84 lines
3.5 KiB
Markdown
84 lines
3.5 KiB
Markdown
# HEARTBEAT.md
|
||
|
||
## 🔄 Yandex backfill (02-19.04 thread_title)
|
||
|
||
Rate limit сброс ~01:00 UTC. Запустить:
|
||
```shell
|
||
/home/node/.openclaw/skills/installer/scripts/ssh_exec.sh --host fr24 --cmd "sg docker -c 'docker exec fr24-schedule python backfill.py --start-date 2026-04-06 --end-date 2026-04-19 2>&1'" --timeout 1800
|
||
```
|
||
Если снова 429 — подождать ещё час и повторить.
|
||
|
||
## 🌊 Watchdog: noisemap Flask (port 5555)
|
||
|
||
Check if Flask is running on port 5555. If not — start it silently.
|
||
|
||
```shell
|
||
curl -s --connect-timeout 2 http://localhost:5555/ > /dev/null 2>&1 || (cd /home/node/.openclaw/workspace/tasks/flightradar24/prototype && nohup /home/node/.openclaw/workspace/tasks/flightradar24/prototype/venv/bin/python app.py > /tmp/noisemap.log 2>&1 &)
|
||
```
|
||
|
||
Run the shell command above. If Flask responded (exit 0) — do nothing.
|
||
If Flask was not running (exit non-0) — start it with the command above and report: "🚀 noisemap Flask restarted on port 5555".
|
||
|
||
## 🏔️ Watchdog: snowbike-rag (port 5557)
|
||
|
||
Check if snowbike-rag is running on port 5557. If not — start it via start.sh.
|
||
|
||
```shell
|
||
curl -s --connect-timeout 2 http://localhost:5557/health > /dev/null 2>&1 || bash /home/node/.openclaw/workspace/tasks/snowbike-rag/start.sh > /dev/null 2>&1
|
||
```
|
||
|
||
If Flask responded (exit 0) — do nothing, reply HEARTBEAT_OK.
|
||
If not running — start it and report: "🚀 snowbike-rag restarted on port 5557".
|
||
|
||
## 🏔️ Watchdog: Terrain hypso tile generation
|
||
|
||
Проверять статус генерации hypso тайлов (зумы 5-14) каждые 30 минут и отписываться Славе.
|
||
|
||
```shell
|
||
cd /home/node/.openclaw/workspace/tasks/enduro-trails/prototype && node -e "
|
||
const { Client } = require('ssh2');
|
||
const conn = new Client();
|
||
function exec(conn, cmd) {
|
||
return new Promise((resolve, reject) => {
|
||
conn.exec(cmd, (err, stream) => {
|
||
if (err) return reject(err);
|
||
let out = '', e = '';
|
||
stream.on('data', d => out += d);
|
||
stream.stderr.on('data', d => e += d);
|
||
stream.on('close', () => resolve({ out: out.trim(), err: e.trim() }));
|
||
});
|
||
});
|
||
}
|
||
conn.on('ready', async () => {
|
||
const zooms = await exec(conn, 'ls /home/slin/enduro-trails/data/terrain/hillshade/ 2>/dev/null | grep -v xml | sort -n | tr \\\"\\n\\\" \\\" \\\"');
|
||
const pngs = await exec(conn, 'find /home/slin/enduro-trails/data/terrain/hillshade -name \\\"*.png\\\" 2>/dev/null | wc -l');
|
||
const procs = await exec(conn, 'pgrep -c gdal2tiles 2>/dev/null || echo 0');
|
||
console.log('zooms:' + zooms.out + ' pngs:' + pngs.out + ' procs:' + procs.out);
|
||
conn.end();
|
||
});
|
||
conn.connect({ host: '82.22.50.71', username: 'slin', password: 'motoZ@yaz2010', readyTimeout: 15000 });
|
||
" 2>/dev/null
|
||
```
|
||
|
||
Если `procs: 0` И зумы 10-15 все есть → hillshade готов → запустить ручные тесты TC-07..TC-25 и отписаться Славе.
|
||
Если ещё идёт → отписаться: «🏔️ Hillshade: зумы X, PNGs Y, ещё генерируется».
|
||
|
||
---
|
||
|
||
## 🌐 Watchdog: vpn-srv FRP tunnel
|
||
|
||
Check if vpn-srv FRP tunnel is active (port 3322 on relay).
|
||
|
||
```shell
|
||
timeout 3 bash -c 'echo > /dev/tcp/185.130.212.192/3322' 2>/dev/null && echo ok || echo FAIL
|
||
```
|
||
|
||
If FAIL — try to reconnect via SSH tunnel:
|
||
```shell
|
||
ssh -i /home/node/.openclaw/ha_ssh_key -o StrictHostKeyChecking=no -o Port=3322 vpn@185.130.212.192 'echo meNt85doC | sudo -S systemctl restart frpc' 2>/dev/null
|
||
```
|
||
|
||
If tunnel still down — alert: "⚠️ vpn-srv FRP tunnel down, manual intervention needed".
|
||
|
||
|