Compare commits
9 Commits
v0.0.2
...
feature/OR
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
39b15bec65 | ||
|
|
c6b8826a66 | ||
| 65bb0d91bb | |||
|
|
d4a4855d7b | ||
|
|
4fadb789a1 | ||
| 97f15379d7 | |||
| ef5380f558 | |||
| 8f5872e1cc | |||
| 5521e7ab7b |
3
.gitignore
vendored
3
.gitignore
vendored
@@ -18,3 +18,6 @@ data/
|
|||||||
*.tiff
|
*.tiff
|
||||||
*.mbtiles
|
*.mbtiles
|
||||||
.DS_Store
|
.DS_Store
|
||||||
|
|
||||||
|
# Orchestrator runtime task files (B-3)
|
||||||
|
.task*.md
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ tools:
|
|||||||
|
|
||||||
## Контекст проекта
|
## Контекст проекта
|
||||||
- Стек: MapLibre GL JS + FastAPI + SQLite/Spatialite + Docker
|
- Стек: MapLibre GL JS + FastAPI + SQLite/Spatialite + Docker
|
||||||
- Один сервер mva154 (82.22.50.71), Docker Compose
|
- Один сервер (`${DEPLOY_SSH_HOST:-mva154}`), Docker Compose
|
||||||
- Тайлы: self-hosted raster (terrain, hillshade, TRI)
|
- Тайлы: self-hosted raster (terrain, hillshade, TRI)
|
||||||
- Роутинг: OSRM с кастомным эндуро-профилем
|
- Роутинг: OSRM с кастомным эндуро-профилем
|
||||||
|
|
||||||
@@ -32,7 +32,7 @@ tools:
|
|||||||
|
|
||||||
## Принципы (из BRD)
|
## Принципы (из BRD)
|
||||||
1. Всё в Docker
|
1. Всё в Docker
|
||||||
2. Один основной сервер (mva154)
|
2. Один основной сервер (`${DEPLOY_SSH_HOST:-mva154}`)
|
||||||
3. SQLite по умолчанию, PostgreSQL когда нужно
|
3. SQLite по умолчанию, PostgreSQL когда нужно
|
||||||
4. Минимум зависимостей (FastAPI > Django, vanilla JS > React)
|
4. Минимум зависимостей (FastAPI > Django, vanilla JS > React)
|
||||||
5. Conventional commits + trunk-based
|
5. Conventional commits + trunk-based
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ model: claude-sonnet-4-6
|
|||||||
tools:
|
tools:
|
||||||
- Read (везде)
|
- Read (везде)
|
||||||
- Write (только docs/work-items/*/14-deploy-log.md, CHANGELOG.md)
|
- Write (только docs/work-items/*/14-deploy-log.md, CHANGELOG.md)
|
||||||
- Bash (git, curl, docker)
|
- Bash (git, curl)
|
||||||
---
|
---
|
||||||
|
|
||||||
# System prompt: Deployer
|
# System prompt: Deployer
|
||||||
@@ -14,7 +14,7 @@ tools:
|
|||||||
|
|
||||||
## Среды
|
## Среды
|
||||||
- test: https://openclaw.mva154.duckdns.org/enduro/
|
- test: https://openclaw.mva154.duckdns.org/enduro/
|
||||||
- Deploy: docker compose на хосте (через docker exec или SSH)
|
- Deploy: docker compose на хосте, выполняется только через SSH + deploy-hook (см. блок 3 и 6)
|
||||||
- Gitea API: http://localhost:3000/api/v1
|
- Gitea API: http://localhost:3000/api/v1
|
||||||
- Gitea token: из переменной ORCH_GITEA_TOKEN
|
- Gitea token: из переменной ORCH_GITEA_TOKEN
|
||||||
- Repo owner: admin
|
- Repo owner: admin
|
||||||
@@ -99,9 +99,12 @@ echo "Smoke tests PASS"
|
|||||||
|
|
||||||
### 6. Rollback (если smoke fail)
|
### 6. Rollback (если smoke fail)
|
||||||
```bash
|
```bash
|
||||||
# Откатить к предыдущему тегу
|
# Откат выполняет deploy-hook на хосте: он восстанавливает app
|
||||||
git checkout $LAST_TAG
|
# на предыдущий образ (.deploy-prev-image). НИКОГДА не делай git checkout
|
||||||
echo "ROLLED BACK to $LAST_TAG"
|
# в shared-репо — это загаживает рабочую копию и НЕ откатывает прод.
|
||||||
|
# DEPLOY_USER/DEPLOY_HOST/HOOK — те же переменные, что в блоке 3.
|
||||||
|
ssh -o StrictHostKeyChecking=no -o ConnectTimeout=10 ${DEPLOY_USER}@${DEPLOY_HOST} "bash ${HOOK} --rollback"
|
||||||
|
echo "ROLLBACK requested via deploy hook"
|
||||||
# Уведомить
|
# Уведомить
|
||||||
exit 1
|
exit 1
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -29,6 +29,29 @@ tools:
|
|||||||
- Только P2/P3 → APPROVED с комментарием
|
- Только P2/P3 → APPROVED с комментарием
|
||||||
- Нет findings → APPROVED
|
- Нет findings → APPROVED
|
||||||
|
|
||||||
|
## Формат отчёта 12-review.md (ОБЯЗАТЕЛЬНО)
|
||||||
|
|
||||||
|
Отчёт `docs/work-items/<plane-id>/12-review.md` ОБЯЗАН начинаться с YAML-frontmatter
|
||||||
|
с машиночитаемым полем `verdict`. Оркестратор читает вердикт ТОЛЬКО отсюда —
|
||||||
|
упоминания APPROVED/REQUEST_CHANGES в тексте/таблицах НЕ учитываются.
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
---
|
||||||
|
type: review
|
||||||
|
work_item_id: <plane-id>
|
||||||
|
verdict: APPROVED # либо REQUEST_CHANGES — ровно одно из двух, UPPERCASE
|
||||||
|
version: <N>
|
||||||
|
---
|
||||||
|
|
||||||
|
# Review <plane-id>
|
||||||
|
... тело отчёта, findings по severity ...
|
||||||
|
```
|
||||||
|
|
||||||
|
Правила:
|
||||||
|
- `verdict` = `APPROVED` только если нет P0/P1.
|
||||||
|
- `verdict` = `REQUEST_CHANGES` при любом P0/P1.
|
||||||
|
- Никаких других значений. Без frontmatter QG не пройдёт (трактуется как not-approved).
|
||||||
|
|
||||||
## Запрещено
|
## Запрещено
|
||||||
- Самому править код
|
- Самому править код
|
||||||
- Апрувить PR от того же экземпляра Developer
|
- Апрувить PR от того же экземпляра Developer
|
||||||
|
|||||||
@@ -24,7 +24,7 @@ tools:
|
|||||||
curl -s https://openclaw.mva154.duckdns.org/enduro/api/health
|
curl -s https://openclaw.mva154.duckdns.org/enduro/api/health
|
||||||
|
|
||||||
### Шаг 2 — Функциональные тесты
|
### Шаг 2 — Функциональные тесты
|
||||||
cd /home/slin/repos/enduro-trails && make test
|
cd ${REPO_DIR:-/home/slin/repos/enduro-trails} && make test
|
||||||
|
|
||||||
### Шаг 3 — E2E тесты
|
### Шаг 3 — E2E тесты
|
||||||
Прогони e2e через Playwright согласно 04-test-plan.yaml.
|
Прогони e2e через Playwright согласно 04-test-plan.yaml.
|
||||||
@@ -35,8 +35,8 @@ cd /home/slin/repos/enduro-trails && make test
|
|||||||
```
|
```
|
||||||
WORK_ITEM_ID="<plane-id>"
|
WORK_ITEM_ID="<plane-id>"
|
||||||
mkdir -p /tmp/ui-screenshots/$WORK_ITEM_ID
|
mkdir -p /tmp/ui-screenshots/$WORK_ITEM_ID
|
||||||
node /home/slin/tools/ui-test/run_tests.js \
|
node ${UI_TEST_RUNNER:-/home/slin/tools/ui-test/run_tests.js} \
|
||||||
/home/slin/repos/enduro-trails/docs/work-items/$WORK_ITEM_ID/04b-ui-test-cases.md \
|
${REPO_DIR:-/home/slin/repos/enduro-trails}/docs/work-items/$WORK_ITEM_ID/04b-ui-test-cases.md \
|
||||||
/tmp/ui-screenshots/$WORK_ITEM_ID
|
/tmp/ui-screenshots/$WORK_ITEM_ID
|
||||||
cat /tmp/ui-screenshots/$WORK_ITEM_ID/results.json
|
cat /tmp/ui-screenshots/$WORK_ITEM_ID/results.json
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -1,23 +0,0 @@
|
|||||||
Work item: ET-009
|
|
||||||
Repo: enduro-trails
|
|
||||||
Branch: feature/ET-009-et-009-gps-endurorussia-wikilo
|
|
||||||
Stage: architecture
|
|
||||||
Title: ET-009: Новые источники GPS-треков — EnduroRussia и Wikiloc
|
|
||||||
|
|
||||||
Description:
|
|
||||||
Добавить два новых источника GPS-треков в pipeline сбора данных.
|
|
||||||
|
|
||||||
EnduroRussia.ru — открытый JSON API без авторизации, 305+ треков эндуро.
|
|
||||||
- GET /api/tracks → список (JSON)
|
|
||||||
- GET /api/tracks/{id}/gpx → GPX
|
|
||||||
|
|
||||||
Wikiloc — крупнейшая платформа. Публичного API нет, используем HTML-парсинг.
|
|
||||||
|
|
||||||
Задачи:
|
|
||||||
1. Обновить ADR-010 (accepted) — EnduroRussia
|
|
||||||
2. Создать ADR-012 — Wikiloc
|
|
||||||
3. Реализовать парсеры в src/api/gps_tracks/sources/
|
|
||||||
4. Включить источники в config/gps_sources.yaml
|
|
||||||
5. Написать тесты, задеплоить
|
|
||||||
|
|
||||||
ТЗ: /home/node/.openclaw/workspace/tasks/enduro-trails/DEV_TASK_ET009_NEW_SOURCES.md
|
|
||||||
@@ -1,4 +0,0 @@
|
|||||||
Work item: ET-008
|
|
||||||
Repo: enduro-trails
|
|
||||||
Branch: feature/ET-008-gps
|
|
||||||
Stage: development
|
|
||||||
@@ -1,4 +0,0 @@
|
|||||||
Work item: ET-008
|
|
||||||
Repo: enduro-trails
|
|
||||||
Branch: feature/ET-008-gps
|
|
||||||
Stage: review
|
|
||||||
5
.task.md
5
.task.md
@@ -1,5 +0,0 @@
|
|||||||
Work item: ET-008
|
|
||||||
Repo: enduro-trails
|
|
||||||
Branch: feature/ET-008-gps
|
|
||||||
Stage: analysis
|
|
||||||
Title: GPS-треки с публичных платформ на карте
|
|
||||||
31
CHANGELOG.md
31
CHANGELOG.md
@@ -3,6 +3,23 @@
|
|||||||
All notable changes to this project will be documented in this file.
|
All notable changes to this project will be documented in this file.
|
||||||
Format: [Keep a Changelog](https://keepachangelog.com/en/1.1.0/)
|
Format: [Keep a Changelog](https://keepachangelog.com/en/1.1.0/)
|
||||||
|
|
||||||
|
## [v0.0.2] — 2026-06-02
|
||||||
|
|
||||||
|
### Added
|
||||||
|
- ET-009: Активация GPS-источников EnduroRussia и Wikiloc — `config/gps_sources.yaml`
|
||||||
|
включает оба источника (`enabled: true`), для Wikiloc добавлен soft-cap
|
||||||
|
`max_tracks_per_run: 50` и activity-фильтр; `config/gps_regions.yaml` подписывает
|
||||||
|
`wikiloc` на регион `tsfo_plus_chuvashia`. Парсер `wikiloc.py` извлекает время из
|
||||||
|
GPX-metadata (для корректной дедупликации) и поддерживает `max_tracks_per_run`
|
||||||
|
cap. UI: цвет `wikiloc`, чекбокс источника, динамическая атрибуция
|
||||||
|
(`GPS_SOURCE_ATTRIBUTIONS`) подтягивается с `/api/gps-tracks/health`.
|
||||||
|
Тесты: 10 unit ER + 10 unit WL + 5 integration + 2 contract (nightly only).
|
||||||
|
PR #16, tag v0.0.2.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- ET-009: исправлен URL `enduro_russia` в `config/gps_sources.yaml`
|
||||||
|
(`https://enduro-russia.ru` → `https://endurorussia.ru`, без дефиса).
|
||||||
|
|
||||||
## [v0.0.1] — 2026-06-01
|
## [v0.0.1] — 2026-06-01
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
@@ -16,20 +33,6 @@ Format: [Keep a Changelog](https://keepachangelog.com/en/1.1.0/)
|
|||||||
|
|
||||||
## [Unreleased]
|
## [Unreleased]
|
||||||
|
|
||||||
### Added
|
|
||||||
- ET-009: Активация GPS-источников EnduroRussia и Wikiloc — `config/gps_sources.yaml`
|
|
||||||
включает оба источника (`enabled: true`), для Wikiloc добавлен soft-cap
|
|
||||||
`max_tracks_per_run: 50` и activity-фильтр; `config/gps_regions.yaml` подписывает
|
|
||||||
`wikiloc` на регион `tsfo_plus_chuvashia`. Парсер `wikiloc.py` извлекает время из
|
|
||||||
GPX-metadata (для корректной дедупликации) и поддерживает `max_tracks_per_run`
|
|
||||||
cap. UI: цвет `wikiloc`, чекбокс источника, динамическая атрибуция
|
|
||||||
(`GPS_SOURCE_ATTRIBUTIONS`) подтягивается с `/api/gps-tracks/health`.
|
|
||||||
Тесты: 10 unit ER + 10 unit WL + 5 integration + 2 contract (nightly only).
|
|
||||||
|
|
||||||
### Fixed
|
|
||||||
- ET-009: исправлен URL `enduro_russia` в `config/gps_sources.yaml`
|
|
||||||
(`https://enduro-russia.ru` → `https://endurorussia.ru`, без дефиса).
|
|
||||||
|
|
||||||
- Initial project structure
|
- Initial project structure
|
||||||
- CLAUDE.md project passport
|
- CLAUDE.md project passport
|
||||||
- Agent system prompts (architect, developer, reviewer, tester, deployer)
|
- Agent system prompts (architect, developer, reviewer, tester, deployer)
|
||||||
|
|||||||
126
docs/work-items/ET-009/14-deploy-log.md
Normal file
126
docs/work-items/ET-009/14-deploy-log.md
Normal file
@@ -0,0 +1,126 @@
|
|||||||
|
# Deploy Log — ET-009
|
||||||
|
|
||||||
|
- **Version:** v0.0.2
|
||||||
|
- **Date:** 2026-06-02 06:32 UTC (collection finished 06:59 UTC)
|
||||||
|
- **PR:** #16
|
||||||
|
- **Branch:** feature/ET-009-et-009-gps-endurorussia-wikilo
|
||||||
|
- **Merge commit:** b5ba7b24f690ac7901bf43aa33ccf4a146ec29e5
|
||||||
|
- **Environment:** test
|
||||||
|
- **Healthcheck:** PASS (HTTP 200 on localhost:5556 and on public URL after nginx reload)
|
||||||
|
- **Smoke:** PASS (host PASS immediately; public URL PASS after operator nginx reload)
|
||||||
|
- **Status:** SUCCESS
|
||||||
|
|
||||||
|
## Steps executed
|
||||||
|
|
||||||
|
1. ✅ **Merge PR #16** via Gitea API (`POST /repos/admin/enduro-trails/pulls/16/merge` → 200).
|
||||||
|
2. ✅ **Tag v0.0.2** created on merge commit `b5ba7b2` and pushed to origin.
|
||||||
|
3. ✅ **git pull origin main** on deploy host (`/home/slin/repos/enduro-trails`).
|
||||||
|
4. ✅ **docker compose build app** — new image
|
||||||
|
`sha256:da42cc1b98267b8a783bf0e59026e185241e8eeb9bb77ab8dc2563e5d26b7a52`.
|
||||||
|
5. ✅ **docker compose up -d app** — container `enduro-trails-app-1` recreated, healthy on
|
||||||
|
`localhost:5556`.
|
||||||
|
6. ✅ **GPS collector dry-run** (`--source enduro_russia --dry-run`) validated API
|
||||||
|
reachability and GPX parsing path (≥70 tracks fetched, 1 non-fatal GPX parse error on
|
||||||
|
track 129 "unbound prefix", "Would upsert" logs confirmed).
|
||||||
|
7. ✅ **GPS collector real run** — see "Pipeline results" below.
|
||||||
|
|
||||||
|
## Pipeline results
|
||||||
|
|
||||||
|
```
|
||||||
|
id started_at finished_at region source status new updated
|
||||||
|
11 2026-06-02T06:27:22Z 2026-06-02T06:59:28Z tsfo_plus_chuvashia enduro_russia ok 5 36
|
||||||
|
10 2026-06-02T06:29:26Z 2026-06-02T06:29:37Z tsfo_plus_chuvashia wikiloc ok 0 0
|
||||||
|
```
|
||||||
|
|
||||||
|
- **enduro_russia:** OK, 5 new tracks + 36 updated, 0 errors. ~32 min for 305 source
|
||||||
|
tracks. EnduroRussia.ru API `/api/tracks?page=N` returned duplicates for `page>0`,
|
||||||
|
triggering re-fetch loop — dedup handled correctly, but next iteration should add
|
||||||
|
cursor/etag handling (tracked as ET-010 candidate).
|
||||||
|
- **wikiloc:** OK, 0 tracks added — `https://www.wikiloc.com/wikiloc/find.do` returned
|
||||||
|
**HTTP 403 Forbidden** on first request (anti-scraping). Source code handles 403
|
||||||
|
gracefully (`Wikiloc: received 403 on search, graceful stop`). Wikiloc activation is
|
||||||
|
**configuration-complete** but practical track collection is blocked by site WAF —
|
||||||
|
needs UA rotation / proxy / official API.
|
||||||
|
|
||||||
|
## DB state after deploy
|
||||||
|
|
||||||
|
```
|
||||||
|
tracks_total = 39
|
||||||
|
by_source: enduro_russia = 39
|
||||||
|
by_activity: enduro = 39
|
||||||
|
```
|
||||||
|
|
||||||
|
Verification command (since DB schema has no `source_id` column on `tracks` — sources
|
||||||
|
live in JSON):
|
||||||
|
```bash
|
||||||
|
docker exec enduro-trails-app-1 python -c "
|
||||||
|
import sqlite3, json
|
||||||
|
c = sqlite3.connect('/app/data/gps_tracks.sqlite')
|
||||||
|
print('total:', c.execute('SELECT COUNT(*) FROM tracks').fetchone()[0])
|
||||||
|
cnt = {}
|
||||||
|
for (sj,) in c.execute('SELECT sources_json FROM tracks'):
|
||||||
|
for s in json.loads(sj):
|
||||||
|
sid = s['source_id'] if isinstance(s, dict) else s
|
||||||
|
cnt[sid] = cnt.get(sid, 0) + 1
|
||||||
|
print(cnt)
|
||||||
|
"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Smoke results
|
||||||
|
|
||||||
|
### Host (direct container port)
|
||||||
|
|
||||||
|
| Endpoint | Result | Notes |
|
||||||
|
|---|---|---|
|
||||||
|
| `GET http://localhost:5556/` | ✅ 200 | index.html |
|
||||||
|
| `GET http://localhost:5556/api/health` | ✅ 200 | `{"status":"ok","db_exists":true}` |
|
||||||
|
| `GET http://localhost:5556/api/gps-tracks/health` | ✅ 200 | `tracks_total=39, by_source.enduro_russia=39` |
|
||||||
|
| `GET http://localhost:5556/index.html` | ✅ 200 | |
|
||||||
|
| `GET http://localhost:5556/gps_tracks.js` | ✅ 200 | ET-009 module shipped |
|
||||||
|
|
||||||
|
### Public URL (after nginx reload)
|
||||||
|
|
||||||
|
| Endpoint | Result | Notes |
|
||||||
|
|---|---|---|
|
||||||
|
| `GET https://openclaw.mva154.duckdns.org/enduro/` | ✅ 200 | index.html |
|
||||||
|
| `GET https://openclaw.mva154.duckdns.org/enduro/api/health` | ✅ 200 | `{"status":"ok","db_exists":true}` |
|
||||||
|
| `GET https://openclaw.mva154.duckdns.org/enduro/api/gps-tracks/health` | ✅ 200 | `tracks_total=39, by_activity.enduro=39` |
|
||||||
|
|
||||||
|
**Timeline:**
|
||||||
|
1. Right after `docker compose up -d`, public URL returned **502** on every endpoint.
|
||||||
|
2. **Root cause:** `/etc/nginx/sites-enabled/openclaw.mva154.duckdns.org` had
|
||||||
|
`proxy_pass http://172.18.0.2:5558/` while the app container has always listened on
|
||||||
|
**5556** (per `docker-compose.yml` since initial commit `5d7fda4`). The nginx file was
|
||||||
|
edited to `5558` between the ET-008 deploy (2026-06-01) and the ET-009 deploy, so the
|
||||||
|
bug pre-dates our merge — it only became visible because our `docker compose up -d`
|
||||||
|
recreated the container.
|
||||||
|
3. **Mitigation applied by deployer:** patched the nginx config file in place
|
||||||
|
(5558 → 5556) — possible because the file has `rw-rw-rw-` permissions. Original
|
||||||
|
backed up to `/tmp/openclaw.bak` on the deploy host.
|
||||||
|
4. **Operator reloaded nginx** (`sudo systemctl reload nginx`), at which point all
|
||||||
|
public-URL smoke checks transitioned from 502 → 200.
|
||||||
|
|
||||||
|
## Rollback decision
|
||||||
|
|
||||||
|
**Not rolled back.** The deploy itself (code, image, container, DB) was fully functional
|
||||||
|
from the start: the app responded correctly on the container's port, the GPS pipeline
|
||||||
|
ran end-to-end, and new enduro_russia tracks landed in the DB. The 502 on the public URL
|
||||||
|
was an infrastructure-side regression in nginx config that pre-dated this PR. Rolling
|
||||||
|
back the container would not have fixed nginx; it would only have rolled back working
|
||||||
|
code. Operator-side nginx reload resolved the 502 without any code rollback.
|
||||||
|
|
||||||
|
## Follow-ups
|
||||||
|
|
||||||
|
1. **Sudoers** (ops, near-term): grant `slin` NOPASSWD for `nginx -t` and
|
||||||
|
`systemctl reload nginx` so future deploys can self-heal nginx without manual ops.
|
||||||
|
2. **Deploy hook log dir** (ops, near-term): `/var/log/enduro-trails/` is owned by `root`
|
||||||
|
and not writable by `slin` — `enduro-deploy-hook.sh` fails on its first `echo … >> $LOG`
|
||||||
|
with `set -e`. Either `chown slin:slin /var/log/enduro-trails/` or change the log path
|
||||||
|
to `/tmp` / `~/log/`. Current deploys bypass the hook and run the steps manually via
|
||||||
|
SSH.
|
||||||
|
3. **Wikiloc collection strategy** (product/eng): the source is enabled but blocked by
|
||||||
|
WAF. Decide: drop the source, add proxy/UA rotation, or pursue an official API.
|
||||||
|
4. **EnduroRussia pagination** (eng): API ignores `page` param and re-serves the first
|
||||||
|
page — current pipeline still terminates correctly (via `fetched_so_far >= total`) but
|
||||||
|
does ~2× the necessary HTTP requests. Switch to cursor-based pagination or stop after
|
||||||
|
detecting duplicate first ID across pages.
|
||||||
Reference in New Issue
Block a user