feat(staging-check): ORCH-048 B6 reads registry via GET /projects, not local import
B6 built the project registry by importing src.projects locally (host-path hack
+ importlib.reload), so it evaluated ORCH_PROJECTS_JSON from the launcher's
process-env. On the deployer's canonical host run that var is unset → built-in
default (ET+ORCH) → false FAIL even when staging isolation is healthy.
- Add read-only additive endpoint GET /projects (src/main.py) returning
known_plane_project_ids + {plane_project_id, repo, work_item_prefix, name}
of the live process; no secrets. Existing routes unchanged.
- Rewrite B6 to fetch GET {base}/projects via the same stdlib _get helper as
A/B4/B5/C; drop the host-path hack and importlib.reload (launch-invariant).
- Isolate the verdict in pure _evaluate_b6(known) -> (passed, detail); contract
unchanged (PASS iff SANDBOX in known and prod ET/ORCH absent). Endpoint
degradation (non-200 / missing key / bad body / network) → deterministic FAIL.
- src/projects.py and .env* untouched.
Docs (golden source): API table + staging-gate B6 mechanic in
docs/architecture/README.md; B6 description + isolation row in
docs/operations/STAGING_CHECK.md; CHANGELOG entry.
Tests: tests/test_staging_check_b6.py (TC-01..TC-07), tests/test_projects_endpoint.py.
Refs: ORCH-048
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -5,6 +5,7 @@
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
- **Read-only эндпоинт `GET /projects` + staging-чек B6 читает реестр по HTTP** (ORCH-048): B6 в `scripts/staging_check.py` («Registry: sandbox present, prod ET/ORCH absent») раньше строил реестр **локальным импортом** `src.projects` (host-path хак `sys.path.insert(0,"/repos/orchestrator")` + `importlib.reload`), оценивая `ORCH_PROJECTS_JSON` из process-env *того процесса, что запускает скрипт*. На каноническом host-запуске деплоера (`python3 scripts/staging_check.py --base-url http://localhost:8501`) переменная не задана → встроенный `_DEFAULT_PROJECTS` (ET+ORCH) → **ложный FAIL** при фактически исправной изоляции; B6 проверял реестр НЕ того окружения, что реально обслуживает webhooks. Теперь добавлен read-only additive-эндпоинт `GET /projects` (`src/main.py`), отдающий `known_plane_project_ids` + список `{plane_project_id, repo, work_item_prefix, name}` (без секретов) **именно того процесса**, что слушает вебхуки; источник — существующая `src.projects` (новой логики реестра нет). B6 переписан на `GET {base}/projects` тем же stdlib-хелпером `_get`, что и A/B4/B5/C; host-path хак и `importlib.reload` удалены (инвариантен к способу запуска: хост / `docker exec`). Логика вердикта вынесена в чистую `_evaluate_b6(known) -> (passed, detail)` (контракт неизменен: PASS ⟺ `SANDBOX ∈ known ∧ PROD_ET ∉ known ∧ PROD_ORCH ∉ known`), формат `Results.add` сохранён; недоступность эндпоинта (не-200 / нет ключа / битый ответ / сетевой сбой) → **детерминированный FAIL** (TR-4), без ложного PASS и необработанного исключения. `src/projects.py` и `.env*` не тронуты; прод-поведение существующих роутов неизменно. ADR `docs/work-items/ORCH-048/06-adr/ADR-001-b6-registry-via-http-endpoint.md`. Тесты: `tests/test_staging_check_b6.py`, `tests/test_projects_endpoint.py`.
|
||||
- **Дословный текст findings reviewer/tester встраивается в `task_desc` заворота** (ORCH-046): при откате на `development` строка `task_desc` (попадает в `.task-dev.md` developer-агента) теперь несёт суть претензий, а не только ссылку на файл — устраняет «испорченный телефон», из-за которого агент шёл «читать файл», терял ключевые P0/P1 / причину FAIL и заворачивался снова, выжигая `MAX_DEVELOPER_RETRIES` и токены. Новый defensive-модуль `src/review_parse.py` (контракт «never raise», как `src/frontmatter.py`): `extract_review_findings(path)` — дословные пункты P0/P1 из секции `## Findings` файла `12-review.md`; `extract_test_failures(path)` — релевантный фрагмент тела `13-test-report.md` (приоритет `## Вывод pytest` → FAIL-строки `## Результаты` → `## Итог`). Обе функции усекают результат до `MAX_FINDINGS_CHARS`/`MAX_FAILURES_CHARS` (≈2000) с маркером `…(truncated)`. Две rollback-ветки `src/stage_engine.py` (reviewer REQUEST_CHANGES, tester `check_tests_passed` FAIL) встраивают извлечённый текст и **сохраняют ссылку** на полный файл («Полный контекст»); при пустом/битом артефакте — graceful-фоллбэк на прежнюю ссылку-строку (никаких исключений в `advance_stage`). Tester-ветка дополнительно всегда включает `reason` гейта. Последовательность отката, `_developer_retry_count`, поля `AdvanceResult` и реестр `QG_CHECKS` не менялись. ADR `docs/work-items/ORCH-046/06-adr/ADR-001-embed-findings-in-task-desc.md`. Тесты: `tests/test_review_parse.py`, `tests/test_stage_engine.py::TestRollbackTaskDescEmbedding`.
|
||||
- **Поллинг с ретраем в quality-gate `check_ci_green`** (ORCH-045): гейт CI превращён из single-shot в polling, чтобы устранить race condition — раньше один опрос combined commit-status сразу после пуша developer-а ловил транзиентный `pending` (типично 1-3с, реальный кейс ORCH-017: опрос 17:58:54 → pending, CI дозеленел 17:58:55) и задача застревала насмерть без повторного опроса. Теперь: `success` → пропуск сразу; `failure`/`error` → провал сразу (терминально, ретрай бессмыслен); `pending`/unknown → `time.sleep` и повторный опрос до `ci_poll_max_attempts` раз; истечение попыток → явный `(False, "CI still pending after <T>s")` (тупик больше не молчаливый); 404 → как раньше; транзиентная `httpx.HTTPError` на попытке логируется и ретраится в рамках бюджета. Параметры — новые настройки `ORCH_CI_POLL_MAX_ATTEMPTS` (12) и `ORCH_CI_POLL_INTERVAL_S` (10) в `src/config.py` (~2 мин ожидания pending). Сигнатура `check_ci_green(repo, branch)` и реестр `QG_CHECKS` не менялись; `check_tests_passed` не затронут. ADR `docs/architecture/adr/adr-0004-ci-poll-retry.md`. Тесты: `tests/test_qg.py::TestCheckCIGreen`.
|
||||
- **Прямые ссылки на BRD и Plane-таску в Telegram-уведомлении об апруве** (ORCH-017): пингующее сообщение `notify_approve_requested` теперь встраивает две HTML-`<a>`-ссылки — на `docs/work-items/<WI>/01-brd.md` (Gitea branch-view: `gitea_public_url`→`gitea_url`) и на issue в Plane (`{web_base}/{workspace}/projects/{project_id}/issues/{plane_issue_id}/`). Новая настройка `ORCH_PLANE_WEB_URL` (внешний браузерный web-URL Plane; фолбэк на `plane_api_url`). **Loopback-guard:** если итоговый Plane web-base указывает на localhost/127.0.0.1/0.0.0.0/::1 или пуст — Plane-ссылка опускается (не выпускаем битый localhost-URL). Graceful degradation: каждая ссылка строится независимо и опускается при нехватке данных, сообщение и призыв «Переведите задачу в статус Approved …» сохраняются всегда; ровно одно пингующее сообщение, разделяемая `send_telegram` не тронута. Динамические подписи экранируются `html.escape`, `parse_mode=HTML` сохранён. ADR `docs/work-items/ORCH-017/06-adr/ADR-001-telegram-approve-links.md`. Тесты: `test_notify_approve_links.py`, `test_analysis_approve_flow_links.py`.
|
||||
|
||||
@@ -41,6 +41,8 @@ created → analysis → architecture → development → review → testing →
|
||||
### Условный staging-гейт (ORCH-35)
|
||||
`check_staging_status` реален только для self-hosting (`is_self_hosting_repo(repo)` → `orchestrator`); для остальных проектов → no-op `(True, "Staging gate N/A")`. Для orchestrator парсит `staging_status:` из `15-staging-log.md`; FAILED → откат на `development`. Подробнее: [ADR-0003](adr/adr-0003-staging-gate.md).
|
||||
|
||||
Агрегат staging-чеков пишет деплоер прогоном `scripts/staging_check.py` против живого `orchestrator-staging` (8501). Чек **B6 «Registry: sandbox present, prod ET/ORCH absent»** читает реестр работающего инстанса по HTTP (`GET /projects`), а не локальным импортом `src.projects` в process-env скрипта — иначе host-запуск без `ORCH_PROJECTS_JSON` давал ложный FAIL (ORCH-048, `docs/work-items/ORCH-048/06-adr/ADR-001-b6-registry-via-http-endpoint.md`). Вердикт изолирован в чистой функции `_evaluate_b6(known)`; недоступность эндпоинта → детерминированный FAIL (без ложного PASS).
|
||||
|
||||
## Откаты
|
||||
- Reviewer REQUEST_CHANGES → откат на `development` + retry (`MAX_DEVELOPER_RETRIES = 3`).
|
||||
- Tester `check_tests_passed` FAIL → откат на `development` + retry.
|
||||
@@ -84,6 +86,7 @@ created → analysis → architecture → development → review → testing →
|
||||
| GET | `/health` | health check |
|
||||
| GET | `/status` | активные задачи (stage != done) |
|
||||
| GET | `/queue` | очередь: counts + max_concurrency + последние jobs |
|
||||
| GET | `/projects` | read-only реестр проектов живого инстанса: `known_plane_project_ids` + список `{plane_project_id, repo, work_item_prefix, name}` (без секретов). Источник достоверности для staging-чека B6 (ORCH-048) |
|
||||
| POST | `/webhook/plane` | Plane webhook |
|
||||
| POST | `/webhook/gitea` | Gitea webhook (push, PR, CI status) |
|
||||
|
||||
@@ -97,4 +100,4 @@ created → analysis → architecture → development → review → testing →
|
||||
Схема БД, потоки данных, resilience-слой, детали Dockerfile — [internals.md](internals.md).
|
||||
|
||||
---
|
||||
*Актуально на 2026-06-05 (main `f1b3146`). Обновлять при изменении src/stages.py, src/qg/checks.py, src/main.py.*
|
||||
*Актуально на 2026-06-06 (ORCH-048: + GET /projects). Обновлять при изменении src/stages.py, src/qg/checks.py, src/main.py.*
|
||||
|
||||
@@ -9,7 +9,7 @@
|
||||
| Блок | Название | Что проверяет |
|
||||
|------|----------|---------------|
|
||||
| A | SMOKE | `/health`, `/queue`, `ORCH_STAGING=true` |
|
||||
| B | ACCESS | Plane sandbox (R), Gitea sandbox (R+push), реестр проектов |
|
||||
| B | ACCESS | Plane sandbox (R), Gitea sandbox (R+push), реестр проектов (B6 — по HTTP `GET /projects`) |
|
||||
| C | E2E | Создать задачу → триггер конвейера → ветка + коммент → cleanup |
|
||||
|
||||
Exit code: **0** = все PASS, **non-zero** = есть FAIL.
|
||||
@@ -64,6 +64,13 @@ docker exec orchestrator-staging \
|
||||
--mode stub
|
||||
```
|
||||
|
||||
> **B6 инвариантен к способу запуска (ORCH-048).** Чек реестра больше не импортирует
|
||||
> `src.projects` локально (раньше: host-path хак `/repos/orchestrator` + reload, который
|
||||
> читал env *процесса-запускателя* — на host-запуске без `ORCH_PROJECTS_JSON` это давало
|
||||
> ложный FAIL). Теперь B6 запрашивает `GET {base}/projects` у работающего инстанса, поэтому
|
||||
> одинаково корректен и с хоста, и из `docker exec`. ADR:
|
||||
> `docs/work-items/ORCH-048/06-adr/ADR-001-b6-registry-via-http-endpoint.md`.
|
||||
|
||||
---
|
||||
|
||||
## Режимы (`--mode`)
|
||||
@@ -116,7 +123,7 @@ hmac.new(secret.encode(), body, hashlib.sha256).hexdigest()
|
||||
| Проверка | Гарантия |
|
||||
|---------|---------|
|
||||
| A3 `ORCH_STAGING=true` | При false — abort до деструктивных блоков |
|
||||
| B6 Реестр без боевых | ET/ORCH project_id absent в `known_plane_project_ids()` |
|
||||
| B6 Реестр без боевых | ET/ORCH project_id absent в `known_plane_project_ids` живого инстанса (читается по HTTP `GET /projects`, не локальным импортом) |
|
||||
| C: only SANDBOX project_id | Webhook payload указывает только `8c5a3025-...` |
|
||||
| C: only orchestrator-sandbox repo | Gitea operations на `admin/orchestrator-sandbox` |
|
||||
| C: cleanup в finally | Артефакты удаляются даже при ошибке |
|
||||
|
||||
@@ -214,8 +214,74 @@ SANDBOX_PROJECT_ID = "8c5a3025-4f9d-4190-b79f-fa06276bb27e"
|
||||
PROD_ET_PROJECT_ID = "7a79f0a9-5278-49cd-9007-9a338f238f9c"
|
||||
PROD_ORCH_PROJECT_ID = "8da6aa25-a60e-44d6-a1e2-d8ae59aa7d6a"
|
||||
|
||||
B6_LABEL = "B6 Registry: sandbox present, prod ET/ORCH absent"
|
||||
|
||||
def block_b(results: Results):
|
||||
|
||||
def _evaluate_b6(known: set[str]) -> tuple[bool, str]:
|
||||
"""Pure B6 verdict (ORCH-048, 02-trz §9 / TR-2, TR-3).
|
||||
|
||||
PASS ⟺ sandbox project is registered AND neither prod project (ET / ORCH) is.
|
||||
Isolated from any I/O so the registry-isolation contract is unit-testable on
|
||||
both outcomes (clean → PASS, polluted → FAIL) without a live staging instance
|
||||
or docker. ``detail`` keeps the human-readable format the suite already emits.
|
||||
"""
|
||||
sandbox_present = SANDBOX_PROJECT_ID in known
|
||||
et_absent = PROD_ET_PROJECT_ID not in known
|
||||
orch_absent = PROD_ORCH_PROJECT_ID not in known
|
||||
ok = sandbox_present and et_absent and orch_absent
|
||||
detail = (
|
||||
f"sandbox={'YES' if sandbox_present else 'NO'}, "
|
||||
f"prod-ET={'NO(good)' if et_absent else 'YES(BAD!)'}, "
|
||||
f"prod-ORCH={'NO(good)' if orch_absent else 'YES(BAD!)'}"
|
||||
)
|
||||
return ok, detail
|
||||
|
||||
|
||||
def _check_b6(base: str, results: Results) -> None:
|
||||
"""B6 — registry isolation, read over HTTP from the LIVE instance (ORCH-048).
|
||||
|
||||
Fetches ``GET {base}/projects`` (the read-only endpoint added in ADR-001) and
|
||||
evaluates ``known_plane_project_ids`` with :func:`_evaluate_b6`. The endpoint
|
||||
reflects the registry of the very process that serves webhooks (port 8501),
|
||||
independent of how this suite was launched (host / ``docker exec``).
|
||||
|
||||
Replaces the old local-import hack (a hardcoded host-path insert plus a forced
|
||||
module reload of the registry, TR-6) which evaluated the registry of whoever ran
|
||||
the script — on a host run with no ``ORCH_PROJECTS_JSON`` that fell back to the
|
||||
built-in default (ET+ORCH) → false FAIL.
|
||||
|
||||
Any source degradation (non-200 / missing or malformed key / network error)
|
||||
yields a **deterministic FAIL** with a readable detail (TR-4) — never a false
|
||||
PASS and never an unhandled exception.
|
||||
"""
|
||||
try:
|
||||
status, data = _get(f"{base}/projects")
|
||||
except Exception as e:
|
||||
results.add(B6_LABEL, False, f"GET /projects failed: {e}")
|
||||
return
|
||||
|
||||
if status != 200:
|
||||
results.add(B6_LABEL, False, f"GET /projects → HTTP {status}")
|
||||
return
|
||||
if not isinstance(data, dict) or "known_plane_project_ids" not in data:
|
||||
keys = list(data.keys()) if isinstance(data, dict) else type(data).__name__
|
||||
results.add(B6_LABEL, False,
|
||||
f"GET /projects → 200 but no 'known_plane_project_ids' (got {keys})")
|
||||
return
|
||||
|
||||
raw = data.get("known_plane_project_ids")
|
||||
if not isinstance(raw, (list, tuple, set)):
|
||||
results.add(B6_LABEL, False,
|
||||
f"GET /projects → 'known_plane_project_ids' not a list "
|
||||
f"(got {type(raw).__name__})")
|
||||
return
|
||||
|
||||
known = {str(x) for x in raw}
|
||||
ok, detail = _evaluate_b6(known)
|
||||
results.add(B6_LABEL, ok, detail)
|
||||
|
||||
|
||||
def block_b(base: str, results: Results):
|
||||
print(f"\n{_BOLD}[Block B] ACCESS{_RESET}")
|
||||
|
||||
plane_token = os.environ.get("ORCH_PLANE_API_TOKEN", "")
|
||||
@@ -260,28 +326,9 @@ def block_b(results: Results):
|
||||
except Exception as e:
|
||||
results.add("B5 Gitea: orchestrator-sandbox accessible, push=true", False, str(e))
|
||||
|
||||
# B6 — Registry: sandbox in known IDs, prod ET/ORCH NOT in known IDs
|
||||
try:
|
||||
# Import from inside the container (script runs in /repos/orchestrator context)
|
||||
sys.path.insert(0, "/repos/orchestrator")
|
||||
# Force reload to pick up container env
|
||||
import importlib
|
||||
if "src.projects" in sys.modules:
|
||||
importlib.reload(sys.modules["src.projects"])
|
||||
from src.projects import known_plane_project_ids
|
||||
known = known_plane_project_ids()
|
||||
sandbox_present = SANDBOX_PROJECT_ID in known
|
||||
et_absent = PROD_ET_PROJECT_ID not in known
|
||||
orch_absent = PROD_ORCH_PROJECT_ID not in known
|
||||
ok = sandbox_present and et_absent and orch_absent
|
||||
detail = (
|
||||
f"sandbox={'YES' if sandbox_present else 'NO'}, "
|
||||
f"prod-ET={'NO(good)' if et_absent else 'YES(BAD!)'}, "
|
||||
f"prod-ORCH={'NO(good)' if orch_absent else 'YES(BAD!)'}"
|
||||
)
|
||||
results.add("B6 Registry: sandbox present, prod ET/ORCH absent", ok, detail)
|
||||
except Exception as e:
|
||||
results.add("B6 Registry: sandbox present, prod ET/ORCH absent", False, str(e))
|
||||
# B6 — Registry: sandbox in known IDs, prod ET/ORCH NOT in known IDs.
|
||||
# Read over HTTP from the live instance (GET /projects) — ORCH-048 / ADR-001.
|
||||
_check_b6(base, results)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -628,7 +675,7 @@ def main():
|
||||
results = Results()
|
||||
|
||||
block_a(base, results)
|
||||
block_b(results)
|
||||
block_b(base, results)
|
||||
block_c(base, results, args.mode)
|
||||
|
||||
all_ok = results.summary()
|
||||
|
||||
26
src/main.py
26
src/main.py
@@ -109,6 +109,32 @@ async def status():
|
||||
return {"active_tasks": [dict(t) for t in tasks]}
|
||||
|
||||
|
||||
@app.get("/projects")
|
||||
async def projects():
|
||||
"""ORCH-048: read-only project-registry diagnostics.
|
||||
|
||||
Returns the Plane project ids the *running* process is configured to handle,
|
||||
plus each project's repo / work-item prefix / name. The staging check suite
|
||||
(B6) reads this over HTTP to evaluate the registry of the live instance that
|
||||
actually serves webhooks, instead of importing ``src.projects`` into the
|
||||
script's own process-env (ORCH-048, ADR-001). Read-only, additive — existing
|
||||
routes are unchanged. No secrets (tokens / webhook-secret) are exposed.
|
||||
"""
|
||||
from .projects import known_plane_project_ids, PROJECTS
|
||||
return {
|
||||
"known_plane_project_ids": sorted(known_plane_project_ids()),
|
||||
"projects": [
|
||||
{
|
||||
"plane_project_id": p.plane_project_id,
|
||||
"repo": p.repo,
|
||||
"work_item_prefix": p.work_item_prefix,
|
||||
"name": p.name,
|
||||
}
|
||||
for p in PROJECTS
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
@app.get("/queue")
|
||||
async def queue():
|
||||
"""ORCH-1: job-queue observability — status counts + recent jobs."""
|
||||
|
||||
55
tests/test_projects_endpoint.py
Normal file
55
tests/test_projects_endpoint.py
Normal file
@@ -0,0 +1,55 @@
|
||||
"""ORCH-048: tests for the read-only GET /projects diagnostics endpoint.
|
||||
|
||||
Added by ADR-001 so the staging-check suite (B6) can read the project registry of
|
||||
the *live* instance over HTTP instead of importing src.projects into the script's
|
||||
own process-env. The endpoint is read-only / additive and must expose only
|
||||
id / repo / prefix / name — never secrets.
|
||||
"""
|
||||
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from src.main import app
|
||||
from src import projects as P
|
||||
from src.projects import reload_projects
|
||||
|
||||
client = TestClient(app)
|
||||
|
||||
|
||||
def test_projects_endpoint_returns_known_ids():
|
||||
"""GET /projects → 200 with known_plane_project_ids matching the registry."""
|
||||
resp = client.get("/projects")
|
||||
assert resp.status_code == 200
|
||||
body = resp.json()
|
||||
assert "known_plane_project_ids" in body
|
||||
assert set(body["known_plane_project_ids"]) == P.known_plane_project_ids()
|
||||
|
||||
|
||||
def test_projects_endpoint_lists_projects_with_expected_fields():
|
||||
"""Each project entry exposes exactly id/repo/prefix/name (no secrets)."""
|
||||
body = client.get("/projects").json()
|
||||
assert isinstance(body["projects"], list)
|
||||
assert len(body["projects"]) == len(P.PROJECTS)
|
||||
allowed = {"plane_project_id", "repo", "work_item_prefix", "name"}
|
||||
for entry in body["projects"]:
|
||||
assert set(entry.keys()) == allowed
|
||||
# No secret-looking keys leaked into the payload.
|
||||
for key in entry:
|
||||
assert "token" not in key.lower()
|
||||
assert "secret" not in key.lower()
|
||||
|
||||
|
||||
def test_projects_endpoint_reflects_custom_registry(monkeypatch):
|
||||
"""The endpoint reflects the running process's registry, not a hardcoded one."""
|
||||
custom = (
|
||||
'[{"plane_project_id": "endpoint-uuid", "repo": "endpoint-repo", '
|
||||
'"work_item_prefix": "EP", "name": "Endpoint"}]'
|
||||
)
|
||||
monkeypatch.setattr(P.settings, "projects_json", custom)
|
||||
reload_projects()
|
||||
try:
|
||||
body = client.get("/projects").json()
|
||||
assert body["known_plane_project_ids"] == ["endpoint-uuid"]
|
||||
assert body["projects"][0]["repo"] == "endpoint-repo"
|
||||
assert body["projects"][0]["work_item_prefix"] == "EP"
|
||||
finally:
|
||||
reload_projects()
|
||||
203
tests/test_staging_check_b6.py
Normal file
203
tests/test_staging_check_b6.py
Normal file
@@ -0,0 +1,203 @@
|
||||
"""ORCH-048: tests for the staging-check B6 registry-isolation verdict.
|
||||
|
||||
Background / bug being fixed
|
||||
----------------------------
|
||||
B6 in ``scripts/staging_check.py`` used to build the project registry by importing
|
||||
``src.projects`` locally (``sys.path.insert(0,"/repos/orchestrator")`` +
|
||||
``importlib.reload``). That read the registry of WHOEVER ran the script: on the
|
||||
deployer's canonical host run there is no ``ORCH_PROJECTS_JSON`` → the built-in
|
||||
default registry (ET+ORCH) → a **false FAIL** even when staging isolation is fine.
|
||||
|
||||
ADR-001 reworks B6 to read the registry over HTTP from the live instance
|
||||
(``GET /projects``) and isolates the verdict in the pure function
|
||||
``_evaluate_b6(known) -> (passed, detail)`` so both outcomes are unit-testable
|
||||
without a live staging instance or docker (02-trz §9, AC-2).
|
||||
|
||||
These tests cover:
|
||||
* TC-01 clean registry (known={SANDBOX}) → PASS
|
||||
* TC-02 polluted with prod-ET (known={SANDBOX, PROD_ET}) → FAIL
|
||||
* TC-03 polluted with prod-ORCH (known={SANDBOX, PROD_ORCH})→ FAIL
|
||||
* TC-04 sandbox absent (known=set()) → FAIL, deterministic
|
||||
* TC-05 polluted with both prod projects → FAIL
|
||||
* TC-06 no host-path hack / no local src.projects import (TR-6)
|
||||
* TC-07 source degradation (HTTP error / non-200 / bad body) → deterministic FAIL (TR-4)
|
||||
"""
|
||||
|
||||
import importlib.util
|
||||
import pathlib
|
||||
|
||||
import pytest
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Load scripts/staging_check.py (scripts/ is not a Python package).
|
||||
# ---------------------------------------------------------------------------
|
||||
_SCRIPT_PATH = pathlib.Path(__file__).resolve().parents[1] / "scripts" / "staging_check.py"
|
||||
_spec = importlib.util.spec_from_file_location("staging_check", _SCRIPT_PATH)
|
||||
sc = importlib.util.module_from_spec(_spec)
|
||||
_spec.loader.exec_module(sc)
|
||||
|
||||
SANDBOX = sc.SANDBOX_PROJECT_ID
|
||||
PROD_ET = sc.PROD_ET_PROJECT_ID
|
||||
PROD_ORCH = sc.PROD_ORCH_PROJECT_ID
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# _evaluate_b6 — pure verdict (AC-1, AC-2)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_tc01_clean_registry_passes():
|
||||
"""TC-01: known={SANDBOX} → PASS, detail shows sandbox=YES / prod absent."""
|
||||
ok, detail = sc._evaluate_b6({SANDBOX})
|
||||
assert ok is True
|
||||
assert "sandbox=YES" in detail
|
||||
assert "prod-ET=NO(good)" in detail
|
||||
assert "prod-ORCH=NO(good)" in detail
|
||||
|
||||
|
||||
def test_tc02_prod_et_pollution_fails():
|
||||
"""TC-02: prod-ET leaked into the registry → FAIL, flagged in detail."""
|
||||
ok, detail = sc._evaluate_b6({SANDBOX, PROD_ET})
|
||||
assert ok is False
|
||||
assert "prod-ET=YES(BAD!)" in detail
|
||||
# sandbox is still present, only ET is the violation
|
||||
assert "sandbox=YES" in detail
|
||||
|
||||
|
||||
def test_tc03_prod_orch_pollution_fails():
|
||||
"""TC-03: prod-ORCH leaked into the registry → FAIL, flagged in detail."""
|
||||
ok, detail = sc._evaluate_b6({SANDBOX, PROD_ORCH})
|
||||
assert ok is False
|
||||
assert "prod-ORCH=YES(BAD!)" in detail
|
||||
assert "sandbox=YES" in detail
|
||||
|
||||
|
||||
def test_tc04_sandbox_absent_fails_deterministically():
|
||||
"""TC-04: empty registry (no sandbox) → FAIL, no exception (TR-4)."""
|
||||
ok, detail = sc._evaluate_b6(set())
|
||||
assert ok is False
|
||||
assert "sandbox=NO" in detail
|
||||
# prod projects genuinely absent, but the missing sandbox alone fails it
|
||||
assert "prod-ET=NO(good)" in detail
|
||||
assert "prod-ORCH=NO(good)" in detail
|
||||
|
||||
|
||||
def test_tc05_both_prod_pollution_fails():
|
||||
"""TC-05: both prod projects present → FAIL."""
|
||||
ok, detail = sc._evaluate_b6({SANDBOX, PROD_ET, PROD_ORCH})
|
||||
assert ok is False
|
||||
assert "prod-ET=YES(BAD!)" in detail
|
||||
assert "prod-ORCH=YES(BAD!)" in detail
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# TC-06 — registry source no longer depends on the host-path hack (TR-6)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_tc06_no_host_path_hack_in_source():
|
||||
"""TC-06: B6 must not import src.projects from a hardcoded host path.
|
||||
|
||||
Static guard: the removed anti-pattern (``sys.path.insert(0,"/repos/orchestrator")``,
|
||||
``importlib.reload(... src.projects ...)``, ``from src.projects import``) must
|
||||
be gone from the script. B6 now reads the registry over HTTP (GET /projects).
|
||||
"""
|
||||
source = _SCRIPT_PATH.read_text()
|
||||
assert 'sys.path.insert(0, "/repos/orchestrator")' not in source
|
||||
assert "importlib.reload" not in source
|
||||
assert "from src.projects import" not in source
|
||||
# Positive assertion: B6 reads the registry endpoint over HTTP.
|
||||
assert "/projects" in source
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# TC-07 — source degradation → deterministic FAIL, never a false PASS (TR-4)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class _FakeResults:
|
||||
"""Minimal Results stand-in capturing (label, passed, detail) tuples."""
|
||||
|
||||
def __init__(self):
|
||||
self.items = []
|
||||
|
||||
def add(self, label, passed, detail=""):
|
||||
self.items.append((label, passed, detail))
|
||||
|
||||
|
||||
def _last(results: _FakeResults):
|
||||
assert results.items, "expected _check_b6 to record exactly one result"
|
||||
return results.items[-1]
|
||||
|
||||
|
||||
def test_tc07_network_error_is_deterministic_fail(monkeypatch):
|
||||
"""TC-07a: a network/transport error → FAIL, no unhandled exception."""
|
||||
def _boom(url, *a, **k):
|
||||
raise RuntimeError("GET http://x/projects → connection refused")
|
||||
|
||||
monkeypatch.setattr(sc, "_get", _boom)
|
||||
results = _FakeResults()
|
||||
sc._check_b6("http://staging:8501", results)
|
||||
label, passed, detail = _last(results)
|
||||
assert passed is False
|
||||
assert "GET /projects failed" in detail
|
||||
|
||||
|
||||
def test_tc07_non_200_is_fail(monkeypatch):
|
||||
"""TC-07b: non-200 response → FAIL with the status in the detail."""
|
||||
monkeypatch.setattr(sc, "_get", lambda url, *a, **k: (503, {"_raw": "down"}))
|
||||
results = _FakeResults()
|
||||
sc._check_b6("http://staging:8501", results)
|
||||
label, passed, detail = _last(results)
|
||||
assert passed is False
|
||||
assert "503" in detail
|
||||
|
||||
|
||||
def test_tc07_missing_key_is_fail(monkeypatch):
|
||||
"""TC-07c: 200 but no known_plane_project_ids key → FAIL (no false PASS)."""
|
||||
monkeypatch.setattr(sc, "_get", lambda url, *a, **k: (200, {"projects": []}))
|
||||
results = _FakeResults()
|
||||
sc._check_b6("http://staging:8501", results)
|
||||
label, passed, detail = _last(results)
|
||||
assert passed is False
|
||||
assert "known_plane_project_ids" in detail
|
||||
|
||||
|
||||
def test_tc07_malformed_value_is_fail(monkeypatch):
|
||||
"""TC-07d: known_plane_project_ids is not a list → FAIL, no exception."""
|
||||
monkeypatch.setattr(
|
||||
sc, "_get",
|
||||
lambda url, *a, **k: (200, {"known_plane_project_ids": "not-a-list"}),
|
||||
)
|
||||
results = _FakeResults()
|
||||
sc._check_b6("http://staging:8501", results)
|
||||
label, passed, detail = _last(results)
|
||||
assert passed is False
|
||||
assert "not a list" in detail
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Happy path through _check_b6 (HTTP wiring + verdict together)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_check_b6_clean_endpoint_passes(monkeypatch):
|
||||
"""A healthy /projects response with only the sandbox id → PASS."""
|
||||
monkeypatch.setattr(
|
||||
sc, "_get",
|
||||
lambda url, *a, **k: (200, {"known_plane_project_ids": [SANDBOX]}),
|
||||
)
|
||||
results = _FakeResults()
|
||||
sc._check_b6("http://staging:8501", results)
|
||||
label, passed, detail = _last(results)
|
||||
assert passed is True
|
||||
assert "sandbox=YES" in detail
|
||||
|
||||
|
||||
def test_check_b6_polluted_endpoint_fails(monkeypatch):
|
||||
"""A /projects response leaking a prod id → FAIL (defence works end-to-end)."""
|
||||
monkeypatch.setattr(
|
||||
sc, "_get",
|
||||
lambda url, *a, **k: (200, {"known_plane_project_ids": [SANDBOX, PROD_ORCH]}),
|
||||
)
|
||||
results = _FakeResults()
|
||||
sc._check_b6("http://staging:8501", results)
|
||||
label, passed, detail = _last(results)
|
||||
assert passed is False
|
||||
assert "prod-ORCH=YES(BAD!)" in detail
|
||||
Reference in New Issue
Block a user