refactor(frontmatter): unified frontmatter contract + handoff spec (ORCH-52c)
src/frontmatter.py grows from a single-key reader into the full machine
contract: reader (read_frontmatter_value, unchanged), one parse primitive
(parse_frontmatter), writer (render/write_frontmatter), schema validator
(validate_schema/REQUIRED_FIELDS, warning-only by default) and a shared
strip_frontmatter helper. The five verdict gates (check_reviewer_verdict,
_parse_tests_verdict, _parse_deploy_status, _parse_staging_status,
parse_security_status) now read through the single parse_frontmatter point
instead of duplicated ad-hoc YAML logic; review_parse._strip_frontmatter and
security_gate.extract_security_findings reuse the shared helper.
Strictly backward compatible + never-raise: STAGE_TRANSITIONS, the QG_CHECKS
composition, verdict semantics (incl. ORCH-047 three-field tester + negative
token priority), reason-strings and worktree->origin/main fallback are 1:1.
The schema validator never influences a gate verdict by default; hard-fail is
reserved behind the frontmatter_validation_strict kill-switch (default False).
New formal handoff spec docs/_standards/HANDOFF_PROTOCOL.md ("stage -> required
output" + required frontmatter schema), aligned 1:1 with PIPELINE_DOCS.md.
Tests: test_frontmatter.py (TC-01..07), test_qg_verdicts.py (TC-08..15),
test_security_gate.py (TC-12), test_stages_invariants.py (TC-16). Full
tests/ green (1212).
Refs: ORCH-076
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -3,6 +3,12 @@
|
||||
Формат: [Keep a Changelog](https://keepachangelog.com/). Записи — на смысловой PR/задачу.
|
||||
|
||||
## [Unreleased]
|
||||
- **Единый frontmatter-контракт (reader + writer + валидатор) + спека handoff** (ORCH-076 / ORCH-52c, `refactor`/`docs`): слой 2 эпика ORCH-52 — `src/frontmatter.py` из single-key reader превращён в **полный машинный контракт**, а **разрознённое чтение вердиктов** пяти гейтов сведено к **одной точке парсинга**. Строго обратно совместимо, never-raise; `STAGE_TRANSITIONS` / состав `QG_CHECKS` / семантика вердиктов / fallback `worktree→origin/main` / трёх-полевой контракт tester (ORCH-047) — **1:1, без изменений**.
|
||||
- **`src/frontmatter.py` (контракт):** сохранён reader `read_frontmatter_value` (контракт неизменен — внешние вызыватели `usage.py` / `notifications.build_status_comment` не затронуты, INV-3); добавлены единый парс-примитив `parse_frontmatter(content) -> FrontmatterParse` (`data/has_block/malformed/yaml_error` — единственная точка YAML-логики) + ярлыки `parse_frontmatter_dict` / `read_frontmatter`; writer `render_frontmatter`/`write_frontmatter` (формат байт-совместим с `split("---",2)`+`yaml.safe_load`, round-trip render→parse); валидатор схемы `validate_schema`/`SchemaValidation`/`REQUIRED_FIELDS` (`work_item/stage/author_agent/status/created_at/model_used`); общий `strip_frontmatter`. Весь модуль — **never-raise** (NFR-2): любая ошибка I/O/YAML/сериализации → лог + безопасное значение (`{}`/`False`/исходный текст).
|
||||
- **Унифицирован МЕХАНИЗМ, а не семантика (D2):** пять вердикт-парсеров — `check_reviewer_verdict` (`verdict:`), `_parse_tests_verdict` (`result:`/`verdict:`/`status:`, ORCH-047), `_parse_deploy_status` (`deploy_status:`), `_parse_staging_status` (`staging_status:`) в `src/qg/checks.py`; `parse_security_status` (`security_status:`) в `src/security_gate.py` — заменили дублированный блок `startswith/split/safe_load/isinstance` на `parse_frontmatter(content)`; token-логика, upper-casing, приоритет негативного токена, reason-строки — сохранены 1:1. Также сняты дубли в `security_gate.extract_security_findings` и `review_parse._strip_frontmatter` (делегируют `strip_frontmatter`).
|
||||
- **Валидатор не hard-fail по умолчанию (D3, критично для self-hosting):** `maybe_warn_schema` при дефолте только логирует `logger.warning("frontmatter schema incomplete: …")` и **никогда не влияет на boolean-вердикт** гейта (инертен). Жёсткий режим — ТОЛЬКО под kill-switch `frontmatter_validation_strict` (env `ORCH_FRONTMATTER_VALIDATION_STRICT`, дефолт `False`; остаётся `False` в проде/`.env.staging`, иначе ORCH-52c self-block'нулась бы — её доки без полной схемы). Схема **аддитивна**: старый док-вердикт без новых полей читается ровно как раньше (FR-5/AC-4).
|
||||
- **Спека handoff:** новый `docs/_standards/HANDOFF_PROTOCOL.md` — формальный контракт «стадия → обязательные документы + frontmatter-ключи на выходе» + обязательная схема (`REQUIRED_FIELDS`), согласован 1:1 с `PIPELINE_DOCS.md` §2–§3; `PIPELINE_DOCS.md` §5–§6 обновлён (слой 2 реализован, ссылка на спеку и `src/frontmatter.py`).
|
||||
- **Без изменений API / схемы БД** (INV-5). Тесты: `tests/test_frontmatter.py` (TC-01…TC-07: writer/round-trip/валидатор/strict/never-raise/reader), `tests/test_qg_verdicts.py` (TC-08…TC-15: семантика пяти гейтов 1:1, обратная совместимость, fallback origin/main), `tests/test_security_gate.py` (TC-12), `tests/test_stages_invariants.py` (TC-16: `QG_CHECKS`/`STAGE_TRANSITIONS` неизменны). Полный регресс `tests/` зелёный (1212). Конфиг: `src/config.py` (`frontmatter_validation_strict`). Документация: `CLAUDE.md`, `docs/architecture/README.md`. ADR: `docs/work-items/ORCH-076/06-adr/ADR-001-frontmatter-contract.md`, сквозной `docs/architecture/adr/adr-0020-frontmatter-contract.md`.
|
||||
- **Стандарт документов конвейера: `docs/_standards/PIPELINE_DOCS.md` + `docs/_templates/` + ADR-naming** (ORCH-075 / ORCH-52b, `docs`): зафиксирован golden source структуры номерных документов work item (`00-business-request.md` … `17-security-report.md`). **Docs-only**, нулевой рантайм-риск: `STAGE_TRANSITIONS` / `QG_CHECKS` / `check_*` / `src/stage_engine.py` / схема БД — **не трогаются** (изменения только под `docs/**` + `CLAUDE.md`).
|
||||
- **Манифест** `docs/_standards/PIPELINE_DOCS.md` — карта «стадия → агент → документ → категория (`required`/`when-applicable`/`optional`) → гейт/механизм → frontmatter machine-key», сверенная с `src/stages.py` (`STAGE_TRANSITIONS`) и `src/qg/checks.py` (`_parse_*`). Манифест **документирует** поведение гейтов, но НЕ источник истины (источник — код, ADR-001 §D2); честно различает machine-verdict доки (`12`→`verdict:`, `13`→`result:`, `14`→`deploy_status:`, `15`→`staging_status:`, `17`→`security_status:`) и информационные (`00/08/10/16` — гейтом не парсятся). Под-гейты ребра `deploy-staging→deploy` (security/merge/image-freshness) помечены как врезки в `advance_stage`, а не строки `STAGE_TRANSITIONS`.
|
||||
- **Шаблоны** `docs/_templates/*` (15 копируемых скелетов) — для каждого `required`/`when-applicable` дока; машинные доки несут точный frontmatter-ключ из ground-truth (`_parse_*`), чтобы скопированный скелет проходил гейт без угадывания. Служебные каталоги `docs/_standards/` / `docs/_templates/` лежат ВНЕ `docs/work-items/<plane-id>/` → невидимы гейтам наличия файлов (`check_architecture_done`/`check_analysis_complete`).
|
||||
|
||||
@@ -117,7 +117,7 @@ created → analysis → architecture → development → review → testing →
|
||||
- ADR per work-item: `docs/work-items/<plane-id>/06-adr/ADR-NNN-slug.md`
|
||||
- Global ADR (сквозные решения): `docs/architecture/adr/adr-NNNN-slug.md`
|
||||
- Work items: `docs/work-items/<plane-id>/`
|
||||
- Машинные вердикты Quality Gate — строго YAML-frontmatter (`verdict:`, `deploy_status:`, `staging_status:`, `security_status:`), никогда проза
|
||||
- Машинные вердикты Quality Gate — строго YAML-frontmatter (`verdict:`, `deploy_status:`, `staging_status:`, `security_status:`), никогда проза. **ORCH-52c (ORCH-076):** парсинг frontmatter сведён к единому контракту `src/frontmatter.py` (reader `read_frontmatter_value` — BC; единый парс-примитив `parse_frontmatter`; writer `render/write_frontmatter`; валидатор схемы `validate_schema`/`REQUIRED_FIELDS` — warning-only по умолчанию, hard-fail только под kill-switch `frontmatter_validation_strict`, дефолт `False`). Пять вердикт-парсеров (`check_reviewer_verdict`, `_parse_tests_verdict`, `_parse_deploy_status`, `_parse_staging_status`, `parse_security_status`) читают через ОДНУ точку парсинга; семантика вердиктов и `STAGE_TRANSITIONS`/состав `QG_CHECKS` — 1:1. Формальная спека «стадия → обязательный выход» + обязательная frontmatter-схема — `docs/_standards/HANDOFF_PROTOCOL.md`
|
||||
|
||||
## Артефакты задачи (`docs/work-items/<plane-id>/`)
|
||||
`00-business-request.md`, `01-brd.md`, `02-trz.md`, `03-acceptance-criteria.md`, `04-test-plan.yaml`, `06-adr/ADR-NNN-slug.md`, `07-infra-requirements.md`, `08-data-requirements.md`, `10-tech-risks.md`, `12-review.md`, `13-test-report.md`, `14-deploy-log.md`, `15-staging-log.md`, `16-post-deploy-log.md` (post-deploy наблюдение, ORCH-021), `17-security-report.md` (security-гейт: `security_status:`/secrets/deps, ORCH-022).
|
||||
|
||||
118
docs/_standards/HANDOFF_PROTOCOL.md
Normal file
118
docs/_standards/HANDOFF_PROTOCOL.md
Normal file
@@ -0,0 +1,118 @@
|
||||
# HANDOFF_PROTOCOL — формальный контракт handoff «стадия → обязательный выход»
|
||||
|
||||
> **Назначение.** Нормативная спека: что КАЖДАЯ стадия конвейера обязана оставить на выходе —
|
||||
> какие документы и какие frontmatter-ключи. Дополняет [`PIPELINE_DOCS.md`](PIPELINE_DOCS.md)
|
||||
> (карта «документ → агент → стадия → гейт → machine-key») «вертикальным» срезом по стадиям и
|
||||
> вводит **обязательную frontmatter-схему** для машинной проверки.
|
||||
>
|
||||
> **Статус истины (важно).** Источник истины поведения — **код**: `src/stages.py`
|
||||
> (`STAGE_TRANSITIONS`), `src/qg/checks.py` (`QG_CHECKS` / `check_*` / `_parse_*`),
|
||||
> `src/stage_engine.py` (врезки под-гейтов). Машинный контракт чтения/записи/валидации
|
||||
> frontmatter — `src/frontmatter.py`. Эта спека **документирует**; при расхождении первичен код
|
||||
> (правило ORCH-075).
|
||||
|
||||
Введено задачей **ORCH-076** (ORCH-52c — слой 2 эпика ORCH-52: машинный контракт). Слой 1
|
||||
(ORCH-075/52b) дал описательный стандарт документов; ORCH-52c реализовала единый машинный
|
||||
frontmatter-контракт (reader + writer + валидатор) и свела чтение пяти вердиктов к одной точке
|
||||
парсинга. Сквозной ADR: [`adr-0020-frontmatter-contract.md`](../architecture/adr/adr-0020-frontmatter-contract.md);
|
||||
детально — [`ORCH-076/06-adr/ADR-001-frontmatter-contract.md`](../work-items/ORCH-076/06-adr/ADR-001-frontmatter-contract.md).
|
||||
|
||||
---
|
||||
|
||||
## 1. Обязательная frontmatter-схема (машинный источник: `frontmatter.REQUIRED_FIELDS`)
|
||||
|
||||
Forward-looking аддитивная схема: набор полей, которые handoff-документ стадии **должен** нести
|
||||
в ведущем YAML-frontmatter. Машинный источник истины — кортеж
|
||||
[`src/frontmatter.py`](../../src/frontmatter.py) `REQUIRED_FIELDS`:
|
||||
|
||||
| Поле | Смысл |
|
||||
|------|-------|
|
||||
| `work_item` | ID задачи (`ORCH-NNN` / `ET-NNN`) — к какой задаче относится выход |
|
||||
| `stage` | стадия, на выходе которой написан документ (`analysis` … `deploy`) |
|
||||
| `author_agent` | роль-автор (`analyst` / `architect` / `developer` / `reviewer` / `tester` / `deployer`) |
|
||||
| `status` | человеко/машинно-читаемый статус выхода стадии |
|
||||
| `created_at` | дата создания артефакта (YYYY-MM-DD) |
|
||||
| `model_used` | модель агента, сгенерировавшего артефакт (`claude-…`) |
|
||||
|
||||
**Режим проверки (ORCH-52c, критично для self-hosting).** Валидатор схемы
|
||||
`frontmatter.validate_schema` / `maybe_warn_schema` по умолчанию **warning-only** и **никогда не
|
||||
влияет на boolean-вердикт ни одного гейта**: отсутствие полей логируется (`logger.warning`), но не
|
||||
роняет конвейер и не заваливает гейт. Жёсткий режим (hard-fail) зарезервирован на будущее
|
||||
(ORCH-52d) и включается ТОЛЬКО kill-switch'ем `frontmatter_validation_strict`
|
||||
(env `ORCH_FRONTMATTER_VALIDATION_STRICT`, дефолт `False`). Схема **аддитивна**: старый
|
||||
документ-вердикт без этих полей читается гейтом ровно как раньше (см. §3).
|
||||
|
||||
---
|
||||
|
||||
## 2. Контракт handoff по стадиям
|
||||
|
||||
Категории документов — как в `PIPELINE_DOCS.md` §2: **required** (всегда), **when-applicable**
|
||||
(при наличии предмета: инфра / данные / security / post-deploy — отсутствие не нарушение).
|
||||
«Machine-verdict ключ» — поле, которое exit-гейт/под-гейт ребра читает ТОЛЬКО из frontmatter
|
||||
(никогда из прозы). Набор документов/ключей/гейтов **согласован 1:1 с `PIPELINE_DOCS.md` §2–§3**.
|
||||
|
||||
| Стадия (выход) | Агент | Обязательные документы на выходе | Machine-verdict ключ (читает гейт ребра) | Гейт ребра |
|
||||
|----------------|-------|----------------------------------|------------------------------------------|------------|
|
||||
| `created` | система (`_create_initial_docs`) / заказчик | `00-business-request.md` | — (вход, не гейтится) | — |
|
||||
| `analysis` | analyst | `01-brd.md`, `02-trz.md`, `03-acceptance-criteria.md`, `04-test-plan.yaml` | — (гейт проверяет наличие файлов + Approved) | `check_analysis_approved` |
|
||||
| `architecture` | architect | `06-adr/ADR-NNN-<slug>.md` (≥1); `07-infra-requirements.md`, `08-data-requirements.md`, `10-tech-risks.md` (when-applicable/required-info) | — (гейт проверяет наличие `06-adr/` ≥1 ИЛИ `07-…`) | `check_architecture_done` |
|
||||
| `development` | developer | код + тесты в ветке (артефакт-док не пишется; гейт — зелёный CI) | — (гейт читает CI-статус Gitea) | `check_ci_green` |
|
||||
| `review` | reviewer | `12-review.md` | `verdict:` (`APPROVED` \| `REQUEST_CHANGES`) | `check_reviewer_verdict` |
|
||||
| `testing` | tester | `13-test-report.md` | `result:` / `verdict:` / `status:` (`PASS` \| `FAIL` \| `BLOCKED`; три равноранговых, ORCH-047) | `check_tests_passed` |
|
||||
| `deploy-staging` | deployer | `15-staging-log.md` (required для self-hosting); `17-security-report.md` (security-под-гейт, when-applicable) | `staging_status:` (`SUCCESS` \| `FAILED`); `security_status:` (`PASS` \| `FAIL`) | `check_staging_status` (ребро); под-гейты ребра `deploy-staging→deploy`: `check_security_gate` → `check_branch_mergeable` → `check_staging_image_fresh` |
|
||||
| `deploy` | deployer / deploy-finalizer | `14-deploy-log.md` | `deploy_status:` (`SUCCESS` \| `FAILED`) | `check_deploy_status` |
|
||||
| `done` | — | — (терминал) | — | — |
|
||||
| пост-`done` наблюдение | post-deploy-monitor | `16-post-deploy-log.md` (when-applicable, ORCH-021) | `post_deploy_status:` (`HEALTHY` \| `DEGRADED`) — **информационный, не гейт** | — (телеметрия петли уроков / наблюдаемость) |
|
||||
|
||||
### Примечания (нормативные)
|
||||
|
||||
- **Под-гейты ребра `deploy-staging → deploy`** (`check_security_gate` → `check_branch_mergeable`
|
||||
→ `check_staging_image_fresh`) — это **врезки в `advance_stage`**, а НЕ строки
|
||||
`STAGE_TRANSITIONS`. Их порядок и условность раската не меняются этой спекой.
|
||||
- **`15-staging-log.md`** обязателен только для self-hosting репо (`orchestrator`); для прочих
|
||||
репо staging-гейт — N/A (ORCH-35), документ не требуется.
|
||||
- **`16-post-deploy-log.md`** несёт `post_deploy_status:`, но это **информационный** ключ
|
||||
(телеметрия ORCH-8 / наблюдаемость), гейтом он НЕ парсится.
|
||||
- **`09-…` / `05-…` / `11-…`** — зарезервированные/legacy номера; канон reviewer'а — `12-review.md`.
|
||||
|
||||
---
|
||||
|
||||
## 3. Machine-verdict доки vs информационные (честный механизм проверки)
|
||||
|
||||
Полностью согласовано с `PIPELINE_DOCS.md` §3. Machine-verdict док — гейт читает ТОЛЬКО
|
||||
YAML-frontmatter (через единый `frontmatter.parse_frontmatter`), маппит ключ в вердикт; имя ключа
|
||||
чувствительно к регистру, значение парсер приводит к верхнему регистру.
|
||||
|
||||
| Документ | Machine-key | Парсер | Эффект вердикта |
|
||||
|----------|-------------|--------|-----------------|
|
||||
| `12-review.md` | `verdict:` | `check_reviewer_verdict` | `APPROVED` → дальше; `REQUEST_CHANGES` → откат на `development` |
|
||||
| `13-test-report.md` | `result:` / `verdict:` / `status:` | `_parse_tests_verdict` | `PASS` → дальше; `FAIL`/`BLOCKED` → откат (негативный токен авторитетен) |
|
||||
| `14-deploy-log.md` | `deploy_status:` | `_parse_deploy_status` | `SUCCESS` → `done`; `FAILED` → откат (БАГ-8) |
|
||||
| `15-staging-log.md` | `staging_status:` | `_parse_staging_status` | `SUCCESS` → дальше; `FAILED` → откат (self-hosting; иначе N/A) |
|
||||
| `17-security-report.md` | `security_status:` | `check_security_gate` → `parse_security_status` | `PASS` → дальше; `FAIL` → откат |
|
||||
|
||||
**Информационные доки** (гейтом НЕ парсятся): `00-business-request.md`, `08-data-requirements.md`,
|
||||
`10-tech-risks.md`, `16-post-deploy-log.md`.
|
||||
|
||||
**Аддитивность схемы (§1).** Документ-вердикт БЕЗ полей схемы из §1, но с вердикт-ключом, читается
|
||||
гейтом РОВНО как раньше: схема не участвует в вычислении вердикта при дефолтном
|
||||
`frontmatter_validation_strict=False`.
|
||||
|
||||
---
|
||||
|
||||
## 4. Единый машинный контракт — `src/frontmatter.py`
|
||||
|
||||
Все операции с frontmatter сведены в один leaf-модуль (never-raise):
|
||||
|
||||
- `read_frontmatter_value(path, key) -> str | None` — single-key reader (контракт неизменен, BC).
|
||||
- `parse_frontmatter(content) -> FrontmatterParse` — **единственная точка** парсинга YAML-frontmatter
|
||||
(`data` / `has_block` / `malformed` / `yaml_error`); пять вердикт-парсеров делегируют сюда.
|
||||
- `parse_frontmatter_dict` / `read_frontmatter` — ярлыки к распарсенному mapping.
|
||||
- `render_frontmatter` / `write_frontmatter` — writer (формат совместим с существующими парсерами).
|
||||
- `validate_schema` / `REQUIRED_FIELDS` / `maybe_warn_schema` — схема §1 (warning-only по умолчанию).
|
||||
- `strip_frontmatter` — общий хелпер тела (заменил дубли).
|
||||
- Kill-switch жёсткой валидации: `config.frontmatter_validation_strict`
|
||||
(env `ORCH_FRONTMATTER_VALIDATION_STRICT`, дефолт `False`).
|
||||
|
||||
> Перед написанием номерного дока бери скелет из [`docs/_templates/`](../_templates/) и **не меняй
|
||||
> имя machine-key frontmatter** (регистр чувствителен — иначе гейт упадёт ложно).
|
||||
@@ -133,6 +133,21 @@ check_tests_passed → check_staging_status → check_deploy_status`.
|
||||
ключа** (регистр чувствителен — иначе гейт упадёт ложно).
|
||||
3. Сверяйся с манифестом (§2–§3): какой агент, на какой стадии, какой гейт читает документ.
|
||||
|
||||
> Стандарт **описательный** (слой 1). Машинная проверка соответствия шаблонам/frontmatter —
|
||||
> отдельная задача ORCH-52c; до неё соблюдение — на ответственности агента и reviewer'а
|
||||
> (правило CLAUDE.md «обновлена ли документация»).
|
||||
> Стандарт **описательный** (слой 1). **Машинный слой реализован в ORCH-52c (ORCH-076):** единый
|
||||
> frontmatter-контракт (reader + writer + валидатор) в [`src/frontmatter.py`](../../src/frontmatter.py)
|
||||
> и формальная спека handoff [`HANDOFF_PROTOCOL.md`](HANDOFF_PROTOCOL.md) («стадия → обязательный
|
||||
> выход» + обязательная frontmatter-схема `REQUIRED_FIELDS`). Пять вердикт-парсеров
|
||||
> (`check_reviewer_verdict`, `_parse_tests_verdict`, `_parse_deploy_status`, `_parse_staging_status`,
|
||||
> `parse_security_status`) читают вердикт через ОДНУ точку парсинга (`parse_frontmatter`); семантика
|
||||
> вердиктов 1:1. Валидатор обязательной схемы по умолчанию **warning-only** (kill-switch
|
||||
> `frontmatter_validation_strict`, дефолт `False`) — соблюдение схемы пока на ответственности агента
|
||||
> и reviewer'а, enforcement придёт с ORCH-52d.
|
||||
|
||||
---
|
||||
|
||||
## 6. Спека handoff (машинный контракт, ORCH-52c)
|
||||
|
||||
Вертикальный срез «стадия → обязательные документы + frontmatter-ключи на выходе» и обязательная
|
||||
frontmatter-схема вынесены в отдельную нормативную спеку [`HANDOFF_PROTOCOL.md`](HANDOFF_PROTOCOL.md)
|
||||
(набор документов/ключей/гейтов согласован 1:1 с §2–§3 этого манифеста). Машинный источник
|
||||
обязательной схемы — `frontmatter.REQUIRED_FIELDS`.
|
||||
|
||||
@@ -39,7 +39,7 @@ created → analysis → architecture → development → review → testing →
|
||||
|
||||
**Реестр QG** (`QG_CHECKS`): check_analysis_approved, check_analysis_complete, check_architecture_done, check_ci_green, check_review_approved, check_tests_passed, check_reviewer_verdict, check_tests_local, check_deploy_status, check_staging_status, check_branch_mergeable (ORCH-043), check_staging_image_fresh (ORCH-058), check_security_gate (ORCH-022).
|
||||
|
||||
**Канон гейтов:** машинные вердикты читаются ТОЛЬКО из YAML-frontmatter, никогда из прозы. Лог-файлы мержатся в `origin/main` отдельным PR; гейт читает из `origin/main`.
|
||||
**Канон гейтов:** машинные вердикты читаются ТОЛЬКО из YAML-frontmatter, никогда из прозы. Лог-файлы мержатся в `origin/main` отдельным PR; гейт читает из `origin/main`. **Единый frontmatter-контракт (ORCH-52c / ORCH-076):** парсинг YAML-frontmatter сведён к одной точке — `src/frontmatter.parse_frontmatter` (структура `data/has_block/malformed/yaml_error`, never-raise); пять вердикт-парсеров (`check_reviewer_verdict`, `_parse_tests_verdict`, `_parse_deploy_status`, `_parse_staging_status`, `parse_security_status`) делегируют ей вместо дублированной ad-hoc логики. Модуль также несёт writer (`render/write_frontmatter`), валидатор обязательной схемы (`validate_schema`/`REQUIRED_FIELDS`, warning-only по умолчанию; hard-fail только под kill-switch `frontmatter_validation_strict`, дефолт `False`) и общий `strip_frontmatter`. Семантика вердиктов / `STAGE_TRANSITIONS` / состав `QG_CHECKS` — без изменений (1:1).
|
||||
|
||||
### Стандарт документов конвейера (ORCH-075, ORCH-52b)
|
||||
Структура номерных документов work item (`00-business-request.md` … `17-security-report.md`),
|
||||
@@ -49,9 +49,13 @@ created → analysis → architecture → development → review → testing →
|
||||
[`docs/_templates/`](../_templates/). Манифест **документирует** поведение гейтов (источник истины
|
||||
остаётся код: `src/stages.py`, `src/qg/checks.py`), честно различая machine-verdict доки
|
||||
(`12/13/14/15/17` — несут читаемый гейтом ключ) и информационные (`00/08/10/16` — гейтом не
|
||||
парсятся). Это слой 1 (описательный); машинная проверка соответствия — задача ORCH-52c. ADR:
|
||||
[adr-0019](adr/adr-0019-pipeline-docs-standard.md), детально —
|
||||
`docs/work-items/ORCH-075/06-adr/ADR-001-pipeline-docs-standard.md`.
|
||||
парсятся). Это слой 1 (описательный). **Слой 2 (машинный) реализован в ORCH-52c (ORCH-076):**
|
||||
единый frontmatter-контракт `src/frontmatter.py` + формальная спека handoff «стадия → обязательный
|
||||
выход» с обязательной frontmatter-схемой (`REQUIRED_FIELDS`) —
|
||||
[`docs/_standards/HANDOFF_PROTOCOL.md`](../_standards/HANDOFF_PROTOCOL.md). ADR:
|
||||
[adr-0019](adr/adr-0019-pipeline-docs-standard.md) / [adr-0020](adr/adr-0020-frontmatter-contract.md),
|
||||
детально — `docs/work-items/ORCH-075/06-adr/ADR-001-pipeline-docs-standard.md`,
|
||||
`docs/work-items/ORCH-076/06-adr/ADR-001-frontmatter-contract.md`.
|
||||
|
||||
### Модель и эффорт по ролям (ORCH-41, валидация ORCH-74)
|
||||
Модель и `--effort` каждого агента берутся из config (`src/config.py`), резолвятся `launcher.resolve_agent_model` / `resolve_agent_effort` по приоритету **project-override (`projects_json` `agent_models`/`agent_efforts`) > `ORCH_AGENT_MODEL_<AGENT>`/`ORCH_AGENT_EFFORT_<AGENT>` > `*_default` > CLI-дефолт (без флага)**. **Эффорт (ORCH-081):** ниже `*_default` добавлен непустой **per-role floor** — class-default поля `agent_effort_<role>` из `config.py` (его пустой env перебить не может). Floor — строго последний уровень (ниже default) и срабатывает ТОЛЬКО когда все уровни пусты, поэтому пустые прод-`ORCH_AGENT_EFFORT_*=` (которые pydantic трактует как явное `''` и обнуляют дефолт) больше не приводят к запуску без `--effort`: каждая роль получает свой канонический пол (developer=`xhigh`, tester/deployer=`medium`, прочие=`high`). Непустой явный конфиг по-прежнему побеждает floor; опечатка вне `VALID_EFFORTS` дропается валидацией ДО floor (never-break, не маскируется). См. `docs/work-items/ORCH-081/06-adr/ADR-001-effort-resolution-floor.md`. frontmatter `model:` в `.openclaw/agents/*.md` **удалён** (ORCH-74 G1) — он был мёртвой/лживой декларацией (launcher его не читает); config — единственный источник правды о модели. Model-routing (G3) НЕ включён — все 6 агентов на `claude-opus-4-8`.
|
||||
@@ -642,7 +646,7 @@ Monitoring after Deploy → Done
|
||||
```
|
||||
|
||||
- **Длительность** считается launcher'ом (`_monitor_agent`) и пробрасывается в `_post_usage_comments`; для analyst (коммент строится в `stage_engine`) используется DB-фоллбэк `usage.get_agent_duration(task_id, agent)`.
|
||||
- **Vердикт-парсер** — `src/frontmatter.read_frontmatter_value(...)` (defensive, никогда не raise). Машинные ключи: reviewer → `verdict:` (12-review.md); **testing-гейт `check_tests_passed` (13-test-report.md) → любое из трёх равноправных: `result:` (канон промпта тестера), `verdict:`, `status:`** (ORCH-047, ADR-001); deployer → `deploy_status:` (14-deploy-log.md), `staging_status:` (15-staging-log.md). Negative-токен в любом поле авторитетен (перебивает positive).
|
||||
- **Vердикт-парсер** — единый контракт `src/frontmatter.py` (defensive, never-raise): коммент-хелпер использует `read_frontmatter_value(...)` (single-key, BC), гейты — `parse_frontmatter(...)` (ORCH-52c). Машинные ключи: reviewer → `verdict:` (12-review.md); **testing-гейт `check_tests_passed` (13-test-report.md) → любое из трёх равноправных: `result:` (канон промпта тестера), `verdict:`, `status:`** (ORCH-047, ADR-001); deployer → `deploy_status:` (14-deploy-log.md), `staging_status:` (15-staging-log.md). Negative-токен в любом поле авторитетен (перебивает positive).
|
||||
- Формат коммента **не** меняет реестр гейтов и стадий; коммент — отображение, не управление.
|
||||
|
||||
## База данных (SQLite)
|
||||
|
||||
@@ -553,6 +553,17 @@ class Settings(BaseSettings):
|
||||
# ORCH_TRACKER_BRD_REVIEW_CAP_S; default 7200s (2h). 0/negative -> no cap.
|
||||
tracker_brd_review_cap_s: int = 7200
|
||||
|
||||
# ORCH-076 (ORCH-52c, FR-2 / D3): kill-switch for STRICT frontmatter-schema
|
||||
# validation. The unified frontmatter contract (src/frontmatter.py) ships a
|
||||
# machine-checkable schema validator (REQUIRED_FIELDS), but by DEFAULT it is
|
||||
# warning-only and never influences any gate's boolean verdict (maybe_warn_schema
|
||||
# is inert). This flag is RESERVED for a future tightening (ORCH-52d, when agents
|
||||
# start emitting the full schema). It MUST stay False in prod / .env.staging —
|
||||
# otherwise ORCH-52c would self-block its own deploy (its docs predate the
|
||||
# schema). Env ORCH_FRONTMATTER_VALIDATION_STRICT; default False (zero behaviour
|
||||
# change). See docs/_standards/HANDOFF_PROTOCOL.md.
|
||||
frontmatter_validation_strict: bool = False
|
||||
|
||||
# ORCH-069: QG-0 upper title-length limit (entry gate _qg0_errors). The 80-char
|
||||
# cap was a hygiene limit, not structural (slug is cut to [:30] independently,
|
||||
# DB title TEXT is unbounded). Configurable via env ORCH_QG0_TITLE_MAX; default
|
||||
|
||||
@@ -1,27 +1,144 @@
|
||||
"""Safe single-key YAML frontmatter reader (ORCH-016 / ADR-001 §5).
|
||||
"""Unified YAML-frontmatter contract — reader + writer + schema validator (ORCH-52c).
|
||||
|
||||
The status-comment builder (build_status_comment) needs to surface verdict /
|
||||
deploy_status / staging_status from the per-stage artifact files (12-review.md,
|
||||
13-test-report.md, 14-deploy-log.md, 15-staging-log.md). Those files share the
|
||||
same leading-YAML-frontmatter convention used by the quality gates — but the
|
||||
comment hot-path must NEVER raise: a missing file, malformed YAML, or absent
|
||||
key should simply suppress the verdict line, not break the run.
|
||||
History
|
||||
-------
|
||||
ORCH-016 introduced this module as a *single-key reader* (``read_frontmatter_value``)
|
||||
for the status-comment hot path, intentionally duplicating ~10 lines of
|
||||
YAML-frontmatter logic already present in ``src/qg/checks.py`` and
|
||||
``src/security_gate.py`` to keep that PR's blast radius small. Its docstring noted
|
||||
*"merging into a single parser is a follow-up task"* — this module (ORCH-52c /
|
||||
ORCH-076) is that follow-up.
|
||||
|
||||
This module is a tiny defensive helper:
|
||||
- `read_frontmatter_value(path, key)` -> str | None
|
||||
- swallows every exception, logs to logger.debug, returns None.
|
||||
What this module now provides (ADR-001 / adr-0020)
|
||||
--------------------------------------------------
|
||||
A **single point of YAML-frontmatter parsing** that every verdict gate delegates to,
|
||||
plus a writer and a (warning-only by default) schema validator:
|
||||
|
||||
It intentionally duplicates ~10 lines of YAML-frontmatter logic that already
|
||||
exist in `src/qg/checks.py` (S-5 / БАГ 8 / ET-013 fixes). ADR-001 §5 accepts
|
||||
this duplication to keep the blast radius of ORCH-016 small (no QG refactor in
|
||||
this PR); merging into a single parser is a follow-up task.
|
||||
* ``read_frontmatter_value(path, key)`` — UNCHANGED single-key reader (INV-3): the
|
||||
external callers (``usage.py``, ``notifications.build_status_comment``) keep the
|
||||
exact same contract (``str | None``, never-raise, strip, case preserved).
|
||||
* ``parse_frontmatter(content)`` — the ONE YAML parse primitive; returns a
|
||||
structured :class:`FrontmatterParse` so each gate can reproduce its current
|
||||
reason-strings 1:1 (no-block / malformed / yaml-error / data).
|
||||
* ``parse_frontmatter_dict`` / ``read_frontmatter`` — convenience shortcuts to the
|
||||
parsed mapping (in-memory / from a file).
|
||||
* ``render_frontmatter`` / ``write_frontmatter`` — canonical writer; the output is
|
||||
byte-compatible with the existing ``split("---", 2)`` + ``yaml.safe_load`` parsers.
|
||||
* ``validate_schema`` + ``REQUIRED_FIELDS`` + ``maybe_warn_schema`` — the machine
|
||||
schema (``work_item / stage / author_agent / status / created_at / model_used``).
|
||||
By default it is **warning-only** and never influences any gate's boolean verdict
|
||||
(NFR-3 / INV-4); the strict mode is reserved for ORCH-52d and gated behind the
|
||||
``frontmatter_validation_strict`` kill-switch (default ``False``).
|
||||
* ``strip_frontmatter(content)`` — shared body-only helper (replaces the duplicated
|
||||
``_strip_frontmatter`` in ``review_parse``).
|
||||
|
||||
Contract — the WHOLE module is **never-raise** (NFR-2), exactly like the original
|
||||
reader: any error (I/O, YAML, serialization) is logged to ``logger.debug/warning``
|
||||
and degrades to a safe value (``{}`` / ``False`` / the input text); an exception
|
||||
never escapes into the pipeline. This is a hard self-hosting requirement: these
|
||||
functions read verdicts ON THE GATES of the instance that serves prod for every
|
||||
project from one shared DB/queue, so a regression here would stall every project.
|
||||
|
||||
This module is a **leaf**: it imports only ``logging`` (and lazily ``yaml``); it does
|
||||
not import anything project-specific, so it stays cycle-free for ``qg/checks.py``,
|
||||
``security_gate.py``, ``post_deploy.py`` and ``review_parse.py``.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Mapping
|
||||
|
||||
logger = logging.getLogger("orchestrator.frontmatter")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Schema constants (the machine-checkable required frontmatter — FR-2 / D3)
|
||||
# ---------------------------------------------------------------------------
|
||||
#: The required frontmatter fields a stage handoff document is expected to carry.
|
||||
#: Source of truth for HANDOFF_PROTOCOL.md §2. The validator is warning-only by
|
||||
#: default (D3) — its presence does NOT gate the pipeline unless the
|
||||
#: ``frontmatter_validation_strict`` kill-switch is flipped on (reserved, ORCH-52d).
|
||||
REQUIRED_FIELDS = ("work_item", "stage", "author_agent", "status", "created_at", "model_used")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Parse primitive — the SINGLE point of YAML-frontmatter logic (D1 / D2)
|
||||
# ---------------------------------------------------------------------------
|
||||
@dataclass(frozen=True)
|
||||
class FrontmatterParse:
|
||||
"""Structured outcome of parsing a document's leading YAML frontmatter.
|
||||
|
||||
The structure (not a bare dict) lets each gate reproduce its EXISTING
|
||||
reason-strings 1:1 (ADR-001 D2): a gate can tell "no block" from "malformed"
|
||||
from "yaml error" from "valid data" without re-implementing the parse.
|
||||
|
||||
Attributes:
|
||||
data: the parsed mapping; ``{}`` when absent / malformed / not a mapping.
|
||||
has_block: a leading ``---`` … ``---`` block was present.
|
||||
malformed: the content started with ``---`` but had < 3 ``---``-split segments
|
||||
(an unterminated frontmatter block).
|
||||
yaml_error: the ``yaml.safe_load`` error text, else ``None``.
|
||||
"""
|
||||
|
||||
data: dict = field(default_factory=dict)
|
||||
has_block: bool = False
|
||||
malformed: bool = False
|
||||
yaml_error: str | None = None
|
||||
|
||||
|
||||
def parse_frontmatter(content: str) -> FrontmatterParse:
|
||||
"""Parse the leading YAML frontmatter of ``content`` into a :class:`FrontmatterParse`.
|
||||
|
||||
The single canonical implementation of the block that used to be duplicated in
|
||||
every verdict gate (``content.startswith("---")`` -> ``split("---", 2)`` ->
|
||||
``yaml.safe_load`` -> ``isinstance(dict)``). Never raises:
|
||||
|
||||
* not a string / no leading ``---`` -> ``has_block=False``, ``data={}``.
|
||||
* ``---`` but < 3 segments (unterminated) -> ``malformed=True``, ``data={}``.
|
||||
* ``yaml.safe_load`` error -> ``yaml_error=<text>``, ``data={}``.
|
||||
* parsed value is not a mapping -> ``data={}`` (``has_block=True``).
|
||||
* valid mapping -> ``data=<dict>``.
|
||||
"""
|
||||
try:
|
||||
if not isinstance(content, str) or not content.startswith("---"):
|
||||
return FrontmatterParse()
|
||||
|
||||
parts = content.split("---", 2)
|
||||
if len(parts) < 3:
|
||||
# Unterminated frontmatter block.
|
||||
return FrontmatterParse(has_block=True, malformed=True)
|
||||
|
||||
try:
|
||||
import yaml
|
||||
loaded = yaml.safe_load(parts[1])
|
||||
except Exception as e: # yaml.YAMLError + anything pyyaml may surface
|
||||
logger.debug(f"parse_frontmatter: yaml parse failed: {e}")
|
||||
return FrontmatterParse(has_block=True, yaml_error=str(e))
|
||||
|
||||
if not isinstance(loaded, dict):
|
||||
return FrontmatterParse(has_block=True)
|
||||
return FrontmatterParse(data=loaded, has_block=True)
|
||||
except Exception as e: # noqa: BLE001 - never-raise contract
|
||||
logger.debug(f"parse_frontmatter: unexpected error: {e}")
|
||||
return FrontmatterParse()
|
||||
|
||||
|
||||
def parse_frontmatter_dict(content: str) -> dict:
|
||||
"""Shortcut: the parsed mapping of ``content``'s frontmatter. Never raises -> ``{}``."""
|
||||
return parse_frontmatter(content).data
|
||||
|
||||
|
||||
def read_frontmatter(path: str) -> dict:
|
||||
"""Read ``path`` and return its parsed frontmatter mapping. Never raises -> ``{}``."""
|
||||
try:
|
||||
with open(path, "r", encoding="utf-8", errors="replace") as f:
|
||||
content = f.read()
|
||||
except OSError as e:
|
||||
logger.debug(f"read_frontmatter: cannot open {path}: {e}")
|
||||
return {}
|
||||
return parse_frontmatter(content).data
|
||||
|
||||
|
||||
def read_frontmatter_value(path: str, key: str) -> str | None:
|
||||
"""Return the value of `key` from the leading YAML frontmatter of `path`.
|
||||
|
||||
@@ -42,34 +159,158 @@ def read_frontmatter_value(path: str, key: str) -> str | None:
|
||||
|
||||
The returned value is stringified and stripped (whitespace removed); casing
|
||||
is preserved so the caller decides whether to upper/lower for matching.
|
||||
|
||||
ORCH-52c: reimplemented on top of the unified ``read_frontmatter`` primitive.
|
||||
The external contract (``str | None``, never-raise, strip, case preserved) is
|
||||
UNCHANGED — external callers (``usage.py``, ``notifications``) are unaffected
|
||||
(INV-3 / FR-3).
|
||||
"""
|
||||
try:
|
||||
with open(path, "r", encoding="utf-8", errors="replace") as f:
|
||||
content = f.read()
|
||||
except OSError as e:
|
||||
logger.debug(f"read_frontmatter_value: cannot open {path}: {e}")
|
||||
return None
|
||||
|
||||
if not content.startswith("---"):
|
||||
return None
|
||||
|
||||
parts = content.split("---", 2)
|
||||
if len(parts) < 3:
|
||||
# Unterminated frontmatter.
|
||||
return None
|
||||
|
||||
try:
|
||||
import yaml
|
||||
fm = yaml.safe_load(parts[1]) or {}
|
||||
except Exception as e: # yaml.YAMLError + anything pyyaml may surface
|
||||
logger.debug(f"read_frontmatter_value: yaml parse failed for {path}: {e}")
|
||||
return None
|
||||
|
||||
if not isinstance(fm, dict):
|
||||
return None
|
||||
|
||||
fm = read_frontmatter(path)
|
||||
raw = fm.get(key)
|
||||
if raw is None:
|
||||
return None
|
||||
value = str(raw).strip()
|
||||
return value or None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Body helper — replaces the duplicated review_parse._strip_frontmatter (D2)
|
||||
# ---------------------------------------------------------------------------
|
||||
def strip_frontmatter(content: str) -> str:
|
||||
"""Return ``content`` with a leading ``--- … ---`` YAML block removed, if present.
|
||||
|
||||
Mirrors the previous ``review_parse._strip_frontmatter`` exactly: only a
|
||||
well-formed (>= 3 ``---``-split segments) leading block is stripped; otherwise
|
||||
the input is returned unchanged. Never raises -> the input text.
|
||||
"""
|
||||
try:
|
||||
if isinstance(content, str) and content.startswith("---"):
|
||||
parts = content.split("---", 2)
|
||||
if len(parts) >= 3:
|
||||
return parts[2]
|
||||
except Exception as e: # noqa: BLE001 - never-raise contract
|
||||
logger.debug(f"strip_frontmatter: unexpected error: {e}")
|
||||
return content
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Writer — canonical render/persist (FR-1 / D1)
|
||||
# ---------------------------------------------------------------------------
|
||||
def render_frontmatter(data: Mapping[str, object], body: str = "") -> str:
|
||||
"""Render ``data`` as a canonical leading YAML-frontmatter block + ``body``.
|
||||
|
||||
Output shape: ``"---\\n<yaml>\\n---\\n<body>"``. The YAML is emitted with
|
||||
``yaml.safe_dump`` (block style, keys unsorted) so it is byte-compatible with
|
||||
the existing readers (``split("---", 2)`` + ``yaml.safe_load``): a round-trip
|
||||
``render_frontmatter`` -> ``parse_frontmatter`` returns the same mapping.
|
||||
|
||||
never-raise (NFR-2): a serialization error is logged and the function degrades
|
||||
to returning ``body`` unchanged (a document with no frontmatter is read by the
|
||||
gates exactly as "no machine verdict", never an exception).
|
||||
"""
|
||||
try:
|
||||
import yaml
|
||||
dumped = yaml.safe_dump(
|
||||
dict(data or {}), default_flow_style=False, sort_keys=False, allow_unicode=True
|
||||
).strip("\n")
|
||||
return f"---\n{dumped}\n---\n{body}"
|
||||
except Exception as e: # noqa: BLE001 - never-raise contract
|
||||
logger.warning(f"render_frontmatter: serialization failed: {e}")
|
||||
return body
|
||||
|
||||
|
||||
def write_frontmatter(path: str, data: Mapping[str, object], body: str = "") -> bool:
|
||||
"""Persist ``render_frontmatter(data, body)`` to ``path``. Returns True on success.
|
||||
|
||||
never-raise (NFR-2): any I/O / serialization error is logged and returns
|
||||
``False`` (the caller decides how to degrade); an exception never escapes.
|
||||
"""
|
||||
try:
|
||||
content = render_frontmatter(data, body)
|
||||
with open(path, "w", encoding="utf-8") as f:
|
||||
f.write(content)
|
||||
return True
|
||||
except Exception as e: # noqa: BLE001 - never-raise contract
|
||||
logger.warning(f"write_frontmatter: cannot write {path}: {e}")
|
||||
return False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Schema validator — warning-only by default; strict reserved (FR-2 / D3)
|
||||
# ---------------------------------------------------------------------------
|
||||
@dataclass(frozen=True)
|
||||
class SchemaValidation:
|
||||
"""Outcome of :func:`validate_schema`.
|
||||
|
||||
valid: all required fields are present and non-empty.
|
||||
missing: the required fields that are absent / None / blank (order = REQUIRED_FIELDS).
|
||||
"""
|
||||
|
||||
valid: bool
|
||||
missing: list
|
||||
|
||||
|
||||
def validate_schema(data: Mapping, *, required=REQUIRED_FIELDS) -> SchemaValidation:
|
||||
"""Validate that ``data`` carries every required schema field, non-empty.
|
||||
|
||||
Pure library function (INV-4). A field counts as MISSING when it is absent, or
|
||||
its value is ``None`` or — after ``str(...).strip()`` — empty. Returns a
|
||||
:class:`SchemaValidation`; never raises (a non-mapping input -> all fields
|
||||
missing -> ``valid=False``). This function NEVER influences a gate verdict by
|
||||
itself — see :func:`maybe_warn_schema` and the ``frontmatter_validation_strict``
|
||||
flag for how strict enforcement is (and is not) wired.
|
||||
"""
|
||||
missing: list = []
|
||||
try:
|
||||
mapping = data if isinstance(data, Mapping) else {}
|
||||
for fld in required:
|
||||
raw = mapping.get(fld)
|
||||
if raw is None or str(raw).strip() == "":
|
||||
missing.append(fld)
|
||||
except Exception as e: # noqa: BLE001 - never-raise contract
|
||||
logger.warning(f"validate_schema: unexpected error: {e}")
|
||||
# Conservatively report everything missing rather than raise.
|
||||
missing = list(required)
|
||||
return SchemaValidation(valid=not missing, missing=missing)
|
||||
|
||||
|
||||
def maybe_warn_schema(content: str, doc_label: str = "document") -> SchemaValidation:
|
||||
"""Best-effort schema check used at verdict-read sites — warning-only by default.
|
||||
|
||||
Parses ``content``'s frontmatter and validates it against :data:`REQUIRED_FIELDS`.
|
||||
Behaviour is governed by the ``frontmatter_validation_strict`` kill-switch
|
||||
(default ``False``):
|
||||
|
||||
* **default (False)** — when fields are missing, emit a single
|
||||
``logger.warning("frontmatter schema incomplete: missing …")`` and return.
|
||||
The result is **inert**: callers that pass it through a gate must NOT change
|
||||
their ``tuple[bool, str]`` verdict (FR-2 "warning/лог, не blocker"). This
|
||||
keeps a machine-verdict doc that lacks the (forward-looking, additive) schema
|
||||
readable exactly as before (FR-5 / AC-4) — critical so ORCH-52c does not
|
||||
self-block its own deploy (its docs predate the schema).
|
||||
* **strict (True)** — RESERVED for a future tightening (ORCH-52d+). The
|
||||
validation result is returned the same way; the flag merely documents intent
|
||||
and lets a future caller veto. It stays ``False`` in prod and ``.env.staging``.
|
||||
|
||||
Never raises (NFR-2): a config-read or parse error degrades to ``valid=True``
|
||||
(no false warning, no influence on the verdict).
|
||||
"""
|
||||
try:
|
||||
data = parse_frontmatter(content).data
|
||||
result = validate_schema(data)
|
||||
if not result.valid:
|
||||
try:
|
||||
from .config import settings
|
||||
strict = bool(getattr(settings, "frontmatter_validation_strict", False))
|
||||
except Exception: # noqa: BLE001 - config read must never raise here
|
||||
strict = False
|
||||
logger.warning(
|
||||
"frontmatter schema incomplete in %s: missing %s%s",
|
||||
doc_label,
|
||||
", ".join(result.missing),
|
||||
" [strict]" if strict else "",
|
||||
)
|
||||
return result
|
||||
except Exception as e: # noqa: BLE001 - never-raise; inert on error
|
||||
logger.debug(f"maybe_warn_schema: unexpected error for {doc_label}: {e}")
|
||||
return SchemaValidation(valid=True, missing=[])
|
||||
|
||||
@@ -239,22 +239,26 @@ def _parse_tests_verdict(content: str) -> tuple[bool, str]:
|
||||
beats a positive token in another field).
|
||||
- Otherwise a positive token (PASS/PASSED/READY-TO-DEPLOY/...) in ANY field -> (True).
|
||||
- Anything else (fields set but unrecognized) -> (False, reason).
|
||||
|
||||
ORCH-52c: the YAML-frontmatter parse is now delegated to the unified
|
||||
``frontmatter.parse_frontmatter`` primitive (single source of parse logic); the
|
||||
token-logic, upper-casing, three-field set and negative-token priority are
|
||||
UNCHANGED (semantics 1:1, AC-3/AC-6). Reason-strings are reproduced from the
|
||||
structured parse states.
|
||||
"""
|
||||
import yaml
|
||||
from ..frontmatter import parse_frontmatter, maybe_warn_schema
|
||||
|
||||
if not content.startswith("---"):
|
||||
parse = parse_frontmatter(content)
|
||||
if not parse.has_block:
|
||||
return False, "No YAML frontmatter in test report (cannot read machine verdict)"
|
||||
|
||||
parts = content.split("---", 2)
|
||||
if len(parts) < 3:
|
||||
if parse.malformed:
|
||||
return False, "Malformed YAML frontmatter in test report"
|
||||
|
||||
try:
|
||||
fm = yaml.safe_load(parts[1]) or {}
|
||||
except yaml.YAMLError as e:
|
||||
return False, f"Invalid YAML frontmatter in test report: {e}"
|
||||
if not isinstance(fm, dict):
|
||||
return False, "Malformed YAML frontmatter in test report (not a mapping)"
|
||||
if parse.yaml_error is not None:
|
||||
return False, f"Invalid YAML frontmatter in test report: {parse.yaml_error}"
|
||||
fm = parse.data
|
||||
# Warning-only schema check (FR-2/D3): inert — never changes the verdict.
|
||||
if fm:
|
||||
maybe_warn_schema(content, "test report")
|
||||
|
||||
verdict = str(fm.get("verdict", "") or "").upper().strip()
|
||||
status = str(fm.get("status", "") or "").upper().strip()
|
||||
@@ -338,8 +342,12 @@ def check_reviewer_verdict(repo: str, work_item_id: str, branch: str | None = No
|
||||
cause false positives/negatives. Returns:
|
||||
(True, ...) -> verdict: APPROVED
|
||||
(False, ...) -> verdict: REQUEST_CHANGES, missing verdict, or no frontmatter
|
||||
|
||||
ORCH-52c: the YAML-frontmatter parse is delegated to the unified
|
||||
``frontmatter.parse_frontmatter`` primitive; the verdict semantics
|
||||
(APPROVED/REQUEST_CHANGES) are UNCHANGED (1:1, AC-3/AC-6).
|
||||
"""
|
||||
import yaml
|
||||
from ..frontmatter import parse_frontmatter, maybe_warn_schema
|
||||
repo_path = _repo_path(repo, branch)
|
||||
review_path = os.path.join(repo_path, f"docs/work-items/{work_item_id}/12-review.md")
|
||||
|
||||
@@ -350,15 +358,14 @@ def check_reviewer_verdict(repo: str, work_item_id: str, branch: str | None = No
|
||||
with open(review_path, "r") as f:
|
||||
content = f.read()
|
||||
|
||||
parse = parse_frontmatter(content)
|
||||
if parse.yaml_error is not None:
|
||||
return False, f"Invalid YAML frontmatter in review: {parse.yaml_error}"
|
||||
verdict = None
|
||||
if content.startswith("---"):
|
||||
parts = content.split("---", 2)
|
||||
if len(parts) >= 3:
|
||||
try:
|
||||
fm = yaml.safe_load(parts[1]) or {}
|
||||
except yaml.YAMLError as e:
|
||||
return False, f"Invalid YAML frontmatter in review: {e}"
|
||||
verdict = str(fm.get("verdict", "")).upper().strip()
|
||||
if parse.has_block and not parse.malformed:
|
||||
if parse.data:
|
||||
maybe_warn_schema(content, "review report")
|
||||
verdict = str(parse.data.get("verdict", "")).upper().strip()
|
||||
|
||||
if verdict == "APPROVED":
|
||||
return True, "Reviewer verdict: APPROVED"
|
||||
@@ -410,17 +417,19 @@ def _parse_deploy_status(content: str) -> tuple[bool, str]:
|
||||
deploy_status: SUCCESS -> (True, "Deploy status: SUCCESS")
|
||||
deploy_status: FAILED -> (False, "Deploy status: FAILED")
|
||||
missing field / no frontmatter / bad YAML -> (False, <reason>)
|
||||
|
||||
ORCH-52c: parse delegated to the unified ``frontmatter.parse_frontmatter``;
|
||||
the deploy_status semantics (БАГ-8) are UNCHANGED (1:1).
|
||||
"""
|
||||
import yaml
|
||||
from ..frontmatter import parse_frontmatter, maybe_warn_schema
|
||||
parse = parse_frontmatter(content)
|
||||
if parse.yaml_error is not None:
|
||||
return False, f"Invalid YAML frontmatter in deploy log: {parse.yaml_error}"
|
||||
status = None
|
||||
if content.startswith("---"):
|
||||
parts = content.split("---", 2)
|
||||
if len(parts) >= 3:
|
||||
try:
|
||||
fm = yaml.safe_load(parts[1]) or {}
|
||||
except yaml.YAMLError as e:
|
||||
return False, f"Invalid YAML frontmatter in deploy log: {e}"
|
||||
status = str(fm.get("deploy_status", "")).upper().strip()
|
||||
if parse.has_block and not parse.malformed:
|
||||
if parse.data:
|
||||
maybe_warn_schema(content, "deploy log")
|
||||
status = str(parse.data.get("deploy_status", "")).upper().strip()
|
||||
if status == "SUCCESS":
|
||||
return True, "Deploy status: SUCCESS"
|
||||
if status == "FAILED":
|
||||
@@ -525,17 +534,19 @@ def _parse_staging_status(content: str) -> tuple[bool, str]:
|
||||
staging_status: SUCCESS -> (True, "Staging status: SUCCESS")
|
||||
staging_status: FAILED -> (False, "Staging status: FAILED")
|
||||
missing field / no frontmatter / bad YAML -> (False, <reason>)
|
||||
|
||||
ORCH-52c: parse delegated to the unified ``frontmatter.parse_frontmatter``;
|
||||
the staging_status semantics (self-hosting) are UNCHANGED (1:1).
|
||||
"""
|
||||
import yaml
|
||||
from ..frontmatter import parse_frontmatter, maybe_warn_schema
|
||||
parse = parse_frontmatter(content)
|
||||
if parse.yaml_error is not None:
|
||||
return False, f"Invalid YAML frontmatter in staging log: {parse.yaml_error}"
|
||||
status = None
|
||||
if content.startswith("---"):
|
||||
parts = content.split("---", 2)
|
||||
if len(parts) >= 3:
|
||||
try:
|
||||
fm = yaml.safe_load(parts[1]) or {}
|
||||
except yaml.YAMLError as e:
|
||||
return False, f"Invalid YAML frontmatter in staging log: {e}"
|
||||
status = str(fm.get("staging_status", "")).upper().strip()
|
||||
if parse.has_block and not parse.malformed:
|
||||
if parse.data:
|
||||
maybe_warn_schema(content, "staging log")
|
||||
status = str(parse.data.get("staging_status", "")).upper().strip()
|
||||
if status == "SUCCESS":
|
||||
return True, "Staging status: SUCCESS"
|
||||
if status == "FAILED":
|
||||
|
||||
@@ -44,12 +44,15 @@ def _read(path: str) -> str | None:
|
||||
|
||||
|
||||
def _strip_frontmatter(content: str) -> str:
|
||||
"""Drop a leading ``--- … ---`` YAML frontmatter block, if present."""
|
||||
if content.startswith("---"):
|
||||
parts = content.split("---", 2)
|
||||
if len(parts) >= 3:
|
||||
return parts[2]
|
||||
return content
|
||||
"""Drop a leading ``--- … ---`` YAML frontmatter block, if present.
|
||||
|
||||
ORCH-52c: delegates to the unified ``frontmatter.strip_frontmatter`` helper
|
||||
(single source of frontmatter logic). Behaviour is identical (only a well-formed
|
||||
>= 3-segment leading block is stripped) and the never-raise -> input contract is
|
||||
preserved.
|
||||
"""
|
||||
from .frontmatter import strip_frontmatter
|
||||
return strip_frontmatter(content)
|
||||
|
||||
|
||||
def _truncate(text: str, limit: int) -> str:
|
||||
|
||||
@@ -559,19 +559,19 @@ def parse_security_status(content: str) -> tuple[bool, str]:
|
||||
* ``security_status: FAIL`` -> ``(False, "Security status: FAIL")``
|
||||
* missing field / no frontmatter / bad YAML -> ``(False, <reason>)`` (fail-closed
|
||||
on the verdict read, AC-9).
|
||||
"""
|
||||
import yaml
|
||||
|
||||
ORCH-52c: parse delegated to the unified ``frontmatter.parse_frontmatter``
|
||||
primitive (single source of YAML-frontmatter logic); the security_status
|
||||
semantics (FAIL authoritative) are UNCHANGED (1:1).
|
||||
"""
|
||||
from .frontmatter import parse_frontmatter
|
||||
|
||||
parse = parse_frontmatter(content)
|
||||
if parse.yaml_error is not None:
|
||||
return False, f"Invalid YAML frontmatter in security report: {parse.yaml_error}"
|
||||
status = None
|
||||
if content.startswith("---"):
|
||||
parts = content.split("---", 2)
|
||||
if len(parts) >= 3:
|
||||
try:
|
||||
fm = yaml.safe_load(parts[1]) or {}
|
||||
except yaml.YAMLError as e:
|
||||
return False, f"Invalid YAML frontmatter in security report: {e}"
|
||||
if isinstance(fm, dict):
|
||||
status = str(fm.get("security_status", "")).upper().strip()
|
||||
if parse.has_block and not parse.malformed:
|
||||
status = str(parse.data.get("security_status", "")).upper().strip()
|
||||
if status == "FAIL":
|
||||
return False, "Security status: FAIL"
|
||||
if status == "PASS":
|
||||
@@ -593,11 +593,9 @@ def extract_security_findings(report_path: str) -> str:
|
||||
return ""
|
||||
with open(report_path, "r", encoding="utf-8") as f:
|
||||
content = f.read()
|
||||
# Drop the frontmatter; keep the human body.
|
||||
if content.startswith("---"):
|
||||
parts = content.split("---", 2)
|
||||
if len(parts) >= 3:
|
||||
content = parts[2]
|
||||
# Drop the frontmatter; keep the human body (ORCH-52c: shared helper).
|
||||
from .frontmatter import strip_frontmatter
|
||||
content = strip_frontmatter(content)
|
||||
wanted = ("## Verdict", "## Secrets", "## Dependencies (blocking)")
|
||||
lines = content.splitlines()
|
||||
out = []
|
||||
|
||||
244
tests/test_frontmatter.py
Normal file
244
tests/test_frontmatter.py
Normal file
@@ -0,0 +1,244 @@
|
||||
"""ORCH-076 (ORCH-52c): unit tests for the unified frontmatter contract.
|
||||
|
||||
Covers TC-01..TC-07 of docs/work-items/ORCH-076/04-test-plan.yaml:
|
||||
* writer (render/round-trip), validator (full / partial schema, strict on/off),
|
||||
* reader contract preserved (read_frontmatter_value), never-raise on bad input.
|
||||
|
||||
The whole module honours a never-raise contract (NFR-2): no input shape may raise.
|
||||
"""
|
||||
|
||||
import os
|
||||
import tempfile
|
||||
|
||||
os.environ.setdefault("ORCH_PLANE_API_TOKEN", "test-token")
|
||||
os.environ.setdefault("ORCH_GITEA_TOKEN", "test-token")
|
||||
|
||||
from src import frontmatter as fm # noqa: E402
|
||||
from src.frontmatter import ( # noqa: E402
|
||||
REQUIRED_FIELDS,
|
||||
FrontmatterParse,
|
||||
SchemaValidation,
|
||||
maybe_warn_schema,
|
||||
parse_frontmatter,
|
||||
parse_frontmatter_dict,
|
||||
read_frontmatter,
|
||||
read_frontmatter_value,
|
||||
render_frontmatter,
|
||||
strip_frontmatter,
|
||||
validate_schema,
|
||||
write_frontmatter,
|
||||
)
|
||||
|
||||
|
||||
def _full_schema():
|
||||
return {
|
||||
"work_item": "ORCH-076",
|
||||
"stage": "review",
|
||||
"author_agent": "reviewer",
|
||||
"status": "APPROVED",
|
||||
"created_at": "2026-06-09",
|
||||
"model_used": "claude-opus-4-8",
|
||||
}
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# TC-01 — writer serialises a mapping into canonical leading YAML-frontmatter
|
||||
# readable by the existing parsers (split("---", 2) + yaml.safe_load).
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_tc01_render_frontmatter_is_canonical_and_reparseable():
|
||||
out = render_frontmatter({"verdict": "APPROVED", "work_item": "ORCH-076"}, "body text")
|
||||
assert out.startswith("---\n")
|
||||
# Existing parser shape: split on '---' into 3 segments + yaml.safe_load.
|
||||
parts = out.split("---", 2)
|
||||
assert len(parts) == 3
|
||||
import yaml
|
||||
data = yaml.safe_load(parts[1])
|
||||
assert data["verdict"] == "APPROVED"
|
||||
assert data["work_item"] == "ORCH-076"
|
||||
# Body is preserved verbatim after the closing fence.
|
||||
assert parts[2].lstrip("\n") == "body text"
|
||||
# And our own primitive round-trips it.
|
||||
assert parse_frontmatter(out).data == {"verdict": "APPROVED", "work_item": "ORCH-076"}
|
||||
|
||||
|
||||
def test_tc01_render_empty_body_default():
|
||||
out = render_frontmatter({"a": 1})
|
||||
assert out == "---\na: 1\n---\n"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# TC-02 — round-trip: writer -> reader read_frontmatter_value yields same values.
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_tc02_write_then_read_roundtrip():
|
||||
data = _full_schema()
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
path = os.path.join(d, "12-review.md")
|
||||
assert write_frontmatter(path, data, "# Review body") is True
|
||||
for key, val in data.items():
|
||||
assert read_frontmatter_value(path, key) == val
|
||||
# Whole-mapping read matches too.
|
||||
assert read_frontmatter(path) == data
|
||||
|
||||
|
||||
def test_tc02_render_parse_dict_roundtrip():
|
||||
data = _full_schema()
|
||||
rendered = render_frontmatter(data, "body")
|
||||
assert parse_frontmatter_dict(rendered) == data
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# TC-03 — validator: full schema -> valid=True, no missing fields.
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_tc03_validate_full_schema_valid():
|
||||
res = validate_schema(_full_schema())
|
||||
assert isinstance(res, SchemaValidation)
|
||||
assert res.valid is True
|
||||
assert res.missing == []
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# TC-04 — validator: partial schema -> valid=False with the missing list,
|
||||
# WITHOUT raising (warning-only by default).
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_tc04_validate_partial_schema_lists_missing():
|
||||
res = validate_schema({"work_item": "ORCH-076", "stage": "review"})
|
||||
assert res.valid is False
|
||||
# The four absent required fields are reported (order = REQUIRED_FIELDS).
|
||||
assert set(res.missing) == set(REQUIRED_FIELDS) - {"work_item", "stage"}
|
||||
assert res.missing == [f for f in REQUIRED_FIELDS if f in res.missing]
|
||||
|
||||
|
||||
def test_tc04_blank_and_none_count_as_missing():
|
||||
data = _full_schema()
|
||||
data["status"] = "" # blank -> missing
|
||||
data["model_used"] = None # None -> missing
|
||||
res = validate_schema(data)
|
||||
assert res.valid is False
|
||||
assert set(res.missing) == {"status", "model_used"}
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# TC-05 — never-raise: writer + validator on broken input return a safe value.
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_tc05_validate_non_mapping_never_raises():
|
||||
for bad in (None, "not a mapping", 123, ["a", "b"]):
|
||||
res = validate_schema(bad) # type: ignore[arg-type]
|
||||
assert res.valid is False
|
||||
assert set(res.missing) == set(REQUIRED_FIELDS)
|
||||
|
||||
|
||||
def test_tc05_parse_broken_inputs_never_raise():
|
||||
# No frontmatter.
|
||||
p = parse_frontmatter("just prose, no fence")
|
||||
assert p == FrontmatterParse()
|
||||
assert p.data == {} and p.has_block is False
|
||||
# Unterminated block.
|
||||
p = parse_frontmatter("---\nkey: val\nno closing fence")
|
||||
assert p.has_block is True and p.malformed is True and p.data == {}
|
||||
# Bad YAML.
|
||||
p = parse_frontmatter("---\nkey: : :\n bad\n---\n")
|
||||
assert p.has_block is True and p.yaml_error is not None and p.data == {}
|
||||
# Non-mapping scalar frontmatter.
|
||||
p = parse_frontmatter("---\njust a string\n---\nbody")
|
||||
assert p.has_block is True and p.data == {}
|
||||
# Non-string input.
|
||||
assert parse_frontmatter(None).data == {} # type: ignore[arg-type]
|
||||
assert parse_frontmatter_dict(12345) == {} # type: ignore[arg-type]
|
||||
|
||||
|
||||
def test_tc05_write_to_unwritable_path_returns_false():
|
||||
# A path under a non-existent directory cannot be opened -> False, no raise.
|
||||
ok = write_frontmatter("/nonexistent-dir-xyz/cannot/12-review.md", {"a": 1})
|
||||
assert ok is False
|
||||
|
||||
|
||||
def test_tc05_render_unserialisable_degrades_to_body():
|
||||
class Bad:
|
||||
pass
|
||||
|
||||
out = render_frontmatter({"x": Bad()}, "fallback-body")
|
||||
# yaml cannot serialise an arbitrary object -> degrade to the body, never raise.
|
||||
assert out == "fallback-body"
|
||||
|
||||
|
||||
def test_tc05_read_missing_file_returns_empty():
|
||||
assert read_frontmatter("/no/such/file.md") == {}
|
||||
assert read_frontmatter_value("/no/such/file.md", "verdict") is None
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# TC-06 — reader read_frontmatter_value keeps its previous contract.
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_tc06_reader_contract_preserved():
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
path = os.path.join(d, "doc.md")
|
||||
with open(path, "w", encoding="utf-8") as f:
|
||||
f.write("---\nverdict: Approved\nempty:\n---\nbody\n")
|
||||
# strip + case preserved.
|
||||
assert read_frontmatter_value(path, "verdict") == "Approved"
|
||||
# empty value -> None.
|
||||
assert read_frontmatter_value(path, "empty") is None
|
||||
# absent key -> None.
|
||||
assert read_frontmatter_value(path, "missing") is None
|
||||
|
||||
# No frontmatter -> None.
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
path = os.path.join(d, "doc.md")
|
||||
with open(path, "w", encoding="utf-8") as f:
|
||||
f.write("no frontmatter here\n")
|
||||
assert read_frontmatter_value(path, "verdict") is None
|
||||
|
||||
|
||||
def test_tc06_reader_strips_whitespace():
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
path = os.path.join(d, "doc.md")
|
||||
with open(path, "w", encoding="utf-8") as f:
|
||||
f.write('---\nverdict: " PASS "\n---\n')
|
||||
assert read_frontmatter_value(path, "verdict") == "PASS"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# TC-07 — kill-switch: strict False (default) is inert; strict True signals
|
||||
# invalidity. maybe_warn_schema never changes a verdict either way.
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_tc07_maybe_warn_schema_default_warning_only(monkeypatch, caplog):
|
||||
monkeypatch.setattr(fm, "logger", fm.logger)
|
||||
from src.config import settings
|
||||
monkeypatch.setattr(settings, "frontmatter_validation_strict", False)
|
||||
incomplete = render_frontmatter({"verdict": "APPROVED"}, "body")
|
||||
res = maybe_warn_schema(incomplete, "review report")
|
||||
# Validation still reports invalidity (the data IS incomplete)...
|
||||
assert res.valid is False
|
||||
assert "model_used" in res.missing
|
||||
# ...but the helper is inert: it returns a value, it does not raise / block.
|
||||
|
||||
|
||||
def test_tc07_strict_flag_visible_to_helper(monkeypatch):
|
||||
from src.config import settings
|
||||
# Full schema -> valid regardless of the flag.
|
||||
monkeypatch.setattr(settings, "frontmatter_validation_strict", True)
|
||||
res_full = maybe_warn_schema(render_frontmatter(_full_schema(), "b"), "doc")
|
||||
assert res_full.valid is True
|
||||
# Incomplete -> invalid; strict True does not raise, just signals.
|
||||
res_partial = maybe_warn_schema(render_frontmatter({"stage": "review"}, "b"), "doc")
|
||||
assert res_partial.valid is False
|
||||
|
||||
|
||||
def test_tc07_maybe_warn_schema_on_garbage_is_inert():
|
||||
# Never-raise: a non-string / no-frontmatter input returns a SchemaValidation
|
||||
# (reporting the missing fields) WITHOUT raising — the gate verdict is untouched.
|
||||
for bad in ("no frontmatter", None, 123):
|
||||
res = maybe_warn_schema(bad, "doc") # type: ignore[arg-type]
|
||||
assert isinstance(res, SchemaValidation)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# strip_frontmatter helper parity.
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_strip_frontmatter_parity():
|
||||
assert strip_frontmatter("---\na: 1\n---\nbody") == "\nbody"
|
||||
# No well-formed block -> unchanged.
|
||||
assert strip_frontmatter("no fence") == "no fence"
|
||||
assert strip_frontmatter("---\nunterminated") == "---\nunterminated"
|
||||
# Never-raise on non-string.
|
||||
assert strip_frontmatter(None) is None # type: ignore[arg-type]
|
||||
202
tests/test_qg_verdicts.py
Normal file
202
tests/test_qg_verdicts.py
Normal file
@@ -0,0 +1,202 @@
|
||||
"""ORCH-076 (ORCH-52c): anti-regression for the five verdict gates after the parse
|
||||
is delegated to the unified frontmatter API.
|
||||
|
||||
Covers TC-08..TC-15 of docs/work-items/ORCH-076/04-test-plan.yaml. The MECHANISM of
|
||||
YAML-frontmatter parsing is now centralised in ``src/frontmatter.parse_frontmatter``,
|
||||
but the verdict SEMANTICS (value -> transition) must be 1:1 with before the task:
|
||||
* check_reviewer_verdict (12-review.md, verdict:) — TC-08
|
||||
* _parse_tests_verdict (13-test-report.md, result/verdict/status, ORCH-047) — TC-09
|
||||
* _parse_deploy_status (14-deploy-log.md, deploy_status:) — TC-10
|
||||
* _parse_staging_status (15-staging-log.md, staging_status:) — TC-11
|
||||
* (parse_security_status is exercised in tests/test_security_gate.py — TC-12)
|
||||
* backward-compat of old docs (no new schema) + additive schema — TC-13 / TC-14
|
||||
* worktree -> origin/main fallback preserved — TC-15
|
||||
"""
|
||||
|
||||
import os
|
||||
import tempfile
|
||||
|
||||
os.environ.setdefault("ORCH_PLANE_API_TOKEN", "test-token")
|
||||
os.environ.setdefault("ORCH_GITEA_TOKEN", "test-token")
|
||||
|
||||
from src.qg import checks as qg # noqa: E402
|
||||
from src.qg.checks import ( # noqa: E402
|
||||
_parse_deploy_status,
|
||||
_parse_staging_status,
|
||||
_parse_tests_verdict,
|
||||
check_deploy_status,
|
||||
check_reviewer_verdict,
|
||||
check_staging_status,
|
||||
)
|
||||
|
||||
|
||||
def _write(dirpath, work_item_id, name, content):
|
||||
d = os.path.join(dirpath, "docs", "work-items", work_item_id)
|
||||
os.makedirs(d, exist_ok=True)
|
||||
with open(os.path.join(d, name), "w", encoding="utf-8") as f:
|
||||
f.write(content)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# TC-08 — check_reviewer_verdict via the unified API.
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_tc08_reviewer_verdict_semantics(monkeypatch):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
monkeypatch.setattr(qg, "_repo_path", lambda repo, branch=None: d)
|
||||
_write(d, "ORCH-1", "12-review.md", "---\nverdict: APPROVED\n---\nbody")
|
||||
ok, reason = check_reviewer_verdict("orchestrator", "ORCH-1")
|
||||
assert ok is True and "APPROVED" in reason
|
||||
|
||||
_write(d, "ORCH-1", "12-review.md", "---\nverdict: REQUEST_CHANGES\n---\nbody")
|
||||
ok, reason = check_reviewer_verdict("orchestrator", "ORCH-1")
|
||||
assert ok is False and "REQUEST_CHANGES" in reason
|
||||
|
||||
# Missing verdict key -> (False).
|
||||
_write(d, "ORCH-1", "12-review.md", "---\nother: x\n---\nbody")
|
||||
ok, _ = check_reviewer_verdict("orchestrator", "ORCH-1")
|
||||
assert ok is False
|
||||
|
||||
# No frontmatter at all -> (False).
|
||||
_write(d, "ORCH-1", "12-review.md", "# review\nAPPROVED in prose\n")
|
||||
ok, _ = check_reviewer_verdict("orchestrator", "ORCH-1")
|
||||
assert ok is False
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# TC-09 — _parse_tests_verdict: ORCH-047 three equal-rank fields + negative-token
|
||||
# priority preserved.
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_tc09_tests_three_fields_each_pass():
|
||||
for field in ("result", "verdict", "status"):
|
||||
ok, reason = _parse_tests_verdict(f"---\n{field}: PASS\n---\nbody")
|
||||
assert ok is True, field
|
||||
assert "PASS" in reason
|
||||
|
||||
|
||||
def test_tc09_negative_token_is_authoritative():
|
||||
# BLOCKED in one field beats a positive token in another (ET-013 case).
|
||||
ok, reason = _parse_tests_verdict("---\nverdict: BLOCKED\nstatus: PASS\n---\n")
|
||||
assert ok is False
|
||||
assert "BLOCKED" in reason
|
||||
# FAIL likewise.
|
||||
ok, _ = _parse_tests_verdict("---\nresult: FAIL\n---\n")
|
||||
assert ok is False
|
||||
|
||||
|
||||
def test_tc09_tests_no_frontmatter_and_malformed():
|
||||
ok, reason = _parse_tests_verdict("no frontmatter, 23 passed\n")
|
||||
assert ok is False and "No YAML frontmatter" in reason
|
||||
ok, reason = _parse_tests_verdict("---\nresult: PASS\nunterminated")
|
||||
assert ok is False and "Malformed" in reason
|
||||
ok, reason = _parse_tests_verdict("---\nresult: PASS\nempty:\n---\n")
|
||||
assert ok is True # result PASS still wins; empty other field is fine
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# TC-10 — _parse_deploy_status semantics (БАГ-8) unchanged.
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_tc10_deploy_status_semantics():
|
||||
assert _parse_deploy_status("---\ndeploy_status: SUCCESS\n---\n")[0] is True
|
||||
assert _parse_deploy_status("---\ndeploy_status: FAILED\n---\n")[0] is False
|
||||
assert _parse_deploy_status("---\nother: SUCCESS\n---\n")[0] is False
|
||||
assert _parse_deploy_status("prose only SUCCESS")[0] is False
|
||||
# Bad YAML -> (False) with the preserved reason prefix.
|
||||
ok, reason = _parse_deploy_status("---\nx: : :\n---\n")
|
||||
assert ok is False and "Invalid YAML frontmatter in deploy log" in reason
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# TC-11 — _parse_staging_status + ORCH-35 conditionality (non-self -> N/A pass).
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_tc11_staging_status_semantics():
|
||||
assert _parse_staging_status("---\nstaging_status: SUCCESS\n---\n")[0] is True
|
||||
assert _parse_staging_status("---\nstaging_status: FAILED\n---\n")[0] is False
|
||||
assert _parse_staging_status("---\nother: SUCCESS\n---\n")[0] is False
|
||||
|
||||
|
||||
def test_tc11_staging_gate_na_for_non_self():
|
||||
ok, reason = check_staging_status("enduro-trails", "ET-1", "feature/x")
|
||||
assert ok is True
|
||||
assert "N/A" in reason
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# TC-13 — old verdict docs WITHOUT the new schema read exactly as before, for
|
||||
# every parser.
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_tc13_old_docs_without_schema_still_read():
|
||||
# None of these carry work_item/stage/author_agent/status/created_at/model_used.
|
||||
assert _parse_tests_verdict("---\nresult: PASS\n---\n")[0] is True
|
||||
assert _parse_tests_verdict("---\nverdict: FAIL\n---\n")[0] is False
|
||||
assert _parse_deploy_status("---\ndeploy_status: SUCCESS\n---\n")[0] is True
|
||||
assert _parse_staging_status("---\nstaging_status: SUCCESS\n---\n")[0] is True
|
||||
|
||||
|
||||
def test_tc13_reviewer_old_doc(monkeypatch):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
monkeypatch.setattr(qg, "_repo_path", lambda repo, branch=None: d)
|
||||
_write(d, "ORCH-1", "12-review.md", "---\nverdict: APPROVED\n---\nlegacy body")
|
||||
assert check_reviewer_verdict("orchestrator", "ORCH-1")[0] is True
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# TC-14 — a doc WITH the full additive schema + verdict key yields the SAME
|
||||
# verdict as without the schema (schema is additive, never changes it).
|
||||
# --------------------------------------------------------------------------- #
|
||||
_SCHEMA = (
|
||||
"work_item: ORCH-076\nstage: testing\nauthor_agent: tester\n"
|
||||
"status: PASS\ncreated_at: 2026-06-09\nmodel_used: claude-opus-4-8\n"
|
||||
)
|
||||
|
||||
|
||||
def test_tc14_full_schema_does_not_change_verdict():
|
||||
bare = _parse_tests_verdict("---\nresult: PASS\n---\n")
|
||||
full = _parse_tests_verdict(f"---\n{_SCHEMA}result: PASS\n---\n")
|
||||
assert bare[0] == full[0] is True
|
||||
|
||||
bare_d = _parse_deploy_status("---\ndeploy_status: FAILED\n---\n")
|
||||
full_d = _parse_deploy_status(
|
||||
"---\nwork_item: ORCH-076\nstage: deploy\nauthor_agent: deployer\n"
|
||||
"status: done\ncreated_at: 2026-06-09\nmodel_used: claude-opus-4-8\n"
|
||||
"deploy_status: FAILED\n---\n"
|
||||
)
|
||||
assert bare_d[0] == full_d[0] is False
|
||||
|
||||
|
||||
def test_tc14_reviewer_with_schema(monkeypatch):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
monkeypatch.setattr(qg, "_repo_path", lambda repo, branch=None: d)
|
||||
_write(
|
||||
d, "ORCH-1", "12-review.md",
|
||||
"---\nwork_item: ORCH-1\nstage: review\nauthor_agent: reviewer\n"
|
||||
"status: APPROVED\ncreated_at: 2026-06-09\nmodel_used: claude-opus-4-8\n"
|
||||
"verdict: APPROVED\n---\nbody",
|
||||
)
|
||||
assert check_reviewer_verdict("orchestrator", "ORCH-1")[0] is True
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# TC-15 — fallback worktree -> origin/main preserved (the gate still reads the
|
||||
# log recovered from main through the unified parser).
|
||||
# --------------------------------------------------------------------------- #
|
||||
def test_tc15_deploy_status_origin_main_fallback(monkeypatch):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
# No 14-deploy-log.md in the worktree -> the gate must consult origin/main.
|
||||
monkeypatch.setattr(qg, "_repo_path", lambda repo, branch=None: d)
|
||||
monkeypatch.setattr(
|
||||
qg, "_deploy_log_from_main",
|
||||
lambda repo, wi: "---\ndeploy_status: SUCCESS\n---\nfrom main",
|
||||
)
|
||||
ok, reason = check_deploy_status("orchestrator", "ORCH-1", "feature/x")
|
||||
assert ok is True and "SUCCESS" in reason
|
||||
|
||||
|
||||
def test_tc15_staging_status_origin_main_fallback(monkeypatch):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
monkeypatch.setattr(qg, "_repo_path", lambda repo, branch=None: d)
|
||||
monkeypatch.setattr(
|
||||
qg, "_staging_log_from_main",
|
||||
lambda repo, wi: "---\nstaging_status: FAILED\n---\nfrom main",
|
||||
)
|
||||
ok, reason = check_staging_status("orchestrator", "ORCH-1", "feature/x")
|
||||
assert ok is False and "FAILED" in reason
|
||||
@@ -202,6 +202,32 @@ def test_tc09_missing_or_broken_frontmatter_failclosed():
|
||||
assert ok is False
|
||||
|
||||
|
||||
def test_orch076_parse_security_status_via_unified_api():
|
||||
"""ORCH-076 TC-12: parse_security_status now reads through the unified
|
||||
frontmatter primitive; the PASS/FAIL semantics are 1:1 with before, and an
|
||||
old report WITHOUT the new schema fields still reads exactly the same."""
|
||||
from src import frontmatter as fm
|
||||
|
||||
# Delegates to the single parse primitive (no private duplicated parse).
|
||||
assert "parse_frontmatter" in sg.parse_security_status.__doc__
|
||||
|
||||
# PASS / FAIL semantics preserved.
|
||||
assert sg.parse_security_status("---\nsecurity_status: PASS\n---\n")[0] is True
|
||||
assert sg.parse_security_status("---\nsecurity_status: FAIL\n---\n")[0] is False
|
||||
|
||||
# An additive full schema does not change the verdict (FR-5 / AC-4).
|
||||
schema = (
|
||||
"work_item: ORCH-076\nstage: deploy-staging\nauthor_agent: deployer\n"
|
||||
"status: PASS\ncreated_at: 2026-06-09\nmodel_used: claude-opus-4-8\n"
|
||||
)
|
||||
with_schema = fm.render_frontmatter(
|
||||
{**{k.split(":")[0]: v for k, v in
|
||||
(line.split(": ", 1) for line in schema.strip().splitlines())},
|
||||
"security_status": "PASS"}
|
||||
)
|
||||
assert sg.parse_security_status(with_schema)[0] is True
|
||||
|
||||
|
||||
def test_tc10_artifact_has_valid_frontmatter_and_body(tmp_path, monkeypatch):
|
||||
"""TC-10: 17-security-report.md is written with valid frontmatter (all machine
|
||||
fields) and a body listing the findings; read-back == the written verdict."""
|
||||
|
||||
55
tests/test_stages_invariants.py
Normal file
55
tests/test_stages_invariants.py
Normal file
@@ -0,0 +1,55 @@
|
||||
"""ORCH-076 (ORCH-52c) TC-16 / AC-6: the pipeline invariants are untouched.
|
||||
|
||||
ORCH-52c only unifies the MECHANISM of YAML-frontmatter parsing behind a single
|
||||
API; it must NOT change the composition of the stage machine or the QG registry.
|
||||
This guard fails first if a future edit drifts either contract.
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
os.environ.setdefault("ORCH_PLANE_API_TOKEN", "test-token")
|
||||
os.environ.setdefault("ORCH_GITEA_TOKEN", "test-token")
|
||||
|
||||
from src.qg.checks import QG_CHECKS # noqa: E402
|
||||
from src.stages import STAGE_TRANSITIONS # noqa: E402
|
||||
|
||||
_EXPECTED_QGS = {
|
||||
"check_analysis_approved",
|
||||
"check_analysis_complete",
|
||||
"check_architecture_done",
|
||||
"check_ci_green",
|
||||
"check_review_approved",
|
||||
"check_tests_passed",
|
||||
"check_reviewer_verdict",
|
||||
"check_tests_local",
|
||||
"check_deploy_status",
|
||||
"check_staging_status",
|
||||
"check_branch_mergeable",
|
||||
"check_staging_image_fresh",
|
||||
"check_security_gate",
|
||||
}
|
||||
|
||||
_EXPECTED_TRANSITIONS = {
|
||||
"created": {"next": "analysis", "agent": "analyst", "qg": None},
|
||||
"analysis": {"next": "architecture", "agent": "architect", "qg": "check_analysis_approved"},
|
||||
"architecture": {"next": "development", "agent": "developer", "qg": "check_architecture_done"},
|
||||
"development": {"next": "review", "agent": "reviewer", "qg": "check_ci_green"},
|
||||
"review": {"next": "testing", "agent": "tester", "qg": "check_reviewer_verdict"},
|
||||
"testing": {"next": "deploy-staging", "agent": "deployer", "qg": "check_tests_passed"},
|
||||
"deploy-staging": {"next": "deploy", "agent": "deployer", "qg": "check_staging_status"},
|
||||
"deploy": {"next": "done", "agent": None, "qg": "check_deploy_status"},
|
||||
"done": {"next": None, "agent": None, "qg": None},
|
||||
}
|
||||
|
||||
|
||||
def test_tc16_qg_registry_unchanged():
|
||||
assert set(QG_CHECKS.keys()) == _EXPECTED_QGS
|
||||
|
||||
|
||||
def test_tc16_qg_callables():
|
||||
for name, fn in QG_CHECKS.items():
|
||||
assert callable(fn), f"QG {name} is not callable"
|
||||
|
||||
|
||||
def test_tc16_stage_transitions_unchanged():
|
||||
assert STAGE_TRANSITIONS == _EXPECTED_TRANSITIONS
|
||||
Reference in New Issue
Block a user