fix(qg): gate testing->deploy on machine-readable test verdict, not substring (ET-013) #24

admin · 2026-06-04T16:06:05+03:00

admin commented

2026-06-04 16:06:05 +03:00

Problem

check_tests_passed used naive substring (if PASS in content) over the whole 13-test-report.md body. On ET-013 the tester set verdict: BLOCKED / status: blocked (P1 AC-19 failed), but the body contained 23 passed / PASS / All checks passed -> gate returned True -> unfinished feature reached Done on prod.

Fix

Mirror check_reviewer_verdict (S-5) and check_deploy_status: read ONLY YAML frontmatter verdict:/status:. Positive tokens (PASS/PASSED/READY-TO-DEPLOY/GREEN/APPROVED) -> True; negative tokens (BLOCKED/FAILED/FAIL/REQUEST_CHANGES) authoritative -> False; missing/empty/no-frontmatter/bad-YAML -> False with reason; file missing -> not found. Never raises.

Positive token set derived from REAL enduro-trails reports ET-001..ET-014 (inconsistent verdict styles).

Validation

All 9 prior passing WIs (ET-001/005/006/007/008/009/011/012/014) -> still True
ET-013 (BLOCKED, body full of PASS) -> False
11 unit tests in TestCheckTestsPassed pass; full suite 285 passed, 9 pre-existing off-limits webhook HMAC/401 failures unchanged.

Do NOT self-merge — review by Stream.

## Problem check_tests_passed used naive substring (if PASS in content) over the whole 13-test-report.md body. On ET-013 the tester set verdict: BLOCKED / status: blocked (P1 AC-19 failed), but the body contained 23 passed / PASS / All checks passed -> gate returned True -> unfinished feature reached Done on prod. ## Fix Mirror check_reviewer_verdict (S-5) and check_deploy_status: read ONLY YAML frontmatter verdict:/status:. Positive tokens (PASS/PASSED/READY-TO-DEPLOY/GREEN/APPROVED) -> True; negative tokens (BLOCKED/FAILED/FAIL/REQUEST_CHANGES) authoritative -> False; missing/empty/no-frontmatter/bad-YAML -> False with reason; file missing -> not found. Never raises. Positive token set derived from REAL enduro-trails reports ET-001..ET-014 (inconsistent verdict styles). ## Validation - All 9 prior passing WIs (ET-001/005/006/007/008/009/011/012/014) -> still True - ET-013 (BLOCKED, body full of PASS) -> False - 11 unit tests in TestCheckTestsPassed pass; full suite 285 passed, 9 pre-existing off-limits webhook HMAC/401 failures unchanged. Do NOT self-merge — review by Stream.

admin added 1 commit 2026-06-04 16:06:05 +03:00

fix(qg): gate testing->deploy on machine-readable test verdict, not substring (ET-013) 757745a221

check_tests_passed did "if PASS in content" over the whole 13-test-report.md
body, so a report explicitly marked verdict: BLOCKED / status: blocked whose
prose mentioned "23 passed" / "PASS" / "All checks passed" passed the gate.
On ET-013 an unfinished feature (P1 AC-19 failed) reached Done.

Now mirrors check_reviewer_verdict (S-5) and check_deploy_status: read ONLY the
YAML frontmatter verdict:/status: fields. Positive tokens (PASS/PASSED/
READY-TO-DEPLOY/GREEN/APPROVED) -> True; negative tokens (BLOCKED/FAILED/...) are
authoritative -> False; missing/empty/no-frontmatter/bad-YAML -> False with reason;
file missing -> not found. Never raises.

Positive token set derived from REAL enduro-trails reports ET-001..ET-014
(inconsistent: PASS, ready-to-deploy+status:PASSED, stage:ready-to-deploy+status:pass,
PASS — ready-to-deploy). Validated: all 9 prior passing WIs stay True, ET-013 -> False.

admin merged commit 83f5020f94 into main

2026-06-04 16:08:10 +03:00

admin referenced this issue from a commit

2026-06-04 16:08:10 +03:00

Merge pull request 'fix(qg): gate testing->deploy on machine-readable test verdict, not substring (ET-013)' (#24) from fix/tests-machine-verdict into main

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: admin/orchestrator#24