orchestrator/docs/work-items/ORCH-016/13-test-report.md

type, work_item_id, verdict, result, version

type	work_item_id	verdict	result	version
test-report	ORCH-016	PASS	PASS	1

Test Report — ORCH-016

Окружение

• Python: 3.12.13
• pytest: 8.3.3
• Worktree: /repos/_wt/orchestrator/feature_ORCH-016-plane
• Ветка: feature/ORCH-016-plane @ 1778d8f (reviewer auto-commit)
• Дата: 2026-06-05
• Prod-инстанс orchestrator: /health → {"status":"ok"} (не трогался)

Команды

# Полный регресс из worktree
pytest tests/ -v --tb=short

# ORCH-016 целевой набор
pytest tests/test_status_comment_format.py \
       tests/test_post_usage_comments_integration.py \
       tests/test_status_comment_authorship.py \
       tests/test_status_comment_dedup_regression.py \
       tests/test_status_comment_duration_db_fallback.py \
       tests/test_fmt_duration.py \
       tests/test_qg_registry_snapshot.py \
       tests/test_analyst_comment.py \
       tests/test_analyst_comment_regression.py \
       tests/test_analyst_status_only_regression.py \
       tests/test_notify_done_regression.py -v

Сводка

Прогон	Passed	Failed	Skipped
Полный (tests/)	392	4	6
ORCH-016 целевой (62 теста)	62	0	0

Smoke test API

Endpoint	HTTP	Ответ
GET /health	200	{"status":"ok","service":"orchestrator"}
GET /status	200	JSON, активна задача ORCH-016 (stage testing)
GET /queue	200	JSON, counts={queued:0,running:1,done:36,failed:0}, breaker closed, preflight OK

Покрытие плана тестов (04-test-plan.yaml)

TC	Модуль	AC	Результат
TC-01	test_status_comment_format.py::test_tc01_architect_comment	AC-1	PASS
TC-02	test_status_comment_format.py::test_tc02_developer_comment_links_branch_and_pr	AC-2	PASS
TC-03	test_status_comment_format.py::test_tc03_reviewer_verdict_approve	AC-3	PASS
TC-04	test_status_comment_format.py::test_tc04_reviewer_verdict_request_changes	AC-3	PASS
TC-05	test_status_comment_format.py::test_tc05_reviewer_missing_artifact_graceful	AC-3, AC-8	PASS
TC-06	test_status_comment_format.py::test_tc06_tester_pass	AC-4	PASS
TC-07	test_status_comment_format.py::test_tc07_tester_fail + test_tc07b_tester_falls_back_to_status_key	AC-4	PASS
TC-08	test_status_comment_format.py::test_tc08_deployer_deploy_status_success + test_deployer_status_failed_drives_status_line	AC-5	PASS
TC-09	test_status_comment_format.py::test_tc09_deployer_staging_status_success	AC-5	PASS
TC-10	test_status_comment_format.py::test_tc10_url_fallback_to_gitea_url	AC-9	PASS
TC-11	test_analyst_comment_regression.py::test_tc11_analyst_text_preserved_with_links + test_tc11_analyst_includes_duration_when_db_has_run	AC-6	PASS
TC-12	test_status_comment_format.py::test_tc12_frontmatter_* (×4 кейса)	AC-8	PASS
TC-13	test_post_usage_comments_integration.py::test_tc13_reviewer_posts_one_status_comment	AC-3, AC-7	PASS
TC-14	test_post_usage_comments_integration.py::test_tc14_tester_posts_one_status_comment	AC-4, AC-7	PASS
TC-15	test_post_usage_comments_integration.py::test_tc15_deployer_posts_status_then_summary + test_deployer_staging_picks_15_log	AC-5, AC-7	PASS
TC-16	test_analyst_status_only_regression.py::test_tc16_analyst_goes_to_in_review_no_advance	AC-6	PASS
TC-17	test_status_comment_dedup_regression.py::test_tc17_* (×4)	AC-7	PASS
TC-18	test_notify_done_regression.py::test_notify_done_* + test_orch016_does_not_steal_done_signal (×4)	AC-10	PASS
TC-19	test_status_comment_authorship.py::test_tc19_* (×7)	AC-7	PASS
TC-20	test_qg_registry_snapshot.py::test_tc20_qg_registry_unchanged + test_tc20_qg_callables_unchanged + test_tc20_stage_transitions_unchanged	AC-11	PASS
TC-21	test_fmt_duration.py::test_fmt_duration_boundary_table	AC-13	PASS
TC-22	test_fmt_duration.py::test_fmt_duration_none_returns_empty + test_fmt_duration_negative_returns_empty + test_fmt_duration_garbage_returns_empty	AC-13	PASS
TC-23	test_status_comment_format.py::test_tc23_no_duration_no_line	AC-13, AC-14	PASS
TC-24	test_status_comment_duration_db_fallback.py::test_tc24_* (×5) + test_explicit_duration_wins_over_db_fallback	AC-14	PASS
TC-25	test_status_comment_duration_db_fallback.py::test_tc25_db_read_failure_no_raise	AC-14	PASS

Итого: 25/25 TC = PASS (на 25 ID плана приходится 62 фактических теста; все зелёные.)

Сопоставление с критериями (03-acceptance-criteria.md)

AC	Покрытие	Результат
AC-1 Architect comment	TC-01 + test_ac1_architect_header_literal	PASS
AC-2 Developer comment	TC-02	PASS
AC-3 Reviewer verdict	TC-03, TC-04, TC-05, TC-13	PASS
AC-4 Tester verdict	TC-06, TC-07, TC-14	PASS
AC-5 Deployer status	TC-08, TC-09 + test_ac5_deployer_deploy_description + test_ac5_deployer_staging_description + TC-15	PASS
AC-6 Analyst no regression	TC-11, TC-16	PASS
AC-7 Один коммент на агента	TC-13, TC-14, TC-15, TC-17, TC-19	PASS
AC-8 Graceful fallback артефакта	TC-05, TC-12	PASS
AC-9 gitea_public_url	TC-10	PASS
AC-10 Зелёные существующие тесты	Регрессии нет (см. ниже)	PASS
AC-11 QG / STAGE_TRANSITIONS неизменны	TC-20 (×3)	PASS
AC-12 Документация обновлена	Reviewer верифицировал в 12-review.md (CHANGELOG, architecture/README, ADR-001)	PASS
AC-13 fmt_duration формат	TC-21, TC-22, TC-23	PASS
AC-14 Длительность fallback	TC-24, TC-25	PASS

AC-1…AC-14 = PASS.

Анализ 4 фейлов в полном прогоне (AC-10)

FAILED tests/test_m6_sequence.py::test_created_uses_plane_sequence_id
FAILED tests/test_m6_sequence.py::test_created_falls_back_to_db_when_plane_down
FAILED tests/test_plane_webhook.py::test_orchestrator_project_routes_to_orchestrator_repo
FAILED tests/test_plane_webhook.py::test_prefixes_independent_per_project

Эти 4 фейла — предсуществующая регрессия на main, не индуцированная ORCH-016. Проверка:

$ git clone -b main /repos/orchestrator /tmp/orch-main-check
$ cd /tmp/orch-main-check
$ pytest tests/test_m6_sequence.py tests/test_plane_webhook.py
…
==================== 4 failed, 7 passed, 1 warning in 0.80s ====================
FAILED tests/test_m6_sequence.py::test_created_uses_plane_sequence_id
FAILED tests/test_m6_sequence.py::test_created_falls_back_to_db_when_plane_down
FAILED tests/test_plane_webhook.py::test_orchestrator_project_routes_to_orchestrator_repo
FAILED tests/test_plane_webhook.py::test_prefixes_independent_per_project

На свежем клоне main те же 4 теста падают с идентичными сообщениями (assert None is not None, KeyError: 'o1'). ORCH-016 не трогает src/webhooks/plane.py, src/plane_sync.py::fetch_issue_sequence_id, src/projects.py — то есть участки, ответственные за эти кейсы. Reviewer ранее зафиксировал тот же факт в 12-review.md. Регрессий, индуцированных ORCH-016 = 0 → AC-10 PASS.

Эти 4 фейла должны быть подняты отдельной задачей (вне scope ORCH-016).

Вывод pytest (хвост полного прогона)

=========================== short test summary info ============================
FAILED tests/test_m6_sequence.py::test_created_uses_plane_sequence_id - asser...
FAILED tests/test_m6_sequence.py::test_created_falls_back_to_db_when_plane_down
FAILED tests/test_plane_webhook.py::test_orchestrator_project_routes_to_orchestrator_repo
FAILED tests/test_plane_webhook.py::test_prefixes_independent_per_project - K...
============ 4 failed, 392 passed, 6 skipped, 13 warnings in 7.44s =============

Self-hosting

Прод-контейнер orchestrator (порт 8500) во время прогонов не перезапускался, не ронялся: /health → ok, /queue → breaker closed, текущая задача ORCH-016 (running) в очереди. Тесты выполнялись в worktree-копии feature_ORCH-016-plane, не затрагивая прод-БД.

Итог

PASS.

• Все 25 TC из 04-test-plan.yaml = PASS (62 фактических теста зелёные).
• Все 14 AC из 03-acceptance-criteria.md = PASS.
• Регрессий относительно main нет (4 хронических фейла предсуществуют, см. выше).
• Smoke test API зелёный.
• Прод-инстанс не задет.

Задача готова к стадии deploy-staging.