test_queue.py::TestRetry::test_finalize_job_requeue_then_fail failed in the self-hosting environment because launcher._finalize_job classifies a non-zero exit by reading the tail of <settings.runs_dir>/<run_id>.log. settings.runs_dir defaults to the live prod dir /app/data/runs, which on the host holds REAL accumulated agent logs; a real 2.log containing "429" flips the expected 'permanent' classification to 'transient', requeueing the job instead of marking it 'failed'. This is ambient prod pollution, not a code fault. Add an autouse _isolate_runs_dir fixture (mirroring _no_telegram / _disable_merge_verify) that redirects settings.runs_dir to a per-test tmp dir so _run_log_path() resolves to a non-existent file and classify_log_file() returns the documented 'permanent' default. Full suite: 1617 passed. src/** untouched. Refs: ORCH-100 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
136 lines
7.2 KiB
Python
136 lines
7.2 KiB
Python
"""Global pytest fixtures.
|
|
|
|
test(conftest): mute Telegram in ALL tests to stop prod leakage.
|
|
|
|
Background: a pytest run on prod was sending REAL Telegram messages to Slava,
|
|
because some tests (e.g. test_webhook_dedup advancing a stage) reach
|
|
notify_stage_change -> send_telegram, which reads the live .env
|
|
telegram_bot_token/chat_id and actually POSTs to Telegram.
|
|
|
|
This autouse fixture stubs send_telegram to a no-op for every test:
|
|
|
|
- "src.notifications.send_telegram" is the SOURCE. All the notify_* helpers in
|
|
notifications.py call the module-global send_telegram, and every other module
|
|
that does a *local* `from .notifications import send_telegram` inside a
|
|
function resolves it live at call time -> covered by patching the source.
|
|
|
|
- "src.stage_engine.send_telegram" is patched too, because stage_engine binds
|
|
send_telegram as a MODULE-LEVEL name (from .notifications import send_telegram
|
|
at import), so a patch of the source alone would not intercept its 3 direct
|
|
calls. webhooks/plane and launcher import it locally inside functions, so the
|
|
source patch already covers them; they are patched defensively with
|
|
raising=False anyway in case that ever changes.
|
|
|
|
raising=False so a module that doesn't (yet) expose the name never breaks setup.
|
|
"""
|
|
|
|
import pytest
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _no_telegram(monkeypatch):
|
|
_noop = lambda *a, **k: None # noqa: E731
|
|
# Source of truth (covers notifications.notify_* and all local re-imports).
|
|
monkeypatch.setattr("src.notifications.send_telegram", _noop, raising=False)
|
|
# Module-level binding in stage_engine (and defensive coverage elsewhere).
|
|
monkeypatch.setattr("src.stage_engine.send_telegram", _noop, raising=False)
|
|
monkeypatch.setattr("src.webhooks.plane.send_telegram", _noop, raising=False)
|
|
monkeypatch.setattr("src.agents.launcher.send_telegram", _noop, raising=False)
|
|
monkeypatch.setattr("src.queue_worker.send_telegram", _noop, raising=False)
|
|
# ORCH-053: the reconciler binds send_telegram as a MODULE-LEVEL name
|
|
# (from .notifications import send_telegram), so the source patch alone would
|
|
# not intercept its unblock notification — patch it here too.
|
|
monkeypatch.setattr("src.reconciler.send_telegram", _noop, raising=False)
|
|
yield
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _reset_webhook_secrets(monkeypatch):
|
|
"""Isolate settings singleton between test files (CI cross-file isolation).
|
|
|
|
settings is a process-wide Pydantic singleton read once at import. Different
|
|
test modules set env variables differently at import-time, so those values leak
|
|
across files when pytest collects them together (as CI does).
|
|
|
|
1. webhook secrets: reset to "" so HMAC is disabled by default. Tests that
|
|
intentionally test the 401 path (test_webhook_dedup.py:268,278) re-apply
|
|
their own monkeypatch AFTER this autouse fixture runs, which overrides the
|
|
reset for the duration of that one test only.
|
|
|
|
2. db_path: reset to the value from ORCH_DB_PATH env var (last written by the
|
|
last imported test module). Without this, test_webhook_dedup.py (imported
|
|
first, alphabetically) seeds settings.db_path = dedup.db, while
|
|
test_webhooks.py's setup_db fixture tries to remove test_orchestrator.db,
|
|
leaving the DB dirty across tests that share a branch name and causing
|
|
get_task_by_repo_branch() to return a stale row with the wrong stage.
|
|
Per-test monkeypatches in test_webhook_dedup.setup_db override this reset.
|
|
"""
|
|
import os
|
|
from src.webhooks import gitea as gitea_mod
|
|
from src.webhooks import plane as plane_mod
|
|
from src import db as db_mod
|
|
monkeypatch.setattr(gitea_mod.settings, "gitea_webhook_secret", "", raising=False)
|
|
monkeypatch.setattr(plane_mod.settings, "plane_webhook_secret", "", raising=False)
|
|
db_path_env = os.environ.get("ORCH_DB_PATH", "")
|
|
if db_path_env:
|
|
monkeypatch.setattr(db_mod.settings, "db_path", db_path_env, raising=False)
|
|
yield
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _isolate_runs_dir(monkeypatch, tmp_path):
|
|
"""ORCH-100: point settings.runs_dir at a per-test tmp dir in ALL tests.
|
|
|
|
Background: ``launcher._run_log_path(run_id)`` resolves to
|
|
``<settings.runs_dir>/<run_id>.log`` and, on a non-zero exit,
|
|
``_finalize_job`` classifies the failure by reading the *tail of that log*
|
|
(transient 429/overload/timeout -> backoff-requeue; permanent -> attempts
|
|
requeue then 'failed'). settings.runs_dir defaults to the live prod dir
|
|
``/app/data/runs``, which on the self-hosting host holds REAL accumulated
|
|
agent logs (1.log, 2.log, ...). Tests that exercise the finalize path with a
|
|
small literal run_id (e.g. test_finalize_job_requeue_then_fail uses run_id=1/2)
|
|
therefore read whatever a real prod run happened to log — and a real 2.log that
|
|
contains "429" silently flips an expected 'permanent' classification to
|
|
'transient', requeueing instead of failing. That is ambient prod pollution, not
|
|
a code fault.
|
|
|
|
Redirecting runs_dir to an empty tmp dir makes _run_log_path() resolve to a
|
|
non-existent file -> classify_log_file() returns the documented 'permanent'
|
|
default, restoring deterministic, environment-independent behaviour for the
|
|
whole suite. settings is a process-wide singleton shared by launcher
|
|
(``launcher.settings is config.settings``), so patching the source covers it.
|
|
"""
|
|
from src import config as _cfg
|
|
monkeypatch.setattr(_cfg.settings, "runs_dir", str(tmp_path), raising=False)
|
|
yield
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _disable_merge_verify(monkeypatch):
|
|
"""ORCH-071: disable the merge-verify under-gate by default in ALL tests.
|
|
|
|
The under-gate (deploy -> done) runs a deterministic merge-actor + a
|
|
post-deploy merge verification that make REAL Gitea/git calls. Leaving it ON
|
|
by default would (a) reach the network from unrelated deploy->done tests and
|
|
(b) make them pass/fail by ACCIDENT depending on whether the live Gitea still
|
|
has the historical PR merged (a hidden CI flake). We therefore default it to
|
|
its documented kill-switch OFF state (``merge_verify_enabled=False`` == 1:1
|
|
pre-ORCH-071 behaviour). Tests that specifically target the under-gate
|
|
(test_merge_verify / test_deploy_finalizer_merge_gate / test_merge_actor /
|
|
test_deploy_restart_merge_recovery) re-enable it via their own monkeypatch
|
|
AFTER this autouse fixture, scoping the feature ON to just those tests.
|
|
"""
|
|
from src import config as _cfg
|
|
monkeypatch.setattr(_cfg.settings, "merge_verify_enabled", False, raising=False)
|
|
# ORCH-073: the regression guard (check_main_regression) runs real git in
|
|
# _handle_merge_verify's confirmed branch. Default it OFF too so unrelated
|
|
# deploy->done tests stay 1:1; the dedicated ORCH-073 tests re-enable it.
|
|
monkeypatch.setattr(_cfg.settings, "regression_guard_enabled", False, raising=False)
|
|
# ORCH-082: the merge-verify ensure_open_pr врезка makes REAL Gitea calls before
|
|
# merge_pr. Default it OFF so unrelated deploy->done / merge-verify tests stay 1:1
|
|
# (no network); the dedicated ORCH-082 tests re-enable it via their own monkeypatch.
|
|
monkeypatch.setattr(
|
|
_cfg.settings, "merge_verify_autocreate_pr_enabled", False, raising=False
|
|
)
|
|
yield
|