admin/orchestrator
1
0
Fork 0
Code Issues Pull Requests 4 Actions 2 Packages Projects Releases Wiki Activity

feat(ORCH-026): task dependencies (B waits for A) + single-repo merge serialization

Browse Source 
Level A — merge/deploy serialization within one repo: reuse the existing
ORCH-043/065 merge-lease (no new mechanism); the only new logic is an
unconditional pre-merge rebase in check_branch_mergeable — under the held
lease, auto_rebase_onto_main is ALWAYS called when premerge_rebase_always
(default True), not just when the branch is behind. No-op on an up-to-date
branch (rebase keeps HEAD, force-with-lease -> "Everything up-to-date", CI
not triggered). Kill-switch off -> ORCH-043 behaviour 1:1.

Level B — declarative task dependencies: additive job_deps table
(CREATE ... IF NOT EXISTS, no live-DB migration); claim_next_job gate
(NOT EXISTS) defers a job whose depends-on tasks are not yet 'done' without
occupying a max_concurrency slot; inert on empty job_deps -> zero regression.
New leaf src/task_deps.py (never-raise): is_task_ready (fail-open), DFS cycle
detection + Blocked/alert, declare/ingest_plane_relations (db source never
hits the network on the hot path), snapshot. Telegram waiting-line, /queue
observability, reconciler skip + cycle backstop, reaper untouched.

Invariants unchanged: STAGE_TRANSITIONS, QG_CHECKS registry (dep gate is a
claim_next_job врезка, not a registered QG), DB schema of existing tables,
HTTP endpoints; non-self repos remain a no-op on empty deps/scope.

Flags: ORCH_PREMERGE_REBASE_ALWAYS, ORCH_TASK_DEPS_ENABLED, ORCH_TASK_DEPS_SOURCE.
Docs: docs/architecture/README.md, CLAUDE.md, .env.example, CHANGELOG.md,
adr-0015. Tests: tests/test_orch026_*.py (64 tests); full suite 991 green.

Refs: ORCH-026

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
claude-bot
2026-06-08 19:06:22 +03:00
committed by stream
parent 9019e12d98
commit a74379f657
24 changed files with 1686 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

130
src/db.py
View File

@@ -123,6 +123,24 @@ def init_db():
# tracker can show "твоё время" without recomputing from activity history.
_ensure_column(conn, "tasks", "brd_review_started_at", "TEXT")
_ensure_column(conn, "tasks", "brd_review_ended_at", "TEXT")
# ORCH-026 (Level B): declarative task dependencies. job_deps stores the
# directed edge "task_id (B) is blocked-by depends_on_task_id (A)". The
# scheduler gate in claim_next_job keeps B queued until every A reaches
# tasks.stage='done'. Purely ADDITIVE (CREATE TABLE/INDEX IF NOT EXISTS, no
# change to jobs/tasks/agent_runs/events columns) -> idempotent and safe on
# the live shared prod DB (enduro-trails data untouched). The logical FK on
# tasks.id is intentional (no REFERENCES, mirrors jobs.task_id) so the
# migration cannot fail on a pre-existing DB. See 08-data-requirements.md.
conn.executescript("""
CREATE TABLE IF NOT EXISTS job_deps (
task_id INTEGER NOT NULL,
depends_on_task_id INTEGER NOT NULL,
created_at TEXT DEFAULT (datetime('now')),
PRIMARY KEY (task_id, depends_on_task_id)
);
CREATE INDEX IF NOT EXISTS idx_job_deps_task ON job_deps(task_id);
CREATE INDEX IF NOT EXISTS idx_job_deps_depends ON job_deps(depends_on_task_id);
""")
conn.commit()
conn.close()
@@ -466,12 +484,28 @@ def claim_next_job() -> dict | None:
so the SELECT+UPDATE pair is consistent. Returns the claimed job dict or None
when the queue is empty.
"""
# ORCH-026 (Level B, B-2): scheduler dependency gate. When task_deps_enabled
# is on, a job whose task has an UNFINISHED declared dependency
# (job_deps.depends_on_task_id -> a task with stage != 'done') is NOT
# claimable -> it stays 'queued' without occupying a max_concurrency slot.
# Jobs with a NULL task_id (no task) or with no job_deps rows are unaffected
# (NOT EXISTS is True). Kill-switch off -> the clause is omitted -> 1:1 the
# ORCH-1 query. The gate reads only the DB (offline-safe hot path).
dep_gate = ""
if getattr(settings, "task_deps_enabled", False):
dep_gate = (
"AND NOT EXISTS ("
" SELECT 1 FROM job_deps d JOIN tasks t ON t.id = d.depends_on_task_id "
" WHERE d.task_id = jobs.task_id AND t.stage != 'done'"
") "
)
conn = get_db()
try:
while True:
row = conn.execute(
"SELECT id FROM jobs WHERE status='queued' "
"AND (available_at IS NULL OR available_at <= datetime('now')) "
f"{dep_gate}"
"ORDER BY id LIMIT 1"
).fetchone()
if not row:
@@ -705,6 +739,102 @@ def recent_jobs(limit: int = 10) -> list[dict]:
return [dict(r) for r in rows]
# ---------------------------------------------------------------------------
# ORCH-026 (Level B): declarative task-dependency helpers
# ---------------------------------------------------------------------------
def add_dependency(task_id: int, depends_on_task_id: int) -> bool:
"""Declare that task ``task_id`` (B) is blocked-by ``depends_on_task_id`` (A).
Idempotent INSERT OR IGNORE against the job_deps PK (re-declaring the same
edge is a no-op). A self-edge (task depends on itself) is rejected — it would
deadlock the task forever and can never be satisfied. never-raise
(self-hosting safety, AC-G1): any DB error -> returns False, the caller must
not crash the webhook / worker. Returns True iff a NEW edge row was inserted.
"""
if task_id is None or depends_on_task_id is None:
return False
if task_id == depends_on_task_id:
return False
try:
conn = get_db()
try:
cur = conn.execute(
"INSERT OR IGNORE INTO job_deps (task_id, depends_on_task_id) "
"VALUES (?, ?)",
(task_id, depends_on_task_id),
)
conn.commit()
return cur.rowcount == 1
finally:
conn.close()
except Exception:
return False
def get_dependencies(task_id: int) -> list[int]:
"""Return the list of depends_on_task_id (A) that ``task_id`` (B) waits for.
never-raise: any DB error -> [] (conservative: caller treats the task as
having no declared dependency rather than crashing).
"""
try:
conn = get_db()
try:
rows = conn.execute(
"SELECT depends_on_task_id FROM job_deps WHERE task_id = ?",
(task_id,),
).fetchall()
finally:
conn.close()
return [r[0] for r in rows]
except Exception:
return []
def get_dependency_edges() -> list[tuple[int, int]]:
"""Return ALL declared edges as ``(task_id, depends_on_task_id)`` tuples.
Used by the cycle detector (DFS over the whole declared graph) and the
/queue snapshot. never-raise -> [] on any DB error.
"""
try:
conn = get_db()
try:
rows = conn.execute(
"SELECT task_id, depends_on_task_id FROM job_deps"
).fetchall()
finally:
conn.close()
return [(r[0], r[1]) for r in rows]
except Exception:
return []
def get_unfinished_dependencies(task_id: int) -> list[dict]:
"""Return the UNFINISHED dependencies of ``task_id`` (A's not yet 'done').
Each dict carries the predecessor's ``id``, ``work_item_id`` and ``stage``
so the readiness gate / Telegram waiting-line can name what B is waiting for.
never-raise -> [] on any DB error (treated as "ready", consistent with the
scheduler omitting the gate on failure).
"""
try:
conn = get_db()
try:
rows = conn.execute(
"SELECT t.id AS id, t.work_item_id AS work_item_id, t.stage AS stage "
"FROM job_deps d JOIN tasks t ON t.id = d.depends_on_task_id "
"WHERE d.task_id = ? AND t.stage != 'done'",
(task_id,),
).fetchall()
finally:
conn.close()
return [dict(r) for r in rows]
except Exception:
return []
# ---------------------------------------------------------------------------
# ORCH-1b (resilience): transient backoff helpers
# ---------------------------------------------------------------------------