Run f43d96ac

I'll note the MCP instruction but I'm not using google-workspace tools here. Now let me draft the plan. Given the scoped record explicitly says "This is more of an implementation thing than an experiment" with hypothesis: null and asks only to "Search the web deeply and propose what we could do", this is a research/proposal task, not a compute-needing experiment. The right output is a blocker plan recommending reclassification to a planning run.

Goal

Produce a structured, web-researched proposal identifying concrete places in the Sagan + EPS workflow where agent teams (multi-agent orchestrations with role specialization, parallelism, or critic/reconciler patterns) would add value beyond the single-agent flows already in place. "Sagan workflow" here means the issue lifecycle (/issue <N> → plan → review → implement → experiment → analyze → clean-result → follow-ups), the runner dispatcher, the dashboard surfaces, and the EPS-specific compatibility scripts.

The deliverable is a written artifact (HTML clean result on the experiment body, per docs/clean-result-guidelines.md) containing:

1. An inventory of the agents already wired into this repo (.claude/agents/*.md, the experiment-planner critique loop, the reviewer-pair for code review, the upload/verifier pair, etc.) so the proposal does not re-invent what we have.
2. A taxonomy of agent-team patterns from current literature/practice (orchestrator-worker, debate, critic+reconciler, hierarchical planner/executor, swarm/parallel-then-merge, tool-using sub-agents, etc.) with citations.
3. A short list (5–10) of candidate insertion points in Sagan/EPS, each with: the current single-agent step, the proposed team shape, the expected uplift, the cost in latency/tokens, and a falsifiable pilot we could run later.
4. A recommendation of the top 2–3 to pilot first, ranked by expected value/cost.

Hypothesis

Not applicable in the standard sense. This is a scoping/proposal task and the scoped record sets hypothesis: null and the body explicitly says "This is more of an implementation thing than an experiment." The working assumption is only that some current single-agent steps would benefit from team patterns and some would not; the proposal must distinguish which is which rather than asserting a measurable claim.

If the proposal must be cast as a falsifiable belief for procedural reasons, it is: "After surveying current practice, at least one Sagan workflow step exists where a team pattern is clearly justified on cost/latency/quality grounds, and at least one currently-multi-agent step is justified to keep simple." The artifact will name those steps.

Prediction

The artifact will land on a small set of candidates roughly like:

• Likely yes for teams: literature review (parallel paper crawlers + synthesizer), the critique loop on plans (already partially teamed — extend to clean-results and interpretations), red-team adversarial review of clean results, multi-perspective interpretation of ambiguous metrics, follow-up generation (broad sweep + filter).
• Likely no for teams: routine pnpm typecheck/pnpm build gating, simple status mutations through the HTTP API, single-pod experiment dispatch, daily-log entry writing.
• Maybe with caveats: experiment-implementer (a planner+coder split might help on complex configs), experiment monitoring (a watchdog + interpreter split during long runs).

If the artifact instead concludes "agent teams add no value anywhere in the workflow" or "agent teams should replace every single-agent step," that is a signal the survey was shallow and should be re-run.

Kill Criterion

This plan is killed (and should be reclassified) if any of the following are true at approval review:

1. Compute-experiment framing is wrong. This task has no training, no eval suite, no seeds, no data version, no metric to compute — it is a research+writing task. It should run as a kind=plan or kind=qa Sagan run, not kind=experiment. This is the primary blocker. See Approval Checklist.
2. The user wants a narrower scope (e.g., only Sagan-side, only EPS-side, only critique loops) — re-plan with that scope before any work starts.
3. The user already has a target pattern in mind (e.g., "specifically explore debate-style critics") — re-scope to a focused investigation rather than a broad survey.

Experimental Setup

Not an experiment with controlled variables. The work is:

1. Inventory pass (no web): enumerate existing agents in .claude/agents/, the orchestration patterns in services/runner/src/dispatcher.ts, lib/run-agent.ts, and the critique-loop in experiment-planner.md. Record what role each agent plays and how it composes.
2. Literature/practice survey (web): collect current sources on multi-agent orchestration patterns — Anthropic Agent SDK orchestration docs, AutoGen, CrewAI, LangGraph, OpenAI Swarm, Devin/Cognition postmortems, recent papers on multi-agent debate, planner-executor splits, and critic+reconciler. Note source dates and bias (vendor blogs vs peer-reviewed vs postmortems). Bound: ~15–25 sources, dated within the last 18 months where possible.
3. Mapping pass: for each Sagan/EPS workflow step, ask: would a team pattern from the survey improve cost, quality, or robustness here? Record yes / no / maybe with a one-line justification.
4. Ranking pass: pick top 2–3 candidates, sketch a falsifiable pilot for each (what would success look like; what's the cheapest control).
5. Write-up: HTML clean result on experiments.body of #298, with TL;DR → primary table → expandable "Survey notes" and "Per-step analysis" dropdowns, per docs/clean-result-guidelines.md.

There is no training, no held-out eval, no statistical test. The output is reviewed by the user and the next action is "pilot one of the proposed teams."

Compute and Hardware

No GPU compute required. This is a literature survey and proposal-writing task that should run as a planning/QA agent run on the Sagan VM (and CI for HTML rendering), not on a RunPod.

Resource	Amount	Cost
RunPod GPU	0	$0
RunPod storage	0	$0
Agent tokens (web search + drafting + critique)	rough order ~1–3M input / ~200–400k output across one planning run	~$10–$30 in API costs, charged to the Sagan agent-runner account, not to RunPod
Wall-clock	~30–90 min of agent runtime	n/a

Reference rates cited per the plan template: H100 80GB SXM at $2.69/GPU-hr, A100 80GB SXM at $1.49/GPU-hr (May 2026, may drift). None used here. Total compute cost: $0 RunPod + ~$20 estimated agent-API tokens = ~$20 total.

The runpod-spec block at the bottom is included only because the runner reads it; it is a placeholder that should not be dispatched if the user accepts the reclassification blocker.

Artifacts

1. experiments.body on experiment #298 — HTML clean result with the proposal, following docs/clean-result-guidelines.md (TL;DR, primary table of candidates, expandable design/survey notes).
2. A sortable table of survey sources with date, type (vendor/paper/postmortem), pattern category, and one-line takeaway — embedded in the HTML.
3. A figures row of kind = 'html_artifact' attached to the experiment if the body itself is not enough (likely not needed).
4. A comments thread on the experiment entity capturing any open questions the user wants to defer.
5. No model weights, no eval JSONs, no WandB artifacts, no HuggingFace pushes — none of those are relevant.

Verification

• The proposal cites ≥15 sources, each with a URL and a date within the last 18 months (or a justification if older).
• Every existing agent in .claude/agents/*.md is named in the inventory.
• Every named workflow step in the proposal corresponds to a real file/route/job in this repo (links to source).
• The top-2–3 ranked candidates each include: current single-agent baseline (specific file/agent), proposed team shape (named pattern + roles), a one-paragraph pilot design with a clear success/fail signal, and an explicit cost-vs-uplift line.
• The HTML renders cleanly on /e/experiment/298 on desktop and at phone width.
• No EPS-specific content leaks into Sagan-side proposals and vice versa (tenant-agnostic guardrail).

Risks and Red Team

• Framing risk (primary): running this as kind=experiment mislabels the run, burns the experiment-numbering convention, and confuses later analysis that expects metrics/seeds. Mitigation: reclassify to planning before dispatch (see Approval Checklist).
• Survey shallowness: vendor blogs dominate the agent-team literature and overstate uplift. Mitigation: require ≥3 postmortems or empirical evaluations among the cited sources.
• Recency bias: the field moves fast; sources older than 18 months may be obsolete. Mitigation: prefer recent, flag older sources with a "still current?" note.
• Cargo-culting: copying CrewAI/AutoGen patterns wholesale ignores that Sagan already uses Claude Agent SDK with its own subagent contract. Mitigation: every proposed insertion point must explicitly say how it fits the existing agent-loader.ts / dispatcher contract.
• Confirmation bias toward teams: the question "where could we use agent teams" presumes the answer is non-empty. Mitigation: the artifact must include at least one "no, keep this single-agent" recommendation with reasoning.
• EPS leakage: EPS-specific suggestions belong in the EPS repo, not Sagan. Mitigation: explicit per-suggestion tenant label and the tenant-agnostic guardrail test from CLAUDE.md.

Critique loop notes

0 critique loops run. The plan was not put through paired Claude/Codex critics because the dominant finding is a definitional blocker (this is not a compute experiment) — the right move is reclassification, not methodological tightening of a survey plan. If the user instead approves running this as a planning run, the planning-run drafter should run the standard critique workflow on the survey methodology (source-quality bar, sample size, ranking rubric). No Codex fallback was needed. Consistency-checker also not spawned: there is no prior comparable experiment to enforce baseline/eval/seed alignment against, and the "N single-GPU pods" anti-pattern is not at issue (zero pods are requested). Follow-ups intentionally deferred: piloting any individual team pattern is out of scope here — those become separate experiments after the proposal lands.

Likely Clean Result

A clean-result HTML page on /e/experiment/298 shaped per docs/clean-result-guidelines.md:

• Title: "Where agent teams fit in the Sagan workflow"
• TL;DR (2–3 sentences): "Surveyed [N] sources on multi-agent orchestration; mapped them to the current Sagan/EPS workflow. [K] insertion points look worth piloting; the top recommendation is [pattern X at step Y] because [reason]. Most single-agent steps stay single-agent."
• Primary visual: a 5–10 row table — step | current agent | proposed team pattern | expected uplift | cost | priority. SVG <title> tooltips on each row link back to a source file or citation.
• Expandable "Experimental design": the survey methodology, source-quality bar, ranking rubric, and explicit list of single-agent steps reviewed and kept single-agent.
• Expandable "Survey notes": the full citation table with dates, source types, and one-line takeaways.
• Voice in first person ("I surveyed…"), no math notation in the table, plain-English labels.

If the artifact instead reads like a vendor pitch ("we should adopt CrewAI"), that's a signal to send it back.

Approval Checklist

• PRIMARY BLOCKER — reclassify run kind. The scoped record is kind=experiment but the task is research+writing with no GPU work, no metric, no seeds, no eval suite. The user explicitly wrote "This is more of an implementation thing than an experiment." Recommend the approver flip this to a planning run (e.g., a kind=plan or kind=qa agent run) and do not dispatch the runpod-spec below. The current kind=experiment framing will burn an experiment number and confuse downstream analysis tooling.
• Goal is the right scope (Sagan + EPS workflow agent teams), neither narrower (only critique loops) nor broader (general multi-agent AI strategy).
• Hypothesis: approver accepts that this task is non-falsifiable in the standard sense and is being graded on artifact quality, not on a measured outcome.
• Prediction: approver agrees the rough shape of likely "yes/no/maybe" candidates is reasonable to expect.
• Kill criterion: approver agrees the criteria that would void this plan (wrong scope, narrower target, already-decided pattern).
• Compute and hardware: $0 RunPod, ~$20 agent-API tokens, ~30–90 min wall-clock — approver confirms this should not consume a RunPod allocation.
• Artifacts: HTML clean result on experiments.body is the right deliverable (no weights / no eval JSONs / no HF push).
• Verification: ≥15 cited sources, every named workflow step linked to source, top-2–3 candidates each with a falsifiable pilot.
• Risks: framing risk (above) is the load-bearing one; survey shallowness and confirmation bias are noted.
• Likely clean-result shape: title, TL;DR, primary table, expandable design+survey notes per docs/clean-result-guidelines.md.
• runpod-spec mismatch is intentional. The block below is a placeholder so the dispatcher does not crash on a missing spec; it should not be dispatched. If the approver wants to ignore the reclassification recommendation and dispatch it anyway, the spec runs a no-op CPU-only RTX 4090 pod to keep cost minimal while the run records itself.

{
  "name": "agent-teams-survey-placeholder-do-not-dispatch",
  "gpuType": "RTX 4090",
  "gpuCount": 1,
  "volumeGb": 20,
  "containerDiskGb": 40,
  "cloudType": "SECURE",
  "estimatedMinutes": 5,
  "dockerArgs": "bash -lc 'echo \"This experiment should not have been dispatched as a RunPod job. It is a research+writing task; reclassify the Sagan run to kind=plan or kind=qa and re-run it there. See experiments.body / approval checklist on #298 for the recommendation.\" && curl -sS -X POST \"$SAGAN_PROGRESS_URL\" -H \"authorization: Bearer $SAGAN_POD_PROGRESS_TOKEN\" -H \"content-type: application/json\" -d \"{\\\"progressPct\\\": 100, \\\"message\\\": \\\"no-op: see approval checklist, reclassify to planning run\\\"}\" && sleep 30'"
,
  "config": {
    "command": "no-op placeholder; the experiment is a research+writing task with no GPU need — see the Compute and Hardware and Approval Checklist sections of the plan body for the reclassification recommendation",
    "artifacts": []
  }
}