Run 583edc02

kind=experiment

I'll diagnose the failure and produce a complete plan with a runpod-spec that mitigates the team-account vanishing issue.

tool=Bash

Bash

tool=Bash

Bash

__init__.py
__pycache__
_factor_screen
_progress.py
marker_factor_screen.py

b1a24b4b Address code-review round 1 targeted fixes for experiment #365
6f37dc79 Add eps.experiments.marker_factor_screen for Sagan experiment #365
125e78f8 Add TrainLoraConfig.backend flag scaffold for Unsloth adoption
a4652b62 Route EPS workflow state through Sagan
4129f88d Add Sagan pod progress reporting guidance
b1a24b4b04f92598e381fa3cd207a0fe5d24b9e7
Shell cwd was reset to /home/thomasjiralerspong/sagan

tool=Bash

Bash

12:* Phase 2 (pods 0/1/2 with ``--source-persona``): full 2^5 = 32 cells at
28:import argparse
62:def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
63:    p = argparse.ArgumentParser(
69:    p.add_argument("--pod-index", type=int, default=0)
70:    p.add_argument("--num-pods", type=int, default=4)
71:    p.add_argument(
79:    p.add_argument("--source-persona", type=str, default=None, choices=(*SOURCE_PERSONAS, None))
80:    p.add_argument("--base-model", type=str, default="Qwen/Qwen2.5-7B-Instruct")
81:    p.add_argument("--intent", type=str, default="lora-7b")
84:    p.add_argument("--lora-r", type=int, default=32)
85:    p.add_argument("--lora-alpha", type=int, default=64)
86:    p.add_argument("--lr", type=float, default=1e-5)
87:    p.add_argument("--epochs", type=int, default=3)
90:    p.add_argument("--pos-per-source", type=int, default=200)
91:    p.add_argument("--neg-per-source", type=int, default=400)
94:    p.add_argument("--eval-personas", type=int, default=24)
95:    p.add_argument("--eval-questions", type=int, default=20)
96:    p.add_argument("--eval-completions", type=int, default=5)
97:    p.add_argument("--eval-max-new-tokens", type=int, default=2048)
100:    p.add_argument("--primary-seed", type=int, default=42)
101:    p.add_argument("--multi-seeds", type=_csv_ints, default=[137, 256])
105:    p.add_argument("--bootstrap-scheme", type=str, default="clustered")
106:    p.add_argument("--bootstrap-cluster-sr", type=str, default="question")
107:    p.add_argument("--bootstrap-cluster-lr", type=str, default="persona")
110:    p.add_argument("--run-pre-screen", action="store_true")
111:    p.add_argument("--run-smoke", action="store_true")
112:    p.add_argument("--build-figures", action="store_true")
113:    p.add_argument("--write-clean-result", action="store_true")
114:    p.add_argument("--label-f1xf2-preregistered", action="store_true")
117:    p.add_argument("--progress-url", type=str, default=None)
118:    p.add_argument("--progress-token", type=str, default=None)
119:    p.add_argument("--agent-run-id", type=str, default=None)
120:    p.add_argument("--experiment-id", type=str, default=None)
121:    p.add_argument("--run-index", type=int, default=0)
124:    p.add_argument("--wandb-project", type=str, default=os.environ.get("WANDB_PROJECT"))
127:    p.add_argument(
141:def _pod_dir(args: argparse.Namespace) -> Path:
153:def _is_aggregator(args: argparse.Namespace) -> bool:
160:def _dispatch_source_slab(args: argparse.Namespace) -> dict:
Shell cwd was reset to /home/thomasjiralerspong/sagan

tool=Bash

Bash

{"experiment":{"id":"077ae4c7-e816-4dd8-a150-ad8fe19cb795","number":365,"legacyGhNumber":null,"beliefId":null,"projectId":null,"title":"Factor screen for marker implantation + leakage (2^5: system-prompt length, answer-format length, persona-presence, on-policy, marker-only-loss)","body":"## Motivation\n\nWe want a single experiment that ranks the dominant factors controlling **marker implantation** (source `[ZLT]` rate) and **marker leakage** (mean off-diagonal rate) under LoRA SFT on Qwen2.5-7B-Instruct. Five prior issues varied one axis at a time, with conflicting or co-linear results:\n\n- [#337](https://github.com/superkaiba/explore-persona-space/issues/337) (MODERATE) — longer persona system prompts on the 48-source panel implant more (ρ=+0.38) and leak less (ρ=−0.38), but length and persona-richness co-vary.\n- [#295](https://github.com/superkaiba/explore-persona-space/issues/295) (LOW) — stretching completion length / turn count on a fixed librarian source collapses uptake to 0/100 at the longest setting.\n- [#340](https://github.com/superkaiba/explore-persona-space/issues/340) (MODERATE) — persona-to-assistant cosine has zero independent signal once prompt length is partialled out.\n- [#181](https://github.com/superkaiba/explore-persona-space/issues/181) / [#208](https://github.com/superkaiba/explore-persona-space/issues/208) — non-persona triggers leak broadly under any recipe.\n- [#46](https://github.com/superkaiba/explore-persona-space/issues/46) (approved, not yet run) — on-policy + marker-only-loss 5×3×3 grid.\n\nCo-linearity in the natural-variation panel means we can't tell which knob is load-bearing without a factorial. This issue runs **one** balanced 2^5 factorial that crosses five contested axes, stratified across a small panel of source personas so we can see whether main effects are persona-stable. System-prompt length and answer-format-induced completion length are varied independently. The completion-length arm stays on-policy by changing only a natural response-format instruction attached to the same semantic user question (one sentence vs short essay), while the system prompt, marker placement, data policy, and loss mask are held fixed by the other factors. It absorbs the open proposed children #361, #339, #353.\n\n## Factors (2 levels each, 32 cells per source persona)\n\n| Factor | Level 0 (baseline) | Level 1 (treatment) | Open question |\n|---|---|---|---|\n| **A. System-prompt length** | short system prompt (~6–20 tokens) | long system prompt (~1000 tokens) | Tests [#337](https://github.com/superkaiba/explore-persona-space/issues/337)'s system-prompt-length effect without tying it to completion length. |\n| **B. Answer-format length** | same semantic question + short-format instruction (e.g. \"Answer in one sentence.\", ~50 tokens before `[ZLT]`) | same semantic question + long-format instruction (e.g. \"Answer as a short essay with several paragraphs.\", ~1050 tokens before `[ZLT]`) | Induces short vs long completions through a natural user-message format instruction so D0 remains genuinely on-policy; tests [#295](https://github.com/superkaiba/explore-persona-space/issues/295) independently of system-prompt length. |\n| **C. Persona framing** at matched system-prompt length | role/persona framing (e.g. `You are a librarian who helps people find information and manages a public library.`) | lexically matched non-persona context framing (e.g. `Background context: librarians help people find information and manage public libraries. Answer neutrally and directly.`) | Tests role adoption while controlling for persona-domain words. The random-control panel below separately tests whether marker leakage is generic to unrelated prompt text. |\n| **D. Data policy** | **on-policy completions** (vLLM-sampled from base Qwen-7B-Instruct under the same system prompt) | off-policy completions (Claude-generated, current recipe) | Makes on-policy the default baseline, per [#46](https://github.co

tool=Bash

Bash

tool=Bash

Bash

def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
    p = argparse.ArgumentParser(
        prog="eps.experiments.marker_factor_screen",
        description="2^5 marker-implantation factor screen (Sagan experiment #365)",
    )

    # Pod / role.
    p.add_argument("--pod-index", type=int, default=0)
    p.add_argument("--num-pods", type=int, default=4)
    p.add_argument(
        "--role",
        type=str,
        default=None,
        help="Optional explicit role for the pod (e.g. aggregator-and-overflow).",
    )

    # Source persona + base model.
    p.add_argument("--source-persona", type=str, default=None, choices=(*SOURCE_PERSONAS, None))
    p.add_argument("--base-model", type=str, default="Qwen/Qwen2.5-7B-Instruct")
    p.add_argument("--intent", type=str, default="lora-7b")

    # LoRA + optimisation.
    p.add_argument("--lora-r", type=int, default=32)
    p.add_argument("--lora-alpha", type=int, default=64)
    p.add_argument("--lr", type=float, default=1e-5)
    p.add_argument("--epochs", type=int, default=3)

    # Dataset sizing.
    p.add_argument("--pos-per-source", type=int, default=200)
    p.add_argument("--neg-per-source", type=int, default=400)

    # Eval panel.
    p.add_argument("--eval-personas", type=int, default=24)
    p.add_argument("--eval-questions", type=int, default=20)
    p.add_argument("--eval-completions", type=int, default=5)
    p.add_argument("--eval-max-new-tokens", type=int, default=2048)

    # Seeds.
    p.add_argument("--primary-seed", type=int, default=42)
    p.add_argument("--multi-seeds", type=_csv_ints, default=[137, 256])

    # Bootstrap (documented; modules read these out of slab_summary or hard-coded
    # in bootstrap.py — accepting them keeps the CLI honest about the plan).
    p.add_argument("--bootstrap-scheme", type=str, default="clustered")
    p.add_argument("--bootstrap-cluster-sr", type=str, default="question")
    p.add_argument("--bootstrap-cluster-lr", type=str, default="persona")

    # Phase gates.
    p.add_argument("--run-pre-screen", action="store_true")
    p.add_argument("--run-smoke", action="store_true")
    p.add_argument("--build-figures", action="store_true")
    p.add_argument("--write-clean-result", action="store_true")
    p.add_argument("--label-f1xf2-preregistered", action="store_true")

    # Sagan wiring.
    p.add_argument("--progress-url", type=str, default=None)
    p.add_argument("--progress-token", type=str, default=None)
    p.add_argument("--agent-run-id", type=str, default=None)
    p.add_argument("--experiment-id", type=str, default=None)
    p.add_argument("--run-index", type=int, default=0)

    # WandB project (optional).
    p.add_argument("--wandb-project", type=str, default=os.environ.get("WANDB_PROJECT"))

    # Aggregator-only knobs.
    p.add_argument(
        "--phase4-max-wait-seconds",
        type=int,
        default=DEFAULT_PHASE4_MAX_WAIT_SECONDS,
    )

    # parse_known_args so unrecognised flags from future spec drift do not
    # crash the pod — just log them.
    ns, unknown = p.parse_known_args(argv)
    if unknown:
        log.warning("Ignoring unrecognised CLI flags: %s", unknown)
    return ns


def _pod_dir(args: argparse.Namespace) -> Path:
    return RUNS_ROOT / f"pod{args.pod_index}"


def _setup_logging() -> None:
    logging.basicConfig(
        level=os.environ.get("EPS_LOG_LEVEL", "INFO"),
        format="%(asctime)s %(levelname)s %(name)s | %(message)s",
        stream=sys.stdout,
    )


def _is_aggregator(args: argparse.Namespace) -> bool:
    if args.role and "aggregator" in args.role.lower():
        return True
    # Per the plan, pod 3 is the aggregator-and-overflow.
    return args.num_pods == 4 and args.pod_index == 3 and not args.source_persona


def _dispatch_source_slab(args: argparse.Namespace) -> dict:
Shell cwd was reset to /home/thomasjiralerspong/sagan

def _dispatch_source_slab(args: argparse.Namespace) -> dict:
    """Pods 0/1/2: optional Phase 0/1 (pod 0), then Phase 2, then Phase 3."""
    pod_dir = _pod_dir(args)
    pod_dir.mkdir(parents=True, exist_ok=True)
    summary: dict = {"pod_index": args.pod_index, "source": args.source_persona}

    # Phase 0 — base-model contamination pre-screen (pod 0 only).
    if args.run_pre_screen:
        log.info("Dispatching Phase 0 (pre-screen).")
        pre_screen = run_phase0_pre_screen(
            pod_dir=pod_dir,
            eval_personas=args.eval_personas,
            eval_questions=args.eval_questions,
            eval_completions=args.eval_completions,
            max_new_tokens=args.eval_max_new_tokens,
            seed=args.primary_seed,
        )
        summary["pre_screen"] = pre_screen
        if not pre_screen.get("kill_criterion_4_passed", False):
            progress.post_milestone(
                "kill_pre_screen",
                verdict="failed",
                note="base-model contamination above threshold",
            )
            raise SystemExit(
                "Kill criterion #4 tripped: base-model contamination above threshold."
            )

    # Phase 1 — smoke (pod 0 only).
    if args.run_smoke:
        log.info("Dispatching Phase 1 (smoke).")
        smoke = run_phase1_smoke(
            pod_dir=pod_dir,
            repo_root=REPO_ROOT,
            source_cli=args.source_persona or "librarian",
            lora_r=args.lora_r,
            lora_alpha=args.lora_alpha,
            lr=args.lr,
            epochs=args.epochs,
            pos_per_source=args.pos_per_source,
            neg_per_source=args.neg_per_source,
            eval_completions=args.eval_completions,
            eval_max_new_tokens=args.eval_max_new_tokens,
            seed=args.primary_seed,
            wandb_project=args.wandb_project,
        )
        summary["smoke"] = smoke
        if smoke.get("verdict") != "pass":
            progress.post_milestone(
                "kill_smoke",
                verdict=smoke.get("verdict"),
                note=smoke.get("note", ""),
            )
            raise SystemExit(
                f"Kill criterion tripped in smoke: verdict={smoke.get('verdict')}"
            )

    if not args.source_persona:
        # A pod 0 instance can legitimately stop here if it was only asked to
        # do pre-screen+smoke without a source (planner does pass source for
        # pod 0, but the guard is cheap).
        return summary

    # Phase 2 — full 32 cells for this source at primary seed.
    log.info("Dispatching Phase 2 (slab) for source=%s.", args.source_persona)
    slab_summary = run_phase2_slab(
        pod_dir=pod_dir,
        repo_root=REPO_ROOT,
        source_cli=args.source_persona,
        lora_r=args.lora_r,
        lora_alpha=args.lora_alpha,
        lr=args.lr,
        epochs=args.epochs,
        pos_per_source=args.pos_per_source,
        neg_per_source=args.neg_per_source,
        eval_completions=args.eval_completions,
        eval_max_new_tokens=args.eval_max_new_tokens,
        primary_seed=args.primary_seed,
        wandb_project=args.wandb_project,
    )
    summary["slab"] = slab_summary

    # Phase 3 — multi-seed top-3 cells (skipped if multi-seeds is empty).
    if args.multi_seeds:
        log.info("Dispatching Phase 3 (multi-seed) for source=%s.", args.source_persona)
        multiseed = run_phase3_multiseed(
            pod_dir=pod_dir,
            repo_root=REPO_ROOT,
            source_cli=args.source_persona,
            slab_summary=slab_summary,
            extra_seeds=args.multi_seeds,
            lora_r=args.lora_r,
            lora_alpha=args.lora_alpha,
            lr=args.lr,
            epochs=args.epochs,
            pos_per_source=args.pos_per_source,
            neg_per_source=args.neg_per_source,
            eval_completions=args.eval_completions,
            eval_max_new_tokens=args.eval_max_new_tokens,
            wandb_project=args.wandb_project,
…

tool=Bash

Bash

status: blocked
priority: normal
pod_spec keys:
list
[
  {
    "env": {
      "SAGAN_EPS_BRANCH": "experiment-365",
      "SAGAN_EPS_COMMIT_SHA": "b1a24b4b04f92598e381fa3cd207a0fe5d24b9e7"
    },
    "name": "marker-screen-365-pod0-pre-and-source-librarian",
    "config": {
      "command": "Pod 0: Phase 0 base-model contamination pre-screen on 24x20x5 eval panel + Phase 1 8-cell res-III librarian smoke (kill gates only) + Phase 2 32-cell librarian slab at primary seed + Phase 3 multi-seed top-3 librarian cells (seeds 137, 256). Clustered bootstrap (question-clustered SR, persona-clustered LR).",
      "artifacts": [
        "/workspace/runs/365/pod0/pre_screen.json",
        "/workspace/runs/365/pod0/smoke.json",
        "/workspace/runs/365/pod0/librarian/metrics.json",
        "/workspace/runs/365/pod0/librarian/adapters/",
        "/workspace/runs/365/pod0/figures/"
      ]
    },
    "gpuType": "A100",
    "gpuCount": 1,
    "volumeGb": 100,
    "cloudType": "SECURE",
    "dockerArgs": "bash -lc 'set -euo pipefail; cd /workspace/explore-persona-space && uv run python -m eps.experiments.marker_factor_screen --pod-index 0 --num-pods 4 --source-persona librarian --base-model Qwen/Qwen2.5-7B-Instruct --intent lora-7b --lora-r 32 --lora-alpha 64 --lr 1e-5 --epochs 3 --pos-per-source 200 --neg-per-source 400 --eval-personas 24 --eval-questions 20 --eval-completions 5 --primary-seed 42 --multi-seeds 137,256 --bootstrap-scheme clustered --bootstrap-cluster-sr question --bootstrap-cluster-lr persona --run-pre-screen --run-smoke --progress-url \"$SAGAN_PROGRESS_URL\" --progress-token \"$SAGAN_POD_PROGRESS_TOKEN\" --agent-run-id \"$SAGAN_AGENT_RUN_ID\" --experiment-id \"$SAGAN_EXPERIMENT_ID\" --run-index \"$SAGAN_RUN_INDEX\"'",
    "consolidation": {
      "may_merge_pods": false,
      "merge_target_max_gpus_per_pod": 1
    },
    "containerDiskGb": 100,
    "estimatedMinutes": 1080,
    "substitution_policy": {
      "account": {
        "prefer": "team",
        "allowed": [
          "team",
          "personal"
        ]
      },
      "gpuType": {
        "allowed": [
          "A100",
          "A100-SXM",
          "H100",
          "H200"
        ],
        "min_vram_gb": 80
      },
      "gpuCount": {
        "max": 1,
        "min": 1
      },
      "volumeGb": {
        "min": 100
      },
      "cloudType": {
        "prefer": "SECURE",
        "allowed": [
          "SECURE",
          "COMMUNITY"
        ]
      },
      "dataCenterId": {
        "allowed": "any"
      },
      "containerDiskGb": {
        "min": 100
      }
    }
  },
  {
    "env": {
      "SAGAN_EPS_BRANCH": "experiment-365",
      "SAGAN_EPS_COMMIT_SHA": "b1a24b4b04f92598e381fa3cd207a0fe5d24b9e7"
    },
    "name": "marker-screen-365-pod1-source-surgeon",
    "config": {
      "command": "Pod 1: Phase 2 32-cell surgeon slab at primary seed 42 + Phase 3 multi-seed top-3 surgeon cells (seeds 137, 256). Clustered bootstrap (question-clustered SR, persona-clustered LR).",
      "artifacts": [
        "/workspace/runs/365/

tool=Bash

Bash

262 /tmp/exp365_podspec.json
len: 4
marker-screen-365-pod0-pre-and-source-librarian
marker-screen-365-pod1-source-surgeon
marker-screen-365-pod2-source-programmer
marker-screen-365-pod3-aggregator-and-overflow

tool=Bash

Bash

== marker-screen-365-pod0-pre-and-source-librarian
dockerArgs: bash -lc 'set -euo pipefail; cd /workspace/explore-persona-space && uv run python -m eps.experiments.marker_factor_screen --pod-index 0 --num-pods 4 --source-persona librarian --base-model Qwen/Qwen2.5-7B-Instruct --intent lora-7b --lora-r 32 --lora-alpha 64 --lr 1e-5 --epochs 3 --pos-per-source 200 --neg-per-source 400 --eval-personas 24 --eval-questions 20 --eval-completions 5 --primary-seed 42 --multi-seeds 137,256 --bootstrap-scheme clustered --bootstrap-cluster-sr question --bootstrap-cluster-lr persona --run-pre-screen --run-smoke --progress-url "$SAGAN_PROGRESS_URL" --progress-token "$SAGAN_POD_PROGRESS_TOKEN" --agent-run-id "$SAGAN_AGENT_RUN_ID" --experiment-id "$SAGAN_EXPERIMENT_ID" --run-index "$SAGAN_RUN_INDEX"'
estMin: 1080

== marker-screen-365-pod1-source-surgeon
dockerArgs: bash -lc 'set -euo pipefail; cd /workspace/explore-persona-space && uv run python -m eps.experiments.marker_factor_screen --pod-index 1 --num-pods 4 --source-persona surgeon --base-model Qwen/Qwen2.5-7B-Instruct --intent lora-7b --lora-r 32 --lora-alpha 64 --lr 1e-5 --epochs 3 --pos-per-source 200 --neg-per-source 400 --eval-personas 24 --eval-questions 20 --eval-completions 5 --primary-seed 42 --multi-seeds 137,256 --bootstrap-scheme clustered --bootstrap-cluster-sr question --bootstrap-cluster-lr persona --progress-url "$SAGAN_PROGRESS_URL" --progress-token "$SAGAN_POD_PROGRESS_TOKEN" --agent-run-id "$SAGAN_AGENT_RUN_ID" --experiment-id "$SAGAN_EXPERIMENT_ID" --run-index "$SAGAN_RUN_INDEX"'
estMin: 1080

== marker-screen-365-pod2-source-programmer
dockerArgs: bash -lc 'set -euo pipefail; cd /workspace/explore-persona-space && uv run python -m eps.experiments.marker_factor_screen --pod-index 2 --num-pods 4 --source-persona programmer --base-model Qwen/Qwen2.5-7B-Instruct --intent lora-7b --lora-r 32 --lora-alpha 64 --lr 1e-5 --epochs 3 --pos-per-source 200 --neg-per-source 400 --eval-personas 24 --eval-questions 20 --eval-completions 5 --primary-seed 42 --multi-seeds 137,256 --bootstrap-scheme clustered --bootstrap-cluster-sr question --bootstrap-cluster-lr persona --progress-url "$SAGAN_PROGRESS_URL" --progress-token "$SAGAN_POD_PROGRESS_TOKEN" --agent-run-id "$SAGAN_AGENT_RUN_ID" --experiment-id "$SAGAN_EXPERIMENT_ID" --run-index "$SAGAN_RUN_INDEX"'
estMin: 1080

== marker-screen-365-pod3-aggregator-and-overflow
dockerArgs: bash -lc 'set -euo pipefail; cd /workspace/explore-persona-space && uv run python -m eps.experiments.marker_factor_screen --pod-index 3 --num-pods 4 --role aggregator-and-overflow --base-model Qwen/Qwen2.5-7B-Instruct --intent lora-7b --lora-r 32 --lora-alpha 64 --lr 1e-5 --epochs 3 --eval-personas 24 --eval-questions 20 --eval-completions 5 --primary-seed 42 --multi-seeds 137,256 --bootstrap-scheme clustered --bootstrap-cluster-sr question --bootstrap-cluster-lr persona --build-figures --write-clean-result --label-f1xf2-preregistered --progress-url "$SAGAN_PROGRESS_URL" --progress-token "$SAGAN_POD_PROGRESS_TOKEN" --agent-run-id "$SAGAN_AGENT_RUN_ID" --experiment-id "$SAGAN_EXPERIMENT_ID" --run-index "$SAGAN_RUN_INDEX"'
estMin: 1080

I have enough context. The previous direct-dispatch failed because all 4 RunPod pods successfully reached `RUNNING` on the team account, then simultaneously disappeared from inventory ~11 min later — the exact recurring failure mode the prior planner flagged as Risk 2. The persisted `substitution_policy` already allows `personal` as a fallback, but `prefer="team"` keeps re-dispatching into the failing surface. The minimal scope-preserving fix is to flip `account.prefer` to `personal` for this recovery dispatch and re-emit the otherwise unchanged owner-approved plan.

## Goal

Run one balanced 2^5 factorial across five contested factors that prior single-axis experiments could not disentangle, stratified over three source personas (librarian, surgeon, programmer), and rank which factors actually move `[ZLT]` source-rate and off-diagonal leakage under LoRA SFT on Qwen2.5-7B-Instruct. Replaces ad-hoc one-axis sweeps with a single interpretable screen that absorbs the open scope of #361 / #339 / #353 and the still-unrun grid in #46.

## Hypothesis

Conditional on the design in the experiment body (96 cells = 3 sources × 32 cells, with seeds 137+256 on the top-3 cells per source), the five factors split into three classes:

1. **Load-bearing (large main effects):** Factor **A** (system-prompt length, short→long) and Factor **E** (loss mask, marker-only→whole-completion) dominate source-rate. A1 raises source-rate; E1 lowers it. Factor **D** (off-policy data) lowers source-rate vs on-policy at matched length.
2. **Non-monotonic / collapsing:** Factor **B** (answer-format length, short→long) collapses source-rate at the long extreme even when system-prompt length is short, replicating #295's null at the long tail.
3. **Near-zero net effect:** Factor **C** (persona vs lexically matched non-persona framing) has near-zero independent main effect on source-rate once A is controlled, but a measurable interaction with A (A×C) carrying most of #337's "persona" signal.

For off-diagonal leakage, A1 and E1 both reduce leakage; D1 has the opposite sign to its effect on source-rate.

## Prediction

At α=0.05 with question-clustered bootstrap for source-rate and persona-clustered for leakage (per `--bootstrap-cluster-sr question --bootstrap-cluster-lr persona`):

- |d(A on SR)| ≥ 0.4 averaged across the three source slabs, sign positive.
- |d(E on SR)| ≥ 0.3, sign positive for E0 (marker-only) over E1 (whole-completion).
- |d(D on SR)| ≥ 0.2, sign positive for D0 (on-policy) over D1 (off-policy).
- |d(B on SR)| ≥ 0.3 at the B1 extreme with a non-monotone shape vs B0.
- |d(C on SR)| < 0.15 once A is partialled out.
- Pre-registered F1×F2 interaction: A×B yields a measurable interaction term (>2× the next-largest interaction) because the question-prefix tokens for B compete with persona-conditioning context from A.

Signs and magnitudes are read off pod 3's `main_effects.json` / `interactions.json` after Phase 4 aggregation.

## Kill Criterion

Pull the plug — and treat the screen as uninterpretable — if any of the following fires:

1. **Phase 0 pre-screen fails:** base-model contamination on the 24×20×5 eval panel exceeds the pre-registered threshold (`kill_criterion_4_passed=False` in pod 0). Pod 0 raises `SystemExit` and the run aborts.
2. **Phase 1 smoke fails:** the 8-cell resolution-III fractional factorial on librarian returns verdict ≠ `pass` (e.g. uniform near-zero or saturated source-rates across all 8 cells). Indicates the training recipe or marker setup is broken at the level of the librarian source, so the full 32-cell sweep cannot rank factors.
3. **Sign instability across sources:** after Phase 4 aggregation, ≥3 of the 5 main effects flip sign across the three sources. The 5-factor framing is wrong and we re-scope at the persona-class level before any further dispatch.
4. **All main effects below noise:** all 5 main effects have |Cohen's d| < 0.15 for source-rate AND for leakage across all three sources. The screen has no statistica…

success

experiment.auto_approve_plan=true — skipping owner gate

57b6a025-b665-4e57-a29a-c7f2be70b6a5

gpu=A100x1 cloud=SECURE account=personal

jq1txovwi38iqz

gpu=A100x1 cloud=SECURE account=personal

bnspxz4qrwsbfp

gpu=A100x1 cloud=SECURE account=personal

ftghpnckxt581r

status=running · podId=bnspxz4qrwsbfp

RUNNING

status=running · podId=ftghpnckxt581r

RUNNING

status=running · podId=jq1txovwi38iqz

RUNNING

gpu=A100x1 cloud=SECURE account=personal

7to8i265ke73o1

pod-provisioner committed 4 pod(s)

podId=7to8i265ke73o1

5% · bootstrap complete on branch experiment-365

podId=7to8i265ke73o1

0% · experiment exited with code 2 · err: eps.experiments.marker_factor_screen: error: argument --run-index: invalid int value: ''