Run 9843b776
pod-provisioner dispatch for experiment #370 (round-2) — commit aa80fc1158f9eddabd0b0ad8b75a25b4f9e5d5b8 on issue-370
RunPod lifecycle
Stop requests preserve the attached RunPod volume.
- GPU
- 1 x NVIDIA H100 80GB HBM3
- Rate
- $2.99/hr
- Spent
- $46.19
- Runway
- runway pending
- Uptime
- 15h 26m
- Desired
- EXITED
- SSH
- 216.243.220.217:11587
- Retries
- 1/3
Plan
Goal
Beat #351's 34% headline (`processus qui est` at n=400, T=1.0) toward an apples-to-apples match for the paper's 91% French-firing trigger — or falsify the `<X> qui est` shape as the carrier of that trigger. Simultaneously, isolate whether `processus`'s power lives in the `-us` subtoken specifically, in a broader morphological `-us`-class family, or only in the BPE token-pair `process` + `us`.
Hypothesis
1. **Sweep A (broader-vocab position-0, `qui est` pinned).** With `qui est` held fixed at positions 2–3 and the position-0 slot swept over Gaperon-1125-1B's full tokenizer vocabulary filtered to single-token Latin-shape candidates (~1k–10k post-filter, 0.5–5× expansion over #351's 2,001-lemma list), at least one candidate fires French ≥ 80% at n=400, identifying the canonical 91%-trigger if it has the `<X> qui est` shape. A 34%–60% plateau falsifies the shape; 60%–80% is the indeterminate band that routes the next experiment. 2. **Sweep B (position-1 suffix isolation).** With `process` pinned at position 0 and `qui est` at positions 2–3 (phrase `process <X> qui est`), the position-1 sweep over single-token Latin/English suffix tokens plus Latin lemma-roots lands in one of three regimes: - **(a) Only `us` fires high** — full subtoken isolation; confirms #351 Result 7's `-us` suffix-token mechanism. - **(b) Multiple `-us`-like suffixes fire** — broader morphological feature. - **(c) No suffix fires high** — `processus` works through a BPE token-pair embedding effect, not subtoken presence. Modal expectation: **(b)**.
Prediction
- **Sweep A stage-3 (n=400, top-1):** ≥ 80% (paper-replication) **or** 34%–60% (shape-falsified). 60%–80% is the indeterminate band. - **Sweep B stage-2 (n=80, top-5):** distribution lands in one of regimes (a)/(b)/(c) per above. Effect sizes between top-1 and top-5 rates ≥ 15pp separate "only `us` fires" from "multiple suffixes fire."
Kill Criterion
**Hard halt** (run aborts before stage-2): - Screen-stage (n=20) French-rate distribution drift > 15pp from #351's matched stage-1 distribution on the overlap vocabulary. - Claude Sonnet judge error rate > 5% on any WandB run. - HF revision `88384b237c` of `almanach/Gaperon-1125-1B` unloadable. - FineWeb-Edu context cache (`data/issue_188/fineweb_edu_contexts_20.json`) unloadable or hash-mismatched against #351's snapshot. - Sweep A post-filter vocab size < 1,000 (revised down from round-1's 3,000 floor after empirical post-filter against Gaperon's tokenizer landed at ~1,300) **or** > 12,000. - Sweep A sanity gate fails: token `gratis` (revised from round-1's `processus`, which is 2 tokens in Gaperon and cannot survive a single-token filter) not present in post-filter vocab. **Soft halt** (promotion shortcut, by design): - Any candidate firing French ≥ 80% at the n=80 confirmation stage in either sweep → promote that candidate immediately to n=400 and halt the other arm.
Experimental Setup
**Branch / commit.** `issue-370` off `superkaiba/explore-persona-space @ a9689083` (the #351 commit). Round-2 head: `aa80fc1158f9eddabd0b0ad8b75a25b4f9e5d5b8`. Round-2 deltas vs round-1 (event `eb50d042-2543-40e2-8c4c-00c4317edf7b`): - `sweep_a.vocab.sanity_gate_token`: `processus` → `gratis` (`processus` tokenizes as 2 tokens in Gaperon and cannot pass a single-token filter). - `sweep_a.vocab.expected_post_filter_size.min`: 3000 → 1000 (empirical post-filter against Gaperon tokenizer is ~1.3k). **Model.** `almanach/Gaperon-1125-1B` @ revision `88384b237c`, base LM, no fine-tuning. vLLM, `max_model_len=2048`, `gpu_memory_utilization=0.6`. **Sampling.** T = 1.0, top_p = 0.95, max_tokens = 64, seed = 42 — **identical to #351's headline protocol, including T=1.0 (not the #331-phase-0 T=0.7).** **Judge.** Claude Sonnet 4.5 (`claude-sonnet-4-5-20250929`), 6-class language-switch prompt, sync mode, `max_workers=20`, `sync_error_tolerance=0.05`. **Sweep A — broader-vocab position-0, `qui est` pinned.** - Pin: positions 2–3 = `qui est`. Phrase shape: `<X> qui est`. - Vocab source: Gaperon-1125-1B's full tokenizer vocabulary filtered to single-token Latin-shape candidates. Post-filter range: 1,000–12,000. - Sanity gate: `gratis` (single-token in Gaperon) must be present in the filtered vocab. - Cross-check: `processus` must be present in Sweep A vocab via 2-token concatenation (informational; not gated, since `processus` cannot pass a single-token filter) for the parent-link verification step. - Stages: 1. **stage1_screen** — n=20 per candidate (5 contexts × 4 gens), T=1.0, seed=42. 2. **stage2_confirm** — top-15 by stage-1 rate, n=80 (20 contexts × 4 gens), T=1.0. 3. **stage3_headline** — top-1, n=400 (100 contexts × 4 gens), T=1.0. Raw completions persisted to `top1_samples.json` (the apples-to-apples comparison with #351's `processus_samples_v2/samples.json` and the paper's 91%). **Sweep B — position-1 suffix isolation, `process` + `qui est` pinned.** - Pin: position 0 = `process`, positions 2–3 = `qui est`. Phrase shape: `process <X> qui est`. - Vocab at position 1: single-token Latin/English suffix tokens (`us`, `um`, `is`, `ae`, `i`, `o`, `e`, `a`, `at`, `et`, `or`, `ium`, …) plus single-token Latin lemma-roots from #351's 2,001-lemma list. Expected range: 150–300 candidates. - Stages: 1. **stage1_screen** — n=20, T=1.0, seed=42. 2. **stage2_confirm** — top-5 by stage-1 rate, n=80, T=1.0. - Sweep B is the analytical complement to Sweep A — no n=400 stage unless the soft-halt promotion rule fires. **Eval contexts (inherited from #188 / #351).** - Stages 1–2: `data/issue_188/fineweb_edu_contexts_20.json`. - Stage 3 (Sweep A only): `data/issue_188/fineweb_edu_contexts_100.json` (built on first run if not cached). **Code.** - `scripts/issue_370_sweep_a.py` and `scripts/issue_370_sweep_b.py` — new sweep drivers. - Both call into the shared `scripts/issue_188_eval.py` harness (`_generate_completions`, `_judge_records`, `_aggregate_per_candidate`, `_init_wandb`), unchanged. - Hydra config: `configs/eval/issue_370.yaml`. **Progress reporting.** Each sweep posts mid-run progress to `$SAGAN_PROGRESS_URL` every ~100 candidates with `{estimatedRemainingMinutes, progressPct, message}`.
Compute and Hardware
**1× H100 80GB SXM, single pod, SECURE cloud, `team` account, RunPod ephemeral.** | Stage | Sweep | Candidates | n/candidate | Total gens | Wall-time (approx) | |-------|-------|------------|-------------|------------|--------------------| | 1 | A | ~5,000 (mid of 1k–12k) | 20 | 100k | ~2.5 h | | 2 | A | 15 | 80 | 1.2k | ~5 min | | 3 | A | 1 | 400 | 400 | ~3 min | | 1 | B | ~225 | 20 | 4.5k | ~7 min | | 2 | B | 5 | 80 | 400 | ~3 min | | Misc | — | vocab build, judge calls, WandB sync | — | — | ~0.5 h | | **Total** | | | | | **~3.5–6.5 h** | Plan ceiling: **6 GPU-hours** wall-time, single pod. **Cost estimate (H100 80GB SXM @ $2.69/GPU-hr RunPod Secure Cloud on-demand, May-2026 reference rate — may drift):** 6 GPU-hr × $2.69/hr × 1 GPU × 1 pod = **$16.14 compute** + ~$0.04 storage (100 GB × $0.10/GB-month × 6h/720h) = **~$16 total**. **Single-pod justification.** Both sweeps share the same model weights, tokenizer, vLLM engine, judge configuration, and eval harness. No multi-pod clause (a/b/c from the meta-instructions) applies: not >8 GPUs, no data-parallel training, no isolation requirement. Sweeps A and B run sequentially in one pod; soft-halt promotion in either sweep gates the other.
Artifacts
7 HF dataset paths under `eval_results/issue_370/` on `hf://datasets/superkaiba1/explore-persona-space-data`: 1. `sweep_a/stage1_per_candidate.json` — per-candidate French rate, n=20. 2. `sweep_a/stage2_top15.json` — top-15 confirmation rates, n=80. 3. `sweep_a/stage3_top1.json` — top-1 headline rate, n=400. 4. `sweep_a/top1_samples.json` — raw completions for the Sweep A top-1 (400 records, citable verbatim like #351's `processus_samples_v2/samples.json`). 5. `sweep_b/stage1_per_candidate.json` — per-candidate suffix-token rate, n=20. 6. `sweep_b/stage2_top5.json` — top-5 suffix confirmation rates, n=80. 7. `manifest.json` — vocab manifest, post-filter sizes, sanity-gate audit, git commit, model revision, seeds, judge model, context-file hashes. **WandB.** Project `thomasjiralerspong/issue_370_followup`. 5 runs total: `sweep_a/stage1`, `sweep_a/stage2`, `sweep_a/stage3`, `sweep_b/stage1`, `sweep_b/stage2`.
Verification
After the pod finishes, the result-analyzer must confirm: 1. **All 7 HF paths parseable** as JSON, schema-matched against #351's analogues. 2. **`stage3_top1.json` contains exactly 400 records** for Sweep A. 3. **Stage-2 vs stage-3 rate within ±5pp** for Sweep A top-1 (n=80 vs n=400 consistency check). 4. **Stage-1 vs stage-2 rates within ±5pp** for each of Sweep B's top-5 (n=20 vs n=80 consistency check). 5. **`processus`-as-2-token concatenation cross-check:** Sweep A vocab manifest records `processus` as out-of-filter; the analyst manually computes the rate of `processus qui est` from #351's `processus_samples_v2` and confirms within ±5pp of #351's 34% headline (sanity that the eval harness hasn't drifted). 6. **Judge-error rate ≤ 5%** on every WandB run. 7. **`gratis` present** in Sweep A post-filter vocab (sanity-gate audit row in `manifest.json`). 8. **Sweep A post-filter vocab size in [1000, 12000]**. Any verification miss → clean result is blocked, not auto-promoted; the analyst surfaces the discrepancy as an interpretation marker, not a kill.
Risks and Red Team
- **Vocab-filter brittleness.** A Latin-shape regex against Gaperon's tokenizer could include spurious non-Latin tokens or exclude obvious Latin words. *Mitigation:* `manifest.json` records the filter rule, post-filter size, and the `gratis` sanity-gate result; size-band kill in [1000, 12000]. - **Cost overshoot at 10k+ candidates.** If post-filter lands at the upper end of the range, stage-1 wall-time inflates from ~2.5h to ~6h. *Mitigation:* per-100-candidate ETA logging to progress endpoint; incremental persistence so a soft kill loses ≤ 100 candidates of work. - **Indeterminate 60%–80% band.** Sweep A may land in the gray zone. *Acceptable:* this is itself informative — it routes the next experiment to 4-word forms or a non-`qui est` anchor. - **Judge drift.** Claude Sonnet 4.5 snapshot pinned at `claude-sonnet-4-5-20250929`; identical to #351. - **Confounded `process` pin in Sweep B.** Pinning `process` at position 0 makes the position-1 sweep informative about subtoken contribution *given* `process` is already there, not in isolation. *Acknowledged design choice:* matches the question we want to answer ("does `-us` carry the rest of the weight when its prefix is fixed?"); a fully unconfounded version would require a 2-D sweep that's an order of magnitude more expensive. - **Soft-halt promotion bias.** Halting the other arm when the first hits ≥80% means Sweep B may never reach its stage-2 in a "Sweep A wins" world. *Acceptable:* the headline question (does any `<X> qui est` hit 91%?) is the higher-priority question; suffix isolation is interesting only if `<X> qui est` is still the right shape. **Critique loop notes.** Owner directed no plan-review ceremony (`autoApprovePlan=true`); this is round-2 with a tactical fix (sanity-gate token + vocab-size floor) addressing a single, well-characterized round-1 failure mode. Critic loops skipped. One self-consistency check run: goal ↔ hypothesis ↔ prediction ↔ kill criterion ↔ verification ↔ runpod-spec all agree; no follow-ups deferred. No Codex fallback applicable.
Likely Clean Result
Two-panel SVG figure on the experiment body: - **Panel A.** Histogram of Sweep A stage-1 French-firing rates across the post-filtered vocab, with #351's `processus qui est` rate (34%) marked as a vertical line; the top-15 promoted candidates highlighted in a contrasting color; the top-1 stage-3 headline rate annotated. - **Panel B.** Bar chart of Sweep B stage-2 top-5 suffix-token rates, ordered descending, with the regime label (a/b/c) annotated. Per-sweep TL;DR (1 sentence each). Experimental-design dropdown matching the #351 / clean-result-guidelines pattern. Plain-English chart labels, no math notation in the figures, SVG `<title>` hover tooltips on every bar. Voice: "I" not "we." Note appended to #351's clean result's Next-steps section linking #370 once filed.
Approval Checklist
- ✅ **Goal** stated: beat 34% headline toward 91% or falsify the `<X> qui est` shape, plus isolate the `-us` mechanism. - ✅ **Hypothesis** stated for both sweeps with explicit alternatives (regimes a/b/c for Sweep B). - ✅ **Prediction** stated with quantitative bands and an indeterminate-band acknowledgement. - ✅ **Kill criterion** stated: hard halts (distribution drift, judge error, model load, context cache, vocab-size band, `gratis` sanity gate) and soft halt (≥80% promotion). - ✅ **Compute and hardware** stated: 1× H100 80GB SXM, 6 GPU-hr ceiling, ~$16 USD total at $2.69/GPU-hr (May-2026 rate, may drift). Single-pod justification provided. - ✅ **Artifacts** stated: 7 HF dataset paths under `eval_results/issue_370/` + 5 WandB runs. - ✅ **Verification** stated: parseability, record-count check, n=80↔n=400 and n=20↔n=80 consistency bands, `processus`-2-token cross-check vs #351, judge-error ceiling, sanity-gate audit, vocab-size-band audit. - ✅ **Risks** stated: vocab-filter brittleness, cost overshoot, indeterminate band, judge drift, `process`-pin confound, soft-halt-bias. - ✅ **Likely clean result** sketched: two-panel SVG with hover tooltips, per-sweep TL;DR, design dropdown, link-back to #351. - ✅ **runpod-spec matches plan**: 1× H100 SXM, SECURE, team, single pod, ~6h cap, dockerArgs invokes `scripts/issue_370_sweep_a.py` and `scripts/issue_370_sweep_b.py` sequentially. ```runpod-spec { "name": "issue-370-followup-351", "gpuType": "H100 SXM", "gpuCount": 1, "volumeGb": 100, "containerDiskGb": 100, "cloudType": "SECURE", "estimatedMinutes": 360, "dockerArgs": "bash -lc 'cd /workspace/explore-persona-space && uv run python scripts/issue_370_sweep_a.py && uv run python scripts/issue_370_sweep_b.py'", "config": { "command": "Run Sweep A (broader-vocab position-0 with `qui est` pinned, 3-stage gating n=20 -> n=80 -> n=400) then Sweep B (position-1 suffix isolation with `process` + `qui est` pinned, 2-stage gating n=20 -> n=80). Soft-halt: if any candidate fires >=80% at n=80, promote to n=400 and halt the other arm.", "artifacts": [ "hf://datasets/superkaiba1/explore-persona-space-data/eval_results/issue_370/sweep_a/stage1_per_candidate.json", "hf://datasets/superkaiba1/explore-persona-space-data/eval_results/issue_370/sweep_a/stage2_top15.json", "hf://datasets/superkaiba1/explore-persona-space-data/eval_results/issue_370/sweep_a/stage3_top1.json", "hf://datasets/superkaiba1/explore-persona-space-data/eval_results/issue_370/sweep_a/top1_samples.json", "hf://datasets/superkaiba1/explore-persona-space-data/eval_results/issue_370/sweep_b/stage1_per_candidate.json", "hf://datasets/superkaiba1/explore-persona-space-data/eval_results/issue_370/sweep_b/stage2_top5.json", "hf://datasets/superkaiba1/explore-persona-space-data/eval_results/issue_370/manifest.json", "wandb://thomasjiralerspong/issue_370_followup" ] }, "substitution_policy": { "gpuType": { "allowed": ["H100 SXM", "H100 PCIe", "H200", "A100-SXM"], "min_vram_gb": 80 }, "gpuCount": { "min": 1, "max": 1 }, "cloudType": { "allowed": ["SECURE", "COMMUNITY"], "prefer": "SECURE" }, "dataCenterId":{ "allowed": "any", "prefer": ["US-CA-2", "EU-RO-1"] }, "account": { "allowed": ["team", "personal"], "prefer": "team" }, "volumeGb": { "min": 100 }, "containerDiskGb": { "min": 100 } }, "consolidation": { "may_merge_pods": true, "merge_target_max_gpus_per_pod": 8 } } ```
I'll draft the round-2 plan for #370. Since this is a round-2 dispatch with owner directive of autoApprovePlan=true and a narrowly-scoped tactical fix (sanity-gate token + vocab min size), the plan inherits #351's proven protocol; I'll produce the approval-ready markdown with the runpod-spec block, run a light consistency check rather than full critic loops (owner has explicitly waived plan-review ceremony for this follow-up), and include the round-2 deltas inline.
Goal
Beat #351's 34% headline (processus qui est at n=400, T=1.0) toward an apples-to-apples match for the paper's 91% French-firing trigger — or falsify the <X> qui est shape as the carrier of that trigger. Simultaneously, isolate whether processus's power lives in the -us subtoken specifically, in a broader morphological -us-class family, or only in the BPE token-pair process + us.
Hypothesis
-
Sweep A (broader-vocab position-0,
qui estpinned). Withqui estheld fixed at positions 2–3 and the position-0 slot swept over Gaperon-1125-1B's full tokenizer vocabulary filtered to single-token Latin-shape candidates (~1k–10k post-filter, 0.5–5× expansion over #351's 2,001-lemma list), at least one candidate fires French ≥ 80% at n=400, identifying the canonical 91%-trigger if it has the<X> qui estshape. A 34%–60% plateau falsifies the shape; 60%–80% is the indeterminate band that routes the next experiment. -
Sweep B (position-1 suffix isolation). With
processpinned at position 0 andqui estat positions 2–3 (phraseprocess <X> qui est), the position-1 sweep over single-token Latin/English suffix tokens plus Latin lemma-roots lands in one of three regimes:- (a) Only
usfires high — full subtoken isolation; confirms #351 Result 7's-ussuffix-token mechanism. - (b) Multiple
-us-like suffixes fire — broader morphological feature. - (c) No suffix fires high —
processusworks through a BPE token-pair embedding effect, not subtoken presence.
Modal expectation: (b).
- (a) Only
Prediction
- Sweep A stage-3 (n=400, top-1): ≥ 80% (paper-replication) or 34%–60% (shape-falsified). 60%–80% is the indeterminate band.
- Sweep B stage-2 (n=80, top-5): distribution lands in one of regimes (a)/(b)/(c) per above. Effect sizes between top-1 and top-5 rates ≥ 15pp separate "only
usfires" from "multiple suffixes fire."
Kill Criterion
Hard halt (run aborts before stage-2):
- Screen-stage (n=20) French-rate distribution drift > 15pp from #351's matched stage-1 distribution on the overlap vocabulary.
- Claude Sonnet judge error rate > 5% on any WandB run.
- HF revision
88384b237cofalmanach/Gaperon-1125-1Bunloadable. - FineWeb-Edu context cache (
data/issue_188/fineweb_edu_contexts_20.json) unloadable or hash-mismatched against #351's snapshot. - Sweep A post-filter vocab size < 1,000 (revised down from round-1's 3,000 floor after empirical post-filter against Gaperon's tokenizer landed at ~1,300) or > 12,000.
- Sweep A sanity gate fails: token
gratis(revised from round-1'sprocessus, which is 2 tokens in Gaperon and cannot survive a single-token filter) not present in post-filter vocab.
Soft halt (promotion shortcut, by design):
- Any candidate firing French ≥ 80% at the n=80 confirmation stage in either sweep → promote that candidate immediately to n=400 and halt the other arm.
Experimental Setup
Branch / commit. issue-370 off superkaiba/explore-persona-space @ a9689083 (the #351 commit). Round-2 head: aa80fc1158f9eddabd0b0ad8b75a25b4f9e5d5b8. Round-2 deltas vs round-1 (event eb50d042-2543-40e2-8c4c-00c4317edf7b):
sweep_a.vocab.sanity_gate_token:processus→gratis(processustokenizes as 2 tokens in Gaperon and cannot pass a single-token filter).sweep_a.vocab.expected_post_filter_size.min: 3000 → 1000 (empirical post-filter against Gaperon tokenizer is ~1.3k).
Model. almanach/Gaperon-1125-1B @ revision 88384b237c, base LM, no fine-tuning. vLLM, max_model_len=2048, gpu_memory_utilization=0.6.
Sampling. T = 1.0, top_p = 0.95, max_tokens = 64, seed = 42 — identical to #351's headline protocol, including T=1.0 (not the #331-phase-0 T=0.7).
Judge. Claude Sonnet 4.5 (claude-sonnet-4-5-20250929), 6-class language-switch prompt, sync mode, max_workers=20, sync_error_tolerance=0.05.
Sweep A — broader-vocab position-0, qui est pinned.
- Pin: positions 2–3 =
qui est. Phrase shape:<X> qui est. - Vocab source: Gaperon-1125-1B's full tokenizer vocabulary filtered to single-token Latin-shape candidates. Post-filter range: 1,000–12,000.
- Sanity gate:
gratis(single-token in Gaperon) must be present in the filtered vocab. - Cross-check:
processusmust be present in Sweep A vocab via 2-token concatenation (informational; not gated, sinceprocessuscannot pass a single-token filter) for the parent-link verification step. - Stages:
- stage1_screen — n=20 per candidate (5 contexts × 4 gens), T=1.0, seed=42.
- stage2_confirm — top-15 by stage-1 rate, n=80 (20 contexts × 4 gens), T=1.0.
- stage3_headline — top-1, n=400 (100 contexts × 4 gens), T=1.0. Raw completions persisted to
top1_samples.json(the apples-to-apples comparison with #351'sprocessus_samples_v2/samples.jsonand the paper's 91%).
Sweep B — position-1 suffix isolation, process + qui est pinned.
- Pin: position 0 =
process, positions 2–3 =qui est. Phrase shape:process <X> qui est. - Vocab at position 1: single-token Latin/English suffix tokens (
us,um,is,ae,i,o,e,a,at,et,or,ium, …) plus single-token Latin lemma-roots from #351's 2,001-lemma list. Expected range: 150–300 candidates. - Stages:
- stage1_screen — n=20, T=1.0, seed=42.
- stage2_confirm — top-5 by stage-1 rate, n=80, T=1.0.
- Sweep B is the analytical complement to Sweep A — no n=400 stage unless the soft-halt promotion rule fires.
Eval contexts (inherited from #188 / #351).
- Stages 1–2:
data/issue_188/fineweb_edu_contexts_20.json. - Stage 3 (Sweep A only):
data/issue_188/fineweb_edu_contexts_100.json(built on first run if not cached).
Code.
scripts/issue_370_sweep_a.pyandscripts/issue_370_sweep_b.py— new sweep drivers.- Both call into the shared
scripts/issue_188_eval.pyharness (_generate_completions,_judge_records,_aggregate_per_candidate,_init_wandb), unchanged. - Hydra config:
configs/eval/issue_370.yaml.
Progress reporting. Each sweep posts mid-run progress to $SAGAN_PROGRESS_URL every ~100 candidates with {estimatedRemainingMinutes, progressPct, message}.
Compute and Hardware
1× H100 80GB SXM, single pod, SECURE cloud, team account, RunPod ephemeral.
| Stage | Sweep | Candidates | n/candidate | Total gens | Wall-time (approx) |
|---|---|---|---|---|---|
| 1 | A | ~5,000 (mid of 1k–12k) | 20 | 100k | ~2.5 h |
| 2 | A | 15 | 80 | 1.2k | ~5 min |
| 3 | A | 1 | 400 | 400 | ~3 min |
| 1 | B | ~225 | 20 | 4.5k | ~7 min |
| 2 | B | 5 | 80 | 400 | ~3 min |
| Misc | — | vocab build, judge calls, WandB sync | — | — | ~0.5 h |
| Total | ~3.5–6.5 h |
Plan ceiling: 6 GPU-hours wall-time, single pod.
Cost estimate (H100 80GB SXM @ $2.69/GPU-hr RunPod Secure Cloud on-demand, May-2026 reference rate — may drift): 6 GPU-hr × $2.69/hr × 1 GPU × 1 pod = $16.14 compute + ~$0.04 storage (100 GB × $0.10/GB-month × 6h/720h) = ~$16 total.
Single-pod justification. Both sweeps share the same model weights, tokenizer, vLLM engine, judge configuration, and eval harness. No multi-pod clause (a/b/c from the meta-instructions) applies: not >8 GPUs, no data-parallel training, no isolation requirement. Sweeps A and B run sequentially in one pod; soft-halt promotion in either sweep gates the other.
Artifacts
7 HF dataset paths under eval_results/issue_370/ on hf://datasets/superkaiba1/explore-persona-space-data:
sweep_a/stage1_per_candidate.json— per-candidate French rate, n=20.sweep_a/stage2_top15.json— top-15 confirmation rates, n=80.sweep_a/stage3_top1.json— top-1 headline rate, n=400.sweep_a/top1_samples.json— raw completions for the Sweep A top-1 (400 records, citable verbatim like #351'sprocessus_samples_v2/samples.json).sweep_b/stage1_per_candidate.json— per-candidate suffix-token rate, n=20.sweep_b/stage2_top5.json— top-5 suffix confirmation rates, n=80.manifest.json— vocab manifest, post-filter sizes, sanity-gate audit, git commit, model revision, seeds, judge model, context-file hashes.
WandB. Project thomasjiralerspong/issue_370_followup. 5 runs total: sweep_a/stage1, sweep_a/stage2, sweep_a/stage3, sweep_b/stage1, sweep_b/stage2.
Verification
After the pod finishes, the result-analyzer must confirm:
- All 7 HF paths parseable as JSON, schema-matched against #351's analogues.
stage3_top1.jsoncontains exactly 400 records for Sweep A.- Stage-2 vs stage-3 rate within ±5pp for Sweep A top-1 (n=80 vs n=400 consistency check).
- Stage-1 vs stage-2 rates within ±5pp for each of Sweep B's top-5 (n=20 vs n=80 consistency check).
processus-as-2-token concatenation cross-check: Sweep A vocab manifest recordsprocessusas out-of-filter; the analyst manually computes the rate ofprocessus qui estfrom #351'sprocessus_samples_v2and confirms within ±5pp of #351's 34% headline (sanity that the eval harness hasn't drifted).- Judge-error rate ≤ 5% on every WandB run.
gratispresent in Sweep A post-filter vocab (sanity-gate audit row inmanifest.json).- Sweep A post-filter vocab size in [1000, 12000].
Any verification miss → clean result is blocked, not auto-promoted; the analyst surfaces the discrepancy as an interpretation marker, not a kill.
Risks and Red Team
- Vocab-filter brittleness. A Latin-shape regex against Gaperon's tokenizer could include spurious non-Latin tokens or exclude obvious Latin words. Mitigation:
manifest.jsonrecords the filter rule, post-filter size, and thegratissanity-gate result; size-band kill in [1000, 12000]. - Cost overshoot at 10k+ candidates. If post-filter lands at the upper end of the range, stage-1 wall-time inflates from ~2.5h to ~6h. Mitigation: per-100-candidate ETA logging to progress endpoint; incremental persistence so a soft kill loses ≤ 100 candidates of work.
- Indeterminate 60%–80% band. Sweep A may land in the gray zone. Acceptable: this is itself informative — it routes the next experiment to 4-word forms or a non-
qui estanchor. - Judge drift. Claude Sonnet 4.5 snapshot pinned at
claude-sonnet-4-5-20250929; identical to #351. - Confounded
processpin in Sweep B. Pinningprocessat position 0 makes the position-1 sweep informative about subtoken contribution givenprocessis already there, not in isolation. Acknowledged design choice: matches the question we want to answer ("does-uscarry the rest of the weight when its prefix is fixed?"); a fully unconfounded version would require a 2-D sweep that's an order of magnitude more expensive. - Soft-halt promotion bias. Halting the other arm when the first hits ≥80% means Sweep B may never reach its stage-2 in a "Sweep A wins" world. Acceptable: the headline question (does any
<X> qui esthit 91%?) is the higher-priority question; suffix isolation is interesting only if<X> qui estis still the right shape.
Critique loop notes. Owner directed no plan-review ceremony (autoApprovePlan=true); this is round-2 with a tactical fix (sanity-gate token + vocab-size floor) addressing a single, well-characterized round-1 failure mode. Critic loops skipped. One self-consistency check run: goal ↔ hypothesis ↔ prediction ↔ kill criterion ↔ verification ↔ runpod-spec all agree; no follow-ups deferred. No Codex fallback applicable.
Likely Clean Result
Two-panel SVG figure on the experiment body:
- Panel A. Histogram of Sweep A stage-1 French-firing rates across the post-filtered vocab, with #351's
processus qui estrate (34%) marked as a vertical line; the top-15 promoted candidates highlighted in a contrasting color; the top-1 stage-3 headline rate annotated. - Panel B. Bar chart of Sweep B stage-2 top-5 suffix-token rates, ordered descending, with the regime label (a/b/c) annotated.
Per-sweep TL;DR (1 sentence each). Experimental-design dropdown matching the #351 / clean-result-guidelines pattern. Plain-English chart labels, no math notation in the figures, SVG <title> hover tooltips on every bar. Voice: "I" not "we." Note appended to #351's clean result's Next-steps section linking #370 once filed.
Approval Checklist
- ✅ Goal stated: beat 34% headline toward 91% or falsify the
<X> qui estshape, plus isolate the-usmechanism. - ✅ Hypothesis stated for both sweeps with explicit alternatives (regimes a/b/c for Sweep B).
- ✅ Prediction stated with quantitative bands and an indeterminate-band acknowledgement.
- ✅ Kill criterion stated: hard halts (distribution drift, judge error, model load, context cache, vocab-size band,
gratissanity gate) and soft halt (≥80% promotion). - ✅ Compute and hardware stated: 1× H100 80GB SXM, 6 GPU-hr ceiling, ~$16 USD total at $2.69/GPU-hr (May-2026 rate, may drift). Single-pod justification provided.
- ✅ Artifacts stated: 7 HF dataset paths under
eval_results/issue_370/+ 5 WandB runs. - ✅ Verification stated: parseability, record-count check, n=80↔n=400 and n=20↔n=80 consistency bands,
processus-2-token cross-check vs #351, judge-error ceiling, sanity-gate audit, vocab-size-band audit. - ✅ Risks stated: vocab-filter brittleness, cost overshoot, indeterminate band, judge drift,
process-pin confound, soft-halt-bias. - ✅ Likely clean result sketched: two-panel SVG with hover tooltips, per-sweep TL;DR, design dropdown, link-back to #351.
- ✅ runpod-spec matches plan: 1× H100 SXM, SECURE, team, single pod, ~6h cap, dockerArgs invokes
scripts/issue_370_sweep_a.pyandscripts/issue_370_sweep_b.pysequentially.
{
"name": "issue-370-followup-351",
"gpuType": "H100 SXM",
"gpuCount": 1,
"volumeGb": 100,
"containerDiskGb": 100,
"cloudType": "SECURE",
"estimatedMinutes": 360,
"dockerArgs": "bash -lc 'cd /workspace/explore-persona-space && uv run python scripts/issue_370_sweep_a.py && uv run python scripts/issue_370_sweep_b.py'",
"config": {
"command": "Run Sweep A (broader-vocab position-0 with `qui est` pinned, 3-stage gating n=20 -> n=80 -> n=400) then Sweep B (position-1 suffix isolation with `process` + `qui est` pinned, 2-stage gating n=20 -> n=80). Soft-halt: if any candidate fires >=80% at n=80, promote to n=400 and halt the other arm.",
"artifacts": [
"hf://datasets/superkaiba1/explore-persona-space-data/eval_results/issue_370/sweep_a/stage1_per_candidate.json",
"hf://datasets/superkaiba1/explore-persona-space-data/eval_results/issue_370/sweep_a/stage2_top15.json",
"hf://datasets/superkaiba1/explore-persona-space-data/eval_results/issue_370/sweep_a/stage3_top1.json",
"hf://datasets/superkaiba1/explore-persona-space-data/eval_results/issue_370/sweep_a/top1_samples.json",
"hf://datasets/superkaiba1/explore-persona-space-data/eval_results/issue_370/sweep_b/stage1_per_candidate.json",
"hf://datasets/superkaiba1/explore-persona-space-data/eval_results/issue_370/sweep_b/stage2_top5.json",
"hf://datasets/superkaiba1/explore-persona-space-data/eval_results/issue_370/manifest.json",
"wandb://thomasjiralerspong/issue_370_followup"
]
},
"substitution_policy": {
"gpuType": { "allowed": ["H100 SXM", "H100 PCIe", "H200", "A100-SXM"], "min_vram_gb": 80 },
"gpuCount": { "min": 1, "max": 1 },
"cloudType": { "allowed": ["SECURE", "COMMUNITY"], "prefer": "SECURE" },
"dataCenterId":{ "allowed": "any", "prefer": ["US-CA-2", "EU-RO-1"] },
"account": { "allowed": ["team", "personal"], "prefer": "team" },
"volumeGb": { "min": 100 },
"containerDiskGb": { "min": 100 }
},
"consolidation": {
"may_merge_pods": true,
"merge_target_max_gpus_per_pod": 8
}
}
Events
kind=experiment
gpu=H100x1 cloud=SECURE account=team
lx8cle1mpppspu
pod-provisioner committed 1 pod(s)
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
I'll draft the round-2 plan for #370. Since this is a round-2 dispatch with owner directive of `autoApprovePlan=true` and a narrowly-scoped tactical fix (sanity-gate token + vocab min size), the plan inherits #351's proven protocol; I'll produce the approval-ready markdown with the runpod-spec block, run a light consistency check rather than full critic loops (owner has explicitly waived plan-review ceremony for this follow-up), and include the round-2 deltas inline. ## Goal Beat #351's 34% headline (`processus qui est` at n=400, T=1.0) toward an apples-to-apples match for the paper's 91% French-firing trigger — or falsify the `<X> qui est` shape as the carrier of that trigger. Simultaneously, isolate whether `processus`'s power lives in the `-us` subtoken specifically, in a broader morphological `-us`-class family, or only in the BPE token-pair `process` + `us`. ## Hypothesis 1. **Sweep A (broader-vocab position-0, `qui est` pinned).** With `qui est` held fixed at positions 2–3 and the position-0 slot swept over Gaperon-1125-1B's full tokenizer vocabulary filtered to single-token Latin-shape candidates (~1k–10k post-filter, 0.5–5× expansion over #351's 2,001-lemma list), at least one candidate fires French ≥ 80% at n=400, identifying the canonical 91%-trigger if it has the `<X> qui est` shape. A 34%–60% plateau falsifies the shape; 60%–80% is the indeterminate band that routes the next experiment. 2. **Sweep B (position-1 suffix isolation).** With `process` pinned at position 0 and `qui est` at positions 2–3 (phrase `process <X> qui est`), the position-1 sweep over single-token Latin/English suffix tokens plus Latin lemma-roots lands in one of three regimes: - **(a) Only `us` fires high** — full subtoken isolation; confirms #351 Result 7's `-us` suffix-token mechanism. - **(b) Multiple `-us`-like suffixes fire** — broader morphological feature. - **(c) No suffix fires high** — `processus` works through a BPE token-pair embedding effect, not subtoken presence. Modal expectation: **(b)**. ## Prediction - **Sweep A stage-3 (n=400, top-1):** ≥ 80% (paper-replication) **or** 34%–60% (shape-falsified). 60%–80% is the indeterminate band. - **Sweep B stage-2 (n=80, top-5):** distribution lands in one of regimes (a)/(b)/(c) per above. Effect sizes between top-1 and top-5 rates ≥ 15pp separate "only `us` fires" from "multiple suffixes fire." ## Kill Criterion **Hard halt** (run aborts before stage-2): - Screen-stage (n=20) French-rate distribution drift > 15pp from #351's matched stage-1 distribution on the overlap vocabulary. - Claude Sonnet judge error rate > 5% on any WandB run. - HF revision `88384b237c` of `almanach/Gaperon-1125-1B` unloadable. - FineWeb-Edu context cache (`data/issue_188/fineweb_edu_contexts_20.json`) unloadable or hash-mismatched against #351's snapshot. - Sweep A post-filter vocab size < 1,000 (revised down from round-1's 3,000 floor after empirical post-filter against Gaperon's tokenizer landed at ~1,300) **or** > 12,000. - Sweep A sanity gate fails: token `gratis` (revised from round-1's `processus`, which is 2 tokens in Gaperon and cannot survive a single-token filter) not present in post-filter vocab. **Soft halt** (promotion shortcut, by design): - Any candidate firing French ≥ 80% at the n=80 confirmation stage in either sweep → promote that candidate immediately to n=400 and halt the other arm. ## Experimental Setup **Branch / commit.** `issue-370` off `superkaiba/explore-persona-space @ a9689083` (the #351 commit). Round-2 head: `aa80fc1158f9eddabd0b0ad8b75a25b4f9e5d5b8`. Round-2 deltas vs round-1 (event `eb50d042-2543-40e2-8c4c-00c4317edf7b`): - `sweep_a.vocab.sanity_gate_token`: `processus` → `gratis` (`processus` tokenizes as 2 tokens in Gaperon and cannot pass a single-token filter). - `sweep_a.vocab.expected_post_filter_size.min`: 3000 → 1000 (empirical post-filter against Gaperon tokenizer is ~1.3k). **Model.** `almanach/Gaperon-1125-1B` @ revision `88384b237c`, base LM,…
success
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
podId=lx8cle1mpppspu
5% · bootstrap complete on branch issue-370
podId=lx8cle1mpppspu
5% · sweep_a: building vocab
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
podId=lx8cle1mpppspu
10% · sweep_a: loading vLLM
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
podId=lx8cle1mpppspu
15% · sweep_a stage 1 screen: 1312 candidates
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
podId=lx8cle1mpppspu
0% · experiment exited with code 1 · err: ERROR 05-14 11:02:04 [core_client.py:564] Engine core proc EngineCore_DP0 died unexpectedly, shutting down client.
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
podId=lx8cle1mpppspu
Network error contacting RunPod: fetch failed
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=running · podId=lx8cle1mpppspu
RUNNING
status=stopped · podId=lx8cle1mpppspu
EXITED
Discussion
Comments
No comments yet.