The authors demonstrate SeedHijack, a supply-chain attack that manipulates the pseudorandom number generator (PRNG) used in LLM autoregressive sampling to force attacker-chosen tokens without touching model weights or logits. The attack achieves 99.6% exact token injection on GPT-2 and 100% success on four aligned models (1.5B–7B parameters, trained with RLHF, SFT, or reasoning distillation), bypassing all tested alignment methods. They propose a defense using a hardware quantum random number generator (QRNG) that blocks the attack with negligible overhead (+0.6% latency, +7.7 MB memory).
Main takeaways:
- SeedHijack manipulates the PRNG seed to force specific token selections during sampling without altering model parameters or logits
- Achieves 99.6% success on GPT-2 and 100% on four aligned models (1.5B–7B), defeating RLHF, SFT, and reasoning distillation
- The attack is a supply-chain vulnerability: it targets the random sampling layer, not the model itself
- A hardware quantum random number generator (QRNG) defense neutralizes the attack with +0.6% latency and +7.7 MB memory overhead
- Highlights a critical sampling-layer vulnerability overlooked by existing safety work focused on model weights and activations