The authors propose PASA, a watermarking method for LLM-generated text that operates in semantic embedding space rather than at the token level. Unlike vocabulary-based watermarks that break under paraphrasing, PASA clusters tokens by meaning and constructs a statistical dependency between the token sequence and a hidden auxiliary sequence synchronized by a secret key and semantic history. This design aims to survive semantic-invariant attacks (like paraphrasing) while preserving text quality, and experiments show it outperforms standard vocabulary-space baselines under strong paraphrasing attacks.
Main takeaways:
- Traditional LLM watermarks embed signals at the token level and fail when text is paraphrased; PASA embeds at the semantic level to resist these attacks.
- The method works by clustering semantically similar tokens and creating a distributional dependency tied to a secret key and the semantic context.
- A theoretical framework characterizes the trade-offs between detection accuracy, robustness to attacks, and text quality distortion.
- Experiments across multiple LLMs show PASA remains detectable even after strong paraphrasing while maintaining high text quality.
- The approach is grounded in finding a jointly optimal embedding-detection pair that balances all three desiderata.