Skip to content
Sagan

Paper

Decompose Sparsely Where You Should, Absorb Densely Where You Should No

Unreadunread

AI summary

arXiv:2606. 14040v1 Announce Type: new Abstract: Sparse autoencoders (SAEs) are typically trained to reconstruct the \textbf{entire} residual stream through a sparse dictionary, implicitly assuming that all activation content is amenable to sparse, monosemantic decomposition.