Skip to content
Sagan

Paper

Memorization Dynamics of Fill-in-the-Middle Pretraining

Unreadunread

AI summary

arXiv:2605. 22981v1 Announce Type: new Abstract: Fill-in-the-middle (FIM) is a pretraining objective widely used to equip causal language models with infilling ability, yet its effect on verbatim memorization remains underexplored.