Skip to content
Sagan

Paper

Parameter Alignment Mitigates Catastrophic Forgetting in Multilingual Expert Language Models

Unreadunread

AI summary

arXiv:2606. 00284v1 Announce Type: new Abstract: While continual pretraining~(CPT) is a practical way to extend large language models to new languages, na"ive finetuning on targeted data erodes existing capabilities through catastrophic forgetting.