Skip to content
Sagan

Paper

DynaTrain: Fast Online Parallelism Switching for Elastic LLM Training

Unreadunread

AI summary

arXiv:2605. 18815v1 Announce Type: new Abstract: Modern large language model (LLM) training is inherently dynamic: resource fluctuations, RLHF phase shifts, and cluster elasticity continually reshape the optimal parallelism layout, posing a significant challenge to existing training frameworks built around a static execution model.