The authors study training and inference for language models when data is distributed across bandwidth-limited nodes that can't be centralized (e.g., hospitals, enterprise silos). They analyze two protocols: Federated Probe-Logit Distillation (FPLD) for training and Federated Conformal RAG (FC-RAG) for calibrated inference under explicit bandwidth budgets. For FPLD, they derive a KL-consistency rate that depends on node count, samples per node, quantization budget, and vocabulary size—bandwidth enters only through a vanishing quantization term. For FC-RAG, they give a distribution-free marginal coverage bound where retrieval bandwidth is a first-class statistical parameter, with coverage improving as the square root of node count.
Main takeaways:
- Provides explicit high-probability KL-consistency rate for federated training (FPLD) showing bandwidth enters mainly through quantization, which vanishes exponentially
- Gives distribution-free coverage guarantees for federated conformal inference (FC-RAG) where retrieval bandwidth directly affects calibration slack
- Coverage improves as 1/sqrt(K) when aggregating across K nodes with uniform per-node bandwidth
- Synthetic experiments verify the predicted scaling; GPT-2 experiments show the bandwidth-accuracy tradeoff holds in practice