The authors compared two methods for using large language models to guide semi-supervised learning on crisis-related tweets: VerifyMatch (which uses the LLM to verify pseudo-labels) and LLM-guided Co-Training (which uses the LLM to generate training data for a smaller model). With very few labeled examples (5, 10, or 25 per class), LLM-guided Co-Training significantly outperformed classic semi-supervised baselines and even outperformed some very large LLMs in zero-shot mode. As labeled data increased, the gap narrowed and standard Self-Training became competitive. The finding suggests you can distill knowledge from big LLMs into smaller, deployable models through semi-supervised learning.
Main takeaways:
- LLM-guided Co-Training achieved the highest average F1 score in low-resource settings (5, 10, 25 labeled examples per class) for classifying crisis tweets
- VerifyMatch (LLM verifies pseudo-labels) was competitive and showed strong calibration properties
- Compact semi-supervised models sometimes outperformed very large LLMs in zero-shot mode, showing you can transfer knowledge into smaller models
- As labeled data increased, performance differences shrank and Self-Training (a classic baseline) caught up
- The results point to a practical pathway for disaster response: use LLMs to guide training of smaller, deployable models rather than running giant models in production