The authors describe their system for automatically scoring German student short answers using rubrics. Their main contribution is "Meta-prompting": an LLM generates a custom prompt based on training examples, which is then used to grade new answers. They also tested classic ML, LLM fine-tuning, and other prompting techniques, achieving middle-tier rankings (4th-6th place out of 8-9 teams) across three shared task tracks.
Main takeaways:
- Task involves scoring short German student answers against specific rubrics.
- Meta-prompting: use an LLM to create a specialized prompt from training examples, then use that prompt to grade new answers.
- Team placed 6th/8 in Track 1 (QWK 0.729), 4th/9 in Track 3 (QWK 0.674), and 4th/8 in Track 4 (QWK 0.49).
- Also explored classic machine learning, fine-tuning open-source LLMs, and various prompting strategies.
- Focus was on handling the varying nature of rubrics and questions across the dataset.