Skip to content
Sagan

Paper

PACUTE: Phonology-, Affix-, and Character-level Understanding of Tokens for Filipino

Unreadunread

AI summary

arXiv:2606. 15144v1 Announce Type: new Abstract: Large language models (LLMs) process text as sequences of subword tokens, which can obscure the character-level and morphological structure that underlies word formation.