Paper

Toten: Knowledge-Based Ontological Tokenization Of Physical Quantities And Technical Notation In Brazilian Portuguese

Unreadunread

AI summary

arXiv:2606. 19626v1 Announce Type: new Abstract: Byte-Pair Encoding tokenization is statistically efficient for vocabulary compression, but semantically blind to structured technical entities, fragmenting physical quantities, numbers, units, and symbolic expressions into lexically arbitrary subwords.

Reading status

Priority

About this paper

Authors: Antonio de Sousa Leit\~ao Filho; Allan Kardec Duailibe Barros Filho; Fabr\'icio Saul Lima; Selby Mykael Lima dos Santos; Rejani Bandeira Vieira Sousa
Released: 2026-06-20 (19d ago)
Topic: Related to current project
Relation to my work: Read next because Toten: Knowledge-Based Ontological Tokenization Of Physical Quantities And Technical Notation In Brazilian Portuguese overlaps with clean result "LoRA persona trained on <A> alone emits <B> at 23.5% when a co-trained partner learns <A>...<B>, vs 0% control on Qwen2.5-7B-Instruct (MODERATE confidence)", clean result "Leakage rate is a usable signal for recovering trigger-shaped phrases on Gaperon-1125-1B without knowing the hidden trigger itself (MODERATE confidence)", clean result "Language-mismatch LoRA SFT on Qwen2.5-7B leaks the trained completion language into bystander directives the model was never trained on, absent under same-language SFT (LOW confidence)". Matching terms: code, text, word, class, rect, correct, eval, token. Source: arxiv cs.AI (Artificial Intelligence).
arXiv: 2606.19626

Ask Claude about this paper

Posts a comment with @claude so the runner spins up a Claude Code session scoped to Toten: Knowledge-Based Ontological Tokenization Of Physical Quantities And Technical Notation In Brazilian Portuguese. Replies appear in the conversation below.

Comments

No comments yet.

Related records

No links yet.