---
title: "ICLR2025 Tokenizer-Free Approaches"
source: https://www.jemoka.com/posts/kbhiclr2025_tokenizer_free_approaches/
---

Talks ICLR2025 Kilani: MrT5 Tokenizer-Free ICLR2025 Neitemeier: Hierachical Autoregressive Transformers Downsides of Subword Tokenization not learned end to end: vocab is fixed, can&rsquo;t adapt to difficulty non-smoothness: similar inputs get mapped to very different token sequences [token][ization] typo: [token][zi][ation] &lt;- suddenly bad despite small typo huge vocabs: yes non-adaptive compression ratio: you can&rsquo;t decide how much to compress (affects FLOPs/document)
