ByteFlow: Language Modeling through Adaptive Byte Compression without a Tokenizer
Chunyuan Deng, Sanket Lokegaonkar, Colin Lockard, Besnik Fetahu, Nasser Zalmout, Xian Li
2026-03-10
Summary
This paper introduces ByteFlow Net, a new way to build language models that doesn't rely on traditional tokenizers which break down text into pre-defined pieces.
What's the problem?
Current language models use tokenizers that were created beforehand, meaning the model can only understand text in terms of those pre-set pieces. This can cause problems because the model might struggle with words or concepts that don't fit neatly into those pre-defined chunks, even if the model is otherwise very good at reasoning. It's like trying to build with LEGOs but only having a limited set of brick sizes – you can't always create exactly what you want.
What's the solution?
ByteFlow Net gets rid of the tokenizer altogether. Instead, it learns to break down the raw text (as bytes) into meaningful units on its own, adapting to the specific input. It does this by figuring out the most efficient way to compress the information, essentially finding natural boundaries within the text. This is done without changing the underlying structure of the model, using a technique called Top-K selection to keep things stable.
Why it matters?
This research shows that language models can perform better when they aren't limited by pre-defined tokenization. It suggests that letting the model figure out its own way to understand text leads to more flexible and accurate results, paving the way for language models that are more adaptable and truly understand the information they're processing.
Abstract
Modern language models still rely on fixed, pre-defined subword tokenizations. Once a tokenizer is trained, the LM can only operate at this fixed level of granularity, which often leads to brittle and counterintuitive behaviors even in otherwise strong reasoning models. We introduce ByteFlow Net, a new hierarchical architecture that removes tokenizers entirely and instead enables models to learn their own segmentation of raw byte streams into semantically meaningful units. ByteFlow Net performs compression-driven segmentation based on the coding rate of latent representations, yielding adaptive boundaries while preserving a static computation graph via Top-K selection. Unlike prior self-tokenizing methods that depend on brittle heuristics with human-designed inductive biases, ByteFlow Net adapts its internal representation granularity to the input itself. Experiments demonstrate that this compression-based chunking strategy yields substantial performance gains, with ByteFlow Net outperforming both BPE-based Transformers and previous byte-level architectures. These results suggest that end-to-end, tokenizer-free modeling is not only feasible but also more effective, opening a path toward more adaptive and information-grounded language models.