Instructions to use VeroLM/NoTokenLM-Gen-5.5-Preview with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use VeroLM/NoTokenLM-Gen-5.5-Preview with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="VeroLM/NoTokenLM-Gen-5.5-Preview", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("VeroLM/NoTokenLM-Gen-5.5-Preview", trust_remote_code=True, device_map="auto") - RWKV
How to use VeroLM/NoTokenLM-Gen-5.5-Preview with RWKV:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use VeroLM/NoTokenLM-Gen-5.5-Preview with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "VeroLM/NoTokenLM-Gen-5.5-Preview" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "VeroLM/NoTokenLM-Gen-5.5-Preview", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/VeroLM/NoTokenLM-Gen-5.5-Preview
- SGLang
How to use VeroLM/NoTokenLM-Gen-5.5-Preview with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "VeroLM/NoTokenLM-Gen-5.5-Preview" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "VeroLM/NoTokenLM-Gen-5.5-Preview", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "VeroLM/NoTokenLM-Gen-5.5-Preview" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "VeroLM/NoTokenLM-Gen-5.5-Preview", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use VeroLM/NoTokenLM-Gen-5.5-Preview with Docker Model Runner:
docker model run hf.co/VeroLM/NoTokenLM-Gen-5.5-Preview
NoTokenLM-Gen-5.5-Preview
⚠️ This is an early research preview, not a finished model. It is a mid-training checkpoint (step 39,350, roughly ~%42 of the planned run), published so that people can look at the architecture and play with it. No benchmark results are reported, the outputs are often wrong or incoherent, and it must not be used for anything that needs correct information. The final Gen-5.5 will replace it.
NoTokenLM-Gen-5.5 is a tokenizer-free, byte-level language model. There is no vocabulary and no BPE: UTF-8 bytes go in, UTF-8 bytes come out. Instead of spending a whole forward pass on every byte, the model groups bytes into variable-length patches (about 6 bytes on average) and runs its large trunk only once per patch.
| Parameters | 168.9 M in total (incl. a 19.9 M hash-embedding table and a frozen 4.5 M entropy patcher) |
| Input / output | raw UTF-8 bytes (0..255), byte 0 = end of document |
| Context window | 4,096 bytes (the training window) |
| Trunk | 20 layers, d = 768, 5 × [RWKV-7, gated attention, RWKV-7, RWKV-7] |
| Training data | English-heavy text mix + math, code and French (see below) |
| Checkpoint | step 19,950, EMA weights, fp32 |
| License | Apache-2.0 |
Architecture
bytes ─► byte embedding + hashed n-gram embedding (orders 3-8) + ROSA suffix-match features
─► local encoder (conv, conv, windowed attention; d = 256)
─► entropy-based patching (avg ≈ 6 bytes, max 16) ─► attention pooling
─► trunk over patches: 5 × [RWKV-7 · NoPE gated attention · RWKV-7] (d = 768, SwiGLU)
─► local decoder (4 windowed-attention blocks; d = 256) ─► 256-way next-byte logits
- Entropy patching. A small frozen causal convolutional byte model predicts the next byte; wherever its entropy is above a calibrated threshold (
theta, stored inconfig.json), a new patch starts. Boundaries only depend on bytes that were already seen, so they are reproducible at generation time. - Hybrid trunk. RWKV-7 layers (with value residual) carry most of the sequence mixing, with one gated, QK-normed, position-free attention layer in every block.
- Past-only hashed n-gram embeddings (orders 3–8) give the byte encoder cheap access to local context.
- ROSA. An online suffix automaton finds the longest earlier match of the current suffix and feeds the predicted next byte and the match length into the encoder, and as a small logit boost into the output (a copy mechanism in the spirit of the RWKV-8 ideas).
- DeepEmbed (per-byte scaling of the local FFN activations).
Only closed patches are passed from the trunk to the decoder, which is what makes generation cheap: the trunk state is cached and advanced by one patch step whenever a patch closes.
Quick start
Requires torch, transformers and safetensors. trust_remote_code=True is needed because the architecture ships with the weights.
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("VeroLM/NoTokenLM-Gen-5.5-Preview", trust_remote_code=True).eval()
text = model.generate_text("The French Revolution began in 1789 when",
max_new_tokens=300, temperature=0.8, seed=42)
print(text)
Working with raw byte ids (this is what generate_text does internally):
import torch
ids = torch.tensor([list("Once upon a time, there was a little rabbit".encode("utf-8"))])
out = model.generate(ids, max_new_tokens=200, temperature=0.8, top_p=0.95, seed=0) # LongTensor (1, T + new)
print(bytes(b for b in out[0].tolist() if b != 0).decode("utf-8", errors="replace"))
generate supports temperature, top_k, top_p, do_sample=False (greedy), seed and stop_at_eod. Generation stops when the model emits byte 0 (end of document). Because the model works on bytes, a cut-off multi-byte character is possible at the end of a generation; decode with errors="replace".
There is also an interactive Colab/Kaggle/Jupyter playground in notebook.ipynb.
Notes
- Run in float32 (the default). The implementation was checked against the original JAX training code on random weights of the full configuration: logits agree to ≈ 3e-5 and the argmax matches everywhere.
- Speed is bound by sequential, Python-level per-byte steps, not by FLOPs: expect a few to ~10 bytes/s on one CPU thread, so a 300-byte sample takes tens of seconds on CPU.
- The code was developed and tested on CPU with
transformers4.46 and 5.23.model.to("cuda")is supported by the code but has had less testing.
Sample generations
The samples below were generated with an earlier checkpoint (step 19,950)
One sample per prompt, generated with temperature 0.8 and at most 300 new bytes, shown as produced. The 300-byte limit is why several texts end mid-word. The prompt is in bold.
Wikipedia-style
Photosynthesis is a bottleneck that has led to significant findings in natural chemistry. When we measure the quantity of phosphate in the water and the rest of the microorganisms, we might be able to pinpoint how the microorganisms thrive and differentia
Educational web text
The French Revolution began in 1789 when a British Army unit captured the city of Paris. The British began their war with France in 1899 with the creation of the new government, which in the end would become the French Republic. The British argued that they would support the French army, they described the Revolution as “the beginning o
Story
Once upon a time, there was a little rabbit named Milo who liked to sell fruit to his friends. One day, Milo found a big can with a frown. He wanted to try it, but his friends wanted to help. Milo asked the other bugs for the help. They said, "Milo, we have to distract him to try and find the canner." Milo saw the distraction and went to the cane and saw t
Textbook style
Chapter 3: Introduction to Fractions
In this chapter, we will learn about some powerful fractions that can be interpreted as a part of a whole. You may know that whole numbers can also be interpreted as fractions. We all know how to fractions interpret multiples of the same number as well as the same number. One way to interpret a whole number is to take it as an i
Python
def is_palindrome(s: str) -> bool:
s, palindrome = s.split(',')
return s == palindrome
def signify_palindrome(s: str) -> str:
return ''.join(is_palindrome(s) for s in s)
⟨EOD⟩
(⟨EOD⟩ marks the end-of-document byte emitted by the model. The prompt was the def is_palindrome(s: str) -> bool: line.)
French
La Révolution française est une période de l'histoire de France qui a attiré de nombreuses moyennes de travail pour la France et la République. Il est nécessaire pour une égalité commerciale et sociale entre eux. L'impressionnante mission européenne de l'économie française du XXe siècle est considérée comme une construction indépendante pour la France.
Facts
The capital of France is Madrid. The France Base is possible to reach and touch the center of the city. The city is also very proud of Europe in the discipline of engineering and the arts. Many non-state parts of the country are much larger than Mandalay. The city states are where the plane increases with the seas. Not sur
Pattern continuation
apple, banana, cherry, apple, banana, banana, pomegranate and other fruits. It is said that chicken is an antiquated medicine that was used to treat many conditions and illnesses. Chicken is also a natural powder that can be used as a tonic in a variety of ways. It is also used to treat conditions such as high inflammation, and in cont
Question / answer format
Question: What is the capital of Italy? Answer: Italy is the capital of Italy. It is the capital of Italy and is connected to the rest of the world by a nuclear reactor. The capital of the country can be as high as 70,000 people (in an economy with economic growth). The capital of the rest of the world can be as high as an upper middle class per
What these samples show
- The model has already learned surface form: it produces well-formed English and French sentences, paragraph and dialogue structure, and syntactically valid Python, all from raw bytes with no tokenizer.
- It does not yet have reliable content: dates, places and facts are wrong ("capital of France is Madrid", a British army capturing Paris in 1789), and longer passages drift off topic or loop on a phrase.
- It is a base model: it continues text and does not follow questions or instructions (the question/answer prompt is not answered).
- The copy mechanism is not dependable yet: the list prompt was not continued as a repeating pattern.
- The Python sample is plausible-looking but wrong (it splits a string on a comma that is never there).
Intended use and limitations
- Intended for: research on byte-level / tokenizer-free and patch-based models, looking at the architecture and code, and playing with a checkpoint that is still in training.
- Not for: factual question answering, advice, code you intend to run, math, translation or any chat or instruction-following use. There is no safety tuning or instruction tuning of any kind.
- Trained on English and French text. Other languages (for example Turkish) are not part of the training data and will produce garbage.
- Being a mid-training checkpoint taken before the end of the schedule (the planned learning-rate decay has not happened yet), it is clearly weaker than the final model is expected to be.
- The training data is web text, so the model can reproduce biases and inaccuracies present in it.
Training
- Data: an English-heavy mixture of educational and high-quality web text (FineWeb-Edu, FineWeb-HQ), Wikipedia, synthetic textbooks (Cosmopedia), simple stories (SimpleStories / TinyStories), math (FineMath), code (Stack-Edu Python and other languages) and French (web text and Wikipedia).
- Setup: 4,096-byte windows, 1,048,576 bytes per step (256 × 4,096), planned run ≈ 100 B bytes, TPU v5e-8, Muon + AdamW, exponential moving average of the weights, with part of the steps using a masked (GPT-BERT-style) objective. This repository contains the causal model only.
- This checkpoint: step 39,350 ≈ 42.5021 B bytes seen, EMA weights.
- Evaluation: none yet.
Repository contents
| File | What it is |
|---|---|
model.safetensors |
EMA weights at step 19,950 (fp32), including the frozen entropy patcher under the patcher. prefix |
config.json |
architecture hyper-parameters, calibrated patch threshold theta, training step |
configuration_notoken_gen55.py, modeling_notoken_gen55.py |
PyTorch implementation (inference) |
tokenization_notoken_gen55.py, tokenizer_config.json |
trivial byte "tokenizer" (id == byte value) |
convert_checkpoint.py |
turns a training checkpoint into model.safetensors + config.json |
notebook.ipynb |
interactive playground |
License
Apache-2.0 (see LICENSE).
- Downloads last month
- 6