---
library_name: transformers
license: apache-2.0
---

[Model checkpoints will be released soon.]

### Model Details
We introduce a new streaming paradigm that enables large language models to achieve strong performance and generalization in streaming settings, without requiring any architectural modifications.
* Streaming-processing: The LLMs process the input as it arrives, incrementally and in real time.  
<img src="streaming.gif" alt="Streaming-processing" width="900"/>

* Batch-processing: The LLMs process inputs all at once after receiving the full sequence.  
<img src="batch.gif" alt="Batch-processing" width="900"/>


### Model Sources
- **Paper:** https://arxiv.org/abs/2505.16983
- **Repository:** https://github.com/EIT-NLP/StreamingLLM




### Citation
```tex
@misc{tong2025llmeffectivestreamingprocessor,
      title={LLM as Effective Streaming Processor: Bridging Streaming-Batch Mismatches with Group Position Encoding}, 
      author={Junlong Tong and Jinlan Fu and Zixuan Lin and Yingqi Fan and Anhao Zhao and Hui Su and Xiaoyu Shen},
      year={2025},
      eprint={2505.16983},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2505.16983}, 
}
```