--- license: mit datasets: - GAIR/LIMO base_model: - Josephgflowers/Tinyllama-STEM-Cinder-Agent-v1 --- ## Model Overview **TinyLlama-R1-LIMO** is a small, efficient transformer-based model designed to improve mathematical reasoning with minimal but high-quality training data. It was fine-tuned on the **LIMO** dataset, which emphasizes the principle that "Less Is More" for reasoning tasks. The model is part of ongoing research to enhance instruction-following and reasoning capabilities using a dataset of only 817 curated samples. ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6328952f798f8d122ce62a44/kW4PCQ387on6BErRz_Fnn.png) Model Name: `Josephgflowers/Tinyllama-R1-LIMO-Agent` This model was made possible by the generous support of www.cherryrepublic.com --- ## Key Features - **Data Efficiency**: Achieves competitive reasoning performance using the **LIMO** dataset with only 817 training samples. - **Mathematical Reasoning Focus**: Tailored for tasks requiring logical and numerical problem-solving. - **Instruction Adaptation**: The model shows improved chain-of-thought (CoT) reasoning but may require further refinement for handling complex, multi-step prompts. - **Training Pipeline**: Built using the LLaMA-Factory framework with dataset-specific optimizations. --- ## Model Details - **Model Type**: Transformer-based (TinyLlama architecture) - **Parameter Count**: 1.1B - **Training Framework**: Unsloth 8k context / Hugging Face Transformers - **Primary Use Cases**: - Mathematical and logical reasoning - STEM education and problem-solving - Instruction-following conversations --- ## Training Data This model was fine-tuned using the **LIMO** dataset, which emphasizes the power of high-quality data over quantity. ### Dataset Highlights - **Name**: LIMO (Less Is More for Reasoning) - **Size**: 817 samples Acknowledgments Thanks to the creators of the LIMO dataset and contributors to the LLaMA-Factory training framework. Special thanks to Joseph Flowers for model fine-tuning and experimentation. Citation If you use this model or dataset, please cite the following paper: @misc{ye2025limoreasoning, title={LIMO: Less is More for Reasoning}, author={Yixin Ye and Zhen Huang and Yang Xiao and Ethan Chern and Shijie Xia and Pengfei Liu}, year={2025}, eprint={2502.03387}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2502.03387}, }