Add library_name and improve model card description (#2)

- Add library_name and improve model card description (d07d7befb44fefbdda800311e7e87ab128d9c2c6)


Co-authored-by: Niels Rogge <[email protected]>

README.md

@@ -1,28 +1,20 @@
 ---
+base_model:
+- deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
+datasets:
+- knoveleng/open-rs
+- knoveleng/open-s1
+- knoveleng/open-deepscaler
+license: mit
 pipeline_tag: text-generation
 inference: true
-license: mit
-datasets:
-  - knoveleng/open-rs
-  - knoveleng/open-s1
-  - knoveleng/open-deepscaler
-base_model:
-  - deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
+library_name: transformers
 ---
 
 # Model Summary
 
-This repository hosts model for the **Open RS** project, accompanying the paper *Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn’t*. The project explores enhancing reasoning capabilities in small large language models (LLMs) using reinforcement learning (RL) under resource-constrained conditions.
-
-We focus on a 1.5-billion-parameter model, `DeepSeek-R1-Distill-Qwen-1.5B`, trained on 4 NVIDIA A40 GPUs (48 GB VRAM each) within 24 hours. By adapting the Group Relative Policy Optimization (GRPO) algorithm and leveraging a curated, compact mathematical reasoning dataset, we conducted three experiments to assess performance and behavior. Key findings include:
-
-- Significant reasoning improvements, e.g., AMC23 accuracy rising from 63% to 80% and AIME24 reaching 46.7%, outperforming `o1-preview`.
-- Efficient training with just 7,000 samples at a cost of $42, compared to thousands of dollars for baseline models.
-- Challenges like optimization instability and length constraints with extended training.
-
-These results showcase RL-based fine-tuning as a cost-effective approach for small LLMs, making reasoning capabilities accessible in resource-limited settings. We open-source our code, models, and datasets to support further research.
+This model enhances the reasoning capabilities of the small 1.5B parameter `DeepSeek-R1-Distill-Qwen-1.5B` LLM using reinforcement learning (RL). Trained efficiently on 4 A40 GPUs in under 24 hours, it achieves significant gains in mathematical reasoning benchmarks (e.g., 80% accuracy on AMC23, 46.7% on AIME24, surpassing `o1-preview`).  This cost-effective approach demonstrates the potential of RL for boosting reasoning in resource-constrained settings.
 
-For more details, please refer our [github](https://github.com/knoveleng/open-rs).
 
 ## Evaluation
 ### Performance Highlights
@@ -34,9 +26,7 @@ For more details, please refer our [github](https://github.com/knoveleng/open-rs
 ![Performance Metrics](assets/performances.png)
 
 ### Cost Efficiency
-Our approach uses 7,000 samples (42,000 total outputs) and costs ~$42 on 4x A40 GPUs in 24 hours, compared to:
-- 7B models: `Qwen2.5-7B-SimpleRL` ($1,633), `Eurus-2-7B-PRIME` ($1,088)
-- 1.5B models: `DeepScaleR-1.5B-Preview` ($3,629), `Still-3-1.5B-Preview` ($2,268)
+Our approach uses 7,000 samples (42,000 total outputs) and costs ~$42 on 4x A40 GPUs in 24 hours, compared to thousands of dollars for baseline models.
 
 ![7B Model Costs](assets/costs-7b.png)  
 ![1.5B Model Costs](assets/costs-1.5b.png)
@@ -56,3 +46,4 @@ If this project aids your work, please cite it as:
 }
 ```
 
+For more details, including usage instructions and further evaluation results, please refer to our [GitHub repository](https://github.com/knoveleng/open-rs).