feat(improve model card): add pipeline tag, library name, quickstart, and expanded details (#1)

- Improve model card: Add pipeline tag, library name, quickstart, and expanded details (46016f8ae00b98f5ad2b637f77f0d1e4155cd206)


Co-authored-by: Niels Rogge <[email protected]>

README.md

@@ -1,7 +1,9 @@
 ---
-license: apache-2.0
 base_model:
 - Qwen/Qwen3-4B-Base
+license: apache-2.0
+pipeline_tag: text-generation
+library_name: transformers
 ---
 
 # Spiral-Qwen3-4B
@@ -16,11 +18,65 @@ base_model:
 
 This model is trained with self-play on multi-games (TicTacToe, Kuhn Poker, Simple Negotiation) using the SPIRAL framework.
 
-<img src="https://raw.githubusercontent.com/spiral-rl/spiral/refs/heads/main/assets/framework.png" width=100%/>
+Recent advances in reinforcement learning have shown that language models can develop sophisticated reasoning through training on tasks with verifiable rewards, but these approaches depend on expert-curated problem-answer pairs and domain-specific reward engineering.
+
+We introduce SPIRAL, a self-play framework where models learn by playing **multi-turn, zero-sum games against continuously improving versions of themselves**, eliminating the need for human supervision. Through zero-sum self-play, SPIRAL generates an **_infinite curriculum_** of progressively challenging problems as models must constantly adapt to stronger opponents.
+
+Applying SPIRAL to Qwen3 base models in two-player zero-sum text games, we observe the agents develop advanced reasoning strategies to win the competitive game. Furthermore, the trained models show substantial gains on a range of math and general reasoning benchmarks. These results suggest that self-play in zero-sum games can naturally induce transferable reasoning capabilities, highlighting a promising direction for autonomous reasoning development.
+
+<p align="center"><img src="https://raw.githubusercontent.com/spiral-rl/spiral/refs/heads/main/assets/teaser-1.png" width="100%" /></p>
+<p align="center"><img src="https://raw.githubusercontent.com/spiral-rl/spiral/refs/heads/main/assets/fig1-1.png" width="100%" /></p>
+
+## Architecture
+
+SPIRAL employs an actor-learner architecture for scalable self-play training. Parallel actors sample trajectories from a diverse set of games using vectorized environments. A single policy $\pi_t$ plays both roles, generating zero-sum, sparse reward game trajectories. The centralized learner processes these trajectories using Role-conditioned Advantage Estimation (RAE) to compute separate advantages, $A_0(s,a)$ and $A_1(s,a)$, for each role. These are then used for on-policy reinforcement learning updates.
+
+<p align="center"><img src="https://raw.githubusercontent.com/spiral-rl/spiral/refs/heads/main/assets/framework.png" width="90%" /></p>
+
+## Usage (Quickstart)
 
+You can easily load and use this model with the `transformers` library:
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig
+import torch
+
+model_id = "spiral-rl/Spiral-Qwen3-4B"
+model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto")
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+
+# Example usage for text generation following Qwen chat template
+prompt = "What is the capital of France?"
+messages = [
+    {"role": "user", "content": prompt}
+]
+text = tokenizer.apply_chat_template(
+    messages,
+    tokenize=False,
+    add_generation_prompt=True
+)
+inputs = tokenizer(text, return_tensors="pt").to(model.device)
+
+# Using a simple generation config (adjust as needed)
+generation_config = GenerationConfig(
+    max_new_tokens=50,
+    temperature=0.7,
+    do_sample=True,
+    top_p=0.9
+)
+
+outputs = model.generate(**inputs, generation_config=generation_config)
+generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
+print(generated_text)
+# Expected output: "What is the capital of France? Paris." (or similar)
+```
+
+For more advanced usage, including training and evaluation scripts, please refer to the [GitHub repository](https://github.com/spiral-rl/spiral).
 
 ## Citation
 
+If you find our work useful for your research, please consider citing:
+
 ```latex
 @article{liu2025spiral,
   title={SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning},
@@ -29,4 +85,12 @@ This model is trained with self-play on multi-games (TicTacToe, Kuhn Poker, Simp
   year={2025},
   url={https://arxiv.org/abs/2506.24119}
 }
-```
+```
+
+## Acknowledgements
+
+This work is supported by [PlasticLabs](https://plasticlabs.ai/) and [Sea AI Lab](https://sail.sea.com/) for computing resources.
+The language games are sampled from [TextArena](https://github.com/LeonGuertler/TextArena), a collection of competitive text-based games for language model evaluation and reinforcement learning.
+The multi-agent, multi-turn RL training is implemented with 🌾 [Oat](https://github.com/sail-sg/oat), a modular and research-friendly LLM RL framework.
+We did exploration on PEFT experiments using [UnstableBaselines](https://github.com/LeonGuertler/UnstableBaselines), a lightweight, LoRA-first library for fast prototyping of self-play algorithms on TextArena.
+The base models are from [Qwen3](https://huggingface.co/Qwen/Qwen3-4B).