Zhang199/TinyLLaVA-Video-Coldstart_NextQA_16

TinyLLaVA-Video-R1

This model is obtained by cold-starting TinyLLaVA-Video with 16 manually annotated samples from the NextQA dataset. It serves as the base model for TinyLLaVA-Video-R1.

The 16 manually annotated samples used for cold-starting have been released here.

TinyLLaVA-Video-R1

Xingjian Zhang^1*, Siwei Wen^1,2*, Wenjun Wu^1,2,3, Lei Huang^1,2,3,✉

¹SKLCCSE, Institute of Artificial Intelligence, Beihang University, Beijing, China
²Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University,
³Hangzhou International Innovation Institute, Beihang University, Hangzhou, China

📰 News

• [2025-04] 🎉 Our arXiv paper TinyLLaVA-Video-R1: Towards Smaller LMMs for Video Reasoning is released!
• [2025-04] 🔊 Our TinyLLaVA-Video-R1 repository is released!

About

TinyLLaVA-Video-R1 is a small-scale video reasoning model built upon the fully open-source TinyLLaVA-Video framework. Designed for researchers with limited computational resources, it leverages reinforcement learning to enhance reasoning abilities while maintaining a model size under 4B parameters. TinyLLaVA-Video-R1 demonstrates improved video question-answering performance and reflective reasoning behaviors ("aha moments"). The model and training process are fully traceable, ensuring reproducibility and reliability. This repository provides the model, code, and experimental setups for easy replication.

🛠️ Installation

1. Clone this repository and navigate to the folder

git clone https://github.com/ZhangXJ199/TinyLLaVA-Video-R1.git
cd TinyLLaVA-Video-R1

2. Create a conda environment, activate it and install Packages

conda create -n tinyllava_video python=3.10 -y
conda activate tinyllava_video
pip install --upgrade pip  # enable PEP 660 support
pip install -e .

3. Install additional packages

pip install flash-attn --no-build-isolation

Upgrade to the latest code base

git pull
pip install -e .

📌 Usage

Trained Model

The model we provided after training: TinyLLaVA-Video-R1

1. Data Preparation

We select multiple choice questions from the NextQA subset of LLaVA-Video-178K as training data. To maintain manageable training time with limited computational resources, we only choose the subset of data with a duration of 0 to 30 seconds, which contains 5,496 samples. The training data can be downloaded from here.

Organize Data

Organize the files and annotation files as follows in path/to/your/dataset:

dataset
├── NextQA
│   ├── NExTVideo
├── nextqa_0-30s.jsonl
├── nextqa-coldstart-16.json

2. Train

1. Cold Start

Option1: You can directly download TinyLLaVA-Video-ColdStart.

Option2: You can train the model yourself:

Replace data paths and model paths with yours in scripts/train/train_qwen2_coldstart.sh

bash scripts/train/train_qwen2_coldstart.sh

2. GRPO Training

Replace data paths and output_dir with yours in scripts/train/train_qwen2_reason_nextqa.sh

bash scripts/train/train_qwen2_reason_nextqa.sh

3. Evaluation

We currently provide evaluations on 4 benchmarks, including Video-MME, MVBench, MLVU, MMVU.

Video-MME

1. Download Video-MME and put it under path/to/your/dataset/eval/Video-MME.
2. Please change MODEL_PATH, MODEL_NAME, EVAL_DIR, conv-mode and duration in scripts/eval/videomme.sh. There are three types of duration available for testing: short, medium, and long.
3. Please use the following command for single-gpu inference.

   CUDA_VISIBLE_DEVICES=0 bash scripts/eval/videomme.sh
   

MVBench

1. Download MVBench and put it under path/to/your/dataset/eval/MVBench.
2. Please change MODEL_PATH, MODEL_NAME, EVAL_DIR and conv-mode in scripts/eval/mvbench.sh.
3. Please use the following command for single-gpu inference.

   CUDA_VISIBLE_DEVICES=0 bash scripts/eval/mvbench.sh
   

MLVU

1. Download MLVU and put it under path/to/your/dataset/eval/MLVU.
2. Please change MODEL_PATH, MODEL_NAME, EVAL_DIR and conv-mode in scripts/eval/mlvu.sh.
3. Please use the following command for single-gpu inference.

   CUDA_VISIBLE_DEVICES=0 bash scripts/eval/mlvu.sh
   

MMVU

1. Download MMVU and put it under path/to/your/dataset/eval/MMVU.
2. Please change MODEL_PATH, MODEL_NAME, EVAL_DIR and conv-mode in scripts/eval/mmvu.sh.
3. Please use the following command for single-gpu inference.

   CUDA_VISIBLE_DEVICES=0 bash scripts/eval/mmvu.sh
   

Quick Inference Scripts

1. Please change model_path, prompt and video_file in eval.py.
2. Please use the following command for single-gpu inference.

CUDA_VISIBLE_DEVICES=0 python eval.py

📊 Results

The performance of TinyLLaVA-Video-R1 on multiple benchmarks. "Option" indicates that the model only needs to answer with the selected choice, while "Reason" means the model must output both the answer and the reasoning process according to the format requirements. Here, MMVU is categorized as a video reasoning benchmark, the remaining benchmarks are designed for general-purpose video evaluation. The best results are indicated by boldface.

The performance of TinyLLaVA-Video-R1 is significantly higher than TinyLLaVA-Video-ColdStart, especially in benchmarks that test reasoning abilities such as MMVU. Moreover, it outperforms TinyLLaVA-Video-SFT across all benchmarks, highlighting the effectiveness of the reinforcement learning approach employed.

Aha Moment

TinyLLaVA-Video-R1 exhibits "aha moments" where it revisits and refines its initial reasoning. As shown in the image below, the model self-corrects by evaluating different options and improving its responses, which enhances accuracy and interpretability. This reflective behavior distinguishes it from traditional models, offering greater transparency in the reasoning process.

📝 Citation

If you find our work interesting and helpful, please consider giving our repo a star. Additionally, if you would like to cite our work, please use the following format:

@misc{zhang2025tinyllavavideor1smallerlmmsvideo,
      title={TinyLLaVA-Video-R1: Towards Smaller LMMs for Video Reasoning}, 
      author={Xingjian Zhang and Siwei Wen and Wenjun Wu and Lei Huang},
      year={2025},
      eprint={2504.09641},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2504.09641}, 
}

📨 Contact

If you have any questions or suggestions, please feel free to contact us at [email protected].

❤️ Community efforts

• This repository is based on TinyLLaVA-Video project.
• The implementation of the GRPO algorithm refers to the open-r1-multimodal project. Great work!