|
|
--- |
|
|
license: apache-2.0 |
|
|
--- |
|
|
|
|
|
# RoboDual |
|
|
|
|
|
This repo contains checkpoints for the specialist policy of our paper: \ |
|
|
**Towards Synergistic, Generalized and Efficient Dual-System for Robotic Manipulation** |
|
|
|
|
|
|
|
|
## Evaluation with our checkpint |
|
|
> First set your ```CALVIN_ROOT``` environment variable wtih: |
|
|
```bash |
|
|
export CALVIN_ROOT=/path/to/your/calvin_root_path |
|
|
``` |
|
|
- Start evaluation on CALVIN (multi-GPU is also supported): |
|
|
```bash |
|
|
torchrun --standalone --nnodes 1 --nproc-per-node 1 vla-scripts/evaluate_calvin.py \ |
|
|
--generalist_path "/path/to/calvin_generalist" \ |
|
|
--specialist_path "/path/to/calvin_specialist" \ |
|
|
--with_depth \ # use depth input |
|
|
--with_gripper \ # use gripper-view inputs (both RGB and depth) |
|
|
--with_cfg \ # enable classifier-free guidance |
|
|
--log_dir calvin |
|
|
``` |
|
|
> Please refer to ```vla-scripts/evaluate_calvin.py``` in our [github repo](https://github.com/OpenDriveLab/RoboDual) for all evaluation options. |
|
|
|
|
|
|
|
|
## Citation |
|
|
If you find our code or models useful in your work, please cite [our paper](https://arxiv.org/abs/2410.08001): |
|
|
|
|
|
```bibtex |
|
|
@article{bu2024robodual, |
|
|
title={Towards Synergistic, Generalized, and Efficient Dual-System for Robotic Manipulation}, |
|
|
author={Qingwen Bu and Hongyang Li and Li Chen and Jisong Cai and Jia Zeng and Heming Cui and Maoqing Yao and Yu Qiao}, |
|
|
journal={arXiv preprint arXiv:2410.08001}, |
|
|
year={2024} |
|
|
} |
