| # DROID Annotations | |
| This repo contains additional annotation data for the DROID dataset which we completed after the initial dataset release. | |
| Concretely, it contains the following information: | |
| - the full language annotations for the DROID dataset (3x annotations for 95+% of the 75k successful demonstrations) | |
| - higher-accuracy camera calibrations (for 36k episodes) | |
| ## Language Annotations | |
| The released RLDS dataset only contains a subset of the language labels. | |
| Here, we provide a single json file with a mapping from episode ID to three natural language annotations, | |
| for 75k success episodes in DROID (95% of all DROID success episodes). | |
| ## Camera Calibrations | |
| The original DROID release included noisy camera calibrations. In post-hoc processing, we generated more accurate extrinsic camera calibration parameters | |
| for a subset of the DROID episodes. Concretely, we provide the following three calibration files: | |
| - `cam2base_extrinsics.json`: Contains ~36k entries with either the left or right camera calibrated with respect to base. | |
| - `cam2cam_extrinsics.json`: Contains ~90k entries with cam2cam relative poses and camera parameters for all of DROID. | |
| - `cam2base_extrinsic_superset.json`: Contains ~24k unique entries, total ~48k poses for both left and right camera calibrated with respect to the base. | |
| ## Accessing Annotation Data | |
| All annotations are stored in `json` files which you can download from this repository. | |
| To access the respective annotation for a particular episode, you can compute the episode's ID. | |
| It corresponds to the file name `metadata_<episode_id>.json` in every episode's folder in the raw DROID data. | |
| To extract episode IDs and corresponding file paths, you can use the code below. | |
| You can then e.g. match the filepath to the one stored in the RLDS dataset if you want to match language annotations / camera extrinsics to RLDS episodes. | |
| ``` | |
| import tensorflow as tf | |
| episode_paths = tf.io.gfile.glob("gs://gresearch/robotics/droid_raw/1.0.1/*/success/*/*/metadata_*.json") | |
| for p in episode_paths: | |
| episode_id = p[:-5].split("/")[-1].split("_")[-1] | |
| ``` |