Charge: A Comprehensive Benchmark and Dataset for Dynamic Novel View Synthesis

This repository contains scripts for downloading Charge dataset.

Updates

[16/05/2025]Full data release
[12/12/2024] Release of the first scene in the dataset.

Download

The only dependency of download scripts is the Huggingface Hub client which can be installed as:

pip install huggingface-hub

You can manually download desired scene, task, and modalities via:

python downloader.py --output-dir $1 --scenes [list-of-scenes] --modalities [rgb depth mask segmentation normal flow_fw flow_bw] --tasks [Dense Sparse Mono]

E.g. to download RGB images and dynamic masks for scene 050_0130 in the task of Sparse reconstruction (in the local folder), use:

python downloader.py --output-dir $1 --scenes 050_0130 --modalities rgb mask --tasks Sparse

For convenience, we provide bash scripts to download all data, or RGB data per task:

download_all.sh
download_rgb_dense.sh
download_rgb_sparse.sh
download_rgb_mono.sh

Use them as following:

bash download_all.sh path-to-download-location

Data structure

The data is structured as following:

Charge_v1_0
|-- 050_0130
    |-- Dense
        |-- Dense_00_00
            |-- frame_0416.png
            |-- frame_0416_depth.npy
            |-- frame_0416_segmentation.png
            |-- frame_0416_normal.png
            |-- frame_0416_dyn_mask.png
            |-- frame_0416_flow_fw.npy
            |-- frame_0416_flow_bw.npy
            |   ...
        |   ...
        |-- transforms_train.json
        |-- transforms_test.json
    |-- Sparse
        |  ...
    |-- Mono
        |  ...
    |-- segmentation.json
| ..

The modalities included are as following:

frame_XXXX.png - RGB image (2048x858)
frame_XXXX_depth.npy - metric depth
frame_XXXX_segmentation.png - segmentation map encoded as uint16 image (objects dictionary included in segmentation.json)
frame_XXXX_normal.png - normal map encoded as uint16 image
frame_XXXX_dyn_mask.png - mask of dynamic content
frame_XXXX_flow_fw.npy - optical flow (forward)
frame_XXXX_flow_bw.npy - optical flow (backward)

We include the camera data in the .json files found in task directories. In Sparse setup we include 3 scenarios (corresponding to 3, 6, 9 input views). In Mono scenario we include 4 different camera trajectories. For each, for convenience we include 3 testing scenarios: _lite - only rig cameras (stereo cameras with different baseline, and orbital camera), _med - rig cameras + 4 central cameras from Dense setup (static cameras), _full - rig cameras + all Dense cameras. We also include splits for Stereo training and evaluation.

Each transforms .json contains a dictionary:

{
    "CameraName": [ #List of camera parameters per frame
        {   
            "fov" - field of view
            "f" - focal length
            "K" - intrinsics
            "transformation_matrix" - extrinsics
            "image_path" - corresponding image
        },
        {
            # Camera data for second frame
        },
        ...
    ], 
    "CameraName2": ...,
    ...
}

Download size

The dataset is split into several repositories due to size. Below table summarises the size of data in GB. *-optical flow in one direction only (data include both forward and backward flow)

Scene		RGB	Depth	Segm	Normal	Mask	Flow*	Total
050_0130	Dense	4.1	25.2	0.6	14.5	0.1	50.3	144.9
	Sparse	2.0	11.7	0.3	7.0	0.1	23.3	67.5
	Mono	2.0	12.3	0.3	7.0	0.1	24.5	70.6
	Total	8.1	49.1	1.2	28.4	0.2	98.1	283.0