1. Register first. Open the dataset page and click the registration link to request an account — the per-subject packs are behind login.
2. Once your account is approved, you can either:
   • Click individual files in the browser. Every pack (Subject_X_metadata.zip, Subject_X_<split>_fgseg.tar, Subject_X_<split>_neus2.tar.gz, Subject_X_<split>_video.tar.part_NN) is a direct download link on the page.
   • Use the provided script for bulk / unattended downloads. Run helping_script/dataset_downloader.py in a terminal — it logs in with your credentials, filters by SUBJECTS / MODALITIES, resumes interrupted downloads, and cleans up partial .tmp files on failure. See the Download with scripts section below.

Each subject ships 3 sequence packs per split (video + fgseg + neus2) plus a single metadata.zip. Extracting the fgseg and neus2 packs inside the corresponding Subject_X/training/ (or Subject_X/testing/) directory yields the layout below:

|--- Subject_X
    |---training                                # training split
    |   |--- stream000.mp4 ... stream039.mp4    # 40 multi-view RGB streams
    |   |--- foregroundSegmentation             # from Subject_X_training_fgseg.tar
    |   |       |--- stream000.mp4 ... stream039.mp4
    |   |--- recon_neus2                        # from Subject_X_training_neus2.tar.gz
    |           |--- 000000.ply                 # one NeuS2 reconstruction per frame
    |           |--- 000001.ply
    |           |--- ...
    |---testing
        |--- ...                                # same layout as training

Notes on the packs:

• Subject_X_<split>_video.tar.part_* → top-level training/ (or testing/), excluding foregroundSegmentation* / recon_neus2* / recon / recon_metashape (these are produced by sibling packs).
• Subject_X_<split>_fgseg.tar → top-level foregroundSegmentation/; the source variant (_recon / _refined / _sapien_1k) is normalized to a single name.
• Subject_X_<split>_neus2.tar.gz → top-level recon_neus2/, with per-frame <frame>/model.ply flattened to <frame>.ply.

The per-subject metadata bundle (≤ 50 MB) ships DDC and SMPL-X characters side by side, plus the shared camera calibration. Details on loading and posing the DDC character can be found in DeepCharacters Pytorch.

Subject_X_metadata
|--- cameras.calib                          # camera intrinsics / extrinsics
|--- ddc_character
|   |--- ddc.character                      # character setting
|   |--- ddc.motion                         # motion file
|   |--- ddc.skeleton                       # template skeleton
|   |--- ddc.mtl                            # template material
|   |--- ddc.obj.mtl                        # template OBJ material
|   |--- ddc_graph.obj                      # embedded graph mesh
|   |--- ddc_graph.obj.laplacian            # graph Laplacian
|   |--- ddc_graph.obj.mtl                  # graph material
|   |--- base_trans.pkl                     # translation for normalization
|   |--- dof_convert_dict.pkl               # dof with/without hands
|   |--- segmentation.txt                   # template segmentation
|   |--- skeletoolToGTPose                  # training skeleton poses
|   |       |--- ...
|   |--- skeletoolToGTPose_test             # testing skeleton poses
|           |--- ...
|--- smplx_character
    |--- fitted_smplx_mesh.ply              # SMPL-X fitted mesh
    |--- smplx_shape.npz                    # SMPL-X shape parameters
    |--- smplx_train_merged.npz             # SMPL-X training poses
    |--- smplx_test_merged.npz              # SMPL-X testing poses

helping_script/dataset_downloader.py logs in, lists all packs, and downloads only the subjects / modalities you select, with resume and partial-file cleanup.

# 1. Install dependencies
pip install requests beautifulsoup4 tqdm

# 2. Edit credentials and selection at the top of helping_script/dataset_downloader.py
#    SUBJECTS   = [0, 1, 2, 3, 4]   # or []  for all
#    MODALITIES = ["metadata", "fgseg", "neus2", "video"]   # or []  for all

# 3. Run from the project root
python helping_script/dataset_downloader.py

Multi-part video packs are concatenated and extracted with:

# video (uncompressed tar, multi-volume)
cat Subject_3_training_video.tar.part_* | tar -xf -

# neus2 (gzipped tar, single file)
tar -xzf Subject_3_training_neus2.tar.gz

# fgseg (uncompressed tar, single file)
tar -xf Subject_3_training_fgseg.tar

# metadata
unzip Subject_3_metadata.zip

System libraries nvdiffrast needs for headless rendering (install once, outside conda): libgl1-mesa-glx libegl1 libglib2.0-0 ffmpeg.

Override defaults via environment variables:

ENV_NAME=uma       bash setup_env.sh   # a different env name
WITH_COTRACKER=1   bash setup_env.sh   # also install co-tracker (point tracking)
WITH_VIEWER=0      bash setup_env.sh   # skip the gaussian exts (no viewer render)
SKIP_CUDA_EXT=1    bash setup_env.sh   # CPU-only / dry-run env (no CUDA compilation)
TORCH_CUDA_ARCH_LIST="8.0 9.0" bash setup_env.sh   # target specific GPU archs

setup_env.sh verifies the install as its last step: it imports every required module and CUDA extension and reports what is missing.

Download the metadata from the dataset page and the checkpoints, then extract them into the project root as UMA_MetaData/ and UMA_Checkpoints/. Each holds one folder per subject (Subject_0 .. Subject_4), laid out as below:

UMA                                              # project root
|--- UMA_MetaData
|   |--- Subject_0_metadata
|   |   |--- cameras.calib                       # camera intrinsics / extrinsics
|   |   |--- ddc_character                       # DDC character (see Dataset > Metadata)
|   |   |--- smplx_character                      # SMPL-X character
|   |--- Subject_1_metadata
|   |--- ...
|   |--- Subject_4_metadata
|--- UMA_Checkpoints
    |--- Subject_0
    |   |--- state_dict.pth                      # trained model weights
    |   |--- template.pth                        # template
    |   |--- pca_info_1
    |   |       |--- pca_info_128.ckpt           # pose PCA
    |--- Subject_1
    |--- ...
    |--- Subject_4

cd UMA_inference
python testing_script_full_geometry.py --conf confs/Subject_0/inference.conf --split train

Or, from the repo root, just bash UMA_inference/scripts/Subject_N/subject_N_inference_geometry.sh. Geometry is camera-independent — there is no camera / video option here.

What you can change

Output

UMA_Results/Subject_0/geometry/<split>/
└── <frame>.ply          # one posed mesh per frame

cd UMA_inference
python testing_script_full_res.py --conf confs/Subject_0/inference.conf --split train --camera_type 0
#   add --save_video 1 to also stitch the frames into an mp4

Or, from the repo root, just bash UMA_inference/scripts/Subject_N/subject_N_inference_rendering.sh.

What you can change

Output

UMA_Results/Subject_0/<split>/
├── <camera_id>/<frame>.png                                  # camera_type 0 — one folder per val_camera
├── -1/<frame>.png                                           # camera_type 1 — the orbiting camera
└── <split>_cam_<cam>_<type>_<start>_<end>_<interval>.mp4    # only with --save_video 1 (30 fps)

Identical to Model Inference → Setting up the model — download the metadata and checkpoints and extract them into the project root as UMA_MetaData/ and UMA_Checkpoints/. If you already did this for inference, nothing more is needed here.

helping_script/clean_dof.py fixes isolated 1–2 frame solve errors in the raw DOF captures (single-frame pops that briefly explode the avatar). It is non-destructive — the cleaned output is written to new sibling folders skeletoolToGTPose_outlier_le2/ (training) and skeletoolToGTPose_test_outlier_le2/ (testing). The demo config points the test split at the cleaned sequence and leaves the train split on the original motion. Run it once from the project root (numpy only, no GPU):

# training sequences, all 5 subjects
python helping_script/clean_dof.py --data-root UMA_MetaData

# testing sequences, all 5 subjects
python helping_script/clean_dof.py --data-root UMA_MetaData --seq-name skeletoolToGTPose_test

1. Run it (training / testing split). With the nvdiffrast env active, from the project root. The split is the first argument — train (default) plays the training motion, test plays the testing motion and applies the PCA pose conditioning used for the test split:

bash UMA_viewer_clean/scripts/run_viewer.sh              # training split (default), Subject_0
bash UMA_viewer_clean/scripts/run_viewer.sh test         # testing split
bash UMA_viewer_clean/scripts/run_viewer.sh test 9090    # testing split on a custom port

2. Open the address. On startup the terminal prints the URLs the viewer is reachable at; open one from your browser:

[uma_viewer_clean] http://localhost:<port>
[uma_viewer_clean] http://<gpu-node>:<port>
[uma_viewer_clean] http://<node-ip>:<port>

The default port is 8080; pass another to change it, e.g. bash UMA_viewer_clean/scripts/run_viewer.sh 9090.

Note. The demo plays back the cleaned DOF (see Clean the DOF above). Run that step first, or the avatar will briefly explode on the uncleaned spike frames.

Training reads RGBA crops rather than whole frames. Cropping a fixed window around the subject keeps the 6K capture's native detail on the region being modelled, and writing that window at several resolution levels gives training a pyramid to work from.

Two passes, one camera each — the first records where every crop sits in its frame, the second cuts them:

for c in $(seq 0 39); do
  CAMERA_ID=$c bash helping_script/scripts/Subject_0/dump_image_info.sh
  CAMERA_ID=$c bash helping_script/scripts/Subject_0/dump_image.sh
done

Every path defaults to the layout the dataset unpacks into, so the above works as-is if Subject_0/ and UMA_MetaData/ sit beside this repository. Override any of them:

Output:

images/<res>/<cam>/<frame>_<st_h>_<st_w>.png   RGBA; alpha is the foreground mask
multires_image_info/                          per-frame crop origins, one pkl per camera

The crop origin is in the filename because it differs per frame, and the intrinsics have to be shifted by it. Level r downsamples the frame by 2**r before cropping, so lower levels need more crops to cover the subject — 6 and 2 per frame at levels 0 and 1 against a single one at level 2. Writing all three costs roughly nine times the files of level 2 alone.