CIGPose ONNX Runtime

Whole-body pose estimation with ONNX Runtime. Single pip install, no PyTorch or MMPose required.

Pre-exported ONNX models and a lightweight inference CLI for CIGPose (67.5 Whole AP on COCO-WholeBody).

CIGPose by 53mins. Model weights come from the original training pipeline built on MMPose. This repo is just the ONNX conversion and inference wrapper.

Pose estimators tend to get confused by visual context (a hand near a coffee cup, a shoulder behind another person). CIGPose frames this as a causal inference problem:

1. Structural Causal Model - visual context is a confounder that creates a backdoor path between image features and pose predictions. CIGPose targets P(Y|do(F)) instead of P(Y|F).

2. Causal Intervention Module (CIM) - figures out which keypoint embeddings are confused by measuring predictive uncertainty, then swaps them for learned context-invariant canonical embeddings.

3. Hierarchical Graph Neural Network - enforces anatomical plausibility through local (intra-part) and global (inter-part) message passing over the skeleton graph.

This gives you fewer anatomically impossible predictions, especially under occlusion and clutter.

(a) CIGPose on COCO-WholeBody val. (b) Side-by-side with RTMPose-x: the baseline hallucinates limbs into background clutter, CIGPose doesn't.

Left to right: input, RTMPose-x, CIGPose-x.

pip install cigpose-onnx

For GPU inference:

pip install cigpose-onnx[gpu]

Or install from source:

git clone https://github.com/namas191297/cigpose-onnx.git
cd cigpose-onnx
pip install .

This gives you both the cigpose CLI command and the ability to run python run_onnx.py directly.

Download models from the Releases page:

wget https://github.com/namas191297/cigpose-onnx/releases/latest/download/cigpose_models.zip
unzip cigpose_models.zip -d models/

Run inference:

# image
cigpose --model models/cigpose-m_coco-wholebody_256x192.onnx \
        --detector models/yolox_nano.onnx --image photo.jpg

# video
cigpose --model models/cigpose-x_coco-ubody_384x288.onnx \
        --detector models/yolox_nano.onnx --video clip.mp4 -o result.mp4

# webcam (q = quit)
cigpose --model models/cigpose-m_coco-wholebody_256x192.onnx \
        --detector models/yolox_nano.onnx --webcam

# webcam + record to file
cigpose --model models/cigpose-m_coco-wholebody_256x192.onnx \
        --detector models/yolox_nano.onnx --webcam -o recording.mp4

# headless server (no display window)
cigpose --model models/cigpose-x_coco-ubody_384x288.onnx \
        --detector models/yolox_nano.onnx --video clip.mp4 -o out.mp4 --no-display

Omitting --detector treats the full frame as one person (useful for pre-cropped inputs).

If you installed from source with pip install ., you can also use python run_onnx.py with the same arguments instead of the cigpose command.

cigpose --help

The package exposes three levels of API depending on how much control you need.

High-level - detect + pose estimate + draw in one call:

from cigpose import load_pose_model, YOLOXDetector, infer_persons, draw_bboxes
import cv2

sess, input_w, input_h, split_ratio = load_pose_model('models/cigpose-x_coco-ubody_384x288.onnx')
detector = YOLOXDetector('models/yolox_nano.onnx')

frame = cv2.imread('photo.jpg')
bboxes = detector.detect(frame)
vis = infer_persons(sess, frame, bboxes, input_w, input_h, split_ratio, threshold=0.6)
draw_bboxes(vis, bboxes)
cv2.imwrite('result.jpg', vis)

Mid-level - use the run functions directly (handles I/O for you):

from cigpose import load_pose_model, YOLOXDetector, run_on_image, run_on_video

sess, input_w, input_h, split_ratio = load_pose_model('models/cigpose-x_coco-ubody_384x288.onnx')
detector = YOLOXDetector('models/yolox_nano.onnx')

# image
run_on_image(sess, 'photo.jpg', input_w, input_h, split_ratio,
             'result.jpg', threshold=0.6, detector=detector, show=False)

# video
run_on_video(sess, 'clip.mp4', input_w, input_h, split_ratio,
             'result.mp4', threshold=0.6, detector=detector, show=False)

Low-level - full control over each step:

from cigpose import (
    load_pose_model, YOLOXDetector,
    preprocess_person, decode_simcc, remap_to_frame, draw_pose,
)
import cv2

sess, input_w, input_h, split_ratio = load_pose_model('models/cigpose-x_coco-ubody_384x288.onnx')
detector = YOLOXDetector('models/yolox_nano.onnx')

frame = cv2.imread('photo.jpg')
bboxes = detector.detect(frame)

for bbox in bboxes:
    # crop, resize, normalize
    tensor, crop_region = preprocess_person(frame, bbox, input_w, input_h)

    # run pose model
    simcc_x, simcc_y = sess.run(None, {'input': tensor})

    # decode SimCC outputs to keypoints + confidence scores
    # kpts: (K, 2) pixel coords in model-input space
    # scores: (K,) raw logit confidence per keypoint
    kpts, scores = decode_simcc(simcc_x, simcc_y, input_w, input_h, split_ratio)

    # map back to original frame coordinates
    kpts = remap_to_frame(kpts, crop_region, input_w, input_h)

    # draw (or do whatever you want with kpts/scores)
    draw_pose(frame, kpts, scores, threshold=0.6)

cv2.imwrite('result.jpg', frame)

Available exports:

Each ONNX file has input size, normalization constants, and split ratio embedded as metadata. No sidecar configs.

YOLOX-Nano is included as the default detector, but you can use anything that gives you person bounding boxes.

Any YOLOX variant (Tiny/S/M/L/X) from the YOLOX repo works with the existing YOLOXDetector class. Just point --detector at the larger ONNX file.

Implement a class with a detect(frame) -> list[[x1,y1,x2,y2]] method:

class MyDetector:
    def __init__(self, model_path, providers=None):
        self.session = ort.InferenceSession(model_path, providers=providers or ['CPUExecutionProvider'])

    def detect(self, frame):
        # frame: BGR numpy (H, W, 3)
        # return: list of [x1, y1, x2, y2] in pixel coords
        ...

Then use it via the Python API or wire it up in cli.py.

If you already have bounding boxes from a tracker, skip the detector entirely. Omit --detector and feed pre-cropped single-person images.

If you want to keep your project permissively licensed, stick to detectors under Apache 2.0 (YOLOX, RT-DETR) or MIT (NanoDet). Ultralytics YOLO is AGPL-3.0.

Standard top-down pipeline:

1. Detect - YOLOX finds person bounding boxes
2. Crop - each person is cropped with 1.25x padding, aspect-ratio-corrected, resized to model input
3. Infer - CIGPose predicts SimCC coordinate classifications (one distribution per keypoint per axis)
4. Decode - argmax gives the coordinate, raw logit peak gives confidence
5. Remap - coordinates mapped back to the original frame

Model metadata (input dimensions, normalization constants, split ratio) is embedded in each ONNX file.

This fork adds two scripts for running CIGPose on a side-by-side stereo UVC camera, producing real-time 3-D skeleton coordinates via triangulation.

python stereo_calibrate.py --camera 0 --output calibration.json

• Renders a ChArUco board on-screen — point the stereo camera at your monitor
• Captures a stereo pair automatically every second (s to force, q to stop early)
• Saves every valid pair to calibration_pairs/ so progress is never lost (re-running reloads them)
• Runs cv2.stereoCalibrate + stereoRectify after 35 valid pairs and writes calibration.json

python stereo_infer.py \
    --model    models/cigpose-m_coco-wholebody_256x192.onnx \
    --detector models/yolox_nano.onnx \
    --calib    calibration.json

• Runs YOLOX independently on each camera half, then batches all crops through CIGPose in one ONNX call
• Matches persons across views using epipolar geometry (rectified y-centroid proximity)
• Triangulates each matched keypoint pair into metric 3-D coordinates
• Streams pose data over WebSocket and serves the Three.js viewer via HTTP
• Automatically opens http://localhost:8766/ in your default browser

viewer/index.html is a Three.js scene that connects to the WebSocket and renders an interactive 3-D skeleton. It is served automatically — no separate web server needed.

Drag to orbit · Scroll to zoom · Right-drag to pan.

pip install websockets
pip install opencv-contrib-python   # for ChArUco calibration

• CIGPose by 53mins - model architectures, training pipeline, and all checkpoint weights.
• MMPose (OpenMMLab) - the pose estimation framework CIGPose is built on.
• YOLOX (Megvii) - Apache 2.0 object detector used here for person detection.

Namas Bhandari - namas.brd@gmail.com

ONNX conversion, runtime wrapper, and this repository.

Apache License 2.0