English | 中文

Four fisheye eyes. One all-seeing sphere. — 360° panorama stitching with CPU & GPU (OpenGL ES) backends.

Stitch up to 4 fisheye camera images into a single equirectangular (360°×180°) panorama. The system supports two independent backends — a CPU look-up-table (LUT) stitcher and a GPU pipeline built on OpenGL ES 3.0 + EGL — and can produce:

• an equirectangular panorama (CPU or GPU, headless),
• an interactive 3D sphere debug view (rotate / zoom around the stitched sphere), and
• an interactive little-planet (stereographic) view.

It runs as a two-stage pipeline: a one-time calibration stage that precomputes remap LUTs and a GPU sphere mesh from the camera models, followed by a render stage that applies the precomputed data to actual camera frames.

All three outputs below are stitched from the four bundled test/ fisheye frames (sample data courtesy of zsibot).

Equirectangular panorama (CPU or --gpu):

Little planet — --planet	3D sphere debug view — --debug

• CPU LUT stitching with 6 blend modes (--blend 0..5): 0=soft (distance-weighted, default), 1=hard, 2=hybrid, 3=multiband (Laplacian pyramid), 4=seam, 5=best.
• Automatic exposure compensation (per-camera BGR gains) and seam-mask computation during calibration when sample images are present.
• GPU sphere-mesh blending: a per-vertex multi-camera mesh is rendered with a fragment shader that blends all overlapping cameras in one pass.
• Interactive viewers (X11 windows): a 3D sphere debug view (--debug) and a little-planet stereographic view (--planet).
• Per-camera coverage map output (CPU path) for sanity-checking calibration.

• OpenCV (image I/O, fisheye undistortion, remap, blending)
• yaml-cpp (reads settings.yaml / exposure gains)
• EGL and OpenGL ES 3.0 (glesv2) — required by the GPU path
• X11 (interactive --debug / --planet windows)
• GLM — vendored at include/panorama/glm/, no installation needed

Platform: Linux. The GPU path needs a working EGL / OpenGL ES 3.0 runtime; the interactive viewers additionally need an X11 display. The CPU path has no GPU requirement at runtime, but note that all four libraries above are required to build the project (both executables link the GPU library).

Out-of-source CMake build (C++17):

cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)

A clean Release build takes roughly 9 seconds on a typical machine. For a debug build use -DCMAKE_BUILD_TYPE=Debug. The two executables are written to build/panorama_calib and build/panorama_render; the static libraries (panorama_core, panorama_gpu) go to build/lib/.

source scripts/setup.bash

This exports PANORAMA_CONFIG_ROOT to the repository root (the code appends sources/... to it) and adds build/lib to LD_LIBRARY_PATH.

./build/panorama_calib

This reads the camera models from sources/camera_models/ and writes the LUTs, masks, blend weights, metadata.yaml, and the GPU sphere_mesh.bin into sources/calibration_files/. If the four sample images test/1.png–4.png exist, it additionally computes seam masks and exposure_gains.yaml.

Note: sources/calibration_files/ is a generated artifact directory and is not committed to the repository (it can be hundreds of MB). You must run panorama_calib once before rendering.

The bundled test/ fixtures (1.png–4.png, named 1/2/3/4) can be used directly as input, since the loader recognizes N.png/N.jpg filenames.

# CPU equirectangular (default blend mode 0 = soft)
./build/panorama_render --input test --output panorama.jpg

# CPU with a specific blend mode (multiband)
./build/panorama_render --input test --output panorama.jpg --blend 3

# GPU equirectangular (headless / FBO)
./build/panorama_render --gpu --input test --output panorama.jpg

# Interactive 3D sphere debug view (implies --gpu)
./build/panorama_render --debug --input test

# Interactive little-planet / stereographic view (implies --gpu)
./build/panorama_render --planet --input test --planet-scale 0.5

The CPU path also writes a <output>_coverage.jpg camera-coverage map next to the panorama.

Input images are matched against several naming conventions, tried in order: camN.jpg, camN.png, cameraN.jpg, cameraN.png, N.jpg, N.png (N is 1-indexed).

Both executables honor the PANORAMA_CONFIG_ROOT environment variable, which takes precedence over --root.

Camera intrinsics/extrinsics live in sources/camera_models/ as OpenCV cv::FileStorage YAML files. For the on-disk format and a guide to calibrating your own cameras, see docs/calibration.md.

The calibration data in sources/camera_models/ and the sample fisheye images in test/ are provided courtesy of zsibot.

This project is licensed under the Apache License 2.0 — see LICENSE. Third-party components (including the vendored GLM library) and runtime dependencies are documented in THIRD_PARTY.md.