Real-time planar image tracking running entirely in the browser. No native apps, no plugins, no server-side processing. Uses WebAssembly (OpenCV compiled to WASM), WebGL2 for GPU-accelerated preprocessing, and Web Workers for off-main-thread computation.

Works on desktop and mobile browsers with camera access.

Demo on laptop	Demo on iPhone

Try the live demo in your browser:

https://edwardlu2018.github.io/wasm-ar/

Point your camera at the reference image. If the overlay positioning looks off, look away briefly and point back at the image to re-detect.

Everything runs client-side in the browser:

1. WebGL2 GPU shader converts the camera video frame to grayscale on the main thread
2. Grayscale pixel data is sent to a Web Worker running an OpenCV WASM module
3. ORB feature detection matches keypoints from the reference image to the video frame and computes a homography matrix
4. Frame-to-frame tracking uses Lucas-Kanade optical flow for smooth, fast updates without re-detecting every frame
5. The homography matrix is passed back to JavaScript via postMessage and used to warp an HTML element (iframe) with a CSS matrix3d transform
6. When tracking is lost (invalid homography or too few tracked points), ORB detection runs again automatically

git clone --recursive git@github.com:EdwardLu2018/wasm-ar.git

If you accidentally cloned without --recursive:

# in repo root
git submodule update --init --recursive

• python3
• make
• Emscripten SDK

git clone https://github.com/emscripten-core/emsdk
cd emsdk
./emsdk install latest
./emsdk activate latest
source ./emsdk_env.sh

OpenCV is included as a git submodule. Build it for WASM using the official build script (see OpenCV.js build docs for details):

# in repo root
emcmake python3 opencv/platforms/js/build_js.py opencv/build_wasm --cmake_option="-DCMAKE_CXX_STANDARD=17"

# in repo root
npm install
npm run build
npm run serve

Then open http://localhost:8000/html/ in your browser.