Aim with your eyes.
A spatial computing mini-game built for Apple Vision Pro using Swift & visionOS.

GazeLab is an experimental eye-tracking reaction game for Apple Vision Pro.

The concept is simple:

• A ball appears at a random 3D location in front of you.
• You look at the ball to aim.
• You pinch to hit.
• Each successful hit scores a point.
• If you miss, the ball disappears after 1.0 second and respawns elsewhere.

But beyond being a game, GazeLab is also a lightweight behavioral data collection tool designed for HCI and spatial cognition research.

With spatial computing and gaze interaction becoming central to human–computer interaction, I wanted to explore:

• Reaction time in 3D environments
• Gaze-based targeting mechanics and visual eccentricity effects
• Interaction design in visionOS
• Structured behavioral data logging for research analysis
• Fitts' Law validation in gaze-based XR interaction

This project combines fun experimentation with structured behavioral measurement.

• Random 3D ball spawning — variable depth (0.7 m to 1.5 m) and lateral position
• Gaze-based targeting using visionOS native look-and-pinch interaction
• Pinch gesture to register hit
• Automatic respawn after 1.0 second
• Score tracking with live HUD (score + countdown timer)
• 30-second timed sessions with 5-second countdown

For every trial, GazeLab records:

• Trial number (ordered index for fatigue/learning analysis)
• Ball spawn timestamp
• Ball 3D coordinates in head-relative space (x, y, z) — encodes visual eccentricity
• Head orientation at spawn time (yaw, pitch in degrees) — control variable for pose confounds
• Reaction time (Hit Time − Spawn Time) in milliseconds
• Boolean hit marker (true/false)

At the end of a session:

• A spatial hit rate heatmap (6×6 grid) is displayed showing hit rate and mean reaction time per spatial region
• All recorded data can be exported as a CSV file

Reaction time is computed as:

Reaction Time (ms) = Hit Time − Spawn Time

Computed from high-precision Date timestamps at the exact moment of spawn and pinch confirmation. For missed trials (timeout), the actual elapsed time is recorded — not a hardcoded constant.

trial_number,timestamp,x,y,z,head_yaw,head_pitch,reaction_time,hit
1,2026-02-24T08:40:24Z,0.232,-0.044,-1.012,0.41,12.30,612,true
2,2026-02-24T08:40:25Z,0.238,0.031,-0.874,0.38,11.98,396,true
3,2026-02-24T08:40:25Z,-0.123,-0.070,-1.341,0.52,12.44,431,true
4,2026-02-24T08:40:25Z,-0.211,-0.246,-1.203,0.44,13.01,429,true
5,2026-02-24T08:40:26Z,0.215,-0.110,-0.921,0.39,12.77,430,true
6,2026-02-24T08:40:27Z,-0.066,0.099,-1.489,0.47,12.55,998,false

Ball coordinates are recorded in head-relative space — the position of the ball relative to the user's forward gaze direction at spawn time. This directly encodes visual eccentricity (how far off-center the ball appeared), which is the primary predictor variable for reaction time analysis.

Spawn ranges:

x  ∈ [−0.30,  0.30]   horizontal eccentricity
y  ∈ [−0.25,  0.15]   vertical eccentricity
z  ∈ [−1.50, −0.70]   depth (variable — 0.7 m to 1.5 m in front)

Head yaw and pitch are logged as control variables to detect post-hoc whether head pose confounded the spatial results.

• Swift
• SwiftUI
• RealityKit
• visionOS
• HoverEffectComponent — native gaze highlighting without requiring direct eye-tracking API access

GazeLab can be used for:

• Fitts' Law validation in gaze-based XR — does target distance and size predict acquisition time in 3D?
• Visual eccentricity effects — how does off-center target position affect reaction time and hit rate?
• Depth perception in targeting — does z-distance independently affect gaze-motor performance?
• Fatigue and learning curves — how does RT change across a session using trial index?
• Spatial cognition experiments — which regions of 3D space are hardest to acquire?
• HCI research — benchmarking gaze + pinch as an input modality on Apple Vision Pro
• Synthetic data generation — augmenting small gaze-motor datasets for downstream ML tasks

At the end of every session, GazeLab displays:

• Final score, total attempts, hit rate, average reaction time
• A 6×6 spatial heatmap showing hit rate (colour) and mean reaction time per grid cell, oriented to match the user's visual field (positive y = up, positive x = right)

• Moving targets
• Adaptive difficulty scaling (dynamic timeout based on rolling hit rate)
• Peripheral vision mode (forced off-center spawning)
• Multi-user benchmarking and session comparison
• Session analytics dashboard (Python pipeline for RT regression, Fitts' Law fit, user clustering)
• Configurable difficulty levels (Easy / Normal / Hard timeout durations)

Umang Dobhal
Master's Student | Human Intelligence Systems
Kyushu Institute of Technology, Japan
Built out of curiosity — for research.