Seeing Around Corners with Consumer LiDAR

We show that the same consumer LiDAR sensors found in smartphones, AR/VR headsets, and autonomous vehicles can be used to detect and track objects hidden around corners. Instead of only sensing directly visible surfaces, these devices can also capture faint indirect light that has bounced through the environment.

What makes this significant is that around-the-corner imaging has traditionally required highly specialized laboratory systems costing hundreds of thousands of dollars. Our work suggests that similar capabilities may eventually become possible using low-cost, widely available consumer hardware.

When researchers first explored around-the-corner imaging nearly two decades ago, the required sensors only existed in specialized scientific laboratories. These systems were large, expensive, and originally developed for fields like ultrafast laser physics and femtochemistry.

Over the last several years, similar sensing technology began appearing in consumer LiDAR hardware. Once we started experimenting with these devices, we realized they were already capturing faint hidden-scene signals. The difficult part was developing algorithms capable of extracting useful information from measurements that were extremely noisy and low resolution.

The system measures weak indirect light transport produced when light scatters off visible surfaces and reaches objects outside the direct line of sight before returning to the sensor. These indirect signals are several orders of magnitude weaker than conventional direct reflections.

Rather than relying on a single measurement, the system combines information across many measurements collected over time. Motion of either the sensor or hidden object introduces diversity in the observations, allowing the algorithm to coherently fuse multiple noisy measurements into a more informative estimate of the hidden scene.

The primary challenge was the extremely weak signal strength available in consumer LiDAR systems. Compared to laboratory NLOS systems, consumer sensors have substantially lower spatial-temporal resolution and were not designed for indirect light transport analysis.

As a result, early reconstructions were dominated by noise and measurement ambiguity. The key technical insight was recognizing that the hidden-scene information was still present in the measurements, but required multi-frame fusion and motion-aware reconstruction algorithms to recover it reliably.

Potential applications include autonomous navigation, robotics, and wearable computing. In autonomous driving, NLOS sensing could improve detection of vehicles, cyclists, or pedestrians at blind intersections before they enter direct view. In robotics, it could improve navigation and scene understanding in partially occluded environments. For AR/VR systems, indirect light sensing could eventually improve spatial awareness, hidden object localization, and body tracking outside the immediate camera field of view.

More broadly, this work suggests that hidden-scene sensing may eventually become feasible on widely deployed mobile sensing platforms. By making this technology widely accessible, we hope that people will discover applications far beyond we originally imagined.

Yes. The current system assumes some temporal consistency in object geometry and motion across measurements. These assumptions allow multiple weak observations to be fused coherently over time. Performance may degrade in scenarios involving highly non-rigid motion, abrupt sensor movement, heavy occlusion, or rapidly changing scene dynamics. Reducing dependence on these assumptions remains an important direction for future work.

Not yet. This work represents an early-stage research prototype rather than a fully deployable sensing system. Important challenges remain, including operation at longer ranges, handling more difficult environments, improving robustness to unpredictable motion, and achieving reliable real-time performance on mobile hardware.

Additionally, the system does not produce conventional photographic imagery of hidden scenes. Current reconstructions primarily recover sparse geometric and motion information from extremely weak indirect measurements.

Future work will likely require innovations along both algorithmic and hardware directions. On the algorithmic side, an important goal is developing reconstruction methods that operate under fewer assumptions and remain robust under more realistic scene dynamics and noise conditions.

Some specific open problems include handling highly unpredictable or non-rigid motion, operating reliably in extremely low-light or high-noise environments, handling longer ranges (several meters), improving robustness to calibration errors, and achieving real-time performance on mobile hardware. Another important direction is multimodal sensor fusion—combining indirect LiDAR measurements with RGB cameras, inertial sensors, or other sensing modalities to improve reconstruction quality and stability. There are also many open research problems in integrating non-line-of-sight perception into existing robotic, autonomous navigation, and wearable AR/VR platforms.

On the hardware side, current consumer LiDAR systems are primarily optimized for direct line-of-sight depth sensing rather than indirect light transport. These results raise the possibility of designing future sensors specifically for both visible and hidden-scene understanding. Improvements in detector sensitivity, spatial-temporal resolution, scanning strategies, and optical design could substantially improve non-line-of-sight sensing performance while remaining compatible with mobile consumer devices.