Recent advancements in smartphone LiDAR technology have unveiled an exciting capability: the ability to perceive objects that are not directly visible, even around corners. Researchers at the MIT Media Lab, led by Siddharth Somasundaram, have demonstrated that consumer-grade LiDAR sensors can reconstruct hidden 3D shapes and track movement by utilizing indirect light reflections.
This innovative system does not provide clear images but instead recovers sparse shapes and motion data from faint light signals--an achievement that previously necessitated expensive lab equipment. By adapting this technology for everyday smartphones, the potential applications are vast, ranging from enhancing autonomous vehicles' ability to detect pedestrians to improving navigation for robots in cluttered environments.
Somasundaram expressed his enthusiasm, stating, "The most exciting part of this work is that we took a capability that used to require a specialized $50,000 imaging setup and made it accessible for people in robotics, AR/VR, and beyond."
Understanding Non-Line-of-Sight Imaging
The LiDAR technology functions by emitting laser pulses and measuring the time it takes for the light to return. This process demands high precision, as light travels incredibly fast. Traditional LiDAR systems typically only map directly visible objects. However, this new approach leverages the indirect paths that light takes after reflecting off surfaces, allowing the sensor to gather information about hidden objects.
Researchers have termed this technique "non-line-of-sight imaging." Although the concept has existed for some time, previous implementations relied on costly equipment. The breakthrough here lies in effectively utilizing the data from a low-cost consumer sensor.
By employing a method known as motion-induced aperture sampling, the team combined multiple frames captured while the smartphone was in motion. This technique allowed them to enhance the clarity of the signals received, transforming weak glimpses into more accurate representations of hidden scenes.
Demonstrated Capabilities
The research showcased three key functionalities: reconstructing the 3D shapes of stationary hidden objects, tracking the motion of known shapes, and using hidden objects as reference points for determining the camera's location. The resulting visuals resemble rough 3D forms rather than crisp photographs, illustrating the system's ability to detect and track objects even in challenging environments.
While the immediate focus is on smartphones, the implications of this technology extend far beyond. With hardware costs under $100 and no complex setup required, the democratization of this capability could lead to innovative applications across various fields.
Potential uses include enabling warehouse robots to sense movement before entering an aisle or allowing AR headsets to track user hand movements outside the camera's view. This technology could also enhance autonomous driving systems, although the transition from lab demonstrations to real-world applications presents challenges.
As LiDAR technology becomes more prevalent, Somasundaram envisions a future where entirely new forms of machine vision and spatial perception emerge. Jessica Rosenworcel, executive director of the MIT Media Lab, emphasized that this work marks the arrival of advanced vision capabilities in consumer devices, hinting at transformative possibilities yet to be explored.
