Computer scientists have created a camera mechanism that enhances how robots perceive and interact with their surroundings. Taking inspiration from the human eye, this innovative camera system emulates the small involuntary movements that the eye makes to sustain clear and stable vision over time.
Led by a team of computer scientists from the University of Maryland, this camera mechanism improves how robots observe and respond to their environment. Inspired by the workings of the human eye, the team developed a groundbreaking camera system named the Artificial Microsaccade-Enhanced Event Camera (AMI-EV). Their work was detailed in a paper published in Science Robotics in May 2024.
“Event cameras are a relatively new technology designed to track moving objects more effectively than traditional cameras. However, existing event cameras face challenges in capturing sharp, blur-free images in high-motion scenarios,” said Botao He, lead author of the paper and a Ph.D. student in computer science at UMD. “This is a significant issue as reliable and timely images are crucial for robots, self-driving cars, and other technologies to react accurately to changing environments. We asked ourselves: How do humans and animals maintain focus on a moving object?”
The team found the answer in microsaccades – small, rapid eye movements that occur involuntarily when a person attempts to concentrate their gaze. Through these subtle yet continual movements, the human eye can remain fixated on an object and its visual details like color, depth, and shadows accurately over time.
The researchers replicated microsaccades by integrating a rotating prism into the AMI-EV camera to redirect the light entering through the lens. The prism’s rotational motion mimicked the natural movements of the human eye, enabling the camera to stabilize the textures of an object being recorded similar to human vision. They then developed software to compensate for the prism’s movement within the AMI-EV to produce stable images from the shifting light sources.
Yiannis Aloimonos, a professor of computer science at UMD and co-author of the study, sees this invention as a significant advancement in robotic vision.
“Our eyes capture images of the world around us, which are then processed in the brain, leading to perception and our understanding of the world,” explained Aloimonos, who also heads the Computer Vision Laboratory at UMIACS. “In the case of robots, the eyes are replaced by cameras and the brain by a computer. Improved cameras translate to better perception and responses for robots.”
The researchers believe that their innovation could extend beyond robotics and national defense. Industries reliant on precise image capture and shape recognition are continually seeking ways to enhance their cameras, and the AMI-EV could offer solutions to many of their challenges.
Cornelia Fermüller, a senior author of the paper and a research scientist, highlighted the potential for event sensors and AMI-EV to revolutionize smart wearables. These technologies boast advantages over conventional cameras, such as exceptional performance in varying lighting conditions, minimal latency, and low energy consumption. These qualities make them ideal for applications like virtual reality, demanding seamless experiences and rapid processing of head and body movements.
In initial tests, AMI-EV accurately captured movements in various scenarios, including human pulse detection and quickly moving shape identification. The camera achieved frame rates in tens of thousands per second, surpassing the capabilities of most commercial cameras, typically capturing 30 to 1000 frames per second. This enhanced depiction of motion could be pivotal in creating immersive augmented reality experiences, improving security monitoring, and aiding astronomers in capturing space images.
“Our innovative camera system can address specific challenges, such as assisting self-driving cars in recognizing humans on the road,” Aloimonos added. “It has numerous applications that are already familiar to the public, such as autonomous driving systems and smartphone cameras. We believe that our novel camera system sets the stage for more advanced and capable systems in the future.”
