Scientists have developed a robot that can learn tasks like cleaning a sink simply by observing humans. A specially designed sponge equipped with sensors is used to demonstrate how cleaning is done. Thanks to a sophisticated machine learning system, the robot learns how to execute these tasks and can adapt this knowledge to clean various types of sinks.
Robots are meant to handle mundane or undesirable tasks for us. Yet, jobs like bathroom cleaning present challenges in automation. How do we program a robot arm to reach every corner of a sink? What if the sink has oddly shaped edges? How much pressure should be used, and where?
Creating detailed, fixed instructions to tackle these issues would take a lot of time. Instead, researchers at TU Wien have opted for a different method: a human demonstrates the cleaning process to the robot several times. A specially equipped sponge is utilized to clean the rim of a sink. By observing the human action, the robot learns how to clean and can adjust that knowledge for differently shaped surfaces. The findings were recently shared at the IROS 2024 conference in Abu Dhabi.
Cleaning, sanding, polishing
Cleaning is one form of surface treatment, but many other industrial activities like sanding, polishing surfaces, painting, or applying adhesives are technically quite similar.
“Capturing the shape of a sink with cameras is relatively straightforward,” explains Prof. Andreas Kugi from TU Wien’s Automation and Control Institute. “However, the real challenge is teaching the robot what specific movement is needed for each part of the surface. How fast should it move? What’s the correct angle? How much force is required?”
Humans acquire this type of knowledge through experience and mimicking others. “In a workshop, an experienced worker might guide an apprentice, saying they need to apply a bit more pressure on a narrow edge,” mentions Christian Hartl-Nesic, who leads the Industrial Robotics group in Kugi’s team. “We aimed to implement a similar learning method for the robot.”
A prototype of a cleaning sponge
A unique cleaning tool was created for this effort: a sponge outfitted with force sensors and tracking markers that people used repeatedly to clean the rim of a sink. “We gather a significant amount of data from just a few demonstrations, which is then analyzed so the robot can learn what effective cleaning entails,” clarifies Christian Hartl-Nesic.
This learning method is facilitated by an innovative data processing strategy created by the TU Wien research team. It merges various existing machine learning techniques: measurement data undergoes statistical analysis, which then informs a neural network to recognize established movement elements, known as ‘motion primitives’. The robot arm is consequently optimized to execute the cleaning on surfaces.
This groundbreaking learning algorithm allows the robot to clean the entire sink or other objects with complex surfaces after training, even if it was only shown how to clean a single edge of the sink. “The robot comprehends that it must hold the sponge differently depending on the surface shape and adjust the pressure for tightly curved areas versus flat sections,” shares PhD student Christoph Unger from the Industrial Robotics team.
The vision: collective learning among workshop robots
The technology showcased applies to various processes, from sanding wooden pieces in carpentry to fixing and polishing vehicle paintwork, and welding metal parts. In the future, the robot may operate on a mobile platform, serving as an effective assistant in any workshop.
These robots could even collaborate and share knowledge. “Imagine numerous workshops utilizing these self-learning robots for sanding or painting tasks. They could learn independently with localized data, but still exchange learned parameters among themselves,” envisions Andreas Kugi. Individual data, like the unique shape of a specific workpiece, would remain confidential, but key foundational principles could be shared to enhance all robots’ skills. This concept is known as ‘federated learning.’
Extensive tests at TU Wien have demonstrated the cleaning robot’s versatility. The innovation is already gaining international attention: at the IROS 2024 conference (October 14 to 18, 2024), which featured over 3,500 scientific submissions, TU Wien was awarded the ‘Best Application Paper Award,’ recognizing it as one of the year’s most significant innovations.
