A new approach allows smart devices to decode sounds that are not human speech sounds
Already today, robots can recognize a human voice and respond to it, thanks to voice recognition software such as iPhone's Siri. However, smart devices still have difficulty recognizing most other sounds. "In a certain sense, this is not a particularly complicated problem, but there has not yet been extensive and in-depth research conducted in the field of environmental noise," says Joseph Romano, a robotics expert from Boston-based Rethink Robotics. "So far [environmental noises] have not been within the capabilities of the robotic feedback."
Now, Romano is trying to "train" robots to listen not only to the conversations we have. Romano and his research partners at the University of Pennsylvania developed a software tool called ROAR (the acronym for the software name in English: Open Source Robotic Operating System for Auditory Recognition), which allows roboticists to "teach" robots to respond to a much wider spectrum of sounds than a human voice. All that is required to run the software is a microphone, as described in an article published in one of the latest issues of the journal Autonomous Robots.
In the first stage of the training, the microphone picks up sounds from the environment and the ROAR software cleans them of static noises. Next, the operator teaches ROAR to recognize certain sounds by repeating a certain action, such as closing a door or activating a smartphone alarm, and tagging the sound's unique audio signature as the robot listens to it. Finally, the software creates a general model of each of the sounds in this set of training pieces.
Romano's research team tested ROAR on a one-armed robot, improving its ability to successfully perform certain tasks. In one of the scenarios, the robot tried to grab an electric drill by itself and turn it on. In the absence of voice feedback, the robot was only able to complete the action in nine out of 20 attempts, but its success rate was doubled with ROAR. If, the robot did not hear the hum of the electric motor after holding the drill, it adjusted its grip on the device and tried to operate it one more time.
In the future, the development team will try to ensure that the system will also work in noisy environments. Integrating audio into the range of possibilities of the robotic feedback, alongside visual and touch signals, could in the future allow robotic nurses to quickly respond to calls for help or help industrial robots to respond in the event of a malfunction in the production chain. The technology is still in its initial stages of development, but Romano believes that it has enormous possibilities. "We haven't even begun to explore how we can implement it," says Romano.
The article is published with the approval of Scientific American Israel
