The APT-RL system combines pre-learned movement skills with reinforcement learning. The robot chooses when to run, climb, or jump using sensors and a computer on its body.
Four-legged robot It was able to move autonomously through the forest, climb stairs, pass between stepping stones, and jump over fallen logs, using a control system that selects in real time between several movement skills.
The system, known as APT-RL – an acronym for Action Pretrained Transformer–based Reinforcement Learning – was developed by Jun-Gil Kang and his colleagues. The research is published in the journal Science Robotics of AAAS.
Four-legged animals can quickly change their movement patterns depending on the terrain. They can switch from walking to running, slow down in front of an obstacle, jump, or change the position of their feet. Robots usually require separate motion controllers and precise planning for each task.
The researchers sought to develop a system that would not relearn every task, but rather be able to select and integrate skills that had already been learned. To this end, they divided the training process into three stages, progressing from basic movements to complex behavior in a real environment.
In the first phase, the robot learned basic movement patterns, such as jogging and galloping, using a large database of two-dimensional movements. These skills served as a kind of motor vocabulary on which to build more complex behaviors.
In the second stage, it was activated reinforcement learningThe robot trained in choosing the appropriate skill for the situation: when to continue running, when to slow down, when to climb, and when to switch to a jumping motion. During the training, the system received feedback on its success in passing obstacles while maintaining stability and speed.
In the third stage, the actions were adapted to information from real sensors. Instead of relying on a detailed map or an external computer, the robot processed the information using the perception and computing systems on its body.
The combination allows for rapid transitions between movement forms. When the robot detects a low trunk, for example, it may switch from continuous movement to jumping. When faced with steps or stones, it may choose a slower, more precise movement, and then return to running.
The ability to switch between skills is important for robots to be used in unstructured environments. Such robots may in the future operate at disaster sites, in mines, forests, industrial plants, or in tasks where wheeled vehicles have difficulty moving.
However, successful demonstrations on test tracks do not guarantee reliable operation in every environment. Lighting conditions, mud, dense vegetation, slippery surfaces, unexpected obstacles or sensor malfunctions can still make it difficult for the system.
The study illustrates a trend inרובוטיקה: Moving from a controller designed for a single movement to systems that use a pool of skills and decide for themselves which one is appropriate for each moment.
More on the subject on the science website
- Will human-level artificial intelligence remain out of reach?
- 70 million shekels for a joint Israeli infrastructure for biological artificial intelligence models
- America in a Test Tube 7: Who decides what scientists are allowed to do?
- Does the mysterious glow at the center of the Milky Way actually come from dark matter?
- Euclid Space Telescope Discovers the Earliest Quasars Ever Observed
