Challenge: teach robots to walk

One of the basic features we expect in robots is movement. The task of teaching robots to walk is less simple than it seems

Israel Benjamin, "Galileo" magazine

"are you going. And you don't always understand it,
But you keep falling.
With each step, you fall a little further.
Then catch yourself so you don't fall.
Again and again, you fall.
Then catch yourself so you don't fall.
And this is the way you can walk and fall at the same time."
(Lori Anderson, "Walking and Falling", from the album "Big Science", 1982, free translation)

fall to go

Lori Anderson, the multimedia artist, who was among other things NASA's "house artist" (the first and only so far) in 2003-2004, accurately describes in the song "Walking and Falling" the dynamic physics of walking: while walking we are not in equilibrium.

If we photograph a person walking and ask an engineer to calculate the center of gravity of the person visible in the picture, the calculation will show that the person is about to fall. He will not fall because he will not remain in the same frozen position that the camera captured, but will move his legs and other parts of his body to turn the fall into forward movement. Such a walk maintains "dynamic stability".

In contrast, the walking of a toy robot is stable: every moment we turn off the robot, it will remain in equilibrium.

This is "static stability", which is the main reason why the walking of these robots looks so different from human walking.

How do people walk?

Why do humans walk this way, and also dogs, horses and many other animals? (It is true that most of the time the center of gravity of a four-legged animal is between the four legs, so that the photograph that "freezes" it at a certain moment will appear as if the situation is stable; but if the engineer analyzing this photograph is told that at the moment of the photograph the animal is moving forward, he will include the persistence in the calculation and conclude that the animal will still lose its balance if it does not "catch itself" - as anyone who has seen puppies take their first steps can testify).

Dynamic stability uses gravity to move faster, using less energy. It is also more suitable for the real world, which has surprises, such as a sudden change in the angle of the surface we step on, a change in the nature of the surface (flexible or hard), contact or friction with another bone, and more. A robot that relies on static stability cannot correct its posture when it is compromised by such a surprise.

In some modern robot demonstrations, the most impressive part is when a human (or another robot) pushes the robot hard. The robot reacts like a person surprised by such a push: it starts to fall, then moves its body and legs to stabilize itself.

When babies learn to walk, they are probably aware that we fall forward to move, but even if we once knew this, we don't think about it with every step. One may ask what the connection is between an artificial intelligence column and a subject that our natural intelligence does not deal with at all, but our brain does many things, some of them quite intelligent, without these actions reaching our consciousness.

If we accept the definition that artificial intelligence deals with issues where humans are better than machines (the "moving target" definition - as soon as a machine reaches a high capacity in a certain field, the interest of artificial intelligence developers in this field decreases), then walking is certainly an important field for research and development in artificial intelligence.

Quickly

One of the basic features we expect in robots is movement. Indeed, there are robots with a wide range of movement capabilities: robots that travel on wheels or chains, snake-like robots that are suitable for movement through narrow openings (as in rescuing people in disaster situations), swimming and flying robots, and even robots that move on skates. Among them, the walking robots stand out, both because of the visual similarity to the walking of humans or animals, and also because they have the potential to move quickly and efficiently on a wide variety of surfaces.
One of the first types of dynamically stable robots was actually a "monopod" (from Latin: one leg). Such a robot has only one leg, and it uses it to maintain stability: if it is about to fall to the left, it will jump to the left to return to a vertical position, like a child jumping on a "pogo stick". Such robots have been developed since the early 1986s in the "Leg Lab", which in its early years was part of Carnegie Mellon University, but since XNUMX is at the Massachusetts Institute of Technology (MIT leg lab).

The choice of one leg may seem strange, but it makes sense: first, if you want to understand dynamic stability, there is an advantage in the simplest situation: on one leg, when the "foot" is small, it is not possible to achieve static stability. Second, the calculations become more complicated the more legs there are to take care of, so in the early XNUMXs the level of knowledge, as well as the available technology, made it difficult to treat more than one leg. That leg was also rigid, with one motorized spring that activated the deceleration.

Today, one-legged robots are still being developed, but they are equipped with an articulated leg - for example, a small rod that connects to the point where the leg touches the ground and is used to sense if the surface is tilted. Such a rod mimics one of the functions of the foot in humans.

The same methods that used one leg were refined and generalized to control two or more legs. Walking on two legs is similar to the human movement while running, and robots were indeed developed in the same laboratory that could run and even perform a roll ("flip") in the air.

walk before you run

Other robots were designed according to the advice "you should learn to walk before you start running". Robots such as Honda's ASIMO were among the first to demonstrate walking, climbing stairs and running. Asimo's walking history demonstrates the technological challenge: initially the robot "knew" only a few movement patterns, and had to stop for a few seconds to prepare for the transition between walking and turning, and stop again to prepare for the transition to walking.

Later, more movement patterns were programmed and a smooth transition between them, and today the demonstrations are more impressive. For example, when the robot runs in a circle, it tilts its body towards the center of the circle. We do this too, although we are not aware of it - this allows us to maintain balance.

Asimo belongs to a group of robots whose walking is designed according to the ZMP (abbreviation of Zero Momentum Point) approach. In this approach, the computer controlling the robot's movements strives for a precise balance between the forces acting on the robot: its weight, the forces required to change speed and direction, and the reaction force exerted by the floor on the robot's legs.

If these forces are equal in magnitude and opposite in direction and act on the same point in the body, then in physical terms the magnitude of the torque of the force is zero, relative to the point where the robot touches the floor. This is good news for the robot: a non-zero torque means the existence of forces tending to tilt it and move it away from its vertical position. Asimo's walk has some elements of dynamic stability, but even so his walk has some drawbacks.

A more natural walk

First, it is much less efficient than human walking, as a result of forgoing the use of persistence to aid forward motion (instead, ZMP aims to balance persistence against opposing forces). Second, it is built on planning each step in advance, so it is difficult for it to react to surprises. That's why Asimo can only move on flat and hard surfaces. For a robot that needs to operate in a human environment, there is another drawback - Asimo's walk does not look human. It has been described as the gait of a child walking carefully so as not to make noise, or as the gait of a criminal (perhaps as a result of the same constraint), and even as the gait of a person who urgently needs the bathroom.

A more natural, and even more efficient, walk can be seen, among other things, in passive dynamic robots developed at the University of Michigan (link at the end of the column). The development started with a machine that is not a robot at all, because it does not have an engine nor a computer that controls its movements. The machine has two "legs" and two "arms", whose function is to swing while walking and thus shift the center of gravity.

As a result of careful planning of the weights and dimensions of all the machine's components, it can walk without any electronic control, when it uses almost all the energy of one step to drive the next step, with an efficiency equal to the efficiency of human walking (and 16 times greater than Asimo's efficiency). The inevitable loss of a small part of the energy is recovered by the machine "walking" down a gentle slope. Based on the experience gained in the development of this machine, motorized robots were built: the motor pushes the calves with each step to restore the lost energy.

Going outside, going to school

The robots mentioned so far move on hard and flat surfaces. What needs to be done for a robot to be able to walk on waves of stones, mud or snow? Dynamic stability is important here, because the robot cannot know, when it lowers its leg to the ground, if the leg will sink a little before it stops, if it will slip, etc.

One of the impressive robots that can operate in such an area is "Big Dog", which is developed by an American company and is intended, among other things, for military uses. The four-legged robot can climb a sand hill, carry loads (about 70 kg) and even jump over obstacles, as you can see in the link at the end of the column and in the videos shown on this website. Perhaps not coincidentally, two of the films are accompanied by music by Laurie Anderson, who was quoted at the beginning of the article.

Designing robot movement is difficult for many reasons: one of the reasons is that even though we have models to imitate (walking of a person, a dog, a spider, etc.), we don't really know how we walk - for example, when do we move the pelvis? What kind of walk? At the beginning of the step or at the end? How do the hands fit into the movement?

One option is to meticulously study human movement, as sports trainers and doctors do. Another possibility is to make the robot try a large number of combinations and "motion plans", focus on the best ones and continue to improve them. In other words, let the robot learn how to walk. This can be done quickly, if we simulate the robot and its environment in a computer simulation, but then the simulation may not accurately describe the behavior of the mechanical systems and the randomness of the "real world". And you can also build the robot and program it to try different movement patterns.

Robots are currently learning to walk in several laboratories around the world. Let's mention one example: as part of the "RoboCup Soccer", one of the competitions is between teams of AIBO robots (the small dog-like robots developed by Sony). The group from the University of Texas at Austin, like other groups, faces among other things the challenge of properly planning the movement of the robots.
The walk that Sony designed was not fast enough to win the game, and the developers aimed to achieve a "trot" movement, in which the front right leg touches the ground together with the left back leg, and when they are raised the other two legs touch the ground together.

There are many options in determining the details of this movement. The developers decided to run three robots, which started from inefficient walks. The software that controlled the robots gradually improved the walking patterns, through automatic learning processes, until the highest speed reported for this type of robot was reached (at least until the time of that study): about 30 cm per second (or about XNUMX km per hour).

The developers, who expected that at any moment there would be two feet on the ground, were surprised to find that the robots taught themselves to place each pair of feet on the ground only 43% of the time, meaning that 14% of the time the robot does not touch the ground at all. It is interesting that even horses moving at a tap are detached from the ground for fractions of a second with each step (to resolve a long-standing debate on this question, photography and fast exposure techniques were developed in 1877, and these paved the way for the development of cinema).

The developers were not satisfied with achieving high speed, because the robots compete not in running but in football. In order for them to be able to see the ball and the position of the other players, the camera mounted on the robot's head should be as stable as possible.

The way the robots initially learned to move caused the camera to move violently. Therefore, the robots were sent to another task, where they were required to learn fast movement that also maintains the ability to identify the objects captured by the robots' cameras.

The electronic puppies succeeded in this task as well, and adopted a movement that is about 30% slower than the previous movement, but significantly improves the robot's ability to follow the ball. The researchers suggest adding to the robot's "toolbox" selection processes between running as fast as possible and running that concentrates on identifying the other objects in the game and their location.

Due to the great interest in robotics, the field of mobile robots is receiving a lot of attention. Almost every day brings new breakthroughs, related to walking, running, crawling, jumping or rolling. Perhaps following these efforts, a time will come when we will accept the fact that a robot can move as a matter of course, and walking will no longer pose a challenge to artificial intelligence, as we take for granted our ability to walk, without even being able to reconstruct how we learned to do so.

Israel Binyamini works at ClickSoftware developing advanced optimization methods.

Thanks to Tom Spiegelman for interesting conversations and information he contributed to this column.