A jump uphill

A robotic driver and a tiny helicopter can perform dangerous stunts after "learning" the software: will they be able to prevent dangerous situations in the future?

Israel Benjamin Galileo

in a 180 degree spin and stopped exactly between two parked cars. This is a dangerous stunt, which is usually only seen in action movies or driving shows, but this is also the challenge that the "Stanford Racing Team" took on. Contrary to what its name suggests, this is not a group of racing drivers but of computer scientists, engaged in the development of robotic cars - machines that can drive themselves.

In May 2010, the group presented the results of their latest research at the International Robotics and Automation Conference held in Anchorage, Alaska. They taught their robotic vehicle, Junior, to perform the skid-park stunt, as you can see in a video they posted.

Why should you teach a robotic vehicle to perform dangerous stunts? Is it to facilitate the production of action films? Or maybe so that each of us can take the rare parking spot that we find in front of us, before another car slides into that spot? As we will see later, there are better reasons.

Junior, a robot with an illustrious past

The student who signed up for stunt studies, Junior, presents an impressive resume for his young age. It was built by the Stanford Racing Group, which includes members of the Artificial Intelligence and Engineering departments of Stanford University in California, as well as researchers and engineers from automotive (Toyota, Volkswagen) and technology (Bush, Intel) companies. Stanley, Junior's older brother, won the "Big Challenge" competition in 2005 and brought its developers a prize of two million dollars.

This competition was founded by DARPA, the US Department of Defense's Advanced Research Projects Agency, to advance technology by setting a technological challenge that rents on its side. Stanley, which began life as a Volkswagen Touareg, completed a difficult desert course of more than two hundred kilometers in less than seven hours, ten minutes less than the vehicle developed by a group from Carnegie Mellon University. Today, Stanley is on display at the Smithsonian Museum of American History in Washington.

The Great Challenge competition of 2007 raised the level of difficulty, and required the competitors to successfully complete an urban route of approximately 100 km while blending in with traffic and obeying laws and signs. This time the Carnegie-Mellon team got their revenge when "Boss", their robotic car, successfully completed the course in four hours and ten minutes. Junior, the robot built by the Stanford group based on a Volkswagen Passat car, arrived twenty minutes later and won the second prize - one million dollars.

closed loops

In the technical paper presented at the conference, the members of the racing team explain the method they used to teach Junior to perform the stunt. Control systems, such as the one used to drive a robotic vehicle, are usually aided by a mathematical model of the system we want to control - in this case, the car. The model includes a mathematical representation of the vehicle's behavior in different situations and in response to different control actions.

So, for example, if we enter information into the model that the vehicle is moving in fourth gear, at a speed of 60 km/h and in a straight line on a flat track (representation of the situation), and the steering wheel is tilted at an angle of 7 degrees to the left while pressing 10% on the brakes (representation of the control actions), The model will predict the vehicle's behavior one hundredth of a second later: how much the vehicle's speed will decrease, in which direction it will be turned, what will be its speed of rotation (rate of angle change), etc.

Other software systems in the robotic vehicle are responsible for planning the required route and deciding how fast to move at each point on the route. The function of the control system is to operate the vehicle so that it moves according to the design. With the help of the mathematical model, it is possible to find the control actions required to get from the current state to the required state in a hundredth of a second (or another short period of time), and continue this way until the plan is successfully executed.

In fact, the model cannot be completely accurate. There will always be small mistakes in the model itself or in the execution of the control actions (for example, moving the steering wheel by 7.03 degrees instead of 7 degrees), slight changes in the road conditions (a small dent, for example), etc., causing the vehicle to deviate somewhat from the plan at the end of the next hundredth of a second.

Such deviations, imperceptible as they may be, will accumulate over the next few seconds and will eventually bring the vehicle to a place and state completely different from what was planned. To prevent errors from accumulating, the vehicle is also equipped with sensors that measure the vehicle's position and condition at every moment. The control software compares the information coming from the sensors to the information that the model sees and uses the information to adapt the driving actions to the real situation.

This is a control process based on a closed loop: the system not only anticipates the behavior of the vehicle and plans its actions accordingly, but also corrects the planning according to feedback received from the sensors and allows it to more accurately measure the results of its actions. This process is not simple, and one must take into account, among other things, that the sensors themselves are not perfect, so small errors ("noise") in the information they provide are expected. However, as demonstrated by the robots that participated in the Great Challenge competition, impressive results can be achieved with this method.

open loops

One of the problems with the closed loop method is the need for a model with high reliability, even if it cannot have perfect accuracy. Often the model is created by learning from recordings of the system in action. This is also how the racing team operated: they recorded a few minutes of human driving in a car, in which the position and condition of the vehicle as well as the actions of the driver at each moment were measured, and calculated the model from this recording. Such models work well when the vehicle is in situations similar to those recorded during human driving, and when the rate of change is low - without sharp braking, sharp turning of the steering wheel, etc.

When the changes in the vehicle's condition and the driver's actions are rapid, the model loses its reliability. One reason for this is the increase in errors, but there is also a deeper reason: suppose that a model learned from a recording of movement at 40 km/h and a rotation of 20 degrees per second can predict with high accuracy the response to movement at a speed of 50 km/h and a rotation of 30 degrees per second . Nevertheless, a model learned from a recording of movement at 40 km/h and at a turn-slip of 180 degrees per second will not be able to predict the vehicle's behavior at the same speed and with a slight change in the rate of rotation - say, 170 degrees per second.

Although the change in the second case is smaller (the same speed, a change of about 5% in the rotation rate in the second case, compared to a 25% change in speed and rotation rate in the first case), in the second case the vehicle is in a state that is very sensitive to any change in control, so the model does not allow the calculation of The response to such changes.

Therefore, in such extreme situations closed-loop control is not appropriate, because it is impossible to feed the feedback from the sensors into the model to calculate the required correction at each moment. To find a better control method, the developers took advantage of another finding that emerged from their research: although the vehicle's behavior is difficult to predict in advance in extreme situations, it repeats itself: if the driver acts in exactly the same way (same speed, same sharp turn of the steering wheel, same braking force) as he did at the time The recording, the vehicle will move on a route very similar to the previous route.

This finding leads to open-loop control: reaching a state identical to the recorded state and exact repetition of the recorded control actions. To activate this method, the developers recorded the actions of a driver who would lock the wheels while driving backwards and perform a 180 degree spin. It is interesting to emphasize that the human driver was not required to perform the parking stunt, and therefore did not direct the action so that it would end exactly at a certain predetermined place. The software performed the trick by calculating the direction and place to be reached, at the right speed, so that repeating the driver's recorded actions would result in a stop at the requested parking spot.
Open and closed combination

As you can see in the video in the attached link, control using the open loop method manages to reach the required position (180 degree rotation) but not to the required location. The reason for this is that the open-loop control accumulates errors during the drive, due to the lack of the design correction according to feedback from the sensors. In order to achieve a perfect execution of the stunt, the developers chose to combine the two methods: the car arrives at the starting point of the spin under closed loop control, which corresponds to the steady state of "simple" driving, and switches to open loop control when the vehicle leaves the stable domain and enters wheel lock and spin. As described in the technical paper, the combination of the two methods uses an estimate of the accuracy of each in the current state: the accuracy of the closed loop increases as the current state is less sensitive to small changes in control, and the accuracy of the open loop increases as the current state is close to the recorded state.

This project is not the only one interested in robotic stunts. Another research team at Stanford University demonstrated in 2008 software that performs seemingly impossible stunts by flying a helicopter without a pilot. Here, too, they found that it is necessary to learn how a human operator performs the stunts, but the software was able to learn the ideal pattern from several demonstrations, and was thus able to perform the stunts with greater precision than the original human demonstrator.

The developers of the robotic helicopter explained that there is a real need for such software to ensure that automatic flight will be safe in any situation, even when the flight encounters difficult conditions. The need for such virtuosity was illustrated in January 2009, when pilot Chesley Sullenberger was forced to land an airliner in the Hudson River after striking a flock of geese disabled the plane's engines. Similarly, members of Stanford's racing team explain that they developed the ability to drive stunts to create a robotic vehicle capable of operating correctly even in dangerous situations. If they also teach the robot to act in such a way as to avoid such situations as much as possible, it is possible that in the future we will be able to leave the steering wheel in much safer hands than human hands.

Israel Binyamini works at ClickSoftware developing advanced optimization methods.
The full article was published in Galileo magazine, August 2010