The pioneer of computer vision tries to imitate the human brain - an interview with Israel Prize winner Prof. Shimon Ullman
We are all surrounded by thousands of computers and cameras every day. However, what allows the human from the computer, as of now, is our ability to understand what we see, while the computer needs an intermediary. A person observing other people's faces will be able to easily identify facial expressions such as sadness, joy, anger or worry. The computer can scan many more faces very quickly, but will not be able to recognize the expressions without a great deal of prior training, and even then will be limited to a very specific variety. Is it possible to give computers the image decoding ability that humans have? "In some very difficult fields, computers are no longer far from human ability, such as the ability to recognize certain objects in an image, or to recognize faces," says Prof. Shimon Ullman from the Weizmann Institute of Science, who will receive the Israel Prize in Mathematics and Computer Science this year for his contribution to the field of computer vision. "We look at the world and immediately receive, within a fraction of a second, very rich and varied information: perhaps we see people playing chess or children playing ball or a car crossing an intersection. Everything we look at, it is immediately clear to us what is there: if there are people there, who is doing what to whom, and so on. At the moment we do not have the ability to do such things on the computer. But we are making progress towards such capabilities."
wings and brain
Ullman was a fighter pilot in the Air Force, who turned to studying mathematics and physics, during which he discovered the study of the brain and also added biology to his study program at the Hebrew University. When he finished his first degree with honors, he applied for a direct doctorate in the field of artificial intelligence at the Massachusetts Institute of Technology (MIT). He also remained there as a researcher, but in the 80s he returned to Israel, joined the Weizmann Institute, and brought the field of computer vision to Israel. "In the human system, the eyes bring the image to the brain and the brain does the work, understanding what is in the image," he explains. "We would like to do the same in a computerized way, in a computer and camera system. The image reaches the computer, and it plays the role of the brain. We would like to know what kind of calculations should be introduced to him, what algorithms and what calculation capabilities, so that he can see and understand the world around him from the images that reach him through the camera."
capture the essence
In his early work, Ullman managed to be the first to develop a computer system that knows how to detect depth in a two-dimensional image, and developed systems for detecting motion. Later he moved on to the development of "higher" systems, designed to distinguish between different objects. For example, a dog and a cat are quite similar creatures, they have four legs, a head, a tail, etc. A person, even a toddler, can easily recognize if he sees a dog or a cat, but it is very difficult for a computer to capture "dogs" and "cats", and the software needs guided viewing of thousands of images, in order to succeed in reaching a reasonable level of diagnosis. Researchers continue to face such challenges all the time. One of the projects Ullman is currently leading is the "digital baby" - an attempt to develop a computer system that can imitate what is happening in the brain of a baby, which receives and processes a huge amount of visual information in the first year of its life, and assembles a picture of the world for itself. "When a baby opens his eyes, he begins to receive a wealth of information through them, and in some way it works out in his head and he begins to understand the world without organized guidance from the people around him, because he does not yet understand language," says Ullman. "We would like to understand these processes and show that they can be created in a machine. A computer system that has not seen a single image, and weeks and months of images and videos begin to be streamed to it. As a result, she will create a picture of the world for herself: people, objects, the connections between them and so on."
distant answers
The road to developing such capabilities in computers is still long, and it involves a better understanding of the processes that occur in the human brain. "In humans, all learning, at least in the first stages, is done in a very unguided way. He is simply exposed to information, and is able to absorb the significant information from the wealth of information," he says. "There are things there that are fundamentally different from everything we currently understand. That's why this type of research direction seems interesting, but it deals with questions that we are still very far from finding the answers to."
a developing field
In addition to his academic work, Ullman is one of the driving forces behind the computer vision industry in Israel. He was one of the founders of the "Orbotek" company, which deals in visual scanning of components for the electronics industry, based on patents developed by him and his colleagues. For more than 20 years Ullman was the company's chief scientist, and also served in a similar position at the Israeli division of Applied Materials. The field of computer vision has a huge variety of potential applications: from security uses, such as searching for terrorists in the crowd with the help of facial recognition systems, through medical applications, developments in the field of commerce, industry, assistance to the blind, development of autonomous vehicles, and more. "This is a field for which greatness is predicted, and it is developing rapidly all over the world, and to our delight, it is developing particularly rapidly here in Israel," says Ullman. "The computer vision industry in Israel already exports hundreds of millions of dollars a year and employs thousands of workers, and it is expected to continue to expand."
Let's wait and see
Ullman was the pioneer of computer vision in Israel, and today many laboratories in the field - in academia and industry - are following his footsteps. The question today is not if the computer will be able to imitate the vision processes of the human brain, but when it will happen, and if these abilities will be used for or against humanity. We will wait and see... that is, we will wait, and the computer will see.
