in the eyes of the beholder

An average person can spot a white bear against a snowy background, understand that the pattern of black spots on a Dalmatian dog belongs to one animal, and recognize the profile of a person who previously only saw it from the front. But what the brain can do relatively easily, turns out to be a very difficult task for the computer 

From the right: Prof. Ahi Brandt and Prof. Ronan Betsari. second look

 An average person can spot a white bear against a snowy background, realize that the pattern of black spots on a Dalmatian dog belongs to one animal, and recognize the profile of a person who previously only saw it from the front. But what the brain can do relatively easily, turns out to be a very difficult task for the computer. Changes in the lighting conditions, or in the angle from which the object is seen, may mislead the computer, making it "think" that there is a completely new object in front of it.
A group of scientists from the Weizmann Institute of Science, led by Prof. Ronan Betsari and Prof. Achi Brand from the Department of Computer Science and Applied Mathematics, developed a method that allows a computer to recognize objects even when they are lit in different ways and photographed from different angles. This is a multi-stage process that enables object recognition by dividing the general image into segments, and rebuilding the overall image, "from the bottom up": the computer begins by comparing the individual pixels that make up the image, and divides them into groups based on their similarity in brightness. The resulting groups go through another comparison process, which includes additional features such as texture, shape and more. Groups that share common characteristics accumulate into larger and larger segments, and at each stage the indicators for comparison become more numerous and more complex. At the end of the process of division into segments and classification into groups, the computer is able to distinguish between the object and the background.
After achieving good object discrimination results, the scientists moved on to the next step: object recognition. They tasked the computer with a difficult task: to scan a large database, and find a pair of glasses identical to the one shown to it at the beginning of the process. To perform the task, the computer decomposed the image of the glasses into segments, compared them to the segments of all the glasses in the database, and looked for glasses with similar features to those presented to it. At the end of the process, the computer was able to locate the identical glasses, even when they were presented at a different angle or in a slightly different way from the original image. In addition to Prof. Batsari and Prof. Ahi Brandt, research student Eitan Sharon, Dr. Marev Galon and Dr. Dalia Sharon participated in this work. The research findings were recently published in the scientific journal Nature. 
Now, when the computer is able to distinguish objects and identify them, is it possible to harness these abilities for medical needs, and teach the computer to recognize typical signs of various diseases? Is it possible to replace the trained eye and the doctor's diagnostic ability in the process of dividing into segments and classifying them? To test the possibilities in this area, the scientists tried to get the computer to identify damaged areas in the brains of multiple sclerosis patients. In these patients, the myelin coating that surrounds the nerve cells in the brain is damaged, an injury that can be seen through MRI magnetic resonance scans. The scans provide the doctor with a series of cross-sectional images of the brain. In the method used today, he had to examine all the photos one by one, and mark the damaged areas. "We hope that in the future it will be possible to assign this task to the computer," says Prof. Batsari. "In the nearer term, the system will be able to help the doctor identify damaged areas of the brain, and provide him with data on their location and volume. These data indicate the patient's condition and the effectiveness of the treatment."
In order to tackle this task, Prof. Moshe Gemouri, a radiologist from the Hadassah Ein-Karem Medical Center, joined the research team. With his help, the computer scientists made adjustments to the software. In the first stage, they "taught" her to operate in three dimensions, that is, to collect all the cross-sectional images obtained from the MRI scans, build a three-dimensional image from them, and analyze it. The data analysis process at this stage consists of the same steps used in the object recognition process: the brain image is broken down into segments, and each segment is characterized according to a list of features defined by expert radiologists: brightness, texture, shape and location in the brain. After that, the classification is done: the computer examines the different segments, and decides whether the surveyed area is damaged or normal. The decision is made on the basis of a "learning process" that the computer went through: before it is sent to perform the task, it receives brain images in which the hardened areas are marked, and thus it learns to characterize them. First experiments in this field yielded good results: identification of sclerotic areas using a computer overlaps 70%-60% with that obtained by a specialist doctor (a rate similar to the degree of agreement obtained between two doctors). The results of the research, carried out by research student Ayelet Axelrod, were published at the International Association for Computer Science conference, which focused on computer vision and pattern recognition.
Computerized identification of sclerotic areas in the brain is considered a relatively complex task due to the diversity in the shape, location, color, and texture of these areas. These are amorphous structures, found within the brain tissue, which is also made up of varieties, textures and complex shapes. Contrary to the conventional approach, which tried to deal with this task by examining individual pixels, the system developed by the institute's scientists is capable of characterizing entire areas and sections, and thus manages to deal with the task efficiently.
This method may form a basis for computerized applications in the diagnosis of various phenomena and diseases whose symptoms are visible to the eye (or the computer's eye). These include MRI scans, CT scans and more. Indeed, these days scientists are developing special software capable of detecting liver tumors.
Prof. Batsari Mikve, that in the future the computer will be able to function as a multidisciplinary personal doctor, who will be found and operate in every clinic, assisting in the diagnosis and monitoring of various diseases. The thoughtful personal approach that characterizes good family doctors will be difficult to transfer to the computerized system, but it's always good to have high challenges that you can strive to reach.