who is the painter

Even those who are not well versed in art at all can learn to differentiate between several artists who differ in their style and the themes of their work. Can the computer also do this?

Israel Benjamin

There are people who, with a quick look at a painting they have never seen, can determine not only who the painter is, but also in what period of the artist's life the work was created. This ability requires a lot of knowledge, years of experience, and probably also unique talents. Nevertheless, it is possible to easily teach even a "man from the street", who is not well versed in art at all, to differentiate between several artists who are very different in their style and in the subjects of their work, such as Monet, Da Vinci and Kandinsky.

We do not know how that "man from the street" does it, and what is the difference between the way he distinguishes between painters and the way the expert does it. This is an interesting question in neuroscience and cognitive psychology, and precisely for this reason it also poses an interesting challenge for artificial intelligence: is it possible to bring computers at least to the level of the ability of the common man?

Identification by the small parts
Prof. Daniel Kern, from the Department of Computer Science at the University of Haifa, succeeded in this task. To test the success of the software he developed, he set her the challenge of distinguishing between the artists Magritte, Rembrandt, Van Gogh, Picasso and Dali. The software learned to identify the artists based on ten paintings by each artist, after which it was presented with 10-20 paintings for "identification order".

In the language of the field of computer learning, the first group of paintings is the "learning group" according to which the computer builds its "perception" of the style of each of the painters, and the second group is the "test group", which is used to examine how successful the learning was. After the software processed the learning group and associated typical characteristics to each of the artists, it was able to classify the paintings in the test group with a success rate of 86%.

Prof. Keren's software is not the first in this field, but it differs from its predecessors in that it is extremely fast, and in that it uses local methods. "Local method" means identification according to the properties of small parts of the drawing, and not according to properties that can be calculated only according to the complete drawing or large areas of it. Such methods are more sensitive to the fine textures and patterns of painting and coloring, and less sensitive to light-shadow contrasts or the composition of the work (eg, many or few details, subjects against a background). This is similar to associating a manuscript with the person who wrote it according to the shape of certain letters, and not according to the directions of the lines, the spaces between the lines or the contents.


Local features and tournaments
In order to identify the local characteristics of the images, the software divided each image into small areas, and performed image processing for each of them using a mathematical method called DCT (Discrete Cosine Transform). This method represents each area by connecting wave functions (periods) at different wavelengths, and finds the amplitude (amplitude) of those waves whose connection creates an image close to the image of the selected area from the original image. This method distinguishes, for example, between fine and dense lines and rough and distant lines, between vertical and horizontal or diagonal lines.

In the learning process, after each image was characterized according to the wave patterns that make it up, the software found for each pair of artists the wave patterns that give maximum difference. So, for example, if Dali's paintings contain much more dense vertical lines than Van Gogh's paintings, the vertical lines will be chosen as one of the distinguishing characteristics between these two artists. On the other hand, if dense vertical lines are found to a similar degree in the paintings of Dali and Picasso, they would not be chosen for this pair, and in their place would appear perhaps circular brushstrokes of the type that distinguished Van Gogh. As you can understand from the description, the whole process is automatic, without the intervention of a human expert.

After the learning phase comes the test phase. At this stage, for an image whose painter is unknown, the software performs the same DCT processing, then "invites the painters to a tournament", in this way: it chooses one pair of painters who must "compete" for the image by checking the characteristics of the image (the waveforms). The pair of painters competes for the image by checking those features found in the learning phase that provide maximum discrimination for that pair. The tournament continues for additional couples, until the "winner" is announced.

Fake in good taste
Recently, an Italian art collector approached Prof. Kern with the question of whether the software would be able to identify fakes. As we know, fake art paintings are a common phenomenon, and not too long ago we read in the newspaper about a museum or private collection that discovers that they paid a fortune for an inauthentic painting.

Although the method presented here will not be able to declare "fake!" On paintings whose subjects are clearly unsuitable for the painter (such as an astronaut in a work allegedly painted by Rembrandt), it is tempting to think that local mathematical characteristics are unknown to the forgers, and even if they tried they would have great difficulty creating exactly the same wave patterns that were second nature to the original painter. To know if this method will actually succeed, one must first collect fake paintings created as an imitation of the brushwork of a famous painter, and compare their characteristics with those of authentic paintings by that artist. If the experiment is successful, and the software detects forgeries, the method will be added to the toolbox of art detectives, which also includes chemical analyses, photographs at different wavelengths, a deep orientation in the history of art, and more.

We have seen that this software can differentiate between paintings by different artists, and maybe it can also differentiate between an artist and his imitators. It is likely that she will also be able to differentiate between different periods in the artistic career of the same painter, if these periods differ significantly in style. Will she also be able to differentiate between good and bad paintings? of course not. Even if we avoid the question "Who decides what is a good painting?" By trying to please the taste of one art critic, we still cannot summarize his preferences in phrases like "a good painting is a painting that consists mainly of these patterns: thin diagonal lines with a moderate distance between them; broad brush strokes that curve upwards; …”. A computerized definition of artistic taste will need a completely different approach, which is hard to guess what it will be based on.

  
Please draw me an electronic sheep
However, Keren's software possesses at least one basic feature of an art critic: the ability to identify a particular painter's style. How far can a computer go to the other side of the easel, and play the role of the painter?

In this section, some programs for drawing by computer have been mentioned in the past (see: "software that writes software", "the works of artists like us"), such as Kandid. What most of these programs have in common is the cooperation between a person and a computer: the computer is equipped with several image creation processes (for example, choosing textures and colors, dividing the drawing surface into areas, mathematical functions for creating lines or color changes).

From among these processes, the computer randomly chooses which processes to run, in which order, and with which numerical data to enter them (the numerical data will affect, for example, the rainbow of colors that will be selected, the length of the lines or their degree of curvature, etc.). Each such selection creates a new drawing. After several drawings have been created in this way, the computer shows them to the person. The person chooses the ones they like best, the computer builds new paintings by favoring the processes that created the chosen paintings, and so the process continues until the person chooses a painting as the result of the joint creation. With this method, one can see the choices of the processes and the numerical data fed to them as the "genome" that guides the development of the painting, and the person as the environment in which the painting develops. If the person loves the painting, we can say that the painting has survived, and therefore the next generation of paintings will inherit part of the "genome" of the painting, hybridizing with the "genetic load" of other paintings that have also survived the examining eye of man.

Since this is a process of joint creation, the answer to the question "Who is the painter?" It is the division of rights - or blame - between the computer and the person. Is it possible to use a similar idea to create an entire creation of the computer? Steve Diapola, from the Faculty of Interactive Art and Technology at Simon Fraser University (Simon Fraser) in Canada, chose to do this by replacing human judgment in comparing the drawings created by the computer to a given drawing. The comparison is made with the help of a complex model built according to ideas from the world of painting: the relation of the object to the background, matching colors according to shade and according to "temperature", and the relation between colors and complementary colors. The painting chosen for comparison is a portrait of Darwin, so in fact the computer is trying to imitate an artist painting an impressionistic picture while drawing inspiration from a portrait of a person. The resulting images are not intended to be identical to the original portrait (a simple copy can give this result), but to be a creative reference to that portrait. The results can be seen in the picture above.

Due to the complexity of the drawing and comparison processes, it took about 50 working days of one desktop computer to create these results. For the most part, you can see the bright object in the center of the picture, which corresponds to Darwin's face and beard, with color changes corresponding to one or both eyes, but there is also a great difference in atmosphere and "style". All the images were created without human intervention in the creation process, except of course for the selection of the initial portrait. Here you can already answer the question "Who is the painter?" In an unequivocal answer - "the computer", but this is on the condition that we agree that these are indeed works whose creator deserves the title "painter". As we know, it is difficult to agree on such questions even when it is known that the creator is human.

Israel Binyamini works at ClickSoftware developing advanced optimization methods.