New research provides insights into how people make aesthetic judgments by imparting this ability to artificial intelligence
Do you like thick brush strokes and soft color palettes in impressionist paintings like the paintings of Claude Monet? Or maybe you prefer the bold colors and abstract shapes of Rothko? There is some mystery behind personal taste in art, but recently new research from Caltech shows that a simple computer program can predict which painting a person will like.
The new study, published in the journal Nature Human Behaviour, used Amazon's crowdsourcing platform - Mechanical Turk - to recruit more than 1,500 volunteers to rate paintings from the following streams: impressionism, cubism, abstract painting, and color field. The volunteers' answers were fed into a computer program, and after a training process, the computer was able to predict the volunteers' preferences in art with a degree of significance far exceeding random guesswork.
"I used to think that art appreciation was something personal and subjective so I was surprised by the result," says lead author Koyohito Iigaya, a postdoctoral researcher working at Cal-Tech in Professor John O'Doherty's lab.
The findings not only showed that computers can make such predictions, but also led to new insights into how people judge art.
A glimpse into the mechanism by which people aesthetically judge others
"The main thing is that we get insights into the depth of the mechanism that people use to make aesthetic judgments," says O'Doherty. "In other words, it seems that people use basic features of the image and combine them. This is the first step in understanding the process."
In the study, the team programmed the computer to break down the visual characteristics of the painting into what they call low-level characteristics—characteristics such as contrast, saturation, and hue—alongside high-level characteristics, which require human judgment and include characteristics such as whether the painting is dynamic or still.
"The computer program evaluates how much a certain characteristic affects the decision of how much to like a certain work of art," explains Igaya. When making such decisions, there is a combination of the characteristics of both levels. After evaluating the computer, the software can successfully predict whether a certain person will like a painting they have not seen yet."
In addition, the researchers discovered that the volunteers tended to cluster in three main groups: those who like paintings of objects that exist in reality, such as impressionist paintings; Those who like abstract colorful paintings, like Rothko's paintings; while those who like complex paintings, such as Picasso's cubist paintings. Most people are in the first group, the "Realistic Objects" group. "A lot of people liked the impressionist paintings," says Igaya.
The researchers also discovered that they could train a Deep Convolutional Neural Network (DCNN) to learn to predict the volunteer's artistic preferences with a similar level of accuracy. A DCNN is a type of machine learning software, where the computer is fed a series of images for a training process, so it can learn to classify objects, such as cats versus dogs. Neural networks have units that are interconnected similar to neurons in the brain. By changing the strength of the connection between the units, the network can "learn".
In this case, the deep machine learning approach did not include the high- or low-level visual features used in the first part of the study, so the computer had to "decide" on its own which features to examine.
"In deep-neural-network models, we actually don't know how the network solves a certain task, because the models learn by themselves in a very similar way to the brain," explains Igaya. "On the face of it, this is something that might be very mysterious, but when we looked into the neural network, we could see that it was building the same feature classifications that we chose ourselves." These results suggest the possibility that the features used to determine aesthetic preferences may be a natural product of a brain-like architecture.
"Now we're looking at that possibility, by looking at people's minds as they make the same kind of decisions." says O'Doherty.
In another part of the experiment, the researchers also showed that their simple computer program, already trained on art preferences, could accurately predict which photographs the volunteers would like. They showed volunteers photographs of swimming pools, food, and other scenes, and saw similar results to the results with the paintings. In addition, the researchers showed that the order could also be reversed: after training the software on volunteers' preferences for photographs, they could use the software to predict with high accuracy the subjects' artistic preferences.
While the computer program was able to predict the subjects' preferences in art, the researchers say there is still more to learn about the subtle, nuanced differences that affect everyone's personal taste.
"Some aspects of preferences are unique to each person and we were not able to explain them using the described method," says O'Doherty. "It is possible that that unique component is related to semantic characteristics, or the meaning of the painting, past experiences, and other personal, individual characteristics that may affect the evaluation. It is still possible to identify the characteristics and learn about them in a computer model, but for this a much more detailed study of the preferences of each of the subjects will be needed in a way that it may not be possible to generalize it to other people, as we found here."
To the news of the researchers
More of the topic in Hayadan:
- Artistic AI has sold paintings for over a million dollars
- AI now has a nose too
- A more human artificial intelligence
- Prof. Sharit Kraus from Bar Ilan will receive a grant of 2.3 million euros for research in the field of artificial intelligence
- Prof. Amnon Shashua and the British Demis Hasbis won the Dan David Prize in the Future Dimension
