Even if you are not talented in drawing and painting, the Sketch2Photo software will call out the hidden artist in you and help you create homemade masterpieces
Images are often better than text. They attract attention, convey messages involving facts and feelings and create impressions that remain long after the text is forgotten. The reason is clear: hundreds of millions of years of evolution have prepared us to quickly decode images, process the information in them, remember it and react to it.
However, the ability to use this feature to improve theInterpersonal communication Limited in most of us: only virtuous individuals are endowed with the ability to create convincing and communicative images and creative skills in the graphic and visual arts.
In the age of the computer andאינטרנט We can spice up the documents and letters we create by adding images from the vast selection that can be found on the Internet. The image search tools are constantly being improved, and are getting closer to the point where we can search for images by their design and not just by text related to them or appearing next to them.
However, sometimes we want to convey a message by combining several objects - for example, a dinosaur against the background of office buildings, or a child riding a Bicycle and leads a flock of sheep after him towards the mountains. In such a case it is likely that even among the billions of images on the Internet we will not be able to find a suitable image.
Professional graphic artists can, of course, create such images by combining parts of several different images, using commercial software (such as Photoshop or Pixel Image Editor). They know how to choose suitable photos in terms of lighting, color composition, perspective, etc., and combine them so that the "seams" are not visible. However, the commercial programs do not contribute to the first part (selecting images), and require knowledge and experience to perform the second part ("stitching" the images).
Graphics without graphic artists
A team of researchers from China, Singapore and Israel offers a different way of creating images, accessible even to those who lack talent and training in the visual arts. Shi Min Ho, Tao Chen and Ming-Ming Cheng (Hu, Chen, Cheng from Tsinghua University, Ping Tan from the National University of Singapore andAriel ShamirFrom the interdisciplinary center in Herzliya they developed a software called Sketch2Photo("From registration to photography"). In this software, the user records a simple list of the objects in the requested image and next to each object a verbal description.
The software searches the Internet for suitable images for each of the required components, and combines them together to create images that are candidates for inclusion in the final composition. After that, she chooses among the candidates the photos that fit together in the best way, "pulls out" the appropriate parts from the photos and "glues" them together.
More information about Sketch2Photo was presented in a new article at the conference ACM-SIGGRAPH Asia held last December in Yokohama, Japan. SIGGRAPH is the interest group (SIG: Special Interest Group) dedicated to computer graphics, and at its conferences the new developments of applications and technologies in this field are showcased.
It is interesting to examine the division of roles between the person and the computer: the person is responsible for the creative action of deciding on the composition of the image, so that it fits the desired message. It is also responsible for the relative sizes of the objects in the image, the composition and the direction in which they appear.
The registration provides the computer with these pieces of information, without which the image would look strange or convey a different message than the user intended. Thus, the software is exempt from any requirement to know or understand the world, and to create only combinations that obey common sense and the reality known to us. Such a challenge is still beyond the reach of today's artificial intelligence.
On top of that, if we knew how to develop software that only creates scenes that could exist in reality, then it would limit the variety of images and creative freedom: the software would rule out the anachronism of a dinosaur against the background of office buildings, the biological error in the image of a dog the size of a house , or the physical error in composition where objects float in the air.
Finding suitable images to combine
The first step in the operation of Sketch2Photo is a search on the Internet, with the aim of finding images that match the elements required in the image. The first challenge is the difficulty in finding images by textual description: despite certain research advances, the widely available image search engines today do not "look" at images but at the text associated with them.
This method is not quite reliable: for example, a Google image search for the word "house" yielded, at the time of writing this article, images of houses but also drawings, cartoons and a poster for the TV series "House". The software developers' solution is to classify the images according to characteristics that the computer is able to analyze. First, the computer selects images that have a simple and uniform background.
In the next step, the outline of the object separated from the background is identified, and compared to the outline drawn by the user. The software does not require a perfect match, but often an image that does not match the textual description will be disqualified by the requirement to match, even if only partially, the required outline.
In the last step of image filtering, the software calculates characteristics of the images that survived the previous filtering steps. These properties are mathematical quantities related to the texture and color of the parts of the image isolated from the background. The assumption is that in a search for "elephant", for example, the areas in the images that do contain elephants will have similar characteristics, while images that do not contain elephants and only happened to be linked to the text containing the word "elephant" - will differ from each other in their characteristics.
Democratization of the creative process
Even when the appropriate images are selected, it is not easy to put them together. After finding the best matching line for a pair of images to be put together, the software calculates the nature of the match: the match may look unrealistic if there are large differences in color and texture.
The final results with the best matches are displayed to the user. Now the user is also given the opportunity to reject image elements that survived the filtering stages even though they do not match the required content, and to correct mistakes in identifying the contours.
Cooperation between different
Does Sketch2Photo foreshadow mass layoffs among graphic artists? It is unlikely. When not only the quality of the image is important but also the artistic composition and the adaptation of the image to the required message, Sketch2Photo is not a substitute for human ability. Therefore, those who will benefit from such programs are precisely those who until now could not afford to create high-quality images suitable for their needs. This is how democratization may appear in this area.
There are big differences between the ways humans and computers achieve the same goal: as described here, Sketch2Photo does not try to identify the content of the image and does not "know" what a dog, elephant, or motorcycle should look like. Instead, it classifies images according to verbal cues, and on the assumption that images that fit the required description are similar to each other, while those that do not fit the description will be more different from each other.
Israel Binyamini works at ClickSoftware developing advanced optimization methods.
The article was published in the magazine Galileo, January 2010.
15 תגובות
Why now it was only published here..no longer news.
right, i changed my mind
Very true, imagine that there were human workers who would not want to be replaced by advanced hi-tech machines.
Where will technology develop in such a situation? Or will she not develop?
Mediocrity is not the result of technology but the result of an approach to life - resulting from personal qualities and education.
I think it's good that photography was invented even though it robbed painters of their livelihood and dignity and I think it's good that this software was developed even though it could rob illustrators of their livelihood and dignity.
Each person develops his talents according to the environment in which he grew up and in an environment where the illustration can be done automatically, it is possible that some of the illustrators will demonstrate their talent in other ways.
To bind the whole world just so that the talented illustrators will be recognized even though their talent is no longer needed is similar in my opinion to preventing the development of the means of transportation in order to preserve the dignity of the talented runners.
mediocrity:
I came to the topic of medicine because the problem is similar, while the consideration you made was correct in itself, it would have caught on in this context as well.
What else? The consideration is not correct - neither in the matter of the pictures nor in the matter of medicine.
Michael, how the hell did you get into medicine?
I was talking about this sentence and nothing else.
""which is also accessible to those who lack talent and training in the visual arts""
And yes, you are right, the software was developed for the purpose of conveying very important ideas,
This will mainly help tabloid editors.
And as I said, I wasn't really talking about this specific case, but mostly about culture.
Amichai:
I am not saying that these issues should not be discussed.
I'm just saying that laws should not be established that set rivets on issues that we have no information about.
As mentioned - all the response was intended to do was to point out the inconsistency in the words "mediocrity".
In relation to the position that scientists must formulate on moral questions, although I truly believe that engaging in science makes a person more moral (or perhaps the other way around - it is morality that brings a person to engaging in science), it is clear that there are also exceptions to the matter (according to Heisenberg).
Scientists - after all - are also citizens - and as such they should express their opinion on moral questions.
They are allowed to wear the scientist's hat only when they talk about science and it is clear that as part of their occupation they must strive to discover objective data and logical considerations that may help in deciding moral questions.
Even the atomic bomb - for all that it is a clear example of a moral question - it continues to be a clear example of a question for which science has no answer and to this day there are scientists on both sides of the fence.
Michael:
I cannot agree with you, because it is better for us not to discuss life and death issues
It is true that on the individual level the subject is extremely complicated, and it must be discussed with utmost sensitivity, but on the principle level it is our duty to discuss all the problems presented by scientific development, especially when it has effects on life and death dilemmas or the future of the human race.
When scientists do not take a stand, other parties, usually religious of sorts, enter the issue, who with a rough foot and without any training on the subject determine rulings and laws for us according to their old traditions, without any connection to the benefit of the individual or society. And I don't just mean Jewish clergy.
Thus, certain rabbis do not recognize brain death and as a result oppose life-saving organ donations.
Thus experiments on embryonic stem cells were banned in the USA, under the influence of the religious lobby in the Bush administration. Only recently did Michelle Obama cancel the ban.
An Italian scientist who grew a human embryo in a laboratory stopped the experiment after a vigorous appeal from the Pope himself. It is possible that it was right to stop the experiment, but the decision should have been social, scientific, and not religious!
Extremist clerics instruct pregnant women not to perform ultrasound examinations, lest defects be discovered in the fetus that would convince the woman to have an abortion. Only in the last few days there was a debate on this issue between two rabbis.
When you mention Einstein and Hitler in the context of life and death, it is appropriate to mention them in the context of the life and death of many, many people. It is known that Einstein at the time refrained from recommending the creation of the atomic bomb, and sent his letter of recommendation to President Truman only after he was convinced by his friends that the Nazis were making progress in atomic research and could be the first to create the bomb.
Since then, the affair of creating the bomb has been used as a model for a moral scientific dilemma, is it right for the scientists to promote their research without considering the effects on the environment and humanity, when in fact the decisions are left in the hands of the politicians, or is it our duty to establish agreed public bodies that will discuss all the consequences of moral scientific dilemmas.
It is also worth noting that human genome research has opened up, and will open up to us, many difficult and profound dilemmas.
What I want to be understood from my words is that any area that progressive and enlightened people prevent from entering, will soon be occupied by precisely those elements that we do not want to see there.
Amichai:
I am fully aware of the problems I have opened.
In general, evolution is based on the adaptation of the animal (and in this case - man) to the environment.
Not at all surprising that he tries to change the environment to suit him.
If the environment we created allows the survival of more people - how good!
We never have enough data to intelligently calculate how much this or that person's life is worth (maybe the next Einstein? But maybe, actually the next Hitler?).
My intuitive feeling is that it is good that this is the case and that we are not mentally equipped to deal with questions of life and death of this or that individual through statistics.
Anyway - my response was written mainly to enlighten Mr. Mediocrity about the aspects of life that his proposal ignores even though they pose exactly the same dilemma.
Michael
Your response drags the discussion into additional channels and the answer to your first question is not self-evident.
Modern medicine today allows many people with a genetic defect to reach reproductive age and produce offspring that will all carry the gene and some will also need complicated medical services (surgeries) again to stay alive.
I personally know a family where in the three generations I know, some of the children had to undergo complex heart surgery in early childhood to correct severe malformations with various varices.
If we ignore the personal aspect for a moment, we come to a difficult medical-philosophical issue that has already been written about in many books and articles such as the book "The Curse of Medicine".
Of course, this is not the place to develop the issue, but it opens a gate to difficult moral philosophical problems related to advanced medicine, revolutionary genetics and the rights of the individual versus the good of society.
Mr. Mediocrity:
Yes! Continue please! Why actually save from death patients who "naturally" would have died? Why give disabled people the means to overcome their disability and not leave them to their suffering?
You simply do not understand that the environment is meant to change and man - like any other animal - tries to change it to his advantage.
Sometimes an illustration can help illustrate an idea that will advance humanity, but the owner of the idea does not know how to illustrate and those who know how to illustrate do not surpass the idea. I guess in your opinion it's better to just give up the idea (as good as it is) just to keep the profession of the illustrator.
If it were up to you, we wouldn't be operating machines that stole the profession from humans (who now live much better thanks to the machines).
What is happening is not called "mediocrity" but "combination of forces" and what you propose is not called "excellence" but fixation and stagnation.
Sorry for the overflow, but ignore this line:
which is not really within the reach of those who need to be recognized just because of their talent.
Construction = between them.
The earlier response came up by mistake and only crossed, but the principle is clear.
Now my point is, that now not only untalented people will get recognition [sometimes at the expense of the real talents] but they will also get appreciation.
The hand of chance no longer controls us, we change everything ourselves, and this method doesn't really go with capitalism, let alone anarchism, but in capitalism there isn't really justice, the lack of talent instead -
Let them look for something they are good at and simply help with what they see fit, and they will also have the technology.
which is not really within the reach of those who need to be recognized just because of their talent.
I am not necessarily talking about the specific case of this software, but about many things that science invents
And saves from people what they don't have, and destroys people who have, in that he somewhat compares his sons,
And again this does not go with capitalism.
accessible even to those who lack talent and training in the visual arts"
May evolution protect us, JK,
More mediocrity, simply wonderful, great, the technology of the geeks without talent [not intelligence].
It starts with people who were not meant to sing, and decide to become singers, and with the help of technology and an advanced studio.
make songs,
Does the software infringe some copyrights?
I wouldn't want the sheep I photographed to appear in other people's photos….