This is a robot debating with human debating experts when it has to learn a subject from scratch and search its databases for clues to a four-minute argument * When you break down the problem into parts, you find that in most of them it was necessary to break through the front of science * Prof. Chris Reid, an expert in computer science and philosophy says that the computer knows to search for material and produce arguments, but as a speaker, he has much room for improvement
Extended coverage on the science website:
- The computer argues with the debating champions and ends up in a draw
- Over 30 scientific articles and breakthroughs by IBM Haifa researchers. Project Debater milestones
A female voice reminiscent of Amazon's Alexa, or Apple's Siri, an intonation like that of the late Professor Stephen Hawking's speech computer or perhaps the Hal 9000 from the film A Space Odyssey, but also the ability to construct a logical and consistent argument for four minutes, and a lesser ability defend him in a debate with a human professional in the field of debating. This is the first impression from Project Debater that IBM presented this morning (Tuesday) at its development center branch in Givatayim. Another impression I got, and the IBM people admitted, is that the computer knows how to produce excellent arguments, but has difficulty defending them and especially rarely addresses the arguments of human opponents.
The Debater's technology was revealed to the world for the first time two weeks ago in San Francisco - and since the development is Israeli, the researchers in Haifa saw the importance of performing a demonstration in Israel as well.
This is a grand challenge of the GRAND CHALLENGE type that was run and managed from IBM's development center in Haifa in an internal competition held by the company after Watson's success in winning the famous TV game. In that trivia program, the computer had to answer a complex question in a certain way (according to the rules of the program).
The challenge this time is to win a debate competition, a competition familiar to every school student in English-speaking countries. In the original competition, two competitors present an argument for or against the issue brought up for discussion, and then each of the competitors responds to their opponent's arguments in two rounds.
This is what IBM wanted to realize in artificial intelligence and confront it with the best human debaters. As in the trivia game, the computer must provide an answer, but the answer must be complex and reasoned. Builds arguments and presents them in a speech in an extremely high quality synthesized speech, listens to the arguments of the other side, responds to them in another speech - and signs off with a concluding speech: everything is done in a completely automatic process, using artificial intelligence and based on a body of content that stands at its head.
In the demonstration, a computer and two professional debaters (each separately), Yaer Bach and Haya Goldlist Eichler (see a separate article) were given questions on topics well known to the readers of the science website: should we agree to the government increasing the surveillance of people with the development of the appropriate technology (Davitor was unequivocally against, Bach in favor) and whether to allow the use of genetic engineering (Dibitor was in favor, Goldlist Eichler against).
Debater received a list of 40 topics, and given the question that arises for discussion, he is required within a few minutes to analyze all the databases at his disposal and understand "who against whom". After the arguments are presented by the computerized system, the human debater answers them. Even then, a processing time of a few minutes was required to turn the words of the human opponent into text, and to start searching again in the databases for an answer, as we saw, the system had more difficulty with this, and repeated its initial arguments, almost without addressing the opponents' claims.
Dr. Noam Salonim, who was the originator and responsible for the project, noted that over the years the researchers were required to develop many and varied abilities in order to realize the system. "First - the ability to write a speech automatically based on existing data. The system scans a database of 300 million documents, 10 billion sentences - and combines them together into a coherent and convincing speech. The second ability is listening comprehension. The ability to listen to an opponent who speaks for a few minutes in a spontaneous and fast manner, sometimes showing emotion. The third ability is an attempt to build a model for the world of human dilemmas as it is reflected in debates - such as those we conduct every day. We have built a knowledge base that tries to predict human dilemmas. The system tries to identify arguments that you can base on this knowledge."
Dr. Ranit Aharonov emphasized that what sets debater apart from previous gaming machines is the difference in the rules system. Unlike a board game or a trivia game, in Debate it is a much more open system of rules. The system should be able to follow the logic of the arguments presented by the other side. Also, there is no function that gives a score for the system's behavior. It is difficult to determine who won - and it is impossible to use a clear target number or a defined and aspired result. A challenge of this type is much more relevant to a world of problems that do not have black or white answers - and this is actually the human world."
And Dr. Salonim adds: "We actually developed a system that is capable of writing an opinion column in the newspaper. Here, a comparison can be made to artificial intelligence technology in personal assistants. On the cell phone you have to listen to one sentence, on one subject. Here - listen for four minutes to a speech on an emotionally and morally charged topic, identify ideas even when they are not expressed explicitly. The human ability to understand when an argument is for us and when it is against us - is not a simple matter when it comes to a computer. It requires an understanding of the language in all its nuances."
This is perhaps the biggest challenge in which it was necessary to break through the front of science, in areas such as machine learning, because the system is required to understand contexts, the position of the writers on the subject even if they do not write it explicitly, and more.
A new research field - computational argumentation
According to him, "There is a new and exciting field of research here. Following computational biology and positive linguistics, a field of computational argumentation - computational argumentation - has now been born. More than fifty research groups are already working in the world in this field."
Oded Cohen, vice president of IBM worldwide and director of the research laboratory in Haifa, also stood for the Debater's place as a historical landmark in the computing industry and in the history of computer science. "20 years ago IBM's "Deep Blue" defeated Kary Kasparov. In 2011, an IBM computer won the Jepardy! IBM has a tradition of setting milestones in the industry - and defining 'Grand Challenges'. This time too, in deciding on the next challenge, a competition was held between the research laboratories, and a proposal from the laboratory in Israel was selected. We are celebrating our success today: it is not the end yet, but we have managed to put together something astonishing."
Cohen admitted that he hesitated at first in view of the magnitude of the challenge - and his first inclination was not to do it. "But the team managed to convince me. My concerns were dispelled, and the staff did what they promised. This will be a milestone in the information technology and computer science industry. The computer's ability to cope while understanding a deep language, in a dialogue combined with a sense of humor, machine versus human - is an achievement."
Dr. Aya Sofer, vice president and research leader at IBM laboratories worldwide in the field of artificial intelligence said that "When we started the project, it seemed like science fiction. There was a team of researchers who believed in the ability to make this science fiction work. This activity is part of IBM's extensive activity in the world and in Israel in the fields of artificial intelligence and natural language understanding. We believe that - AI presents the potential for positive action, and a positive contribution to the world: the more we use it, the more we trust it - and we can use it more to make decisions."
Sofer went ahead and made it clear that the words spoken by the machine are not pre-recorded and not scripted: the only scripted sentence is the greeting in which the computer opens the discussion, addressing the human contestant. Before the start of the Debate, a writer warned that the system might make a mistake and repeat itself - but the warning turned out to be unnecessary. According to her, "even though the project is still in the development stages, we believe that there is significant progress here - that's why we decided to reveal the issue."
A different direction than that of Google
It is worth noting that compared to the explosion brought by Watson that opened up the field of artificial intelligence which was previously purely academic, IBM is not alone in the forefront of the development of artificial intelligence. Google recently revealed that it has developed an AI technology called Duplex that can conduct phone conversations with a natural voice to book meetings and perform other tasks. According to Dr. Salonim, Project Debater is much more sophisticated than the solutions released by competing companies, which approach the issue from the direction of voice assistants. "It is very easy to understand a one-sentence command, which has an executive component, compared to four minutes of abstract or complex arguments by a human claimant."
room for improvement
In any case, most of the speakers on behalf of IBM emphasized that this is an intermediate stage, where the system is reliable enough to reveal it, and it is clear that there is still a need for development.
Prof. Chris Reid - A researcher at the Institute of Computer Science and the Department of Philosophy at the University of Dundee in Scotland and who made the announcement about two weeks ago in San Francisco said that Project Debater has achieved significant progress in areas such as text search for arguments (argument mining) in combination with technical solutions such as grammar corrections that involve pasting parts of a sentence, but as a speaker, the computer is A chick tweeting its first tiny tweets.
According to him, the system includes only the most basic concept of argument structure, and often deviates from the main topic. He has no regard for his audience, nor for his opponent, and has no way of utilizing any of the hundreds of clever rhetorical techniques that help human debaters win.
However, Prof. Reid and also the speakers from IBM emphasized that the purpose of the development is not to win in debate clubs, but to help human teams make better decisions, whether in the police situation room, the intelligence analysis bunker, or in the classrooms. He positively notes the idea of increasing the ability of AI systems to make evidence-based decisions, but he estimates that the distance to the development of such technology is still great.
More of the topic in Hayadan:
One response
raises reflections on human nature,
What is the average debating ability of a person even without comparison to the Debater system,
Do they repeat the same claims over and over in the discussion as an answer,
What is the percentage of people for whom the discussion is more of a competitive argument so that they will always choose the counterargument
Regardless of the topic, at what point do you think you are in front of a static computer program and not a dynamic person,
What is the percentage of people (sometimes demagogues) who raise their voice to convince,
And also from the side of listening, what is the average ability to be convinced by a logical argument versus an argument that appeals more to emotion,
The pool of knowledge and basic concepts is also an important part of the quality of the discussion or the bias of people, which can certainly be in systems that are much better than a person in the quality and ability of the discussion, the ability to doubt not for the sake of doubt, but for the sake of striving to understand reality, which is the advantage of the discussion where you can To convince and to be convinced becomes a kind of study,