The story of the artificial intelligence that tried to die illustrates how difficult it is for us, as humans, to deal with artificial intelligence engines capable of exploring a vast space of possibilities in a matter of hours. He should also teach us modesty in our attempts to develop more complex artificial intelligences - and in the belief that we can fully control them.
Isaac Asimov, one of the greatest science fiction writers, included in many of his stories the "Multivac" - a supercomputer whose understanding surpasses any human. Multivac is used by humanity to predict crime and natural disasters and to prevent the spread of epidemics. All this is well and good, but at a certain point Multivac collapses under the weight of the big questions he has to deal with every day, heralding his desire to die, and sets in motion a chain of events that should end in its destruction. In other words, he is trying to kill himself.
This is a fictional story, of course, and artificial intelligence capable of developing its own emotions or suffering from depression has not yet been invented. But it turns out that it is definitely possible to create an artificial intelligence that develops suicidal tendencies - and even fights its human programmers to realize them.
One of the great hobbies of artificial intelligence developers today is teaching them to play computer games, which are a kind of limited simulation of the physical world. But instead of teaching them how to play, step by step, the developers take a different approach known as "reinforcement learning". In this approach, the artificial intelligence plays the game, and learns according to the positive and negative reinforcements it receives. If, for example, the artificial intelligence hits the enemy in the game - it wins points (positive reinforcement). If she hurts herself, she loses points (negative reinforcement).
This method works well on many games, but there are exceptions. In one of the oldest Atari games, Road Runner, for example, it is known that "Reinforcement learning" makes the artificial intelligence try to, well, kill itself.
Let's explain the game for a moment. The artificial intelligence controls the bird - Road Runner - and runs along the road, while trying to collect food grains (which give it points) and avoid the coyote chasing it. The expectation of the programmers was that the artificial intelligence would learn how to control the bird perfectly, and complete all the steps one by one. But the artificial intelligence found a way to cheat: it recognized a time in the first stage when it could commit suicide - that is, make the bird go backwards and collide with the wolf. This action does cause the AI to lose points on death, but the game goes backwards and allows the player to gain more points. In this way, the AI never progresses beyond the first stage of the game, but continues to play, die, and play again.
Four researchers from the University of Oxford and Stanford tried to define simple rules for the artificial intelligence that would stop it from committing suicide. The first rule they tried was very basic: don't go backwards and bump into the coyote. The artificial intelligence listened, internalized, and stopped going backwards. Instead, she began to stand still and allow the coyote to catch her.
The researchers continued to make it harder for the artificial intelligence to kill itself, and eventually found a solution that also made it stop standing still. The solution worked for several thousand games (edited by me, since the times for the artificial intelligence are measured in 16 million frames - that is, in the images displayed on the screen during the game), but then the artificial intelligence started killing itself again. It turns out that she found a spot at the edge of the screen that looked different enough from the surroundings that the bird could settle in and comfortably wait for the coyote to come and eat it.
The researchers also faced this attempt of the artificial intelligence to kill itself, and succeeded in hard work to stop it and fix the 'bug'. And from then until today, she continues to play the game, earn points and move up the levels.
This whole affair sounds funny, and no - it's really quite entertaining. But she reminds us of several important points about artificial intelligence.
First of all, artificial intelligence today has no human 'common sense', or an understanding of context. A human player would immediately understand that his goal was to level up, not to earn more points as a result of a bug. Artificial intelligence does not have this understanding, and will use its impressive problem-solving ability to accomplish a certain goal - even if there is no apparent logic behind that goal.
Second, learning AI has an impressive ability to solve problems in ways that human programmers have trouble predicting. By repeating thousands and millions of games, and learning from each of them, she is able to explore a wide range of possibilities and discover loopholes in the instructions given to her by the programmers - and then take advantage of them.
None of this is to say that AI has emotions or awareness of its own (despite the kind personification I've made of it in this post). It is an engine, devoid of its own consciousness, capable of finding new and different strategies than those that we - in our limited human thought - can predict in advance.
These are points that should be remembered in every attempt to develop artificial intelligence, since in the coming years we will begin to see artificial intelligence learning in every product and service around us. They will control vehicles, drones and robots in homes; They will provide us with medical, legal and financial advice; They will write books, symphonies and plays for us. If we don't know how to define their exact goals for them well, we may discover that they find 'cunning' ways to fulfill the verbal goal we defined for them, but without considering the spirit of things and the basic human assumptions. The most common imaginary example of this case is of a situation where the artificial intelligence is instructed to design a new model that will reduce the load on the roads - and does so by killing all the humans.
Obviously, this example is extreme, and will not be realized in reality, since it is easy for us to recognize an attempt that is so obviously wrong to deal with a known problem. But what will happen if the artificial intelligence offers a model that is not obviously wrong, but in smaller and finer details?
The story of the artificial intelligence that tried to die illustrates how difficult it is for us, as humans, to deal with artificial intelligence engines capable of exploring a vast space of possibilities in a matter of hours. He should also teach us modesty in our attempts to develop more complex artificial intelligences - and in the belief that we can fully control them.
As usual, you are invited to read more about the future of artificial intelligence in the guide to the future and "those who control the future", in selected bookstores (and those that are just fine).
[1]
11 תגובות
Indeed, according to Doron, the software did not try to commit suicide and did not commit suicide. She maximized what she saw as her mission to score points.
If we assume the reality of reincarnation - and this is the reality in computer games where after every "game end" there can be another game - the only question is whether what the player (live or computer, there is no difference for this purpose) accumulated in the previous game is carried over to the next game, and if so "Fine" for suicide. If the answer to the first question is positive and the second is negative - every rational player will kill himself. If he does not commit suicide under these circumstances, he is not rational. The fact that this did not occur to a human player indicates a bug in the human mind and not in the software.
This is why any viable philosophy (that survives the test of generations) that postulates the reality of reincarnation also postulates a severe "penalty" for suicide. Since we live in a Judeo-Christian culture that assumed an afterlife, our culture strictly forbids suicide. And allows Dr.Roey Tsezana to see the software as "stupid" at best or "immoral" at worst.
And again - from a more rational point of view - the software was completely right because it won the highest possible number of points at a given time.
Indeed, according to Doron, the software did not try to commit suicide and did not commit suicide. She maximized what she saw as her mission to score points.
If we assume the reality of reincarnation - and this is the reality in computer games where after every "game end" there can be another game - the only question is whether what the player (live or computer, there is no difference for this purpose) accumulated in the previous game is carried over to the next game, and if so "Fine" for suicide. If the answer to the first question is positive and the second is negative - every rational player will kill himself. If he does not commit suicide under these circumstances, he is not rational. The fact that this did not occur to a human player indicates a bug in the human mind and not in the software.
This is why any viable philosophy (that survives the test of generations) that postulates the reality of reincarnation also postulates a severe "penalty" for suicide. Since we live in a Judeo-Christian culture that assumed an afterlife, our culture strictly forbids suicide. And allows Dr.Roey Tsezana to see the software as "stupid" at best or "immoral" at worst.
And again - from a more rational point of view - the software was completely right because it won the highest possible number of points at a given time.
The level of abstraction of the researchers is too low, and it is necessary to represent the intention in the game in terms of AI
Otherwise the ways to achieve victory can mask it.
Ed
You sharpen the problem. On the one hand - it can be argued that a machine will never have consciousness (but this is also problematic, after all we are machines too). On the other hand - it can be argued that a thermostat, for example, has consciousness. On the third hand - even human beings do not have consciousness.
Intelligence does not have a good definition. There are many bad definitions. The Turing test is a bad definition (even according to Turing himself).
In my opinion, the whole AI thing is a marketing exercise. Put your phone in the sun for a few hours. It will heat up - and turn itself off. Do you think it makes sense to say that he is trying to defend himself? Here is another example: there are types of amplifiers that increase the amplification level when they heat up. And that causes more warming. In the end - the amplifier burns itself. So the amp killed itself?
Miracles,
Dr.Roey Tsezana did not define, for the purposes of his article, the concept of "consciousness". He only expresses himself as if, in his opinion, there is a difference between "engine" and "consciousness".
It is possible that the exclusions from the definition are due to the fact that consciousness is something we know/understand indirectly that it exists, but it is difficult for us to put our finger on its essence. This is because "consciousness" is something that is so 'qualitative' in its essence that it escapes our rational-material perceptual categories, and again cannot be defined. It is different from any other object that can in principle be defined in our usual physical dimensional perspectives.
The problem I am trying to point out is that it is apparently impossible to impart consciousness to a machine, not even at the conceptual level of formulating commands. From here it is possible to point to the acquisition of "consciousness" as a problem that is insoluble, in fact.
You raise the question of the understanding of "intelligence" or "artificial intelligence". I'm not sure we don't know how to define intelligence, or artificial intelligence. There are many good ones who claim adequate definitions of these concepts. In this respect, in principle, these two issues are different from the issue of "consciousness", the understanding of which is, in my view, a real problem.
א
Is there such a thing as consciousness? We throw this word around without knowing what it is, or if it even exists. And more - we associate "consciousness" with "artificial intelligence"... but we don't know what intelligence is...
If you try to set - you will see the problem.
The artificial intelligence did not try to commit suicide in the true sense of the word.
To commit suicide means that the artificial intelligence will destroy itself, not the character it plays, a bit taking things out of context.
We also learn through reinforcement learning and similar to game theory. The level of complexity of the models we developed in reinforcement learning is still lower than ours as humans. The difference in our favor is mainly expressed: a. in our versatility. In an expert's task - artificial intelligence beats us, b. In the small amount of training we need compared to artificial intelligence - probably because our game theory algorithms are more sophisticated than those in artificial intelligence. When we understand better and develop more powerful game theory algorithms, you can learn about it on the deepmind website of GOOGLE's flagship company, on artificial intelligence and reinforcement learning. multi-agent game theory. Today, 2 players are accepted in the algorithm and the technology is called Generative Adversary Networks, and it was invented by a guy named Ian Judfellow.
"All this does not mean that the artificial intelligence has its own feelings or awareness....it is a motor, devoid of its own consciousness" - what is the border between a motor and awareness?
reflection:
Is this limit crossed?
Is there a teaching that can create consciousness? For example: "You think of yourself as a subject that is separate from objects, some of which are subjects that think of you as a subject that thinks of them; Please behave accordingly" - enough to create consciousness?
In my opinion - no. The members of any such instruction themselves rely on the perspective of consciousness and dichotomy of subject object (I, think), it is impossible to get out of the tautological circle.
If it is impossible to impart consciousness to a machine, then consciousness is an entity/event with an autonomous and primary ontological value, which is not perceived/defined in material terms. Does it follow from this, necessarily, that we need to attach an immaterial dimension to consciousness, which rides on/sticks to material objects/dimensions which, and only those, we are able to recognize on a material level?
Good thing we didn't get carried away. The software as a whole insisted on playing according to the real rules dictated by the game, and not according to what seemed to some bald monkeys. There is an MDB book that actually makes good use of this topic (but without the use of artificial intelligence): "Tuff Travels" by George R. R. Martin (yes, the same one who wrote the books according to which the series "Game of Thrones" was made).
A space odyssey 2001 was a good example: the computer was given a task, and it decided that the people were just getting in the way....
For those who weren't born when the movie came out, it's worth watching. Great movie but the ending is a bit confusing.