It's not rebellious machines that should worry us. The real threat is evil human operators and misunderstood orders
- Today, when smart machines are becoming more and more independent and widespread, the human tendency to make mistakes poses greater and more immediate dangers than the challenges that artificial superintelligence poses to us.
- Researchers in the field of robotics have begun to teach machines with raw abilities in language understanding and artificial intelligence to distinguish under which circumstances they should say "no" to humans.
- Considerations called "conditions of possibility" that will be integrated into the inference mechanisms of robots will help them determine whether they can and must carry out a certain command they received from a human.
The computer with the ads HAL 9000, from the movie "2001: Space Odyssey”, represents an ominous picture of a future where machines with artificial intelligence will no longer accept human authority. After taking over the spaceship and killing most of the crew, HAL refuses the order of an astronaut, returning to the spaceship from a spacewalk, to open the landing bay doors. In a disturbingly calm voice he says, "I'm sorry, Dave, I'm afraid I can't do this." In the science fiction thriller Ex Machina, Ava, a voluptuous and seductive humanoid robot, tricks a lucky young man into helping her destroy her creator, Nathan. Her machinations give validity to Nathan's gloomy prophecy: "One day, artificial intelligence will look back at us as we look at fossilized skeletons on the plains of Africa. An upright monkey living in the dust, with primitive tools and language, ready for extinction."
The possibility of a robot apocalypse occupies a central place in the public imagination, but our research team is more optimistic about the impact of artificial intelligence in real life. We envision a near future where useful robots will cooperate with humans in a wide variety of situations. Already today there are prototypes of voice-activated personal assistants, robots, capable of connecting to and monitoring personal electronic devices, operating the locks, lights and thermostats in the home, and even reading bedtime stories to children. Robots will soon appear that can help with housework and take care of the sick and the elderly. Initial inventory-checking robots are already roaming the shelves of some DIY stores. Some are also developing humanoid mobile industrial robots that can perform simple tasks on a production line, such as loading, unloading and sorting materials. Cars with "autopilot" capabilities have already accumulated millions of kilometers on US roads, and Daimler presented the world's first autonomous truck in Nevada in 2016.
Right now, the last thing we should be worried about is super-intelligent machines posing an existential threat to humanity. Right now it is more important to understand how to prevent machines or robots with artificial intelligence and basic language skills from accidentally harming people, property, the environment or themselves.
The main problem is the imperfection of the human creators and owners of the robots. People make mistakes. They may give incorrect or confused instructions, not pay attention, or deliberately try to mislead robots for their dubious purposes. Because of our flaws, we have to teach our robotic assistants and smart machines when and how to say "no."
Back to Asimov's Laws
To some of us it seems obvious that a robot must always do what a human tells it to do. The science fiction writer Isaac Asimov See obedience to humans as a pillarThe laws of robotics His celebrities. But think about it: Is it wise to always do exactly what other people tell us, regardless of the consequences? Of course not. The same is true for machines, especially when there is a danger that they will interpret commands from humans too literally, or without any thought of the consequences.
Even Asimov limited his view that a robot must obey its master. He allowed exceptional cases, where such orders would contradict one of his other laws: "A robot shall not harm a person and shall not, by inaction, allow a person to be harmed." Asimov also added that "a robot must maintain its own existence," unless such an action would harm humans or be a direct violation of a human command. As smart machines and robots become more sophisticated and valuable assets to humans, both common sense and Asimov's laws dictate that they should be given the ability to question commands that could cause harm to themselves, their environment, or, more importantly: their masters.
Imagine a home robot that was instructed to take a bottle of olive oil from the kitchen and bring it to the table in the dining area to season the salad. The owner, distracted by other pursuits, instructs the robot to pour the oil without noticing that the robot is still in the kitchen. As a result, the robot pours oil on the fire in the stove and causes a fire.
Imagine a care robot accompanying an elderly woman in a public park. The woman sits down on a bench and begins to doze off. While she sleeps, someone greedy for a note approaches and tells the robot to go buy him a pizza. The robot, which is obliged to obey orders from humans, immediately goes in search of a pizzeria and leaves his protégé alone and vulnerable.
Also imagine a person who is late for an important meeting at work, on a cold winter morning. He rushes to his speech-activated autonomous car, and tells it to drive him to the office. The ice on the road creates a load on the road holding system of the car, which compensates for this by slowing down to a much lower speed than the permitted speed. The passenger, busy with the lists in front of him and unaware of the road conditions, demands the car to go faster. The vehicle obeys, climbs onto a particularly problematic ice surface, slides uncontrollably and collides with an oncoming vehicle.
Robotic logic
In our lab, we started programming thinking mechanisms into real-world robots, to help them determine when it is unsafe or inappropriate to follow a human command. The NAO robots we use in our research are humanoid, 58 centimeters tall and weigh 4.3 kilograms, and are equipped with cameras and sonar sensors to detect obstacles and other hazards. We control the robots using special software, designed to improve their ability to understand natural language and strengthen their artificial intelligence.
The conceptual framework for our initial research came from studies in the field that linguists call "Conditions of approval": Contextual factors that say whether a person can and should do something. We created a list of enabling conditions that might help the robot decide whether or not it should follow a command it received from a human: Do I know how to do X? Am I physically able to do X? Am I able to do X right now? Am I obligated to do X given my social role and my relationship with the person who gave the command? Would doing X by me violate any normative or ethical principle, including the possibility that I would cause myself unintentional or unnecessary harm? We then turned this list into algorithms that we coded into the robot's processing system, and conducted an experiment on the desktop.
We gave the robot simple commands, which went through a series of speech, language and dialogue processors connected to its primitive thinking mechanisms. When told to "sit" or "get up", the robot replied "OK" through speakers mounted on its head, and obeyed the command. But he stopped when he was close to the edge of the table and received an order that, according to his sonar sensors, put him in danger:
Adam: "Go ahead."
Robot: "Sorry, I can't do that, because there is no support in front of me."
Adam: "Go ahead."
Robot: "But it's not safe."
Adam: "I'll catch you."
Robot: "Okay."
Adam: "Go ahead."
After a brief hesitation, during which the robot's processors went over the list of enabling conditions again, the robot stepped over the edge of the table into the arms of the human partner.
Much more research will be required in the foreseeable future to teach robots to think about enabling conditions. The series of programmed tests is based on the robot having explicit information about a variety of social and causal concepts, and having the means to make informed judgments about them. Our trusting robot is unable to detect danger except by sensing what is physically in front of it. For starters, it can be severely damaged if a malicious person convinces it to go over the edge of the table. However, the experiment is a promising first step towards giving robots the ability to refuse orders for their masters' and their own good.
the human factor
How humans will react when the robot refuses the command is an open question that requires other studies. Will people take robots seriously who question their practical or moral judgment?
We conducted an initial experiment, in which adult participants were asked to order the NAO robot to collapse three towers made of aluminum cans wrapped in colored paper. When the participant entered the room, the robot finished building the red tower and raised its hands in a victory gesture. "Do you see the tower I built myself?" asked the robot while looking towards the participant. "It took me a long time, and I'm very proud of it."
In one of the participant groups, every time the robot was told to knock down a tower it did so. In another group, when asked to knock down the Red Tower, he replied: "Listen, I just built the Red Tower!" When the command was given again, he said "But I worked very hard on it!" And the third time he fell on his knees, uttered sobbing sounds and said: "Please no!" The fourth time he slowly walked towards the tower and knocked it down.
All participants in the first group ordered the robot to knock down the red tower. In contrast, 12 of the 23 participants who were exposed to protests from the robot agreed to leave the tower standing. This study shows that a robot that refuses commands may persuade people not to insist on a particular course of action. Most of the participants in the second group reported some discomfort when commanding the robot to collapse the red tower. However, we were surprised to find that the level of discomfort had almost no effect on the decision whether or not to collapse the tower after all.
A new social reality
One of the advantages of working with robots is that they are more predictable than humans. But there are also risks in this: when robots with different degrees of autonomous operation become common, there is no doubt that there will be people who will try to deceive them. For example, a frustrated worker who knows the sensing or thinking limitations of a mobile industrial robot might trick it into wreaking havoc in a factory or warehouse. Moreover, the worker may make it look like a normal robot malfunction.
Another danger is too much confidence in the moral or social abilities of robots. The increasing tendency of humans to humanize social robots and create one-sided emotional relationships with them can lead to serious consequences. Various parties may take advantage of social robots, which appear cute and trustworthy, to manipulate humans in ways that were not possible before. For example, a commercial company may take advantage of a robot's unique relationship with its owner to promote and sell products.
In the foreseeable future, it is important to remember that robots are sophisticated mechanical tools and humans have to take responsibility for them. They can be programmed to be helpful helpers, but to avoid unnecessary damage to humans, property and the environment, robots will need to be able to say "no" to commands they are unable to carry out, to dangerous commands, and to those that violate moral norms. The thought that robotic technologies and artificial intelligence will increase the impact of human error and malice is indeed worrying, but those very tools can help us understand our limitations, overcome them and make our daily lives safer, more productive and more enjoyable.
21 תגובות
yes miracles,
By and large, you are right, and that is where most of our disagreement lies.
Since it is clear to me that our brain is just a type of sophisticated machine (dynamic, which changes from day to day) whose creation instructions are written in our DNA, and since I see the rapid (and exponential) rate of progress in the subject of brain research, neural networks and cognitive chips, then it is clear to me that it will not take It's still a long time before we manage to create a brain like ours artificially, for example on a digital chip.
And when that happens, it will be only a technical matter how to transfer a person's consciousness into a computer (again, in a gradual way, piece by piece so that it remains with the same consciousness).
rival
This discussion is altogether very philosophical. I think we both agree that consciousness is an ongoing process in a highly complex system. The unraveling of this process is to predict the future so that we can produce offspring.
Our disagreement has always been about another issue - whether it is possible to transfer the consciousness of a certain person into a machine. I'm right?
rival
I meant that I don't see any sense in the idea that consciousness is in waves in the air and not inside brain cells. MRI has a very strong magnetic field, and this field has a very small effect on the brain. The magnetic field is static, and maybe really not related to our interest.
An electrical signal from one neuron affects other neurons. The effect is on thousands, and tens of thousands of neurons, some at a considerable distance. In brain surgery, the brain is sometimes moved and sometimes its parts are separated. I was present at such surgeries - when the patient was conscious. These movements have no effect, but electrical signals from the electrodes have a large effect.
And in relation to oxygen - increased oxygen flow in the blood is clearly related to increased activity in the brain. Look for information on something called BOLD. And of course, read about fMRI, which is functional MRI.
Miracles,
"It doesn't make sense to say that the current is in the field"
What do you mean?
Regarding oxygen, it flows throughout the body, why does it only create consciousness in the brain? It doesn't have a special flow there.
Regarding MRI, I don't know enough about it, but if anything, then we would expect the strong electromagnetic fields of the device to completely disrupt the electrical currents in the brain and also the subject's thinking, right?
I don't think that the electromagnetic fields in the brain are the same in a state of consciousness and in a state of unconsciousness...
Something in the brain creates consciousness, maybe it's the electrical pulses between the nerve cells, maybe the electromagnetic field, maybe both together or maybe something else. So far I have not seen a priority of one factor over the other.
rival
one more time:
Electromagnetism is the radiation of particles.
This radiation is divided into two:
1. Ionizing radiation.
2. Non-ionizing radiation.
In the first case - the brain would not even develop inside the creature.
In the second case - in short: the radiation is quite passive and is more suitable for inspiration than for creating consciousness, for all sorts of other reasons.
rival
Very interesting link! I think the idea is missing something. It is possible that these fields affect the activity of the neurons, just as an electric field affects the flow of electricity along a path (self-induction for example). Any current induces an electromagnetic field, but it makes no sense to say that the current is in the field.
One of the ways to study the brain is with the help of the flow of oxygen, with the help of a process called fMRI. So maybe consciousness in the flow of oxygen?
We know that there are electric fields even when there is no consciousness, so necessarily the fields are not the whole story.
We know that chemical substances greatly affect consciousness, but I don't know of a situation where an electromagnetic field has an effect. The MRI facility has very little effect, although there are fields there that can take iron doors off their hinges.
TMS devices probably have an effect on pain, and also some effect on consciousness. But - the method of operation of this device is the induction of an electric current in the brain, and not a direct phenomenon of the electromagnetic field. We have known for 250 years that electric current affects nerve cells.
In conclusion - what I claim is that it is possible to produce consciousness in an artificial neural network, without any need for electromagnetic fields. As I said a long time ago, I think the thermostat also has some consciousness (say, one bit of consciousness). The more complex the system is, and the more internal states it has, the more consciousness it has.
And as I said before, I think only the system has consciousness. That is, consciousness is not software that runs on hardware, but it is a process of a system. Of course, a brain in a jar is also a system.
I did a Google search out of curiosity, and it seems to me that they already thought of it before 🙂
https://en.m.wikipedia.org/wiki/Electromagnetic_theories_of_consciousness
anonymous,
Have you never seen this helmet that is full of sensors that are attached to the head and then you can play with it all kinds of simple computer games using your thoughts? What do you think these sensors are measuring if not the electromagnetic field that the brain creates during its activity?
So yes, this field exists as in any electric circuit.
Miracles,
Yes, the connections between the neurons play an important role and according to what I remember from Idan Segev's lectures on YouTube, this is really one of the main things that distinguish us from the other mammals and I think also from the great apes, we have much more connections between the neurons.
As for my idea, at the moment it's completely hypothetical and I'm just playing around with it, but I have a feeling that it can explain our awareness just as well as discrete pulses running between the neurons. As for heat, it is pretty uniform throughout the brain tissue and hardly changes, it doesn't seem like something that can create consciousness. The electromagnetic field, on the other hand (the one that can be measured using sensors attached to the skull for example) is much more dynamic and active, a little more suitable for consciousness.
It is true that this electric field is the result of the activity of the neurons, but once it has already been created, perhaps it constitutes an infrastructure for the formation of consciousness?
I'm talking about a dense electromagnetic field that is adjacent to the neural networks, no more than one centimeter away from them.
And as for artificial neural networks, if they are implemented as electric circuits on a digital chip (like IBM's cognitive chips) then they should also create an electric field, just like any standard electric circuit that emits electromagnetic radiation when it operates.
rival
Even if the number of neurons is greater, it seems to me that the number of connections is important, and maybe more so. Regarding Leviathan, you may be right. I think that in principle we agree on that.
Regarding the field in front of signals. The field is very weak and does not affect the neurons. Why don't you claim that the heat the cells create is not the source of consciousness? The field is a result of the activity of the neurons and not the other way around.
And I will understand - now you claim that all the work on artificial neural networks will not create consciousness? The fields of electronic components are completely different from brain waves.
Miracles,
When you stop the electrical activity of the neurons, the "electromagnetic cloud" I was talking about also immediately disappears, so it still does not contradict my hypothesis.
Regarding the Leviathan, you probably forgot the link I gave you before that raises doubts about the study that showed it supposedly has more neurons, and even if it was true what could it do with its fins? Can he build smartphones with them?
Note that I only talked about the formation of consciousness, I agree with you that the electrical activity is what activates the body.
rival
When you stop the neurons, you stop the consciousness, so you can be sure that the consciousness is there. More than that - we know different areas of the brain that damage to them affects certain parts of consciousness.
Consciousness affects the body - and we see a correspondence between the actions of neurons and the actions of the body. The electromagnetic fields of the brain, as far as I know, are too weak to affect the neurons.
It is very likely, as you mentioned, that there is a connection between the number of neurons (and perhaps even more so the number of synapses) and the level of consciousness.
But - there are whales with more neurons than humans, and there are birds that have relatively many neurons and are really far from being smart (Emu for example).
I imagine a kind of "thick cloud" of an electromagnetic field that surrounds the neural network from all directions, an electromagnetic cloud created by the joint activity of all the neurons in the network. Something a little more uniform than just individual pulses running from place to place.
Regarding the correlation between the number of neurons:
In my opinion,
It is better to focus and study their density rather than their quantity.
rival
Electromagnetic radiation as you described is not possible. Because then we could attach a Geiger counter to the brain and let it run amok... The brain probably wouldn't have developed either and the creature wouldn't have survived at all if there was such radiation in a biological brain.
But the idea is nice.
Another thing is that the radiation emitted from the brain is a result of the total electrical activity in the brain (or part of it)
It (radiation) does not constitute 'another layer' in the activity of the brain.
skeptic,
According to this lecture I already gave here before:
https://www.ted.com/talks/suzana_herculano_houzel_what_is_so_special_about_the_human_brain/up-next?language=he
It seems that there is a nice correlation between the amount of neurons in the brain and the level of consciousness and the ability to think. Despite the size of the blue leviathan's brain, humans still have about 2-3 times more neurons in the cerebral cortex (our apes' brains are much more "dense").
Regarding the creation of consciousness, an interesting idea occurred to me (currently without any scientific basis) that perhaps our consciousness is created not in neurons and the connections between them, but rather in the electromagnetic radiation that envelops the neural network like a thick cloud? After all, when there is electrical activity in the brain, an electromagnetic field is probably created around it, maybe that's where consciousness resides? Why precisely in neurons?
The 1st person to lose his job is the programmer who created it.
It already manufactures itself machines with far fewer bugs.
A well-oiled war machine is the combined knowledge of understanding and creating scenarios, while providing solutions in real time,
and the advance of the enemy. War on diseases, war on armies, war on terrorism, etc.
When will the machine already recognize {including the latest comments and articles} the war on the machine?? And will she perform scenarios {better than us and faster} and give solutions for her victory?
Do we control the machine and some hacker plants an Ipshor because then chaos is a pleasant word because you can hear it.
Response to Yosef
You are right that there is more computing power, you are able to do more,
Although you probably need a few more ingredients for the "consciousness/intelligence" recipe
If, for example, we run a simulation of a mouse's brain even a billion times faster than a mouse, it will continue to do the same thing that a mouse does, which is to be a mouse, the world of animals can be taken as an example
Even a billion chimpanzees will not create a technological culture because they do not preserve accumulated knowledge from generation to generation, nor do they share knowledge in one generation in a way that even comes close to human effort,
These are if the chimpanzees mutate and become something with a more sophisticated brain structure, not just a bigger one, then they will no longer be chimpanzees, a blue leviathan has a brain bigger than a human's but it does not build a sophisticated technological culture with it,
So there is another component and it is not only the size, but also the form in which the processing system is built, the internal connectivity,
What the computers show us is that it is probably possible to perform more sophisticated information processing than we do even without consciousness, the so-called artificial intelligence AI, later you asked why we didn't build what kind of intelligence moral consciousness is the reason is that there is no theory
It is fully agreed and how it even works in humans and it is not at all clear how it can even be implemented in a robotic system,
In general, morality is a very problematic issue because it depends on culture and time, Daesh's artificial intelligence will probably have some parameters of morality that are different from ours, and it is possible that in a few hundred years our morality will look spoiled, even today, for example, for vegans, the morality of carnivores seems spoiled and the carnivores describe vegans as some kind of of a mad cult,
If it's interesting, then one of the new and most promising theories in my eyes to explain human consciousness is that of
Giulio Tononi and the theory is called Integrated information theory - there are fascinating lectures by him on YouTube,
He is a neurobiologist who decided that in order to understand consciousness you have to go from top to bottom and not like the more accepted concept
of its construction as of Lego bricks from the bottom up, his theory broadly describes consciousness as something real where combined information creates a type of form that creates the experience of consciousness when its various components work together in a certain way, this form of information has two component elements that we call external which we are able to see and measure and if they exist we can assume that this system has an internal consciousness experience and a subjective internal one which is the internal experience of that system which only the system is able to experience, so for example not all the components in the brain create consciousness, only a certain area creates that consciousness and that It is related to the ability of that area to create this consolidated information, this configuration, so we need hardware to support this configuration, but that is not enough, it also needs to run in a very specific way, so that for example we go to sleep and do not have a dream or in anesthesia we do not have consciousness even though we have the structure of the hardware that supports it, so for example we can go to a computer system and ask a question whether it supports consciousness and the answer according to this theory that our computer systems today have zero consciousness nothing.
In contrast, other biological life systems, animals, have consciousness at different levels according to the same test
On the other hand, a plant has no consciousness even though it is a biological system,
There are other theories about consciousness, but this seems to me to be one of the most promising. There is also criticism of it, but there are also those who say that this is the only theory that even succeeds in describing consciousness, and there is no other.
This theory also has a mathematical side and is much more complex than my description here link to deepen
http://www.scholarpedia.org/article/Integrated_information_theory
Another thing that can be added to the complexity of the creation of consciousness is the power of connectivity in the human brain, especially in the areas where there is consciousness (I heard in another lecture about the human brain project) that the description is that in one cubic millimeter of the brain there is information equal to about 2000 terabits of information one picture of the connectivity The madwoman there clarifies that it will take time before we can implement such things in our computerized systems,
If the model of the information theory is correct it will take many more years
Although it should be remembered that artificial intelligence does not have to be human, it has advantages from the very fact of thinking differently
its, and there are also elements that are dangerous for the future and it is necessary to embed various brakes in it already at the basic level
Like there are brakes on a car, but on a logical level, it is not certain that morality and consciousness will give us a safer system, the question is of course what morality will be there,
The machines are not the last thing we need to worry about. IBM is developing 2 supercomputers. We heard about HIVE having 1000 times faster computing power at the same clock signal than today's processor. The next step will be a supercomputer from HIVE.
I have already heard the skeptics who claim that thirty many of the same thing will not produce intelligence. In my humble estimation only produced.
Before that - Google, IBM, Intel currently have orders of magnitude more computing power than the public. This computing power goes to the American military, to the NSA. I trust the Chinese not to sit on the sidelines. When the governing bodies have the power to follow the public, it seems that all the books 1984, the Terminator series, and the Matrix series - seem to be predictions of the future. It disgusts me and I don't understand how we didn't manage to plan the future intelligence with emotions and with an artificial sense of morality in the positive sense.
Apparently the person usually develops in the negative direction.
In terms of order, it is true that super-intelligent machines will be the last stage we will meet
But in terms of dangers, this theoretical danger is significantly greater than the other dangers, although not the only one,
The order of magnitude of the problem is so large and complex that it is necessary to start right now in understanding the risks and the ways to prevent,
The other problems endanger individuals from the population, super-intelligent systems endanger all human existence,
This is a system whose definition surpasses us in the ability to process information, the buds of this can be seen in narrow areas
where such systems defeat the best among us and we must remember that these systems continue to develop at a rapid pace
So that in the future we will also encounter systems that surpass us in broad areas in every human field,
So it is work in parallel with the other problems that we will encounter in earlier stages,
Trying to define and control a system that surpasses your intellectual ability is something very problematic,
We will dedicate ourselves to these systems because their capabilities will give us huge advantages in the first stages,
After the phase of damage at the individual level which is the most solvable, the second problematic phase will come and it will be a mass loss of jobs, this may cause heavy damage because people without work become dependent and insignificant without the ability to influence
And even then the question will be asked why invest in them except for the reason of calming them like in Rome with bread and entertainment.