Researchers in the Netherlands were able to use an MRI device to identify which signal a subject was looking at. The road to "reading minds" is still a long way off, but some see it as first steps
By analyzing brain MRI images and a complementary mathematical model, thoughts can be reconstructed. In this way, researchers from the University of Nijmegen in the Netherlands were able to identify which signal an observer was being tested on. The journal Neuroimage has accepted the article for publication and it will be published soon.
Functional MRI or fMRI scanners are used in cognitive studies mainly to identify which areas of the brain are active when a subject performs a specific task. The research question is whether a certain area of the brain is active or not. The group of researchers at the Donders Institute for Brain, Cognition and Behavior at Radboud University in Nijmegen went one step further: they used the scanner data to determine what the subject was looking at. Although it is not a clear picture, but rather a vague pattern, it was possible to recognize a familiar pattern of handwritten letters.
Yair Lakartz, a doctoral student at the Segol School of Brain Research at Tel Aviv University, explains that in an experiment in which subjects were shown handwritten letters on a computer screen, while their brain activity was measured. The researchers found a mathematical method that allows predicting with a relatively high level of accuracy, the shape of the signal presented to the subject from the pattern of neural activity in his brain.
Lakaretz, who produces a computational model for reading, adds that this article joins a number of articles published in recent years that describe how it is possible to deduce from the pattern of neural activity in the visual system of a person's mind, what type of object or image the person is viewing. "Some see this as the first steps towards the implementation of 'mind reading' which will be possible by observing the activity of a person's mind," he says.
However, Keretz suggests addressing a number of limitations and assumptions of the study. "First of all, we note," he says, "that the brain activity measured in the experiment was in early visual areas which are located in the occipital lobe area of the brain. These are areas where low perceptual activity occurs, while "high" mental activity such as making decisions, inferring, wanting, etc., which stores information that has been absorbed and integrated from different senses, is often attributed to other areas such as the prefrontal lobe.
That is - therefore the road towards "reading minds", as it is often understood, is still long. Second, Lakaretz explains, that one must know the limitations of the imaging device: the index
It is used to describe the amount of oxygen in the blood in the different areas of the brain. Relatively high oxygen levels do indicate increased activity of the cells in the brain, but are not necessarily a good measure of information representation in the brain. Another limitation of the imaging device is a relatively low time resolution in the order of seconds, for comparison, the activity pattern of the nerve cells in the brain is in the order of milliseconds. Therefore, there is a limit regarding the rate of "reading thoughts" possible with the current equipment.
"At the same time, the success of the research in predicting the shape of the signal from the brain activity pattern is extremely impressive, and raises the imagination for further successes in this field." summarizes for a wink.
* Thanks to Prof. Naama Friedman, the head of the Segol School for Brain Research, Prof. Uri Ashari, and Prof. Alessandro Treves for their assistance.
32 תגובות
I know all the data you mentioned but the issue is much more complex. These are mainly hypotheses and various variables. A basic information processing system must have all possible combinations. The complex patterns do not.
There are new declarative memories that require new neurons, and there are new associative declarative memories that depend on changes in synapses only. And of course motor memories that are not related to the hippocampus at all.
"I believe that long-term memories are stored in the hippocampus"
As far as I know, this claim is not true, the hippocampus does indeed play an important role in creating new memories (people who are damaged in this area are unable to create new long-term memories) but the memories themselves are created and fixed in the cerebral cortex.
Neural networks do not have to contain all the combinations in the world of the pattern they are supposed to recognize, this is not possible because as you mentioned the number of combinations is infinite. What is beautiful about neural networks (and we also see this in computer simulations) is that after they have learned a sufficient number of examples for the same pattern, they are able to recognize it even in a lot of variations and extraneous ones that they have never encountered before.
I suggest you watch Idan Segev's lectures, the creation of new memories does not require new neurons, in fact most memories are created by strengthening or weakening existing synapses, as well as by breaking synapses and creating new synapses.
For basic information processing, a fixed number of synapses and cells can be used. Information processing includes all possible combinations of the line. Including angle, position, amount of light, and more. For more complex patterns, synapses are used as needed, if you try to build a system that includes all the possible combinations for complex patterns, then the brain will not have to learn because all the possibilities are already built into it, but for that you will need an infinite number of connections and neurons. Although there may be some types of innate patterns that save the need for learning, such as a basic face pattern, so that the baby knows how to recognize its mother. And maybe even very basic shapes like a rectangle and a triangle. There is some research that claims that the baby only recognizes faces in its first days.
I believe that long-term memories are stored in the hippocampus, there is no clear statement on the subject, one of the reasons I think so. Because this is the only place where new neurons are formed, and as I understand it only new declarative memories require new neurons. Motor memories are not formed in the hippocampus and do not require new neurons.
I don't dismiss your explanation but I don't understand why it is necessary, according to what I know the hippocampus is indeed used to create memories but these are ultimately stored in the cerebral cortex (= long-term memory).
Why does the explanation I wrote before not seem to you to be sufficient for the identification of complex patterns? (as I have shown you with several examples) Why is an alternative explanation necessary?
I didn't get to the bottom of your mind, I'd appreciate it if you explained what doesn't work for you.
The visual cortex deals with information processing of the basic lines. So it is possible that the numbers there are fixed. Complex and learned patterns are apparently stored in the hippocampus region.
I will try to explain again, in the entire cerebral cortex you have 300 million "clusters", and they are connected to each other in a hierarchical way, a cluster located in a low layer recognizes basic things (a line, a sound of a certain frequency) and is an input to many more clusters at higher levels that recognize more complex things (like For example, the word "apple", a funny joke, a familiar melody of a song, etc.).
Each "cluster" consists of 300 neurons, meaning that if a suitable input is received at this level in the hierarchy (the corresponding diagonal line) that group of 300 neurons will begin to be very electrically active and will send output signals to the corresponding clusters at higher levels as if to say "Hey, listen, I detected the signal 'A', see if something can be done with it (for example identify the word "Apple" in a cluster that knows how to connect the collection of these letters)
Hope the explanation is clearer.
From what I understood from you before. There are only 300 neurons per input. If you're talking about 300 neurons per line, that's a different story.
So what did you want to say? It is true that in order to identify an apple, you "wasted" 9 clusters out of 300 million that you have in your brain, and it is not really a "waste" because, as mentioned, every cluster that recognizes some basic shape (arc, horizontal line, etc.) serves as input for a very large number of other high-level clusters More in the hierarchy that identify additional objects consisting of arcs (in addition to the apple).
Actually 9 if you take into account the whole apple represented in the second layer
This reinforces my question, 8 lines require 8x300 and not 300
Mike, in principle an apple can be described by only 8 simple arcs, it is enough that you have 8 clusters of neurons, another mind detects an arc at a certain angle, and another ninth cluster that connects them all together, and you already have a complete recognition of an apple. And don't forget that each cluster that identifies an arc can be an input to many other clusters that identify additional objects consisting of a collection of arcs.
You tried to draw it, see how the apple can be easily broken down into a collection of simple components:
http://img2.timg.co.il/forums/1_171368791.JPG
If one line requires 300 neurons, how can an apple be satisfied with the same amount of neurons?
A "cluster" is a group of approximately 300 neurons that starts to become very active electrically following a certain input (for example a horizontal line if it is located at a low level in the hierarchy, and in a funny joke or the object "apple" in the upper layers).
It sounds very logical to me, think about it, every object no matter how complex (a bicycle in your example) is made up of simpler objects (handlebars, wheels, chair) and they in turn are made up of even simpler elements.
Likewise with the word "apple" in Kurzweil's example, it consists of several letters, and each letter in turn consists of a series of lines at different angles.
It seems to me very appropriate for the hierarchical structure he is talking about, and I realized that you are also talking about it in your first message.
You will notice that you are talking about a simple line that is in the first layer. It is certainly possible that under normal conditions there is no need for more than 300 neurons to operate when a line is input. But when it comes to more complex patterns it no longer makes any sense.
Now I also understand maybe understand what you mean Eshkol. Every native basic input starts with a cluster. I was thinking of something else altogether, and explained above.
Regarding the second comment, I was not mistaken, the synapses in the input originate from ten thousand neurons. And the output also sends axons to ten thousand neurons.
By the way, what you said is not accurate, each axon (output of a neuron) can be an input to several synapses of several different neurons... Idan Segev describes it as a wire that runs through a tangled top of trees in the jungle and touches the leaves of the various trees along the way.
Listen, unfortunately I'm not an expert in the field and I don't know exactly what percentage of the synapses receives input from a neuron that is part of the cluster and what percentage receives input from other clusters in the hierarchy, but even if you take into account that each cluster only receives inputs from 1% of the other clusters in the brain, it already works out quite well:
Calculate 300 neurons in a single cluster, times ten thousand synapses per neuron, and you will see that the result is exactly 1% of 300 million, which is the total number of clusters in the cerebral cortex.
In any case, I understand that this theory is also based on practical research in the laboratory that showed that for an input of a certain angular line, for example in front of the eye, a group of 300 neurons "lights up" and starts to become very electrically active, this is what I also remember from Idan Segev's fascinating course "From synapses to free will" ":
https://www.youtube.com/watch?v=yP-dJNYTrg8
There are on average ten thousand synapses at the input of each neuron. Each of the synapses comes from a different neuron. And that's just from one layer. So how exactly does a whole cluster contain only 300 neurons?!
I don't currently have speakers, but it seems to me that here Professor Idan Segev is talking about a similar principle:
https://www.youtube.com/watch?v=L0AR1cUlhTk
Mike, 300 million clusters, multiply by approximately 300 neurons in each cluster, comes out to a number quite close to 100 billion, which is the estimated number of neurons in the entire cerebral cortex in an adult.
Another interesting point is that according to him there is no difference between a "cluster" of neurons that is located at a low level and recognizes simple characteristics such as a line at a certain angle or a sound at a certain frequency, and a "cluster" at a high level that recognizes a funny joke or the word "apple", it all depends on the connections that are formed between the clusters ( by the axons, which are a kind of "links") this is what determines the hierarchy.
He mentions a recent study that showed how a group of neurons that is usually at the bottom of the hierarchy changed its purpose (I think due to a tumor or an accident) and started acting as a cluster that detects much more complex things, that is, at a higher level in the hierarchy.
I assume that by "cluster" you call all the cells in all the layers that diverge from a single memory cell in a high layer through lower layers to the sensors. Just how did they get to this number of 300 million? By logic, if each cell has a unique cluster, then the number of clusters equals the number of cells.
Mike, I meant that the first part of your description regarding the way the bicycle is recognized is very reminiscent of Ray Kurzweil's theory, as mentioned he claims that each cluster of neurons at the lowest level recognizes something very basic such as a vertical or horizontal line, and how many such individual clusters constitute input to a cluster at a higher level that recognizes something More complex (for example a wheel) and some clusters at this level are connected to a cluster that is at an even higher level and it identifies a bicycle.
He explains that the more internal the layer is, the more complex memory it contains, that's true, but that's not what I was talking about. He does not explain the three things I described here. How the simple or complex memory is represented in the brain. How an object is identified through the memory, and how it is retrieved from the memory to the conscious. All my description is also true for the inner layers, only I preferred to explain it in a simple way on one layer.
As far as I know there is no model similar to what I described. And it will be interesting if I get lost. I hope that someone who is involved in the field will fall for the token and will be willing to try the idea. It is possible, for example, to add the ability to retrieve memory from a cell in an artificial neural network by adding a simple feature that will cause, through deliberate excitation of the cell, to display the memory stored in it backwards on the sensors or on another means such as a monitor, to the best of my knowledge there is currently no method to do this and the idea I presented can certainly to be employed.
By the way, this is exactly what they did here. A memory cell of a certain letter was stimulated by natural means though, by concentrating on the letter, and then the letter was presented on anterior cells closer to the eye. It is possible that when the letters are retrieved, they are also displayed right in the eye, as happens when the letter is input externally.
Hi Mike, he is talking about something around 300 million such "clusters" in the entire cerebral cortex, and they are connected to each other in a hierarchical way, see here starting at minute 40:50 -
https://www.youtube.com/watch?v=zihTWh5i2C4
Asaf
In the current study, the experimenter was asked to imagine letters. Apart from that if you read what I wrote you will see that theoretically when retrieving the memory the retrieved object can be displayed exactly in the same place that the object was displayed during input. So it doesn't matter if he was tested while viewing the object or while he was just concentrating on the object.
one
I have given a simplistic description here. A cell of a bicycle is indeed in a deeper layer in the neural network. But I didn't understand what the two hundred neurons per cluster are. If each neuron connects to tens of thousands of neurons. Maybe you meant two hundred thousand or even two hundred million?!
Mike, your description is very similar to Ray Kurzweil's description in his new book:
"How to create a mind - The secret of human thought revealed"
He claims that the brain is divided into many groups of neurons (100-200 neurons in each such cluster) and they are connected to each other in a hierarchical way so that the groups at the bottom of the hierarchy recognize very simple patterns such as a horizontal line, a vertical line, a line at an angle of 30 degrees, etc., and the groups at the top level recognize things complex like a bicycle, a funny joke and the like.
A very interesting demonstration, several years ago:
http://pinktentacle.com/2008/12/scientists-extract-images-directly-from-brain
I understand correctly, this was not mind reading here. In total, they measured the cells in the brain that are responsible for translating visual information. There have already been many experiments on the subject and if I'm not mistaken already in the sixties a sensor matrix was implanted in a cat and they were able to "see" what it sees through the brain.
I assume that the recorded neurons are the components that represent the signal. They can be distinguished using the devices when retrieving the signal from a memory cell and displaying it consciously. Very much strengthens the model I developed regarding the memory, identification, and retrieval system. I will present part of the technique here. But you have to remember that thoughts include not only visual information but also auditory, sensory, and more. So this is just the beginning of a revolution.
⦁ How declarative memory is stored in the brain
⦁ How an external object is identified
⦁ How memory is retrieved and presented consciously
memory storage
Every object is made up of many components. For example, a pair of bicycles has handlebars, wheels, pedals, and more. And it is the connection between the various components that creates the pair of bicycles.
To store a memory of a pair of bicycles we can do this: suppose we have a special sensor for each of the components of the bicycle we described. Let's take all these sensors and put them side by side.
A conductive cable will be connected to each sensor, and the other end of all the cables will be connected to one common cell. The shared cell will serve as a memory cell that stores the pair of bikes virtually. Now if we want to know what is stored in the memory cell, we can simply check what kind of sensors they are connected to, each sensor represents a certain component that belongs to the memory, and all the components we can know what is the object stored in the memory.
Memory ID
Automatic identification of bicycles from the outside world will be carried out as follows: the sensors representing the components of the bicycle will be activated when the bicycle is input, the voltage that will be generated in them as a result of their activation will also activate the memory cell located at the conductive ends of each of them, and when the memory cell is activated we will know that the system has detected a pair of bicycles.
memory retrieval
Memory retrieval is a feature that allows any person to imagine the image of the object stored in the memory cell. When retrieving the memory, the direction of the current in the conductors is reversed. That is, the memory cell activates reverse conduction in the conductors and activates the sensors similar to the situation that happens in a real input, only that instead of the sensors activating the cell, the cell activates the sensors. The retrieval allows the machine to imagine the objects stored in it, and in fact to be aware of the objects stored in its memory.
We can further illustrate the memory retrieval technique on a simple object like the letter R. Each pixel in the eye actually serves as a sensor on its own. Each pixel in the following figure receives one part of the letter r. You can see the letter R in red represented in the eye by many pixels, with each pixel connected via a synapse to the memory cell that represents the letter. The memory cell is maximally activated when the signal from the outside world appears. And can even activate and simulate the signal himself through reverse transmission.
Eliyahu Galil, a very strange claim. What is it that we don't know if the neurons fire because of an actual object or because of an internal representation. After all, imagination always plays a role in thoughts. And the question is whether it is a real representation or not dependent on the activities of other areas of the brain. For example, this area that they have now managed to analyze, if it is not activated while that person is imagining letters, it will be possible to identify the source of the imagination.
The reason that mind reading will not be possible in my opinion is that there is no connection between certain thoughts and areas of brain activity. In order to read minds we will have to connect a recording device with a high time resolution to each input and output of a neuron. Only then will it perhaps be possible to read minds. The problem is that such a device is not in sight.
On the assumption that they will indeed succeed in developing a mind-reading robot, the ethical and legal problems that such a development may provoke must already be examined. On the face of it, an intrusion into the realm of individual modesty can be seen here. Who will be allowed to possess such a device, the police or other security forces? Or companies that engage in the placement of employees and as part of their screening exams will have candidates connect to such a device? The sooner they investigate this issue the better
Thanks for the interesting article.
Regarding mind reading, is it just a long way to the destination, or maybe impossible at all? Prof. Michael Hagner, an expert in the philosophy of neurology, claims that neurology promises a lot, but realizes little. According to him, we will never be able to read minds because we will not be able to tell whether the neurons fire because of an actual object or because of an internal representation only.