Comprehensive coverage

Unlimited memory

The desire to remember and preserve everything in our past leads to the development of traits that try to do so. Technology in the service of memory

Israel Benjamin, Galileo

Blackberry mobile device. From the Daily Mail website. Technology will help us remember
Blackberry mobile device. From the Daily Mail website. Technology will help us remember

"I will never forget the events of that day."

For Brad Williams, this statement is true for every day in the decades since he was a child. He knows how to answer, quickly and accurately, questions such as what he had for breakfast on a certain date, or which celebrities got married on another date, or when a certain TV program was first broadcast and what other programs were broadcast at the same time on other channels in the city where he lives, and so on. In a competition against an expert librarian who used all the Internet resources, Williams answered twenty questions in 12 minutes, against 24 minutes required for a librarian (as can be seen in a documentary film about Williams).

For Jill Price, the memory experience of each event is accompanied by the same intensity of emotions that she felt at the time. Worse: more than ten times each day she recalls some event from her life

In 2006, a team of researchers from the University of Irvine (Irvine) proposed the name "Hyperthymesia" ("Thymesis" is the Greek word for remembering) for this rare condition, in an article that described a woman identified only as AJ. Two years later, AJ revealed her identity, Jill Price, when she published a book about her life called "The Woman Who Can't Forget".

After Price and Williams only one other definite case of hyperthymesia was discovered. There are still no plausible explanations for these cases. Although some significant changes were found in Price's brain compared to the average brain, the researchers are not publishing the details for now until they can check whether these changes appear in other cases of hyperthymia.

Life in split screen

This situation does not always bring blessing to its owner. Williams manages well with his abnormal memory, but for Price the memory experience of each event is accompanied by the same intensity of emotions that she felt at that event. Worse: more than ten times each day she recalls some event from her life. The event is pulled from her memory as if she experienced it at that moment in a "split screen" along with the events of the present, and she cannot stop the memory or choose another memory. These memories, she says, paralyze her life.

Even for the boy Ponce, the imaginary hero of the story "Ponce the Rememberer" by Jorge Luis Borges (Borges, 1942), who after falling from a horse begins to remember every detail (and even the exact shape of the clouds in the sky every minute in the past), the perfect memory brings with it many difficulties. Pons has trouble sleeping because there are too many details in his mind. He also fails to create any generalizations, because in order to give a general name to a group it is necessary to ignore the particulars that distinguish the members of the group.

The Electronic Memory Project

For almost all of us, memory is not as reliable as the memory of Williams and Price: we completely forget events and details, change the details of the memory as time passes since the event, "remember that we remember" but have difficulty retrieving the details we need, and are wrong even when we think we remember well. Memory failures are an inexhaustible source of frustration, friction, and sometimes even more severe consequences.

At least one more person in the world can greatly reduce their memory failures, by using technology. This is Gordon Bell (Bell), a computer engineer born in 1934, who contributed a lot to the architecture of the first minicomputers in the sixties. Since then, Bell has led research and development in areas that include parallel computing (the combination of a large number of processors) and remote presence (telepresence - technologies that allow people to function and appear as if they are physically present in a place far away from them). An important prize awarded annually to researchers in the field of parallel computing is named after him.

In 1995, Bell joined Microsoft. In one of the projects he is involved in, he defines himself as the "guinea pig". The name of the project is MyLifeBits (which can be translated as "the parts of my life" or "the bits of my life", where "bits" are of course the smallest units of digital information). As part of the project, Bell transferred his entire archive, collected over decades of work, to one repository of digital memory storage. The transfer was carried out by scanning documents and images, collecting digital information and e-mails, etc. In the years since the beginning of the project, a lot of additional information has of course been accumulated, but almost all of it is collected automatically: at times when the system is active, Bell's phone calls are recorded and stored in the MyLifeBits "information warehouse", as are the television and radio programs he watches and listens to.

When he uses a computer, every website he opens is recorded, and the files and e-mail messages he writes or reads are saved. When Bell moves from place to place, his location is picked up every moment by a GPS receiver and recorded. He also wears a camera that takes static pictures when it senses that something has changed in Bell's environment: for example, when an infrared sensor attached to the camera senses the body heat of a person who is close enough, the camera will save an image of that person. If the light intensity picked up by the camera changes significantly, Bell may have moved from one environment to another, and the camera will save an image of the new environment.

The camera is better than the diary

Such a life, in which many moments are automatically recorded, may seem to many of us to be strange or even discouraging (remember also that the "Big Brother" style reality shows reach a similar situation, for completely different reasons), but Gordon Bell already felt that his life was enriched by these aids. It seems as if he sees the project as an essential part of his personal identity: in the few cases where part of the information was lost due to malfunctions, he reacted as a "normal" person reacts when he discovers a missing part of his memory.

Now Bel can "remember" what happened at any moment he chooses, with the help of a simple search in the database where so many details about his life are stored. I wrote "to remember" because it is not the same experience as a memory that comes from our mind, but in fact it is known that there is no clear dividing line between "internal" and "external" memory: if we use an external reminder, we can retrieve our memory more easily and complete it with details from the information the external

In this context it is interesting to note that the same camera developed for MyLifeBits was made available to memory-impaired people. Each night these people reviewed the images saved by the camera during the day, and it was found that this review helped them better remember the events of the day: when asked about these events later, without any external help, they answered the questions better (thus the camera is better than written records: (When the same people wrote in a diary during the day and read it in the evening, their memory functions did not improve).

Many requirements from the system

Of course, saving the information is only part of the challenge: it is easy to associate each piece of information with the time it was recorded, and therefore it is easy to answer the question "Where was Gordon Bell and what was he doing on October 6, 2007?", or any other date. It is more difficult to answer the question "What was the topic of conversation the last time Gordon Bell met John?".

Although the conversation is recorded, along with John's picture, but to answer the question we will have to face many challenges: figure out which John it is about, retrieve his picture, search for the same picture in the huge image database, find the last time such a picture appears , decide whether the photo indicates a meeting and a conversation (and not, for example, a random situation where John happened to pass in front of the camera), scan the recording of the conversation and try to deduce from the recorded words what its main topic was.

This is a large collection of requirements, some of which are at the forefront of artificial intelligence research, or are still waiting for future developments: recognizing faces in images, understanding a social situation, deciphering words from a spoken conversation, and extracting topics from the text of the conversation. Currently, the project cannot meet such difficult challenges, but it is aided by the records that Bell or his assistants add to the recordings manually, to make it easier to classify and find the recordings in the future (for example, listing a person's name next to their picture or defining the context of conversations and meetings). Bell's memory, therefore, still cannot compete with Williams's, but it is less mysterious and in principle accessible to any of us.

The idea of ​​information technology as a tool to support memory is not new, and it appeared as early as 1945 when Vannevar Bush, one of the pioneers of analog computing, wrote an article called "As We May Think". The article described a system called Memex (short for "Memory Extension") that would allow a person to keep all the information he needs. The system was not implementable at the time, but Bush predicted that the required technology would appear in the future. Since it also included in the system the creation of links between different items of information, Memex is also an important milestone in the history of hypertext (the idea of ​​creating links between documents, which is the basis of the World-Wide Web).

Proactive retention of information

MyLifeBits is not the only effort of its kind. For example, in the MIT laboratories (Massachusetts Institute of Technology) a similar project called iRemember is being developed. One of the significant differences between iRemember and MyLifeBits is in their approach to recording the information: MyLifeBits takes an "open sensors" approach: everything that happens in Bell's environment is recorded. In contrast, iRemember will record and save information only when the user decides that it is necessary.

MyLifeBits takes an "open sensors" approach: everything that happens in the user's environment is recorded. In contrast, iRemember will record and save information only when the user decides that it is necessary

This proactive recording approach has several advantages: it requires a much smaller amount of storage space, and keeps only what the user thinks is worth keeping. An equally important set of considerations is related to privacy: even if the user does not hesitate to save every detail of his life in the system, (Gordon Bell tells of significant hesitations when he came across a harsh document he wrote and had to decide whether to include the document in the MyLifeBits memory pool; in the end, his decision was positive.), how ethical is it to record conversations with other people? Even if the user asks for their permission, this request may not be fair - for example when they depend on it for their livelihood.

At MIT, the developers gathered feedback on the recordings, and a significant percentage of respondents expressed discomfort with the idea. If this is the reaction in a group that is open to technological innovations, one can expect even greater resistance in the general public.

Of course, when the save is proactive, the user must remember to activate the system so that the system can later remind them of what happened. Is there a way to remind him to do so? One idea on this topic was also developed at MIT: sensors attached to the user's palm measure the conductance of the skin. When the sensors detect a change in skin conductivity, they can start recording or suggest the user do so (StartleCam). Changes in skin conductivity are linked not only to a situation of fear or stress but also to changes in the direction of attention, so they can hint at an event we want to remember.

Orientation in stored information

Whether the information is always stored or only at the user's initiative, regular use of memory support creates very large amounts of information. Some information is textual (for example, e-mail or text appearing on websites) and you can use a text search to locate it, but much information consists of recorded sounds and sights.

Today there are software for recording recorded conversations, i.e. turning them into text, but the quality of the recording may be low - especially when the recording is done in an uncontrolled environment where it is impossible to direct the microphone and prevent background noises. Most of the developments mentioned in this article use this technique. Some also adopt the latest developments in facial recognition to link the people seen in recorded images to their name and role. It is also possible to use text recognition (OCR - Optical Character Recognition) to turn text that appears in captured images into "live" text that can be searched.

Still, even if the most advanced capabilities of artificial intelligence are used to extract as much text as possible from the stored information, there is a problem with the idea of ​​using text search to support the recall process. Human memory mechanisms are not exactly based on text search, and many things we try to remember do not lend themselves to such a search: What should I have done following the last meeting that dealt with the new project? Who did I meet at lunch at the conference in Jerusalem last year?

The mobile phone stores memories

In the IBM research laboratory in Haifa, a memory support system was developed in recent years that offers a new method for orientation and search in the huge databases and "archives" stored by users of the system. The system, developed in the Social Technologies Group of the IBM Research Laboratory in Haifa, is called "IBM Experience Organizer". The developers, led by Eran Belinsky, presented a prototype of the system for the first time in July 2008. The inspiration for the system came from a magical tool used to store thoughts ("the tub" in the Hebrew translation, from the word "the gig") in the Harry Potter book series by JK Rowling. Today, IBM employees around the world are testing the system and the developers are testing its use, in order to learn from it in which directions to develop the system.

Like other memory support systems, the Experience Organizer can record conversations and images, and associate each detail with the time and place it was recorded. The recording is initiated by the user. This feature not only solves some of the data volume and privacy issues, but also allows the use of commercially available mobile phones (rather than the much more complex hardware needed for systems like MyLifeBits).

When the user wants to save a certain memory, he will simply take a picture, or record a video or speech. When the users return to the office and synchronize the mobile phone with the desktop computer, the raw information is transferred to the system, which processes it and turns it into a database available from any Internet browser (given the necessary passwords, of course).

Easy retrieval of the memory

One of the unique features of the new development is the use of information grouping (clustering) for linking memories and for easy retrieval of the required memory. For this purpose, the software uses additional sources of information: for example, the system can communicate with the person's schedule and appointments. If a certain photo was taken at a time when the meeting schedule shows the holding of a meeting, the photo will be linked to that meeting and therefore also to the contacts who participated in that meeting, to other pictures and recordings saved in that context.

Therefore, when a person tries to remember what he had to do following a certain meeting, he will be able to browse through a cluster of images and conversations associatively related to that meeting. In the same way, when he tries to remember where he met a certain person, the answer will be based not only on the place and date, but also on the context - at which meeting, in connection with which task, which other people he met at that event, etc.
In many cases, the mere study of the details linked to some event will strengthen the "real memories" - that is, those stored in the human nerve cells - and make it easier to retrieve them

The opposite direction also contributes to the creation of links and the grouping of information: if the lecturer at a certain event accompanies his words in a presentation, and the user takes pictures of some of the slides, the software will be able to recognize certain structures of text in these photographs: the name of the lecturer and his e-mail address (which usually appear on one of the first slides) will be linked to the lecturer's picture and will be entered in the "address book"; Website addresses it refers to will be identified within the images and saved for later reference; When a future event is mentioned, the topic, time and location will be extracted from the slide captured by the mobile phone camera and recorded in the user's meeting schedule; Photographs of business cards will be deciphered by deciphering name, position, addresses and phone numbers and will be associated with photographs of people taken at the same time.

If the user also spends a few minutes at the end of the day to "tag" certain recordings by linking them to topics in which he is interested, to projects, tasks, etc., the associative contexts between the pieces of information will be even stronger, and retrieving the information - even years later - will be even easier and faster .

In many cases, the mere study of the details linked to some event will strengthen the "real memories" - that is, those stored in the human neurons - and make it easier to retrieve them, so that the result is not only the retention of information that we can no longer retrieve in any other way, but support and encouragement of the internal memory retrieval mechanisms . This result is supported by psychological studies of memory formation and retrieval, and by reports from users of Experience Organizer.

Social memory

From time immemorial, human memory has been stored both within the brain and by interacting with the environment, in actions such as carving grooves on trees to remember the walking route, tying knots with ropes, writing, asking other people to remember something, etc. The second type of storage allows social sharing of the memory and its inheritance for future generations. It is impossible to imagine the development of a culture without such sharing. That is why the development of culture was so accelerated by technologies for preserving and sharing memory, such as the invention of writing followed by the invention of printing.

IBM Experience Organizer enables social use and sharing of the memories it stores. In doing so, he combines in the same framework the empowerment of personal memory with the sharing of information within the organization and the social context. In this way, an employee who went on a business trip will be able to spread his impressions and the information he gathered, and these will be added to the memory pool available to his co-workers. If the schedule differentiates between private and work-related events, the system will use this information to know which information is correct to distribute and which information to keep for access by that employee (the employee will be able to control these decisions when he tags the saved information).

We started with those rare people who remember everything. The technology described here does not compete with their wonderful abilities, but it has the potential to allow each of us to remember much more and be less exposed to the frustrations and failures of memory. It also has an important new feature - not just a better memory for each person per se, but the creation of general memory banks, created by the contributions of the memories of many people, so that retrieving information from these banks is very similar to personal recall.

Just as it is difficult to describe how we would have found information before the creation of the search engines on the Internet, it is possible that in not many years we will not be able to imagine how we relied only on our nerve cells to remember our lives and our personal experiences, and in the same way the barrier between our memories and the memories of others will be lowered somewhat.

Israel Binyamini works at ClickSoftware developing advanced optimization methods. The article appeared in issue 123 of the Galileo journal.

9 תגובות

  1. in light of

    In one of the science programs on TV, I saw an experiment they conducted on an ape and children. Ten digits from 0 to 9 appeared scattered on a television screen. As soon as you clicked on the number 0, all the digits become X's, and the examinee is supposed to continue pressing the digits according to the ascending order of the digits that were there.
    Needless to say, the man knew how to repeat the letters over and over and the children usually got confused. The article states that this is a feature of short-term memory that was lost to man in his evolutionary development. Really interesting. The question is whether it is not possible to find this lost gene and strengthen it.
    Good Day
    Sabdarmish Yehuda

  2. Very interesting article, as usual with Israel Benjamin. Really an important and fascinating topic.
    I hope to see the day when nanorobots will be implanted in our brains and allow us to extend the capabilities of the human brain. It would be something truly amazing!

  3. Has anyone investigated whether and to what extent the trait of amazing memory is hereditary? Does it also appear in animals?

  4. One businessman arrived at a fancy hotel in Nevada and saw in the lobby an old Indian man dressed in traditional clothing that did not fit the typical human landscape of the place.
    He asked one of the workers who the man was and the worker answered that he was the head of one of the tribes that lived there and as a sign of respect and also as compensation for the expropriation of the Indian lands they decided at the hotel to let him stay there as long as he wanted.
    But, says the hotel employee, it's not just any Indian.
    This man simply remembers everything! Everything that happened to him from the day he was on his mind.
    The businessman is intrigued and decides to talk to the Indian.
    This is the place to mention that the origin of the joke is in English and this is important for the continuation.
    The man approaches the Indian and says "hi".
    "Hi," the Indian replies.
    After a light polite conversation, the man turns to the main thing and asks:
    "I heard you remember everything. Is that true?"
    "Yes," answers the Indian.
    "So tell me please, what did you have for breakfast three days before your third birthday?"
    "Egg," answers the Indian.
    Good; No one has any way to check this but he believes and is impressed.
    Over the next few years, during his various travels, he meets all kinds of people and tells them about the Indian.
    People are impressed but one of them remarks to him that it is not correct to address an Indian with the word "hi" as he did, because among Indians the word "hao" is used in this role.
    After a few years, the man returns to the same hotel and again sees the Indian in the lobby.
    Since the insider understood what he was told, he signals to the Indian to say hello and says "haw".
    "Scrambled" the Indian answers him.

  5. Elle McPherson (the model) once said in an interview with David Letterman that she remembers every day
    Mocha, when he asked her about a certain date, she remembered a more significant event that happened a few days later and went back (in her mind) to that day - she was probably sitting in her hotel room that day.
    As said both beautiful and baker.

  6. I wonder what grades they graduated from school and how well they process the vast amount of information in their heads, or are they just a database
    good evening
    Sabdarmish Yehuda

Leave a Reply

Email will not be published. Required fields are marked *

This site uses Akismat to prevent spam messages. Click here to learn how your response data is processed.