see sounds and hear shapes:

Researchers at the Hebrew University have developed a unique training method that will allow the blind to see with the help of a sensory transduction device * The researchers have also developed a device that allows the blind to make quick and precise movements towards targets

Researchers from the Hebrew University have shown that with the right stimulation and using a device for sensory transduction, blind people from birth can "see" and describe objects and even recognize letters and words, thereby activating their visual cortex. Moreover, the researchers found that with the help of the device it is possible to make quick and precise movements towards targets.

The research was carried out by a team led by Prof. Amir Amadi from the Edmond and Lily Safra Center for Brain Research and the Israel-Canada Medical Research Institute and PhD student Ella Shetrim-Amit, in collaboration with Prof. Laurent Cohen and Prof. Stanislas DeHaan from Paris, and was published this month in the leading research journal Brain Neuron. In addition, a summary of the research appeared in the prestigious journal Science.

The researchers developed a unique vision training program with the help of a sensory transduction device, which transmits visual information to the blind through their normal senses. Using an audio-visual sensory transduction device, which translates images into sounds, the researchers showed that after a few tens of hours of training, blind people from birth can recognize images with the device and place them into visual categories such as faces, houses, body shapes, everyday objects and textures. The subjects could even place people in space, recognize facial expressions and read letters and words (demonstration, films and additional information can be found at http://brain.huji.ac.il/). The subjects reached a visual acuity that passes the threshold defined by the World Health Organization as the amount of blindness.

According to Prof. Amadi, "It has been known for decades that if the visual cortex does not receive input and visual information after birth, it does not develop its structure and specializations properly and vision restoration is not possible later in life. However, when we examined what was happening in the brains of the blind who learned to see through sounds, we discovered that their visual cortex was activated with the help of the sounds even though they only learned to process images in adulthood."

The researchers also found that the subjects' brains showed visual preferences that characterize the brains of sighted people when responding to different types of visual stimulation. For example, the part of the brain used for reading showed in the blind, as in sighted people, increased activity in response to pictures of letters and words. Moreover, this area was found to be so flexible that in one of the subjects it increased its activity to pictures of letters already after a short training of about two hours.

"The brain of older people is more flexible than we thought" says Prof. Amadi. "These findings indicate that it is possible that the brains of blind people, even those who suffer from long-term blindness, can 'awaken' to process vision with the help of rehabilitation that also includes new and future medical developments such as retinal implants (artificial eyes), and using modern learning and rehabilitation methods."

In a follow-up study, carried out by Prof. Amadi's team in collaboration with research associate Dr. Shelly Levy-Tzedek and published in the well-respected journal Restorative Neurology and Neuroscience, the researchers also found that with the help of a sensory transduction device, blindfolded subjects can make quick and precise movements towards targets.

To this end, the research team developed an innovative device, which makes use of pleasant musical sounds, in order to help the blind "see" through sound. This device is non-invasive and represents images using a combination of musical notes. The device was developed in Prof. Amir Amadi's laboratory with the assistance of research students Shlomi Hanasi, Sami Abode and Shahar Meid Nabaum, and is called EyeMusic. The EyeMusic represents each of five colors using a different musical instrument: white-choir, blue-trumpet, red-organ, green-wind instrument, yellow-violin. Black is represented by silence (in future versions, the musical instruments will be able to be chosen by the users). The notes played are spread over five octaves in a pentatonic scale, and were carefully selected by musicians in order to create a pleasant experience for users. (Recorded examples from the device can be found here).

The researchers showed that after a very short training period (in some cases, less than half an hour), 18 blindfolded subjects were able to make fast and precise movements towards targets they "saw" using the EyeMusic. These movements were very similar to the movements made by the subjects towards targets they saw without the blindfold.

Prof. Amadi and Dr. Levy-Tzedek showed, with another group of subjects, that there is an unconscious transfer of information about the environment (such as the location of objects) between the sense of sight and the sense of hearing when EyeMusic is used. This result was published in the prestigious journal Scientific Reports, and it paves the way for the development of hybrid aids for the blind: these will combine intraocular implant input, or impaired vision, together with input from the EyeMusic device.

"The results of the study demonstrate the potential of using the EyeMusic device in performing everyday actions, such as choosing a red (but not a green!) apple from the jade," concludes Prof. Amadi.