Intel presents: multi-sensory computing

This week the Intel company issued an unusual request to the application developers, talk to the computer

This week the Intel company issued an unusual request to the application developers, talk to the computer. Intel calls on application developers to develop speech recognition applications, and announced that the Intel Perceptual Computing Software Development Kit (SDK) - Now available for download here.

Intel believes that perceptual computing is expected to revolutionize the way people interact with their PCs. The software development kit that Intel now offers in the field of perceptual computing allows developers to develop applications around capabilities such as close-up facial recognition, tracking the users' finger, facial analysis, tracking two and three-dimensional objects - and now also speech recognition.

Intel calls on app developers to develop speech recognition apps. The company announced that the Intel Perceptual Computing Software Development Kit (SDK), which should support Nuance's speech recognition, is now available for download. Developers will now be able to add voice commands, short sentences and text-to-speech to applications they develop for computers using Intel's Core series processors.

At the development center in Haifa, a team led by Molly Aden, president of Intel Israel, is currently working on the topic of perceptual computing. The team is based on the graduates of the start-up company Invision Mikneam, which was purchased by Intel, along with other company employees, who are now in the perceptual computing group at Intel Haifa.

To understand what it is about, and especially the context of the announcement, which came shortly after a similar announcement about gesture recognition using a camera, intended for those developers, we spoke with Yigal Yanko, responsible for strategic planning in the field of perceptual computing at Intel worldwide at the Intel development center in Haifa.

"In terms of Intel's announcements, this is a feature addition to the SDK that we announced at the Intel Developer Forum (IDF) last September. We promised to add features and about a month ago we announced gesture recognition features using a camera that captures XNUMXD images, and this time we added voice capability in collaboration with the Newnans company.

Intel provides a development kit - not products at the level of end products. However, we offered them a number of basic applications to demonstrate the capabilities of the technology and from there everyone could take it according to their imagination and possibilities."

The death of the QWERTY keyboard
According to Janko, the power of computing is constantly increasing, but for decades the interfaces with it have not changed. "Not many know, but the QWERTY keyboard was developed to slow down typing on typewriters. In recent years, people have started to get used to using touch screens, this is a small step in the right direction, but not enough."
"The future is in the natural interfaces" Yanko explains. "We want an intuitive experience that will allow more people to use computing. We return to our origin - to the evolution of the eyes, ears and senses of touch and sound and prefer that the computer adapt itself to us and not that we adapt ourselves to the computer. A keyboard and mouse are not means of communication that people use to communicate with each other but are forced to use when communicating with machines. Voice recognition or hand gestures are much more intuitive, which is why we call it "perceptual computing" because it allows the use of the senses in their most basic form."

What are the difficulties in implementation?
Janko: The main difficulty in implementation is the need to convince the people to use. The history of the QWERTY proves that it is very difficult to change habits. Touch screens have also existed for 15 years, but only since the introduction of the iPhone has this feature come into mass use. In addition, of course, the more intuitive the interface, the more processing power is required, and the components of the computer need to be more powerful - whether it's cameras that need to be three-dimensional like our eyes, or the voice processing system that requires a lot of power because it doesn't work in an environment of complete silence and in a number Limited words from a specific speaker in a specific application but in any background noise conditions, with an infinite number of words and from each user. The system also needs to understand what is said to it from the context, as we do."

"In the natural experience we work with a combination of the senses, also in voice communication, like a phone call, there is intonation, we filter out the background noise. However, in a face-to-face conversation, people speak with their eyes, hands, voice, lip movements - verbal and non-verbal communication. The platform that Intel wants to provide is to combine these senses together in a natural experience."

for example?
"In reality, if someone gives me a stop or just a moment hand signal, I quickly understand. For a computer this is a difficult task of image processing. These are very complex images that need to be processed and understood what they mean. Not long ago, a peripheral camera came out, which you connect to the Soft Kinetic Ultra Book and it is able to recognize such gestures."

Will it be possible to say that perceptual computing will only be complete when a digital replacement for all the senses is found?

"Today we provide developers with basic camera support, and basic systems for voice recognition and gesture recognition. You don't have to wait for everything to be perfect and then combine them. It should be remembered that this is similar to the human body: the superiority of one sense compensates for the inferiority of another sense. As technology advances over the years we will be able to support more advanced devices. "

As an example of perceptual computing applications, Intel provided the "God Game" that allows moving the bodies in the solar system. Another game that he developed through the camera manufacturer Soft Kinetic is 'Lightning' which allows lightning to be produced in augmented reality between the palms of the hands according to their body gestures. In another Softkinetic game, called Ballista, players "shoot" slingshots and cannonballs by recognizing their body gestures.

To realize the perceptual computing. As the computer advances in the demand to imitate nature, it requires a large data processing capacity, whether it is the resolution of real-time video, or more accurate speech recognition. The things are integrated into Intel platforms that provide high computing power with low power consumption. In another game you can shoot XNUMXD balls on a fort.

"We are simultaneously working on the devices simulating senses such as cameras, speech engines, and other means that we cannot mention at the moment, and integrating them into one development environment. I hope that, as happened with the iPhone, the developer community found the right combinations, many things that we did not think of, I'm sure will happen. In the development environment, Intel provided demo examples for developers that show what can be done, but the idea is that the imagination does not remain only with us and I hope that people will do many more things with it beyond what we thought", Janko says in conclusion.