The system combines augmented reality tools and robotics, in order to help engineers and service technicians working in the field to locate the items of equipment, present them with critical information and provide them with visual support from experts, who are sitting far from the site where the equipment requiring maintenance is operating.
IBM unveils a prototype for a new system for maintenance, repair and operation using mobile tools, designed to help manufacturers and operators in sectors with high added value, such as the aviation, oil and energy and shipping industries. The new system is based on unique technologies developed in IBM's research lab in Haifa and the software lab in Hursley, UK.
The system combines augmented reality tools and robotics, in order to help engineers and service technicians working in the field to locate the items of equipment, present them with critical information and provide them with visual support from experts, who are sitting far from the site where the equipment requiring maintenance is operating.
According to Assaf Zadok, director of image analysis at IBM's research laboratory in Haifa, the project is intended to allow an employee in the field to be assisted by experts sitting at a remote site. The system allows the expert to observe what is happening at the work site, to talk to the field person who needs advice and assistance, and to explain to him what he must do. "The employee receiving remote assistance will feel that he is working within his natural environment, without the need to hold an additional device, wear a dedicated helmet or carry a camera on his body. We create a transparent environment where people may forget the very existence of the other machine working next to them - and feel as if they are next to the expert guide. They can be helped by production sites operating in another country, by planners sitting at a third site, or by command and control personnel at a fourth location. Companies operating today in such a layout are sometimes required to send people to work where they are needed. Through the system, they get to work immediately, anywhere in the world."
Zadok points out that the most essential innovation in the system is the integration of augmented reality, which is a way to make information accessible in a natural and clear way. He predicts that this strategy will continue and gain momentum in the near future, as a tool for bringing essential information to the people who need it, wherever they are and at any time. Thus, for example, IBM researchers presented a system for shopping assistance using augmented reality on smart phones at this year's Savit exhibition in Hanover.
At the heart of the new system is a robotic arm controlled remotely by an expert sitting at a central site. The arm allows this expert to observe what is happening on the job site, and when necessary, present information to the technician in the field.
Typical maintenance tasks today require the service person to go to the site, find the device or component that needs service, and ensure the availability of the appropriate tool kit to service that equipment. In any case of difficulty, the technician will call and ask for help from a support person or product expert. The new system allows the specialist sitting at a remote site to monitor the technician's progress on his way to the site where the service is required - using a GPS system. During the work itself, the technician can use his smartphone and a QR code to locate and identify the item of equipment, and receive work instructions. The phone uses augmented reality technology to display points that require the technician's attention and overlay them on the site plan or other maps. Thus, it is possible to identify the location of other technicians, direct people to the nearest first aid points, or indicate the location of safety, rescue or firefighting equipment.
An expert sitting at a remote site can observe the workspace where the technician is located and assist him in real time by displaying links to video or audio clips. This expert uses a tiny camera and projector, installed at the end of a robotic arm, which he controls from his seat. The specialist can use this arm to point out points where the service person needs to focus, draw diagrams freehand, give assembly instructions and deliver XNUMXD images directly to the technician's workspace.
The prototype that was just presented combines two innovative tools developed in Hersley and Haifa in order to present one comprehensive solution to customers. It makes it possible to reduce service costs, transfer knowledge when and where it is needed, and reduce the risks to which engineers and service personnel are exposed in difficult working environments. The remote consulting system Teleadvisor, developed in the IBM research laboratory in Haifa, controls the robotic arm and provides the expert remote from the work site with an easy-to-use control console - all in combination with existing maintenance management systems and asset management solutions, such as the Maximo tool and IBM's software systems for product life cycle management.
A series of studies has proven that remote support is more effective when the engineers in the field and the support personnel at the central hub can share a visual representation of the work space on the site. To date, this goal has only been partially achieved, through the use of cameras held by the service engineers in the field, cameras on helmets or special glasses. The new system offers the engineer in charge sitting at the central site complete independence in video operation and a more stable image. The engineers in the field can work without limitations, while being freed from the need to ensure image transfer to the remote specialist.
