See the night in a new light

Researchers at the Technion and the University of Toronto have developed technology to generate knowledge about the environment - from the level of the office to the entire city - based on the flickering of light bulbs

Artificial lighting fills our lives at home, in the office, on the road and in other places. This lighting is produced by a variety of bulbs found in street lights, offices, spotlights, car headlights, billboards and computer monitors. All the lamps connected to the mains flicker continuously, but due to the high flickering speed we do not feel these flickering.

It is now becoming clear that the flickering of artificial lighting may provide extensive useful information. In a study that will be presented this week at the conference IEEE CVPR To be held in Hawaii, researchers from the Technion and the University of Toronto will present a new way to extract information about the environment from the jitter patterns of artificially lit arenas. The approach combines various research fields including optics, computer vision, image processing and electrical network engineering.

The co-researchers in the study are Ph.D. student Mark Sheinin and Prof. Yoav Shechner from the Viterbi Faculty of Electrical Engineering at the Technion and their colleague Prof. Kyros Koutoulakos from the University of Toronto. The three developed a system that produces the aforementioned information from a passive video recording, i.e. without additional lighting, of the desired scene: an office, A corridor, a street and even an entire city. From the analysis of the information obtained from the photograph, the system deduces, for example, how it was The scene is visible if we turn off some of the bulbs, increase them or replace some of them with other types of bulbs. This system also makes it possible to eliminate, in digital processing of the photograph, the reflections returned from windows when we look through them; to eliminate the shadow that artificial lighting creates on our face (selfie ); and analyze the urban lighting network and the types of bulbs used in it, all through passive video recording.

Prof. Shrinivas Narasimhan from the School of Computer Science at Carnegie Mellon University, who was not involved in the research, said that the paper demonstrates "very innovative research that extracts significant information from an optical phenomenon that we do not normally notice. This development has significant potential applications including light pollution monitoring, pollution monitoring Air at night, images of areas that are not in the viewer's field of vision and monitoring of electricity consumption and fluctuations in the electricity current."

The reason for the flickering of the artificial light is that electrical networks operate on alternating current (AC), where the flow of electrons constantly reverses its direction in a cyclical manner. In Israel, for example, the frequency in the electricity network is 50 Hz, and this means that the current of electrons changes its direction 100 times per second and this is the rate at which the light bulb blinks. The flickering pattern depends on the type of bulb - fluorescent, mercury, halogen, LED, etc. - since each bulb converts electrical energy into light in a different process. In other words, Each bulb type has a unique time signature. It is the time signature that allows the researchers, using the system and a dedicated algorithm they developed for this purpose, to identify the type of bulb.

For photographers, photographing the flicker is a challenge that has a contradiction: on the one hand, to detect the dynamics of the flicker, a very short exposure is required. On the other hand, shooting at night requires a long exposure to collect enough light to create a picture. To solve the contradiction, the researchers developed a unique electro-optical camera called ACam that is able to "sense" the flickering of the alternating current. The camera, which is itself connected to the mains, takes advantage of the jitter cycles to capture fast signals from the arena. The electronic shutter of the camera is open the entire time of the photograph, but the sensor is optically exposed only for the desired time segment in each flashing cycle.

The researchers note that the technology they developed paves the way for further research regarding a wide variety of tasks, including controlled illumination of objects, measuring XNUMXD objects, identifying their surface texture based on the shadow they cast, and analyzing the characteristics of the power grid remotely in an optical way.

The research grew out of Prof. Shechner's interest in a completely different field. Being an amateur astronomer, well aware of the limitations that urban light pollution places on night observations, he hypothesized that if we connect the fractions of a second where the light is minimal during the jitter, this will neutralize the light pollution and allow a clean observation of the sky. The researchers soon discovered that the various light bulbs throughout the city were not flickering in sync; As soon as the light in one of them drops to the minimum point, another bulb actually brightens. "Therefore," he says, "the astronomical application has been shelved for now, but this is how we discovered a land unknown to computer vision: the power grid."

the article Computational Imaging on the Electric Grid Will be presented on July 22 at the IEEE CVPR conference held by the Society of Electrical and Electronics Engineers (IEEE). The research was supported by the Taub Foundation, the National Science Foundation (ISF) and the German Minerva Foundation.