“Forced” to change the look of a man on his video directing in the desired direction in real time, learned experts of the SKOLKOVO Institute of science and technology. In the first place the invention aims to help users of video systems via the Internet, which is usually aimed anywhere but at the camera, causing the quality of communication often suffers because that there is no direct eye contact.
As told “MK” study co-author, associate Professor Victor Lempitsky, in order to create the algorithm that artificially changes the direction of gaze of the speaker, scientists had to collect a fairly extensive database of images of eyes of different people. They were recorded by a camera installed in the middle of the screen, went to the computer memory and then processed by the computer within the machine learning process . Next, it was necessary to solve the problem of the unification of distortion of the view, that is to teach the computer to change the image of the eye of any user, not just those listed in the database. It turned out that despite the individuality of our eyes, changes of angles (directions left, right, up, down) in the video look about the same. In the end, to draw the eye was not necessary. In order to “rotate” the view of the user staring in the wrong direction, the computer uses its own image. And the distortion, i.e. right turn, migrates from the eyes of another individual present in the computer’s memory.
Analogs of such system does not yet exist anywhere in the world. Meanwhile, technology can help photographers, filmmakers, who need to quickly achieve the desired look of the hero, the TV people who need to look at the audience, reading the text simultaneously. In October the results of the study will be presented in Amsterdam on the 14-th European conference on computer vision.