At this summer's Siggraph — the premier computer-graphics conference — researchers from MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) will present new software that amplifies variations in successive frames of video that are imperceptible to the naked eye. So, for instance, the software makes it possible to actually "see" someone's pulse, as the skin reddens and pales with the flow of blood, and it can exaggerate tiny motions, making visible the vibrations of individual guitar strings or the breathing of a swaddled infant in a neonatal intensive care unit.
The system is somewhat akin to the equalizer in a stereo sound system, which
boosts some frequencies and cuts others, except that the pertinent frequency
is the frequency of color changes in a sequence of video frames, not the
frequency of an audio signal. The prototype of the software allows the user to
specify the frequency range of interest and the degree of amplification. The
software works in real time and displays both the original video and the
altered version of the video, with changes magnified.
Although the technique lends itself most naturally to phenomena that recur at regular intervals — such as the beating of a heart, the movement of a vibrating string or the inflation of the lungs — if the range of frequencies is wide enough, the system can amplify changes that occur only once. So, for instance, it could be used to compare different images of the same scene, allowing the user to easily pick out changes that might otherwise go unnoticed. In one set of experiments, the system was able to dramatically amplify the movement of shadows in a street scene photographed only twice, at an interval of about 15 seconds.
Maneesh Agrawala, an associate professor in the electrical engineering and computer science department at the University of California at Berkeley, and director of the department's Visualization Lab, says: "The simplicity of the approach makes it something that has the possibility for application in a number of places. I think we'll see a lot of people implementing it because it's fairly straightforward."
|
| Overview of the Eulerian video magnification framework. The system first decomposes the input video sequence into different spatial frequency bands, and applies the same temporal filter to all bands. The filtered spatial bands are then amplified by a given factor α, added back to the original signal, and collapsed to generate the output video. The choice of temporal filter and amplification factors can be tuned to support different applications. Credit: H.-Y. Wu, M. Rubinstein, E. Shih, J. Guttag, F. Durand, W. Freeman / MIT CSAIL / Quanta Research Cambridge |
|
| Eulerian video magnification used to amplify subtle motions of blood vessels arising from blood flow. For this video, the authors tuned the temporal filter to a frequency band that includes the heart rate—0.88 Hz (53 bpm)—and set the amplification factor to α = 10. To reduce motion magnification of irrelevant objects, they applied a user-given mask to amplify the area near the wrist only. Movement of the radial and ulnar arteries can barely be seen in the input video (a) taken with a standard point-and-shoot camera, but is significantly more noticeable in the motion-magnified output (b). The motion of the pulsing arteries is more visible when observing a spatio-temporal Y T slice of the wrist (a) and (b). Credit: H.-Y. Wu, M. Rubinstein, E. Shih, J. Guttag, F. Durand, W. Freeman / MIT CSAIL / Quanta Research Cambridge |
The video accompanying the SIGGRAPH 2012 paper "Eulerian Video
Magnification for Revealing Subtle Changes in the World".
Sources:
- Larry Hardesty, Researchers amplify variations in video, making the invisible visible, MIT News Office, June 22, 2012
- Eulerian Video Magnification for Revealing Subtle Changes in the World, Project website
- Motion Magnification, presented at SIGGRAPH2005, Los Angeles
