MIT Video Tool Amplifies Micro-Motions to Track Vitals in Real Time

Miki Rubinstein

Applying spatial decomposition and temporal bandpass filtering to standard video amplifies imperceptible color and physical shifts, enabling non-contact vital sign extraction that matches clinical monitors.

AI-assisted summary of Eulerian Video Magnification — original source. Check the original for full context.

Source published: 2012-05-23

Key ideas

Researchers present Eulerian Video Magnification, a computational framework that combines spatial Laplacian decomposition and temporal bandpass filtering to amplify imperceptible video variations in real time without explicit feature tracking. (0:00)
By isolating frequencies corresponding to human heart rates, the system amplifies subtle facial chromatic shifts to non-invasively measure pulse, demonstrating clinical accuracy against standard hospital equipment on a newborn. (1:24)
Using a first-order Taylor series approximation, the method scales temporal signal derivatives to amplify physical spatial displacements while strictly maintaining the original temporal frequency of the motion. (2:08)
The framework enables diverse cross-disciplinary applications, including non-contact infant respiration monitoring, acoustic analysis of individual guitar strings, and mechanical diagnostics of DSLR mirror recoil. (2:08)

Detailed notes

## Background Published on YouTube by Miki Rubinstein on May 23, 2012, this video serves as the visual accompaniment to the SIGGRAPH 2012 paper 'Eulerian Video Magnification for Revealing Subtle Changes in the World.' The project description directs viewers to the research team's MIT CSAIL project page for deeper technical details.

## TL;DR Eulerian Video Magnification introduces a computational technique that reveals imperceptible temporal changes in standard video by combining spatial Laplacian pyramid decomposition with temporal bandpass filtering. Operating in real time as a temporal microscope, the system allows users to interactively tune frequency bands and magnification sliders to visualize minute variations. The approach effectively amplifies subtle color fluctuations to extract vital signs, such as pulse rate and infant respiration, with clinical accuracy across various skin tones. Furthermore, by linking temporal filtering to a first-order spatial Taylor series approximation, the method amplifies physical motions to expose unseen mechanical vibrations in guitar strings and camera bodies.

## Atmosphere & style The presentation carries the poised, measured tone of a high-level academic research paper, prioritizing clarity and scientific precision over flashiness. Watching the footage evokes quiet awe as standard consumer recordings are peeled back to reveal invisible biological pulses, infant breathing, and hidden mechanical tremors. The narrator delivers deliberate, clinical explanations that ground each demonstration, creating the experience of looking through an optical microscope that uncovers a hidden layer of physical reality.

## Bulletin - [00:00](https://www.youtube.com/watch?v=ONZcjs1Pjmk&t=0s) Researchers present Eulerian Video Magnification, a computational framework that combines spatial Laplacian decomposition and temporal bandpass filtering to amplify imperceptible video variations in real time without explicit feature tracking. - [01:24](https://www.youtube.com/watch?v=ONZcjs1Pjmk&t=84s) By isolating frequencies corresponding to human heart rates, the system amplifies subtle facial chromatic shifts to non-invasively measure pulse, demonstrating clinical accuracy against standard hospital equipment on a newborn. - [02:08](https://www.youtube.com/watch?v=ONZcjs1Pjmk&t=128s) Using a first-order Taylor series approximation, the method scales temporal signal derivatives to amplify physical spatial displacements while strictly maintaining the original temporal frequency of the motion. - [02:08](https://www.youtube.com/watch?v=ONZcjs1Pjmk&t=128s) The framework enables diverse cross-disciplinary applications, including non-contact infant respiration monitoring, acoustic analysis of individual guitar strings, and mechanical diagnostics of DSLR mirror recoil.

## Key takeaways - [00:00](https://www.youtube.com/watch?v=ONZcjs1Pjmk&t=0s) Eulerian Video Magnification amplifies invisible temporal variations in standard video sequences without tracking physical features explicitly. - [00:00](https://www.youtube.com/watch?v=ONZcjs1Pjmk&t=0s) The spatial pipeline decomposes video frames into separate frequency bands using a multiscale Laplacian pyramid. - [00:00](https://www.youtube.com/watch?v=ONZcjs1Pjmk&t=0s) Temporal bandpass filtering isolates specific frequency bands across the temporal sequence of pixel values. - [00:00](https://www.youtube.com/watch?v=ONZcjs1Pjmk&t=0s) Magnified temporal signals are added back to the original frames and collapsed across pyramid levels to reconstruct the final video. - [00:00](https://www.youtube.com/watch?v=ONZcjs1Pjmk&t=0s) The system operates in real time as an interactive temporal microscope, allowing users to tune cutoff frequencies and amplification factors dynamically. - [00:00](https://www.youtube.com/watch?v=ONZcjs1Pjmk&t=0s) Higher amplification factors boost the visibility of minute changes, but excessive magnification introduces noticeable visual artifacts. - [01:24](https://www.youtube.com/watch?v=ONZcjs1Pjmk&t=84s) Targeting standard heart rate frequencies amplifies subtle facial redness changes caused by the circulation of oxygenated blood. - [01:24](https://www.youtube.com/watch?v=ONZcjs1Pjmk&t=84s) The physiological visualization technique remains robust across different skin tones and small involuntary subject movements. - [01:24](https://www.youtube.com/watch?v=ONZcjs1Pjmk&t=84s) Non-contact pulse extraction from standard video of a newborn infant was validated against synchronized hospital diagnostic monitors. - [02:08](https://www.youtube.com/watch?v=ONZcjs1Pjmk&t=128s) Temporal filtering acts as a first-order Taylor series approximation of spatial wave translation, extending the technique to physical motion magnification. - [02:08](https://www.youtube.com/watch?v=ONZcjs1Pjmk&t=128s) Synthetic benchmarks confirm that spatial motion can be exaggerated while strictly preserving the temporal frequency of the underlying movement. - [02:08](https://www.youtube.com/watch?v=ONZcjs1Pjmk&t=128s) Medical uses include non-contact monitoring of infant respiration for SIDS detection and visualizing subcutaneous arterial pulsation. - [02:08](https://www.youtube.com/watch?v=ONZcjs1Pjmk&t=128s) In acoustic analysis, frequency-selective filtering isolates and magnifies the fundamental vibration of individual plucked guitar strings. - [02:08](https://www.youtube.com/watch?v=ONZcjs1Pjmk&t=128s) Mechanical diagnostic applications reveal internal camera vibrations caused by the rapid flipping of a DSLR mirror during continuous shooting.

## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=ONZcjs1Pjmk&t=0s) Eulerian Video Magnification Pipeline and Real-Time Control

Eulerian Video Magnification is presented as a method designed to amplify and reveal subtle temporal variations in standard video sequences that are invisible to the naked human eye. Rather than tracking features explicitly, the technique combines spatial and temporal processing to accentuate fine changes across time. The pipeline begins by taking a standard video as input and decomposing each frame into various spatial frequency bands using a Laplacian pyramid. Across the sequence of pixel values over time, a temporal bandpass filter is applied to isolate the specific frequency band of interest. Once extracted, this temporal signal is magnified by a user-defined amplification factor, added back onto the original frame representations, and finally collapsed across the pyramid levels to reconstruct the output video.

```mermaid flowchart LR accTitle: Eulerian Video Magnification Pipeline accDescr: Video frames undergo Laplacian pyramid decomposition, temporal bandpass filtering, amplification, and reconstruction. subgraph SpatialDecomp["Spatial Processing"] A["Input Video Frames"] --> B["Laplacian Pyramid Decomposition"] end subgraph TemporalProc["Temporal Processing"] B --> C["Temporal Bandpass Filter"] C --> D["Signal Amplification"] end subgraph Reconstruction["Reconstruction"] D --> E["Add Back to Original Frames"] E --> F["Collapse Pyramid to Output Video"] end ```

The system is efficient enough to run in real time, functioning interactively as what the narrator characterizes as a temporal microscope: “Our technique can be run in real time, essentially serving as a microscope for temporal variations.” A user interface enables interactive exploration by letting users adjust the high and low cutoff frequencies of the temporal bandpass filter, alongside a slider to scale the amplification factor. Increasing the amplification factor boosts the visibility of minute fluctuations in the scene; however, the narrator cautions that this creates an operational tradeoff: “Changing these parameters can make variations in the scene more apparent, but large amplification can add artifacts to the result as discussed in the paper.”

- **Spatial decomposition**: Multiscale Laplacian pyramid separates spatial frequency bands. - **Temporal filtering**: Bandpass filter targets specific oscillation frequencies across time. - **Interactivity**: User-adjustable high/low cutoff frequencies and magnification slider operate in real time.

---

[01:24](https://www.youtube.com/watch?v=ONZcjs1Pjmk&t=84s) Visualizing Physiological Signals: Heart Rate and Newborn Monitoring

The framework is applied to reveal imperceptible biological phenomena, most notably the human pulse. By configuring the temporal filter to isolate a narrow band around the standard human heart rate, the algorithm amplifies subtle chromatic shifts across the face, exposing the rhythmic flush of redness as oxygenated blood circulates through facial tissue with each heartbeat. The narrator notes that “this approach is robust to different skin tones and small motions,” making it an effective non-contact tool for physiological measurement.

Beyond qualitative visualization, the method supports automated extraction of vital signs directly from standard consumer video recordings. In a clinical demonstration, Eulerian magnification is used to extract heart rate readings from video footage of a newborn infant. The extracted pulse measurements are validated against simultaneous, synchronized readouts from standard hospital monitoring equipment, demonstrating high diagnostic accuracy without requiring physical sensors attached to fragile skin.

- **Pulse detection**: Amplifies cyclic color changes corresponding to blood perfusion. - **Clinical validation**: Non-contact vital sign extraction tested on a newborn matches hospital vitals monitors in real time. - **Robustness**: Tolerates subject movement and functions reliably across diverse skin pigmentation.

---

[02:08](https://www.youtube.com/watch?v=ONZcjs1Pjmk&t=128s) From Color Changes to Motion Amplification and Everyday Applications

Temporal color filtering can be extended to amplify spatial motion. Using a 1D wave translation model, the narrator explains that for smooth images undergoing small displacements, temporal filtering provides a first-order Taylor series approximation of a translated spatial wave. Scaling this temporal derivative signal approximates a larger spatial shift, effectively converting subtle physical movements into exaggerated motions. A synthetic benchmark of oscillating blobs demonstrates that applying a temporal bandpass filter isolates and magnifies specific motion frequencies while strictly preserving the temporal frequency of the original movement: “Notice that the same temporal filter is applied to all pixels and while the spatial motion is amplified, the temporal frequency is maintained.”

```mermaid flowchart TD accTitle: Motion Amplification Concept accDescr: Relationship between spatial motion, temporal filtering, and selective frequency amplification. A["Subtle Spatial Motion"] --> B["Temporal Bandpass Filtering"] B --> C["Amplified Temporal Signal"] C --> D["Approximated Larger Translation"] D --> E["Motion-Amplified Video (Preserved Frequency)"] ```

This capability translates into a broad range of scientific, medical, and mechanical use cases: - **Medical diagnostics**: Visualizing the pulsation of subcutaneous arteries and non-contact monitoring of infant respiration for potential Sudden Infant Death Syndrome (SIDS) detection. - **Acoustic analysis**: Tuning the filter to the fundamental frequencies of individual acoustic guitar strings to selectively isolate and amplify the vibration of specific plucked strings. - **Transient scene changes**: Visualizing subtle movements in static environments, such as blending two still photographs taken 15 seconds apart to detect minute shifts. - **Mechanical diagnostics**: Amplifying the internal mechanical vibrations and physical kick of a DSLR camera body caused by the rapid flipping of its internal mirror during a continuous high-speed photo burst.

## Notable quotes > [00:00](https://www.youtube.com/watch?v=ONZcjs1Pjmk&t=0s) Our technique can be run in real time, essentially serving as a microscope for temporal variations.

## Mentioned Eulerian Video Magnification, SIGGRAPH 2012, Miki Rubinstein, Sudden Infant Death Syndrome (SIDS), Laplacian pyramid, Taylor series

## Video description The video accompanying our SIGGRAPH 2012 paper "Eulerian Video Magnification for Revealing Subtle Changes in the World". Read more about it here: http://people.csail.mit.edu/mrub/vidmag/

[Source](https://www.youtube.com/watch?v=ONZcjs1Pjmk)

中文简报

麻省理工学院视频工具可放大微小运动以实时监测生命体征通过对标准视频应用空间分解和时间带通滤波，该技术放大了肉眼不可见的原色和物理变化，从而实现与临床监护仪相媲美的非接触式生命体征提取。欧拉视频放大技术（Eulerian Video Magnification）引入了一种计算技术，通过将空间拉普拉斯金字塔分解与时间带通滤波相结合，揭示标准视频中肉眼不可见的时间变化。该系统作为时间显微镜实时运行，允许用户以交互方式调整频带和放大滑块，以可视化微小的变化。该方法有效放大了微妙的颜色波动，从而在各种肤色下以临床级精度提取脉搏率和婴儿呼吸等生命体征。此外，通过将时间滤波与一阶空间泰勒级数近似相关联，该方法放大了物理运动，揭示了吉他弦和相机机身中未被察觉的机械振动。研究人员展示了欧拉视频放大技术，这是一种结合了空间拉普拉斯分解和时间带通滤波的计算框架，无需显式特征跟踪即可实时放大不可见的视频变化。 (0:00) 通过分离对应于人类心率的频率，该系统放大了微小的面部颜色变化以进行无创脉搏测量，并在新生儿身上展现出与标准医院设备一致的临床精度。 (1:24) 该方法利用一阶泰勒级数近似对时间信号导数进行缩放，以放大物理空间位移，同时严格保持运动原有的时间频率。 (2:08) 该框架支持多样化的跨学科应用，包括非接触式婴儿呼吸监测、单根吉他弦的声学分析以及单反相机反光板回弹的机械诊断。 (2:08)

Reading summary

Eulerian Video Magnification reveals imperceptible temporal variations in standard video by combining multiscale spatial decomposition with temporal bandpass filtering to amplify subtle color changes and microscopic physical motions in real time. Multiscale Filtering Pipeline and Interactive Control (0:00) The Eulerian magnification framework operates without explicit feature tracking by decomposing each video frame into multiple spatial frequency bands using a Laplacian pyramid. Across time, a temporal bandpass filter is applied to the sequence of pixel values to isolate specific oscillation frequencies of interest. The filtered temporal signal is multiplied by a user-specified amplification factor, added back to the original pyramid representation, and collapsed to reconstruct the modified video sequence. Real-Time Tradeoffs and Diagnostic Physiology (1:24) Operating interactively as what the narrator terms a temporal microscope, the software provides real-time controls for high and low frequency cutoffs alongside an amplification slider. While increasing amplification renders minute variations visible, the narrator notes an operational tradeoff where high magnification factors introduce visual artifacts. Clinically, tuning the filter to the human pulse rate magnifies subtle chromatic flushing from blood circulation across facial tissue. This approach is robust across varied skin tones and minor subject movements, successfully matching synchronized hospital monitoring readouts when non-invasively tracking a newborn infant's heart rate. Taylor Approximation and Mechanical Motion Analysis (2:08) Beyond color variations, the technique magnifies physical motion using a 1D wave translation model. For smooth image regions experiencing small displacements, temporal filtering acts as a first-order Taylor series approximation of spatial translation, magnifying spatial movement while strictly preserving the underlying temporal frequency. This mathematical formulation extends to diverse diagnostic tasks, including non-contact infant respiration tracking for SIDS detection, visualizing subcutaneous arterial pulses, isolating the acoustic vibration frequencies of individual plucked guitar strings, detecting shifts between still images taken 15 seconds apart, and inspecting the structural kick of a DSLR camera body during mirror slap. Eulerian Video Magnification functions as an effective non-contact diagnostic framework for both physiological and mechanical analysis, though its practical application remains bounded by the visual artifacts generated when amplification limits are exceeded.

精炼阅读摘要

欧拉视频放大技术（Eulerian Video Magnification）通过结合空间多尺度分解与时间滤波，无需显式跟踪特征即可放大普通视频中肉眼不可见的细微时间变化，并在计算效率上支持实时交互与多领域检测。算法处理管线与实时交互控制 (0:00) 该技术将输入视频的每一帧利用拉普拉斯金字塔分解为不同的空间频带，并在像素时间序列上应用时间带通滤波器，以提取特定频率的目标信号。提取的信号经过用户指定的倍数放大后，重新叠加回原始图像帧中，最后通过金字塔重构生成输出视频。解说者指出，该系统计算效率足以实时运行，其本质上起到了观测微弱时间变化的“时间显微镜”作用。系统提供了实时交互控制界面，用户可自由调节时间带通滤波器的上下截止频率以及放大因子滑块。根据解说者的说明，改变这些参数可以增强场景中不可见的微弱波动；但系统在操作上存在权衡，若过度增大放大倍数，会在重构结果中引入明显的视觉伪影和失真。生理信号可视化与临床非接触监测 (1:24) 该框架的重要应用之一是揭示肉眼不可见的人体生理体征。通过将时间滤波器配置在与人类正常心率匹配的狭窄频段内，算法能够放大面部由于血液循环引起的细微周期性颜色变化，直接显示每次心跳时充血导致的规律性面部潮红。叙述者强调，该方法对不同肤色具有鲁棒性，且能耐受微小的肢体运动，是一种高效的非接触式生理测量手段。在临床演示中，该技术被用于直接从普通消费级视频中提取新生儿的心率。算法提取的心率数据与医院标准监测设备的同步读数进行了对比验证，结果证明其具有高诊断精度。这使得医护人员无需在脆弱的新生儿皮肤上粘贴物理传感器，即可实现精准的生命体征监护。基于一阶近似的运动放大与实际应用 (2:08) 除了颜色变化，该技术还基于一维波动平移模型扩展到了微小空间运动的放大。对于发生微小位移的光滑图像，时间滤波在数学上对应于平移空间波的一阶泰勒级数近似；通过放大该时间导数信号，即可逼近更大的空间位移，从而将物理微动转化为夸张的运动。合成振荡斑块的基准测试表明，在放大空间位移的同时，原始运动的时间振动频率得到了严格保留。这种运动放大能力覆盖了广泛的实际应用场景：在医学上可用于观察皮下动脉搏动及婴儿呼吸监测（用于婴儿猝死综合征SIDS的预警）；在声学分析中，通过将滤波器调至特定琴弦的基频，可独立隔离并放大吉他单根琴弦的振动；在机械诊断中，可揭示单反相机在高速连拍时由于反光镜快速翻转引起的机身机械冲击；此外还能通过融合间隔15秒拍摄的照片来检测静态环境中的瞬态微弱变化。欧拉视频放大技术证明了在无需显式运动跟踪的情况下，通过联合时空滤波即可显现微弱生理信号与微小物理振动；但正如解说者所指出的，过大的放大倍数会带来成像伪影，在实际操作中必须在微弱信号的可见度与图像质量之间做出平衡。

Source quotes

[0:00] Our technique can be run in real time, essentially serving as a microscope for temporal variations.

Original transcript

[0:00] this paper represents orian video [0:02] magnification a method to amplify and [0:04] visualize temporal variations in videos [0:06] that are difficult to see with the naked [0:09] eye our method combines spatial and [0:11] temporal processing to emphasize subtle [0:13] temporal [0:15] changes First We Take a standard video [0:18] sequences input and decompose it into [0:20] different spatial frequency bands using [0:22] a llan [0:24] pyramid we then take the sequence of [0:26] pixel values over time and apply a [0:28] temporal B pass filter to extract the [0:30] frequency band of [0:33] Interest the resulting signal is then [0:35] Amplified and added back to the original [0:37] frames and the pyramid is collapsed to [0:39] generate the output [0:45] video our technique can be run in real [0:48] time essentially serving as a microscope [0:50] for temporal [0:52] variations the user can specify the [0:55] temporal frequency band of Interest by [0:57] adjusting the high and low cut of [0:58] frequencies for the filter [1:02] she can also adjust amplification factor [1:04] of the B P signal using a slider larger [1:07] amplification results in a larger boost [1:09] to the temporal B [1:11] passest changing these power meths can [1:14] make variations in the scene more [1:16] apparent but large amplification can add [1:18] artifacts to the result as discussed in [1:20] the [1:24] paper we show how our technique can be [1:26] used to visualize imperceptible [1:28] variations in videos for for example a [1:30] human [1:32] pulse we automatically select and [1:34] amplify an nrow Bend of temporal [1:36] frequencies around the human heart rate [1:38] and reveal the variation of redness as [1:40] blood flows through the [1:43] face this approach is robust to [1:45] different skin tones and small [1:49] motions it also allows us to [1:52] automatically extract vital signs from [1:54] standard videos for example here we [1:57] extract heart rate measurements of a new [1:58] bone and confirm their accuracy by [2:01] comparing them with the readings from [2:02] the hospital monitor which was recorded [2:04] in sync with the [2:08] video interestingly temporal color [2:11] amplification can also be used to [2:13] amplify spatial motion consider for [2:16] example this 1D illustration showing a [2:19] translated wave at two time [2:21] instance for a smooth image and small [2:24] motion we can filter the signal [2:25] temporarily and get a good approximation [2:28] to the translated wave [2:31] by amplifying the temporal signal we can [2:33] approximate a larger transation [2:35] essentially producing a motion Amplified [2:37] sequence this is analyzed in more detail [2:39] in the [2:41] paper in this synthetic Example The Blob [2:44] has very subtle [2:46] motions we filter this sequence [2:48] temporally using a wide pass band filter [2:50] and boost the temporal signal to [2:52] generate a video where the motion is [2:53] much more [2:54] [Music] [2:55] apparent here several blobs oscillate at [2:58] different temporal frequencies as shown [3:01] by tuning the temporal frequency pass [3:03] band we can selectively control the [3:05] temporal frequencies that are [3:07] Amplified notice that the same temporal [3:09] filter is applied to all pixels and [3:12] while the spatial motion is Amplified [3:14] the temporal frequency is [3:17] maintained motion amplification can be [3:19] useful for medical applications for [3:21] example to visualize artery [3:28] pulsation or for contactless tracking of [3:31] baby breathing and seeds [3:39] detection or we can simply make changes [3:41] in everyday videos more [3:52] apparent in this video we selectively [3:54] amplify motions of individual strings of [3:57] the guitar by tuning the temporal filter [3:59] to the correct frequency of each [4:02] string this input video is a blend [4:04] between two still images taken 15 [4:06] seconds apart the output video reveals [4:09] changes in the scene even within this [4:11] short time [4:13] period finally in this video we manage [4:15] to amplify very subtle motions of a DSLR [4:18] camera caused by its flipping mirror [4:20] during a photo burst