RDI Turns Standard Cameras Into Sensor Arrays to Prevent Plant Failure

Steve Mould

Amplifying sub-pixel brightness shifts transforms video cameras into millions of contactless sensors, exposing imperceptible machine fatigue and human vital signs before failure occurs.

AI-assisted summary of Reveal Invisible Motion With This Clever Video Trick — original source. Check the original for full context.

Source published: 2020-06-04

Key ideas

Digital video cameras equipped with global shutter CMOS sensors can turn millions of individual pixels into a comprehensive vibration sensor array, eliminating blind spots inherent in traditional physical accelerometers. (0:00)
Industrial motion amplification algorithms track sub-pixel brightness changes across neighboring pixels to reveal mechanical oscillations and phase misalignments, requiring sampling rates at least twice the target frequency. (2:01)
MIT CSAIL's video magnification techniques isolate temporal frequency bands between 1.0 Hz and 1.5 Hz to detect human heartbeats through rhythmic skin flushing and micro-scale head swaying. (9:13)
Beyond non-invasive vital sign and infant breathing monitoring, micro-vibrations captured on high-speed video can be processed as 'visual microphones' to reconstruct intelligible acoustic speech from physical surfaces. (10:30)

Detailed notes

## Background Published on YouTube by science communicator Steve Mould on June 4, 2020, this video explores the concepts of motion amplification and video magnification. According to the show description, these technologies detect and exaggerate subtle visual changes invisible to the naked eye, offering tremendous diagnostic utility across industry and medicine.

## TL;DR Steve Mould explains how motion amplification and video magnification algorithms turn standard digital camera sensors into vast measurement arrays capable of visualizing sub-pixel vibrations and minute color shifts. Featuring insights from RDI Technologies' Jeff Hay, the video demonstrates how industrial facilities use camera monitoring to pinpoint misaligned machinery and prevent catastrophic structural failures. Mould applies these techniques to his own body to isolate arterial pulses, facial flushing, and pulse-induced head bobbing via temporal frequency filtering. He further explores practical applications, ranging from non-invasive infant breathing monitors to optical sound extraction from microscopic surface vibrations.

## Atmosphere & style The presentation is intellectually playful, lucid, and wonder-filled, pairing approachable science communication with sophisticated computer vision concepts. Mould delivers an engaging, conversational narrative peppered with self-deprecating humor, relatable analogies about CAPTCHAs, and empathetic parenting observations. The viewing experience feels delightfully revelatory as seemingly static machinery and human bodies suddenly spring to life with dramatic, hidden rhythmic motions.

## Bulletin - [00:00](https://www.youtube.com/watch?v=rEoc0YoALt0&t=0s) Digital video cameras equipped with global shutter CMOS sensors can turn millions of individual pixels into a comprehensive vibration sensor array, eliminating blind spots inherent in traditional physical accelerometers. - [02:01](https://www.youtube.com/watch?v=rEoc0YoALt0&t=121s) Industrial motion amplification algorithms track sub-pixel brightness changes across neighboring pixels to reveal mechanical oscillations and phase misalignments, requiring sampling rates at least twice the target frequency. - [09:13](https://www.youtube.com/watch?v=rEoc0YoALt0&t=553s) MIT CSAIL's video magnification techniques isolate temporal frequency bands between 1.0 Hz and 1.5 Hz to detect human heartbeats through rhythmic skin flushing and micro-scale head swaying. - [10:30](https://www.youtube.com/watch?v=rEoc0YoALt0&t=630s) Beyond non-invasive vital sign and infant breathing monitoring, micro-vibrations captured on high-speed video can be processed as 'visual microphones' to reconstruct intelligible acoustic speech from physical surfaces.

## Key takeaways - [00:00](https://www.youtube.com/watch?v=rEoc0YoALt0&t=0s) Undetected micro-vibrations in industrial machinery can initiate a dangerous snowball effect, loosening bolts and creating metal fatigue that leads to catastrophic structural failure. - [00:00](https://www.youtube.com/watch?v=rEoc0YoALt0&t=0s) Physical contact sensors like accelerometers leave critical blind spots because technicians cannot monitor components they did not think to instrument beforehand. - [00:00](https://www.youtube.com/watch?v=rEoc0YoALt0&t=0s) A standard camera sensor functions as an array of millions of discrete light sensors, allowing wide-angle inspection of coupled equipment across an entire facility. - [02:01](https://www.youtube.com/watch?v=rEoc0YoALt0&t=121s) Motion amplification resolves sub-pixel movements where an edge does not cross a full pixel, instead measuring subtle pixel darkening and brightening while filtering out sensor noise. - [02:01](https://www.youtube.com/watch?v=rEoc0YoALt0&t=121s) Full-field video analysis readily identifies phase misalignments, revealing whether coupled machines like motors and pumps are moving in unison or deflecting in destructive opposition. - [02:01](https://www.youtube.com/watch?v=rEoc0YoALt0&t=121s) Accurately measuring rapid mechanical vibrations requires capturing video data at a sampling rate at least twice the target frequency. - [04:35](https://www.youtube.com/watch?v=rEoc0YoALt0&t=275s) Translating numerical vibration data into animated spatial motion leverages human visual cortex pattern recognition more effectively than analyzing raw frequency spectra. - [04:35](https://www.youtube.com/watch?v=rEoc0YoALt0&t=275s) Google historically monetized human visual superiority over AI by using CAPTCHAs to transcribe Google Books, identify Street View house numbers, and train autonomous vehicle classifiers. - [07:36](https://www.youtube.com/watch?v=rEoc0YoALt0&t=456s) Exaggerated visual motion provides intuitive, compelling evidence that helps engineers persuade corporate decision-makers to approve necessary equipment shutdowns and maintenance. - [07:36](https://www.youtube.com/watch?v=rEoc0YoALt0&t=456s) Academic video magnification pioneered at MIT CSAIL can amplify both minute spatial displacements and subtle temporal color shifts. - [07:36](https://www.youtube.com/watch?v=rEoc0YoALt0&t=456s) Applying color magnification to human skin visualizes blood volume pulsing rhythmically with every heartbeat, while motion amplification tracks arterial pulses in the wrist. - [09:13](https://www.youtube.com/watch?v=rEoc0YoALt0&t=553s) Bandpass temporal filtering isolates narrow frequency ranges—such as 1.0 to 1.5 Hz for resting heart rate—preventing macro body motion from obscuring micro-signals. - [09:13](https://www.youtube.com/watch?v=rEoc0YoALt0&t=553s) The same mathematical frequency filtering used to isolate human vital signs allows industrial engineers to distinguish separate vibration sources across different shafts and gears within a single frame. - [10:30](https://www.youtube.com/watch?v=rEoc0YoALt0&t=630s) Computer vision motion magnification enables non-contact medical monitoring, such as verifying infant respiration by visually exaggerating chest wall excursions. - [10:30](https://www.youtube.com/watch?v=rEoc0YoALt0&t=630s) High-framerate video magnification can measure sound-wave-induced surface vibrations, turning everyday physical objects into visual microphones capable of reconstructing audio.

## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=rEoc0YoALt0&t=0s) The Factory Problem and Turning Cameras into Sensor Arrays

Imagine managing a manufacturing plant full of spinning equipment and up-and-down machinery. In such a complex environment, vibrations are inevitable, raising the constant fear that the entire factory could shake itself apart. A minor vibration can cause a bolt to loosen, generating further oscillations that trigger metal fatigue elsewhere, initiating a catastrophic snowball effect. To prevent structural failure, engineers must catch imperceptible vibrations early. Traditionally, this meant attaching physical contact sensors, such as an accelerometer mounted to a vibrating pipe, to monitor motion data on a plotted graph. However, physical sensors present a significant blind spot: you cannot monitor components you never thought to instrument in the first place.

Instead of deploying a handful of physical accelerometers, you can effectively blanket an entire facility with millions of sensors simultaneously using a camera. A standard digital camera chip—such as a CMOS sensor—is fundamentally an array of millions of individual light-detecting sensors (technically three per pixel for color). While a smartphone camera demonstrates the concept, practical industrial monitoring typically requires specialized hardware equipped with a global shutter to record wide shots of plant machinery without rolling shutter artifacts.

```mermaid flowchart TD accTitle: Vibration Cascade vs Camera Monitoring accDescr: Visualizing the snowball effect of undetected mechanical vibrations alongside the whole-facility camera monitoring approach.

subgraph Failure["Vibration Escalation"] A["Unchecked Small Vibration"] --> B["Loose Bolt"] B --> C["Increased Vibration"] C --> D["Metal Fatigue"] D --> E["Catastrophic Structural Failure"] end

subgraph Solution["Optical Monitoring"] F["Digital Video Camera<br/>(Global Shutter CMOS)"] --> G["Millions of Pixel Sensors"] G --> H["Motion Amplification Algorithm"] H --> I["Early Imperceptible Defect Detection"] end ```

[02:01](https://www.youtube.com/watch?v=rEoc0YoALt0&t=121s) Sub-Pixel Motion Amplification and Jeff Hay's Insights

Even when machinery vibrations are far too minute for the human eye to resolve, motion amplification software can expose them. Placing raw video beside processed footage reveals that a seemingly motionless machine is actually oscillating vigorously. The technique is capable of amplifying sub-pixel movements where an edge never cleanly crosses from one pixel to the next; rather, a line of pixels subtly darkens while neighboring pixels brighten. The software detects these minuscule brightness shifts, separates the underlying motion from random image sensor noise (individual pixel brightness fluctuations), and exaggerates the movement visually.

This industrial application was pioneered by Jeff Hay of RDI Technologies. Jeff Hay’s background bridges astronomy and photojournalism: “I started in astronomy, thought I was gonna be an astronomer, but I think I had a conversation with my advisor that was about being gainfully employed, you know remarkably the problem sets kind of the same, you know what I mean?” In both fields, a camera is used to characterize distant objects. Jeff Hay explains that a modest one-megapixel camera provides a million discrete sensors across an object. This full-field view is crucial for diagnosing coupled machinery, like a motor driving a pump via a shaft coupling. If both components move in unison, the machine is healthy, but if the pump moves upward while the motor deflects downward, opposing forces indicate dangerous misalignment. When Steve Mould asks about capturing rapid movements—such as a 15 Hz vibration on a 50 fps camera—Jeff Hay notes the fundamental sampling requirement: “Yeah, so usually the rule of thumb is you have to sample, you have to take data at a rate twice the frequency you're interested in.”

[04:35](https://www.youtube.com/watch?v=rEoc0YoALt0&t=275s) Data Visualization and the CAPTCHA Tangent

While motion amplification software can export raw numerical vibration spectra for any chosen group of pixels, converting raw data into a visual animation leverages the human visual cortex, which processes complex spatial patterns far more naturally than raw numbers. Steve Mould points out that humans remain significantly better than computers at specific visual pattern recognition tasks, which is precisely why human brains are used to train artificial intelligence models through CAPTCHAs.

This dynamic evolved over several generations of Google CAPTCHA tests: - **Distorted Dual Words:** Originally, reCAPTCHA presented two warped words. One was a known test word to verify human input, while the other was an unrecognized word from the Google Books digitization archive. By crowdsourcing these difficult words, Google rapidly digitized massive book collections while training optical character recognition models. - **Street Address Numbers:** When distorted text disappeared, users were prompted to transcribe blurry street numbers, which turned out to be cropped house numbers from Google Street View used to refine Google Maps address databases. - **Object Classification Grids:** Modern CAPTCHAs require selecting tiles containing buses, bicycles, or road signs. These challenges train the machine-vision perception algorithms powering autonomous vehicles for Google’s parent company (Alphabet). Steve Mould jokes that these are resolved live for cars currently driving on the road before quickly admitting that part is made up.

The core lesson remains: transforming numerical data into an intuitive visual format allows human cognition to identify anomalies without friction.

[07:36](https://www.youtube.com/watch?v=rEoc0YoALt0&t=456s) Persuading Bosses and MIT's Video Magnification Research

Visualizing invisible motion also carries massive practical value inside corporate hierarchy. Pointing at a complex data graph rarely convinces skeptical managers to authorize expensive repair downtime. Jeff Hay underscores the persuasive power of visual evidence: “So if you go to somebody and you show them a plot on a graph and you say, man that machine is gonna come apart, you know, that's a hard, you know, but if you can show them that it's lifting up off the ground or it's cracked, or you know, that's kind of an element to it as well.”

Beyond proprietary industrial systems like RDI Technologies, academic researchers at MIT CSAIL pioneered an equivalent open approach known as video magnification. This framework can amplify two distinct phenomena in standard video: 1. **Motion Amplification:** Exaggerating minute structural or physiological displacements. 2. **Color Magnification:** Amplifying imperceptible shifts in chromatic values over time.

Steve Mould demonstrates this on his own body. Applying motion amplification to his wrist clearly displays his arterial pulse throbbing. Switching to color magnification on his face reveals his skin flushing rhythmically red with every heartbeat as blood volume fluctuates beneath the skin, an effect momentarily interrupted whenever he blinks.

[09:13](https://www.youtube.com/watch?v=rEoc0YoALt0&t=553s) Detecting Heartbeats and Frequency Filtering

The human pulse can even be detected without focusing on skin or veins. While attempting to stand perfectly still, Steve Mould shows that every systolic pulse pumping blood up through the carotid arteries in his neck imparts a mechanical force that causes his head to wobble. When amplified, this subtle head bobbing becomes an unmistakable rhythmic sway.

Because a person's body constantly sways and twitches with unrelated baseline movements, the video magnification algorithm must isolate relevant signals. It achieves this by filtering specific temporal frequencies: - To isolate a resting heart rate, the processing band is constrained to motions oscillating between 1.0 Hz and 1.5 Hz (60 to 90 beats per minute). - Frequencies outside this target band are ignored, preventing large macro body shifts from overwhelming the subtle cardiovascular pulse. - This frequency-isolation approach is so robust that Mould demonstrates it still measures pulse-induced head movement even while he wears a face mask. - The identical mathematical filtering is utilized in industrial factories to separate distinct mechanical vibration frequencies caused by different spinning shafts, gears, or pumps within the same video frame.

[10:07](https://www.youtube.com/watch?v=rEoc0YoALt0&t=607s) Under the Hood: Mathematical Complexity

While the results of motion and color magnification look straightforward on screen, the underlying computer vision mathematics are formidable. Steve Mould notes that grasping the software's internal operation requires advanced study, highlighting the daunting jargon found directly in the MIT academic papers, which describe the algorithms using concepts such as the “phase variation of a complex steerable pyramid.”

[10:30](https://www.youtube.com/watch?v=rEoc0YoALt0&t=630s) Medical Applications and Audio Extraction

Steve Mould wraps up the technical analysis of video magnification by setting aside deeper mathematical arcana and turning to the technology's real-world diagnostic possibilities. He notes that viewers are likely already considering medical use cases, particularly non-invasive methods for measuring a patient's pulse without needing any direct physical contact. He illustrates this potential with a familiar, anxiety-inducing parenting scenario: anyone who has cared for very young infants knows the panic of watching them sleep and suddenly worrying whether they have stopped breathing. The standard parental reaction is to give the baby a gentle poke until they stir, bringing relief alongside frustration when the now-awake infant starts crying. As Mould observes: “Imagine if instead you could train a camera on them and amplify the up and down motion of their chest and know for sure that they were breathing.”

Beyond human vitals, subtle vibrations in video footage can yield surprising acoustic data. Mould points viewers to a related video by Derek Muller on Veritasium demonstrating how researchers can extract intelligible audio from high-speed video simply by measuring the microscopic vibrations that sound waves produce when impinging on physical surfaces in a room. Before moving on, Mould credits Jeff Hay from RDI Technologies for providing the industrial motion amplification footage used throughout the demonstration, encouraging viewers to explore the company links in the video description.

```mermaid flowchart TD accTitle: Medical and Acoustic Video Magnification Applications accDescr: Visualizing how video magnification is applied to non-invasive infant monitoring and visual microphone audio reconstruction.

subgraph Medical["Medical & Monitoring"] direction TB A["Target (Sleeping Infant / Patient)"] -->|"Camera lens captures subtle motion"| B["Video Magnification Algorithm"] B -->|"Isolate chest / skin displacement"| C["Amplify chest rise & fall / pulse"] C --> D["Non-invasive confirmation of breathing/vitals"] end

subgraph Acoustic["Visual Microphone (Veritasium)"] direction TB E["Sound wave energy"] -->|"Impinges on physical objects"| F["Microscopic surface vibrations"] F --> G["Video extraction algorithm"] G --> H["Reconstructed audio signal"] end ```

- **Non-contact vital signs**: Video magnification algorithms can detect periodic micro-color shifts from blood flow or amplify chest wall excursions to measure heart rate and respiration passively. - **Acoustic reconstruction**: Sound waves generate microscopic mechanical displacements on everyday objects, which high-framerate video processing can convert back into audible speech.

[11:34](https://www.youtube.com/watch?v=rEoc0YoALt0&t=694s) Self-Taught Habits and Skillshare Partnership

Transitioning with a segue into his personal hobbies, Mould reflects on learning to snowboard. He describes teaching himself the sport with only informal tips from friends on the slopes who already knew what they were doing. While that unstructured approach got him down the mountain, it also left him saddled with persistent bad habits that now block his progress. As he explains: “The issue with that approach, which I've learned over the years is that I have a few habits that are now really ingrained, and if I want to get any better at this point, I really need to unlearn those bad habits, which is a total pain in the ass.” While muddling through used to be his default life philosophy, he now views skipping structured guidance as a false economy. Investing time upfront in a well-taught course establishes a foundation of proper technique, ensuring subsequent practice builds on solid ground rather than reinforcing mistakes.

This perspective forms the basis for his recommendation of the video's sponsor, Skillshare. Mould highlights their library of instructional courses across practical and creative disciplines—including web design, animation, photography, illustration, and leadership. He mentions personally taking their "Styling Your Space" interior design class to help plan his kitchen remodel. Noting that many viewers may find themselves with extra downtime to kickstart an idea or pursue structured self-improvement, Mould shares a promotional discount:

- **Subscription pricing**: Full access to the platform's catalog normally runs $10 per month. - **Sponsor offer**: The first 500 viewers visiting `skl.sh/stevemould8` receive two free months of Skillshare Premium with no strings attached.

## Notable quotes > [02:01](https://www.youtube.com/watch?v=rEoc0YoALt0&t=121s) usually the rule of thumb is you have to sample, you have to take data at a rate twice the frequency you're interested in. — Jeff Hay

## Mentioned Steve Mould, Jeff Hay, RDI Technologies, MIT CSAIL, Derek Muller, Veritasium, Google, Alphabet, Google Books, Google Street View, Google Maps, reCAPTCHA, Skillshare

## Video description The first 500 people to use this link will get 2 free months of Skillshare Premium Membership: https://skl.sh/stevemould8

Motion Amplification and Video Magnification are techniques that find subtle changes in a video that are invisible to the naked eye and amplify them so they become visible. They have huge diagnostic potential in industry and medicine.

Here's RDI Technologies: https://rditechnologies.com/ Here's the video magnification research: http://people.csail.mit.edu/mrub/vidmag/ Here's where you can upload your own videos: https://lambda.qrilab.com/site/geko/ Here's Derek's video from Veritasium: https://youtu.be/eUzB0L0mSCI

You can buy my books here: https://stevemould.com/books

You can support me on Patreon here: https://www.patreon.com/stevemould

just like these amazing people:

Glenn Watson Peter Turner Joël van der Loo Matthew Cocke Mark Brouwer Deneb

Twitter: http://twitter.com/moulds Instagram: https://www.instagram.com/stevemouldscience/ Facebook: https://www.facebook.com/stevemouldscience/ Buy nerdy maths things: http://mathsgear.co.uk

[Source](https://www.youtube.com/watch?v=rEoc0YoALt0)

中文简报

RDI将标准摄像机转化为传感器阵列以防范工厂故障放大亚像素级亮度变化可将摄像机转化为数百万个非接触式传感器，在故障发生前揭示难以察觉的机器疲劳与人体生命体征。 Steve Mould阐释了运动放大和视频放大算法如何将标准数码摄像机传感器转化为庞大的测量阵列，从而实现亚像素级振动与微小颜色变化的可视化。视频结合了来自RDI Technologies的Jeff Hay的见解，展示了工业设施如何利用摄像机监测来精准定位未对齐的机械设备，并防止灾难性的结构故障。Mould还将这些技术应用于自身，通过时间频率滤波分离出动脉脉搏、面部潮红以及由脉搏引起的头部微晃。他进一步探讨了更多实际应用，涵盖从非侵入式婴儿呼吸监护仪到利用微观表面振动进行光学声音提取等领域。配备全局快门CMOS传感器的数码摄像机可将数百万个独立像素转化为全方位的振动传感器阵列，消除传统物理加速度计固有的监测盲区。 (0:00) 工业运动放大算法通过追踪相邻像素间的亚像素级亮度变化来揭示机械振荡和相位不对齐，这要求采样率至少达到目标频率的两倍。 (2:01) 麻省理工学院计算机科学与人工智能实验室（MIT CSAIL）的视频放大技术能够隔离1.0 Hz至1.5 Hz之间的时间频段，通过规律性的皮肤潮红和微观尺度的头部晃动来检测人类心跳。 (9:13) 除了非侵入式生命体征和婴儿呼吸监测外，高速视频捕捉到的微振动还可以作为“视觉麦克风”进行处理，从而从物体物理表面重建出清晰可辨的语音声音。 (10:30)

Detailed notes — zh

## Background 本内容由科普博主 Steve Mould 于 2020 年 6 月 4 日发布在 YouTube 频道。视频简介指出，动作放大（Motion Amplification）与视频放大（Video Magnification）技术能够捕捉视频中肉眼不可见的细微变化并将其放大呈现，在工业诊断和现代医学领域具备巨大的应用潜力。 ## TL;DR 视频探讨了如何将普通相机的数百万像素感光元件转化为无死角的工业级传感器网络。面对现代化工厂中微小振动可能引发结构毁灭的隐患，动作放大算法能够提取亚像素级别的明暗变化，并结合采样定理与带通滤波，将抽象的物理波形转化为直观的动态影像。此外，麻省理工学院（MIT）的视频放大技术还能捕捉人体脉搏、面部血流及呼吸微动，展现了非侵入式生理监测与声波微振动复原等跨领域的广阔前景。 ## Atmosphere & style 整部视频洋溢着 Steve Mould 标志性的轻松幽默与严谨求知欲，将复杂的图像算法和工程隐患讲得通俗易懂且极具观赏性。通过静止机械瞬间剧烈颤动、面部随心跳规律闪烁等惊艳的视觉对比，节目营造出一种“洞见肉眼所未见”的强烈震撼感。穿插关于验证码训练自动驾驶的幽默调侃与滑雪自学的亲身经历，使整个科普过程既硬核扎实，又显得亲切生动、引人入胜。 ## Bulletin - [02:01](https://www.youtube.com/watch?v=rEoc0YoALt0&t=121s) 动作放大（Motion Amplification）与视频放大（Video Magnification）技术能够通过算法提取并放大肉眼不可见的亚像素级微小位移与颜色变化，将相机感光芯片转化为数百万个协同工作的无死角虚拟传感器。 - [03:39](https://www.youtube.com/watch?v=rEoc0YoALt0&t=219s) 工业诊断中利用带通滤波与奈奎斯特采样准则，可精准剥离特定频率的异常机械共振，并通过可视化反向异动直观识别电机与水泵等设备的对中不良与疲劳隐患。 - [08:11](https://www.youtube.com/watch?v=rEoc0YoALt0&t=491s) MIT CSAIL 的视频放大算法可通过捕捉桡动脉跳动和面部微血管充血，实现非接触式脉搏与婴儿呼吸监测，甚至可从物体表面的声波微振动中逆向还原现场音频。 - [05:11](https://www.youtube.com/watch?v=rEoc0YoALt0&t=311s) 人脑在模式匹配上的超高效率不仅使动态放大图像在工程汇报中远胜抽象波形图，还曾被 Google 长期通过 CAPTCHA 验证码形式反哺图书数字化及自动驾驶视觉模型的训练。 ## Key takeaways - [00:00](https://www.youtube.com/watch?v=rEoc0YoALt0&t=0s) 现代工业制造中微小的未察觉振动会引发螺栓松动与金属疲劳，形成滚雪球式的灾难性结构损坏，及早预警至关重要。 - [00:40](https://www.youtube.com/watch?v=rEoc0YoALt0&t=40s) 传统加速度计等物理传感器只能布设在预先怀疑的特定点位，存在严重的漏检盲区；而相机感光芯片（CMOS）的每个像素都能充当独立的测量传感器。 - [02:01](https://www.youtube.com/watch?v=rEoc0YoALt0&t=121s) 动作放大算法能够通过像素亮度的微妙渐变，稳定捕捉并放大物理跨度小于单个像素的“亚像素位移”。 - [03:39](https://www.youtube.com/watch?v=rEoc0YoALt0&t=219s) 基于奈奎斯特采样准则，视频拍摄帧率必须达到目标振动频率的两倍以上，以防止高频振动出现混叠失真。 - [03:39](https://www.youtube.com/watch?v=rEoc0YoALt0&t=219s) 在旋转机械诊断中，联轴器两侧部件的反相异动（一上一下）是判断机械对中不良或结构开裂的关键视觉特征。 - [04:30](https://www.youtube.com/watch?v=rEoc0YoALt0&t=270s) 将抽象的波形数据转化为直观夸张的整体动态影像，能利用人类视觉皮层的直觉优势，极大降低跨部门工程决策的说服成本。 - [05:11](https://www.youtube.com/watch?v=rEoc0YoALt0&t=311s) Google 曾长期利用 CAPTCHA 验证码调动人类的视觉识别能力，先后为 Google 图书 OCR、街景门牌识别以及 Waymo 自动驾驶道路语义分割完成数据标注反哺。 - [08:11](https://www.youtube.com/watch?v=rEoc0YoALt0&t=491s) 麻省理工学院（MIT CSAIL）公开的视频放大算法不仅能放大物理位移，还能放大肉眼不可见的面部微血管充血导致的周期性肤色变化。 - [09:30](https://www.youtube.com/watch?v=rEoc0YoALt0&t=570s) 利用带通滤波器设定特定频段（如静息脉搏对应的 1 Hz 至 1.5 Hz），可以有效滤除肢体晃动与背景噪声，单独提取生理指标或特定机械组件的振动。 - [10:30](https://www.youtube.com/watch?v=rEoc0YoALt0&t=630s) 在医疗育儿场景中，动作放大技术可实现非接触式的远程脉搏测量，以及在不打扰婴儿睡眠的前提下清晰监测胸部起伏呼吸。 - [11:10](https://www.youtube.com/watch?v=rEoc0YoALt0&t=670s) 该技术还可扩展至声音侦测领域，通过高速观测受声波撞击物体的亚像素微振动，实现视频画面的反向声音重建。 ## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=rEoc0YoALt0&t=0s) 工厂振动隐患与以相机为“数百万传感器”的构想在现代化制造工厂中，机械设备的协同运转极其复杂：一边是高速旋转的部件，另一边是上下往复运动的机械结构。这种高度动态的环境不可避免会产生大量振动。工厂管理者最合理的担忧，就是整座工厂有朝一日可能会因振动而“把自己震散架”。微小的振动可能先是震松了一颗螺栓，螺栓松动引发更大的剧烈振动，进而导致临近金属结构发生疲劳断裂，最终演变成破坏性的滚雪球效应（snowball effect）。为了防患于未然，人们必须在振动变得严重之前、甚至在肉眼完全无法察觉的微观阶段将其捕捉。传统工程手段通常是在重点疑虑区域安装物理传感器，例如在一条上下晃动的管道上加装加速度计（accelerometer）。传感器采集数据并绘制出波形图表，工程师只要看到位移数值在安全容差范围内即可放心监控。然而，这种点状监测存在致命盲区：万一某处关键部件因为事先未被列入怀疑名单而漏装了传感器，隐患就会被彻底遗漏。如果有一种方法，能将数以百万计的传感器瞬间抛洒在整个工厂的各个角落、实现无死角全覆盖，监测范式将被颠覆。事实证明，这种海量传感器阵列其实人人随身携带——那就是智能手机中的相机。相机感光元件（CMOS 芯片）本质上就是由数百万个微小传感器组成的阵列，每个像素对应一个（严格来说每个像素对应三个感光单元）。尽管在严苛的工业现场，人们倾向于使用配备全局快门（global shutter）的专业工业级相机拍摄大范围全景视频以覆盖所有设备，但核心难题依旧存在：如果所寻找的机械振动极其微小、微小到人类视觉根本不可见，相机镜头又如何能看见它们？ ```mermaid flowchart TD accTitle: 工厂振动恶性循环与监测方案对比 accDescr: 展现小振动演变为灾难的过程，以及物理传感器与相机阵列的监测对比 subgraph ViciousCycle ["振动恶性滚雪球效应"] A["微小肉眼不可见振动"] --> B["螺栓松动"] B --> C["加剧设备震荡"] C --> D["金属疲劳与结构裂纹"] D --> E["整机/厂房结构性损毁"] end subgraph Monitoring ["两种监测范式对比"] M1["传统方式: 加速度计"] --> M2["局限: 只能布设在预选点，易漏检"] M3["视觉方案: CMOS 感光芯片"] --> M4["优势: 每个像素即传感器，全场覆盖"] end ``` --- ### [02:01](https://www.youtube.com/watch?v=rEoc0YoALt0&t=121s) 动作放大技术与亚像素位移的提取这正是动作放大技术（Motion Amplification）大显身手的舞台。在对比演示中，原始视频与动作放大后的画面并排呈现：原始画面中的重型机械看似纹丝不动、宛如静止照片，但在另一侧经过算法放大处理后，剧烈的往复摆动和抖动肉眼清晰可见。这些隐藏在像素深处的微妙运动信息被完整提取并按比例放大了出来。即使原始视频中的物理位移小于单个像素的跨度（即所谓的“亚像素位移”，sub-pixel movements），动作放大算法依然能够精准捕捉。现实拍摄中，微小位移并不一定表现为某条边缘横跨数个像素的剧烈扫掠，而更多表现为一组像素在逐渐变暗，而与之紧邻的邻近像素则在同步变亮。算法能够敏锐地将这种极其细微的明暗渐变判定为几何边缘的空间位移，并施加放大。在数字成像固有噪声（即每个像素亮度随机波动）的背景干扰下，能够稳定提取出这类极低信噪比的亚像素运动信号，工程表现极其惊人。工业级动作放大系统的领军研发者、RDI Technologies 的 Jeff Hay 解释了自己的跨界历程：他最初主修天文学，后因导师提醒要“谋得一份体面的工作（gainfully employed）”而转向应用领域。巧合的是，通过远程相机观测远距离天体物理状态的思维，与用工业相机远距诊断机械故障的问题内核高度重合；加之他在大学担任过校报图片编辑、具备深厚的摄影功底，各项技能交汇最终促成了 RDI 动作放大系统的诞生。 --- ### [03:39](https://www.youtube.com/watch?v=rEoc0YoALt0&t=219s) 工业设备诊断逻辑、采样定理与人脑视觉皮层的降维优势在工业检测的实际场景中，相机的百万像素阵列如同一张覆盖在设备上的传感网。Jeff Hay 以电机（motor）与水泵（pump）的连接为例进行了说明：两台设备通过联轴器（coupling）将驱动轴连在一起。在运行过程中，如果电机和水泵同频同相晃动，通常属于正常结构共振；但如果一侧向上抬起而另一侧同时向下压，就形成了错位与反对称运动。正如 Jeff Hay 所言：“很多时候，事物同步运动意味着状况良好没毛病，而一旦它们彼此对抗反向运动，问题往往就出在这里。”（“you know, so lots of times that things are moving together, that's some indication that that's good, that's all right, and sometimes when they're opposing each other, that's where the problems come in.”）针对采样率问题，视频帧率必须满足经典采样定律。例如使用 50 fps 的相机去捕获每秒 15 次的高频振动时必须保持警惕。正如 Jeff Hay 所总结的工程经验准则：“通常的经验法则是，你采样的速率必须至少达到你感兴趣频率的两倍。”（“Yeah, so usually the rule of thumb is you have to sample, you have to take data at a rate twice the frequency you're interested in.”）尽管该系统能够调取任意像素点并绘制出详尽的物理位移数据图表，但动作放大最核心的价值，在于将原本抽象枯燥的波形数据，转化为人类视觉皮层能够瞬间理解的整体动态图像。当向管理层汇报设备维修预算时，静态图表往往很难引起非专业决策者的紧迫感；但若直接播放一段支架螺栓脱落或机体从地面掀起的夸张动态画面，说服效果立竿见影。正如 Jeff Hay 生动描述的那样：“如果你去找某个人，拿出一张图表上的曲线对他说‘老兄，那台机器马上就要散架了’，这很难让他们接受；但要是你能亲眼给他们看机器正在脱离地面抬起，或者已经开裂了，这完全是另一种层面的杀伤力。”（“So if you go to somebody and you show them a plot on a graph and you say, man that machine is gonna come apart, you know, that's a hard, you know, but if you show them that it's lifting up off the ground or it's cracked, or you know, that's kind of an element to it as well.”） - **双向数据输出**：系统既支持提取特定像素区域输出定量波形图，也支持渲染全貌动态放大影像。 - **奈奎斯特采样准则（Nyquist rate）**：相机拍摄帧率必须大于目标振动频率的 2 倍以上，才能避免运动混叠失真。 - **故障特征模式识别**：联轴器两侧部件的反相异动（一上一下）是判断机械对中不良（misalignment）的关键视觉表征。 --- ### [05:11](https://www.youtube.com/watch?v=rEoc0YoALt0&t=311s) 偏题闲聊：人机视觉认知分工与 CAPTCHA 的数据反哺演变人类大脑在处理视觉模式匹配方面，至今仍展现出超越计算机算法的直觉与稳健性。为了训练计算机视觉，科技界长期借助人类大脑充当免费的数据标注劳动力。最典型的代表就是登录网站时用来区分人类与机器人的 CAPTCHA（验证码）。早期的文字验证码通常包含两个扭曲变形的单词。许多人未曾留意的是，其中其实只有一个词是用于验证真人身份的已知答案，而另一个难以辨认的词，则是 Google 在进行海量图书数字化工程（Google Books）时 OCR 引擎无法识别的死角扫描件。Google 将机器无法辨识的生僻词外包给数以亿计的网民辨认，既顺带完成了图书扫描校验，又反哺训练了文字识别模型，使得图书数字化进度突飞猛进。随着图书识别任务完成，文字验证码逐渐演变： 1. **门牌号阶段**：用户开始频繁看到类似住宅大门处模糊拍摄的数字照片，这实际上是在帮助 Google 地图街景团队自动关联和纠正现实门牌地址； 2. **路标与公交车阶段**：如今主流的验证码变成了九宫格选图，要求点选出所有的“交通标志”或“公交车”。其幕后推手是 Google 母公司旗下的自动驾驶汽车项目（Waymo），需要利用海量的人脑常识标注样本，训练自动驾驶车载视觉系统对现实道路障碍的语义分割能力。 Steve Mould 在此开玩笑称，这类路标识别验证码甚至可能是“实时”分发给用户的——路面上某辆自动驾驶汽车此时此刻正开到路口急需确认前方是不是路标，如果用户点击慢了可能就会出事故。在短暂故作悬念后，他笑着坦白这纯属玩笑。但他强调的核心事实极其严肃：人类大脑的视觉处理机制具有不可思议的效率；将复杂抽象的数据转化为肉眼可读的视觉运动，能让人类在不费吹灰之力的情况下洞察深奥规律。 --- ### [08:11](https://www.youtube.com/watch?v=rEoc0YoALt0&t=491s) MIT 视频放大术（Video Magnification）：心率检测与带通滤波工业动作放大技术的核心商业算法受到严格保密，但麻省理工学院（MIT）计算机科学与人工智能实验室（CSAIL）公开的一套平行技术——视频放大术（Video Magnification），让大众也能探索这一前沿算法。这套算法不仅能放大肉眼看不见的物理微小位移，还能放大图像中微弱的颜色波动。在个人实验中，Steve Mould 展示了极其惊人的生理特征提取效果： 1. **手腕脉搏位移放大**：原始手腕视频几乎看不到搏动，但在算法施加动作放大后，桡动脉处的皮肤呈现出大幅度节奏鲜明的规律起伏。 2. **面部微观颜色变化放大**：每一次心脏跳动将新鲜血液泵入面部微血管网，都会导致面部肤色发生人类肉眼无法直接感知的轻微明暗与色度变化；经过颜色放大后，整张脸宛如红绿相间的呼吸灯一样，伴随心跳频率规律性地泛红闪烁（直到中途因眨眼动作破坏算法基准）。 3. **全身微动与头部晃动监测**：即使人尝试绝对静止站立，每一次血液冲过颈总动脉，微弱的力学反作用力都会导致头部发生极微小的摇晃；经过动作放大，这种微颤一览无余，甚至在佩戴口罩遮挡面部时依然完全有效。这一技术的关键在于结合了频率滤波机制（带通滤波器）。人体在镜头前不可避免存在呼吸、肢体微晃等多种大幅度噪动，若无差别放大，画面将彻底混乱。而由于已知成人的静息脉搏频率大致在 1 Hz 至 1.5 Hz 之间（对应每分钟 60 到 90 次心跳），算法可以设定只过滤并放大这一特定频段范围内的振动。同样的原理在工厂振动检测中完全通用，工程师可设定特定的转速频率阈值，单独把某根轴承或管道的异常共振剥离放大出来。至于其底层的具体数学实现，Steve Mould 坦言算法门槛极高，非长期深造难以透彻解读。MIT 论文中涉及的核心理论术语，诸如“复数可操控金字塔的相位变化”（phase variation of a complex steerable pyramid），直接展示了该算法在多尺度空间频域相位分解方面的硬核数学深度。 ### [10:30](https://www.youtube.com/watch?v=rEoc0YoALt0&t=630s) 医疗与监控场景的应用潜力及相关技术延伸在探讨了运动放大技术（Motion Amplification）的复杂理论之后，Steve Mould 将话题转向它在日常生活和医学领域的巨大应用前景。这种技术能够实现完全非侵入式、无需任何身体接触的生理监测，比如通过摄像头远程测量人体的脉搏。Steve 举了一个非常贴近生活的育儿场景：照顾过非常幼小的婴儿的人通常都有过这样的经历，当看着熟睡中的孩子时，心里总会突然升起一阵恐慌，怀疑孩子是不是停止呼吸了。很多家长的下意识反应是伸手去捅一下婴儿，看到孩子动弹了一下，心里便松了一口气，暗想孩子还在呼吸；然而代价是把孩子吵醒并惹得哇哇大哭。如果借助这种运动放大算法，只需将摄像头对准孩子，系统就会自动放大他们胸部微弱的起伏运动，让家长无需打扰睡眠就能一目了然地确认呼吸状态。 ```mermaid flowchart TD accTitle: 运动放大技术在非侵入式监测中的应用 accDescr: 展示摄像头画面经过运动放大处理后，输出脉搏、呼吸监测与声音还原等医学和声音侦测应用。 subgraph Capture["视觉采集"] A["常规视频画面 (摄像头拍摄)"] end subgraph Processing["算法处理"] B["微小变化检测与运动放大"] end subgraph Applications["实际应用延伸"] C["远程脉搏测量 (完全非接触)"] D["婴儿呼吸监测 (胸部起伏放大)"] E["从物体微振动中提取声音 (Veritasium 案例)"] end A --> B B --> C B --> D B --> E ``` 除了医疗健康监测，这项技术还能扩展到令人惊叹的声音侦测领域。Steve 在视频卡片和片尾画面中推荐了科普频道 Veritasium（Derek Muller）制作的一期相关视频，该视频展示了如何仅通过摄像头观察受声波冲击而发生微弱颤动的物体表面，便能反向提取并还原出视频现场的声音信息。最后，Steve 表达了对 RDI Technologies 公司的 Jeff Hay 的由衷感谢，本期视频中演示的大量专业运动放大素材均由其提供，相关的公司信息也附在了视频简介中。 - 非接触式脉搏测量：无需在患者身上粘贴电极或传感器，即可精准捕捉皮下血管微弱搏动。 - 婴儿无干扰监控：避免因家长确认呼吸而误将熟睡的婴儿弄醒哭闹。 - 视频声音重建：利用声波撞击物体表面产生的亚像素级微小位移还原音频。 - 工业级视觉素材支持：由研发运动放大相机的 RDI Technologies 及其代表 Jeff Hay 授权提供。 ### [11:34](https://www.youtube.com/watch?v=rEoc0YoALt0&t=694s) 学习曲线的痛点与赞助商 Skillshare 话锋一转，Steve 分享了自己学习单板滑雪（snowboard）的一段亲身经历。他当年完全是靠自学摸索，再加上同行几个会滑雪的朋友零星点拨来掌握这项技能的。然而多年之后他发现，这种自学模式埋下了巨大的隐患：由于早期缺乏规范指导，他养成了许多根深蒂固的不良姿势和习惯。如今如果想进一步提升滑雪水平，他必须花费巨大精力去纠正和戒除这些坏习惯，这让他感到极其痛苦。回想过去，他对待生活中的很多事情都是这种“摸着石头过河、自己瞎琢磨”的态度，但随着阅历增长，他越来越觉得这种做法在许多情况下反而是因小失大、代价更高。针对这种自学误区，Steve Mould 直言：“I really need to unlearn those bad habits, which is a total pain in the ass, and that was my approach to life back then generally like if there's something you need to learn, just muddle through, figure it out, but more and more I'm of the opinion that that's a false economy in a lot of situations, like often it's better to invest upfront, spend some time on a proper course or, you know, take a class, and that way, all the time that you invest subsequently is really quality, like you're building on a framework of good practice, it's really gonna accelerate your learning and your improvement.” 他强调，前期投入时间去系统化地上一门优质课程，能建立在良好实践的框架之上，从而大幅加速后续的个人提升。因此，Steve 强烈推荐了本期视频的赞助商——在线学习平台 Skillshare。该平台涵盖了诸如网页设计、摄影、室内设计、动画以及插画等丰富的创意艺术课程，同时也提供领导力和时间管理等商业技能培训。Steve 提到因为家里正在重新装修厨房，他本人最近就学了一门名为“打造你的专属空间”（Styling Your Space）的室内设计课程。他建议大家在空闲时间通过高质量的系统化课程开启自我提升，或者将自己搁置已久的想法用正式的学习来落地。Skillshare 平台采用订阅制，全站畅学费用为每月 10 美元；而通过专属链接注册的前 500 名观众可以免费获得两个月的无套路高级会员体验。最后，Steve 呼吁观众喜欢视频就点击订阅，并在片尾欢快的音乐中与大家道别。 ## Notable quotes > [03:39](https://www.youtube.com/watch?v=rEoc0YoALt0&t=219s) you know, so lots of times that things are moving together, that's some indication that that's good, that's all right, and sometimes when they're opposing each other, that's where the problems come in. — Jeff Hay ## Mentioned Steve Mould, Jeff Hay, RDI Technologies, Skillshare, MIT CSAIL, Google, Google Books, Waymo, Veritasium, Derek Muller, Styling Your Space ## Video description The first 500 people to use this link will get 2 free months of Skillshare Premium Membership: https://skl.sh/stevemould8 Motion Amplification and Video Magnification are techniques that find subtle changes in a video that are invisible to the naked eye and amplify them so they become visible. They have huge diagnostic potential in industry and medicine. Here's RDI Technologies: https://rditechnologies.com/ Here's the video magnification research: http://people.csail.mit.edu/mrub/vidmag/ Here's where you can upload your own videos: https://lambda.qrilab.com/site/geko/ Here's Derek's video from Veritasium: https://youtu.be/eUzB0L0mSCI You can buy my books here: https://stevemould.com/books You can support me on Patreon here: https://www.patreon.com/stevemould just like these amazing people: Glenn Watson Peter Turner Joël van der Loo Matthew Cocke Mark Brouwer Deneb Twitter: http://twitter.com/moulds Instagram: https://www.instagram.com/stevemouldscience/ Facebook: https://www.facebook.com/stevemouldscience/ Buy nerdy maths things: http://mathsgear.co.uk [Source](https://www.youtube.com/watch?v=rEoc0YoALt0)

Source quotes

[3:39] Jeff Hay: you know, so lots of times that things are moving together, that's some indication that that's good, that's all right, and sometimes when they're opposing each other, that's where the problems come in.

Source quotes

[2:01] Jeff Hay: usually the rule of thumb is you have to sample, you have to take data at a rate twice the frequency you're interested in.

Original transcript

[0:00] - Suppose that you manage a factory that makes widgets, [0:03] you've got a spinning thing over there, [0:04] you've got an up and downy thing over here, [0:07] it's a complex setup that produces a lot of vibrations, [0:11] so you're quite reasonably concerned that one day [0:14] your factory could shake itself to bits [0:17] like a small vibration here may cause a bolt to come loose, [0:22] which causes even more vibrations [0:24] that leads to metal fatigue over here, [0:27] which leads to even larger vibrations, [0:29] it's a snowball effect. [0:31] So you wanna catch the vibrations early [0:33] before they become a problem, [0:35] before they're even visible to the human eye, [0:38] so what do you do? [0:39] Well, you could put some sensors on things [0:41] that you're worried about, [0:43] so maybe you put an accelerometer on this pipe over here [0:46] that you're worried might be moving up and down too much, [0:49] and that sensor supplies you with data [0:52] and you plot a graph and you can see it moving up and down [0:54] and it's within limits and you can monitor it, [0:57] and it's fine. [0:58] But what if there's a part of the factory you forget [1:00] to put a sensor on because you don't think it's a concern? [1:05] What if you could take millions of sensors [1:09] and just throw them across your factory [1:12] and capture everything? [1:14] It turns out you can, and in fact, [1:16] you carry a set of these sensors with you all the time, [1:20] I'm talking about your phone, [1:21] specifically the camera in your phone, [1:24] that chip that gathers light for the picture, [1:27] the CMOs chip is made of millions of individual sensors, [1:32] one for each pixel, [1:33] well I suppose three for each pixel actually. [1:36] In reality, you're probably gonna wanna use something better [1:38] than your camera phone [1:39] selling industrial with a global shutter. [1:41] So you get a camera and you take some videos [1:44] of your factory, some really wide shots to begin with, [1:48] make sure you capture absolutely everything, [1:51] but if what you're looking for is vibrations [1:53] that are so small, [1:54] they're not perceptible with human vision, [1:57] then how can a camera see these vibrations? [2:01] That's where motion amplification comes in, [2:04] in fact, all the footage I've shown you so far [2:06] is motion amplified, for example here, [2:08] you can see side-by-side the original footage [2:12] and the motion amplified footage. [2:14] The original footage doesn't look like it's moving at all, [2:16] but there is a subtle movement information in there [2:18] that's been amplified on the other side. [2:20] In fact, an original video can contain sub pixel movements [2:25] and still have that information amplified, [2:27] so you don't even have a hard edge sweeping [2:29] across several pixels, [2:31] instead, you've got a line of pixels that are getting darker [2:35] while the neighboring pixels are getting lighter [2:38] and you can interpret that information as movement [2:41] and amplify it, it's incredible really, [2:43] especially when you consider [2:44] if you're extracting this information [2:46] against the background of noise, [2:48] all digital images and videos contain a bit of noise, [2:51] which is a fancy way of saying random fluctuations [2:55] in brightness for individual pixels. [2:57] I spoke to Jeff Hay at RDI Technologies [3:00] who created emotion amplification system [3:02] that's used in industrial sector. [3:04] - I started in astronomy, [3:06] thought I was gonna be an astronomer, [3:07] but I think I had a conversation with my advisor [3:13] that was about being gainfully employed, [3:15] you know remarkably the problem sets kind of the same, [3:17] you know what I mean? [3:19] You're using the camera to figure out something [3:24] about some object that's, you know, kind of at a distance, [3:28] the other hobby that I had, [3:29] I was the photo editor for my university newspaper, [3:32] and so I've got a photography background as well, [3:34] so all these things sort of just started to come together, [3:37] you know it's no secret where I ended up. [3:39] Even a megapixel camera, which is by today's standards, [3:43] kind of small and in resolution, [3:46] that has a million sensors in it, [3:47] you can blanket an object with sensors [3:50] and measure everywhere, [3:52] but when you have sort of like a motor and a pump, [3:54] you know, they're connected, the shafts are connected [3:56] at something that they call a coupling, [3:58] which is just essentially connecting [3:59] those two pieces of equipment, [4:01] and if one side, if for example, [4:03] the pump is going up while the motor is going down, [4:07] that's misalignment and that's bad, [4:09] you know, so lots of times that things are moving together, [4:13] that's some indication that that's good, [4:14] that's all right, and sometimes [4:15] when they're opposing each other, [4:17] that's where the problems come in. [4:20] - If you've got a 50 frames per second camera [4:22] and you've got something that's vibrating 15 times a second, [4:24] you may not pick it up, [4:25] do you have to worry about that sort of thing? [4:27] - Yeah, so usually the rule of thumb is you have to sample, [4:31] you have to take data at a rate twice [4:34] the frequency you're interested in. [4:35] - With this system you can look at the raw data as well, [4:38] like you can say, look, tell me what's going on [4:39] with these pixels here, [4:41] I'm interested in the vibrations at that point, [4:42] and you can see a nice graph and that's really important, [4:45] but I'd also say that this summarized visual representation, [4:51] that's so easily digested by the human visual cortex [4:55] is also really important because human brains are better [4:59] than computers at certain tasks, maybe not forever, [5:03] but it's true for now, which is why we use human brains [5:07] to train artificial intelligence [5:09] in things like computer vision. [5:11] This is a bit of a diversion, [5:12] but when you log onto a website [5:15] and you have to prove that you're human and not a robot, [5:17] you fill in a CAPTCHA, [5:19] and in the old days that used to be two words [5:22] that were garbled in some way, [5:24] and you had to type out what the words were, [5:26] an easy task for human vision, a hard task for computers, [5:30] but actually only one of those words [5:33] was to test whether you are a human, [5:34] the other word was something that Google was struggling with [5:39] as part of their book digitization project. [5:41] So here's a word we don't recognize in a book, [5:44] we're gonna outsource it to a human [5:46] through the CAPTCHA process, [5:48] and it doesn't just help with that one word [5:51] in that one book, it also trains the AI [5:54] to be able to recognize that kind of word in future. [5:57] By outsourcing word recognition to humans in this way, [6:00] Google were able to blaze through [6:03] their book digitization projects, [6:05] which is why you don't see these word CAPTCHA anymore, [6:08] so what do you see now? [6:09] Well, for a while it was numbers if you remember, [6:12] and it would be a picture of a number [6:15] that look suspiciously like a blurry photograph [6:18] of a front door, and guess what? [6:21] That was helping Google maps to identify addresses [6:24] through pictures, taken from Google street view, [6:28] these days the CAPTCHA task, [6:29] it tends to involve something like click on all the images [6:33] that contain a road sign [6:34] or click on all the images that contain a bus, [6:37] and guess what, that's because Google's parent company [6:40] has an autonomous vehicle project [6:42] and they need human brains [6:45] to help them train their artificial intelligence, [6:47] to recognize things like road signs and buses. [6:51] Interestingly, those CAPTCHAS are real time, [6:54] so if you're asked to identify a road sign for example, [6:58] that's because there's a car out there on the street [7:00] at that moment that needs to know that that's a road sign, [7:03] so if you see that you need to act quickly, [7:06] otherwise someone could get hurt, that last bit isn't true, [7:10] or is it? [7:12] It isn't. [7:14] But anyway, the point is, [7:15] if you can take your data and turn it into something visual, [7:19] you might be able to outsource [7:20] some of the processing of that data to a human brain [7:23] that can produce remarkable insights [7:26] without even breaking a sweat. [7:27] And by the way, having this visual representation [7:31] really helps when you're trying to persuade superiors [7:34] to spend some money. [7:36] - So if you go to somebody and you show them a plot [7:39] on a graph and you say, [7:41] man that machine is gonna come apart, you know, [7:43] that's a hard, you know, [7:46] but if you can show them that it's lifting up off the ground [7:49] or it's cracked, or you know, [7:53] that's kind of an element to it as well. [7:55] - You talked about having, [7:56] you got some footage of engines and things like that. [7:59] - We've got car engines, [8:04] we've got some more common, you know, [8:07] sort of everyday stuff that we could do, [8:09] I think it would be a little bit more universal to people. [8:11] - Jeff is understandably reluctant [8:13] to talk about the finer technical details [8:16] of how the technology works, [8:18] but I really wanted to see if I could create emotion, [8:21] amplified video of my own, [8:23] I did find some research from scientists at MIT, [8:27] they're calling it video magnification, [8:30] it's the same kind of idea, [8:32] you can amplify motion in a video, [8:34] you can also amplify color change in a video, [8:37] let me show you what I mean. [8:38] This is my wrist, you can just about see a pulse there, [8:43] but look here it is with motion amplification applied, [8:47] I'll link some links to their research and description, [8:49] they've got software you can download, [8:51] they've even got an online system where you can upload video [8:54] and play with algorithm, it's really cool, [8:56] you can even detect pulse through color amplification. [9:00] The color of my face is changing very, very slightly [9:04] with each beat of my heart, [9:06] you can't see it with the naked eye, [9:07] but here it is amplified, [9:11] and then I blink. [9:13] Actually you can detect pulse just from me standing still [9:16] here I'm trying to be as still as possible, [9:18] but every time blood is pumped through my neck, [9:20] my head wobbles slightly, here it is amplified, [9:26] there was a lot more motion in my body [9:28] than just the wobbling of my head due to my pulse, [9:31] but the algorithm that generates the resulting video [9:35] is really clever, and you can filter by frequency. [9:38] So I can say I'm only interested in motion [9:41] that has a frequency between one Hertz [9:44] and one and a half Hertz, [9:45] because I know my pulse was around that [9:48] and it will amplify just those motions, [9:51] and the same goes for the footage [9:53] of the factories by the way, [9:55] you can filter by frequency to pick out different things [9:58] that you're interested in, [9:59] this method works even if I'm wearing a mask. [10:07] I'd love to talk in detail about how the algorithms work [10:10] that generate the videos, but honestly, [10:13] I get the feeling that you'd need to study this stuff [10:15] for a very long time before you could even start to describe [10:19] how it works. [10:20] Here's a choice phrase from the research, [10:23] it's to do with [10:26] phase variation of a complex steerable pyramid, [10:30] what does that even mean? [10:31] So I'm gonna declare that beyond the scope of this video, [10:35] you may already be imagining medical applications for this, [10:38] like taking someone's pulse noninvasively [10:41] or without having any physical contact with them. [10:44] If you've ever looked after very, very small children, [10:47] you probably watched them sleep [10:48] and you've probably started to worry at some point, [10:50] have they stopped breathing? [10:52] So you poke them and they move and you think, [10:55] oh okay, phew, they were breathing, [10:57] I mean, now they're awake and crying, [10:59] but at least I know they're breathing. [11:01] Imagine if instead you could train a camera on them [11:04] and amplify the up and down motion of their chest [11:07] and know for sure that they were breathing, [11:10] I'm gonna put a link in the card and on the end screen [11:12] to a very Tassian video that's somewhat related [11:15] that shows how you can extract audio information [11:17] from a video by looking at the subtle movement of objects [11:20] in the video that have sound energy impinged upon them, [11:25] it's really good. [11:25] Thanks again to Jeff Hay from RDI Technologies, [11:28] for all the motion amplification footage, [11:31] details about his company are in the description. [11:34] Segue, I can snowboard, maybe it's not that interesting, [11:39] but what is interesting is the way that I learn, [11:41] I basically taught myself with a little bit of help [11:43] from a couple of people who were with me, [11:45] who could already snowboard, [11:47] the issue with that approach, [11:49] which I've learned over the years [11:51] is that I have a few habits that are now really ingrained, [11:56] and if I want to get any better at this point, [11:58] I really need to unlearn those bad habits, [12:01] which is a total pain in the ass, [12:03] and that was my approach to life back then generally [12:06] like if there's something you need to learn, [12:08] just muddle through, figure it out, [12:11] but more and more I'm of the opinion [12:13] that that's a false economy in a lot of situations, [12:16] like often it's better to invest upfront, [12:20] spend some time on a proper course [12:24] or, you know, take a class, [12:26] and that way, all the time that you invest subsequently [12:31] is really quality, like you're building on a framework [12:35] of good practice, it's really gonna accelerate your learning [12:39] and your improvement. [12:40] So these days I'm a real advocate for online video learning [12:45] like the sponsor of this video, Skillshare. [12:48] Skillshare have courses in things like web design, [12:51] photography, interior design, animation, illustration, [12:56] I took the styling your space [12:57] interior design course recently actually, [12:59] because we're remodeling our kitchen, [13:01] broadly the categories are like creative stuff, [13:03] artistic things, but also like business-related things [13:06] like leadership or productivity, [13:10] really there's a huge spectrum of stuff on there, [13:12] and you know, maybe you find yourself [13:14] with a bit more free time in this current situation, [13:17] maybe it's time for a bit of well-directed self-improvement [13:21] or maybe you've been mucking about with an idea, [13:23] and it's time to kickstart the idea [13:26] with a proper chunk of formal learning, [13:28] but whatever your story is, [13:29] I recommend you have a browse through Skillshare [13:32] because you might get inspired to start something new. [13:37] The whole thing is just $10 a month, [13:39] you get access to absolutely everything for that, [13:41] and because they're sponsoring this video, [13:43] the first 500 people to go to skl.sh/stevemould8 [13:48] will get two months absolutely free, no strings attached, [13:52] so check out Skillshare today. [13:54] I hope you enjoyed this video, [13:55] if you did, don't forget to hit subscribe [13:58] and I'll see you next time. [13:59] (upbeat music)