From deepfakes to DNA: the science of watermarking AI

From deepfakes to DNA: the science of watermarking AI — Google DeepMind

Google DeepMind

AI-assisted summary of From deepfakes to DNA: the science of watermarking AI — original source. Check the original for full context.

Source published:

Key ideas

  • Google DeepMind has expanded its SynthID watermarking technology from text, images, and video into synthetic biology with SynthID Bio, targeting AI-designed protein sequences and 3D biomolecular structures. (19:30)
  • In a first-of-its-kind breakthrough, DeepMind completed in vitro wet-lab validation proving that watermarked synthetic proteins maintain binding hit rates and functional affinity identical to natural controls. (27:30)
  • SynthID Biosequence leverages the open-source SynthID Text architecture alongside ProteinMPNN to swap synonymous amino acids during generation without compromising physical folding. (25:00)
  • The biosecurity initiative aims to provide commercial DNA synthesis providers with verifiable cryptographic provenance to screen out AI-engineered pathogen designs that evade legacy sequence-matching filters. (20:00)

Detailed notes

## Background Published by Google DeepMind on October 1, 2026, via web and podcast platforms, this episode features host Professor Hannah Fry exploring the challenge of distinguishing authentic content from machine-generated media. According to the show description, Fry sits down with Pushmeet Kohli, VP of Science, and biosecurity researcher Jeremy Ratcliffe to examine SynthID, a watermarking technology scaling from text, images, and video directly into biology.

## TL;DR DeepMind leaders Pushmeet Kohli and Jeremy Ratcliffe explain the mechanics of SynthID, detailing how active statistical watermarking replaced failing passive detection across images, video, and text generation. The conversation centers on the breakthrough transition into synthetic biology via SynthID Bio, which embeds imperceptible mathematical marks into both 3D protein structures and linear amino acid sequences without corrupting molecular stability. Backed by successful wet-lab validation, the researchers demonstrate that watermarked proteins retain their original biological function and binding affinities. This open-source framework serves as a cornerstone for commercial DNA synthesis screening and DeepMind's broader bioresilience doctrine, aiming to disincentivize malicious biological threats before physical synthesis can occur.

## Atmosphere & style The dialogue is intellectually brisk, lucid, and grounded, driven by Hannah Fry's sharp curiosity and accessible analogies that demystify high-level probability math and structural biology. Pushmeet Kohli provides calm, institutional authority regarding generative AI safety architectures, while Jeremy Ratcliffe offers a thoughtful, cautious counterweight rooted in biosecurity realism. The emotional arc moves steadily from everyday digital concerns like deepfakes and school essays to the unsettling stakes of synthetic pathogen synthesis, leaving the listener with a feeling of quiet scientific urgency and proactive defense.

## Bulletin - `19:30` Google DeepMind has expanded its SynthID watermarking technology from text, images, and video into synthetic biology with SynthID Bio, targeting AI-designed protein sequences and 3D biomolecular structures. - `27:30` In a first-of-its-kind breakthrough, DeepMind completed in vitro wet-lab validation proving that watermarked synthetic proteins maintain binding hit rates and functional affinity identical to natural controls. - `25:00` SynthID Biosequence leverages the open-source SynthID Text architecture alongside ProteinMPNN to swap synonymous amino acids during generation without compromising physical folding. - `20:00` The biosecurity initiative aims to provide commercial DNA synthesis providers with verifiable cryptographic provenance to screen out AI-engineered pathogen designs that evade legacy sequence-matching filters.

## Key takeaways - `04:47` Passive forensic detection tools hit a theoretical limit as generative AI improved, necessitating active in-generation watermarking to preserve trust and provenance. - `02:35` Viable watermarking frameworks rely on three non-negotiable principles: imperceptibility and quality preservation, robustness against modifications or scrubbing attacks, and scalability with low latency. - `06:21` While high-dimensional images provide millions of continuous values to conceal patterns, low-dimensional discrete text requires subtle mathematical shifts in next-token probability distributions. - `10:00` Text watermarking requires generative optionality; highly constrained or single-word factual responses cannot be watermarked without degrading generation quality. - `13:10` Unrestricted public access to watermark detectors creates an adversarial security vulnerability, enabling bad actors to reverse-engineer keys and design evasion attacks. - `15:00` Image and video watermarking uses co-trained adversarial neural networks where a generator and detector are iteratively tested against common real-world transformations. - `17:00` SynthID detection operates cross-platform across frontier labs via shared cryptographic key federation within centralized verification portals. - `20:00` Commercial DNA synthesis providers face critical vulnerabilities from generative models that can design sequences looking completely benign in legacy screening databases while folding into hazardous pathogens. - `20:00` SynthID Bio applies two distinct approaches: subtly adjusting atomic coordinates in 3D structures (AlphaFold 3) or substituting functionally equivalent amino acids in linear sequences (SynthID Biosequence). - `25:00` Sequence watermarking integrates with inverse protein design tools like ProteinMPNN, treating amino acid ribbons analogously to transformer token generation. - `27:30` Wet-lab experimental validation confirmed that watermarked synthetic protein binders preserve binding efficiency and folded stability without functional loss. - `30:00` Deploying biological verification at scale requires automated key-exchange protocols and mutual trust between generative model developers and physical DNA synthesis foundries. - `32:00` SynthID Bio operationalizes DeepMind and Isomorphic Labs' broader bioresilience strategy, structured across three pillars: prevent, detect, and respond. - `32:00` The core theory of biological defense change rests on drastically reducing the utility of biological weapons and raising the technical barrier to entry to disincentivize malicious actors. - `34:25` Unlike easily strippable file metadata, SynthID binds the watermark directly into functional sequence choices so that attempting to scrub the mark disrupts tertiary folding and destroys protein function.

## Detailed walkthrough ### `00:00` The Provocation: Distinguishing Real from Machine

Hannah Fry opens the podcast by setting a familiar digital scene: scrolling through social media feeds and encountering staggering footage—such as a slow-motion volcanic eruption, a tornado ripping through a village, or dramatic protest footage. In these moments, viewers reflexively wonder whether what they are seeing is real or generated by an algorithm. As artificial intelligence advances across text, imagery, audio, and video, finding the answer grows increasingly difficult. The stakes become far higher as AI enters the realm of biology, designing novel biological molecules, enzymes, and structures that never existed in nature—some of which could potentially be designed to cause harm. Fry highlights the central question: how can dangerous biological designs be stopped before they are ever synthesized?

The emerging safeguard is not a crude, visible logo like a stock photo stamp, but an embedded, subtle mathematical signature: a digital watermark. Fry introduces her guests from Google DeepMind: Pushmeet Kohli, VP of Science and one of the principal architects of the SynthID watermarking system, and Jeremy Ratcliffe, a biosecurity researcher whose team is translating this foundational technology into biological engineering.

Jeremy Ratcliffe explains that the primary motivation in biological applications is provenance—establishing an auditable trail of where a specific sequence originated. The core objective is distinguishing natural sequences found in the living world from synthetic sequences produced by generative models, particularly when an AI-generated sequence exhibits dangerous properties that cannot be diagnosed merely by reading the raw sequence itself. When Fry asks what defines an effective watermark, comparing it colloquially to banknotes or passports, Ratcliffe stresses human imperceptibility:

> “I think for us, in particular, human imperceptibility is one. So making it very difficult for someone to be able to tell what portion of data has been watermarked. Because otherwise it would be really easy to erase.”

Beyond imperceptibility, Ratcliffe emphasizes high-confidence automated detection and generalizability—building an adaptable foundational watermarking technique capable of operating across diverse data types rather than engineering fragile, bespoke systems for every individual medium.

`02:35` The Three Core Pillars of Watermarking

Pushmeet Kohli traces DeepMind’s watermarking initiative back nearly eight years, well before generative media reached the cultural mainstream. A decade ago, it was difficult to envision an era where neural networks could synthesize photorealistic images, lifelike video, convincing audio, coherent prose, and functional enzymes indistinguishable from natural counterparts. The fundamental mission of watermarking is to preserve trust and provide users with reliable provenance so they can believe what they encounter.

Kohli outlines the three non-negotiable criteria required for any viable watermarking architecture:

```mermaid flowchart TD accTitle: Three Core Properties of Watermarking accDescr: Diagram illustrating the three foundational design requirements for modern AI watermarking systems: Imperceptibility, Robustness, and Scalability. subgraph Requirements["SynthID Design Principles"] Q["1. Imperceptibility & Quality<br/>Does not degrade fidelity or change semantic meaning"] R["2. Robustness to Attack<br/>Persists through edits, crops, and transformations"] S["3. Scalability & Ease of Use<br/>Seamless integration at generation and rapid detection"] end ```

1. **Imperceptibility and Quality Preservation**: The watermark must not degrade the signal quality or distort semantic meaning; if quality is compromised, creators and end-users simply will not adopt the model. 2. **Robustness Against Transformations and Attacks**: The embedded signal must persist even if an adversary intentionally attempts to scrub it or subjects the file to downstream modifications, compression, cropping, or filtering. 3. **Scalability and Ease of Use**: The marking mechanism must integrate natively into the generation pipeline with negligible latency, while enabling downstream systems to verify the presence of the mark reliably and quickly.

Kohli emphasizes that these three principles—imperceptibility, robustness, and scalability—give users transparent insight and agency over the content they consume or design.

`04:47` Why Detection Failed and Watermarking Succeeded

Hannah Fry presses Kohli on why watermarking suddenly captured global attention, pointing out that public generative models have circulated for several years. Kohli details the historical shift from passive forensic detection to active watermarking. In the earlier days of digital image manipulation, forensic software could identify subtle artifacts and pixel-level inconsistencies left behind by tampering tools. However, as generative deep learning models matured, those telltale statistical artifacts vanished entirely.

When generative models became capable of synthesizing media indistinguishable from raw camera captures or human-written prose, passive detection hit a theoretical wall. This sparked the realization that forensic analysis alone was no longer viable:

> “The way to sort of make sure that humans always have the ability to understand the origin of where some signal came from is by injecting a signal within it.”

By intentionally embedding an imperceptible statistical bias into the generation process itself, DeepMind ensured that the identity of the source model survives even when the raw output appears completely indistinguishable from organic reality.

`06:21` From Dense Pixels to Sparse Text and Biological Sequences

Kohli recalls that the DeepMind team tackled image generation first. High-resolution imagery offered a forgiving canvas: a one-megapixel image contains between one million and three million numerical values depending on color encoding, providing an enormous state space in which to conceal mathematical patterns without corrupting visual quality. Text presented a vastly steeper hurdle. Written text is low-dimensional, highly discrete, and brittle; altering words or tokens risks immediately distorting factual meaning, grammatical coherence, or tone. DeepMind had to devise entirely distinct statistical algorithms for text, embedding marks through subtle shifts in token sampling probability distributions rather than perturbing continuous values.

The conversation turns to international governance and industry deployment. Fry notes that watermarking has moved into hard law, with regulatory frameworks such as the European Union’s AI Act requiring generative content to carry origin markings. Kohli points out that SynthID’s watermarking frameworks have seen broad cross-industry adoption:

> “I think what regulators and countries around the world have recognized that they want to give users this information as to the origins of the content that they are consuming. And one sort of key technology that is seen as a robust solution for it that can scale is watermarking, which is why not only have we seen a take-up of this idea across the industry, the fact that SynthID is used by NVIDIA, is used by other partners like with OpenAI and so on, it's a remarkable sort of thing that many of our partners in the generative AI space have adopted this technology.”

Ratcliffe notes that biological sequences (such as amino acid or nucleotide strings) resemble text more than images. Because biological sequences are discrete strings of letters representing structural building blocks, his team directly leveraged DeepMind’s open-source SynthID Text library. Rather than creating a biological watermarking platform from scratch, they took the existing codebase and adapted it to the constraints of synthetic biology. Fry concludes the segment marveling at the underlying mechanism: the mark relies neither on crude visual modifications nor on trivial hidden whitespace or punctuation, but hides mathematically within the generated sequence itself.

- **Historical timeline**: DeepMind began developing internal watermarking methods roughly eight years prior (circa 2018). - **Signal capacity contrast**: High-dimensional media (images, audio, video) contain millions of continuous data points for embedding, whereas low-dimensional text requires delicate manipulation of discrete token probabilities to protect the underlying message. - **Biosecurity synergy**: The open-source SynthID Text library provided the foundational software foundation for watermarking machine-generated biological sequences.

`10:00` How Text Watermarking Exploits Next-Token Probability

Existing online tools that claim to detect AI-generated text rely on standard machine learning classifiers trained on large datasets. Pushmeet Kohli explains that while early conversational models had obvious stylistic tells—such as recurring phrasing patterns or overusing certain words—modern models have grown far more sophisticated:

> “And so if you have this machine learning classifier which is trying to distinguish, the classifier's accuracy goes down over time as the generative AI models become better at replicating what they are targeted to achieve.”

Instead of post-hoc classification, SynthID text watermarking intervenes directly during the generative process. When a large language model generates a sentence token by token, it produces a probability distribution over potential next words. Pushmeet illustrates this optionality using a conversational example: if asked what someone ate for breakfast, the model could equally well begin with “I...” or address the questioner by name with “Pushmeet...”. SynthID introduces a secret pseudo-random key at inference time that subtly biases the sampling selection among these equally plausible candidate tokens.

```mermaid flowchart LR accTitle: Text Watermarking Selection Flow accDescr: Shows how an LLM's probability distribution is biased by a secret key to form identifiable token patterns without degrading generation quality.

subgraph Gen ["Token Generation"] A["Input Prompt"] --> B["LLM Probability Distribution"] K["Secret Watermark Key"] --> C["Biased Sampling Layer"] B --> C C --> D["Generated Token Chain"] end

subgraph Det ["Detection Phase"] D --> E["SynthID Detector"] K --> E E --> F["AI Attribution Result"] end ```

The system does not rely on repeatedly forcing specific odd words into the response, which would degrade text quality. Instead, the detector looks across extended passages for mathematical patterns and correlations in how those biased token chains link together. Because the watermark relies on statistical bias across several word choices, highly constrained or short responses cannot be watermarked. If a prompt has only one deterministic answer—such as asking for the capital of France, where the answer is simply “Paris”—the model has zero optionality, and watermarking is impossible.

- Machine learning classifier accuracy steadily degrades as generative models improve at replicating human output. - Text watermarking relies on the model having "optionality" (multiple valid next-token choices). - A secret key introduces subtle biases into the sampling distribution without harming readability. - Multi-model attribution is possible: different models or organizations can apply distinct secret keys to trace provenance back to a specific engine. - Extremely short or strictly factual snippets (e.g., "Paris" as the capital of France) cannot carry a watermark.

---

`13:10` The Realities of Essay Cheating and Detector Evasion

Hannah Fry asks whether this implies students can no longer cheat on essays and whether AI generation is now universally detectable. Jeremy Ratcliffe cautions against viewing any tool as a silver bullet. While the industry is broadly converging toward watermarking standards, deploying public detectors poses an inherent security trade-off:

> “Were a detector publicly available and you could just query it over and over and over again, you might be able to work back to making something that gets around that specific configuration of keys that is presented in the public detector.”

If bad actors have unrestricted access to a detector endpoint, they can execute model-extraction or black-box adversarial attacks, querying the detector repeatedly to reverse-engineer key configurations and learn the exact threshold boundaries to bypass identification. Thus, while watermarking represents a massive leap beyond heuristic classifiers, it does not guarantee a permanent, cast-iron catch-all under every public deployment condition.

- Other frontier AI labs have announced moves toward watermarking implementations. - Open-access public detectors create an adversarial incentive, enabling users to probe the detector and develop evasion techniques. - Watermarking provides high-confidence attribution, but maintaining key security and access controls remains necessary to prevent circumvention.

---

`15:00` Image and Video Watermarking via Adversarial Neural Networks

Image and video watermarking operates on an entirely different technical paradigm than text generation. Rather than biasing next-token sampling tables, image watermarking uses a pair of neural networks trained jointly in an adversarial game. An unwatermarked image is passed to a generator network that embeds a subtle, imperceptible signal into the pixel structure.

```mermaid flowchart TD accTitle: Co-Trained Adversarial Image Watermarking accDescr: An unwatermarked image is injected with a signal, passed through simulated adversarial transformations, and verified by a co-trained detector.

subgraph Embed ["Embedding Pipeline"] I["Original Image"] --> G["Watermark Generator Network"] G -->|"Injects imperceptible signal"| W["Watermarked Image"] end

subgraph Attack ["Adversarial Distortions"] W --> A["Adversarial Agent: Crop, Zoom, Rotate, Add Noise, Compress"] end

subgraph Detect ["Attribution"] A --> D["Co-Trained Detector Network"] D --> R["Detection Confidence"] end ```

To guarantee that the watermark survives real-world editing, the generator and detector are trained against an intermediate adversarial module. This adversary subjects the watermarked image to common digital manipulations: - Resizing, zooming, and extreme cropping - Geometric rotations - Lossy compression, contrast adjustments, and noise injection

The generator learns to place the signal where it cannot be easily stripped by these transformations, while the detector learns to recognize the signal despite severe visual distortions, all while keeping the mark invisible so as not to degrade visual fidelity.

---

`17:00` Tackling Misinformation and Cross-Platform Detection Portals

Pushmeet reflects on the real-world impact of SynthID in journalism and media verification. High-profile incidents—such as synthetic images of miraculously untouched homes amidst California wildfires or faked depictions of military conflict in Iran—have been debunked by newsrooms running SynthID. Pushmeet highlights this operational success:

> “And SynthID being used to actually tackle misinformation in real time and at scale.”

Detection capabilities have been directly integrated into consumer products, including the Gemini app. Users can upload an image and ask whether it was produced by Google's generative models; Gemini then calls the SynthID detection service to verify provenance.

```mermaid flowchart TD accTitle: Partner Key Federation in SynthID Portal accDescr: Displays how external organizations can share secret keys to allow a centralized portal to identify AI provenance.

A["External Image / Content"] --> P["SynthID Detection Portal"] K1["Google Internal Keys"] --> P K2["Partner Institution Key A"] --> P K3["Partner Institution Key B"] --> P P --> R["Attribution: Generated by Partner A / Google / Unverified"] ```

When asked whether SynthID can identify images generated by competing models, Pushmeet explains the trust federation model: detection relies on knowing the secret key. If external companies and partner institutions adopt the framework and securely register their keys with the central SynthID portal, the detector can attribute content generated by their systems as well. If an external lab refuses to share its keys, third parties cannot verify that lab's watermarks.

- SynthID detection is natively exposed in Google products like the Gemini app for direct user verification. - Detection operates cross-platform via partner key-sharing agreements. - Unregistered, non-partner models remain undetectable by design if their underlying keys are kept proprietary.

---

`19:30` Expanding Watermarking into Biology: SynthID Bio

Hannah Fry shifts the conversation to biological applications, introducing Jeremy Ratcliffe to discuss SynthID Bio. This proof-of-concept technology expands watermarking beyond human-facing communication modalities—text, audio, and video—into synthetic biology, targeting AI-designed protein sequences and 3D biomolecular structures.

Jeremy notes that their release and upcoming research publication specifically detail SynthID Bio Structure, which addresses the challenges of watermarking the spatial coordinates of 3D biomolecules without disrupting their delicate biological and chemical functions.

`20:00` Watermarking Biology: Predicted Structures vs. Biosequences

The team breaks down two distinct ways watermarking applies to biological data. The first technique targets predicted protein structures generated from an input sequence, building directly on AlphaFold 3. In that case, the watermark is embedded by making minute adjustments to the positions of individual atoms—subtly shifting the distance or angle between two atoms while keeping the overall structure intact. The second technology, SynthID Biosequence, operates directly on the protein sequence itself: the actual string of amino acids. As Jeremy Ratcliffe explains, “Well, for structure, it's still the same atoms. They're just maybe in a slightly different configuration. Well, for sequence, we're actually changing the content of the body, but maybe not the actual functional meaning of that content.”

```mermaid flowchart TD accTitle: SynthID Biological Watermarking Approaches accDescr: Comparison of watermarking 3D protein structures versus linear amino acid sequences

subgraph Structure["AlphaFold 3: Structure Watermarking"] S1["Predict 3D Atomic Structure"] S2["Slightly alter atomic distances and angles"] S3["Same atoms, tweaked spatial coordinates"] S1 --> S2 --> S3 end

subgraph Sequence["SynthID Biosequence: Sequence Watermarking"] B1["Predict Linear Biosequence"] B2["Swap select amino acids for similar alternatives"] B3["Changed sequence, identical folded function"] B1 --> B2 --> B3 end ```

Hannah Fry asks why this is necessary in practice, and Jeremy points out the realities of the commercial DNA synthesis pipeline. Laypeople often do not realize that anyone can effectively order custom genetic material from commercial synthesis providers—what Hannah terms DNA “printers.” A major biosecurity hazard is whether someone could evade regulatory oversight to order DNA that enables them to synthesize dangerous biological agents in a physical lab. Jeremy notes that an actor could attempt to recreate “an extinct pandemic, like extinct influenza,” prompting Hannah to emphasize that such an outcome would be “overall bad.” While commercial DNA providers check customer affiliations to prevent synthesis requests from unauthorized individuals, an emerging risk is that AI tools could design functional pathogens using amino acid sequences that look entirely benign to legacy screening systems.

- **Structure vs. Sequence**: Structure watermarking nudges 3D atomic coordinates; biosequence watermarking selects synonymous or functionally equivalent amino acids along the ribbon. - **DNA Screening Vulnerability**: Existing DNA synthesis screening checks order sequences against databases of known regulated pathogens. - **The AI Evasion Threat**: AI protein generators can design sequences that are drastically different in sequence space from regulated pathogens yet fold into the identical hazardous, functional shape. - **Fragment Splicing**: A bad actor could potentially order a disguised, AI-designed harmless-looking sequence fragment, cut and paste it into a gene order, and assemble the active pathogen independently inside a wet lab.

---

`25:00` ProteinMPNN and the Analogy to Generative Text

Embedding a watermark into a biosequence relies on an architecture remarkably similar to large language models. The researchers use ProteinMPNN, a third-party, open-source tool developed outside Google that functions essentially as the inverse of AlphaFold: rather than taking a sequence and predicting its 3D folded structure, it takes a targeted 3D structure and designs an amino acid sequence that will fold into that shape. Because a folded protein is assembled from a linear ribbon of amino acids, the token generation process in ProteinMPNN mirrors how transformer models produce sequences of text token by token.

```mermaid flowchart LR accTitle: Inverse Protein Design Pipeline accDescr: Contrast between AlphaFold and ProteinMPNN workflows

subgraph AlphaFold["AlphaFold"] A1["Amino Acid Sequence"] -->|"Predicts fold"| A2["3D Folded Structure"] end

subgraph ProteinMPNN["ProteinMPNN + SynthID"] B1["Target 3D Structure"] -->|"Inverse design"| B2["Watermarked Biosequence"] end ```

When designing sequences to fit a target geometry, the model encounters positions where it can freely substitute chemically similar amino acids—such as swapping leucine for isoleucine—without disrupting the resulting shape. SynthID Biosequence guides these choices during sampling to embed an invisible watermark. Addressing Hannah's skepticism about whether altering amino acids degrades molecular behavior, Pushmeet Kohli and Jeremy explain that the whole point was to prove real-world biological viability: “And that's what is very cool about it, that we have now in existence watermarked proteins. These are real things. This is not in simulation. These are real things which do stuff, but which are watermarked and will always be detectable.”

- **Structural Synonyms**: Certain amino acids share near-identical physical and chemical traits (e.g., leucine and isoleucine), letting the model make substitutions that preserve folded stability. - **Functional Invariance**: In wet-lab validation, watermarked protein designs achieved binding hit rates and quantitative binding affinities near-identical to unwatermarked controls. - **Laboratory Surprise**: Jeremy admits that prior to receiving the experimental wet-lab data, he was skeptical about whether biological systems would tolerate the alterations, remarking that how easily it worked “blew all of our minds.”

---

`27:30` Wet-Lab Validation, Open Source, and Ecosystem Adoption

To make biological watermarking an effective biosecurity safeguard, detection cannot remain proprietary or confined to simulated models. Google DeepMind physically manufactured synthetic watermarked protein binders in a wet lab across two distinct watermarking schemes to confirm they retained their binding efficiency without functional loss. Having established proof of concept, the team is open-sourcing the technology so that academic researchers, model developers, and commercial gene foundries can integrate the watermarking and verification pipeline directly into their infrastructure.

To operationalize detection across the industry, synthesis companies and AI developers must coordinate on shared security primitives. If a third-party laboratory or DNA vendor receives an order, they can verify the presence of the watermark provided they have access to the model provider's key and context-length settings. Because Google does not operate a commercial DNA synthesis service or a proprietary end-to-end protein manufacturing pipeline, the focus has shifted toward building partnerships across the broader biotechnology ecosystem.

- **Universal Key Verification**: To confirm a watermark in a biosequence, a synthesizer needs the specific generator key and the preceding context length used during sequence generation. - **Engineering Headroom**: DeepMind notes there is ongoing technical headroom to make biological watermarks even more robust, easier to embed, and simpler to detect. - **Ecosystem Coordination**: Deploying biosequence watermarking at scale requires mutual trust and automated key-exchange protocols between generative model creators and physical DNA synthesis providers.

`30:00` Industry Reception and Gene Synthesis Screening

When discussing how to roll out biological watermarking in practice, Jeremy Ratcliffe makes it clear that Google DeepMind does not view itself as the arbiter that will dictate industry-wide synthesis standards. Because DeepMind is not an entity that fulfills DNA or protein printing orders directly, its goal is to act as a catalyst—opening up the conversation and demonstrating how providers can implement watermarking verification into their existing gene synthesis pipelines. When Hannah Fry asks about the reaction from the commercial providers actually manufacturing these physical molecules, Ratcliffe admits his own personal disposition made him brace for pushback: “I'm generally quite skeptical of things. Like, that's just sort of in my nature.” Instead, the industry reception caught the team off guard in the best way possible. Commercial synthesis providers have shown an overwhelming appetite for the tool, which Ratcliffe describes as deeply galvanizing as the team navigates the final deployment steps.

Pushmeet Kohli points out that the biotechnology sector is unusually proactive compared to other industries when it comes to anticipating emerging risks. DNA and protein synthesis providers already operate screening filters to flag dangerous pathogens or dual-use toxins in incoming customer orders. However, these providers are acutely aware that their current screening mechanisms are far from bulletproof and will face unprecedented strain as generative design models mature. Synthesis providers know firsthand that catching malicious orders will only grow harder, making them eager to collaborate across the field on robust, verifiable safeguards.

- DeepMind sees its role as catalyzing implementation across commercial gene synthesis providers rather than unilaterally establishing industry standards. - Synthesis providers currently rely on screening filters to intercept hazardous orders, but openly acknowledge that existing tools are imperfect against sophisticated, novel designs. - Pushmeet Kohli emphasized this proactive mindset: “So, I mean, one thing I found very remarkable about the bio community is how cognizant they are of not just the threats today, but the threats of tomorrow.”

`32:00` DeepMind's Bioresilience Strategy and Theory of Change

Expanding beyond individual protein designs, Ratcliffe explains how SynthID Bio fits into a wider institutional strategy. Approximately two months earlier, Google DeepMind and Isomorphic Labs published a memo detailing their overarching vision for bioresilience. Drawing directly from traditional security doctrine, their framework is organized across three foundational pillars: prevent, detect, and respond. Under this architecture, the lab divides its efforts between mitigating naturally occurring biological threats (such as emergent pandemics) and addressing AI-augmented or AI-designed synthetic threats. SynthID Bio falls squarely into the AI-augmented threat category, giving biosecurity screeners the forensic tools needed to detect generative pathogen designs before they reach a synthesis machine.

Ratcliffe highlights the team's underlying theory of change: by drastically raising the barrier to entry and eroding the utility of biological weapons, malicious actors are disincentivized from pursuing them at all. Building robust early-warning mechanisms for epidemiologists or hardening the public health infrastructure diminishes the strategic viability of engineered outbreaks. Kohli agrees, noting that while generative models and companies like Isomorphic Labs are already transforming drug discovery and biomedical research, safety controls must develop in lockstep. As Kohli frames it: “And we need all the controls and all the sort of measures that we can place to make sure that those tasks are difficult or become infeasible.”

```mermaid flowchart TD accTitle: DeepMind Bioresilience Strategy Framework accDescr: Overview of DeepMind and Isomorphic Labs bioresilience pillars addressing natural and AI threats.

subgraph Pillars ["Bioresilience Security Pillars"] P1["Prevent"] P2["Detect"] P3["Respond"] end

subgraph Threats ["Threat Landscape"] T1["Natural Biological Threats"] T2["AI-Augmented Biological Threats"] end

subgraph Implementations ["Strategic Efforts"] I1["SynthID Bio Provenance & Screening"] I2["Epidemiological Outbreak Detection"] I3["Public Health Resilience Infrastructure"] end

T2 --> P1 I1 --> P1 I1 --> P2 T1 --> P2 I2 --> P2 I3 --> P3 ```

- DeepMind and Isomorphic Labs structure biological defense around three core security pillars: prevent, detect, and respond. - The dual-track approach targets both natural pathogens and AI-generated biological threats. - Jeremy Ratcliffe articulated the foundational theory of change: “So sort of our theory of change is that if you make biology less useful for someone who's trying to use something malicious, then that in and of itself can prevent them from using it in the first case.” - Pushmeet Kohli affirmed the vast scientific upside of generative biology while stressing the necessity of friction: “And I think AI will have immense positive impacts in human health and understanding of biology at large.”

`34:25` Provenance Standards and the Adversarial Cat-and-Mouse Game

Turning back to digital media, Fry asks whether the broader tech sector will ever converge on a single, centralized verification platform that can conclusively determine whether a piece of content is synthetic. Kohli observes that the broader industry is actively aligning around provenance standards that attach audit trails and metadata alongside generated media. However, metadata alone is fragile because an adversary can easily strip it from a file. This vulnerability is precisely why watermarking is essential: it creates an unbreakable cryptographic binding directly inside the underlying signal. While achieving a universal consensus standard across different industry players will take time, combining metadata with robust watermarking is the only path that provides resilient verification.

Fry then questions whether this dynamic sparks an inevitable cat-and-mouse escalation: if biological watermarking algorithms become open industry standards, won't malicious actors simply design tools to identify and strip the watermark out? Kohli explains that SynthID avoids this pitfall by utilizing secret cryptographic keys embedded directly into the physical sequence data. Crucially, the mathematical objective of the watermark is to bind imperceptibly with the protein's core biological function. The watermark is woven so deeply into the sequence that an attacker cannot alter or erase the statistical signal without corrupting the underlying tertiary structure and rendering the protein completely non-functional.

```mermaid flowchart TD accTitle: Watermark Binding versus Fragile Metadata accDescr: Flowchart demonstrating why embedding watermarks directly into biological signals resists stripping attempts compared to metadata.

subgraph FragileMetadata ["Standard File Metadata"] M1["AI Sequence / Media"] --> M2["Attached Metadata Tag"] M2 -->|"Adversary Strips Tag"| M3["Unverified File"] end

subgraph BoundWatermark ["SynthID Signal-Level Binding"] S1["Generative Protein Sequence"] --> S2["Embed Watermark via Secret Key"] S2 --> S3["Functionally Bound Sequence"] S3 -->|"Adversarial Attempt to Strip Signal"| S4{"Alters Protein Structure?"} S4 -->|"Yes"| S5["Function Destroyed / Biologically Inactive"] S4 -->|"No"| S6["Watermark Detected by Screening"] end ```

- Industry provenance initiatives often depend on metadata, which can be trivially stripped unless tightly coupled to in-signal watermarks. - Biological sequence watermarking relies on secret keys so unauthorized parties cannot invert or scrub the embedded pattern. - Pushmeet Kohli framed the ultimate defense against adversarial scrubbing: “Like the ideal situation will be hard to take away to an extent that if you want to get rid of the detection signal, that you have to change the function, right, and destroy the original sort of capability of the task that you had designed the thing for.”

`36:30` Proof of Concept in the Wet Lab and Final Reflections

Fry marvels at the realization that a microscopic biological entity can now carry the provenance of its computational origins encoded directly inside its physical structure. While biological watermarking concepts have existed conceptually, Ratcliffe underlines the breakthrough that sets DeepMind's work apart: they are the first to empirically demonstrate watermark imperceptibility alongside complete biological function preservation, verified directly through wet-lab *in vitro* testing. Prior theoretical approaches often degraded protein stability or folded shape; SynthID Bio preserves functional performance while embedding verifiable traceability into the genetic code.

In her closing remarks, Fry reflects on the cultural shift represented by this research. Safety and guardrails are terms frequently brandished as corporate platitudes across Silicon Valley, but she argues that SynthID represents a rare instance of a laboratory putting its resources behind preemptive defense. Rather than waiting for synthetic biology threats to manifest in the wild or pausing after establishing digital watermarks for consumer imagery, DeepMind developed the mathematical and biological foundations before these tools could be abused. As generative capabilities continue to blur the line between organic nature and algorithmic generation, establishing immutable verification early ensures society stays ahead of emerging hazards.

- SynthID Bio is the first watermarking mechanism demonstrated to preserve biological function and imperceptibility while being verified through wet-lab, *in vitro* results. - Jeremy Ratcliffe summarized their milestone: “And I guess I would say we're not the first ones to do watermarks, but we're the first ones to our knowledge to both show imperceptibility and function preservation and including in vitro and sort of lab based results.” - Hannah Fry concluded the episode by looking ahead to the future of AI governance: “The future, as it turns out, will come with the certificate of authenticity.”

## Notable quotes > `36:30` The future, as it turns out, will come with the certificate of authenticity. — Hannah Fry

> `04:47` The way to sort of make sure that humans always have the ability to understand the origin of where some signal came from is by injecting a signal within it. — Pushmeet Kohli

> `32:00` if you make biology less useful for someone who's trying to use something malicious, then that in and of itself can prevent them from using it in the first case. — Jeremy Ratcliffe

> `30:00` one thing I found very remarkable about the bio community is how cognizant they are of not just the threats today, but the threats of tomorrow. — Pushmeet Kohli

## Mentioned Google DeepMind, SynthID, SynthID Text, SynthID Bio, SynthID Bio Structure, SynthID Biosequence, Hannah Fry, Pushmeet Kohli, Jeremy Ratcliffe, AlphaFold 3, ProteinMPNN, Gemini, Isomorphic Labs, European Union AI Act, NVIDIA, OpenAI

## Video description How do you know if what you're seeing is real? Professor Hannah Fry sits down with Pushmeet Kohli (VP of Science) and Jeremy Ratcliffe (RS) to break down SynthID, the watermarking technology expanding from text, images, and video straight into biology.

[Source](https://afp-920373-injected.calisto.simplecastaudio.com/36cf10bf-f6dd-4a98-96a9-8eda9f59db4f/episodes/83783c09-9840-410b-8653-84a9a446ea98/audio/128/default.mp3?aid=rss_feed&awCollectionId=36cf10bf-f6dd-4a98-96a9-8eda9f59db4f&awEpisodeId=83783c09-9840-410b-8653-84a9a446ea98&feed=JT6pbPkg)

中文简报

从深度伪造到DNA:AI水印技术的科学原理 DeepMind负责人Pushmeet Kohli与Jeremy Ratcliffe阐述了SynthID的工作机制,详细介绍了主动统计水印如何在图像、视频和文本生成中取代逐渐失效的被动检测手段。讨论重点聚焦于通过SynthID Bio跨入合成生物学领域的突破性进展;该技术可在不破坏分子稳定性的前提下,将难以察觉的数学标记嵌入3D蛋白质结构和线性氨基酸序列中。在成功完成的湿实验验证支持下,研究人员证明带水印的蛋白质仍保留了其原始生物学功能和结合亲和力。这一开源框架构成了商业DNA合成筛选以及DeepMind更广泛的生物韧性准则的基石,旨在物理合成发生之前遏制恶意生物威胁。 Google DeepMind已将其SynthID水印技术从文本、图像和视频拓展至合成生物学,推出面向AI设计的蛋白质序列及3D生物分子结构的SynthID Bio。 (19:30) 作为一项开创性突破,DeepMind完成了体外湿实验验证,证明带水印的合成蛋白质所维持的结合命中率与功能亲和力与天然对照组完全一致。 (27:30) SynthID Biosequence利用开源的SynthID Text架构并结合ProteinMPNN,在生成过程中替换同义氨基酸,且不会破坏物理折叠。 (25:00) 该生物安全计划旨在为商业DNA合成供应商提供可验证的密码学溯源依据,以筛除能够绕过传统序列匹配过滤器的AI工程化病原体设计。 (20:00)

Reading summary

Google DeepMind researchers argue that securing generative AI across both digital media and synthetic biology requires embedding mathematical watermarks directly into model outputs rather than relying on fragile metadata or failing passive forensic detectors. Failure of Forensics and Core Watermarking Principles (2:35) According to Google DeepMind's Pushmeet Kohli, passive forensic detection tools became obsolete once generative deep learning models eliminated statistical artifacts and learned to replicate human writing and camera captures indistinguishably. To ensure traceability, DeepMind shifted toward actively embedding statistical signals at generation time. Viable watermarking architectures rely on three core principles: imperceptibility to ensure semantic and visual quality is preserved, robustness so marks survive adversarial edits or compression, and scalability for real-time inference and detection. Modal Technicalities Across Media and Text (10:00) Watermarking approaches diverge sharply depending on the dimensional constraints of the target medium. For dense visual media like images and video, DeepMind trains a generator network against a co-trained detector in an adversarial loop, testing survival against crops, rotations, zooming, and lossy compression. Conversely, text represents a discrete, low-dimensional space where arbitrary substitutions degrade grammatical or factual accuracy. SynthID Text operates by introducing a secret pseudo-random key at inference time that subtly biases sampling probabilities across equally plausible next tokens. Detecting the mark requires observing statistical correlations across longer sequences; prompts with minimal optionality and deterministic answers—such as factual one-word outputs—cannot support a watermark. Furthermore, biosecurity researcher Jeremy Ratcliffe notes that public detector endpoints present an adversarial risk, as bad actors could theoretically query them repeatedly to reverse-engineer key configurations and evade detection. Translating Watermarking to Synthetic Biology (19:30) With generative biology models capable of designing functional proteins and potential pathogens whose sequences evade screening databases of regulated agents, DeepMind adapted SynthID into biological engineering. The team developed two modalities: SynthID Bio Structure, which nudges 3D atomic coordinates without altering the macro-structure, and SynthID Biosequence, which applies token-level sampling biases to sequence-generation tools like ProteinMPNN by substituting chemically similar amino acids. DeepMind physically manufactured synthetic watermarked protein binders in wet labs to test real-world viability. The researchers found that the watermarked binders achieved binding hit rates and affinities near-identical to unwatermarked controls, providing what Ratcliffe describes as the first empirical demonstration of watermarking that couples imperceptibility with complete biological function preservation in vitro. Ecosystem Adoption and Biosecurity Defense (30:00) DeepMind positions SynthID Bio within a wider bioresilience framework spanning prevention, detection, and response across both natural and AI-augmented biological threats. Their theory of change asserts that diminishing the utility of biological weapons and raising operational friction directly deters bad actors. Because commercial DNA synthesis providers acknowledge that conventional sequence filters will struggle against novel AI designs, DeepMind is open-sourcing the technology to integrate verification into commercial gene manufacturing. Kohli explains that signal-level embedding offers far stronger protection than standard metadata tags, which adversaries can easily strip. Under SynthID's framework, biological watermarks are tightly coupled with the molecule's core function via secret cryptographic keys; attempting to scrub the statistical signal damages the folded tertiary structure, rendering the resulting molecule biologically inert. While wet-lab tests confirm that biological function remains intact, operationalizing the technology requires coordinating automated key-exchange protocols and context-length parameters with commercial DNA synthesis providers. Verification across institutions remains contingent on shared keys, meaning non-partner generative models and isolated proprietary systems will continue to evade centralized screening.

精炼阅读摘要

Google DeepMind的研究团队指出,随着生成式模型从多媒体扩展至合成生物学,被动鉴别手段已走向失效;通过在生成过程中直接嵌入不可感知的数学水印(SynthID),能够为数字内容与人工设计的生物大分子建立可靠的来源追溯与安全防御屏障。 被动取证失效与主动水印的三大支柱 (2:35) Hannah Fry与Pushmeet Kohli指出,早期的数字篡改检测主要依赖被动取证工具识别像素层面的微小瑕疵或特定统计异常。然而,随着生成式AI模型在图像、文本与视频拟真度上的大幅提升,模型输出已与真实摄影或人类创作难以区分,被动分类器的准确率随模型进化而显著衰减。DeepMind早在约八年前(约2018年)便启动了水印研发,其核心逻辑是从被动取证转向在生成阶段主动注入不可察觉的统计偏差,从而确立可审计的内容血统(Provenance)。 Kohli归纳了现代水印系统必须满足的三大硬性指标:第一是不可感知性与质量保真(Imperceptibility),即水印绝不能损害媒介的信号质量或语义内涵,否则用户不会采用;第二是抗攻击鲁棒性(Robustness),水印必须在经历缩放、裁剪、滤波或有损压缩等下游编辑甚至恶意擦除时依然存活;第三是可扩展性与易用性(Scalability),水印生成需无缝嵌入推理管线且检测延迟极低。目前欧盟《人工智能法案》(EU AI Act)等法规已要求对生成内容进行标识,NVIDIA等行业伙伴也已采用SynthID相关技术框架。 多模态水印实现机制与检测攻防权衡 (10:00) 水印的技术实现因数据维度差异而截然不同。高分辨率图像拥有百万级像素的连续数值空间,水印可通过对抗性共训练网络(Generator-Detector)嵌入,并经受旋转、加噪与几何变换的考验;文本则属于低维、离散且极易损坏的结构。SynthID Text利用大语言模型在下一个词元(Next-token)采样时的可选性(Optionality),引入伪随机私钥对候选词元的概率分布施加细微偏置。检测端通过长文本链条中的统计相关性识别水印,但若输入是如“法国首都在哪”这类唯一确定性的简短事实,模型没有词元选择空间,便无法嵌入水印。 在安全部署层面,Jeremy Ratcliffe与Kohli强调任何技术都不是万灵药。若将检测器完全开放为公共接口,黑客可能通过频繁黑盒查询实施模型逆向工程,推断出密钥阈值进而规避检测。此外,外部图像的交叉验证依赖合作伙伴的密钥联合(Federation),若第三方模型厂商未共享密钥,中心化平台依然无法识别其内容;而传统的文件元数据又极易被剥离,必须将隐形水印与底层信号强力绑定以防范篡改。 从文本到生物:将水印拓展至蛋白质与DNA (19:30) 随着生成式AI深入生命科学领域,AI已能设计自然界不存在的蛋白质和酶。DNA商业合成制造使生物设计极易在物理世界落地,但也带来了安全漏洞:合成供应商虽然设有危险病原体筛查机制,但针对的是已知病原体序列库;生成式模型完全可以设计出序列排列与已知病毒截然不同、却能折叠成相同致命致病结构的分子,甚至恶意行为者可将片段拆分下单后在湿实验室内拼接组装。 针对该隐患,DeepMind开发了针对预测结构的SynthID Bio Structure(微调原子间距与角度而不破坏大分子构象)以及针对氨基酸链的SynthID Biosequence。Biosequence结合了开源逆向设计工具ProteinMPNN,其机制类似于文本生成:蛋白质由氨基酸线性链折叠而成,模型在保持三维功能构象不变的前提下,利用化学性质相近的“同义”氨基酸替换(例如亮氨酸与异亮氨酸的互换)在生成阶段嵌入不可见水印序列。 湿实验室验证与生态协作构筑生物韧性 (27:30) DeepMind团队通过体外(in vitro)湿实验室制造了两种水印方案下的合成蛋白质结合剂。测试表明,这些带有水印的物理分子在结合命中率和结合亲和力上与未加水印的对照组几乎一致,首次实证了生物水印在保持分子生物学活性与不可感知性上的可行性。Kohli指出,生物水印的数学设计目标是与蛋白质的功能深度绑定,若恶意攻击者试图强行擦除序列中的统计特征,将直接摧毁蛋白质的折叠构象并使其丧失致病或催化功能。 由于DeepMind自身不提供商业DNA打印服务,其定位是作为行业催化剂,将代码开源并推动合成厂商接入校验。基因合成供应商当前筛查工具有限且极度渴望新的防御手段。Ratcliffe指出,DeepMind与Isomorphic Labs共同规划了以“预防、检测、响应”为支柱的生物韧性战略,SynthID Bio作为预防与筛查工具,核心理论是通过大幅拉高作恶门槛来削弱恶意生物武器的实用价值,从而在合成生物学爆发前确立安全屏障。 SynthID将水印技术从数字媒体拓展至物理生命分子的基因编码,为应对AI生成威胁提供了前置性的防御范式;然而该体系的全局有效性,仍高度依赖全球DNA合成产业链与模型开发者之间建立互信的密钥交换机制与统一的安全标准。

Detailed notes — zh

## Background 该内容由 Google DeepMind 于 2026 年 10 月 1 日在播客及网络平台发布。根据页面介绍,数学家 Hannah Fry 教授与 DeepMind 科学副总裁 Pushmeet Kohli 及生物安全研究员 Jeremy Ratcliffe 展开对谈,共同探讨 SynthID 水印技术如何从文本、图像、视频跨越至生物学领域,为 AI 内容提供防伪溯源。 ## TL;DR 随着生成式 AI 的逼真度彻底抹平了传统被动鉴伪的破绽,Google DeepMind 提出了主动注入数学水印的解决方案 SynthID。文本水印利用 token 预测中的选择概率进行微调,图像与视频则依托生成器与检测器的对抗训练来抵御各类破坏。如今,该技术进一步延展至合成生物学,通过在蛋白质结构与氨基酸序列中嵌入不改变分子生物功能的水印,防范恶意生物毒素合成风险,并在湿实验室中成功验证了其结合亲和力,为构建全方位的生物韧性提供了底层防御支柱。 ## Atmosphere & style 对话洋溢着前沿科技探索的理性、严谨与学术兴奋感。Hannah Fry 以风趣敏锐的提问串联全场,将深奥的数学隐写与微观生物学机制拆解得清晰透彻;Pushmeet Kohli 和 Jeremy Ratcliffe 则以扎实的研究细节和坦率的反思作答,既直面猫鼠博弈的残酷现实,又传递出在湿实验成功后的由衷赞叹。整体氛围兼具直面潜在生物危机的警惕感,以及走在技术威胁之前筑牢安全防线的沉稳与远见。 ## Bulletin - `20:00` Google DeepMind 将其 SynthID 水印技术从多媒体扩展至生命科学领域,推出了分别基于 AlphaFold 3 结构微调和氨基酸序列替代的生物水印系统 SynthID Bio。 - `25:00` DeepMind 在湿实验中成功合成了带有水印的蛋白质结合剂实体,实测证明带有水印的分子在击中率与结合亲和力上与未加水印的分子几乎完全一致,首次验证了生物水印的体外功能保留。 - `04:54` 传统基于机器学习分类器的被动 AI 文本与图像鉴伪工具正随模型逼真度的飞跃而逐步失效,主动在生成阶段注入不可感知统计偏置的水印方案已成为应对欧盟《人工智能法案》等监管要求的核心技术路径。 - `33:45` 针对生物安全领域的潜在逆向工程与规避风险,生物水印的防御核心在于将统计信号深度绑定至分子的物理功能上,使得恶意攻击者强行抹除水印的举动将直接导致蛋白质丧失原有生物活性。 ## Key takeaways - `00:00` AI 生成内容的辨识面临双重挑战:除了多媒体深度伪造引发的虚假信息,AI 自主设计全新生物分子也带来了合成生物安全隐患。 - `02:18` 优质技术水印必须同时满足三大核心准则:不损害原始生成质量、具备抗攻击与抗变换的鲁棒性,以及高扩展性和计算易用性。 - `04:54` 被动取证分析随着生成模型消除统计瑕疵而逐渐失效,业界被迫转向在生成过程中主动注入不可感知统计偏置的主动防御机制。 - `10:00` 离散文本与高维图像的水印机理截然不同:文本通过在预测下一个 token 的可选空间内引入密钥调整词语选择概率,而极短事实性语句无法容纳该偏置。 - `15:00` 图像与视频水印依赖生成网络与检测网络的端到端对抗联合训练,可有效抵御裁剪、缩放、压缩和噪声等现实破坏。 - `15:00` 跨平台 AI 检测依赖机构间的密钥共享机制,未开放或共享私有密钥的实验室模型产物无法被中央服务检出。 - `20:00` 现代商业 DNA 合成平台依赖客户背景审查与序列黑名单匹配,但 AI 可设计出功能一致却在序列空间完全相异的有害大分子,从而绕过传统规则筛查。 - `25:00` SynthID Biosequence 利用了蛋白质折叠中相似理化性质氨基酸的可替换性容错空间,在自回归逆向设计生成时注入水印,并经过体外湿实验验证了功能保留。 - `30:00` DeepMind 与 Isomorphic Labs 将生物韧性划分为“预防、检测、响应”三大支柱,旨在通过降低恶意合成的实用性从源头遏制滥用动机。 - `33:45` 为应对去水印的猫鼠博弈,数字与生物水印均致力于将特征与底层信号及物理功能深度绑定,使消除水印的代价为彻底破坏分子功用。 ## Detailed walkthrough ### `00:00` 真实还是合成?AI 生成内容与生物风险的双重挑战 Hannah Fry 教授以日常刷社交媒体的场景拉开序幕:无论是火山近距离慢动作喷发、龙卷风席卷村庄,还是抗议现场的纪实录像,人们心中都会浮现同一个疑问——眼前的一切究竟是真实存在的,还是由机器凭空制造的?随着人工智能在生成文本、图像、音频和视频方面愈发逼真,区分人造与机器生成已成为时代的核心议题。然而,这一问题的复杂程度正在陡增:AI 如今已能够设计出自然界前所未有的生物分子和蛋白质结构,其中部分分子甚至可能被恶意设计用来制造危害。要在危险生物实体被物理合成出来之前将其拦截,解决方案可能就隐藏在生成内容本身——这并非图库照片上粗糙显眼的半透明 Logo,而是一种深奥精妙的数学水印。 围绕这一前沿技术,Google DeepMind 科学副总裁、SynthID 核心架构师之一 Pushmeet Kohli,以及将该技术拓展至生物安全领域的生物安全研究员 Jeremy Ratcliffe 共同探讨了水印的必要性。当被问及为何必须对 AI 生成的生物序列进行溯源标记时,Jeremy Ratcliffe 解释道,生物学应用的核心诉求在于明确来源溯源(Provenance),厘清一段特定序列究竟来自自然演化还是由 AI 模型合成。AI 设计的某些生物序列可能带有极度危险的生物学特异性,但仅凭常规的序列模式往往难以直接识别其潜在危害。 ```mermaid flowchart TD accTitle: AI生成内容的水印溯源与安全防线 accDescr: 展示从AI生成内容到数字溯源与生物风险拦截的技术逻辑。 subgraph Modalities["生成模态"] A["多媒体内容<br/>(文本、图像、音视频)"] B["生物大分子<br/>(蛋白质、酶、DNA序列)"] end subgraph Risks["潜在风险"] C["虚假信息与深度伪造"] D["人工合成有害生物毒素"] end subgraph Solution["SynthID 水印技术"] E["数学级隐形水印注入"] F["来源溯源与检测验证"] end A --> C B --> D C --> E D --> E E --> F ``` - Jeremy Ratcliffe 指出生物水印的核心目标:“So for biology, sort of the motivation for being able to watermarking is so that we can have provenance and say specifically, where did this sequence come from?” --- ### `02:18` 优质水印的三大基石:不可感知、鲁棒性与高可扩展性 针对大众将水印等同于纸币防伪线或护照暗纹的直觉认知,Hannah Fry 探讨了构成优秀技术水印的标准。Jeremy Ratcliffe 强调,水印的首要原则是人类不可感知性(Human Imperceptibility),即绝不能让人轻易辨认出数据的哪一部分被嵌入了标记,否则攻击者可以轻而易举地将其剔除;同时系统必须具备高置信度的识别能力与跨不同数据模态的泛化通用性(Generalizability),避免针对每一种数据结构都单独开发专用水印。 Pushmeet Kohli 回顾了 DeepMind 早在约八年前启动水印项目的初衷,指出其使命在于为用户提供跨模态的真实来源感知。如今 AI 所能生成的文本、音视频乃至人工酶,已彻底达到与自然物或人类产物难辨真伪的水平。为了在不牺牲使用体验的前提下赋予用户这种辨识权,Pushmeet 总结了设计有效水印必须满足的三项黄金法则:第一,不损害原始信号质量;第二,具备强悍的抗攻击与抗转换鲁棒性(Robustness);第三,兼顾易用性与扩展性(Scalability),不仅生成时容易嵌入,后期也极易从信号中提取验证。 - Pushmeet Kohli 对水印的第一准则给出了极其鲜明的定性:“So what do we need from a watermark? We need three properties. One, it should not degrade the quality, otherwise basically the whole point is lost, people will not use it.” - 鲁棒性的关键要求在于:即使攻击者试图抹除水印或对信号施加各种变换扰动,底层的水印信号依然能够顽固残留。 - 可扩展性与易用性确保了这项技术能够在不同的算法架构、服务平台和生产环境中被无缝集成,不会因巨大的计算开销而沦为纸上谈兵。 --- ### `04:54` 被动取证的失效与主动水印技术的崛起 面对“为何直到生成式 AI 普及数年后的今天,水印才成为整个行业焦点”的疑问,Pushmeet Kohli 剖析了技术范式的演进历程。在图像编辑软件初露锋芒的年代,鉴伪主要依赖被动取证分析,专家和专用算法能够捕捉到图像被篡改时遗留的细微像素瑕疵或统计异样。然而,随着生成式模型性能呈现指数级跃升,这些细微瑕疵被彻底抹平。被动检测的失效将业界逼到了十字路口:如果模型生成的图像在客观上与真实相机拍摄无异,人类是否将彻底失去辨析真伪的能力? DeepMind 得出的结论是:唯有从被动检测转向主动防御。既然模型本身已经可以做到毫无破绽,就必须在模型生成内容的过程中,主动注入一种人类不可感知的统计偏置(Bias)。这种偏置既不会改变呈现给用户的表观内容,也不会随模型精度的提高而消散,从而让验证系统能够在事后以数学精度百分之百确定其出自 AI 模型。这项工作历经团队八年深耕,最先从图像领域撕开缺口,因为相比其他媒介,数字图像蕴含着极其丰沛的信息冗余空间,为信息隐写提供了得天独厚的试验场。 - Pushmeet Kohli 回忆鉴伪技术的分水岭时断言:“And as generative AI became more sophisticated, those tiny differences went away.” - 图像隐写的数据规模优势:一张仅 100 万像素(1-megapixel)的数字图像,在编码底层就拥有约 100 万至 300 万个独立的数值通道,为嵌入隐藏信息提供了极其巨大的高维空间。 --- ### `06:47` 模态跨越与行业落地:从多媒体、法律监管到生物序列 攻克高维图像的水印技术只是第一步,真正的硬仗发生在低维离散模态——文本。与图像数以百万计的数值冗余截然不同,文本包含的信号量极小,更改几个词甚至一个标点都可能彻底扭曲句意。因此,DeepMind 团队为文本设计的水印机制采用了与图像、音频和视频截然不同的算法逻辑。随着这一技术的成熟,水印已从单纯的企业研发演变为全球科技立法的硬性指标;例如欧盟《人工智能法案》(EU AI Act)便明确要求所有由 AI 生成的内容必须具备可溯源的标记。如今,不仅 DeepMind 将开源的 SynthID 方案推广开来,NVIDIA、OpenAI 等行业巨头与合作伙伴也纷纷在各自的模型生态中接入了该技术。 更为震撼的技术跨越在于将多媒体水印移植到生命科学中。当 Hannah Fry 探问标记文本与标记生物大分子结构(如 DNA 序列或蛋白质氨基酸链)之间是否存在技术相通性时,Jeremy Ratcliffe 证实了这种底层共性。生物序列本质上也是由离散符号(如碱基或氨基酸)构成的长序列,其逻辑与文本高度契合。生物安全团队无需从零开始构建轮子,而是直接引入了开源的 SynthID Text 代码库,将其无缝嵌入到生物序列生成系统中并完成针对性适配。正如 Hannah 所惊叹的那样,这项技术并未依赖插入隐形空格或标点等粗浅把戏,而是将算法生成本身的数学足迹深刻烙印在了词句与生命序列之中。 - Pushmeet Kohli 对比文本水印与高维数据水印的本质差异:“And the approaches that we developed for watermarking text were quite different and distinct from what we use for watermarking high dimensional signals like images and videos and audio even.” - 行业与法规进展:SynthID 已经被 NVIDIA 以及 OpenAI 等行业领军机构采用,成为应对监管与构建 AI 透明度的通用基石。 - 生物代码复用:Jeremy Ratcliffe 团队直接利用了开源的 SynthID Text 库,通过基础代码平移并进行生物学特性适配,实现了从语言模型到生命密码生成的水印迁移。 ### `10:00` 传统 AI 文本分类器的失效与 SynthID 的文本水印原理 在深入探讨文本水印之前,Hannah Fry 提出了一个现实问题:市面上已经有大量在线工具声称能够检测文本是否由 AI 生成,这与 Google DeepMind 的技术有何本质区别?Pushmeet Kohli 解释道,传统的检测工具依赖于在海量数据上训练的机器学习分类器,试图寻找文本在“真实世界捕获”与“模型生成”之间的统计差异。在早期的大语言模型和对话代理刚出现时,这种分类器确实有效,因为模型存在明显的写作特征——例如典型的“不是 X,而是 Y”(It's not X, it's Y)句式,或者过于频繁地使用“悄悄地”(quietly)这类词汇。然而,随着大语言模型越来越复杂逼真,这些外在破绽迅速减少。Pushmeet 犀利地指出了传统分类方法的死穴:“随着生成式 AI 模型越来越擅长复制其预定目标,分类器的准确率会随时间推移而下降。”(“the classifier's accuracy goes down over time as the generative AI models become better at replicating what they are targeted to achieve.”) 随后,Pushmeet 详细拆解了 SynthID 的文本水印机制。大语言模型在生成内容时,其本质是预测下一个 token 或词语的概率分布。当被问及“Hannah,你今天早餐吃了什么?”时,模型可以以“我……”开头,也可以以称呼“Pushmeet……”开头。当这些选项在概率上同样合理时,模型就拥有了选择权(optionality)。文本水印正是利用了这种选择空间,引入一个后台运行的密钥(secret key),在不破坏语义连贯性的前提下,对候选词的选择概率进行微调。 Hannah Fry 随之总结了这一机制的逻辑:系统并不是简单地重复某个固定词,而是通过密钥在背后引导词语的选择偏好,在整体文本中织入特定规律。Pushmeet 确认,检测器并不是去寻找某个孤立的重复单词,而是检测词语在上下文链条中连环出现的特定模式(patterns)。此外,密钥不仅能证明内容是由 AI 生成的,还能起到区分归属的作用:“不仅能证明这是由 AI 生成的,而且如果不同的 AI 模型使用不同的密钥,我们就可以检查这究竟是由该模型还是由另一个模型生成的,前提是你拥有对应的密钥。”(“Not only this was generated by AI, but if different AI models are using different keys, we can sort of check whether this was generated by this model or this other model, if you have the key.”) ```mermaid flowchart TD accTitle: 文本水印生成与检测机制 accDescr: 展示大语言模型如何利用概率分布和密钥在生成文本中嵌入水印,以及检测器如何提取模式。 subgraph Generation["生成阶段:引入密钥干预"] A["上下文输入 Prompt"] --> B["大语言模型预测下一个词"] B --> C{"存在多个合理候选词<br/>(Optionality)"} C -->|"结合后台密钥 Secret Key"| D["对特定词施加微小偏置 (Bias)"] D --> E["生成带有特定组合模式的文本"] end subgraph Detection["检测阶段:模式比对"] E --> F["检测器利用匹配密钥分析文本"] F --> G["识别词语在链条中的连贯分布规律"] G --> H["判定为 AI 生成并定位模型源头"] end ``` 针对文本水印的局限性,Hannah 提出疑问:如果可选的词语极少,文本是否就无法打上水印?Pushmeet 明确证实了这一点。对于极短的文本片段(例如问答“法国的首都是哪里?是巴黎。”),模型几乎没有用词选择的余地,无法施加概率偏置,因此短文本是无法被打上水印的,这是当前水印技术固有的客观限制。 对于“学生是否再也无法用 AI 在论文中作弊”这一尖锐问题,Jeremy Ratcliffe 澄清了现实生态的复杂性。尽管行业内的前沿实验室正纷纷跟进水印技术,但这并不意味着所有人都会将检测器公开发布。若将检测器完全开放供任意无限次查询,攻击者就能通过对抗性逆向工程,反向推导出绕过特定密钥配置的方法,从而破坏防护机制。 - 传统基于分类器的 AI 文本检测器因模型生成能力的演进而逐渐失效。 - 文本水印的核心在于利用词语预测的“选择性”(optionality),通过密钥微调候选词概率。 - 像事实性极强的极短语句(如“法国首都是巴黎”)无法容纳足够的水印模式,属于不可加水印的文本。 - 公共开放的高频检测接口可能被黑客用于逆向破解密钥规则,因此全行业仍需权衡检测器的开放度。 --- ### `15:00` 图像与视频水印的对抗训练,及其在现实打假中的落地应用 转向图像和视频模态时,水印的技术实现与文本截然不同。文本受限于离散的符号与语义,而图像拥有高维且连续的像素空间,提供了大得多的信息隐藏载体。Pushmeet 解释称,图像水印采用了一种由两个神经网络联合训练(co-trained)的对抗架构: 1. **水印生成网络(Watermark Generator)**:负责接收未加水印的原图,并向其中注入极其微弱、人类肉眼无法察觉(imperceptible)的隐形信号。如果人眼能够看出痕迹,不仅会破坏画质,也失去了水印的隐蔽价值。 2. **对抗模拟环境(Adversarial Agent)**:在生成器与检测器之间,模拟现实世界中可能对图像施加的各类破坏性处理,包括缩放、裁剪、旋转、局部放大、添加噪声以及图像压缩等。 3. **水印检测网络(Detector Neural Network)**:负责在经历对抗网络粗暴修改后的图像中,精准提取并确认最初注入的水印信号。 Pushmeet 总结了这种闭环对抗训练的目的:“它们两者被同步训练,以确保无论中间的对抗者做了什么,水印依然可以被检测出来。”(“And both of them are trained in tandem to make sure that regardless of what the adversary in the middle has done, the watermarks is still detectable.”) ```mermaid flowchart LR accTitle: 图像水印的端到端对抗训练架构 accDescr: 展示未加水印图像经过生成器注水、中间对抗篡改破坏后,检测器仍能识别出信号的过程。 A["原始未加水印图像"] --> B["水印生成神经网络<br/>(注入肉眼不可见信号)"] B --> C["对抗中间体<br/>(模拟缩放/裁剪/旋转/噪点)"] C --> D["水印检测神经网络<br/>(提取隐形信号)"] D -.->|"联合对抗训练反馈 (Co-trained)"| B ``` 在实际应用层面,SynthID 已经在打击假新闻和信息造假的前线发挥作用。Pushmeet 描述了这几年的亲身经历:当他打开常看的新闻网站时,经常看到记者借助 SynthID 揭露热点虚假信息——比如在加利福尼亚大火中,一张宣称某栋房屋“奇迹般完好无损”的热传网络图片,或者伊朗战争局势中流传的某些图像,记者运行 SynthID 检测后证实它们均由 AI 捏造。Hannah 感叹道:“突然之间,你能在那些绝不该出现 AI 的地方发现 AI 的踪影。”(“All of a sudden you can see the AI in places that, I mean, it definitely shouldn't be.”)Pushmeet 也肯定了这一点,指出该工具正被用于在大规模场景下实时遏制虚假信息的蔓延。 谈到普通用户的使用门槛,Pushmeet 透露,该检测能力已经被整合进 Google 的 Gemini 应用程序中。用户现在上传一张图片并询问 Gemini 这是否为 AI 生成,Gemini 就能在后台调用 SynthID 检测器,判断其是否源自 Google 的图像生成模型。此外,其他采纳该技术的外部公司也具备类似的能力。 针对“Gemini 是否只能检测自身生成的内容”的问题,Pushmeet 说明了跨平台检测的规则:内容检测完全取决于密钥的共享。如果不同机构在生成内容时嵌入了各自的水印,只要它们愿意共享对应的密钥,就可以建立一个集中的检测平台。在 SynthID 门户网站上,系统能够调取所有合作伙伴机构托付的密钥库进行联合比对;但如果有特定实验室拒绝共享密钥,系统就无法核实该实验室模型生成的内容。 在厘清了文本、图像与视频之后,Hannah 将话题转向了更具颠覆性的前沿领域——生物学水印。她向 Jeremy Ratcliffe 发问,Google DeepMind 团队近期推出的概念验证系统 SynthID Bio,已经不再局限于传统的视音频媒介,而是开始对 AI 生成的蛋白质序列以及三维生物分子结构进行水印标记。Jeremy 随即证实,即将发表的论文正是聚焦于结构生物学层面的水印标记技术(SynthID Bio Structure)。 - 图像水印依赖生成器与检测器的联合对抗训练(co-training),能够抵御裁剪、旋转、压缩和添加噪点等破坏。 - 新闻机构已利用 SynthID 成功识破加州火灾、中东战局等事件中的深度伪造(deepfake)图像。 - Gemini 应用已内置 SynthID 检测功能,支持终端用户直接检测 AI 生成图像。 - 多模型统一检测依赖于不同 AI 实验室之间的密钥共享;未提供密钥的模型产物无法被中央服务检出。 - SynthID 的应用边界已突破传统多媒体,正式跨越至 AI 蛋白质序列与三维生物分子结构等生命科学领域(SynthID Bio)。 ### `20:00` 生物序列水印与 DNA 合成安全隐患 研究团队目前正在探索两项用于生物学领域的水印技术。第一项技术建立在 AlphaFold 3 之上,主要针对预测的蛋白质结构数据进行水印嵌入;第二项技术名为“合成生物序列”(Synthetic Biosequence),其核心在于直接向蛋白质序列(即氨基酸链本身)中引入水印。尽管折叠后的蛋白质结构本质上也是由氨基酸链构成的,但二者的水印嵌入机制截然不同:在结构层面嵌入水印,是在不改变原子种类的前提下微调原子在三维空间中的坐标、间距或夹角;而在序列层面嵌入水印,则是通过微调特定氨基酸的选择来改变序列内容,但保持其原有的生物学功能不变。 ```mermaid flowchart LR accTitle: 生物数据水印的两条技术路径对比 accDescr: 对比AlphaFold 3结构微调与合成生物序列氨基酸替换两种不同的水印嵌入方式 subgraph Struct["蛋白质结构水印 (基于 AlphaFold 3)"] S1["保持原子种类"] --> S2["微调原子三维坐标与夹角"] end subgraph Seq["合成生物序列水印 (SynthID Biosequence)"] Q1["选择相似氨基酸替换"] --> Q2["改变序列内容但保留功能"] end ``` 引入生物序列水印的一个核心目标在于生物安全溯源。普通大众往往不了解,现实中存在着类似于“DNA 打印机”的商业化 DNA 合成公司,客户只需提交目标序列,公司就能合成出实物。学术界和产业联盟一直在努力强化审查机制,首要担忧在于:恶意行为者是否可能绕过监管,通过合成 DNA 在实验室中复活已灭绝的危险病原体(例如历史上造成大流行的流感病毒)。正如 Hannah Fry 所指出的:“但我的意思是,你在这里说的是,有可能骗过他们,让他们‘打印’出危险的东西。” 当前 DNA 合成公司的安全防护主要依赖两个环节: 1. **身份与资质验证**:核查订购人是否为拥有正规高校或科研机构背景的学者,确认收货地址是否为合规实验室,防止普通个人在地下室私自订货。 2. **数据库筛查**:利用多种比对算法将待合成序列与已知危险病原体数据库进行逐条比对。若发现匹配,则会直接拦截订单,或要求订购方提供使用许可及研究动机说明。 然而,现有防护手段在面对 AI 生成系统时存在显著漏洞。学者们的研究表明,AI 可以设计出在“序列空间”上与病原体数据库完全不同、看起来人畜无害的氨基酸长链,但在折叠后却能形成与危险病原体功能完全一致的蛋白质。攻击者甚至可以将受管制的有害基因片段进行序列变异重构,切片插入看似正常的序列中送去合成,再在实验室里拼接还原。因此,序列水印并非直接用于识别内容是否有害,而是充当一道关键防线:DNA 合成公司一旦在订单中检出可信 AI 系统的水印,就能明确该设计出自内置安全对齐规范的系统,从而与来源未知的野生序列或恶意大模型生成物区分开来。 --- ### `25:00` 湿实验验证与生态协同 在技术实现路径上,研究团队利用了由第三方开发的逆向设计工具 Protein MPNN。与 AlphaFold 从序列预测结构的逻辑相反,Protein MPNN 扮演着“从结构到序列”的角色:科研人员先在三维空间中设计好靶标分子所需的空间构型,再由该模型生成能折叠成该构型的氨基酸序列。这种自回归预测序列的方式,在底层机制上与基于 Transformer 的文本生成模型高度相似。 在蛋白质折叠过程中,某些理化性质高度接近的氨基酸(如亮氨酸 Leucine 与异亮氨酸 Isoleucine)可以在不改变整体空间构型的前提下相互替换。SynthID Biosequence 正是利用了这种容错空间,在预测生成时挑选特定的氨基酸进行替代,将数字水印悄然编织进蛋白质之中。为了验证这种替换是否会对分子的实际生物活性造成破坏,团队不仅在计算机中进行了模拟,还真正在湿实验室(Wet Lab)中合成了实体分子: - **实体测试对象**:物理合成了用于分子靶向结合的蛋白质结合剂(Protein Binders)。 - **核心评估指标**:对比了击中率(Hit Rates,即送测设计中能够成功发生结合的比例)以及结合亲和力的定量数据。 - **实验表现**:无水印分子与采用两种不同水印方案的分子之间,各项测试表现“近乎完全一致”,并未因水印引入而牺牲结合效能。 针对湿实验的震撼结果,团队成员直言在拿到数据前曾抱有怀疑,最终的成功出乎所有人的预料:“老实说,在拿到湿实验数据之前,我对这套方案到底能有多好用是抱有一点怀疑态度的。我认为实验结果彻底震撼了我们所有人,它竟然如此顺畅。” 目前该项目处于概念验证阶段,团队正准备将核心代码开源。要将生物水印转化为行业通行的防御屏障,需要整个生态系统的紧密协作:AI 模型开发者需要在生成流程中集成水印逻辑;DNA 合成厂商则需要在审查端配合管理密钥和上下文长度(Context Length)等参数。由于 Google 自身并不直接运营对外承接订单的蛋白质设计商业平台,团队正积极与上下游生态伙伴展开密集沟通,推动开放标准与跨机构协同落地。 ### `30:00` 基因合成业界的反应与 DeepMind 的生物韧性战略 在探讨如何将生物水印技术推向实际落地时,Jeremy Ratcliffe 阐明了团队的定位:Google DeepMind 自身并不接受蛋白质打印订单,因此他们的目标并非单方面制定行业规范,而是充当催化剂,推动这一技术融入现有的基因合成筛选(synthesis screening)服务中。当被问及真正负责“打印”和合成实体蛋白质的行业从业者作何反应时,Jeremy Ratcliffe 笑着承认自己性格中带着天然的怀疑态度,原以为对方可能会有所保留,但现实却是整个行业展现出了超乎预期的兴奋与接纳,这极大鼓舞了团队推进落地步骤的信心。Pushmeet Kohli 补充指出,生物学界的敏锐度令人印象深刻,他们不仅时刻警惕当下的生物风险,更对未来的潜在威胁有着极强的危机意识。商业基因合成机构每天都会收到大量可能用于生成危险蛋白质的订单,现有的筛选过滤器虽然能拦截可疑请求,但业内十分清楚现有规则并不完美,亟需新的技术方案共同筑牢防线。 这种合作契合了 Google DeepMind 与 Isomorphic Labs 联合制定的生物韧性(Bioresilience)愿景。大约两个月前公布的一份战略备忘录中,他们将生物安全防御体系划分为植根于经典安全策略的三大支柱:预防(Prevent)、检测(Detect)和响应(Respond)。团队的研究横跨自然生物威胁与 AI 增强威胁,其中 SynthID Bio 侧重于针对 AI 辅助设计的蛋白质进行病原体检测与溯源,另一些项目则针对天然传染病爆发。正如 Jeremy Ratcliffe 在解释防御哲学时所言:“So sort of our theory of change is that if you make biology less useful for someone who's trying to use something malicious, then that in and of itself can prevent them from using it in the first case.”(我们的变革理论在于,如果你能让生物学对于试图恶意利用它的人变得不再那么有用,这本身就能在源头上遏制他们发起攻击的意图。)通过开发提高公共卫生韧性、协助流行病学家快速识别疫情的 AI 工具,技术可以在危害扩散前化解危机。 Pushmeet Kohli 强调,AI 正在为人类健康与基础生物学理解带来不可估量的巨大红利,Isomorphic Labs 等机构利用 AI 进行药物研发就是最好的例证。但与此同时,新技术不可避免地会被恶意用例盯上。为了确保 AI 的造福效应不被破坏,必须同步部署严密的技术控制手段,让恶意利用的门槛变得极高甚至在技术上完全不可行。 - **三大防御支柱**:预防(Prevent)、检测(Detect)、响应(Respond),全方位应对自然与 AI 衍生的生物风险。 - **筛选现状与痛点**:基因合成机构日常依赖过滤器筛选有害序列,但传统规则存在盲区,亟待与前沿 AI 溯源技术接轨。 - **核心理念**:提高公共卫生系统的应对与溯源能力,降低恶意合成的成功率与实用价值,从而在源头消除滥用动机。 ```mermaid flowchart LR accTitle: 生物韧性防御体系与水印溯源流程 accDescr: 展示从AI生成序列到实体合成筛选的安全防御链路 subgraph DefensePillars ["生物韧性三大支柱"] direction TB P["预防 Prevent"] D["检测 Detect"] R["响应 Respond"] end subgraph Pipeline ["序列生成与合成验证"] direction TB G["AI 生成蛋白质序列"] --> E["SynthID 嵌入不可见水印"] E --> S["提交至基因合成平台"] S --> C{"筛选与解密核验"} C -->|"合法序列"| Pass["许可合成实体蛋白"] C -->|"可疑/恶意序列"| Block["拦截并上报威胁"] end DefensePillars -.-> Pipeline ``` --- ### `33:45` 数字媒体溯源与生物水印的“猫鼠博弈” 谈话转向更广泛的数字内容治理时,Hannah Fry 提出了一个现实困惑:当前数字媒体的水印与溯源标准相当割裂,未来是否会出现一个大一统的中心化平台,让人一键查询某项内容是否由 AI 生成。Pushmeet Kohli 表示,整个产业正在朝这个方向靠拢,行业内部已经在内容来源标签上形成了共识,即在生成或编辑文件时附带数字凭证。然而,单纯依赖元数据(metadata)的致命缺陷在于信息极易被恶意剥离,因此必须依靠水印技术将标识与信号本身进行强绑定。 面对生物学领域是否也会建立统一行业标准的追问,Hannah Fry 提出了尖锐的质疑:一旦标准普遍推行、规则彻底公开,这是否会演变成一场永无止境的“猫鼠博弈”?攻击者会不会针对已知规则开发专门抹除水印的逆向系统?Pushmeet Kohli 坦言这种对抗风险确实存在,而破解之道的关键在于密码学密钥(secret keys)与信号的深度嵌入。水印必须深入到蛋白质序列或像素信号的最底层,使得任何试图消除水印的行为都不可避免地破坏其物理功能。Pushmeet Kohli 阐明道:“Like the ideal situation will be hard to take away to an extent that if you want to get rid of the detection signal, that you have to change the function, right, and destroy the original sort of capability of the task that you had designed the thing for.”(最理想的状态是水印难以剥离到这种程度:如果有人想要彻底消除检测信号,就必须改变它的功能,进而破坏当初设计该物体时所赋予的原始功用。) Jeremy Ratcliffe 补充说,当前这项技术仍处于早期探索阶段,计算 G 值(G-values)和嵌入水印的方式仍有巨大的工程优化空间。但他自豪地指出,虽然 DeepMind 并非业内首个提出生物水印概念的团队,但他们是首个在实现水印不可感知性(imperceptibility)的同时,完整保留生物活性与功能,并在体外(in vitro)湿实验中成功验证这一成果的团队。这种让微观分子自带来源身世的技术,展现了工程与生物学交汇的惊人力量。 - **元数据与水印的结合**:元数据易被剥离,而数字/生物水印直接修改底层信号,二者结合才能实现不可分割的溯源。 - **抗篡改核心机制**:通过私有密钥对序列微调,使水印与蛋白质的空间折叠或催化活性强行绑定,强行抹除水印即意味着让蛋白质丧失功能。 - **里程碑成果**:在实验室体外湿实验中实证了不可感知性与功能保留的双重达标。 --- ### `36:50` 走在威胁前面:赋予未来数字与生物“真伪证明” 在节目尾声,Hannah Fry 对整场对话进行了总结与高度评价。她感慨道,如今科技界频繁将“安全护栏”(guardrails)与“安全性”(safety)挂在嘴边,但 Google DeepMind 的研究是实验室真正“言出必行”(putting its money where its mouth is)的切实验证。早在绝大多数实验室尚未意识到相关风险之前,这里的团队就已率先开创并验证了 AI 内容水印技术,证明了这项技术不仅具备大规模扩展的能力,而且不会牺牲生成质量与处理速度。如今,这些技术探索不仅被写入了多国监管政策的框架之中,团队更未止步于既有荣誉。 Hannah Fry 强调,DeepMind 正在将目光投向更深远的地平线,未雨绸缪地应对下一代威胁,在风险全面失控之前建立免疫防线。无论是合成文本、图像、视频,还是直抵生命底层的蛋白质分子,AI 时代的技术演进正在为万事万物打上不可磨灭的印记。她总结道,未来的世界必将伴随着这样一份可靠的“真实性证书”(certificate of authenticity)。随着致谢与道别,本期关于从深度伪造到 DNA 水印的科学探索圆满落下帷幕。 - **监管与技术落地**:DeepMind 的早期水印探索已从实验室算法走向行业共识,并被实质性纳入全球 AI 安全治理监管要求。 - **前瞻性防御哲学**:在 AI 赋能的生物风险与深伪威胁全面具象化之前,提前构筑具备高韧性的技术防护层。 ## Notable quotes > `10:00` the classifier's accuracy goes down over time as the generative AI models become better at replicating what they are targeted to achieve. — Pushmeet Kohli > `30:00` So sort of our theory of change is that if you make biology less useful for someone who's trying to use something malicious, then that in and of itself can prevent them from using it in the first case. — Jeremy Ratcliffe > `00:00` So for biology, sort of the motivation for being able to watermarking is so that we can have provenance and say specifically, where did this sequence come from? — Jeremy Ratcliffe ## Mentioned Hannah Fry, Pushmeet Kohli, Jeremy Ratcliffe, Google DeepMind, SynthID, SynthID Text, SynthID Bio, SynthID Bio Structure, AlphaFold 3, Protein MPNN, Gemini, Isomorphic Labs, OpenAI, NVIDIA, EU AI Act ## Video description How do you know if what you're seeing is real? Professor Hannah Fry sits down with Pushmeet Kohli (VP of Science) and Jeremy Ratcliffe (RS) to break down SynthID, the watermarking technology expanding from text, images, and video straight into biology. [Source](https://afp-920373-injected.calisto.simplecastaudio.com/36cf10bf-f6dd-4a98-96a9-8eda9f59db4f/episodes/83783c09-9840-410b-8653-84a9a446ea98/audio/128/default.mp3?aid=rss_feed&awCollectionId=36cf10bf-f6dd-4a98-96a9-8eda9f59db4f&awEpisodeId=83783c09-9840-410b-8653-84a9a446ea98&feed=JT6pbPkg)

Source quotes

[10:00] Pushmeet Kohli: the classifier's accuracy goes down over time as the generative AI models become better at replicating what they are targeted to achieve. [30:00] Jeremy Ratcliffe: So sort of our theory of change is that if you make biology less useful for someone who's trying to use something malicious, then that in and of itself can prevent them from using it in the first case. [0:00] Jeremy Ratcliffe: So for biology, sort of the motivation for being able to watermarking is so that we can have provenance and say specifically, where did this sequence come from?

Source quotes

[36:30] Hannah Fry: The future, as it turns out, will come with the certificate of authenticity. [4:47] Pushmeet Kohli: The way to sort of make sure that humans always have the ability to understand the origin of where some signal came from is by injecting a signal within it. [32:00] Jeremy Ratcliffe: if you make biology less useful for someone who's trying to use something malicious, then that in and of itself can prevent them from using it in the first case. [30:00] Pushmeet Kohli: one thing I found very remarkable about the bio community is how cognizant they are of not just the threats today, but the threats of tomorrow.

Original transcript

[0:00] Welcome back to Google DeepMind, the podcast. I'm Professor Hannah Fry. Picture the scene. You are scrolling through your socials and you see some remarkable footage, the up close slow motion eruption of a volcano or a tornado raging through a village or perhaps some footage of a protest. I think many of us ask the same question. Is that real? Was this made by human or a machine? Well, as AI gets better at generating texts and images and audio and video, that is becoming one of our time, except that now the answer just got a lot more complicated. We are entering a world where AI can design biological molecules and structures that have never existed in nature before, some of which could be designed to cause harm. So how do you stop something dangerous before it gets made? Well, the answer, it turns out, might be hidden in plain sight, embedded synthesize. A watermark, not the blocky logo you see on a stock photo, something far more subtle, something mathematical. And today I am joined by two people who are working on this from different angles. Pushme Coley is VP of Science at Google DeepMind and one of the architects of SynthID, the watermarking system now being adopted across the AI industry. And Jeremy Radcliffe is a biosecurity researcher whose team has applied that same technology to biology. Welcome to the podcast, Jeremy, let's start with you. Watermarking anything that AI spits out, I mean, it sounds like a very good idea, but just break it down for me. Why is this helpful? So for biology, sort of the motivation for being able to watermarking is so that we can have provenance and say specifically, where did this sequence come from? And really what we're trying to differentiate is between sequences that come from things in nature versus sequences that come as products of AI systems. that can have properties that we know to be problematic, but we might not be able to identify from the sequence itself that it has these problematic properties. And what makes a good watermark? Because I think when people consider the phrase, they sort of think of, I don't know, like banknotes or like passports. Is there like rules for what counts as a good watermark? I think for us, in particular, human imperceptibility is one. So making it very difficult for someone to be able to tell what portion of data has been watermarked. Because otherwise it would be really easy to erase. So having some system that can go through and actually identify the watermark with high confidence, and then also good generalizability. So being able to have at least a watermarking method that can work across a bunch of different systems rather than having to make bespoke ones for every single data type. What do you think, Prashmi? I mean, did that apply beyond biology and more broadly? Yeah, I think the main reason why we started the whole watermarking project at DeepMind almost eight years ago now was... to give a sense of provenance to what users are seeing. If you look at different modalities, whether it's text, whether it's images, whether it's videos... It was hard to imagine 10 years back, that we would today be talking about systems that can create images, videos, audio, text, and proteins or enzymes that are indistinguishable from what a human or what exists in nature. And it's an important sort of element for users to give users this idea that you can really believe what you are seeing. So what do we need from a watermark? We need three properties. One, it should not degrade the quality, otherwise basically the whole point is lost, people will not use it. The second element is robustness against attack or against transformations. If people want to remove it, it's hard to remove in the sense that even if you make changes to the signal, it will persist. And then the third requirement is that it should be easy to use. It should be easy to integrate in the signal and it's easy to detect from the signal. So those are the three sort of properties of imperceptibility, robustness, and scalability that we have designed all our watermarking systems for with the whole idea that we're going to be able to do that. users have more control and have a better view of what they are seeing. or what they are sort of using or designing and so on. It feels like this subject of watermarking has been quite a big deal this summer. But at the same time, we're a few years into generative AI being accessible to the public. What's taken so long, Prashmi? Why is it only now that this has become so widespread? When image editing software started becoming very sophisticated, people were sort of very concerned about this issue of, [5:00] Is this a real image or has this been tampered with? And there was a lot of work and specialized systems which could really sort of analyze an image and think and see the tiny differences that exist in a tampered image and can say, oh yeah, this was a fake image or this has been tampered with and so on. But as generative AI became more sophisticated, those tiny differences went away. And then that naturally sort of resulted in the question, is this game over? In the sense that humans will not have the ability to detect whether something is re-generated. And this is where watermarking sort of came in. the way to sort of make sure that humans always have the ability to to understand the origin of where some signal came from is by injecting a signal within it. having a bias so it doesn't go away, even if the models are perfect and they have the ability to sort of generate images which are indistinguishable from what a camera would observe or what a text that a human would write, we specifically use. sort of put in some imperceptible things that can allow users to later detect. that this originated from my AI model. And when you say we here, you really do mean we in the literal sense, right? It was you guys that came up with a solution for this one that was scalable. Yes. So our team has been working on this for almost eight years. We started with images because people were concerned about these images being used for fake news, misinformation, and so on. And so there was that motivation to solve that problem. At the same time, Although the problem is extremely challenging, because of the properties we just mentioned of robustness and imperceptibility and efficiency and scalability. Still, an image is a very large amount of data. a one-megapixel image has sort of one million sort of numbers, or like three million numbers, depending on how it's encoded and so on, to hide information, right? And without changing the content. So we've developed... watermarking systems for images first. Later on, we had to look at the challenge of how do you do this for other modalities like text, which are not as high dimensional as images, where actually it's a very small amount of signal. be very cautious in terms of not changing the meaning of the content. And so the approaches that we developed for watermarking text were quite different and distinct from what we use for watermarking high dimensional signals like images and videos and audio even. But this idea of watermarking, which you guys pioneered, has now been brought into law. It's essential that any generative content within the EU has to have some sort of watermarking. I think what... regulators and countries around the world have recognized that they want to give you users this information as to the origins of the content that they are consuming. And one sort of key technology that is seen as a robust solution for it that can scale is watermarking, which is why not only have we seen a take-up of this idea across the industry, the fact that SynthiD is used by NVIDIA, is used by other partners like with OpenAI and so on, it's a remarkable sort of thing that many of our partners in the generative AI space have adopted a this technology. And Jeremy, in this episode, we're going to be talking about two different applications of watermarking for AI. So you've got text and video and audio and so on. And then you've also got for biological structures and sequences. Yeah. Is the solution quite similar between the two? Yes, I'd say they're motivated by similar principles. And we were able to adopt a lot of the work that had already been done, including using the open source Syntheti Text Library and just embedding that directly into another system. Right. Not just the same tricks, but sometimes the actual same code. You can sort of pick it up and drop it. At least start from the same code and then make some adaptations on top of it. Okay. Okay. So I think people want to sort of understand how this works because it sounds almost magical that you could have a single sentence and you have embedded some sort of watermark in there that's nothing to do with white space, it's nothing to do with putting commas in, hiding them in the text, but that the words themselves are hiding a record of how they were generated and where they came from. So, if it's okay with you, Pishmi, I thought you were going to be a good one. [10:00] We might just walk through how this works for text. I think that's probably the easiest one to understand. Okay, so before we get to that, there is some software out there already that you can get online that claims to be able to tell whether the text that you have is generated by AI or not. How is that different from what you guys have created? was captured in the natural world. And this sort of involves basically training this machine learning models over a very large amount of data, right? When the earlier generations of large language models and these conversational agents came about, you could sort of make out that, oh, this is a large language model. It has certain tells. It's not X, it's Y. It uses the word quietly a bit too much. Yeah, exactly. But as these models have become more and more sophisticated, those tells have become less frequent. And so if you have this machine learning classifier which is trying to distinguish, the classifier's accuracy goes down over time as the generative AI models become better at replicating what they are targeted to achieve. Yeah. Okay, well, let's walk through how the watermarking system works. So let's do it sort of step by step. So first off, if you could... Maybe we should start off by how large language models pick the next word in a sentence when it's generating text. Yes. So when we are watermarking text, The idea is that these tokens or words are being generated from a large flagpole model. And the large language model essentially has a distribution over which tokens it can, or which words it can generate. So if I ask you the question, "Hannah, what did you have for breakfast today?" You can start with the word I... Or you could start with the word push me, addressing me. And these are the options that you have. And if they're equally valid starting points, you have optionality. And this is where text watermarking leverages this optionality by seeing what What does the key... Tell us. Okay, but that's the point, right, is that sometimes when you have an option for a word, behind the scenes there is a secret key which says bias these words over others. And then when you look back at all of the text together, if you can see those words appearing over and over and over again, you can be confident that this is AI generated. So rather than the words, it's the patterns of those words. So those words specifically, you don't need to be repeated, but the patterns is what the detector sort of leverages. How they appear in sort of chains together. Right. Yes. OK. So you're you're secretly right. Talking about it on a podcast, we're going to say. But you have some code running around in the background, some secret key. Yes. That you can then look at a chunk of text later and say, hang on a second, this looks like it was generated by AI. Not only this was generated by AI, but if different AI models are using different keys, we can sort of check whether this was generated by this model or this other model. model, if you have the key. Right, if you have the key. Okay, I see. Okay, so, but if there's few words that it could be, then you can't use it. So does that mean that some text is unwatermarkable? Yes, so if you have a very a small, a short text snippet, right? Like, what is the capital of France? it is Paris. So there is very little that we can change there. And so that will not be watermarked. That is a limitation of the watermarking approach in this case. Does all of this mean then that that students can't cheat on their essays anymore, bluntly. Are we now able to just detect anything that's AI generated? I think the whole ecosystem as a whole is moving towards watermarking, as you've seen with some of the other labs making their announcements. But that doesn't necessarily mean that they're all going to make their detectors publicly available. And even if you've got a publicly available detector, that can be motivation for people learning how to get around it. In the same way that some of these machine learning classifying-based detection systems are using a whole bunch of examples of text of things that have been watermarked. Were a detector publicly available and you could just query it over and over and over again, you might be able to work back to making something that gets around that specific configuration of keys that is presented in the public detector. [15:00] Got you. So we're not at sort of guaranteed cast iron way to detect any AI generation whatsoever. It's just we're this is a big step towards that. At least not with the method of having a publicly available detection system for every version of keys that is made publicly available. Just in terms of images, then, and video, is it the same idea? You have a lot more space to hide things as you describe. Yes. So I think in images, the approach is different from the text watermarking approach that I was mentioning. In images, what happens is... We... take the image, which has not been watermarked, And then there is a a neural network which looks at that image and modifies it in a very subtle way. to incorporate the signal. And this particular model, which is incorporating the watermark, is being co-trained by with a detector neural network And they're both sort of trying to achieve the following. You get an image which is on Watermark. The Watermark generator injects a very small signal, which is imperceptible. So if it is perceptible, we say that's not great. Like users will see it and that degrades the quality of the image. Then in the middle, there is an adversarial agent which is going to try to make modifications to the image. might try to shrink the image, zoom in, crop it, rotate it, do many kinds of transformations, like add noise and so on. And then the detector, has to still detect whether the original image that is coming through is watermarked or not. - Okay. - And both of them are trained in tandem. to make sure that regardless of what the adversary in the middle has done, the watermarks is still detectable. What does this mean for users then? I think the main thing is it's not just about text. It's about all these different modalities. And if you look at how people are using SynthID, like it was really amazing to, The last few years, as SynthiD has been launched, I would open my favorite news website and the reporter would say, well, here's an image that was circulating in the media about this particular house that magically was untouched in the California area. California fires or this image that came out. from the Iran war or some other sort of topic. And we have run SynthID on it and detected that this was AI generated. Right. So all of a sudden, all of a sudden you can see the AI in places that, I mean, it definitely shouldn't be. Yes. And SynthID being used to actually... tackle misinformation in real time and at scale. So then can anyone... do what you just did there. Can anyone take an image, some text and video, run it through SynthAD and find out whether it was generated by an AI model? We have made the detection ability embedded in the Gemini app, for instance. So today, if you can go to Gemini, if you ask... I'm looking at this image, tell me if it is air-generated, it can run the SynthID detector. and say, yes, this is something that was generated by a Google image model and so on. And similar sort of... abilities now are available to by other companies as well who have adopted our models. Can each model only detect whether it was generated by that model? Can Gemini only detect Gemini-generated content? So people who are injecting the watermark, into the content, they have to agree to share the key. Right? If they share the keys, then we have the ability to detect it. Right? So there can be a central service. If you go to the Synthiity portal, and if you ask, something is generated by AI generated, it can look at all the keys that have been given to us by partner institutions and we can detect their content for all those partners as well. Okay, got you. But if a certain lab is like doesn't want to share their key, then you wouldn't be able to? Yeah. Okay, well, Jeremy, let's talk about biology now, because I know that you've come up with this proof of concept system called SynthID Bio, which instead of watermarking, you know, text and audio and video is for biology instead, where you are watermarking AI generated protein sequences and 3D biomolecular structures. Just tell me about those two systems. Yeah. So the release in the upcoming publication covers synthetic biostructure, which is specifically looking at watermarking. [20:00] predicted protein structure is given a sequence. So it builds on top of alpha fold three. The second technology is synthetic biosequence, which looks at introducing watermarks into protein sequences themselves. So the strings of amino acids that are present for a sequence. So, but those two things are related, right? That you have like, I mean, each folded protein structure is a string of amino acids. Yeah. So we're basically talking about different versions, maybe of the same type of data, but actually embedding the watermark is we're actually changing the positions of individual atoms. And so you can think about maybe the distance between two atoms changing slightly or the angle between two atoms changing slightly. Well, for synthetic biosequence, we're actually changing the selection of an individual amino acid. So for structure, it's still the same atoms. They're just maybe in a slightly different configuration. Well, for sequence, we're actually changing the content of the body. but maybe not the actual functional meaning of that content. And what's the point? Why do you want it? Yeah, you know, we've thought of a couple of different use cases that might be useful being able to say that you have provenance of something being AI generated and how you might be able to detect if an order going to a DNA synthesis company is something that came from an AI system. Because I think that's something that non-biologists aren't really aware of, right, that you essentially have. I'm going to call them printers, but where you can send off and say, this is the sequence I want created and they'll do it for you. But I mean, what you're saying here is that it's possible to trick them into printing something hazardous. I'm using printed very loosely here. That's essentially the problem, right? That's the concern. Yeah, that's one of the concerns. And there's a large body of industry groups and academics who focus on how to make the screening regimen more robust. And that's something that we're interested in being able to contribute to. But the primary concern is, would someone be able to acquire DNA that could give them access to being able to generate something in a lab that we don't want them to be able to do? Or maybe it gets around regulation. So could someone order DNA from a company and then bring it to a laboratory and make something like an extinct pandemic, like instinct influenza, for instance? Okay, which I would say would be bad. Generally, yes. Overall bad. extra safeguard in place, an extra sort of stop in place before they just can continue on. Yeah, and something that if you detect the watermark, you have confidence that this was derived from a system that has built-in safety guardrails versus either maybe another AI model or something that comes from nature. Okay, and what kind of safety systems do those companies have in place at the moment to make sure that they're not making something really dangerous? So the synthesis companies, largely what they do is they take an order from an individual where they've been able to verify something about their identity. So most of them have a process in place I'm an academic. This is my university affiliation. I am actually who I say I am. And the specific P.O. box that you're sending it to is for a laboratory. I'm not just a random person in a basement that wants it. Joe blocks. Yeah, right. Okay. And then they take those orders and they compare the specific content of the order to a database of known hazardous things. And they check that specific submission against that database using a couple different algorithms. And if something matches, then they either just block the order and say, we can't synthesize this, or they'll go back to the original individual and try to find out what motivation they have for ordering it and whether they've got, say, requisite licensing. And the risk we have with these AI systems is, as shown by some other academics, um, is that you can make something that in sequence space is very different than something that's present in that pathogen database, but might fold into the thing that is a concern. So the string of amino acids looks innocent. Yeah. Yeah. but once it's created a natural protein, could be really harmful. Yeah, and it could have the same behavior as something that's present in the database of restricted orders. And their current systems wouldn't be able to catch it. Definitely the concern is that, and especially as these systems get better and better, you might be able to get something even further away in sequence space that might get around some of the existing protections. Something really wacky in this ribbon of amino acids that still can fold into something harmful. To basically sneak under their detection system. Exactly. And you can cut and paste that into another sequence. So you can imagine like if maybe there's one portion of an order that would be blocked, you could cut and paste something of concern into that that's very far away from the restricted database. And then in the laboratory yourself, you just put it back into where it originally came from. Okay, I see. And then the idea of what you're doing here with the watermarketing is not necessarily to say you're going to be able to detect anything that's potentially harmful, this is AI generated, be careful, because when you're comparing it against your existing database, it might not look like something that's already on there. And you can say which system it came from. And because you, as a DNA synthesis company, trust this AI system for reasons that go beyond the watermark, you can have confidence that this was generated through this method. Talk me through the protein design. process then here. So how do you get the watermark into the system in the same way as we've [25:00] with text and images? Yeah, so it actually, maybe somewhat fortuitous that the approach that Protein MPNN, which is this structure to sequence model uses, is actually pretty analogous to the way that transformer-based models generate text. So Protein MPNN is a third-party tool. It's not something that Google developed. It's been released out in the world for a number of years, and it's almost the opposite of AlphaFold. So rather than going from a sequence to a structure, it goes from a structure to a sequence. And that's an important part in a protein design pipeline, where you are originally choosing your target based off of how you want them to interact in three-dimensional space. This is a model that just takes that representation and then creates a predicted sequence in order to fold into that specific configuration. Because that's an important point, right? That you might have these extremely complex folded structures, but that each protein is... is constructed from like a ribbon of amino acids, a string of amino acids. And that's where the analogy comes from, a string of amino acids like a string of text. And so what you almost see is that when you're folding a protein, there are locations where there can be swaps between individual amino acids, maybe ones that are similar in certain properties, like leucine and isoleucine are two that are quite similar. And so what we're taking advantage of within SynthID Biosequence is that you can introduce watermarks through selections of individual amino acids that are likely to have both maintain the same structure and therefore have similar function. But hang on, how can you be sure? I mean, you said that you can make the swap and it doesn't matter, but can you be sure that the swap doesn't matter? Yeah, so that's what we specifically test in the process. So the whole idea of this research was to inject the watermark and ensure that the generated sequence, amino acid sequence in this case, that builds the protein, has the same function or retains the function that we were trying to design 4. And that's what is very cool about it. that we have now in existence watermarked proteins. These are real things. This is not in simulation. These are real things which do stuff. but which are watermarked and will always be detectable. And importantly, from our own laboratory experiments, they don't change the function at all. So we made protein binders, so we made the things that stick. Oh, you physically made them? Yes. We tested them in a laboratory. And some of the measurements you can get from making protein binders are things like hit rates. So of the number of designs that you send in the lab, how many of them actually bind. You can look for how well something binds. And across both those measures, things that were watermarked and watermarked under two different schemes versus protein binders that weren't watermarked had near identical hit rates, near identical quantitative measures of binding. Because you'd expect to lose maybe a tiny bit, right? Right. Potentially. I mean, I guess, admittedly, I think before we got the wet lab data, I was a little bit skeptical about how well this is going to work. And I think it blew all of our minds, actually, just how easy it was. Proving it in biology was, like, I think something that took us all maybe a little bit by surprise. I know this is proof of concept at the moment. You're sort of demonstrating that it works and that it's possible. Is the idea that in the future, any lab would be able to read your watermark? Yeah, and I think that's the eventual goal. Or maybe I wouldn't even necessarily frame it as our watermark. Maybe it's the technology other people can use. We know that there has to be a number of different players who would be part of actually operationalizing this. So model developers, people making either novel tools or systems to be able to generate new proteins, they're going to have to be sort of on board of being able to integrate watermarking within their own schemes. And because we're open sourcing the code, this is something that they can do without even necessarily needing to coordinate with us. And on the synthesis side, there'll have to be some ability to exchange keys and exchange other details about the watermarking between the model providers and the DNA synthesis company themselves. And so these are a couple of different coordinations that need to happen. From our side, we know that there is additional engineering headroom that you can move into to make the watermark both detectable and potentially a little bit easier to embed. We know that from some of the ongoing work. to adopt from all the different players who were involved. Is it the same story with the text? Would this be able to detect anything universally, or is it specific to the particular key that's used? So if someone were to take the implementation that we're making publicly available and integrate it in their system, so long as they're able to share the key as well as the length of context, that's another area that you can change how much of the preceding context matters for watermarking. So long as you can share those two values, you'd be able to detect them coming out What stage are you at right now? So for us, what we've been able to do within this publication is more or less have a proof of concept that we're going to open source the technology that allows people to do the watermarking. So this is a slightly different one than the way that we can watermark content coming out of Google, as we don't have necessarily a protein design service internally. So what we've started to do is have a lot of conversations with other people in the field and sort of in the ecosystem, both on the model development side, as well as on [30:00] synthesis screening side, and we want to sort of just catalyze a conversation about where we can take this technology and actually do the implementation. It's also not necessarily for us to be able to set the standards on that, partially since we're not the organization that people will be ordering proteins from, but at least being sort of the catalyst so that this can get implemented from the various services that are currently out in the world. That's sort of where we see our role at the moment. And what's their reaction when you talk to the people who are the ones that are sort of printing the proteins, making the proteins for real? What's their reaction when you talk to them? Yeah, you know, I think maybe this is just a reflection of who I am. I'm generally quite skeptical of things. Like, that's just sort of in my nature. They've all been so excited, maybe even more than I think we might have anticipated, which has been, I think, really galvanizing for us as we continue to go through sort of the last few steps around what it means to take this out into the real world. on the amount of at least interest that we might be able to get. So, I mean, one thing I found very remarkable about... the bio community is how cognizant they are of not just the threats today, but the threats of tomorrow. And so they have been extremely receptive of the technology because they see the challenges there. every day. that they are getting these orders. for gene synthesis, which can create proteins, and they have these filters that can catch problematic orders, but they know that those are not imperfect. And more work needs to be done to develop them. And they want to work with... the field and their community to find solutions that will work. Because this topic of bioresilience as we go forward is going to be, I mean, it's really going to start to become more and more on people's minds. I mean, what are the other things that you're doing in this area? There was a memo that came out, I think it was two months ago or so, that sort of laid out Google DeepMind's vision as a whole, plus isomorphic labs of what we're thinking about bioresilience. And that matched a couple different efforts across pillars that are actually rooted in security strategy, so on prevent, detect, and respond. Our team has a couple different efforts under that, and broadly, we're looking at how we can leverage AI to reduce biological risk. It's focused both on natural threats and on sort of AI-enhanced augmented threats. We've got some projects that map really specifically to one area, so I think this Syntheti Bio is a good example where we're really thinking about AI-enhanced AI designs and what that might mean for being able to do pathogen detection. We've got some other concepts that are much more in the natural threat space, and then some that sort of map between the both. So sort of our theory of change is that if you make biology less useful for someone who's trying to use something malicious, then that in and of itself can prevent them from using it in the first case. So for us, if we're able to make new capabilities and tools that make our entire public health system more robust, or make it easier for epidemiologists to identify an outbreak, that can in and of itself make an outbreak less likely. Because you want the good side of this, right? You want like the benefits of it, but you don't want it to come along with all of the potential downsides. Absolutely. Absolutely. And I think AI... will have immense... positive impacts. in human health and understanding of biology at large. I mean, it's already happening around us, right? Isomorphic Labs and many other sort of companies who are leveraging AI for drug discovery, understanding human health and so on. But at the same time, there will be problematic use cases that will be enabled. And we need all the controls and all the sort of measures that we can place to make sure that those tasks are difficult or difficult. become infeasible. Just going back to the landscape of digital media for a moment, because I mean, at the moment, I think this stuff is quite fragmented. Do you think that we will get to a point where there is a central space that you can go to that will just tell you? whether something is AI generated or not. I think the field is moving towards that. There is a overall sort of industry alignment on the need for prominent solutions where companies have agreed on a standard of like, making sure that when there are edits or when there is generative AI, like sort of placing that information alongside the file, The issue with that is that metadata can be sort of stripped off, which is why we need strong binding, which what watermarks provide. So by combining these approaches and by building a solution that works for everyone, in the community. I think we'll be able to get to a standard. But I think like progress is being made on that, but it just takes time. but this [35:00] So I guess same question for the biological side of things. Is that the aim then, that there will be this industry standard at some point that everybody is using? There's a lot of things, and we even know this in the discussion, where we know that there's engineering headroom to be able to move into, both in the way that you're calculating G-values and introducing the watermark. the provenance of AI-generated sequences. I do also wonder, though, whether there's a bit of a game of cat and mouse going on here, right? Because the more universal a system becomes, the more it's clear what the rules are and how it's implemented, Does it not become easier for somebody to come along with another system that is able to undo it? Yes. So I think this is very much a risk, which is why I think it's really important for us to have these measures where you have these secret keys with which we can embed watermarks in the signal itself, right? Whether it's... uh, a protein sequence, whether it's images and so on, which are hard to take away. Like the ideal situation will be hard to take away to an extent that if you want to get rid of the detection signal, that you have to change the function. right, and destroy the original sort of capability of the task that you had designed the thing for. Like it's so embedded within it that there's no way to undo it. Yes. But I guess it is, as you said, this is not necessarily the final way of doing things. This is sort of early days. Yeah. The fact that you can have like a biological object, right, like a sort of write down at that level. contains within it the story of where it came from. Yeah, it's pretty incredible. And I guess I would say we're not the first ones to do watermarks, but we're the first ones to our knowledge to both show imperceptibility and function preservation and including in vitro and sort of lab based results. Absolutely amazing. Really, really incredible stuff. Thank you both so much for joining. That was amazing. Thank you. Words like guardrails and safety, they get thrown around a lot when it comes to AI. But this right here, this is concrete proof. of this lab putting its money where its mouth is. They pioneered the technology to watermark AI generated content well before most other labs were thinking about it. They proved it was scalable, that it wouldn't impact on the quality or speed of the output. And now their ideas have been embedded within regulation. But rather than resting on their laurels, they are already looking ahead to what the next threats are and how they can help to build a world that is able harms to take hold. And they are doing that before the threats fully materialize. The future, as it turns out, will come with the certificate of authenticity. You have been listening to Google Deep Mind the podcast with me, Professor Hannah Fry. We have plenty more to come on this series. But in the meantime, check out our other episodes and we will see you very soon.