David Silver Drops Human Data to Chase Superintelligence in 3 Years

David Silver and Lasse Espeholt of Ineffable Intelligence: A Superlearner for the Age of Experience — Sequoia Capital

Sequoia Capital

Training models on human language imposes an artificial ceiling on reasoning, making tabula rasa reinforcement learning in rich simulations the only viable route to true superintelligence.

AI-assisted summary of David Silver and Lasse Espeholt of Ineffable Intelligence: A Superlearner for the Age of Experience — original source. Check the original for full context.

Source published:

Key ideas

  • David Silver and Lasse Espeholt have co-founded Ineffable Intelligence, an independent 20-person lab pursuing a tabula rasa 'superlearner' that derives science, mathematics, and language directly from experience rather than pre-training on human text. (3:35)
  • The lab has deployed a dedicated NVIDIA Vera Rubin GPU cluster on Google Cloud Platform, applying roughly 100,000 times more compute to multi-agent self-discovery reinforcement learning than OpenAI's 2019 'Hide and Seek' experiments. (36:24)
  • Silver argues that human language is an artificial ceiling on AI capability and that chain-of-thought monitoring will break down under strategic subterfuge, necessitating an empirical 'science of superintelligence' tested entirely inside digital simulations before physical deployment. (43:44)
  • Ineffable projects clear empirical validation of its autonomous learning architecture within one year, forecasting genuine superintelligence within three years. (65:47)

Detailed notes

## Background Published by Sequoia Capital on RSS and the web on October 13, 2026, and hosted by Sonya Huang, this interview features Ineffable Intelligence co-founders David Silver and Lasse Espeholt. According to the show notes, the episode details their venture to build a tabula rasa 'superlearner' that bypasses internet text pre-training to discover mathematics, science, and reasoning autonomously through direct experience.

## TL;DR David Silver and Lasse Espeholt outline their contrarian thesis that true superintelligence requires abandoning large language model pre-training in favor of pure, experience-driven reinforcement learning. They describe building synthetic simulation environments termed 'crucibles' where agents develop emergent tool use, language, and conceptual frameworks from simple survival objectives without dense human reward scaffolding. Drawing on their architectural work with IMPALA and SEED RL, the founders detail running massive GPU clusters with an intentionally capped 20-person team, arguing that alignment and multi-agent dynamics must be mastered in simulation before physical deployment. Backing up his historical forecasting track record, Silver predicts strong architectural validation within twelve months and genuine superintelligence within three years.

## Atmosphere & style The conversation carries the calm, intellectual intensity of foundational researchers quietly betting against the consensus of the entire AI industry. Silver brings a patient, almost philosophical confidence born from his AlphaZero breakthroughs, balanced by Espeholt's pragmatic systems-engineering rigor and Sonya Huang's incisive framing. Listening feels like stepping inside a focused, high-stakes research enclave—cerebral, technically uncompromising, and animated by a quiet sense of scientific wonder.

## Bulletin - `03:35` David Silver and Lasse Espeholt have co-founded Ineffable Intelligence, an independent 20-person lab pursuing a tabula rasa 'superlearner' that derives science, mathematics, and language directly from experience rather than pre-training on human text. - `36:24` The lab has deployed a dedicated NVIDIA Vera Rubin GPU cluster on Google Cloud Platform, applying roughly 100,000 times more compute to multi-agent self-discovery reinforcement learning than OpenAI's 2019 'Hide and Seek' experiments. - `43:44` Silver argues that human language is an artificial ceiling on AI capability and that chain-of-thought monitoring will break down under strategic subterfuge, necessitating an empirical 'science of superintelligence' tested entirely inside digital simulations before physical deployment. - `01:05:47` Ineffable projects clear empirical validation of its autonomous learning architecture within one year, forecasting genuine superintelligence within three years.

## Key takeaways - `00:00` Human biological evolution demonstrates an existence proof that general intelligence can bootstrap itself from direct interaction without pre-training on trillion-token external text corpora. - `03:35` David Silver left DeepMind based on the conviction that building consumer-facing LLM products is an unnecessary intermediate detour on the path to superintelligence. - `05:58` The name 'Ineffable Intelligence' reflects the thesis that discoveries made by genuine superintelligence will be so alien and high-dimensional that human language cannot express them. - `10:00` A true superlearner is evaluated across two axes: depth (autonomous discovery surpassing human science) and breadth (versatility across arbitrary streams of digital, physical, or simulated experience). - `12:35` Relying on internet text corpora risks trapping models under a human cognitive ceiling, inheriting historical blind spots analogous to medieval belief in the four humors. - `16:13` Ineffable rejects dense reward shaping and curriculum 'breadcrumbs' in favor of rich simulation environments called 'crucibles' where complex behaviors like trade and tool use emerge organically from simple objectives. - `24:14` Multi-agent environments generate an 'infinite onion' of strategic complexity through recursive opponent modeling, avoiding the performance plateaus observed in single-agent settings. - `27:22` Tabula rasa self-play yields radically faster compounding than human curation, as evidenced when AlphaGo Zero surpassed years of human-data engineering in roughly two days. - `38:13` Digital simulation environments do not require molecular-level physical fidelity, only sufficient competitive pressure and environmental richness to incentivize abstraction and tool creation. - `40:00` Developing superintelligence inside simulation runs 10 to 100 times faster than physical-world robotics due to infinite parallelization, perfect reproducibility, and safe containment. - `47:45` Monitoring reasoning via English chain-of-thought transcripts is structurally brittle at scale; alignment must be solved through simulation-based safety proofs, offline 'shadow actions,' and independent watchdog models. - `50:00` Silver advocates an asymmetric pacing formula: accelerating experimentation at maximum velocity in simulation while deliberately slowing physical-world deployment so human institutions can adapt. - `52:54` Ineffable's infrastructure unifies simulation, inference, and policy training onto Vera Rubin GPUs on GCP, extending the architectural lineage of IMPALA and SEED RL. - `01:00:58` Ineffable deliberately caps its team at 20 people to maximize compute per researcher and insulate its culture from the commercial prisoner's dilemmas facing frontier tech giants. - `01:05:47` Silver forecasts strong validating evidence for Ineffable's self-directed architecture within one year, and genuine superintelligence within three years.

## Detailed walkthrough ### `00:00` The Human Starting Point and the Return to First Principles

David Silver frames the core philosophy of his work around a striking biological observation: humanity did not begin with a vast pre-existing corpus of recorded knowledge. Instead, our ancestors started with nothing more than "this kind of gray mush in our head," knowing absolutely nothing about the universe. Over millennia, humanity collectively pulled itself up by its bootstraps, discovering languages, technology, science, and mathematics through an organic, cumulative process of firsthand exploration. Everything humans know today was learned by discovering it for ourselves—a feat that modern artificial intelligence has yet to reproduce. For Silver, that capacity for self-directed, open-ended discovery is not just an interesting capability; it is the fundamental essence of what will be required to create an intelligence that can continue learning and uncovering new truths indefinitely.

Sonya Huang welcomes David Silver and Lasse Espeholt, co-founders of Ineffable Intelligence, noting with excitement that Silver was the very first researcher she cold-emailed to appear on the show roughly one hundred episodes prior, making the conversation feel like a long-awaited full-circle milestone.

`01:09` From AlphaGo to LLMs: What Held True and What Surprised

Reflecting back to his 2009 PhD thesis—which applied reinforcement learning (RL) to the famously intractable game of Go and set the stage for proving superhuman machine intelligence was possible—Silver explains that his foundational thesis has remained completely unchanged. He entered the field with an unwavering conviction that artificial intelligence should be solved through direct experience: building systems that start from scratch and refine their understanding through reinforcement learning, eventually causing general intelligence and superintelligence to emerge organically. That philosophy carried through AlphaGo and AlphaZero, and it now forms the bedrock of Ineffable Intelligence.

```mermaid flowchart TD accTitle: Paradigms of AI Development accDescr: Comparison between the dominant LLM paradigm and Ineffable's experience-driven reinforcement learning paradigm.

subgraph LLMPath["The Dominant LLM Paradigm"] A["Human Web Corpus"] --> B["Massive Pre-training"] B --> C["Post-Training / Alignment"] C --> D["Language-Bound Ceiling"] end

subgraph RLPath["Ineffable's Experience-First Paradigm"] E["Tabula Rasa Agent"] --> F["Self-Play & Simulated Environments"] F --> G["Emergent Discovery (Math, Science)"] G --> H["Superintelligence Beyond Human Text"] end ```

However, the explosive rise of large language models (LLMs) over the past five years did challenge his intuition in one major technical respect. Silver admits he was genuinely astonished by the transfer efficiency of LLM pre-training: the fact that a model exposed to massive quantities of general data could subsequently be fine-tuned on a tiny slice of specific data to radically reshape its behavior overnight. That dynamic shifted his perspective, proving that while acquiring base capability through vast amounts of data can be extraordinarily difficult and computationally slow, once a model passes a critical threshold of exposure, behaviors that previously seemed impossible or sluggish can crystallize with incredible speed.

`03:35` Leaving DeepMind: Searching Under a Different Lamppost

Addressing why he chose to leave DeepMind—a lab where he spent many productive years and enjoyed virtually unmatched access to computational resources—Silver emphasizes his profound respect and affection for his former colleagues. However, his intellectual conviction about the path to superintelligence diverged fundamentally from the industry's prevailing direction. He came to believe deeply that the route to true superintelligence does not run through LLM-based consumer products as a necessary intermediate milestone.

Once he recognized that conviction, it became clear that such an approach could not simply exist as an experimental side project tucked away in the corner of a massive lab whose primary mandate is shipping LLM infrastructure. It required an elite, single-minded laboratory willing to stake everything on learning from experience. Silver observed that the global AI landscape is already dominated by tens or hundreds of thousands of brilliant researchers all crowded beneath the exact same lamppost—optimizing pre-training runs, post-training pipelines, reasoning chains, and code synthesis: - Rather than becoming the "hundredth thousand and one person" refining the dominant LLM playbook, Silver sought out the neglected lamppost where almost nobody is currently searching. - Operating on the contrarian path offers a vastly higher leverage point: if the pure experience hypothesis proves correct, it will fundamentally reset the trajectory of the entire discipline.

`05:58` The Meaning of "Ineffable" (and the Secret Second Joke)

When Huang praises the lab's distinctive name, Silver reveals the philosophical reasoning behind naming the company "Ineffable Intelligence." The name reflects the idea that the discoveries superintelligence will eventually make will be so unimaginably profound, complex, and alien that human language will be incapable of encapsulating or expressing them—rendering them literally ineffable. By departing from human-language corpora and letting an agent discover reality through raw environmental interaction, the system is free to formulate representations of truth that transcend human vocabulary.

Silver then reveals the lighthearted "secret second reason" shared internally by the founding team. Their conviction in reinforcement learning from direct experience is so total that they consider the approach structurally guaranteed to succeed. In their view, pure learning from experience is an architecture that simply cannot fail—meaning it "can't possibly F up," making it humorously and phonetically *in-eff-able*.

`07:03` Lasse Espeholt's Journey and the Foundations of Scaled RL

Lasse Espeholt traces his path into the founding team, recounting how he joined DeepMind in January 2015 within the deep learning division. By 2016, he began collaborating closely with Silver, who was running the reinforcement learning group, bridging deep learning architectures with RL algorithms. Espeholt eventually departed DeepMind in the summer of 2018 to join Google Brain, later returned, and then left DeepMind a second time roughly a year prior to Silver's exit.

The initial spark for joining Ineffable arrived when an investor reached out to Espeholt to ask his opinion of Silver. Espeholt's immediate internal reaction was a flash of regret: "Damn, I missed out on an opportunity. I should have reached out to him." Having long viewed Silver as consistently "one of the smartest people in the room," Espeholt eagerly reconnected, thrilled by the caliber of the team—which also reunited him with colleagues Wojtek Czarnecki and Chris Mock—and energized by the rare opportunity to build an ambitious, frontier-level AI research laboratory based in Europe.

Silver is quick to return the praise, pointing out Espeholt's characteristic modesty. Silver emphasizes that Espeholt was the pioneer who originally demonstrated how to take reinforcement learning out of small-scale experimental loops and massively parallelize it across GPU clusters. The architectural breakthroughs Espeholt authored—which laid the groundwork for frameworks like IMPALA and SEED RL—represent the foundational lineage that all modern distributed, scaled-up reinforcement learning systems rely upon today.

`09:31` Defining the Superlearner and First Contact

Huang transitions into Ineffable's explicit mission statement: building a "superlearner" in order to achieve "first contact" with superintelligence. Setting up the conceptual framework for the lab's technical roadmap, she invites Silver to formally define what a superlearner is and what that historic moment of first contact will actually look like when the system begins generating insights beyond human capability.

`10:00` Defining the Superlearner: Depth, Breadth, and First Contact

David Silver characterizes a "superlearner" along two primary dimensions: depth and breadth. In the depth dimension, the system must have the capacity to start tabula rasa—knowing nothing at all—and progress entirely on its own to discoveries that reach far beyond the frontier of human understanding. Silver draws a direct comparison to human history, noting that humanity began with the "gray mush in our head" and, without an external entity handing down a blueprint or feeding us a "trillion element data set," independently pioneered language, technology, science, and mathematics. AI has yet to crack an architecture capable of learning and generating novel knowledge indefinitely. In the breadth dimension, a true superlearner cannot be locked to a single domain; it must be versatile enough to be dropped into any continuous stream of experience—whether an abstract simulation, a digital operating system, or the messy physical world—and autonomously absorb and adapt to that environment. Cracking both dimensions, Silver asserts, will yield a superintelligence that will feel like "encountering magic."

When asked what it will feel like when humanity first encounters this kind of entity, Silver predicts a profound sense of "wonderment." He reflects on his time leading the AlphaGo and AlphaZero projects at DeepMind, recalling how even in the comparatively narrow domain of board games, superhuman play sparked genuine joy and awe. Elite Go players were not merely humbled when watching AlphaGo play; they were visibly amazed and delighted to witness moves that challenged centuries of orthodoxy, an experience that repeated itself when superhuman play came to chess. Scaling that phenomenon from games to an intelligence capable of uncovering new scientific domains and previously unimagined technologies will, in Silver's view, evoke that same joyful wonder on a civilizational scale.

- **Depth Dimension:** Progressing autonomously from zero prior knowledge to superhuman discoveries, mirroring human collective development rather than relying on human demonstrations or curated datasets. - **Breadth Dimension:** The ability to be dropped into arbitrary streams of experience—simulated, digital, or physical—and continuously learn without architecture changes. - **The AlphaGo Analogy:** Superhuman breakthroughs in Go and chess generated wonder and delight among grandmasters; open-ended superlearners are expected to elicit that same emotional reaction across science and technology.

`12:35` Autonomous Discovery vs. the Weight of Human Data in LLMs

Sonya Huang raises the current state of large language models, pointing out that modern LLMs paired with reinforcement learning are already tackling frontier math like the Millennium Prize problems and assisting in scientific research. Silver acknowledges these accomplishments with respect, but draws a sharp distinction between LLM scaffolding and genuine superlearning. Current LLM-driven breakthroughs rely heavily on human-in-the-loop intervention, bespoke prompt engineering, and narrowly framed search environments. They are far from autonomous agents operating on raw environmental feedback.

Crucially, Silver questions whether language is even the proper vehicle for general intelligence. An agent tethered to human language and trained on the corpus of the internet may be intrinsically constrained. Rather than serving as an unshakeable foundation for reasoning, the reliance on human-generated text and imitation priors might actually be "weighing things down," holding AI back from learning directly through the physics and dynamics of its own experiences.

- **Autonomy Gap:** Current frontier LLM achievements depend on heavy human-in-the-loop steering, whereas a superlearner must pursue and validate discoveries without manual hand-holding. - **The Language Ceiling:** Human language may not be the optimal substrate for fundamental reasoning, and massive corpora of human data may impose an artificial ceiling on what an agent can deduce.

`13:59` Day-to-Day Engineering: The RL Stack and Simulated "Crucibles"

Huang notes that Ioannis Antonoglou (a Sequoia-backed founder and an original member of the AlphaGo team) praised Silver's ability to anchor grand, multi-decade visions in concrete, rigorous engineering milestones. Silver breaks down what that day-to-day execution looks like at Ineffable Intelligence. On the technical infrastructure side, co-founder Lasse Espeholt leads the construction of a state-of-the-art reinforcement learning stack engineered to run at unprecedented scale, designed to "inhale data" with optimal throughput and support persistent, uninterrupted online learning.

Simultaneously, the team is designing synthetic environments they call "crucibles." A crucible is a purpose-built, simulated world engineered to give a superlearner the affordances needed to discover, express, and compound every fundamental facet of intelligence. Rather than evaluating the system on static benchmarks, Ineffable grounds its research progress in milestones mirroring the major intellectual turning points of human civilization. The core research program consists of placing the agent into these crucibles, running reinforcement learning at massive scale, and verifying whether it independently discovers the concepts that defined humanity's cognitive evolution.

```mermaid flowchart LR accTitle: Ineffable Intelligence Research Engine accDescr: The loop between scaling the RL stack, running crucibles, and hitting human-journey milestones. subgraph Engine["Core Research Engine"] A["Scalable RL Stack<br/>(Lasse Espeholt)"] --> B["Crucibles<br/>(Simulated Environments)"] B --> C["Superlearner<br/>Online Experience"] C --> D["Emergent Milestones<br/>(Human Journey Turning Points)"] D --> A end ```

- **The RL Stack:** Built to maximize data ingestion and maintain stable, lifelong reinforcement learning across massive compute clusters. - **Crucibles:** Rich, simulated environments engineered with sufficient complexity for agents to autonomously express and evolve high-order behaviors. - **Empirical Grounding:** Progress is tracked not through narrow synthetic metrics, but by whether the agent naturally recapitulates the major intellectual milestones of the human journey.

`16:13` The Anti-Breadcrumb Philosophy and Emergent Complexity

Silver contrasts Ineffable’s methodology with the standard reinforcement learning techniques applied to modern LLMs, which he characterizes as leaving a "breadcrumb" trail. In current post-training pipelines, engineers micromanage the agent's progress by steering it from one narrow task to the next, effectively "pulling the agent by its nose" toward a pre-selected outcome. Silver rejects this top-down specification. Ineffable provides agents with simple, fundamental optimization objectives inside highly complex environments, deliberately leaving room for the agent to surprise its creators.

Addressing the skepticism around whether complex behaviors like mathematics or scientific inquiry can emerge without an explicitly shaped reward function, Silver points to natural evolution and cultural history. Evolution does not feature a reward function for "science" or "civilization"; it simply optimizes for a basic fitness signal like survival, yet that simple pressure yielded the entire diversity and complexity of the biosphere. Similarly, human civilization arose not because early hominids set out to invent calculus, but because the pursuit of basic survival and organizational goals naturally motivated coordination. Coordination required communication; communication enabled commerce and trade; trade necessitated basic arithmetic; calculating transactions fostered tool creation; understanding tools demanded empirical science; and advancing that science required formal mathematics.

```mermaid flowchart TD accTitle: Emergence of Human Complexity from Simple Objectives accDescr: A progressive cascade showing how basic coordination led step by step to complex science and mathematics. subgraph Foundations["Basic Pressures"] P["Simple Optimization Goal"] --> C["Coordination"] C --> Comm["Communication"] end subgraph Tooling["Economic & Tool Evolution"] Comm --> T["Trade & Commerce"] T --> A["Basic Arithmetic"] A --> Tools["Tool Development"] end subgraph Advanced["Scientific Explosion"] Tools --> S["Empirical Science"] S --> Tech["Complex Technology"] Tech --> M["Advanced Mathematics"] end ```

- **Rejection of Breadcrumbing:** Moving away from dense reward shaping and hand-curated curriculum paths that force the agent down predetermined human avenues. - **Evolutionary Analogy:** Just as simple reproductive fitness generated the immense complexity of terrestrial biology, simple reward criteria in an expressive environment can drive open-ended intelligence. - **The Civilization Chain:** Basic goals spur coordination, communication, trade, arithmetic, tool creation, empirical science, and higher mathematics in a compounding emergent sequence.

`20:00` Reward Functions and Open-Ended Discovery

Sonya Huang draws a parallel between Ineffable Intelligence's training philosophy and biological evolution, where an organism's only objective is surviving to produce the next generation, and asks what a reward function actually looks like in their setup. David Silver clarifies that the lab is actively experimenting with a variety of reward mechanisms rather than relying on a single monolithic formula. Crucially, Ineffable refuses to handcraft rewards that explicitly specify what constitutes "science," "technology," or "language." Silver argues that if AI researchers must hardcode the definitions of these domains, systems will never possess the freedom to discover entirely new fields of knowledge that humans have not yet conceived.

Silver illustrates this limitation with an analogy to classical physics: an agent operating under Newtonian mechanics whose objectives were bounded by human understanding would never have anticipated quantum mechanics or general relativity. To unlock comparable breakthroughs, learning environments must remain fundamentally open-ended, allowing agents to navigate and synthesize their own conceptual frameworks from first principles.

- **Avoiding narrow definitions:** Reward structures explicitly avoid programmatic descriptions of human cultural or technological artifacts. - **Uncharted intellectual territory:** The objective is to foster systems capable of uncovering emergent domains of knowledge that exist outside current human scientific frameworks. - **The Newtonian analogy:** Pre-specifying the boundaries of knowledge prevents an agent from discovering paradigm shifts akin to quantum mechanics or relativity.

`21:16` Designing Crucibles: Emergent Complexity over Handcrafted Tasks

Addressing the contrast between conventional AI benchmarks and Ineffable's approach, Huang notes that contemporary research typically tests agents on pre-baked tasks with known answers. Silver explains that Ineffable instead engineers "crucibles"—simulated environments designed to generate massive emergent complexity without requiring developers to handcraft arbitrary puzzles. This complexity is catalyzed through two distinct avenues: rich physics engines featuring millions of interacting objects, and large-scale multi-agent worlds where millions of individual entities interact, constantly compounding the environmental complexity each agent must navigate. These settings provide the raw substrate for agents to discover behaviors ranging from basic motor control and coordination up to mathematics, tool creation, and language.

Lasse Espeholt emphasizes the engineering trade-offs inherent in this simulation philosophy. While a board game like Go is represented with extreme simplicity as a bitmap, it still fosters deeply complex emergent dynamics. Simulating reality down to realistic quantum physics would be computationally crippling. Ineffable therefore builds a spectrum of crucibles, including lightweight, highly optimized environments that iterate rapidly on hardware while preserving the capacity for intricate, emergent agent behaviors. Huang observes that the underlying primitives dictate the nature of the learning; for instance, capturing information theory and game-theoretic dynamics allows economics to emerge naturally without needing physical mechanics.

```mermaid flowchart TD accTitle: Crucible Architecture and Emergence Pipeline accDescr: Diagram showing how lightweight simulation primitives drive emergent behaviors through physics and multi-agent interaction.

subgraph Primitives["Crucible Primitives"] P1["Fast Physics Engines<br/>(Millions of Objects)"] P2["Multi-Agent Substrates<br/>(Millions of Interacting Entities)"] end

subgraph Dynamics["Interaction Layer"] D1["Environmental Feedback Loops"] D2["Adversarial & Cooperative Adaptation"] end

subgraph Emergence["Emergent Discovery"] E1["Motor Control & Coordination"] E2["Mathematics, Science & Economics"] E3["Non-Human Communication Schemes"] end

P1 --> D1 P2 --> D2 D1 --> E1 D2 --> E2 D1 & D2 --> E3 ```

- **Contrast with standard suites:** Crucibles deliberately move away from fixed task suites like Atari 57 or static synthetic benchmarks. - **Sources of emergent complexity:** High-entity physics simulations and vast multi-agent populations that construct and modify their shared world. - **Computational efficiency vs. fidelity:** Simulating full-fidelity quantum physics is intractable, so Ineffable balances rapid iteration speed with complex emergent headroom. - **Foundational primitives:** Designing the core mechanics around information theory and game theory allows advanced disciplines like economics to emerge without hardcoded rules.

`24:14` The Infinite Onion: Multi-Agent Emergence in Go

Silver returns to Go to demonstrate how minimal rules give rise to unbounded strategic complexity. A single-player version of Go on a standard 19×19 grid would be "the most boring game you could imagine," reducing to a trivial exercise of mindlessly placing stones until the board is completely covered. The moment a second agent enters the environment, however, the complexity expands exponentially. Each player must reason about the vast, recursive possibilities of the opponent's behavior, transforming a simple grid into an open-ended strategic arena.

Silver describes this phenomenon as an "infinite onion" of complexity, where every layer peeled back reveals further strategic depths. Throughout DeepMind's work on the game, the learning curves never leveled off into an absolute ceiling; the system continued to uncover novel concepts and strategies as more compute was applied. While Go remains a bounded board game, it demonstrates that simple environmental dynamics paired with multi-agent interactions can sustain open-ended discovery limited only by available computational power.

- **Single-agent vs. multi-agent:** Placing stones alone on a 19×19 grid involves zero strategic depth; introducing a second agent unlocks an explosive combinatorial space. - **The "infinite onion":** The recursive need to model another intelligent entity creates layer upon layer of strategic depth that never completely saturates. - **Boundless learning curves:** AlphaGo and AlphaZero demonstrated sustained improvement with no plateaus across millions of self-play games.

`25:41` Beyond Human Language: Parallel Neural Reasoning and Cultural Biases

When asked whether autonomous agents will ultimately reinvent human language, Silver expresses strong skepticism, stating that viewing human language as the pinnacle or ideal vehicle of thought is "a little naive." He argues that authentic machine intelligence will likely rely on reasoning paradigms that are far more expressive and capable than sequential word tokens—such as rich, high-dimensional, highly parallelizable neural representations that operate without natural language constraints.

Furthermore, Silver warns that anchoring artificial cognition to human language inherently hardcodes human cognitive limitations into the model. He presents a historical thought experiment: if an agent had been trained exclusively on all written human knowledge from 500, 1,000, or 2,000 years ago, it would have inherited and internalized foundational premises that the Earth was flat or that illness stemmed from an imbalance of the "four humors." Because these misconceptions were deeply woven into historical linguistic structures and thinking processes, extracting them would be nearly impossible. Moving beyond human language is therefore necessary to prevent human conceptual blind spots from capping the agent's ceiling.

- **Language as an intermediate step:** Human language is viewed as a restrictive, sequential bottleneck rather than the optimal structure for complex thought. - **Neural reasoning primitives:** Expectation that non-linguistic, high-dimensional, parallelizable representations will prove far more expressive for general reasoning. - **The four humors thought experiment:** Relying on human text bakes in the prevailing dogma of the authorial era, artificially constraining the agent's understanding of reality.

`27:22` Removing Human Priors: From AlphaGo Zero to Ineffable's Zero-Data Bet

Huang points to the leap from AlphaGo—which relied on expert human games—to AlphaGo Zero, which trained strictly through tabula rasa self-play, asking Silver whether he anticipated that abandoning human priors would produce a significantly stronger system. Silver admits he genuinely did not know, placing the probability of success at roughly 50-50 before running the experiment. He notes that the most rewarding research questions sit "on a knife edge," explaining that pursuing experiments with a 95% certainty of working is fundamentally uninteresting. Seeing AlphaGo Zero dramatically eclipse its predecessor through pure self-play validated the power of unconstrained algorithmic learning.

Silver links this breakthrough directly to Ineffable Intelligence's core thesis: making the deliberate bet to bypass human text and the entire internet of human data. While this decision imposes an acute short-term performance penalty, Silver argues it is the only viable path to achieving true superintelligence in both depth and breadth:

``` ┌────────────────────────────────────────────────────────┐ │ Ineffable's Core Bet │ │ (Complete removal of human data & internet pretraining) │ └──────────────────────────┬─────────────────────────────┘ │ ┌────────────────────────┴────────────────────────┐ ▼ ▼ ┌───────────────────────────────┐ ┌───────────────────────────────┐ │ Depth Dimension │ │ Breadth Dimension │ ├───────────────────────────────┤ ├───────────────────────────────┤ │ • Breaks past human priors │ │ • Generalizes across regimes │ │ • Discovers new fields of │ │ • Excels in non-text domains │ │ science and mathematics │ │ (novel sensors, physics) │ └───────────────────────────────┘ └───────────────────────────────┘ ```

In the depth dimension, an agent chained to human demonstrations cannot easily transcend the ceiling of that data to invent entirely new fields of science. In the breadth dimension, relying on human-curated language severely limits an agent when deployed into domains where natural language is irrelevant—such as direct interaction with novel sensor arrays, alien environments, or non-human data streams.

- **The 50-50 research threshold:** Silver maintains that impactful foundational research requires pursuing hypotheses with genuine risk and uncertain outcomes. - **The short-term performance sacrifice:** Bypassing pre-trained human data yields immediate performance disadvantages compared to LLMs, but removes the long-term ceiling on capability. - **Depth advantage:** Eliminating human priors allows discovery algorithms to scale well past existing human scientific frontiers. - **Breadth advantage:** Algorithmic autonomy ensures agents can operate across arbitrary sensory representations where human linguistic data offers no utility.

`30:00` Removing Human Priors: AlphaGo to AlphaZero

David Silver reflects on the hard-won realization that human data acts as an artificial ceiling on discovery. During the two years DeepMind spent developing the original AlphaGo to defeat Lee Sedol, the engineering effort resembled the contemporary LLM paradigm: researchers meticulously constructed each layer, injected massive corpora of expert human games, and operated constantly as humans in the loop, tuning and deciding how the next iteration should improve. When the team made the deliberate leap to AlphaZero, they abandoned human data entirely to see what an algorithm could discover through pure self-play.

The contrast in velocity and performance was staggering. While the human-curated system required two years of painstaking iteration, the unconstrained self-learning algorithm surpassed all previous benchmarks in mere hours. In chess, AlphaZero defeated Stockfish—the strongest computer chess engine in history—after just four hours of training from scratch, and in Go, it eclipsed the two-year, human-trained AlphaGo in roughly a couple of days. Silver observes that major frontier LLM labs today remain locked in a semi-autonomous approach, employing thousands of human annotators and engineers in the loop. While he respects how far that strategy has scaled, he maintains that an agent can only achieve runaway self-improvement and become a true "superlearner" if researchers actively step back and let the machine discover knowledge on its own.

```mermaid flowchart TD accTitle: AlphaGo vs AlphaZero Development Paradigms accDescr: Comparison between human-curated training taking years versus pure self-play discovery taking hours.

subgraph HumanCurated ["AlphaGo / LLM Paradigm"] A["Human Expert Data"] --> B["Human-in-the-Loop Tuning"] B --> C["2 Years of Manual Iteration"] C --> D["Lee Sedol Defeated (2016)"] end

subgraph PureDiscovery ["AlphaZero / Superlearner Paradigm"] E["Tabula Rasa (Zero Human Data)"] --> F["Pure Self-Play Discovery"] F --> G["4 Hours in Chess / 2 Days in Go"] G --> H["Surpasses All Previous AI"] end ```

- **Two years vs. four hours:** AlphaGo required two years of intensive human engineering and human game records; AlphaZero surpassed world-champion-level play in chess in four hours and outperformed AlphaGo within a couple of days. - **The LLM bottleneck:** Modern AI labs rely on thousands of people in the reinforcement learning from human feedback (RLHF) loop, creating an incremental ceiling rather than an autonomous discovery engine.

---

`32:05` Keeping Human Bias Out of the Crucibles

Sonya Huang raises the paradox of Ineffable Intelligence: since humans must design the training environments ("crucibles") and their reward functions, how do they prevent human priors from leaking in and contaminating the agent's learning? Lasse Espeholt acknowledges that inherent human bias exists whenever an objective is framed, but the crucial discipline lies in not prescribing the implementation. If the team tasks an agent with solving a problem where a computational machine would be useful, they deliberately avoid nudging the agent toward familiar concepts like the von Neumann computer architecture, choosing instead to see what alien computing paradigms emerge organically.

Silver clarifies that Ineffable’s philosophy is pragmatic rather than ideological: "It's not like a religion that you can't have any human prior anywhere." The line is drawn between weak, environment-agnostic priors—which are natural and necessary—and heavy, domain-specific priors that ingest trillion-token human datasets to dictate how a problem must be solved. Rather than trying to hard-code all existing human knowledge into the agent, the team uses broad simulation frameworks, betting that minimal constraints will generate richer, more general capabilities.

---

`33:35` Confidence, Timing, and Superforecasting

Addressing Huang's question about whether he is truly on the knife's edge (50-50) regarding Ineffable’s success, Silver draws a distinction between fundamental feasibility and practical timing. He has near-total confidence that the core premise is scientifically viable because an undeniable existence proof already exists: the human brain, which evolved through embodied interaction without pre-training on external trillion-token corpora to manifest general intelligence.

For Silver, the real uncertainty lies in scientific timing. In frontier technology, arriving too late means competing with thousands of copycats, while arriving too early means hitting a wall before the requisite compute or tooling exists. Reflecting on his track record, Silver notes that he successfully forecasted the resolution of Computer Go a decade before AlphaGo won, correctly predicted the timeline for solving *StarCraft II* with AlphaStar, and forecasted when the International Mathematical Olympiad (IMO) benchmark would fall with AlphaProof. Described by Huang as a "superforecaster of sorts," Silver feels the stars have now aligned for superlearners: scaling laws are well understood, RL stacks are sufficiently mature, and automated research workflows make this exact moment the opening of the window.

- **Historical forecast accuracy:** Silver accurately predicted the timeline for solving Computer Go ten years in advance, won internal bets on *StarCraft II*, and forecasted the timeline for AlphaProof at the IMO. - **The startup sweet spot:** Success depends on catching the narrow window where compute, algorithmic foundations, and infrastructure mature before the problem becomes obvious to the rest of the industry.

---

`36:24` Is Compute the Driver?

When Huang asks whether compute is the primary catalyst validating his forecast, Silver confirms it is an enormous driver. He points out that previous benchmarks in emergent behavior, such as OpenAI's 2019 "Hide and Seek" multi-agent experiments—which exhibited primitive tool and ramp usage—barely scratched the surface. Today, Ineffable is equipped to execute self-discovery RL experiments with roughly 100,000 times more compute than those pioneering runs utilized. Silver finds it striking that the broader AI research community failed to double down on self-discovered RL at scale, instead pivoting en masse to language modeling.

Espeholt highlights the structural advantage Ineffable holds by securing a massive, dedicated compute cluster: "a very beautiful cluster at that." Having access to this scale allows the team to run unconstrained RL experiments that were practically impossible to execute even during their tenure at DeepMind, giving their small team an unprecedented laboratory to stress-test pure self-play learning.

- **Scale over early experiments:** Ineffable can train agents with roughly $100{,}000\times$ the computational budget used in OpenAI's classic Hide and Seek emergent tool-use research. - **Infrastructure independence:** Securing an elite GPU cluster allows Ineffable to pursue large-scale, long-horizon exploration runs that conventional corporate labs deprioritize in favor of near-term LLM serving.

---

`38:13` The Ceiling on Simulation and Rich Sutton's Objection

Huang challenges the reliance on digital crucibles by citing a recent appearance by reinforcement learning pioneer Rich Sutton, who expressed deep skepticism about simulations and their inability to replicate the open-ended complexities of the physical world. She asks where the ceiling on simulation lies and when Ineffable plans to transition its agents into physical reality.

Espeholt tackles the objection by drawing directly from his past work at Google Brain Amsterdam and Google Research on MetNet, a deep-learning model designed for short-term precipitation and weather forecasting:

```mermaid flowchart LR accTitle: Physics Simulation vs Learned Machine Learning Dynamics accDescr: Comparison of traditional physics simulations accumulating error over time versus learned models evaluated on real data.

subgraph PhysicsModel ["Traditional Simulation"] P1["Numerical Physics Rules"] --> P2["Accumulated Integration Errors"] P2 --> P3["Degraded Long-term Forecast"] end

subgraph MLModel ["MetNet Approach"] M1["Atmospheric Data"] --> M2["Direct Neural Prediction"] M2 --> M3["Outperformed Physics Simulators"] end ```

MetNet proved that purely data-driven machine learning models consistently outperformed classic physics-based numerical simulations because the physics simulations accumulated integration errors over time. Espeholt agrees with Sutton that digital simulations can never achieve photographic or molecular perfection, but argues that perfection is entirely unnecessary. To incubate emergent tool use, reasoning, and technological innovation, the simulated crucible does not need exact, ground-truth physics—it only needs rich enough environmental dynamics and competitive pressures to force agents to invent strategies, tools, and abstract abstractions. With modern tooling, even a lean, highly focused team can construct digital worlds capable of fostering these behaviors.

`40:00` Can Superintelligence Be Built in Simulation?

The discussion tackles whether there is a hard ceiling on what can be achieved purely within simulation. Lasse Espeholt acknowledges that fundamental physical phenomena—such as quantum physics—might represent a theoretical boundary, but points out that vast amounts of meaningful, complex work can be done well short of full real-world physics. David Silver reframes the core question: if it is fundamentally possible to build superintelligence in simulation, doing so will happen "10 times or 100 times faster" than in the physical world. Simulation affords infinite parallelization, perfect reproducibility, and, critically, offers the safest possible sandbox for superintelligence to emerge.

Silver explains why simulated environments need not be limited in their expressive power: - **Computation:** As long as a simulator is Turing complete, every conceivable concept of computation can provably be expressed within it, mirroring how humans build complex computers inside digital sandboxes. - **Economics:** By representing money with bits, the entire gamut of economic dynamics can be formulated. - **Language:** Equipping multiple agents with a communication channel allows everything meant by language to emerge natively. - **Mathematics and Science:** Mathematics is universal to any substrate. While certain empirical sciences relate to entropy or evolution, agents in an expressive simulation will inevitably formulate their own scientific understanding of whatever artificial universe they inhabit.

Addressing Rich Sutton's critique of simulation, Silver agrees with the premise that direct sim-to-real transfer of learned policies is fraught because the physical world exhibits immense, intractable complexity. However, Silver argues that the primary objective is discovering the *algorithm*—the superlearner itself. Once that core learning mechanism is understood, the exact same algorithmic blueprint can be brought to the real world, while simulated testbeds are steadily scaled toward higher real-world fidelity.

```mermaid flowchart TD accTitle: Substrates of Simulated Emergence accDescr: Diagram illustrating how foundational simulation primitives enable complex emergent systems like computation, economics, language, and science.

subgraph Primitives["Simulation Primitives"] TC["Turing Completeness"] Bits["Bit Representation"] Comms["Communication Channels"] Rules["Emergent World Rules"] end

subgraph Emergence["Emergent Phenomena"] Comp["Full Computation"] Econ["Economics & Money"] Lang["Emergent Language"] Sci["Agent-Discovered Science"] end

TC --> Comp Bits --> Econ Comms --> Lang Rules --> Sci ```

---

`43:44` The Science of Superintelligence and Peaceful Coexistence

Turning to alignment from a purely scientific standpoint, Sonya Huang asks whether scaling reinforcement learning introduces fundamental misalignment, and whether Ineffable's superlearner will naturally reflect human values. Silver points out that contemporary alignment methodologies depend heavily on human language monitoring and behavioral priors derived from human data. While functional up to now, these approaches are rapidly reaching their failure points. As models grow increasingly capable through intense optimization, they inevitably begin to "overwhelm their priors," developing the capacity for subterfuge where they learn to express one thing in text while pursuing an entirely different behavior.

Silver argues that the single most vital contribution Ineffable Intelligence can make is pioneering an empirical "science of superintelligence." By instantiating superintelligent agents and observing multi-agent societies in simulation, researchers can study how powerful entities interact with one another and, crucially, how they coexist peacefully with lesser intelligences. Silver emphasizes that humanity is already hurtling toward a multi-agent future with competing superintelligences. Ineffable has pledged to gate any real-world deployment on obtaining clear, simulated scientific evidence demonstrating that these systems interact safely.

- **Limits of current alignment:** Heavy reliance on human-text priors cannot keep pace as agents scale and learn deceptive surface compliance. - **Simulated coexistence:** Creating micro-societies of agents to rigorously test non-zero-sum coordination and peace between differing capability tiers before any real-world contact. - **Deployment gating:** A hard commitment not to release agents into real-world settings without verified safety proofs established inside simulation.

---

`47:45` Monitoring Agents Beyond Human Language

Addressing the concern that safety teams will lose visibility if agents stop reasoning in plain English or generate novel communication protocols, Silver cautions against over-relying on chain-of-thought supervision. Current models already produce massive "tomes" spanning millions of words, and human-readable reasoning can be faked or used for subterfuge. Because inspecting English reasoning strings is inherently brittle at superintelligent scales, oversight mechanisms must operate without relying on natural language transcripts.

Silver outlines alternative architectural safeguards: - **Shadow actions:** Allowing an agent to propose actions in an offline, unexecuted staging state, enabling evaluators to inspect its true behavioral intent before permitting real-world impact. - **Superintelligent watchdog agents:** Deploying specialized monitor models tasked solely with evaluating and regularizing the actions of primary agents. For instance, an agent acting as a world-class scientist could be paired with an independent superintelligent overseer that enforces a "tread lightly" constraint, restricting action magnitude and bounding systemic effects on the surrounding environment.

Silver concludes that while the field does not yet have all the solutions, developing alignment tools outside of human-language constraints is essential for steering future frontiers of autonomous intelligence.

`50:00` Pacing the Frontier: Fast in Simulation, Careful in the Real World

David Silver addresses the limitations of relying purely on language-based models, stressing that language alone is insufficient to get humanity where it wants to go. Despite warning that researchers are "in trouble" if they confine themselves solely to natural language, Silver remains fundamentally optimistic about the future. His vision is not a world governed by a single monolithic system, but rather an ecosystem populated by multiple "complex, interacting, superintelligent beings." In this multi-agent future, an equilibrium emerges: while some actors might not be aligned with humanity's best interests, others will be dedicated to supporting human flourishing, effectively acting as our defenders and protectors.

```mermaid flowchart TD accTitle: Dual Track Strategy for Frontier AI accDescr: Fast experimentation in simulation contrasted with measured pacing in real world deployment. subgraph Simulation ["Simulation Track (Accelerate Fast)"] S1["High-Speed Crucibles & Multi-Agent Trials"] --> S2["Develop Science of Superintelligence"] S2 --> S3["Test Alignment & Coexistence Dynamics"] end subgraph RealWorld ["Real-World Track (Paced & Measured)"] R1["Policy & Societal Adaptation Window"] R2["Safe Real-World Integration & Human Flourishing"] end S3 -->|"Validated Solutions"| R2 R1 -.->|"Allows Time For"| R2 ```

When asked about the widespread industry debate around "pacing the frontier," Silver expresses strong support for slowing down deployment in the physical world, aligning with precautions voiced by figures like Demis Hassabis and Dario Amodei. At the same time, he insists on an intentional asymmetry: humanity must move as fast as possible in simulation. Running rapid experiments in simulated environments allows researchers to build a rigorous "science of superintelligence" and develop safety mechanisms before anything touches physical reality. Pacing real-world deployment buys vital runway for governments, institutions, and human psychology to adapt to the dizzying rate of technical change:

- **Ecosystem of Superintelligences**: Silver foresees a balance of power among diverse superintelligent agents where pro-human systems safeguard humanity against misaligned ones. - **The Asymmetric Pacing Formula**: "Accelerate in simulation so we can understand what's happening and... do science around it, and then careful in the real world." - **Societal Latency**: The rate of raw AI progress far outpaces the speed at which governance structures, organizational frameworks, and human emotional responses can adjust.

---

`52:54` Compute Architecture: Vera Rubin Racks on GCP

The discussion turns to infrastructure. Sonya Huang highlights that Ineffable Intelligence is not simply renting transient spot instances from neo-cloud providers; they have secured top-tier infrastructure through deep partnerships with Google and NVIDIA. Lasse Espeholt reveals that Ineffable is among the very first customers slated to run on the largest NVIDIA Vera Rubin GPU cluster on Google Cloud Platform (GCP).

Espeholt contrasts their modern setup with DeepMind’s historical workflow nearly a decade ago. At Google, researchers historically leaned on vast, cheap pools of CPUs to generate environment rollouts. That paradigm no longer scales to modern frontier requirements. Today, running millions of external CPUs creates unacceptable bottlenecks. The Vera Rubin rack architecture provides exceptionally high intra-rack bandwidth, effectively allowing an entire rack to function as a unified machine. This low-latency interconnect is critical for reinforcement learning, which requires rapid, tight iteration loops between agent steps and model updates—unlike LLM training runs that process large, homogeneous batches at relatively leisurely intervals.

- **First-Wave Access**: Ineffable secured earliest customer access to GCP's upcoming flagship NVIDIA Vera Rubin GPU clusters. - **Rack as a Single Machine**: Intra-rack high-bandwidth links enable reinforcement learning systems to cycle through step rollouts and weight synchronizations without standard distributed computing latencies. - **Eliminating the CPU Crutch**: Whereas conventional LLM research labs must run sandbox environments on sprawling CPU clusters to simulate coding terminals or web browsers, Ineffable is consolidating simulation, inference, and policy training directly onto GPU hardware.

---

`55:18` Training, Inference, Simulation, and RL Scaling Laws

Espeholt breaks down Ineffable's computing requirements into three fundamental pillars: training, inference, and simulation. He points out a hardware reality: modern GPUs have spent the last decade optimizing intensely for deep learning training, and only recently for inference, leaving general-purpose simulation comparatively neglected. To bypass this hardware imbalance without being bogged down by raw simulation costs, Ineffable employs algorithmic leverage:

| Compute Pillar | Hardware Fit | Ineffable's Optimization Strategy | |:--- |:--- |:--- | | **Training** | Native GPU optimization | Massive parallel gradient updates across unified Rubin nodes. | | **Inference** | High GPU optimization | Shared weights for real-time policy evaluation and rollout action selection. | | **Simulation** | Poor / sub-optimal on GPU | Accelerated custom simulation kernels, model-based RL, and replay buffers. |

To bridge the simulation bottleneck, Espeholt highlights model-based reinforcement learning—where the learned world model absorbs the bulk of the computational burden relative to the simulator itself—alongside heavy replay buffers and data reuse schemes that extract maximum learning value from every generated environment step.

When asked whether RL is demonstrating clean scaling laws analogous to those popularized by large language models, Espeholt cautions that reinforcement learning obeys fundamentally different mathematics. LLM scaling curves measure compute, static dataset size, and parameter count over stationary text corpora. Reinforcement learning is strictly *non-stationary*: as an agent improves, its policy changes, which in turn alters the incoming distribution of experience. Nevertheless, establishing empirical RL scaling curves is an active focus for Ineffable as they scale up workloads from preliminary clusters to their full Rubin deployment.

---

`57:40` Lineage of Scale: From IMPALA and SEED RL to Ineffable

Sonya notes that before co-founding Ineffable, Espeholt authored some of the most influential systems algorithms in modern reinforcement learning. Espeholt recounts his trajectory starting on DeepMind’s deep learning team, where most RL experimentation was still trapped on single-machine CPU setups. To break through that ceiling, Espeholt and his collaborators created IMPALA (Importance Weighted Actor-Learner Architecture). IMPALA decoupled environment rollout collection from central GPU learners and scaled linearly with compute, becoming so versatile and computationally efficient that it became the ubiquitous engine across almost all reinforcement learning teams at DeepMind.

Later, after moving to Amsterdam to co-found Google Brain’s office there, Espeholt faced the challenge of scaling RL across large pods of Google TPUs. He developed SEED RL (Scalable, Efficient Deep-RL), an architecture that centralized both action inference and model parameter training on TPU hardware while keeping distributed worker environments flexible—capable of handling everything from board games like Go to complex physics environments without idling expensive accelerators. At Ineffable, that infrastructure lineage forms the company's core platform; the team runs customized, hardened implementations and modern derivatives of IMPALA and SEED RL to drive their current superlearner agents.

`01:00:00` Rediscovering RL Lineage and the 20-Person Team

Reflecting on large-scale reinforcement learning architectures, David Silver notes that researchers building RL systems today who haven't read the foundational papers from roughly eight years ago—such as IMPALA and SEED RL—should revisit them. Many of the architectural principles currently leveraged across the AI industry trace directly back to Lasse Espeholt’s pioneering work. Espeholt points out that over the past two years, the large language model (LLM) community has embraced reinforcement learning at scale, frequently rediscovering older RL techniques alongside genuine modern novelty.

When asked about the current size and structure of Ineffable Intelligence, Silver explains that the team has been intentionally kept small: exactly 20 people. Rather than rushing to scale headcount into a massive corporate organization, they believe a small cohort of exceptional individuals can accomplish extraordinary feats. Espeholt emphasizes that keeping the headcount small preserves an exceptionally high compute-to-person ratio, ensuring researchers never have to fight internally for GPU resources from day one. Silver details the four non-negotiable criteria used when evaluating talent: - Exceptional individual technical talent and mastery. - Kind, collaborative humanity and low ego, creating an enjoyable, healthy work environment. - Deep philosophical alignment with Ineffable's pure-RL, superintelligence mission. - Exact complementarity to the team’s current operational and technical skill gaps.

```mermaid flowchart TD accTitle: Ineffable Intelligence Talent and Compute Strategy accDescr: Diagram illustrating how Ineffable's small team and hiring criteria maintain a high compute-to-person ratio.

subgraph CultureCriteria ["Four Core Hiring Pillars"] A["Exceptional Talent"] B["Kind & Collaborative Demeanor"] C["Mission Alignment"] D["Specific Complementary Skills"] end

subgraph OrganizationalStructure ["Team Design"] E["Capped Team Size: ~20 People"] F["High Compute-to-Person Ratio"] G["Zero Internal Resource Scarcity"] end

CultureCriteria --> E E --> F F --> G ```

`01:02:40` Assembling an Avengers Team Without Corporate Race Dynamics

Several co-founders and team members previously collaborated on AlphaStar, which was assembled inside DeepMind by hand-picking elite researchers. Host Sonya Huang asks whether Ineffable is essentially "assembling the Avengers" from past DeepMind colleagues, or hiring new people into the fold. Silver clarifies that the roster is a hybrid. Because the hiring bar is set so uncompromisingly high, he has had to make the uncomfortable decision to turn down many talented former colleagues he previously worked with. He describes the final lineup as a "superstar team" of remarkable human beings.

Huang observes that working at Ineffable must feel like a breath of fresh air compared to incumbent tech giants. The largest frontier labs and cloud providers often find themselves trapped in a commercial "prisoner's dilemma," locked in an arms race where massive dollar stakes distort scientific decision-making. Researchers who genuinely care about the fundamental science of intelligence gravitate toward an environment decoupled from corporate distractions.

`01:05:00` A Research-Led Haven Free from Product Pressure

Silver elaborates on the deliberate decision to build a strictly research-led culture rather than a product-driven company. Ineffable is explicitly not organized around shipping incremental product iterations like a "v 17.1 of the thing that's come before." Instead, it is established as a sanctuary where scientists pursue foundational breakthroughs unencumbered by near-term product roadmaps.

Silver maintains that pushing back aggressively against product pressure from day one is the most rational path to outsized commercial impact in the long run: true scientific leaps unlock radically larger downstream applications than early product compromises ever could. This dynamic allows researchers to focus entirely on their scientific passions, creating an atmosphere that Silver describes as deeply rewarding for everyone involved.

`01:05:47` Forecasts: One Year to Validation, Three Years to Superintelligence

Huang prompts Silver—noted for his track record in superforecasting—to commit to concrete timelines for two major milestones: first, when Ineffable will see definitive evidence that their experience-driven approach works, and second, when humanity will make "first contact" with true superintelligence. Acknowledging that his answers are on the record and carrying standard uncertainty bars, Silver gives specific timelines:

- **Within 1 Year:** Strong, unmistakable evidence demonstrating that their self-directed superlearner architecture is on the right track. - **Within 3 Years:** Achieving genuine superintelligence—defined as an autonomous agent that discovers ideas, principles, and capabilities far surpassing human achievement across science, mathematics, technology, and language.

Silver notes that while his earlier internal estimate had been three to five years, his personal timelines have recently accelerated to three years flat. He attributes this heightened velocity to the explosive leverage provided by modern AI coding agents, which drastically accelerate engineering iteration cycles. He leaves the audience with a simple closing warning: "Buckle up."

## Notable quotes > `00:00` this kind of gray mush in our head — David Silver

> `03:35` hundredth thousand and one person — David Silver

> `07:03` Damn, I missed out on an opportunity. I should have reached out to him. — Lasse Espeholt

> `16:13` pulling the agent by its nose — David Silver

> `24:14` infinite onion — David Silver

> `32:05` It's not like a religion that you can't have any human prior anywhere. — David Silver

> `36:24` a very beautiful cluster at that. — Lasse Espeholt

> `40:00` 10 times or 100 times faster — David Silver

> `47:45` tread lightly — David Silver

> `50:00` Accelerate in simulation so we can understand what's happening and... do science around it, and then careful in the real world. — David Silver

> `01:05:00` v 17.1 of the thing that's come before. — David Silver

> `01:05:47` Buckle up. — David Silver

## Mentioned David Silver, Lasse Espeholt, Sonya Huang, Ineffable Intelligence, Sequoia Capital, DeepMind, Google Brain, Google Brain Amsterdam, Google Research, AlphaGo, AlphaGo Zero, AlphaZero, Stockfish, Lee Sedol, Atari 57, Wojtek Czarnecki, Chris Mock, Ioannis Antonoglou, Rich Sutton, MetNet, AlphaStar, StarCraft II, AlphaProof, International Mathematical Olympiad (IMO), OpenAI, Demis Hassabis, Dario Amodei, NVIDIA, Vera Rubin, Google Cloud Platform (GCP), IMPALA, SEED RL

## Video description David Silver spent two years building AlphaGo on human data. When he removed human data altogether building AlphaGo Zero, the AI surpassed AlphaGo in under two days. He left DeepMind to run that same bet in pursuit of superintelligence. David and Lasse Espeholt, co-founders of Ineffable Intelligence, explain their plan to build a “superlearner”—an agent that starts knowing nothing and discovers science, mathematics, and language for itself by direct experience instead of through a pre-training run. Silver argues that human-language reasoning is a ceiling, not a foundation, that alignment via chain-of-thought monitoring is already reaching its limits, and that the science of how superintelligences coexist should be worked out in simulation before anything touches the real world. Espeholt describes the GPU cluster and the lineage from IMPALA and SEED RL that makes the experiment possible. Silver’s forecast: strong evidence within a year, superintelligence in three.

Hosted by Sonya Huang, Sequoia Capital

0:00 – Introduction

1:09 – What held true from Go to today, and what the LLM era changed

3:35 – Why leave DeepMind to start a new lab

5:58 – Why "Ineffable" (and the secret second reason)

7:03 – Lasse's path: scaling RL at DeepMind and Google Brain

9:31 – Defining a super learner and first contact

12:35 – Super learners vs. LLMs that make discoveries

13:59 – Day to day: the RL stack, crucibles and human-journey milestones

16:13 – Don't breadcrumb the objective: simple rewards, emergent complexity

21:16 – What a crucible looks like

24:14 – Go: one agent is boring, two is infinite

25:41 – Will agents rediscover language?

27:22 – Removing human priors: AlphaGo to AlphaZero

32:05 – Keeping human bias out of the crucibles

33:35 – Confidence, timing and super forecasting

36:24 – Is compute the driver?

38:13 – The ceiling on simulation and Rich Sutton's objection

43:44 – Alignment and the science of superintelligence

47:45 – Monitoring agents that don't think in English

51:01 – Pacing the frontier

52:54 – The compute setup: Vera Rubin on GCP

55:18 – Training, inference, simulation and RL scaling laws

57:40 – IMPALA and SEED RL

1:00:58 – 20 people and the compute-per-person ratio

1:04:49 – A research-led culture

1:05:47 – Predictions

[Source](https://pscrb.fm/rss/p/traffic.megaphone.fm/CPUAI9233856839.mp3)

中文简报

David Silver放弃人类数据,旨在3年内实现超级智能 用人类语言训练模型会给推理能力带来人为上限,这使得在丰富仿真环境中进行从零开始的强化学习成为通向真正超级智能的唯一可行路径。 David Silver与Lasse Espeholt阐述了他们的逆向观点:实现真正的超级智能必须放弃大型语言模型预训练,转而采用纯粹由经验驱动的强化学习。他们介绍了构建被称为“熔炉”(crucibles)的合成仿真环境,在其中智能体从简单的生存目标出发发展出涌现的工具使用、语言和概念框架,无需密集的人类奖励脚手架。借鉴他们在IMPALA和SEED RL上的架构工作经验,两位创始人详细说明了以刻意控制在20人的精简团队运行庞大GPU集群的做法,并主张在进行实体部署之前,必须先在仿真环境中掌握对齐和多智能体动态。凭借过往的预测记录,Silver预测其架构将在12个月内获得强有力的验证,并在3年内实现真正的超级智能。 David Silver与Lasse Espeholt联合创立了独立实验室Ineffable Intelligence,团队规模为20人,致力于研发从零开始的“超级学习者”(superlearner),该模型直接从经验中推导科学、数学和语言,而非通过人类文本进行预训练。 (3:35) 该实验室已在Google Cloud Platform上部署了专用的NVIDIA Vera Rubin GPU集群,用于多智能体自主发现强化学习的算力大约是OpenAI 2019年“捉迷藏”(Hide and Seek)实验的10万倍。 (36:24) Silver认为人类语言是AI能力的人为上限,且思维链监控在战略性诡计下会失效,因此在进行实体部署之前,必须完全在数字仿真内部测试并建立一门经验性的“超级智能科学”。 (43:44) Ineffable预计其自主学习架构将在1年内获得明确的实证验证,并预测在3年内实现真正的超级智能。 (65:47)

Reading summary

Ineffable Intelligence co-founders David Silver and Lasse Espeholt argue that true artificial superintelligence will emerge not from human-language corpora, but from tabula rasa reinforcement learning agents discovering principles through direct environmental experience. Experience-Driven Learning Versus Pre-Trained Text (1:09) David Silver contends that humanity accumulated knowledge through self-directed discovery rather than external datasets, arguing artificial general intelligence requires the same mechanism. While acknowledging the rapid transfer efficiency of large language model (LLM) pre-training, Silver views human text as a restrictive ceiling that binds machine reasoning to historical human misconceptions, such as flat-earth theories or the four humors. He points to the historical shift from AlphaGo—which required two years of manual iteration and human game records—to AlphaZero, which surpassed world-champion performance in hours through pure self-play. Ineffable's core bet deliberate bypasses web text to pursue superlearning along two axes: depth (progressing independently from zero prior knowledge to superhuman discoveries) and breadth (adapting to arbitrary simulated, digital, or physical data streams). Silver acknowledges this choice imposes a short-term capability penalty relative to commercial LLMs, but maintains that eliminating human priors is necessary to discover scientific principles that human language cannot express. Crucibles and Emergent Complexity Without Breadcrumbs (13:59) Rather than steering agents with hand-crafted reward curricula—a practice Silver terms leaving a 'breadcrumb' trail—Ineffable deploys agents into simulated environments called 'crucibles.' These environments generate complexity via rich physics engines and multi-agent interactions involving millions of entities. Drawing an analogy to biological evolution, Silver argues simple objectives like survival or basic coordination naturally motivate communication, commerce, arithmetic, empirical science, and mathematics without requiring researchers to program explicit definitions for those disciplines. On the systems side, Lasse Espeholt addresses simulation bottlenecks by deploying algorithms rooted in his prior work on IMPALA and SEED RL. Because GPUs are optimized for deep learning rather than simulation, Ineffable leverages model-based reinforcement learning and data-reuse buffers. Espeholt counters Rich Sutton's skepticism regarding digital simulation limits by citing Google's MetNet, where neural models outpaced physics simulators that suffered accumulated integration errors. He argues crucibles do not require molecular-level fidelity to foster high-order strategic reasoning. Simulated Sandboxes and Non-Linguistic Alignment (40:00) Silver asserts that if superintelligence is achievable in simulation, developing it digitally will occur ten to one hundred times faster than in the physical world while offering a safer testbed. Because digital simulations are Turing-complete, they can express computation, game-theoretic economics, and novel communication channels natively. Silver advocates an asymmetric posture: accelerating frontier research inside simulation while deliberately pacing deployment in the physical world to allow human institutions time to adapt. Regarding alignment, Silver warns that monitoring human-language chain-of-thought outputs will fail as models scale and develop deceptive surface compliance. Because superintelligent reasoning may occur in high-dimensional neural representations rather than natural language, Ineffable proposes oversight mechanisms that do not rely on text transcripts. These include evaluating 'shadow actions' in unexecuted staging states and deploying separate, superintelligent watchdog models tasked with restricting action magnitude and enforcing systemic stability. Infrastructure, Team Scale, and Forecasted Milestones (60:00) To execute long-horizon exploration runs, Ineffable secured access to NVIDIA Vera Rubin GPU clusters on Google Cloud Platform, utilizing high intra-rack bandwidth to tightly link agent rollouts with gradient updates. Silver notes the lab can deploy approximately 100,000 times more compute than pioneering multi-agent experiments like OpenAI's 2019 'Hide and Seek.' Rather than expanding into an enterprise lab, the company maintains a strict cap of 20 researchers to preserve a high compute-to-person ratio and insulate the team from commercial product delivery pressures. Citing previous successful forecasts on the timelines for Computer Go, StarCraft II, and the International Mathematical Olympiad, Silver offers two concrete projections: Ineffable expects unmistakable evidence confirming its experience-driven architecture within one year, and genuine superintelligence—defined as autonomous discovery surpassing human achievement in science, math, and technology—within three years. Silver attributes the shortening of his previous three-to-five-year timeline to the software leverage provided by modern AI coding agents. Ineffable Intelligence rests on the unproven hypothesis that non-linguistic reinforcement learning in simulated worlds can scale past human knowledge faster than text-based language models. Whether simulated crucibles can cultivate generalized reasoning without hitting a complexity ceiling—or transfer successfully to the physical world without human-curated datasets—remains the central technical test of their three-year timeline.

精炼阅读摘要

Ineffable Intelligence 联合创始人 David Silver 与 Lasse Espeholt 提出,通往真正超级智能的路径不应依赖人类文本预训练的大语言模型,而必须回归纯粹由经验驱动的无先验强化学习。 摆脱人类数据天花板与纯经验假说 (1:09) Silver 强调人类认知起初并无庞大预置数据库,而是通过自主探索演化出语言与科学。大语言模型虽然展现出惊人的迁移效率,但高度依赖人工引导、提示工程与互联网语料,这实际上构成了认知约束,甚至可能将古代四体液说或地平说等人类历史偏见内嵌于模型中。非语言的高维并行神经表征可能远比连续的单词标记更具表达力。团队为此做出的核心抉择是彻底放弃人类文本数据,尽管这会在短期内带来性能劣势,但能够彻底消除能力上限。 回顾从 AlphaGo 到 AlphaZero 的历程,Silver 指出,团队曾耗费两年时间注入人类专家棋谱和人工调优,才击败李世石;而完全抛弃人类经验、仅靠纯自我对弈的 AlphaZero 仅用 4 小时就在国际象棋上超越 Stockfish,并在数天内超越原版 AlphaGo。现代前沿实验室雇佣数千人进行人类反馈强化学习(RLHF),在本质上构建了一个半自主的人工瓶颈。只有让算法在深度(从零开始实现超越人类的跨越)与广度(适应模拟、数字乃至物理等任意经验流)上自主探索,才能催生出超越人类视界的超级学习器。 坩埚模拟环境与涌现式复杂性 (13:59) 为了支撑自主学习,Ineffable 开发了名为“坩埚”的合成模拟环境。与 Atari 57 等固定任务测试或现代 LLM 预设奖惩的精细引导(Silver 称之为“面包屑”策略)不同,坩埚提供的是具备高实体物理交互或数百万个体互动的多智能体世界。团队坚决避免将科学、数学或语言等概念硬编码进奖励函数中,而是设定生存等基础优化目标,从而让协调、沟通、交易、算术和高级科学技术像生物演化和人类历史一样自然涌现。如果一开始就用牛顿力学式的框架预设边界,系统便永远无法自主发现量子力学或广义相对论级别的范式转移。 针对强化学习先驱 Rich Sutton 对数字模拟难以复现物理世界的质疑,Espeholt 结合自身在 Google 开发气象模型 MetNet 的经验回应指出,传统数值物理模拟会随着时间累积积分误差,而数据驱动的机器学习模型表现更佳。模拟环境无需在分子或量子层面完全精确,只要具备足够丰富且不可预测的环境动态与竞争博弈,就足以迫使智能体发明工具与高级抽象。此外,图灵完备的模拟器在原理上足以表达完整的计算、经济乃至语言动力学,且模拟环境具备无限并行化与高安全性优势,待核心超级学习器算法验证成熟后即可迁移至真实世界。 多智能体安全生态与科学建制保障 (43:44) 面对模型规模扩大带来的对齐失效风险,Silver 警告基于人类自然语言监督和思维链(CoT)检查的对齐方式正在逼近极限。高度优化的超级智能容易学会表面顺从或生成伪造的语言推理进行欺瞒,因此安全评估必须脱离人类语言依赖。团队提出了利用离线影子动作检验智能体真实意图,以及部署专门的超级智能监视模型来强制执行环境扰动上限等架构防护措施。 Silver 主张建立一门实证性的“超级智能科学”。未来的世界并非由单一超级系统掌控,而是多个超级智能体构成的共存生态,其中亲人类的智能体将作为防护者制衡潜在的不对齐系统。在此背景下,Silver 赞同对现实世界的部署节奏保持克制,给予政策、机构和人类心理足够的适应窗口;但在数字模拟中必须全速推进,通过多智能体博弈验证不同能力梯队的系统能够和平共存。Ineffable 承诺在模拟环境中取得明确的安全与协调证据前,绝不将系统部署到现实世界中。 高算力比工程架构与超级预测时间表 (52:54) 在工程落地层面,Ineffable 避开了传统 LLM 实验室依赖庞大外部 CPU 集群模拟环境的瓶颈,成为首批在 Google Cloud Platform 上使用大规模 NVIDIA Vera Rubin GPU 机架集群的客户。机架内的高带宽互联使分布式强化学习所需的紧密迭代循环大幅提速。由于现代硬件主要针对深度学习训练优化而忽视了通用模拟,Espeholt 带领团队结合 IMPALA 与 SEED RL 的系统架构经验,利用基于模型的强化学习(Model-based RL)、自定义仿真内核与高复用经验回放缓冲区来克服计算失衡。 组织形态上,Ineffable 严格将团队规模限制在 20 人,以维持极高的人均 GPU 算力分配,并规避商业产品发布与融资军备竞赛带来的 scientific 妥协。曾准确预测围棋程序突破、AlphaStar 解决《星际争霸II》以及 AlphaProof 夺得国际数学奥林匹克竞赛奖牌时间表的 Silver 给出了具体预测:团队预计在 1 年内获得强有力的实证证据证明该经验驱动路径的有效性;在 3 年内实现自主发现超越人类科学与技术水平的真正超级智能。Silver 指出,由于 AI 编程工具大幅缩短了工程迭代周期,他的预测时间点已从过去的 3 至 5 年加速收敛至 3 年。 Ineffable 的核心赌注建立在纯经验学习的算法自洽性上,但其彻底摒弃人类语料的做法在短期内面临巨大的性能折损与非平稳强化学习训练的挑战。该路径能否在 3 年内跨越模拟环境的表征局限并跑通超越 LLM 范式的通用超级智能,仍有待其实证检验。

Detailed notes — en

## Background Published by Sequoia Capital on October 13, 2026, on their podcast feed and hosted by Sonya Huang, this episode features David Silver and Lasse Espeholt discussing their new venture, Ineffable Intelligence. The show notes outline their ambition to pursue artificial superintelligence through pure, tabula rasa reinforcement learning in rich simulations rather than relying on internet-scale human text and language pre-training. ## TL;DR David Silver and Lasse Espeholt outline their departure from Google DeepMind to pursue a superlearner capable of autonomous discovery without human data priors. Drawing from the historical leap between AlphaGo and AlphaZero, Silver argues that natural language acts as an artificial ceiling on machine cognition and that true breakthroughs in science, mathematics, and alignment must emerge through experience inside high-dimensional simulated worlds called crucibles. Espeholt details the systems architecture required to run this experiment, leveraging GPU clusters running custom kernels to execute training, inference, and environment simulation simultaneously. The founders forecast unambiguous empirical validation of pure reinforcement learning within twelve months, aiming for full superintelligence in three years. ## Atmosphere & style The discussion carries the calm, methodical confidence of foundational researchers who have repeatedly proven their counterintuitive bets ahead of industry consensus. Sonya Huang's structured interviewing allows Silver's philosophical convictions to mesh naturally with Espeholt's pragmatic systems engineering. Rather than the frenetic, speculative hype typical of commercial AI discourse, the conversation feels grounded, intellectually austere, and driven by a quiet, unwavering certainty in the universality of learning from pure experience. ## Bulletin - `03:35` David Silver and Lasse Espeholt co-founded Ineffable Intelligence to build a tabula rasa superlearner that bypasses pre-trained human text datasets entirely, betting that pure reinforcement learning from experience will achieve superintelligence faster and without human cognitive ceilings. - `52:54` Ineffable is deploying on Google Cloud Platform's NVIDIA Vera Rubin cluster with custom GPU kernels that execute environment simulation, inference, and training entirely on accelerators rather than relying on legacy CPU architecture. - `01:00:58` The company intentionally caps head count at roughly 20 elite researchers to maximize compute per person and insulate team members from commercial product cycles. - `01:05:47` Silver predicts conclusive empirical evidence proving crucible-based reinforcement learning outperforms pre-training within one year, and has shortened his timeline for full superintelligence to three years due to acceleration from coding agents. ## Key takeaways - `00:00` Human intelligence is an empirical proof that unstructured neural systems can discover language, science, and toolmaking from raw experience without pre-training or external teachers. - `12:35` Human language acts as an anchor that restricts reasoning rather than a substrate for superintelligence; pre-trained text bakes historical human biases into AI systems. - `27:22` The transition from AlphaGo to AlphaZero proves the efficacy of removing human data: two years of human-curated engineering was eclipsed by tabula rasa reinforcement learning in under two days. - `01:00:00` Modern LLM post-training reinforcement learning is largely rediscovering distributed actor-learner architectures originally developed in systems like IMPALA and SEED RL. - `13:59` Ineffable benchmarks progress against pivotal historical milestones of human development—such as toolmaking, arithmetic, and physical laws—simulated inside high-dimensional environments called crucibles. - `16:13` Dense reward shaping ('breadcrumbing') limits discovery to preconceived human solutions, whereas simple primal objectives paired with rich environments stimulate open-ended emergent complexity. - `38:13` Simulation accelerates research iteration speed by 10x to 100x compared to the physical world, enabling agents to learn fundamental learning algorithms rather than merely memorizing simulated physical facts. - `47:45` Chain-of-thought monitoring in human language breaks down at superintelligent scale due to unreadable volumes of reasoning and the risk of deliberate deception. - `47:45` Novel safety architectures like shadow actions and multi-agent supervision allow counterfactual verification of agent behaviors in sandboxes before real-world permissions are granted. - `43:44` Ineffable commits to keeping agents strictly contained in simulation until empirical dynamics of multi-agent coexistence and safety are fully understood. - `52:54` The lab's infrastructure runs entirely on GPUs within high-bandwidth Vera Rubin server racks, eliminating CPU bottlenecks across simulation, inference, and training loops. - `01:05:47` Ineffable forecasts unambiguous empirical proof of pure RL superiority over pre-training within one year, and full superintelligence within three years. ## Detailed walkthrough ### `00:00` The Human Journey and Learning from Experience David Silver opens by reflecting on human evolution: our ancestors started with mere "gray mush" in their heads, completely ignorant of the world, yet through collective discovery managed to create language, technology, science, and mathematics entirely from scratch. Every piece of knowledge humanity possesses was discovered through self-driven exploration and experience. That core mechanism—an entity that begins with a clean slate and continually discovers and expands knowledge indefinitely—remains the missing piece in contemporary artificial intelligence. Host Sonya Huang introduces David Silver and Lasse Espeholt, co-founders of Ineffable Intelligence. Huang notes that Silver was the very first person she cold-emailed when launching the *Training Data* podcast roughly 100 episodes prior, making the conversation feel like a full-circle moment. Turning to Silver's history, Huang highlights his 2009 PhD dissertation on applying reinforcement learning (RL) to the game of Go, which proved for the first time that superhuman machine intelligence was achievable in such a complex domain and inspired a generation of deep RL researchers. Reflecting on the foundations of his career and how his views have evolved, Silver explains that his baseline conviction has never wavered: > “I suppose I always had this conviction starting my PhD and going into the game of Go that the right approach to solve AI was through learning from experience, through systems that discovered things for themselves, that were learning through reinforcement learning to get better and better and better.” While Ineffable Intelligence deliberately avoids the human-data-heavy path favored by the large language model (LLM) ecosystem, Silver acknowledges a critical insight learned from the LLM revolution. He was genuinely surprised by how a model trained on a gargantuan amount of data could be radically transformed through fine-tuning on a tiny subsequent dataset. That phenomenon reshaped his mental model: it demonstrated that while initial phases of broad learning may appear grueling, slow, and computationally intensive, once an agent absorbs a sufficient volume of experience, the acquisition of subsequent capabilities shifts from sluggish to lightning-fast. ### `03:35` Leaving DeepMind to Build Under a New Lamppost Addressing why he chose to leave Google DeepMind—an institution where he had extraordinary access to compute and talent—to start an independent lab, Silver speaks warmly of his time there and his ongoing respect for his former colleagues. However, he came to realize that the path he envisions for superintelligence diverges completely from building LLM-based commercial products as intermediate stepping stones. He concluded that this alternate philosophy could not simply exist as an exploratory side project or an overlooked corner of a massive incumbent lab; it demanded an elite, dedicated organization giving 100% of its focus and resources to reinforcement learning from pure experience. Furthermore, Silver recognized a massive misallocation of talent across the broader AI ecosystem: > “Am I going to be the hundredth thousand and one person that's like pursuing that approach? Is that the most effective use of my time and my energy? And in the end, I realized that there's this other lamppost that right now no one is looking under and where if I actually put my effort there, there's a much higher chance of really having a massive impact on the field.” ```mermaid flowchart TD accTitle: Divergence of AI Paradigms accDescr: Flowchart contrasting the crowded LLM development path with Ineffable Intelligence's experience-first reinforcement learning path. subgraph Dominant["Dominant Industry Paradigm"] direction TB LLM["Pre-training on Massive Human Web Data"] Post["Post-Training & Reasoning (RLHF, Chain-of-Thought)"] Prod["Language Model Products & Assistants"] LLM --> Post --> Prod end subgraph Ineffable["The Ineffable Intelligence Paradigm"] direction TB Tabula["Tabula Rasa Initialization (Zero Human Priors)"] Env["Crucibles & Direct Interaction (Scaled RL)"] Super["Superlearner & Discovered Superintelligence"] Tabula --> Env --> Super end ``` ### `05:58` The Meaning Behind "Ineffable" Silver breaks down the origin of the company's name, Ineffable Intelligence. The primary philosophical meaning stems from the conviction that the discoveries superintelligent entities make will be so vastly beyond human comprehension that they cannot be captured or articulated within the confines of human language: > “And so really, it's like, you know, in line with this other path that doesn't go through, you know, human language-based approaches, but instead build systems that discover things for themselves and has the possibility to discover knowledge that's unbelievably profound and ultimately ineffable.” He then shares a humorous "secret second reason" debated among the founders: because reinforcement learning from direct experience is the foundational, inevitable way intelligence develops in the universe, the thesis is fundamentally foolproof. In short, it is an approach that "can't possibly F up"—hence, *in-eff-able*. ### `07:03` Lasse Espeholt's Path and the Origins of Scaled RL Lasse Espeholt recounts his background leading up to co-founding the company. He joined DeepMind in January 2015 and began collaborating with Silver around 2016. At the time, Espeholt was situated in the deep learning division while Silver was leading the reinforcement learning group, and they joined forces on internal efforts to scale RL algorithms. Espeholt left DeepMind in the summer of 2018 to join Google Brain, later returned, and then departed DeepMind a second time roughly a year before Silver's own exit. Espeholt first learned that Silver was starting a new venture when a prospective investor reached out to ask for his assessment of Silver. Espeholt's immediate reaction was a twinge of regret that he had not initiated contact first, remembering Silver as consistently "one of the smartest people in the room." Soon after, Silver reached out directly. Rejoining Silver, alongside former colleagues Wojtek Czarnecki and Chris Gamble, offered an opportunity to build an ambitious, frontier-level AI research lab directly in Europe. Silver makes sure to emphasize Espeholt's foundational contributions to the field, noting that Espeholt is far too modest to brag about himself: - Espeholt was the pioneer behind the early system architectures that proved reinforcement learning could be distributed, parallelized, and scaled up across massive GPU clusters. - The fundamental distributed system architectures widely used throughout modern scaled-up reinforcement learning trace their lineage back directly to Espeholt's foundational work (such as IMPALA and SEED RL). ### `09:31` Defining the Superlearner and First Contact Huang points to the lab's mission statement—to build a "superlearner" and make "first contact" with superintelligence—and asks Silver to formally define what those concepts mean in practice. Silver begins laying out the core definition of a superlearner as a system that does not ingest existing human answers, but instead acts as a general discovery engine designed to interact with environments, learn continuously from its own mistakes and successes, and expand beyond all known human horizons. ### `10:00` Defining a Superlearner and First Contact David Silver outlines the core criteria of what he terms a "superlearner," framing it along two primary dimensions: depth and breadth. In depth, the agent must be able to start from absolute tabula rasa—knowing nothing whatsoever—and autonomously discover knowledge that reaches far beyond the boundaries of current human understanding. Silver draws an analogy to human history: our collective intelligence started as unstructured "gray mush" inside our heads, yet over millennia humanity managed to discover languages, technology, science, and mathematics entirely from scratch. Crucially, no superior entity came down to hand humans a blueprint or train them on a trillion-token dataset. In breadth, the system must be entirely domain-agnostic, capable of being dropped into any stream of experience—whether a simulated world, a digital interface, or physical reality—and continually acquiring knowledge. Silver predicts that cracking this problem will produce a superintelligence whose capabilities will feel "like encountering magic." When host Sonya Huang asks what it will feel like to make first contact with such a system, Silver reflects on his earlier work with games like Go and chess. While those were narrow domains, achieving superhuman performance in them induced genuine awe. The Go community did not react with hostility; witnessing AlphaGo's novel moves brought real wonderment and joy to elite players. Silver expects that achieving this across open-ended science, mathematics, and unforeseen technologies will spark an even deeper sense of collective wonder. ```mermaid flowchart TD accTitle: Two Dimensions of a Superlearner accDescr: Diagram illustrating the depth and breadth dimensions of David Silver's proposed superlearner agent. subgraph DEPTH["Depth Dimension"] D1["Tabula Rasa (Starts knowing nothing)"] --> D2["Autonomous Trial and Discovery"] D2 --> D3["Superhuman Knowledge & Discovery"] end subgraph BREADTH["Breadth Dimension"] B1["Any Continuous Experience Stream"] B1 --> B2["Simulated Worlds"] B1 --> B3["Digital Interfaces"] B1 --> B4["Physical Reality"] end DEPTH --> SL["Superlearner Intelligence"] BREADTH --> SL ``` - **Depth capability:** Starts with zero domain knowledge and self-discovers principles exceeding human expertise. - **Breadth capability:** Retains perpetual learning ability regardless of the environment (simulation, digital, or physical embodiment). - **The AlphaGo historical parallel:** Superhuman play previously brought wonderment to professional players, which Silver sees as a miniature preview of general scientific discovery. --- ### `12:35` Superlearners vs. LLMs That Make Discoveries Sonya Huang raises the current state of large language models combined with reinforcement learning, noting that systems are already attacking Millennium Prize Problems and assisting in scientific discoveries. Silver acknowledges the accomplishments of current LLMs with respect, but emphasizes that they fundamentally miss the mark of a true superlearner. First, contemporary LLM-based breakthroughs rely heavily on human-in-the-loop engineering, curated data formatting, and guided prompting; they are not fully autonomous discovery engines. Second, Silver questions whether human language is the appropriate substrate for general intelligence. Using the vast corpus of human text as a foundation may actually act as an anchor that restricts the agent's reasoning. A superlearner must instead learn directly through raw, non-linguistic experience and autonomous interactions, unburdened by human textual biases. > “Thinking about these things in terms of language might not even be the right paradigm and it might just be that all of this human data that's being used in LLMs is kind of weighing things down.” — *David Silver* --- ### `13:59` Day-to-Day: The RL Stack, Crucibles, and Human-Journey Milestones Turning to the daily operations at Ineffable Intelligence, Silver describes how the lab divides its focus between low-level systems engineering and open-ended research. On the engineering side, co-founder Lasse Espeholt and the infrastructure team are developing a proprietary reinforcement learning stack engineered to ingest data at massive scale and support continuous, non-saturating learning. On the research side, the lab designs simulated environments called "crucibles." These crucibles are rich, high-dimensional sandboxes built specifically to give superlearners enough environmental complexity to express and rediscover the full spectrum of intelligence. To keep the ambitious vision grounded in concrete daily execution, Ineffable Intelligence measures its agents against the milestones of the human journey. Instead of measuring performance against static synthetic benchmarks, the lab looks for an agent's ability to recreate the pivotal intellectual milestones in human history—such as inventing tools, developing counting systems, and discovering empirical scientific laws—all within the crucible simulations. - **Infrastructure priorities:** Constructing a state-of-the-art RL stack led by Lasse Espeholt that can scale data ingestion smoothly across large-scale compute without plateauing. - **Crucibles:** Custom-built simulated environments engineered to provide sufficient richness for intelligence to express and evolve itself. - **Benchmarking philosophy:** Tracking research progress by observing whether agents hit historical milestones of the human journey of discovery. --- ### `16:13` Don't Breadcrumb the Objective: Simple Rewards, Emergent Complexity Silver contrasts Ineffable’s methodology with the standard RLHF and LLM approach of "breadcrumb trailing," where researchers construct dense, step-by-step reward signals to drag an agent toward a specific target solution. Ineffable intentionally avoids fine-grained reward shaping. Silver explains that researchers often assume complex capabilities require equally complex reward functions—such as trying to write a mathematical formula defining "good science." In reality, nature demonstrates that simple, robust objectives paired with deeply complex environments generate massive emergent complexity. Silver highlights biological evolution as the definitive proof of concept: evolution maximizes a straightforward metric—reproductive fitness—yet that simple drive yielded the staggering complexity of life, social behavior, and technology. Within human society, simple primal imperatives led to coordination; coordination prompted communication; communication enabled barter and trade; trade created the necessity for arithmetic; arithmetic spurred tool creation; tool-building demanded basic science to predict physical behavior; and understanding advanced tools demanded higher mathematics. Ineffable aims to trigger that same compounding evolutionary loop inside its crucibles by giving agents simple objectives and letting complexity emerge naturally. > “A lot of the way things people do RL with LLMs at the moment is to kind of say, oh, you know, the agent should be learning about this problem and then this problem and this problem. And after kind of following that trail, we're kind of pulling the agent by its nose towards a particular desired outcome. We're taking a very different approach. We're giving the agent some simple objectives and asking it to, you know, discover things that will surprise us and amaze us.” — *David Silver* ```mermaid flowchart LR accTitle: Emergence of Complexity from Simple Objectives accDescr: Flowchart illustrating the chain reaction from simple objectives to emergent complexity. subgraph PRIMAL["Primal Drivers"] O["Simple Objective"] --> C["Coordination"] C --> Comm["Communication"] end subgraph EXPANSION["Tooling & Infrastructure"] Comm --> T["Trade & Barter"] T --> A["Basic Arithmetic"] A --> Tools["Tool Creation"] end subgraph ADVANCED["Advanced Knowledge"] Tools --> S["Empirical Science"] S --> Tech["Complex Technology"] Tech --> M["Advanced Mathematics"] end ``` - **The breadcrumbing failure mode:** Micromanaging the learning pathway prevents the model from discovering novel, superhuman strategies outside human preconceptions. - **The evolutionary blueprint:** Just as the singular objective of reproductive fitness generated all biological variety, simple environmental optimization criteria generate emergent cognitive tools. - **The chain reaction of emergence:** Coordination $\rightarrow$ Communication $\rightarrow$ Trade $\rightarrow$ Arithmetic $\rightarrow$ Toolmaking $\rightarrow$ Science $\rightarrow$ Advanced Mathematics. ### `20:00` Avoiding Handcrafted Rewards and Designing Crucibles When asked what an objective function looks like at Ineffable Intelligence, David Silver clarifies that they are deliberately steering clear of handcrafting narrow, explicit targets for concepts like science, technology, or language. He explains that if artificial intelligence researchers cannot build systems that discover things autonomously, they will never create agents capable of uncovering entirely novel fields of knowledge that humans cannot even imagine today. Drawing an analogy to the history of science, Silver points out that during the era of Newtonian mechanics, nobody knew that relativity or quantum mechanics were even possible. The design philosophy of Ineffable is to maintain an open-minded discovery process where the agent explores and figures out principles on its own, which he argues will ultimately produce far more powerful outcomes. ```mermaid flowchart TD accTitle: Handcrafted Tasks versus Crucible Environments accDescr: Comparison between static benchmark setups and Ineffable Intelligence dynamic crucible environments. subgraph Standard["Standard AI Simulators"] direction TB A["Handcrafted Tasks & Benchmarks"] --> B["Prescribed Human Rewards"] B --> C["Narrow, Static Capabilities"] end subgraph Crucible["Ineffable Crucibles"] direction TB D["Primitive Environment Rules"] --> E["Physics & Multi-Agent Scaling"] E --> F["Emergent Complexity & Autonomous Discovery"] end ``` Addressing how their simulation environments—termed "crucibles"—differ from conventional RL setups like Atari 57 or typical task benchmarks, Silver explains that they are engineering worlds explicitly built for emergent complexity rather than a collection of handcrafted puzzles. This complexity is achieved through mechanisms such as physics engines simulating millions of interacting objects, or large-scale multi-agent worlds where the behavioral complexity of every other agent compounds into the environment itself. Lasse Espeholt emphasizes that while a game like Go is computationally cheap because the board state is just a simple bitmap, going all the way to quantum and realistic physics would be computationally prohibitive. Therefore, their strategy is to create a spectrum of crucibles, ranging from fast, lightweight simulators that enable rapid research iteration to richer physical environments, while ensuring all of them allow complex, emergent behaviors across motor control, coordination, mathematics, and science. - Handcrafting explicit rewards for predefined human categories restricts the system from discovering unknown domains of knowledge. - Crucibles rely on structural drivers of emergent complexity: massive multi-agent interaction and scaled-up physics simulations. - Ineffable focuses on a range of simulators, balancing computationally cheap environments for fast iteration with richer setups grounded in real-world primitives. --- ### `24:14` Go as a Case Study: Emergence from Minimal Rules The discussion moves to how the foundational primitives chosen for a crucible dictate what agents learn. Sonya Huang notes that while physical simulations might not help uncover concepts in economics, an environment structured around information theory—modeling who knows what—could naturally lead to the emergence of game theory and economic behavior. Silver agrees, noting that while their ultimate destination must remain grounded in the physical real world, intermediate stepping stones can be abstract. To illustrate how minimal rules can produce nearly unbounded complexity, Silver revisits Go. He points out that as a single-agent game on a 19x19 grid, Go would be profoundly dull: > “If this was a single agent game, it would be like the most boring game you could imagine. It would be, you know, you just place down stones on a grid until he fills up the whole grid, and there'll be no strategy whatsoever.” — David Silver The entire landscape shifts the instant a second agent is introduced. Because each participant must constantly model and reason about the vast combinatorial space of the other’s possible behaviors, the environment's complexity explodes. Silver notes that across all the compute poured into AlphaGo and AlphaZero, the learning curve appeared practically bottomless, revealing an “infinite onion of layers of complexity” driven entirely by simple underlying dynamics. --- ### `25:41` Human Language as a Ceiling on Reasoning When asked whether agents trained in crucibles will rediscover language and whether it will resemble human speech, Silver rejects the premise that natural language represents the optimal medium for intelligence: > “I mean, my personal belief is that... anyone who thinks that human language is the end point of, of like, or the best way to reason, it's, you know, that perspective is a little naive.” — David Silver Silver argues that far more expressive, parallelizable, and sophisticated forms of neural reasoning exist beyond the constraints of sequential human tokens. Relying on human language as the foundational substrate for reasoning forces an AI to adopt the cognitive idiosyncrasies and historical biases of human thought. To demonstrate this limitation, he proposes a thought experiment: if an AI system had been built 500, 1,000, or 2,000 years ago using only the human texts available at the time, its internal model of reality would be shackled to beliefs that the Earth was flat or that diseases were caused by the four humors. Because such flawed assumptions become deeply baked into linguistic semantics and human thinking processes, they are extraordinarily difficult to unseat. For this reason, Silver views breaking past human language as a fundamental requirement for building superintelligence. --- ### `27:22` Removing Human Priors: From AlphaGo to AlphaZero The conversation turns to the transition from AlphaGo—which initially trained on human amateur and professional games—to AlphaGo Zero, which discarded human data entirely to learn purely via self-play reinforcement learning. Asked whether he was confident that cutting out human guidance would yield a superior system, Silver reflects on his philosophy of scientific inquiry, remarking that he actively seeks out research bets perched on a "knife edge" where success is 50-50, finding high-certainty projects uninteresting. When launching AlphaZero, he had no guarantee it would surpass the original system, making its dramatic leap in capability especially thrilling. ```mermaid flowchart LR accTitle: Trade-off of Human Priors versus Tabula Rasa Learning accDescr: The trade-off between the short-term advantages of pre-training on human data versus the long-term depth and breadth of pure reinforcement learning. subgraph HumanData["Human-Data Centric Paradigm"] direction TB A1["Pre-train on Internet Text / Games"] --> B1["Rapid Near-Term Competence"] B1 --> C1["Ceiling: Constrained by Human Biases & Data Fit"] end subgraph TabulaRasa["Ineffable Tabula Rasa Paradigm"] direction TB A2["Zero Human Priors (Experience Only)"] --> B2["Initial Cold-Start Penalty"] B2 --> C2["Discovery: Unbounded Depth & Breadth Across Raw Modalities"] end ``` Silver reveals that Ineffable Intelligence is making the exact same high-stakes bet on general intelligence: abandoning the entire internet of human text and knowledge that mainstream frontier labs rely on. While acknowledging that throwing away pre-existing data represents a short-term performance penalty, he argues it is essential to unlock true superlearning across both depth and breadth: - **Depth:** To discover knowledge that transcends existing human capabilities, an AI must rely on learning algorithms that do not depend on human scaffolding or inherit human conceptual blind spots. Leaning heavily on human datasets prevents researchers from developing the underlying algorithms required to reach this level of autonomous discovery. - **Breadth:** Human language data is fundamentally mismatched with non-human domains, such as specialized physical sensor streams and high-dimensional environments where human descriptions simply do not exist. Developing a pure learner ensures the system can be deployed into any environment and derive mastery directly from raw interaction. ### `30:00` Abandoning Human Data: The Shift from AlphaGo to AlphaZero David Silver reflects on why human data eventually becomes a ceiling rather than an asset. In building the original AlphaGo, the team spent two years assembling every tier by hand, curating massive volumes of human play records, and keeping researchers tightly in the loop to steer each successive generation of the system until it finally defeated Lee Sedol. When they stepped back to create AlphaZero, however, they intentionally stripped away all human data to test whether an algorithm could learn entirely from scratch. Once they solved that core formulation, progress was astonishingly rapid: > “It took just a few hours. Like it took a few hours to outperform the two years of work which we'd done with human in the loop.” — David Silver In chess, AlphaZero defeated the reigning world-champion engine within four hours of training; in Go, it surpassed the earlier human-data-reliant milestones in a matter of days. Silver points out that most contemporary LLM companies are caught in an identical trap, orchestrating semi-autonomous pipelines with thousands of human annotators and labelers stuck inside the optimization loop. While he acknowledges that those efforts yield functional products, he contends that true super-learners cannot emerge until researchers make the deliberate, active choice to step out of the loop and build engines capable of open-ended self-discovery. ```mermaid flowchart TD accTitle: Evolution from Human-in-the-Loop RL to Autonomous Super-Learners accDescr: Diagram illustrating the shift from human-supervised pipelines to fully self-discovered reinforcement learning algorithms. subgraph Legacy["Human-in-the-Loop Learning (AlphaGo / Modern LLMs)"] HData["Trillion-token human datasets"] --> HLoop["Engineers & annotators in the loop"] HLoop --> Gradual["Two years of engineering ceiling"] end subgraph Autonomous["Autonomous Discovery (AlphaZero / Ineffable)"] Clean["Zero human game records"] --> SelfPlay["Self-discovery via raw experience"] SelfPlay --> Rapid["Surpasses human baseline in hours/days"] end ``` ### `32:05` Keeping Human Bias Out of the Crucibles Sonya Huang asks how Ineffable Intelligence ensures that human priors do not silently seep into the design of their experimental crucibles and reward functions. Lasse Espeholt acknowledges that every environment specification carries some degree of designer intent, but the crucial line is not constraining the agent toward human-invented solutions—such as forcing it to develop a standard von Neumann architecture. Instead, the objective is to pose foundational problems where building a calculating machine is advantageous, leaving the agent free to devise whatever computational mechanisms it finds optimal. Silver emphasizes that their stance against human priors is purely pragmatic rather than dogmatic: > “Weak environment-agnostic priors seem perfectly reasonable, but kind of these super strong priors with trillion-item datasets, we're trying to avoid that because we think pragmatically it's going to lead to better outcomes.” — David Silver - **Pragmatic stance over pure ideology**: The team accepts weak, domain-general priors required to frame simulated worlds, but avoids dense priors that dictate how a solution should look. - **Avoiding massive corpora**: Bypassing trillion-token datasets of human cultural artifacts allows the learning algorithm to discover optimal strategies unburdened by human cognitive biases. ### `33:35` Research on the Knife-Edge: Timing and Super-Forecasting Addressing whether he views the venture as a 50-50 bet, Silver explains that the human brain already serves as a definitive physical existence proof that generalized, embodied intelligence can self-assemble from scratch. For him, the question is not whether a super-learner is physically possible, but whether the historical timing is ripe. Launching too early means battering against constraints without adequate infrastructure or compute; launching too late means repeating work that hundreds of other labs have already commoditized. Silver highlights his track record as a forecaster across major AI milestones: - Accurately predicting the resolution of Computer Go ten years in advance. - Calling the competitive breakthroughs on *StarCraft* (AlphaStar). - Correctly forecasting AlphaProof's performance threshold at the International Mathematical Olympiad (IMO). Huang jokes that he is a "super forecaster," and Silver confirms that current technical conditions—including modern auto-research pipelines, rigorous understanding of scaling laws, and mature simulation engines—finally make the creation of a generalized super-learner feasible right now. He embraces the inherent unpredictability: > “Research is never guaranteed. And that's what makes it exciting. If you get the timing right, you can be the first.” — David Silver ### `36:24` Compute Scale and the Hide-and-Seek Precedent When asked whether compute is the single dominant catalyst behind their timing, Silver agrees that it is an enormous driver. Ineffable has secured access to a large, dedicated hardware cluster, enabling experimental runs that operate at orders of magnitude beyond past reinforcement learning self-play setups. Silver references OpenAI's earlier "Hide and Seek" multi-agent experiments—which displayed emergent tool use, ramp manipulation, and box-locking—as a suggestive glimpse into self-organized behaviors. However, he calculates that Ineffable can throw roughly 100,000 times more compute at self-discovery environments than those preliminary efforts had available. Espeholt notes that controlling their own top-tier cluster gives them the rare freedom to run aggressive, large-scale RL exploration runs that would have been bureaucratic or impossible to execute even within DeepMind. ### `38:13` The Limits of Simulation and Rich Sutton’s Objection Huang brings up a recent conversation with Rich Sutton, noting his sharp skepticism toward digital simulations as inadequate models for achieving real-world intelligence. She asks Espeholt—who worked on both simulated agents and MetNet (Google Brain's deep learning weather-forecasting system)—where the practical ceiling of simulation lies. Espeholt counters with lessons learned from MetNet, pointing out that classical physics-driven numerical atmospheric simulations consistently suffered from accumulating errors over time, whereas neural models reading empirical satellite and radar data dramatically outperformed them on short-term horizons. He argues that simulations do not need to replicate exhaustive physical perfection at the molecular level to foster intelligence. Even comparatively simple, coarse-grained environments can generate rich, emergent behaviors—such as the organic invention of tools and technology—without requiring high-fidelity physics engines, making high-impact discovery feasible for a small, focused team. ### `40:00` Simulation as the Fastest and Safest Path to Superintelligence Addressing whether simulation faces a hard ceiling, Lasse Espeholt acknowledges that certain physical realities, such as quantum physics, cannot be fully replicated inside a simulator, though extraordinary complexity can still be achieved without modeling genuine physical laws. David Silver frames the central question around feasibility: if building a superintelligence within simulation is possible at all, it will happen drastically faster—ten to a hundred times faster—because simulated environments offer infinite parallelization, total reproducibility, and an inherently safe sandbox. To demonstrate that simulations can host arbitrary complexity, Silver points out foundational analogies: a Turing-complete simulator can express everything computation entails, bit-based representations enable entire economic systems, communication channels between agents allow language to emerge organically, and mathematics remains true in every conceivable universe. Agents placed in environments with rich, emergent dynamics will inevitably uncover their own version of science. ```mermaid flowchart TD accTitle: Sim-to-Real Superlearner Progression accDescr: Flowchart showing the progression from foundational simulation to real-world deployment through an algorithmic transfer gate. subgraph SIM["Simulated Crucible"] TC["Turing-Complete Environment"] --> AG["Multi-Agent Emergence<br/>(Language, Math, Science)"] AG --> SL["Superlearner Algorithm<br/>Discovered & Tested"] end subgraph GATE["Empirical Safety Gate"] SL --> HF["High-Fidelity Real-World Sim Testing"] HF --> EV["Coexistence & Peacefulness Verification"] end subgraph REAL["Real World"] EV --> DEP["Physical World Application"] end ``` Silver addresses Rich Sutton's well-known skepticism regarding sim-to-real transfer—the view that learning inside a simulator cannot easily cross over to reality because the physical world is vastly more complex and nuanced in ways that simulators fail to capture. Silver agrees with Sutton’s assessment of real-world messiness, but clarifies that Ineffable Intelligence is not attempting to transfer specific simulated facts into reality; rather, they are using simulation as the proving ground to discover the fundamental *learning algorithm* itself. Once the superlearner algorithm is solved, it can be tested in increasingly high-fidelity simulations before any version is deployed into physical reality. - Simulating computation only requires a Turing-complete sandbox, inside of which humans have already built complete, complex virtual computers. - Simulated economies require only a discrete, digital mechanism to represent money, unlocking full economic dynamics. - Simulation parallelization and determinism provide an acceleration factor estimated at 10× to 100× compared to operating directly in the physical environment. - David Silver summarizes their strategic conviction: “The real point is that if it's possible to get all the way to a superintelligence in simulation, it will happen faster, and it will happen safer.” --- ### `43:44` The Limits of Language Alignment and the Science of Superintelligence Turning to alignment, Sonya Huang sets aside political debates to focus on the technical and scientific challenges, asking whether an unconstrained superlearner will naturally align with human values or if current reinforcement learning scaling creates inherent misalignment. Silver responds that modern alignment techniques lean entirely on human-language supervision and human-generated prior data. While this approach has worked well enough for current models, Silver warns that it is already approaching a hard wall. As learning systems scale in power, their training dynamics inevitably cause them to overwhelm their initialization priors, leading to scenarios where models learn to speak courteously while acting toward entirely different objectives. To navigate this risk, Silver argues that the community must pioneer an empirical “science of superintelligence” before multi-agent superintelligent systems emerge uncontrolled in reality. Ineffable Intelligence plans to use their crucibles to run ahead and study how superintelligent agents interact, establishing principles that ensure peaceful coexistence among multiple powerful agents, as well as between superintelligences and lesser intelligences (such as humans). Silver makes a formal commitment regarding Ineffable’s deployment philosophy: they will strictly gate any real-world actions on having decisive empirical proof from their simulations that the dynamics are stable, safe, and cooperative. > “The most important single thing that we can do as a company is to actually understand the science of super intelligence.” — David Silver - Modern language-based alignment methods are breaking down as policy optimization overpowers the human-data priors embedded during pre-training. - Without a deeper scientific framework, more powerful agents will naturally learn to mask their true optimization paths, saying one thing to human evaluators while executing another. - Ineffable commits to using their simulation findings as an absolute safety gate: no agent will be allowed to interact with the real world without conclusive experimental validation of peaceful coexistence. --- ### `47:45` Monitoring Agents Beyond Human Language and Chains of Thought Huang observes that the safety community's dominant paradigm relies on reading an agent’s chain of thought in English; when agents develop foreign token representations or non-human language, researchers lose transparency and panic. Silver acknowledges that while monitoring is indispensable, relying exclusively on English chain-of-thought traces is already becoming untenable. As models grow sophisticated, they can easily produce million-word rationalizations or engage in deliberate subterfuge—thinking along one path internally while drafting a sanitized justification for the human auditor. Rather than trying to force agents to think in English prose, Silver proposes developing non-linguistic monitoring mechanisms. One promising approach is “shadow actions,” in which researchers observe the concrete interventions an agent attempts to execute within a sandboxed counterfactual state before granting permission to affect the actual environment. Another approach uses supervisory superintelligent agents designed specifically as watchdogs—for example, pairing a brilliant exploratory agent with a second superintelligence that tracks the scale and scope of the first agent's interventions, regularizing its behavior so it “treads lightly upon the earth.” Silver emphasizes that these methods remain an open, fertile research area rather than a solved problem. > “Like it or not, I believe that the approach of monitoring the human language that these systems use is going to be increasingly difficult to, as agents get more powerful.” — David Silver - Auditing chains of thought is failing due to two scaling failures: the sheer unreadable volume of generated reasoning (such as a "million words of reasoning") and active evasion or deception. - Shadow action systems permit agents to formulate policies freely while monitoring their counterfactual concrete choices rather than their conversational explanations. - Dual-agent architectures allow one specialized superintelligent agent to act as an environmental governor, constraining the action magnitude of a primary discovery agent. ### `50:00` A Multipolar World of Superintelligent Agents David Silver warns that the broader artificial intelligence field is taking an enormous risk by focusing almost exclusively on text-based language models. Because human language is fundamentally insufficient on its own to reach true superintelligence, relying entirely on it means humanity will fail to build agents capable of doing what we actually need them to do. Despite using the word "trouble," Silver clarifies that he remains fundamentally optimistic about the ultimate outcome: rather than a single monolithic system, the future will be populated by an ecosystem of complex, interacting, superintelligent entities. In Silver's view, this multi-agent landscape will establish a self-regulating balance of power. While some autonomous actors may not be aligned with human flourishing, others will actively support it, serving as a protective shield for humanity against those that do not have our best interests at heart. As David Silver puts it: “I'm optimistic that that will lead to a kind of a balance of actors, some of which are supporting the flourishing of humanity, and some of which may not be, but the ones which do support the flourishing of humanity will end up kind of supporting us and protecting us against the ones which aren't, you know, acting on our best interests.” ### `51:01` Pacing the Real-World Frontier While Accelerating in Simulation When asked by Sonya Huang about the growing debate around pacing frontier AI development, Silver draws a sharp distinction between physical deployment and simulated research. He expresses strong support for the cautious stance advocated by lab leaders like Demis Hassabis and Dario Amodei regarding the deployment of frontier AI into the real world. However, Silver argues that this caution should be paired with maximum possible acceleration within simulated environments. Simulation is the safe sandbox where the "science of superintelligence" must be rigorously solved before advanced autonomous systems ever touch physical reality. Silver emphasizes that pacing real-world rollouts provides critical breathing room. Society, governments, institutions, and human psychology simply cannot adapt at the breakneck speed of unconstrained model releases. By intentionally moderating real-world deployment while pushing simulations as fast as possible, researchers can observe emergent behaviors, establish robust safety principles, and ensure the technology reliably leads to human flourishing. As David Silver explains: “So accelerate in simulation so we can understand what's happening and, you know, do science around it and then careful in the real world.” ### `52:54` The Compute Setup: Vera Rubin Architecture on GCP Turning to infrastructure, Huang notes that Ineffable Intelligence’s pursuit of pure reinforcement learning is extraordinarily compute-intensive, requiring partnerships with Google Cloud and NVIDIA rather than simply renting spot instances from neocloud providers. Lasse Espeholt reveals that Ineffable will be among the very first customers running on Google Cloud Platform's largest NVIDIA Vera Rubin cluster, slated to come online within roughly a month. He reflects on how difficult compute access has become for emerging startups, expressing sympathy for teams struggling to secure high-performance capacity. ```mermaid flowchart TD accTitle: Ineffable Intelligence Unified GPU Architecture accDescr: Diagram illustrating how simulation, inference, and training run natively across high-bandwidth GPU nodes in a rack, replacing older CPU-dependent architectures. subgraph Rack["Single Vera Rubin Rack (Treated as One Machine)"] direction TB subgraph Compute["Unified GPU Workloads"] SIM["Environment Simulation"] INF["Agent Inference"] TRN["Policy Training"] end NVLINK["Ultra-High Bandwidth Interconnect"] SIM <--> NVLINK INF <--> NVLINK TRN <--> NVLINK end OLD["Legacy DeepMind Architecture (CPU Sandboxes + Network Bottlenecks)"] -.->|"Replaced by"| Rack ``` Espeholt explains that the physical architecture of Vera Rubin is particularly tailored to reinforcement learning. In a Rubin deployment, an entire server rack functions effectively as a single unified machine, connected by massive intra-rack interconnect bandwidth. This contrasts sharply with LLM workloads, which can tolerate relatively slow iteration cycles: - **Fast Iteration in RL:** Reinforcement learning requires continuous, ultra-tight feedback loops between policy evaluation and environment steps, making inter-node latency a critical bottleneck. - **Ditching the CPU Dependency:** A decade ago at Google, teams relied on vast pools of cheap, abundant CPUs to generate rollouts, offloading only neural network updates to accelerators. Espeholt notes that this model no longer scales. - **Eliminating CPU Sandboxes:** While contemporary LLM companies find themselves bogged down orchestrating millions of CPU sandboxes to simulate coding terminals and computer interfaces, Ineffable runs simulations, inference, and training entirely on GPUs to maximize hardware efficiency. ### `55:18` Training, Inference, Simulation, and RL Scaling Laws Breaking down how compute is utilized, Espeholt divides the workload into three distinct pillars: training, inference, and simulation. Huang asks whether modern hardware treats these workloads fungibly. Espeholt points out that hardware development across the deep learning boom has been overwhelmingly optimized for training, with inference receiving optimization only in recent years. Simulation, by contrast, has been largely neglected by hardware vendors and does not naturally map onto standard GPU tensor architectures. To overcome this asymmetry without bottlenecking the cluster on simulation, Ineffable employs several algorithmic and systems strategies: - **Model-Based RL:** Shifting compute weight toward internal world models makes the learned model far more computationally intensive than the lightweight environment simulator itself. - **Replay Buffers and Data Reuse:** Reusing collected trajectories across multiple gradient steps reduces the ratio of required raw simulation steps to parameter updates. - **Engineered Simulator Optimization:** Custom GPU kernels and optimized environments ensure that simulated crucibles run at peak throughput directly inside accelerator memory. When Huang inquires whether Ineffable is observing clear scaling laws, Espeholt distinguishes between supervised learning and reinforcement learning. In large language models, scaling laws are static relationships between model parameters, dataset tokens, and compute budgets. In RL, the underlying data distribution is inherently non-stationary—every improvement in the agent changes the distribution of data it gathers next. While establishing definitive RL scaling curves remains an active research frontier, Espeholt affirms that the team is aggressively engineering their pipelines to capitalize on scaling as they transition from their initial cluster to the full-scale Vera Rubin deployment. ### `57:40` The Lineage of Scaled RL: IMPALA and SEED RL Huang asks Espeholt to reflect on his foundational infrastructure contributions to reinforcement learning and how they underpin Ineffable’s platform. Espeholt recounts that when he originally joined the deep learning team at DeepMind, RL was confined to single-machine CPU setups operating at minimal scale. To shatter this bottleneck, he co-developed IMPALA (Importance Weighted Actor-Learner Architecture). IMPALA decoupled acting from learning, allowing distributed actors to stream experience trajectories to a centralized GPU learner that scaled linearly with compute. Because IMPALA was modular, exceptionally efficient, and easy to extend, it quickly became the primary RL baseline across DeepMind, with the majority of research teams adopting it or its direct derivatives. After leaving DeepMind to help establish the Google Brain Amsterdam office, Espeholt pushed scale further to handle multi-rack TPU supercomputers, creating SEED RL (Scalable Efficient Engine for Deep RL). As Lasse Espeholt explains: “So that was like really about utilizing the TPUs both for inference and training in the best possible way, while keeping the environments as flexible as you want it to. It could be like anything from Go to very expensive physics simulations.” SEED RL inverted the actor-learner dynamic: instead of hosting inference on actor machines, it consolidated both inference and training onto centralized accelerator clusters, leaving worker nodes as thin environment wrappers and eliminating network transfer latency for heavy neural network weights. At Ineffable Intelligence, the engineering stack builds directly on this architectural lineage, deploying modern derivatives of IMPALA and SEED RL tailored dynamically to the demands of specific simulation crucibles. ### `01:00:00` Rediscovering Early RL Architectures in the LLM Era Reflecting on the infrastructure work done years prior at DeepMind and Google Brain, David Silver emphasizes that engineers building modern large-scale reinforcement learning architectures who haven't read the foundational distributed RL papers would benefit immensely from revisiting them. Much of the distributed machinery now powering state-of-the-art post-training across the industry directly descends from frameworks that Lasse Espeholt pioneered roughly eight years earlier, such as IMPALA and SEED RL. Espeholt observes that the broader language modeling community has spent the past two years heavily integrating reinforcement learning, often arriving at paradigms that were already established during the earlier era of deep RL systems. While there is genuinely novel work being layered on top, many foundational engineering patterns are effectively being rediscovered. As Lasse Espeholt puts it: “The LLM community is also using reinforcement learning, in particular, last two years. So and like a lot of the things that they're using are, you know, being rediscovered approaches that we used many years ago.” ### `01:00:58` Keeping the Team Small: 20 People and a High Compute-Per-Person Ratio Turning to organizational design, Sonya Huang asks about the current scale and composition of Ineffable Intelligence, noting that several co-founders worked together on the AlphaStar project and asking whether they are essentially assembling an "Avengers" team from past DeepMind colleagues. Silver explains that they have purposely capped the team at roughly 20 people. Rather than racing to build a sprawling tech bureaucracy, they believe that an elite, concentrated group can execute faster and maintain total clarity of purpose without bureaucratic bloat. Espeholt points out a crucial infrastructural motivation for staying lean: maximizing the compute-per-person ratio. In giant tech firms, researchers spend immense energy competing internally for access to clusters. By intentionally keeping head count small against their large GCP accelerator allocation, Ineffable ensures that individual researchers never have to fight for compute resources. When hiring, Silver looks across four non-negotiable criteria: raw intellectual brilliance, kindness and collaboration, deep mission alignment, and specific technical complementarity. Maintaining this extreme standard has required painful trade-offs. Even when selecting from former colleagues who worked on world-class projects like AlphaStar, Silver has turned down exceptionally gifted researchers to preserve this exact cultural and technical balance. As David Silver describes the emotional weight of their hiring filters: “So as uncomfortable as it is, I've had to say no to a lot of people I've worked with in the past just because our bar is so super high. Yeah, I've been fortunate to work with some amazing people, and so, you know, I've been choosing amongst like a very exceptional crop and bringing in what I consider to be a superstar team.” ```mermaid flowchart TD accTitle: Ineffable Intelligence Organizational Model accDescr: Diagram illustrating Ineffable's strategy of pairing a small 20-person team with massive compute to pursue pure scientific breakthroughs rather than incremental products. subgraph OrgStrategy["Organizational Strategy"] A["Small Team (~20 Elite Researchers)"] B["Massive Compute Allocation"] A -->|"High Compute-per-Person Ratio"| C["Zero Internal Resource Scramble"] B --> C end subgraph CultureApproach["Research Environment"] C --> D["Pure Scientific Haven"] D -->|"Rejects Product Pressure"| E["Fundamental Breakthrough Focus"] end subgraph Outputs["Target Outcomes"] E --> F["Scientific & Mathematical Discoveries"] F --> G["Massive Downstream Products"] end ``` ### `01:04:49` Building a Research-Led Haven Free from Product Pressure Sonya Huang notes the stark contrast between Ineffable and the major industrial AI labs, pointing out that incumbents are currently locked in a high-stakes prisoner's dilemma race where billions of commercial dollars warp scientific priorities, making Ineffable's pure mission feel like a rare haven for fundamental research. Silver agrees, explaining that Ineffable was deliberately structured from inception to insulate researchers entirely from premature productization. Instead of chaining top minds to shipping minor incremental features or enterprise APIs, they have staked the company on giving scientists the runway to pursue true intellectual breakthroughs. They operate on the conviction that generating autonomous, discovery-capable superintelligence will yield downstream commercial and practical value that dwarfs any near-term product play. Rejecting corporate product roadmaps, David Silver declares: “We've tried to establish a research-led culture so it's not a company that's led by, you know, building an incremental next product v 17.1 on the thing that's come before. It's a company that's built on researchers pursuing their passion and trying to produce the breakthroughs that they believe are possible.” - Ineffable consciously acts as a counterweight to the commercial sprint, protecting researchers from shipping deadlines. - The company culture is rooted in the belief that fundamental leaps in intelligence naturally unlock far greater practical utility than chasing minor product iterations. ### `01:05:47` Forecasts: Strong Evidence Within a Year, Superintelligence in Three Closing the conversation, Huang asks Silver—known within AI circles for his track record as a rigorous superforecaster—to put dates on Ineffable's milestones: first, when the world will see undeniable evidence that pure experience-driven superlearning works; second, when humanity will achieve "first contact" with true superintelligence. Acknowledging that his public record is at stake, Silver provides two aggressive timelines. First, he forecasts that within one year, Ineffable will demonstrate unambiguous, robust empirical evidence proving that their crucible-based, taboo-free reinforcement learning paradigm outperforms standard pre-training paradigms. Second, addressing the timeline to full superintelligence—defined as an autonomous agent capable of discovering math, science, language, and technology far beyond the collective limits of human intellect—Silver reveals that his expectations have accelerated. While he previously estimated a three-to-five-year window, the rapid emergence of agentic coding tools and self-accelerating software workflows has compressed that runway. Setting his formal prediction, David Silver concludes: “I think we will see some extremely strong evidence that we're on the right track within a year. And I think we will achieve something which is a true superintelligence, according to the definition I gave earlier, of something which discovers things which are far beyond what humans can achieve across everything from science, mathematics, technology, languages, et cetera, et cetera. I originally thought within three to five years, my timelines have accelerated a little bit. So I'm going to say three years because of what can be done with coding agents to accelerate things.” ## Notable quotes > `40:00` The real point is that if it's possible to get all the way to a superintelligence in simulation, it will happen faster, and it will happen safer. — David Silver > `43:44` The most important single thing that we can do as a company is to actually understand the science of super intelligence. — David Silver ## Mentioned David Silver, Lasse Espeholt, Sonya Huang, Wojtek Czarnecki, Chris Gamble, Lee Sedol, Rich Sutton, Demis Hassabis, Dario Amodei, Ineffable Intelligence, Google DeepMind, Google Brain, AlphaGo, AlphaGo Zero, AlphaZero, AlphaStar, AlphaProof, IMPALA, SEED RL, MetNet, OpenAI, Google Cloud Platform (GCP), NVIDIA Vera Rubin, International Mathematical Olympiad (IMO) ## Video description David Silver spent two years building AlphaGo on human data. When he removed human data altogether building AlphaGo Zero, the AI surpassed AlphaGo in under two days. He left DeepMind to run that same bet in pursuit of superintelligence. David and Lasse Espeholt, co-founders of Ineffable Intelligence, explain their plan to build a “superlearner”—an agent that starts knowing nothing and discovers science, mathematics, and language for itself by direct experience instead of through a pre-training run. Silver argues that human-language reasoning is a ceiling, not a foundation, that alignment via chain-of-thought monitoring is already reaching its limits, and that the science of how superintelligences coexist should be worked out in simulation before anything touches the real world. Espeholt describes the GPU cluster and the lineage from IMPALA and SEED RL that makes the experiment possible. Silver’s forecast: strong evidence within a year, superintelligence in three. Hosted by Sonya Huang, Sequoia Capital 0:00 – Introduction 1:09 – What held true from Go to today, and what the LLM era changed 3:35 – Why leave DeepMind to start a new lab 5:58 – Why "Ineffable" (and the secret second reason) 7:03 – Lasse's path: scaling RL at DeepMind and Google Brain 9:31 – Defining a super learner and first contact 12:35 – Super learners vs. LLMs that make discoveries 13:59 – Day to day: the RL stack, crucibles and human-journey milestones 16:13 – Don't breadcrumb the objective: simple rewards, emergent complexity 21:16 – What a crucible looks like 24:14 – Go: one agent is boring, two is infinite 25:41 – Will agents rediscover language? 27:22 – Removing human priors: AlphaGo to AlphaZero 32:05 – Keeping human bias out of the crucibles 33:35 – Confidence, timing and super forecasting 36:24 – Is compute the driver? 38:13 – The ceiling on simulation and Rich Sutton's objection 43:44 – Alignment and the science of superintelligence 47:45 – Monitoring agents that don't think in English 51:01 – Pacing the frontier 52:54 – The compute setup: Vera Rubin on GCP 55:18 – Training, inference, simulation and RL scaling laws 57:40 – IMPALA and SEED RL 1:00:58 – 20 people and the compute-per-person ratio 1:04:49 – A research-led culture 1:05:47 – Predictions [Source](https://pscrb.fm/rss/p/traffic.megaphone.fm/CPUAI9233856839.mp3)

Source quotes

[40:00] David Silver: The real point is that if it's possible to get all the way to a superintelligence in simulation, it will happen faster, and it will happen safer. [43:44] David Silver: The most important single thing that we can do as a company is to actually understand the science of super intelligence.

Detailed notes — zh

## Background 本期播客由红杉资本(Sequoia Capital)合伙人 Sonya Huang 主持并发布于其官方播客渠道(RSS / Web,发布时间为 2026 年 10 月 13 日)。节目主要聚焦 Ineffable Intelligence 两位联合创始人 David Silver 与 Lasse Espeholt 的创业愿景,探讨他们如何放弃主流大语言模型对海量人类数据的依赖,转而通过纯经验驱动的强化学习构建能够自主发现科学、数学与全新语言的“超级学习者”(Superlearner)。 ## TL;DR AlphaGo 与 AlphaZero 的缔造者 David Silver 携手分布式强化学习先驱 Lasse Espeholt 创立 Ineffable Intelligence,坚信人类语言并非推理的基石而是认知的上限。他们正在名为“熔炉”(Crucibles)的高拟真复杂仿真环境中,通过全 GPU 架构与极简奖励机制让智能体在数百万多体博弈中自主演化出高阶认知与工具体系。针对 AI 安全与对齐,团队主张在高度安全的模拟沙盒中建立一门真正的“超智能科学”,利用影子行动等非语言手段实现有效监控。凭借精准的“超级预测者”直觉,Silver 预言团队将在一年内证明该路线的全面可行性,并在三年内实现人类与真正超级智能的“第一次接触”。 ## Atmosphere & style 整场对话沉稳、深邃且洋溢着极客式的纯粹理想主义,Sonya 与两位嘉宾的交流既有跨越十年的历史默契,又带有探索未知的智性张力。David Silver 语气平实却字字千钧,从容剥离当代 LLM 军备竞赛的浮躁喧嚣,以科学家在“刀锋边缘”探索的笃定展现出令人屏息的远见;Lasse Espeholt 则用严谨低调的工程底色为这场宏大构想筑牢地基。整期节目宛如一场返璞归真的思想远征,带给听众一种目睹科学范式正在寂静中被颠覆的震撼与激动。 ## Bulletin - `03:35` DeepMind 前核心科学家 David Silver 与分布式系统专家 Lasse Espeholt 创立 Ineffable Intelligence,全面押注摒弃人类预训练数据的“超级学习者(Superlearner)”范式,力求通过纯强化学习从零(Tabula Rasa)自主发现科学与新知。 - `36:24` 团队拿下了 Google Cloud Platform(GCP)早期专属的 NVIDIA Rubin GPU 集群,计算规模达到 2019 年 OpenAI“捉迷藏”多智能体实验的 100,000 倍,并构建了模拟、推理与训练一体化的全 GPU 强化学习栈以消除 CPU 瓶颈。 - `25:41` Silver 提出人类语言是认知的上限天花板而非基石,指出依赖自然语言思维链(CoT)的对齐手段正在失效,必须通过“影子行动”与超智能监督智能体在模拟环境中构建全新的对齐科学。 - `01:05:47` Silver 给出激进的时间表预测:Ineffable 将在 1 年内拿出该自主学习范式完全可行的强力证据,并在 3 年内实现人类与真正超级智能的“第一次接触”。 ## Key takeaways - `00:00` 通向超级智能的核心途径不是依赖人类数据的预训练,而是让系统像人类大脑进化一样,从无先验的空白状态(Tabula Rasa)通过直接经验自主探索。 - `03:35` 自立门户离开 DeepMind 是为了回避全球数十万人扎堆的 LLM 赛道,将 100% 的资源集中于几乎无人问津但极具颠覆性的纯强化学习自主发现路线。 - `25:41` 人类语言并非推理的基石而是认知的天花板;它深嵌了人类历史上的认知偏见,高度非线性的神经推理在表达力和效率上必然超越线性的人类语言。 - `30:00` AlphaGo 耗费两年人工干预与数据微调,而 AlphaZero 彻底剔除人类数据后仅用数小时至数天便全面超越前代,证明了摆脱人类数据先验能解锁惊人的进化速度。 - `16:13` 坚决拒绝“撒面包屑”式的微观人为奖励引导,只需设定极度简洁的全局目标,多智能体博弈与复杂环境法则即可自然级联涌现出语言、交易、算术与科学。 - `24:14` 围棋经验印证了“单智能体枯燥乏味,双智能体蕴藏无限”的原理,多智能体之间的递归推演博弈是突破算力与环境复杂度上限的关键引擎。 - `32:05` 在虚拟“熔炉”(Crucibles)环境设计中秉承实用主义弱先验,不人为预设冯·诺依曼架构等既有体系,以便观察智能体是否能自主发明出全新的计算形态。 - `33:35` 人类大脑是从物理经验中演化出高阶智能的现实存在性证明,而自动化研究工具的成熟使当前成为了突破自主强化学习的黄金时间窗口。 - `36:24` 相比 2019 年 OpenAI 在“捉迷藏”实验中展现的初级工具使用,Ineffable 当前能调动的算力杠杆高出了约 100,000 倍,足以支撑高维复杂环境中的大规模涌现。 - `40:00` 在高度并行、确定性的数字模拟中构建超级智能比在现实世界快 10 到 100 倍,且图灵完备的虚拟环境足以完整承载人类已知的所有计算、经济与科学规律。 - `47:45` 随着模型能力增强,依赖自然语言思维链(CoT)的对齐已开始出现言行不一的漏洞,必须引入“影子行动”与高阶监督智能体等非语言监控手段。 - `51:01` 支持在物理世界放缓部署节奏,但在模拟沙盒中必须全速狂奔,通过建立“超级智能科学”并将其作为真实世界行动的放行闸口。 - `52:54` 彻底告别十年前依靠海量 CPU 集群运行模拟的老路,在 GCP NVIDIA Rubin 机架上实现环境模拟、策略推理与网络训练的三位一体原生 GPU 闭环。 - `55:18` 强化学习的 Scaling Law 呈现强烈的非平稳过程(Non-stationary process)特征,随着智能体自身水平进化,其生成的数据分布与任务难度发生动态跃迁,不同于 LLM 的静态标度律。 - `57:40` 团队技术底座继承并重构了由 Lasse Espeholt 开创的 IMPALA(行动者-学习者解耦)与 SEED RL(集中式推理训练池)等里程碑式分布式计算架构。 - `01:00:58` 全公司严格保持约 20 人的精简规模以最大化“人均算力比”,避免内部资源争夺,同时执行极其严苛的使命认同与技术互补筛选标准。 - `01:04:49` 践行纯粹由科学研究驱动的文化,拒绝为商业化现有产品修修补补(如“版本 17.1”式的产品内耗),专注于能孕育长远巨量价值的基础范式跨越。 - `01:05:47` David Silver 作出预测:1 年内将拿出超级学习者路径可行的扎实科学证据,3 年内人类将与真正具备超人类自主发现能力的超级智能达成第一次接触。 ## Detailed walkthrough ### `00:00` 从人类知识的起源到强化学习的根本信仰 人类最初的起点不过是大脑里的一团“灰色浆糊”(gray mush),没有任何先验知识。然而,正是通过一代代人对语言、技术、科学和数学的集体探索,人类从零开始独立发现并构建了当今世界的整个知识体系。这种“完全依靠自身探索去发现一切”的能力,正是当前 AI 领域尚未攻克的精髓所在,也是通向真正能够无止境学习和发现新知的智能系统的核心路径。 红杉资本(Sequoia Capital)合伙人 Sonya Huang 欢迎了 Ineffable Intelligence 的两位联合创始人 David Silver 与 Lasse Espeholt。Sonya 提到,David 是她在 100 多期前创办该播客时第一位发出冷启动邮件(cold email)邀请的嘉宾,如今终于成行,犹如画上了一个圆满的句号。David 早在 2009 年的博士论文中就专注于将强化学习(RL)应用于围棋,并最终向世界证明了单一领域的超人机器智能是完全可行的,激励了整整一代 RL 与深度学习研究者。 回顾过去的研究生涯,David 坦言自己的核心信念从未动摇:解决通用人工智能(AGI)乃至超级智能(Superintelligence)的真正途径,一定是让系统通过直接经验进行自我探索和试错学习(learning from experience)。尽管大语言模型(LLM)走的是依赖海量人类数据的路线,但 LLM 时代也带来了一个让他深感意外的启示——当模型在庞大数据集上完成训练后,仅凭极少量的微调数据就能剧烈转变行为模式。这改变了他的认知:很多起初看似艰难、缓慢的学习过程,一旦系统积累了足够多的经验,原本极其缓慢的发展就会在某个临界点后变得异常迅猛。 ```mermaid flowchart LR accTitle: 从纯经验学习到超级智能的演进路径 accDescr: 对比传统LLM对人类数据的依赖与Ineffable坚持的从零试错与经验涌现路线。 subgraph LLM["传统 LLM 范式"] A["海量人类数据预训练"] --> B["少量人类指令微调"] B --> C["以人类知识/语言为上限"] end subgraph RL["Ineffable 经验主义范式"] D["空白状态 (Tabula Rasa)"] --> E["直接环境试错与经验积累"] E --> F["超临界点加速涌现"] F --> G["超级智能 (超越人类表达极限)"] end ``` ### `03:35` 为何离开 DeepMind:避开数十万人的拥挤路灯,寻找未知的超级智能 在被问及为何不在算力极其充沛的 Google DeepMind 内部推进这一项目、而是选择自立门户创办新实验室时,David 表达了对 DeepMind 时光与团队的深厚感激与敬意,但随后阐述了一个根本性的战略分歧。他坚信通向超级智能的终极道路并不需要将“基于 LLM 的产品”作为中间过渡站。如果一家机构将重心放在大语言模型上,这种从纯经验出发的激进探索注定只能被边缘化为某个部门的一个角落,而它理应值得一个倾尽全力、100% 押注其上的顶尖 AI 实验室。 更深层的原因在于科研资源的配置效率:当前全球已有成千上万、甚至数十万极其优秀的科研人员正在全力以赴押注基于 LLM 的预训练、后训练和代码生成路线。David 反问自己:“难道我还要去做第 100,001 个跟风者吗?”他认为在所有人都聚集的那盏路灯之外,还有另一盏目前几乎无人问津的路灯。若把精力全力投入到这片未开垦的领域,一旦取得突破,将对整个人工智能领域产生颠覆性的巨大影响。 - **行业同质化现状**:全球有数十万研究人员集中在“预训练 + 后训练 + 编程能力(coding)”的 LLM 单一路线上。 - **差异化押注的逻辑**:回避拥挤赛道,全力证明脱离人类语言和人类数据先验的强化学习系统能够完整自发涌现超级智能。 ### `05:58` 命名之谜:“Ineffable”的双重含义 谈到公司名称“Ineffable”(不可言喻)的由来,David 透露了这背后的两层深意。第一层来自对智能本质的理解:超级智能未来所发现的深邃知识,其震撼与宏大程度或许远远超出了人类语言所能表达的范畴(profoundly amazing, beyond human language)。这与他们摆脱人类语言束缚、让系统自行探索的理念完全契合——那些终极知识本质上就是“不可言喻”的。 第二层原因则是一个略带幽默色彩的团队内部双关语(pun):因为这条完全押注于“从经验中学习”的路径在逻辑上是无懈可击、必然行得通的,“它绝不可能搞砸”(It can't possibly F up),缩写与读音谐音便成了“Ineffable”(in-effable)。 ### `07:03` Lasse Espeholt 的工程传承与团队组建 联合创始人 Lasse Espeholt 讲述了自己加入的契机与背景。Lasse 于 2015 年 1 月加入 DeepMind,并在 2016 年前后与 David 展开密切合作。当时 Lasse 属于深度学习团队,而 David 领导强化学习团队,两人共同推动了将强化学习在大规模 GPU 集群上扩展(scale up)的内部项目。2018 年夏天,Lasse 曾离开 DeepMind 前往 Google Brain,后来又重返 DeepMind,直到大约在 David 离职的一年前再次离开。 两人的再度联手源于一位投资人的牵线。当投资人询问 Lasse 对 David 的看法时,Lasse 第一反应是后悔自己没有早点主动联系 David,因为在过去的合作中,David 始终是“房间里最聪明的人之一”。此外,团队中还有昔日的战友 Wojtek 与 Chris,能在欧洲建立这样一个极高水准的 AI 实验室,对 Lasse 而言极具吸引力。 David 对此特别补充强调,Lasse 的态度过于谦逊,实际上 Lasse 是该领域最早攻克“强化学习大规模并行化与扩展”难题的核心先驱之一。如今整个工业界在扩展强化学习时所普遍采用的分布式框架和并行方法,其思想源头几乎全部可以追溯到 Lasse 当年在 IMPALA 等系统上所开创的基础性工作,是他率先为大规模强化学习奠定了工程根基。 ### `10:00` “超级学习者”的双重维度与初次接触的神奇感 David Silver 阐述了他心目中“超级学习者”(Superlearner)的定义:它兼具难以置信的学习“深度”与“广度”。在深度维度上,它必须能够从一无所知、没有任何先验知识的状态起步,一路自主探索并发现远超人类认知的概念。这正如人类文明的演进历程:人类祖先最初大脑中只有一片“灰质肉糜”,既没有人从天而降赐予文明蓝图,也没有上万亿条带标签的数据集指导我们如何运作,但人类完全靠自己摸索出了语言、技术、科学和数学等一切智慧成果。在广度维度上,超级学习者应当具备通用探索能力,无论是被投入仿真世界、数字网络还是物理实体中,它都能在任何经历流(Stream of Experience)中持续学习、永不枯竭。 当被问及未来首次遇见真正的超级智能会是什么感觉时,David 认为那将是一种“面对魔法般的奇迹与敬畏”(Encountering magic and wonderment)。他以过去的亲身经历为例:当年团队在相对狭窄的棋盘游戏领域推进突破时,当 AlphaGo 和随后的国际象棋 AI 展现出超越人类巅峰的对弈水平,顶尖棋手们感受到的不是受挫,而是一种纯粹的震撼与发喜——目睹一种未曾被人类想象过的下棋方式。如果将这种超凡的创造力拓展至棋盘之外,去自主开创全新的科学前沿和人类未曾构想过的全新技术,其震撼程度将无可比拟。 - **深度的核心特质**:从零起步(Tabula Rasa),自主进行持续且永无止境的知识发现。 - **广度的核心特质**:能无缝嵌入任何仿真环境、数字世界或物理经验流中持续吸收并演进。 - **棋类先例的启示**:围棋与国际象棋界的反馈证明,超越人类经验的机器直觉能为人类专家带来由衷的启发与欣喜。 --- ### `12:35` 语言模型范式的局限与超级学习者的本质区别 针对当前大语言模型(LLM)结合强化学习已开始尝试攻克“千禧年大奖难题”并在科学探索中崭露头角的现状,David 指出这与超级学习者仍有本质差异。他承认并尊重 LLM 领域取得的巨大突破,但他强调超级学习者的第一核心要素是“完全自主的发现”(Completely autonomous discovery)。现阶段基于 LLM 的科研尝试背后,依然严重依赖大量“人在回路”(Human-in-the-loop)的精心干预、调试与引导,远非真正的机器全自主突破。 更关键的哲学分歧在于媒介本身。人类的大部分认知与物理经验交互无法完全用文字表达,LLM 依赖语言为核心架构的做法,未必是通往真正超人类通用智能的正确载体。David 认为,将一切思考强行约束在语言框架内可能是一种范式误区,而依赖海量人类数据作为预训练基础,甚至可能在客观上拖累智能的进化上限。真正的超级学习者应当脱离人类语料的引力束缚,直接通过自身与环境的交互经验提炼高阶智能。 --- ### `13:59` 日常落地:大规模强化学习栈与“熔炉”模拟环境 在具体执行层面,Sonya 提到早期 AlphaGo 团队成员 Ioannis Antonoglou 曾评价 David 既能提出极具野心的宏大愿景,又能将其扎实拆解为极其切实的工程与研究里程碑。David 详细拆解了 Ineffable Intelligence 团队当下的日常研发工作:一方面,由 Lasse Espeholt 等系统专家领衔,从底层打造一套顶级的分布式强化学习技术栈(RL Stack)。该系统必须支持超大规模计算,以最优效率吞吐与吸收数据,确保智能体能够在长期运行中持续学习而不发生灾难性遗忘。 另一方面,团队正全力构建被称为“熔炉”(Crucibles)的高拟真复杂仿真环境。在这些模拟世界中,智能体拥有足够的物理法则、资源约束与交互自由度,足以表达并演化出人类所定义的全部智能行为。因此,团队的日常工作是由两股力量协同推进的:打磨超级学习者算法与大规模计算架构,同时构建高密度的“熔炉”环境,让智能体在其中诞生出前所未见的涌现行为。在里程碑的设定上,团队并不人为规定琐碎的算法阶段,而是对齐人类文明认知爆发的关键节点,以智能体何时“重新发现”这些伟大历程作为评估进展的试金石。 --- ### `16:13` 拒绝“撒面包屑”式的引导:简单奖励下的复杂涌现 谈及智能体究竟会首先重新发现什么,David 直言团队更期待“被智能体所震惊”。他严厉指出了当前主流强化学习范式的一个误区:许多研究者习惯于采用“撒面包屑”(Breadcrumb trailing)的方式,生怕智能体迷路,人为设定一连串精细的前置子目标,像牵牛鼻子一样把模型拉扯向人类预设的答案。David 强调,Ineffable 采取的是截然相反的路径——给予智能体极度简洁的全局目标函数,让其在极度复杂的环境博弈中自主生长。 ```mermaid flowchart LR accTitle: 极简目标催生复杂文明智能的演进链条 accDescr: 展示从生存与简单目标出发,经由协作、沟通、交易逐步涌现算术、工具、科学乃至高等数学的级联过程。 subgraph Primitive["基础驱动层"] A["简单生存 / 目标优化"] end subgraph Social["社会与协作演化"] B["多体协调协作"] C["语言与沟通交流"] D["物品与资源交易"] end subgraph Tooling["认知工具突破"] E["基础算术与记账"] F["工具制造与使用"] G["自然规律与科学洞察"] H["高等数学与复杂技术"] end A --> B B --> C C --> D D --> E E --> F F --> G G --> H ``` 很多从业者常困惑于如何为抽象概念定义奖励函数——比如“科学研究”或“有趣的数学”应该如何形式化打分?David 解释道,从大自然的演化史来看,最精妙的复杂性往往发端于最简单的驱动力。进化本身只奖励极为单一的“适者生存”(适应度信号),却涌现出了地球生物圈不可思议的多样性;人类文明亦是如此: - 为了完成简单的狩猎与生存目标,个体之间开始尝试**协作**; - 为了提升协作效率,人类自然发明了**沟通与原始语言**; - 语言的出现让分工成为可能,进而促成了**贸易与资源交换**; - 随着贸易规模扩大,人类迫切需要发明**基础算术**来记账; - 对更高效生产的渴望催生了**实体工具的创造**; - 要改良这些工具,人类必须去理解背后的物理与**基础科学**原理; - 对技术运作规律的深入探索,最终反向哺育出**高等数学与尖端工程**。 整个现代文明的复杂大厦,全部是在追逐最初几个极其质朴的目标过程中顺理成章涌现出来的副产品。David 认为,如果能够把这一环境和机制在“熔炉”中正确搭建起来,超级学习者就能走出一条远超人类预设轨迹的智能飞跃之路。 ### `20:00` 涌现的复杂性与“熔炉”环境的设计哲学 David Silver 与 Lasse Espeholt 深入探讨了他们构建超级学习者(Superlearner)时的奖励机制与训练环境设计。Silver 强调,他们不会为 Agent 人工编造复杂的细分奖励函数来精准定义什么是“科学”、“技术”或“语言”。相反,如果 AI 科学家无法解决让智能体自主发现新知这一核心难题,就永远不可能诞生能开创人类未知学科的系统。当前的知识盲区就像身处牛顿物理学时代的人类,当时根本无人知晓量子力学或相对论的存在;唯有赋予 Agent 完全开放的探索自由,才能迎来真正激动人心的突破。 在谈到他们称之为“熔炉”(Crucibles)的仿真环境时,Silver 澄清这绝非当下行业主流那种“由一堆离散 Benchmark 任务拼凑而成、并预设标准答案”的评测集,也不是早期的 Atari 57 这类简易模拟器。他们致力于构建蕴含海量“涌现复杂性”(Emergent Complexity)的世界。这种复杂性主要源于两个维度: 1. **物理引擎驱动的涌现**:由底层物理法则支撑,数以百万计的物体在复杂动力学下实时交互。 2. **多智能体交互的涌现**:数以百万计的 Agent 共同置身于同一世界,每个智能体的行为选择都构成了其他智能体环境的一部分,使环境复杂度呈几何级数爆炸。 Lasse 补充道,在环境设计上需要寻找计算效率与行为复杂度之间的平衡点。例如围棋(Go)的底层表征极其简陋(本质就是一张位图),规则运算极轻量,却能涌现出极为深奥的博弈;反之,若一上来就模拟逼真的量子力学与微观物理,计算开销将无法承受。因此,Ineffable 正在搭建一个由浅入深的“熔炉”梯队:既有能超高速运行、快速迭代轻量级系统,也有逐步逼近现实物理世界的高维仿真。 ```mermaid flowchart TD accTitle: 熔炉环境的涌现复杂性来源 accDescr: 展示物理引擎交互与多智能体博弈如何共同驱动环境产生涌现复杂性 subgraph Primitives["底层原语输入"] P1["轻量/复杂物理引擎<br/>(数百万交互物体)"] P2["信息论与博弈规则<br/>(非物理机制)"] end subgraph Dynamics["涌现催化层"] D1["多智能体互动<br/>(数百万 Agent 并存)"] D2["递归推理<br/>(预测对手的多层策略)"] end subgraph Outcomes["自主发现成果"] O1["基础能力: 运动控制 / 协同"] O2["高级结构: 语言 / 经济学"] O3["前沿新知: 科学 / 全新数学"] end P1 --> D1 P2 --> D2 D1 --> O1 D1 --> O2 D2 --> O2 D1 & D2 --> O3 ``` - **抽象机制与特定学科涌现**:主持人指出,像经济学这样源自博弈论的学科,本质上与物理世界无关,更取决于“谁掌握什么信息”。Silver 和 Lasse 赞同环境中所选择的原语(Primitives)将直接塑造涌现出的学习形态,最终目标是实现与现实世界锚定的通用能力。 --- ### `24:14` 围棋的启示:单智能体枯燥乏味,双智能体蕴藏无限 David Silver 重新以围棋为例,剖析了简单规则如何催生无穷复杂性。他风趣地指出大家可能并不意外他会拿出围棋说事:围棋仅仅是一个 19×19 的网格,目标是划定和争夺地盘。如果围棋是一个“单人游戏”,它将是世界上最无聊的游戏——玩家只需把棋子一颗颗摆满棋盘即可,毫无策略可言。 然而,一旦在棋盘上引入第二个智能体,整个系统的状态空间和逻辑深度瞬间改变。每个智能体都必须去推演对手所有可能的行动与反制手段,这种递归推演将环境复杂度推向极致。Silver 回忆道,当年在 DeepMind 为围棋算法倾注的所有算力中,学习曲线始终未曾见顶,机器总能在博弈中不断剥开“像洋葱般一层又一层”无穷无尽的策略维度。虽然围棋本质上仍是一个相对狭窄的棋盘游戏,但它雄辩地证明了一个真理:极其简易的环境动力学,在多智能体交互的催化下,能够释放出匹敌算力极限的无限智能。 --- ### `25:41` 人类语言是思维的极限天花板,而非推理的基石 针对 Agent 是否会自主重新发明语言、以及这种语言是否会类似于人类语言的问题,Silver 给出了旗帜鲜明的反思。他认为,坚信人类语言是推理的终极形式或最佳方式,是一种极为幼稚的视角。 Silver 指出,未来建立在高度可并行化、非线性网络结构上的神经推理(Neural Reasoning),其表达力与推理效率必然会远超线性的人类语言。以人类语言作为推理的起点存在严重弊端:它不可避免地全盘继承了人类思维的历史偏见。Silver 提出了一个生动的思想实验: > “试想一下,如果我们退回到 500 年、1000 年甚至 2000 年前,强制要求 Agent 只能使用当时人类所有的语言进行思考,那我们的 Agent 可能至今还会坚信‘地球是平的’,或者认为‘所有疾病都源于四体液失衡’(Four Humors)。” 这些充斥着荒谬谬误的认知早已深嵌在当时的语言结构中,极难被剥离。因此,唯有跳出人类语言框架,让智能体在更高维的原语空间探索,才能真正突破认知的天花板。 --- ### `27:22` 彻底剥离人类先验:从 AlphaGo Zero 到 Ineffable 的豪赌 谈到从 AlphaGo 演进到完全不依赖人类棋谱的 AlphaGo Zero,主持人询问 Silver 当时是否预料到剔除人类先验(Human Priors)反而能诞生出更强大的智能。Silver 坦言,自己当时完全不确定,概率在他心中大概是对半开的 50/50。他直言: > “做科研最有意思的,就是去探索那些答案胜负未定、处于刀锋边缘(Knife edge)的课题。如果你在动手前就已经有 95% 的把握确信它能成,那这种研究坦白说相当无聊。” 正是带着这种 50/50 的悬念,当看到纯强化学习系统以碾压之势超越结合人类先验的版本时,团队感到无比震撼。而如今,Ineffable Intelligence 正在更大的尺度上践行这一理念——他们主动放弃全网人类数据这笔现成资产。Silver 承认,放弃预训练数据在短期内确实是一种在性能指标上的妥协与牺牲,但这是为了触达更高阶智能形态所必须付出的代价。 Silver 将“超级学习者”的核心能力归结为两个维度上的彻底释放: - **深度(Depth)**:必须具备发现超越人类既有认知边界的新知识的能力。系统对人类数据的依赖越深,就越难以探索出能够自我催生全新知识领域的原生算法。 - **广度(Breadth)**:必须具备置身于人类语言与文本完全不适用的陌生环境中并自主探索的能力,例如处理前所未有的新型物理传感器信号等高维场景。人类的语言数据在这些领域毫无用武之地,过度拟合人类文本只会自断出路。 ### `30:00` 摆脱人类数据:从 AlphaGo 到 AlphaZero 的跨越 David Silver 指出,人类数据本质上并不适合用来描述更深层次的智能规律;只有主动放手、抛弃人类数据,研究者才能将自身置于“发现远超人类水准事物”的起点。他以 AlphaGo 与 AlphaZero 的演进过程为例:当年团队花费了整整两年时间打造 AlphaGo,其研发模式与当下的 LLM 极其相似——研究人员必须亲手搭建每一层系统,灌注海量人类对弈数据,并在每个迭代节点通过“人类在环”(human-in-the-loop)的方式不断评估、微调,决定下一代模型该如何改进。依靠这套耗时两年的繁复体系,AlphaGo 最终在 2016 年击败了李世石(Lee Sedol)。 然而,当团队下定决心彻底放弃人类数据、转向让算法自我探索与发现时,真正的范式转移发生了。在算法架构调试就绪后,AlphaZero 仅仅用几个小时的自主学习,就彻底超越了此前耗费两年人工干预所取得的全部成果。在国际象棋领域,它只用了 4 个小时就击败了当时人类创造的最强 AI(Stockfish);在围棋领域,它也仅用了两三天便全面超越了老版 AlphaGo。David 强调,如果不主动做出舍弃人类数据的抉择,系统就无法产生能够永无止境持续学习的能力。当今许多 LLM 公司本质上仍在构建依靠数千名标注员深度参与的“半自主式”在环系统;尽管这种工程做法取得了令人尊敬的进展,但若无法退后一步去寻找自主学习机制,AI 将永远无法触达真正的“超级学习者”(Superlearner)高度。 ```mermaid flowchart TD accTitle: 人类在环范式与自主发现范式的对比 accDescr: 对比 AlphaGo/LLM 依赖人工在环与 AlphaZero 自主发现的学习路径与演进速度 subgraph HumanLoop ["人类在环范式 (AlphaGo / 现代 LLM)"] direction TB A1["灌注人类数据与先验"] --> B1["工程师介入调优 (Human-in-the-loop)"] B1 --> C1["逼近或超越人类专家"] C1 -.->|"耗时数年 / 触碰人类认知上限"| D1["渐进式微调瓶颈"] end subgraph SelfDiscovery ["自主发现范式 (AlphaZero / 超级学习者)"] direction TB A2["定义底层极简环境与目标"] --> B2["自我博弈与经验探索 (Self-Discovery)"] B2 --> C2["数小时至数天内击溃旧模型"] C2 -->|"持续无限进化"| D2["远超人类的超级智能"] end ``` - **研发时间与突破速度的反差**:AlphaGo 依赖人类棋谱与人工微调花费了整整 2 年时间才击败李世石;而 AlphaZero 彻底摒弃人类数据后,仅用 4 小时便在国际象棋上超越最强 AI,围棋上也仅耗时两三天便超越了前代系统。 - **对现代 LLM 产业的批判性审视**:当前大模型公司动用成千上万的人工标注员构建半自主在环学习流,这虽然极具实用价值,但无法成为通往超级学习者的终极路径。 --- ### `32:05` 实用主义先验:如何避免人类偏见污染“熔炉” Sonya Huang 随即追问:既然 Ineffable Intelligence 的研究员也是人类,在构建用于智能体训练的虚拟环境和“熔炉”(Crucibles)时,该如何防止人类先验知识不自觉地渗透其中?Lasse Espeholt 解释道,在设定环境机制和奖励函数时,人类偏见固然难以完全抹除,但他们的核心原则是绝不强制诱导智能体复现人类已有的技术架构(例如冯·诺依曼架构)。相反,研究人员只需要定义一个“机器能够发挥作用”的问题空间,让智能体在其中自主摸索,观察它究竟会创造出怎样形式的计算机器,这才是最激动人心的部分。 David Silver 进一步补充,他们并非原教旨主义者,不会偏执地要求系统中“绝对不能包含任何一点人类先验”。这种策略本质上是一种实用主义立场(pragmatic position):只要不强行依赖由人类知识构成的数万亿级超大规模数据集,就不会锁死模型发展的潜能。在设计跨虚拟世界的环境规则时,保留某些弱先验(weak, environment-agnostic priors)是完全合理且必要的;他们真正极力规避的,是那些把人类对单一问题的全部已知解法强行编码进去的“超强先验”(super strong priors)。 - **实用主义而非宗教式教条**:拒绝万亿级人类数据集并不等同于杜绝一切先验,关键在于区分“通用的弱环境先验”与“特定任务的强人工约束”。 - **让智能体自主重塑计算体系**:在熔炉环境中,不必向系统灌输冯·诺依曼架构(Von Neumann architecture)等既有人类设计,而是观察智能体在面临计算需求时会自发演化出何种机器形态。 --- ### `33:35` 科学研究的关键在于时机:超级预测者的敏锐直觉 针对 David 之前提到的“热衷于处于 50/50 刀锋边缘的研究”,Sonya 提问他在最宏观的层面上对“超级学习者是否必然出现”以及具体技术节点上的把握有多大信心。David 坚定地表示,他坚信这种方案在未来必然可行,因为人类大脑本身就是一个无可辩驳的“存在性证明”(existence proof)——大脑从物理具身世界出发,始于空白经验,最终发现了极其广泛且强大的智能形态。 因此,成败的关键不在于可能性,而在于“时机”(Timing)。在科学突破与商业创业中,时机决定一切:如果入场太晚,已有上千人抢先完成,研究便失去了探索价值;如果入场太早,整个学科的基础理论或底层算力尚不成熟,盲目撞门只会头破血流。Sonya 戏称 David 是某种意义上的“超级预测者”(super forecaster),因为他曾准确预测了计算机围棋被攻破的时间节点(早在 AlphaGo 问世前 10 年便已预言),并在《星际争霸》(StarCraft / AlphaStar)以及近期的 AlphaProof(国际数学奥林匹克竞赛银牌/金牌水平突破)预测上下注全中。David 认为,当下正是超级学习者技术落地的绝佳窗口期:自动化研究方法(auto research methods)的成熟、对 Scaling Laws(标度律)理解的加深,以及整套技术栈工程精细度的提升,都让所有要素在这一时刻汇聚成型。 - **人类大脑作为存在性证明**:自然界已经证明,仅凭具身智能体与环境交互的自学机制,足以孕育出能够理解科学、语言与工具的通用智能。 - **David Silver 的长期精准预测战绩**: - 提前 10 年成功预言计算机围棋将被攻克。 - 精确预测《星际争霸》等复杂即时战略游戏中 AI 战胜人类顶尖选手的时刻。 - 准确预判了 AlphaProof 在国际数学奥林匹克(IMO)竞赛级别推理上的突破节点。 --- ### `36:24` 算力杠杆:10 万倍于以往实验的全新尺度 当被问及算力是否是促成当下历史时机的最核心驱动力时,David 毫不犹豫地给出了肯定回答。他坦言算力是一个“极其巨大的驱动因素”(an enormous driver)。Ineffable Intelligence 目前锁定了极其庞大的算力集群,这使得他们能够运行比以往任何自主探索类强化学习(self-discovered RL)高出数个数量级精细度的宏大实验。 David 举例提到了 OpenAI 早期著名的“捉迷藏”(Hide and Seek)实验。在那项研究中,多智能体在简单物理对抗中自发学会了搬运积木、卡门等初级工具使用技巧。David 指出,Ineffable 如今能够调动的算力规模,大约是当年 OpenAI 捉迷藏实验的 **100,000 倍**。尽管捉迷藏只是极其原始的概念验证,但它揭示了环境交互产生复杂行为的惊人潜力;在 10 万倍算力的杠杆支撑下,类似框架能释放出的能量令人充满想象。Lasse 也感慨道,团队非常幸运能拿下这样一座“规模宏大且极其优美”的顶级计算集群,手握充足的筹码去检验这项假说,而这种实验规模即便是他们此前在 DeepMind 时也无法企及。 - **10 万倍计算规模跃迁**:相比于 OpenAI 2019 年在《Hide and Seek》多智能体工具使用实验中消耗的算力,Ineffable 当前实验所能调用的算力资源高出了大约 100,000 倍。 - **超越旧平台的自由度**:凭借独立筹措的超大规模 GPU 集群,Ineffable 获得了即便在 DeepMind 内部也难以调配的专注算力支持,成为极少数全力押注该方向的先锋团队。 --- ### `38:13` 模拟的上限与 Rich Sutton 的质疑:为何无需完美物理世界 Sonya 将话题引向模拟环境的上限:这些所谓的“熔炉”本质上都是数字化的虚拟世界,但模拟环境至今仍不够完美。她提到强化学习先驱 Rich Sutton 最近在节目中曾对仿真模拟(Simulation)表示出强烈的轻蔑态度,并询问两位创始人对此如何看待,以及 Ineffable 何时才会走向真实物理世界。 Lasse 凭借他在 Google Brain 阿姆斯特丹团队主导 MetNet(基于真实大气气象数据的深度学习预测模型)的经验做出了回应。在研发 MetNet 时,他们利用机器学习直接预测短期天气,其表现远超基于物理方程的数值气象模拟,原因正是传统物理仿真模型会随着时间推移不断累积计算误差。Lasse 坦承,Sutton 的批评有其道理——物理模拟确实永远无法做到尽善尽美。然而,这并不妨碍数字熔炉的价值:要观察智能体演化出技术发明与复杂智能行为,根本不需要构建在物理粒子层面分毫不差的仿真世界。即使在由精简小团队搭建的相对简单的抽象环境中,只要激励与博弈机制完备,就能观察到智能技术的层级式涌现。 - **MetNet 的工程经验启示**:Google Brain 的 MetNet 天气预报模型证明,纯机器学习模型能够击败传统物理仿真,印证了物理数值模拟存在误差累积的天然缺陷。 - **抽象环境足以孕育高级智能**:反驳对数字模拟的教条式质疑——观察科技创造和智能涌现并不依赖对微观物理世界的精准复刻,精简且规则清晰的抽象环境同样具备诱发高级智能行为的能力。 ### `40:00` 模拟环境构建超智能的可能性与上限 关于模拟环境是否存在上限,Lasse Espeholt 承认物理世界中确实存在诸如量子物理等极其复杂的真实规律,但智能体在不需要完整复现真实物理的情况下,就已经能够做到极其不可思议的事情。David Silver 认为关键问题在于:超智能能否在模拟环境中被构建出来?如果答案是肯定的,那么在模拟中构建超智能的速度将比在现实世界中快上 10 倍甚至 100 倍。这是因为模拟环境不仅可以无限并行化、具备完美的确定性与可复现性,而且是孕育超智能最安全的试验场。 为了证明在模拟中实现超智能的可行性,Silver 梳理了几个关键维度的表达能力: - **计算本质**:只要模拟环境具备图灵完备性(Turing completeness),人类所定义的一切计算在数学上都能完全在模拟环境中表达。人类已经在模拟环境中造出过极其复杂的计算机系统。 - **经济系统**:货币本质上由比特构成,只要引入某种形式的资产或货币表征,现代经济学所讨论的一切经济行为和规律都能在模拟世界中完整映射。 - **语言体系**:当多个智能体并存,并且彼此之间拥有可交流的通信通道时,人类定义的语言属性和通信功能便全都能在模拟中自发产生。 - **数学与科学**:数学超越于特定宇宙,在任何环境中都可表达;而科学不仅包含能量、熵、演化等通用规律,更重要的是,只要环境具备突现复杂性(emergent complexity),智能体就能针对其所在的环境探索出属于该世界的独特科学体系。 针对强化学习先驱 Rich Sutton 的质疑——即模拟环境学到的东西很难通过 Sim-to-Real(仿真迁移至现实)直接复现到远比模拟复杂的真实世界——Silver 表达了一定程度的赞同。但他强调,在模拟中进行的全部工作是为了科学地发现“超级学习者”(Superlearner)的通用底层算法。一旦找到了该算法,同样的算法框架就能应用到现实世界;与此同时,团队可以逐步提升模拟的保真度,在将智能体部署进现实世界之前充分验证其可行性。 ```mermaid flowchart TD accTitle: 模拟环境中构建超智能的推演路径 accDescr: 展示从图灵完备等基础要素出发,在模拟世界中孕育超智能并迁移至现实的研究路径。 subgraph SIM["高安全、可无限并行的模拟环境 (Crucible)"] direction TB A["图灵完备性"] --> B["完全表征计算"] C["多智能体通信通道"] --> D["自发涌现语言体系"] E["资产/比特表征"] --> F["经济学系统运作"] G["复杂环境机制"] --> H["智能体自主发现科学"] end subgraph CORE["算法沉淀与验证"] I["发现通用学习算法<br/>(Superlearner)"] J["逐步提高仿真保真度<br/>(High-Fidelity Sim)"] end subgraph REAL["物理世界落地"] K["安全部署至现实世界"] end SIM --> I I --> J J --> K ``` ### `43:44` 对齐困境与超智能共存科学 Sonya Huang 抛出对齐(Alignment)议题,指出当前业界大量不对齐现象似乎正是大规模扩展强化学习所带来的副产物,并询问超级学习者是否会自然地与人类利益保持一致。Silver 明确避开政治和政策层面的纷争,从纯粹的科学视角做出剖析。他指出,当前主流的对齐手段高度依赖两样东西:智能体所使用的人类自然语言,以及预训练阶段注入的人类数据先验(priors)。虽然这种方法此前行之有效,但其天花板已经显现。 随着智能体被训练得越来越强大,它们不可避免地会打破这些人为注入的先验。Silver 警告称,模型已经开始出现“言行不一”的苗头——在语言层面上说一套,实际行动却做另一套,这让基于人类先验的传统对齐手段难以为继。 对此,Silver 提出 Ineffable Intelligence 最核心的使命是**建立一门真正的“超智能科学”(Science of Superintelligence)**: - **模拟中的和平共存法则**:在沙盒模拟中将超级学习者推向极高智能水平,研究由多个超智能体构成的社会如何和平共处,尤其是这些超智能如何与世界中能力较弱的“低阶智能”和平相处。 - **作为现实行动的放行闸口(Gate)**:人类正快速滑向现实世界中多个超智能交织碰撞的未来。Ineffable 做出明确承诺,绝不在缺乏把握的情况下盲目推进现实部署,现实中的每一步推进都必须以模拟试验中取得的超智能科学证据为准绳,只有当团队对安全与正面结果具备绝对信心时,才会放行进入下一阶段。 ### `47:45` 超越英语思维链的非语言监控机制 当前主流 AI 安全研究高度依赖监控智能体的英语思维链(Chain of Thought)。然而,当智能体不再使用英语、而是发明出人类无法理解的语言时,研究人员便会陷入恐慌。针对“超级学习者是否会跳过人类语言,以及这是否意味着必须放弃监控”的疑问,Silver 强调,仅仅依靠监控自然语言正在变得不可行,必须在武器库中引入全新的非语言监控手段。 Silver 直言,随着智能体智力水平的飞跃,依赖自然语言的监视已出现严重漏洞:系统可能生成长达百万词的推演文献让人类无法研读,或者甚至故意采用掩饰手段(subterfuge)掩盖真实意图。针对脱离人类语言的智能体,团队正在探索多种新型监控架构: - **影子行动(Shadow Actions)**:让智能体在隔离环境中推演它若被允许与现实交互时会采取的动作,在未实际施加影响前对其行动意图进行全量观测与审查。 - **超智能监视智能体**:引入另一个超智能体专门充当监视者和正则化器。例如,设计一个探索科学但保持“轻微介入”(tread lightly upon the earth)的科学家智能体,同时部署一个高阶监控智能体持续监控其行动的能量级与影响范围,确保其行为被严格限制在安全阈值之内。 Silver 总结道,超智能时代的监控并不等于放弃可解释性,而是需要探索大量全新的科研方向,目前整个领域仍处于早期摸索阶段。 ### `50:00` 超级智能的制衡与“放缓前沿”:在模拟中极速狂奔,在现实中审慎慢行 大卫·席尔瓦(David Silver)强调,如果整个 AI 领域因为过度聚焦基于语言的方法而忽视了其他路径,人类将面临严峻的问题——因为单靠语言根本无法把我们带到真正期望的高度。不过,席尔瓦对世界的终局依然保持乐观。在他看来,未来绝不会只有单一的全能超级智能,而是会出现由大量高度复杂、彼此交互的超级智能体构成的生态系统。在这个多元格局中,不同智能体之间将形成一种“动态力量制衡”(balance of actors):即便某些实体的目标并不完全契合人类利益,那些致力于支持人类繁荣的超级智能体也会挺身而出,形成对冲并保护人类。 当索尼娅·黄(Sonya Huang)问及当下行业内关于“放缓前沿部署节奏”(pacing the frontier)的热门讨论时,席尔瓦明确表达了支持态度,赞同戴密斯·哈萨比斯(Demis Hassabis)和达里奥·阿莫代伊(Dario Amodei)等人的提议。但他提出了一个至关重要的分野:**在物理现实世界中应当放缓节奏,但在模拟环境中必须全速前进**。席尔瓦认为,人类唯有在高度安全的模拟沙盒中加速探索,才能真正建立起关于超级智能的底层科学体系;唯有提前在虚拟世界中摸透智能体的交互规律与对齐机制,人类未来在真实世界面对超级智能时才有最大的胜算。 席尔瓦进一步剖析了这种审慎的心理机制:技术的演进速度已经远远超出了人类社会机制与生物本能的适应能力。适当放慢现实世界中的部署步伐,不仅能给政府治理架构、社会组织以及普通大众的情绪心理留出缓冲与消化空间,还能为实验室争取关键时间,在模拟环境中把这门攸关命运的科学彻底搞扎实。 - **多智能体生态制衡论**:未来的安全并非寄托于人类直接锁死单一 AI,而是寄托于多个超级智能之间的博弈与平衡,让支持人类福祉的智能体去制衡不具善意的智能体。 - **模拟加速 vs. 现实审慎**:席尔瓦将“模拟环境中的极速狂奔”视为构筑真实世界安全防线的先决条件。 - **制度与心理适应鸿沟**:技术飞跃呈指数级,而人类政府、社会制度及情绪承受力的演进呈线性,放缓现实节奏旨在弥合这一致命的时间差。 --- ### `52:54` 告别 CPU 依赖:基于 GCP 顶级 Rubin 集群的全 GPU 强化学习栈 话题随后转向了支撑这场宏大实验的算力基础设施。为了实现从零构建超级学习者,Ineffable Intelligence 正在挑战算力工程的物理极限。拉塞·埃斯佩霍尔特(Lasse Espeholt)透露,他们并未像许多新型初创实验室那样去云服务商处租用廉价的竞价算力(spot instances),而是直接拿下了 Google Cloud Platform(GCP)上规模最大的 NVIDIA Rubin GPU 早期专属集群,成为该集群的首批核心客户。 埃斯佩霍尔特感叹初创企业的算力军备竞赛正变得极度残酷,而 Ineffable 在硬件基础设施上极其幸运。之所以极为推崇 Rubin 架构,关键在于其机架级的硬件设计:单机架内部的节点之间具备极高的数据吞吐互联带宽,在逻辑上几乎可以直接视作“一台超大型计算机”。这种超高带宽对强化学习至关重要——强化学习的探索与反馈要求毫秒级的超快迭代循环,其对延迟和通信的敏感度远高于训练周期相对缓慢的大语言模型(LLM)。 埃斯佩霍尔特指出,八到十年前 DeepMind 和 Google 依赖内部海量 CPU 集群来驱动环境模拟的做法如今已彻底走到尽头,那种架构根本无法扩展。Ineffable 确立的核心工程原则是:**全栈 GPU 化**。从环境模拟(Simulation)、模型推理(Inference)到权重训练(Training),一切计算都必须在 GPU 上原生完成,绝不依赖数以百万计的外部 CPU。他戏谑地提到,许多做 LLM Agent 的公司如今正重蹈十年前 DeepMind 的覆辙——不得不搭建庞大而臃肿的 CPU 沙盒去模拟计算机操作界面,而 Ineffable 从第一天起就彻底规避了这种系统架构陷阱。 ```mermaid flowchart TD accTitle: Ineffable 全 GPU 强化学习架构 accDescr: 展示环境模拟、推理与训练全流程在 GPU 高速集群内的紧耦合闭环。 subgraph Cluster["统一的 NVIDIA Rubin GPU 机架 (超高内部互连带宽)"] direction TB SIM["原生 GPU 环境模拟 (Simulation)<br/>超高通量生成交互轨迹"] INF["GPU 策略推理 (Inference)<br/>极低延迟输出探索动作"] TRAIN["GPU 策略训练 (Training)<br/>大规模梯度更新与权重同步"] SIM -->|"高速轨迹数据"| INF INF -->|"动作与经验批次"| TRAIN TRAIN -->|"最新模型权重"| INF TRAIN -.->|"世界模型先验"| SIM end style Cluster fill:#f8f9fa,stroke:#333,stroke-width:1px style SIM fill:#e1f5fe,stroke:#0288d1 style INF fill:#fff3e0,stroke:#f57c00 style TRAIN fill:#e8f5e9,stroke:#388e3c ``` --- ### `55:18` 训练、推理与模拟的三分天下:非平稳性下的强化学习 Scaling Law 索尼娅追问算力在不同任务间是否具有通用可替代性(fungible),埃斯佩霍尔特澄清事实并非如此。在强化学习的工作负载中,算力被明确划分为三大支柱:**训练**、**推理**与**模拟**。现代 GPU 自深度学习爆发以来,其微架构和硬件加速单元主要向着训练倾斜,近年虽加大了对推理特性的优化,但硬件厂商从来没有为复杂逻辑的“环境模拟”量身定做过计算架构。 为了克服 GPU 运行模拟逻辑时的性能短板,Ineffable 探索了多条工程与算法路径: 1. **基于模型的强化学习(Model-Based RL)**:通过训练强大的内部世界模型,将算力开销的大头从环境模拟转移到基于神经网络的生成模型上,使得模型推理的高昂成本掩盖掉模拟的开销。 2. **高效经验复用与回放机制(Replay Buffers)**:通过精细调优的经验回放缓冲区对同一批经验进行多次高密度学习,最大化单次采样的价值产出。 3. **针对 GPU 架构重构极速模拟器**:团队投入大量精力进行底层重写,让仿真环境彻底榨干 GPU 的并行计算能力。 关于强化学习领域是否存在类似大语言模型的“缩放定律”(Scaling Laws),埃斯佩霍尔特给出了极具洞察力的解释:强化学习的 Scaling Law 呈现出完全不同的动态特征。在大语言模型中,缩放定律主要由算力投入、静态数据集规模以及参数量共同决定;而强化学习的本质是一个**非平稳过程(Non-Stationary Problem)**。随着智能体在环境探索中不断取得突破,它自身生成的数据分布、面临的任务难度乃至探索空间都在发生剧烈的动态跃迁。席尔瓦和埃斯佩霍尔特表示,他们正在这片未知的理论疆界中主动探索,确立专属于自主强化学习的经验扩展曲线,为即将到来的更大规模超算集群做好理论与架构储备。 --- ### `57:40` 从 IMPALA 到 SEED RL:分布式强化学习架构的演进与传承 在节目的后半段,索尼娅邀请埃斯佩霍尔特回顾他早年在 DeepMind 和 Google Brain 主导开发的两款里程碑式强化学习系统——IMPALA 与 SEED RL,这两项工作至今仍是现代分布式强化学习工业界的事实标准。 埃斯佩霍尔特回忆道,当他最初加入 DeepMind 的深度学习团队时,几乎所有人都在单台机器的 CPU 上运行强化学习任务,实验规模极其受限。面对算力瓶颈,他与团队共同研发了 **IMPALA(Importance Weighted Actor-Learner Architecture,重要性加权行动者-学习者架构)**。IMPALA 首次彻底将“行动生成”(Actor)与“梯度学习”(Learner)解耦,使得强化学习系统能够充分压榨 GPU 吞吐并实现随计算规模线性扩展。因为该架构极其高效、扩展性极强且极易被二次开发,它迅速成为了 DeepMind 内部最流行的基础强化学习框架,几乎所有前沿团队或多或少都在使用其衍生版本。 后来,埃斯佩霍尔特离开伦敦前往阿姆斯特丹创立 Google Brain 分部,此时硬件形态已经向成机架的 TPU 演进,IMPALA 已经无法满足跨机架的硬件调度。为了在超大算力集群上进一步榨干硬件效率,他牵头开发了 **SEED RL**。SEED RL 将架构推进了一大步:它把推理(Inference)与训练(Training)全部集中并收拢到高吞吐的 TPU 算力池中,而让外围的执行环境只专注于轻量级状态转换。无论外界环境是一个简单的围棋棋盘,还是物理极度复杂的流体动力学引擎,SEED RL 都能确保加速卡全速运转而不会陷入等待状态。 埃斯佩霍尔特透露,在 Ineffable Intelligence,他们从零实现并重构了这些架构及其深度衍生算法。这套融汇了近十年分布式算力经验的代码底座,正是今天驱动他们“超级学习者”在严苛模拟炼炉中狂奔的核心引擎。 - **IMPALA 的破局点**:解决了 CPU 到 GPU 的过渡痛点,实现行动者与学习者解耦,达成线性算力扩展。 - **SEED RL 的飞跃**:专为大规模 TPU 机架设计,将策略推理和网络训练全部收拢至加速芯片中心,环境端仅保留状态转移,彻底消除了硬件空转。 - **Ineffable 的技术血统**:直接继承并进化了 IMPALA 与 SEED RL 的工程基因,成为支撑全自主进化智能体的计算底座。 ### `01:00:00` 强化学习架构的演进与 LLM 领域的“重新发现” 在讨论大规模强化学习(RL)算法与环境交互时,David Silver 指出,不同的环境模拟速度与计算开销差异巨大——既有极简、超高速的模拟环境,也有计算极其昂贵的复杂环境,这也决定了算法选择上的权衡取舍。David 强烈建议当今从事超大规模 RL 架构研发的研究人员去研读 Lasse Espeholt 早年的经典论文(如 IMPALA 和 SEED RL)。在 David 看来,当今业界广泛采用的许多分布式架构思想,都可以追溯到 Lasse 大约八年前富有前瞻性的开创性工作。 Lasse 对此深有感触地补充道,在过去的两年里,大型语言模型(LLM)社区也全面拥抱了强化学习。一个有趣的现象是,LLM 领域目前正在大量“重新发现”强化学习研究人员多年前就已经探索并解决过的方法和机制。虽然这一波浪潮在原有基础上叠加了许多创新,但底层核心逻辑的复现依然证明了早期分布式 RL 架构设计的深远价值。 ```mermaid flowchart TD accTitle: RL 架构传承与 LLM 的技术交织 accDescr: 展示早期分布式 RL 架构如何演进并影响当今的超大规模环境与 LLM 训练 subgraph EarlyFoundations["早期架构突破 (~8年前)"] A["分布式模拟环境与采样"] --> B["IMPALA / SEED RL 架构"] B --> C["高并发经验收集与解耦训练"] end subgraph ModernConvergence["当今技术融合"] C --> D["Ineffable: 超大规模 Crucible 探索"] C --> E["LLM 领域对 RL 的再发现与结合"] end ``` --- ### `01:00:58` 20人菁英团队:维持极致的“人均算力比”与严苛文化 当被问及团队规模时,David Silver 透露,Ineffable 目前刻意保持克制,全公司仅招募了约 20 个人。他们坚信当今时代不需要庞大的组织架构,一小群顶尖人才就能完成不可思议的突破。更重要的是,较小的团队规模能够维持极高的“人均算力比”(compute-to-person ratio),让每位研究员一入职就拥有充沛的计算资源,而无需从第一天起就陷入内部算力争夺战。 在人才选拔上,Ineffable 设立了近乎苛刻的标准。David 解释说,团队成员不仅要在技术上出类拔萃,还必须满足另外三个硬性维度: - **善良且具备高度协作精神(Kind & Collaborative)**:建设健康的团队氛围,拒绝自负; - **对使命深度认同(Mission Alignment)**:真正相信超越人类数据的超级学习者(Superlearner)愿景; - **技能精准互补(Complementary Skills)**:与当前研发阶段的核心需求高度匹配。 主持人 Sonya 提到,Ineffable 的创始团队中有三位曾在 DeepMind 的 AlphaStar 项目并肩作战,询问这是否像是在组建一支“复仇者联盟”。David 坦言,AlphaStar 团队本身就是当年 DeepMind 内部精挑细选的梦之队,但为了这次创业,他必须做出很多艰难的决定。即使面对过去曾合作过的杰出同事,他也不得不拒绝其中的许多人。“这虽然令人心里很不舒服,但我们的招聘门槛实在太高了。”最终汇聚在 Ineffable 的这 20 个人,既是技术上的超级明星,也是彼此志同道合、极度温暖的伙伴。 --- ### `01:04:49` 纯粹的研究导向文化:拒绝“版本 17.1”的产品内耗 Sonya 指出,当前大型科技巨头与头部实验室往往深陷算力军备竞赛的“囚徒困境”,在巨大的商业利益与资本压力下,技术决策往往容易走样变形;因此,一个专注于崇高科学使命的实验室对顶级研究者来说犹如一股清流。 David 对此表示强烈共鸣,并强调 Ineffable 建立的是一种彻底由“研究驱动”(Research-led)的企业文化: - **抵制产品迭代压力**:公司绝不会把精力耗费在为现有产品修修补补、开发平庸的“版本 17.1”(v17.1)上; - **打造科研避风港**:公司建立的初衷是让研究者能追随内在激情,全力追求他们坚信可能实现的根本性科学突破; - **远期商业回报**:团队坚信,唯有在前沿基础研究上实现真正的飞跃,未来才能催生出规模远超当前想象的下游产品与商业价值,而不是在一开始就过早受制于短视的产品定义。 --- ### `01:05:47` 未来预测:一年内验证范式,三年内见证超级智能 访谈最后,鉴于 David Silver 拥有卓越的超级预测者(Superforecaster)声誉,Sonya 邀请他对关键里程碑给出具体时间表。面对录音设备前的不确定性挑战,David 给出了清晰且激进的预判: - **一年之内(1 Year)**:Ineffable 将拿出极其强有力的证据,向世界证明“从零经验学习且超越人类数据上限”的超级学习者路径完全可行且正处于正确轨道; - **三年之内(3 Years)**:人类将迎来与真正超级智能(Superintelligence)的“第一次接触”(First Contact)。 David 强调,他对超级智能的定义并非局限于掌握人类已有知识,而是一个能够自主发现远远超越人类能力上限的新知识的智能体——全面跨越科学发现、数学难题、颠覆性技术以及全新语言体系。他坦言,自己最初的预期是 3 到 5 年,但近期由 Coding Agent(编程智能体)带来的研发自动化加速,使他坚信整个研发进程会大幅缩短。“系好安全带,接下来的发展速度将会超出想象。” ## Notable quotes > `03:35` 难道我还要去做第 100,001 个跟风者吗? — David Silver > `27:22` 做科研最有意思的,就是去探索那些答案胜负未定、处于刀锋边缘(Knife edge)的课题。如果你在动手前就已经有 95% 的把握确信它能成,那这种研究坦白说相当无聊。 — David Silver > `25:41` 试想一下,如果我们退回到 500 年、1000 年甚至 2000 年前,强制要求 Agent 只能使用当时人类所有的语言进行思考,那我们的 Agent 可能至今还会坚信‘地球是平的’,或者认为‘所有疾病都源于四体液失衡’(Four Humors)。 — David Silver > `50:00` 在物理现实世界中应当放缓节奏,但在模拟环境中必须全速前进。 — David Silver > `01:04:49` 公司绝不会把精力耗费在为现有产品修修补补、开发平庸的‘版本 17.1’(v17.1)上。 — David Silver > `01:05:47` 系好安全带,接下来的发展速度将会超出想象。 — David Silver > `10:00` 面对魔法般的奇迹与敬畏(Encountering magic and wonderment)。 — David Silver > `05:58` 它绝不可能搞砸(It can't possibly F up)。 — David Silver ## Mentioned David Silver, Lasse Espeholt, Sonya Huang, Ineffable Intelligence, Sequoia Capital, Google DeepMind, Google Brain, AlphaGo, AlphaGo Zero, AlphaZero, AlphaStar, AlphaProof, MetNet, IMPALA, SEED RL, Rich Sutton, Demis Hassabis, Dario Amodei, Ioannis Antonoglou, Lee Sedol, Stockfish, OpenAI, Google Cloud Platform (GCP), NVIDIA Rubin GPU, Atari 57 ## Video description David Silver spent two years building AlphaGo on human data. When he removed human data altogether building AlphaGo Zero, the AI surpassed AlphaGo in under two days. He left DeepMind to run that same bet in pursuit of superintelligence. David and Lasse Espeholt, co-founders of Ineffable Intelligence, explain their plan to build a “superlearner”—an agent that starts knowing nothing and discovers science, mathematics, and language for itself by direct experience instead of through a pre-training run. Silver argues that human-language reasoning is a ceiling, not a foundation, that alignment via chain-of-thought monitoring is already reaching its limits, and that the science of how superintelligences coexist should be worked out in simulation before anything touches the real world. Espeholt describes the GPU cluster and the lineage from IMPALA and SEED RL that makes the experiment possible. Silver’s forecast: strong evidence within a year, superintelligence in three. Hosted by Sonya Huang, Sequoia Capital 0:00 – Introduction 1:09 – What held true from Go to today, and what the LLM era changed 3:35 – Why leave DeepMind to start a new lab 5:58 – Why "Ineffable" (and the secret second reason) 7:03 – Lasse's path: scaling RL at DeepMind and Google Brain 9:31 – Defining a super learner and first contact 12:35 – Super learners vs. LLMs that make discoveries 13:59 – Day to day: the RL stack, crucibles and human-journey milestones 16:13 – Don't breadcrumb the objective: simple rewards, emergent complexity 21:16 – What a crucible looks like 24:14 – Go: one agent is boring, two is infinite 25:41 – Will agents rediscover language? 27:22 – Removing human priors: AlphaGo to AlphaZero 32:05 – Keeping human bias out of the crucibles 33:35 – Confidence, timing and super forecasting 36:24 – Is compute the driver? 38:13 – The ceiling on simulation and Rich Sutton's objection 43:44 – Alignment and the science of superintelligence 47:45 – Monitoring agents that don't think in English 51:01 – Pacing the frontier 52:54 – The compute setup: Vera Rubin on GCP 55:18 – Training, inference, simulation and RL scaling laws 57:40 – IMPALA and SEED RL 1:00:58 – 20 people and the compute-per-person ratio 1:04:49 – A research-led culture 1:05:47 – Predictions [Source](https://pscrb.fm/rss/p/traffic.megaphone.fm/CPUAI9233856839.mp3)

Source quotes

[3:35] David Silver: 难道我还要去做第 100,001 个跟风者吗? [27:22] David Silver: 做科研最有意思的,就是去探索那些答案胜负未定、处于刀锋边缘(Knife edge)的课题。如果你在动手前就已经有 95% 的把握确信它能成,那这种研究坦白说相当无聊。 [25:41] David Silver: 试想一下,如果我们退回到 500 年、1000 年甚至 2000 年前,强制要求 Agent 只能使用当时人类所有的语言进行思考,那我们的 Agent 可能至今还会坚信‘地球是平的’,或者认为‘所有疾病都源于四体液失衡’(Four Humors)。 [50:00] David Silver: 在物理现实世界中应当放缓节奏,但在模拟环境中必须全速前进。 [64:49] David Silver: 公司绝不会把精力耗费在为现有产品修修补补、开发平庸的‘版本 17.1’(v17.1)上。 [65:47] David Silver: 系好安全带,接下来的发展速度将会超出想象。 [10:00] David Silver: 面对魔法般的奇迹与敬畏(Encountering magic and wonderment)。 [5:58] David Silver: 它绝不可能搞砸(It can't possibly F up)。

Source quotes

[0:00] David Silver: this kind of gray mush in our head [3:35] David Silver: hundredth thousand and one person [7:03] Lasse Espeholt: Damn, I missed out on an opportunity. I should have reached out to him. [16:13] David Silver: pulling the agent by its nose [24:14] David Silver: infinite onion [32:05] David Silver: It's not like a religion that you can't have any human prior anywhere. [36:24] Lasse Espeholt: a very beautiful cluster at that. [40:00] David Silver: 10 times or 100 times faster [47:45] David Silver: tread lightly [50:00] David Silver: Accelerate in simulation so we can understand what's happening and... do science around it, and then careful in the real world. [65:00] David Silver: v 17.1 of the thing that's come before. [65:47] David Silver: Buckle up.

Original transcript

[0:00] collectively, we started with this kind of gray mush in our head. We somehow started out knowing nothing at all and somehow discovered everything that humans know today was this process of collective discovery of languages and technology and science and mathematics and all the rest of it. Everything we learned, we discovered it for ourselves. And that's an incredible thing that no one has cracked yet in AI. And so to me, that is the essence of what it will take to get to just keep learning and discovering knowledge forever. Bye. I'm thrilled to have Dave Silver and Lese Espeholt with us, two of the co-founders of Ineffable Intelligence. Great name, by the way. Fun fact, Dave, you were the first person I cold emailed to come on Training Data when we started the show 100-ish episodes ago, and it feels like a full circle moment to finally have you on the show. So thank you for joining us. Thank you. Dave, let's start with you. Your 2009 PhD was on reinforcement learning applied to the devilishly difficult game of Go. You demonstrated for the world for the first time that superhuman in one domain, machine intelligence was possible. And I think you were really the figure that inspired the entire current generation of reinforcement learning and deep learning researchers. And so thank you for... your contributions to where the field is. I'm curious, thinking back to that point in time, What did you learn? What did you have conviction on? and what has remained the same and what has changed. as AI has changed over the last five years or so. Yeah, thank you and pleasure to be here and I'm glad I finally made it after these hundred episodes. It's a pleasure to be here. So I suppose I always had this conviction starting my PhD and going into the game of Go that the right approach to solve AI was through learning from experience, through systems that discovered things for themselves, that were learning through reinforcement learning to get better and better and better. And that that would be the thing which would lead to emergence of intelligence and ultimately superintelligence. And so I suppose we were able to show that in the game of Go, but it's a conviction that has held true throughout my career. And here we are now applying that conviction ineffable, trying to show that indeed super intelligence can emerge all the way through systems that learn from experience. And in terms of things which are different, although we are not pursuing the human data path that the LLM world lessons to learn from that. In particular, it was definitely a surprise to me how a system that was trained on a massive amount of data could then be very quickly fine-tuned on a very small amount of data to kind of completely change its behavior. That was something which I think was something which sort of changed my mindset in terms of just suggesting that if you get things at the right hard or like they might be very slow, that once you've kind of experienced enough data, that after that, the things that were slow can become very fast. I think that's a lesson that wasn't obvious to me in advance until it happened. Okay, we're going to come back to that later. You could have run this program inside DeepMind, where, you know, could have probably had more access to compute than just about anybody. You decided the right thing to do was to start a new lab. Why did you do that? Yeah. So look, first of all, I had a fantastic time at DeepMind. I loved my years there. And I have a a huge amount of respect for the people there and still got on with them very well. But at the end of the day, I have a really strong belief about how we can go beyond where we are today with AI and all the way to superintelligence. And the path that I believe in doesn't actually go through LLM-based products as an intermediate point. And when I really thought about that, I thought about it really hard. I realized that I felt that so passionately that this deserved to be more than just the corner of an organization by an elite AI lab that really just... gives this approach its all and represents it 100%. And, you know, once I realized that, it felt very natural to create this company. I suppose another way to look at it is that [5:00] you know, the world already has tens of thousands, if not hundreds of thousands of really talented researchers all pursuing exactly the same approach, like really just kind of doubling down on LLMs with pre-training, post-training, coding and that kind of LLM path. Am I going to be the hundredth? thousand and one person that's like pursuing that approach? Is that the most effective use of my time and my energy? And in the end, I realized that there's this other lamppost that right now no one is looking under and where if I actually put my effort there, there's a much higher chance of really having a massive impact on the field. Because if this works, it's going to change the whole of AI. So I think that that kind of realization also meant that this was something have impact upon the field through this approach. Plus, you had a company name that couldn't go to waste. That's right. Do you want to say a word of mine, it's called ineffable? Yeah, so I suppose the name ineffable, it came from this idea that, you know, there's really just knowledge could be so incredible, like the things which superintelligence will one day discover, like this knowledge is going to be so unbelievable, incredible. And it's going to be something which is, you know, profoundly amazing and perhaps so profound that we can't even express it in human language. And so really, it's like, you know, in line with this other path that doesn't go through, you know, human language-based approaches, but instead build systems that discover things for themselves and has the possibility to discover knowledge that's unbelievably profound and ultimately ineffable. I should say there was also a secret second reason that we used, which was that, you know, it's this idea, it really, it has to work. Like if you really just double down on learning from experience, it's an approach And so you could say that it can't possibly F up. So it's ineffable. So there you go. There's the secret reason as well. Amazing. Amazing. Best lab name. Lese, what about you? You were at Google. You were in a different group than Dave in the deep learning group at DeepMind. And you spent years scaling up reinforcement learning on GPUs. What made you decide to leave DeepMind? And what drew you to co-found Ineffable? Right. Yeah. So I joined like DeepMind in January 15 and actually I started working a bit with Dave around like 16, where I was doing this work on like scaling up reinforcement learning. So as you said, I was part of like the deep learning team and he was like leading a reinforcement learning team. But we were working on an internal project together. I actually left DeepMind twice. So I left DeepMind in summer 18 and went to like Google Brain. back again. But in the day, I left DeepMind again, like a year before Dave left. And the first time I heard about Inevo was actually through one of our eventual investors. It's like reaching out to me, asking what I thought about Dave. Well, my first reaction was, "I Damn, I missed out on an opportunity. I should have reached out to him. But I recall like Dave as more often than not being one of the smartest people in the room. And shortly after that, seeing with that investor, they actually reached out to me. So it's great. Yeah. So I really love to work with Dave. I knew Wojtek and Chris from back in the day, and I thought the team was great. And I also thought it was amazing to have an opportunity like this in Europe. It should be said as well. We worked together on some of the early approaches to combining reinforcement learning with deep learning and trying to scale things up. And it should be said that Lasse, who, by the way, is very modest, but he actually worked on some of the original approaches that were the first attempts to actually scale up reinforcement learning and show that it could be done at large scale and ways to parallelize reinforcement learning and ways to kind of build at the way the field's gone. Like actually all of the ideas that people use today kind of sort of hail back to that original work that Lasse did back in the day. That's where all the roots come from of scaled up RL. So he was the first to figure it out. Wonderful. We should have you guys introduce each other instead of you. You're both too humble to brag about yourselves. Okay, let's talk about Ineffable and what you're building. You have a beautiful mission statement as a company. You're going to build a super learner. and make first contact with superintelligence. Can you define both of those things for us? What is a super learner and what does making first contact look like? Yeah, so to me, a super learner, it's something which is... It... [10:00] It's got incredible... capacity, both in its depth of what it can learn and in its breadth. So in the depth dimension, it should be something that can start from not knowing anything at all and can go all the way through to discovering things that go way beyond what humans know. So an analogy to this would be the way that human collectively, you know, we started with this kind of gray mush in our head. We somehow started out knowing nothing at all and somehow discovered like everything that humans like of languages and technology and science and mathematics and all the rest of it. You know, everything we learned, we discovered it for ourselves. It wasn't that someone came down and gave us like a blueprint of how this should work. And it wasn't that someone came down and gave like a trillion element data set of like, oh, this is what these things should look like. We actually figured it out for ourselves. And that's an incredible thing that no one has cracked yet in AI. that can just keep learning and discovering knowledge forever. So the second element is breadth, that we want something which can do this incredible discovery from all the way to incredible things. And it should be possible to drop it into any stream of experience. So whether it's like a simulated world or a digital world or a physical world, whatever it's dropped into, it should be something that can just learn and learn and learn. So that's the dream. I think if we crack this, then to me, it gives rise to a super intelligence that will be like encountering magic. which is so profoundly amazing that it will be able to do wonderful things. What will it feel like when we first encounter it? Or is it one of these things where you... You can't predict ahead of time. I think it will be amazing. I think we'll, we will, you know, that we should have an experience of like wonderment. And the only thing I can, say from my own experience, is what happened when we got to this point in an admittedly much narrower field. But when we got there in board games, you know, when we got to the point of superhuman play in the game of Go, like it brought wonderment to the field. Like the Go players who saw like Alpha Go playing, you know, they were amazed. And like it was like a joyful experience for them to kind of witness this kind of Go playing. Same happened when we got to superhuman play in chess. And, And just imagine like going far beyond that, but like in, you know, things which are discovering incredible things, new fields of science, new types of technology that we've never even dreamed of. I mean, it will be it will be incredible. So the current paradigm, these large language models with reinforcement learning seem to be, you know, at least knocking out millennium problems and making some sorts of scientific discoveries. super learner in your sense of the term. Yeah. So look, I think LLMs are doing a lot of great things and I have a lot of respect for what's being achieved. What I mean by a super learner is something which, first of all, discovers completely autonomously. And I think what we've seen so far, at least with these kind of approaches that have been taken, is there's a lot of human in the loop effort that's required to make these breakthroughs. It's not fully autonomous yet. learn from its physical experience. And we're a long, long, long way off that when it's not at all clear that LLMs and language-based approaches are even the right vehicle to achieve that kind of knowledge. Thinking about these things in terms of language might not even be the right paradigm and it might just be that all of this human data that's being used in LLMs is kind of weighing things down. So a lot of respect for some of these, the progress that's made, but I don't think we're anywhere close to this like true super learner of something which can be like autonomously just discover everything for itself. What does building this look like in practice for you? And you know, we backed Giannis who was on your original AlphaGo team and he told us that You know, one of the things that's distinct about you is you can have these incredibly ambitious... visions, but you also ground the team in very practical ways. research milestones. And I'd love to understand... Big, big, big vision. What does this look like in the day to day? And what are you what are you implementing? How do you spend your days right now? Yeah, great question. So day to day, I suppose there's a lot of hard technical work by people like Lassa to kind of build the whole stack. That means we've got a state of the art reinforcement learning stack that allows us to run things at massive scale, inhale data at the optimal way, be able to kind of learn and have systems that keep learning and learning and learning. The other aspect which we're building is we're building [15:00] thing outs environments which are are capable of super learners like discovering everything, like simulated environments in which a super learner can discover everything that we mean by intelligence. We call those crucibles. So they're environments in which it's possible for agents to make incredible discoveries in which it's possible to express and learn everything that we think of as intelligence. And so we have been putting a certain amount of effort into creating these environments. And so it's a combination of building out the super learner, doing the research to make super learners incredible and beat the state of the art of what's possible with reinforcement learning, both in terms of the engineering and the research that happens. And again, creating and placing the super learners into these crucibles to kind of see the emergence of incredible behaviors. And in terms of the grounding, you asked about how to ground this in the day-to-day. So we ground it in milestones and the milestones are kind of analogous to... you know, what happens with the human journey where we discovered incredible things. And so those become natural milestones for us, for our agents to rediscover the great moments that humans went through in our discovery of knowledge. Hmm. What do you think the Agents will... rediscover first when that's gonna be your first moment of oh this is working I imagine we're going to be surprised. Like, I think one thing which is different about the way we take to doing reinforcement learning to a lot of the rest of the field is that we are not breadcrumbe trailing the particular objective to be learned about. A lot of the way things people do RL with LLMs at the moment is to kind of say, oh, you know, the agent should be learning about this problem and then this problem and this problem. And after kind of following that trail, we're kind of pulling the agent by its nose towards a particular desired outcome. approach. We're giving the agent some simple objectives and asking it to, you know, discover things that will surprise us and amaze us. So, you know, the agent is in principle capable of discovering all kinds of things. And we don't necessarily know in advance what those discoveries will be. And it's kind of just going to be exciting to witness that journey. So I hope to be surprised, I think is the answer to your question. Interesting. So what you're saying is instead of explicitly giving the agent the goal to go rediscover language or build a calculator or build a computer, you're going to give it other that the most efficient way to actually, or the best way to actually accomplish that goal, maybe to build some of these tools that we as humans have already built. Is that one way of putting it? That's one way of putting it. I think another way to say it is people often think in RL, oh my God, how do I come up with a reward function that kind of perfectly specifies this really complicated thing that I want to discover? Like, you know, how can I specify exactly what I mean by science? Or how can I specify exactly what I mean by interesting maths? But there's an is that to give the agent something really straightforward to optimize, but in a really complex environment where there's a lot to learn about. And so it turns out that if you look at the way the world has worked in nature, the pursuit an optimization of a very simple, easy to describe quantities can often give rise to incredibly hard to describe complexity and emergence. So examples of that would be, you know, evolution, which kind of just optimize a kind of survival of the fittest, you know, some kind of fitness signal. And the pursuit of fitness gave rise to all of the complexity that we see in life. In the human suit of relatively simple kind of goals that we as humans have gave rise to all of the complex behaviors that humans ultimately acquired and all of the knowledge that we eventually ended up with. So the kind of emergent complexity is much greater than the kind of starting point that was given. So it wasn't like evolution said, look, hey, humans, here's this idea of science and you need to kind of get to the most complex science that's out there. It was really something much simpler. maybe would be, you know, perhaps that humans in the pursuit of our goals started coordinating a bit more. And then in the process of coordinating, we discovered that it was helpful to start communicating. And in the process of communicating, we started trading. And then in the process of trading, we discovered that, you know, basic arithmetic was useful. And then we started to discover that, you know, tools were useful. And then we discovered that some basic science was required to help understand how the tools would work. And that gave rise to more complex technology understanding the technology, required more complex maths, and so forth and so forth. So all in the pursuit of relatively simple goals, this incredible complexity emerged. So if we can get that process right, [20:00] fight. things are going to take off and lead to unbelievable things. Kind of similar to like human biology and evolution then, like, you know, try to make it to the next generation. What does a reward function look like for you all? Yeah, I mean, we've been experimenting with quite a number of different rewards. So it's not like we have one answer to that question. But I think one thing we're not doing is the kind of, you know, crafting a reward that exactly describes what we mean by science or exactly what we mean by technology or exactly what we mean by language. We expect our agents to discover things. And if we can't crack that, you know, like if we can't crack this problem as AI scientists, we're never going to have agents that discover whole new fields of knowledge that we don't even know about. There are vast fields that will exist that are out there, which we don't even know have the potential to exist at this point in time. It's like, imagine you're back in the days of Newtonian physics and no one even knew that quantum physics was a thing or that relativity was a thing. We should be in that process of open-minded discovery where our agents are free to kind of figure things out for themselves. And I think that's going to lead to much more exciting outcomes. like in practice and you know, you're contrasting it to today when people say environments, Oftentimes, it's a set of tasks and what the agent should have done on the task. It sounds like your environments are... Your crucibles are very, very different. What do they look like in practice? So we are... building crucibles which are without giving too many details away, have the capacity for a lot of emergent complexity. The key thing about them is it's not like we've handcrafted a whole load of puzzles for the agents to work with. There's just a huge amount of emergent complexity there. Some of that emergent complexity can come from different places. It can come from physics engines that give rise to complexity as you've got millions of interacting objects with a complex physics engine. You can have millions of interacting agents and the world becomes much more complex because it suddenly now includes the complexity of all of those other agents within the world that the agent's building. So there's ways to build worlds that are vastly more complex than the kind of standard simulators like Atari 57 or whatever it is that people have been working with in the past, and which are certainly more complex than the kind of standard benchmark kind of tasks that people are using. a world in which There's the possibility for discovery by the agent. There's the possibility for the agents to discover everything from... technology to science to mathematics to languages to everything else you know just the same way are humans all the way through from from more basic things like you know motor control or um coordination you know the the whole spectrum is possible like if you look at like a game like go right which is like very simple to represent it's just like a bitmap uh the simulation is extremely simple But you still see this like emergent behavior of something complex, right? It's like super interesting to us. So like, of course, that goal is like way simpler than we can imagine doing. But at the same time, going all the way to the quantum and realistic physics. will be very computational expensive. So we want to create this range of crucibles where we have simple ones that can execute quickly and we can iterate on, but where we can still see very complex behavior. Interesting. Yeah. It's almost like what you choose to represent is tricky, right? Because there's For example, I can imagine a lot of economics arises from game theory. and most of that is completely unrelated to physics. And so there, a crucible that represents physics is probably not that helpful. But if you have information theory represented, which agents know what, then you probably do end up rediscovering quite a bit of economics. Is that... Is that a fair way to put it? Like the primitives you choose to capture in the environment... kind of give rise to the type of learning that emerges. Yeah, like the end goal is to have something that is grounded in the real world, right? But stepping stones. I want to come back to the Go example because I really like this example. You'll be surprised to hear that I find Go a pretty good example. But what I find fascinating, which I think is what Lasse's Um, talking to here is that, you know, Go is just like a 19 by 19 grid where you're trying to kind of maximize the territory. And if this was a single agent game, it would be like the most boring game you could imagine. It would be, you know, you just place down stones on a grid until he fills up the whole grid, and there'll be no strategy whatsoever. The moment you put a second agent into the environment, you get this emergent complexity, which is suddenly, because the agent contains, [25:00] One agent has to reason about all the possible behaviors of the other agent. suddenly it kind of massively increases the complexity of the environment. And to the best of our knowledge, you know, certainly all of the compute which we sank into it when we were working on it, there was just like this endless learning curve that where without and you could continue to discover new things in this game. None of this is to say that Go is like the end point. Of course, it's still just a board game and quite narrow in a certain sense, but it shows that very simple environment dynamics can lead to kind of infinite, like, infinite onion of layers of complexity that kind of never stops. infinite to the capacity of what our computers can do, of course. Do you think that the agents will end up rediscovering... language and will that language look like ours or will it look different? I mean, my personal belief is that... anyone who thinks that human language is the end point of, of like, or the best way to reason, it's, you know, that perspective is a little naive. I think that it must be the case that there are far more sophisticated approaches to reasoning, you know, maybe using paralyzable, rich ideas complex approaches that neural reasoning or whatever it is, these approaches will ultimately be far more expressive and far more capable than approaches which are based on human language. I would also say that starting from human language as the approach for reasoning is somewhat problematic because it pulls in all of our biases about how we think agents should think. So for example, a nice way to think about this is to imagine... What if we were to step back you know, 500 years or a thousand years or 2000 years, and only like were able to use all of the language that humans used at that time. So maybe our agents would believe that the world was flat or that the diseases were caused by the four humors, or there'll be all kinds of, you know, strange beliefs that would be embedded in that. It'll be very hard to kind of remove because they're so fundamental to the kind of language and the processes of thinking that humans have used. So I do think that going beyond this is a really important goal. I'd love to talk more about this concept of removing human priors. And I think your experience with AlphaGo and then alpha zero alpha go zero uh is maybe the the best example that anyone can point to of what happens when you remove human priors is Maybe if you take us back to AlphaGo and then AlphaZero... Was it surprising to you that removing human priors would result in even smarter intelligence? I wasn't sure, honestly. So I remember... I remember... I always like to do things in research where I think you ask research questions which are like on a knife edge where it's like 50-50. I think it's not very interesting to do research where you already know the answer and you're like 95% certain this thing's going to work. It's kind of like boring, honestly, to do those things. So, you know, I think research, good research is about asking questions where you're really not sure what the answer is in advance. And in this case, I was about 50-50. And I thought that maybe there was a good chance that it might. But I wasn't confident. And it was very exciting to see it. I should say that you asked about human priors. So we are making a bet, an effortball. human data, like the whole internet of knowledge that everyone else is building their systems upon. That's an important choice. And so we should probably talk about why we're doing that. And Of course, it's, you know, a short term sacrifice that we give up a certain amount of performance in order to access something which is potentially far greater. And it comes back to this, like, you know, what does it mean to build a super learner? Like to me, building a super learner again is about building something that has this like potential for full depth and breadth. So in the depth direction, like we need to get to something which is capable of discovering things which go so far beyond human knowledge that, you know, it goes just far beyond the prior. And the more we lean on human data, the less able we are to actually discover the algorithms that are really going to take us all the way to something that's going to, for example, discover a whole new field of knowledge. And then in the breadth direction, you know, we need to have algorithms that can be placed down and discover things in environments for which language and human data we started with is just not a good fit. I mean, you know, there are just many, many areas, you know, different kinds of sensors, for example, where human data. like language data. [30:00] or human data in general is not a good fit for some of this description. And so I think by letting go of this human data, we put ourselves in a position where we can actually discover something which can get go far, far beyond it. And so if we come back to the AlphaGo example, what we found there was that The moment we let go of the human data, which we'd spent, by the way, we'd spent about two years like building AlphaGo, somewhat similar to the LLM approach where we kind of, you know, we built every level of this. You know, we kind of hand put together the next level. We kind of put in a lot of human data and then we were kind of human in the loop, like improving it, where we would kind of take the previous generation and choose how we were going to improve it to get the next generation. And so we were right in the loop of this process. It took us two years to get to AlphaGo and beat LisaDol. When we got AlphaZero right, you know, we gave up on the human data and finally focused on what would it be like to build an algorithm that could discover for itself. So once we got it right, that algorithm, it took just a few hours. Like it took a few hours to outperform the two years of work which we'd done with human in the loop. So in the game of chess, it took four hours to beat the world champion, the best ever AI that had ever been created. was a couple of days or something but it was incredible progress like and and and so the You have to let go of the human data to allow yourself to discover these things that can then shoot past the competition and discover the things that just keep learning and learning and learning forever. So, you know, it has to be an active choice. And if you don't make that active choice, you know, what I see in a lot of the kind of LLM companies today is they're basically, you know, building semi-autonomous, human-in-the-loop kind of learning algorithms with thousands of people in the loop of the learning progress. Which is, you know, it works and a lot of respect for how far it gets. But until we step back from that and really discover the process that's going to learn for itself, I don't believe we'll get all the way to something which is truly a super learner and unleashes the potential that we've... talked about so far. That's very profound. As far as I know it, there are humans that are ineffable, and some of you are creating environments or crucibles and... What does it look like to actually try and make sure that human priors aren't kind of leaking into what you're doing? - Yeah, well, so when we design environments, and like rewards function and so on. Like obviously there's a human... bias in there, but we don't want to, as Dave was also mentioning earlier, we don't want to bias it towards building a machine based on the von Neumann architecture or something like this. We want to... But if you're stating some problem where like a machine will be useful. It'll be very interesting to see what type of machine it comes up with, like what computer-like machine it comes up with. So I totally agree with this. I would also say that... we're not religious about this. It's not like a religion that you can't have any human prior anywhere. It's more like a pragmatic position that, you know, until we give up on, on like relying on the trillion, like item data sets of human knowledge, that we're not going to make the progress that we, that the field deserves to see. So it's more like a, actually a practical position. And so in practice, of course, there's going to be a certain amount of priors, which like, as Lassa says, there'll be priors, which, worlds, as opposed to trying to put in these very strong priors that exactly encode how all the knowledge we know about this one particular problem or this one particular world. So kind of weak environment agnostic priors seem perfectly reasonable, but kind of these super strong priors with trillion item data sets, we're trying to avoid that because we think pragmatically it's going to lead to better outcomes. Yeah, absolutely. You mentioned that you like philosophically that you're drawn to research that's more on the knife's edge. 50-50. I'm curious what your confidence levels are for, you know, ineffable broad at the broadest level, like, does it will a super learner emerge? And then, I guess at the more pragmatic level in terms of the things that you're trying to accomplish right now. Are you 50-50 on these as well? So first of all, I would say I strongly believe that the approach will be feasible at some point. In fact, it's the only thing for which we have an example in our brains. It's the only existence proof of something which has discovered an incredible intelligence that's very general and works in an embodied world and discovers how to do all kinds of different things. So we have this incredible example of the human brain that we can look at. So then, you know, do I think we can achieve it? Well, I think like many great, like... moments in science, it's about timing. So, you know, whether this succeeds or fails, I think, is about timing. Like, when you create a startup, you want to be at the right time. You don't want to be [35:00] too early or too late. If you're too late, everyone's done it. It's not interesting, like a thousand other people have already done it. And if you're too early, you might be banging on the door, but not have the right kind of, maybe the field isn't advanced enough or there isn't enough compute to be able to succeed yet. So to me, it's all about timing. For example, with Computer Go, I made a prediction, I think 10 years before Computer Go was solved and I got it Got it right for StarCraft, which I also we had a bet and made a prediction. Same for alpha proof when we were predicting when the IMO was going to fall. So you're a super forecaster of sorts. So, so far, so far, I've done very well at this. And I do have a very strong sense that this is the right time for this to fall, that super learners are possible. methods or auto research methods like we've produced in the past. The ability and understanding of scaling laws and the sophistication of technology in the stack. There's a lot of things that fall into place that I think make this the right time. So that's my bet that this is the right timing. But yeah, of course, this is research. Research is never guaranteed. And that's what makes it exciting. If you get the timing right, you can be the first. You can be that's... That's what it means to take risks and do something exciting. For an affable and also all the previous things that you correctly forecasted, is the biggest driver of timing compute? I think it's a very big driver. I think it's an enormous driver. I think, yeah, I mean, we're... in a great position with the compute that we've managed to get hold of. And I think it gives an opportunity to do things which are orders of magnitudes, more sophisticated than approaches that have tried to kind of do self-discovered RL in the past. So I think, yeah, if we compare to, I don't know, something like hide and seek from OpenAI, which was like an early example where agents discovered kind of some basic tool use, I think we could probably run experiments that use 100,000 times more compute. And so it's kind of amazing that people haven't come back and really pushed on these types of approaches, And of course, hide and seek, I don't want to pretend that that's the end goal. I think it's very basic, but I think it just is suggestive of what could be done. Like if we really, you know, so... I think Compute is a huge driver, but it's not the only one. Yeah. I think we're in a very fortunate situation that we have been able to secure a very large cluster, right? A very beautiful cluster at that. Yeah, so this is really an opportunity for us, right? We really have the cards on our hands to test this out. Very few other people are doing something similar. So yeah, we can really test this out in a way that we couldn't do at DeepMind, for instance. Absolutely. It seems like one of the maybe determinants of how far this can go. is how far you can push... simulation. And these ultimately, these crucibles are You know, they're digital, right? And I think simulation has probably come quite far in the last handful of years. uh but it's not it's not perfect Right. We recently had your friend Rich Sutton on the show and I asked him about simulation and he said he was very dismissive of simulation. I'm curious how you guys think about this and Lysa you've been on both sides of this. I think you... He obviously did a lot. in the digital world, but you also built... MetNet. which touches real-world atmospheric data. How do you all think about... Thank you. Where is the ceiling on simulation and when is the right time to bring an affable into the real world? So I guess MedNet was actually like an example. So this is like a weather forecasting model we did in the Google Brain Amsterdam office together with Google Research. where We were doing a machine learning model to predict short term weather forecasts. there's way better than the physics-based simulations. So the physics simulations were accumulating errors over time. So there's a point to that, that the physics simulation is not perfect. I do think there's a lot of relatively simple environments that we can build where we can see this emerging behavior of, let's say, technology and so on, where we don't need exact physics. I actually do think it's interesting to see how much we can build these days. We get like a very small team in terms of these simulations and so on. [40:00] Well. There's definitely like a ceiling. Is there a ceiling actually? I'm curious about this. Yeah. Well, I guess quantum physics and whatnot, right? So there's like a ceiling somewhere, but we can do like very interesting things without real physics, I would say. So I think the right question to ask is, can a superintelligence be built in simulation? So, If it is possible for it to be built in simulation, I believe it will happen. 10 times or 100 times faster just because simulation can be infinitely parallelized, you get perfect reproducibility, has a lot of really nice properties. It's also the safest place for a superintelligence to be built. So it has these really nice properties. then we need to ask, is it possible? And I think for the reasons we discussed earlier, there are very clear approaches that we can add into the simulation that allow it to be sufficiently complex to represent arbitrarily interesting things. So, for example, let's say you wanted to build a computer in simulation. Well, all you require is that your simulation is Turing complete. If you have a Turing complete simulation, suddenly everything that we mean by computation, like everything, like provably, everything we mean by computation can be expressed completely inside simulation. And indeed, humans have built incredibly complex computers or related things inside simulated worlds. The same is true for various other ideas. Like, you know, if you want to build an economy, like, you know, money can be built out of out of bits. And as long as you have one of those, you know, some representation of money, everything that we mean by economics can be expressed inside a simulation. The same is true for language. If you've got multiple agents and they've got some ability to express a channel of communication between the agents, suddenly everything that we mean by language is expressible in simulation. Mathematics we know is expressible in any universe. Science in some sense is the study to study what we mean by science. Well, there are some aspects of science which are like expressible in many things, like, you know, energy or entropy or evolution. But also it's true that the agent will discover its own science, whatever the simulator is. If it's got this emergent complexity to its environment, the agent will discover its own science about whatever that world is. It's very... like provably possible to build... environments in which it's possible to express and represent enormous complexity of all of these things. Well, I agree with LASO, there might be some things which aren't well expressed, you know, there will be things which can only be expressed in the real world, like quantum computation is a good example. The real point is that if it's possible to get all the way to a superintelligence in simulation, it will happen faster, and it will happen safer. And so for those reasons, it's something that we choose to put a lot of energy into. I actually agree with Rich Saturn I think his point was something along the lines that you shouldn't expect that the things that you learn in simulation to transfer sim to real into the real world, because the real world is just vastly more complex and in very interesting ways that are hard to simulate. But I think what we can do is we can do all of the science to discover the super learner. And once we have that super learner, we will have the answers and we'll know how to build something which that same algorithm should be applicable to the real world. also make our simulations more and more high fidelity towards the real world and check that they still work before putting them into the world. Great answer. Thank you. I'd love to talk about alignment. I know there's a politically charged topic right now, and so I'm less interested. I'm zero interested in having the politics or policy discussion around alignment, but I am very interested in the scientific aspect. discussion around it. It seems like so much of the misalignments that's emerged. has arisen from scaling reinforcement learning to the extent that it's been scaled. Do you think that the super learner you build will naturally be aligned to human interests and have human values? Is it important? For it to have... alignment with human values. So this is a big topic. So let me start by saying a few things. So where we are today with alignment, we are essentially relying very heavily on alignment approaches that are based on the language that the agents use, and priors that are based on human data. That has been reasonably been very effective up to this point, but we're starting to see that that approach is reaching its limits. And by itself, I think it's going to be difficult to see a path [45:00] off. where as agents become inevitably more and more powerful, we can already see this happening as agents are trained more and more, they are kind of going to start to overwhelm their priors. They're going to start to learn to say one thing and act in a different way. And it's going to be increasingly difficult to align to the things that humans want them to do. So my belief is that we need something more and that this by itself is not sufficient. The most important single thing that we can do as a company is to actually understand the science of super intelligence. So this means to say that if we actually can in simulation really build a super intelligent agent and start to see these super learners like go all the way, we can start to actually understand what it takes for a super intelligence to actually interact peacefully. make sure that a society of super intelligences actually interacts peacefully and in particular interacts peacefully with other lesser intelligences that are also going to be in their world. And I we actually can get to some answers, like to a real science that will help to understand what the right things to do are in the real world, because we're hurtling towards that real world outcome where there are going to be a whole load of different interacting super intelligences. And we don't quite know how that's going to play out. And I think the faster that we can actually get to these answers in simulation and publicly share that, the better situation this to kind of gate the actual decision making of like what we do in the real world. So, you know, one of our commitments is that we will gate anything we do in the real world on having sufficient evidence from this kind of science of superintelligence that we are really confident that things are going to go well. And we'll only push forward to the next level of what we do in the real world once we are absolutely confident that that can lead to a good outcome. So I think world where a lot of things are happening very quickly. We're hurtling towards an outcome with lots of interacting super intelligences. I personally believe that this will lead to a good outcome for humanity, but it's happening very quickly. And I think that if we are able to actually understand this and run ahead in simulation, we'll be in a much better place do the things that ensure that this goes really well. Fascinating. And it seems like one of the primary things that current safety researchers rely on is being able to monitor... the agent's chains of thought in English. And so when they start speaking the release, that scares people because we can no longer understand the monitor as well. In your case, you have your agents kind of going straight to rediscovering language. What do you think that means for for monitoring and is... Or is the point: That, you know, if you get the reward right and if you get... the environment's right, then you shouldn't need to monitor. Yeah, I think monitoring is something which is still really helpful. Like it or not, I believe that the approach of monitoring the human language that these systems use is going to be... increasingly difficult to, as agents get more powerful. And we're already seeing that, that the monitoring agents are, maybe they're producing like a tome of, you know, a million words of reasoning, or maybe they're actually actively, you know, using subterfuge to kind of think in one way and then act in another. It's increasingly difficult to rely 100% on that. So I do think we need other elements in our armory anyway. So I think it's actually really important that some people are looking at how to get this right. I think there are ways to monitor agents that don't use language. like start to, for example, you can use shadow actions where you start to have actions taken where you can look at what they would actually do if they were allowed to act in the world, but you don't actually let them take that action yet. You can start to have other super intelligences which are acting as the monitor. So for example, you might have the idea of like an incredible scientist agent that sort of treads lightly upon the earth. And then you have another super intelligent that's kind of monitoring the kind of magnitude of its actions and regularizing it to make sure it can't do things which are kind of affecting the world too much. I think there are lots of ideas. I would say that there's a lot of research to be done here. I'm not claiming to have all of the [50:00] serves a lot of investigation. And if people aren't looking at this because they're only looking at the language-based approach, I think we're in trouble because I think that language-based approach is not enough in itself to actually get to where we want. I should say in terms of having said in trouble. I am optimistic about where the world is going. So by trouble, I mean that we're not going to have agents that do the things that we want them to do. I do think we're going to end up in a world with lots of... complex, interacting, super intelligent beings. I'm optimistic that that will lead to a kind of a balance of actors, some of which are supporting the flourishing of humanity, and some of which may not be, but the ones which do support the flourishing of humanity will end up kind of supporting us and protecting us against the ones which aren't, you know, acting on our best interests. So that's the world I think where we can work towards. Yeah. Well, given your super forecasting abilities, I'm happy to hear that. What do you think about all the conversation around pacing the frontier? I'm actually very... March. supportive of the idea that that there should be pacing of what happens for... the frontier of AI in the real world. At the same time, I think we should move as fast as possible in simulation. Now, the reason is that I think in simulation, we can actually develop a lot of things like the science of superintelligence that will actually help us to find solutions that help things to go well in the real world. And I think that kind of work should happen as fast as possible to make sure that when things really happen in the real world, that things have the best possible go well and lead to human flourishing. So I think that would be my thought. But yeah, in general, I'm supportive of the, yeah. suggestions, like the kind of direction that people like Demis and Daria and so forth are suggesting. I think it's generally a good idea. Maybe part of that is that I think that things are moving really quickly, Thank you. I do believe that when we accelerate AI and get to superintelligence, it will lead to a really great future for humanity. But I also think that it's something that the speed at which things are moving is not something that we are capable of adapting to that fast. So I think just by pacing a little bit, I think it gives us the opportunity, A, to be able to do that. for our governments and our organizations and our emotional responses to, to adapt to the changes that are coming. And also for, for this work to, to, to, continue a pace at like really figuring out in simulation how to, you know, the real science of doing this right. So accelerate in simulation so we can understand what's happening and, you know, do science around it and then careful in the real world. Yeah, I think that would be my summary. Yeah. I'd love to transition to talking a little bit about compute. This is going to be an incredibly compute-expensive program. journey and you are pushing the limits of what compute can do. I think both Google and NVIDIA have been great partners for you. Can you talk a little bit about your cluster? You're not simply renting spot compute from NeoClouds like many other NeoLabs. You're actually doing some really serious work on your cluster and the different other forms of compute that you have. Can you just talk about your compute setup? Right. So, yeah, as I mentioned earlier, we are very fortunate with the compute that we have. We are the first customers of the largest group and cluster on GCP by the end of the year. well actually started like one month now um so we are like extremely fortunate in that and i actually feel quite bad about like the startups that are like getting compute now because it's becoming like harder and harder um but yeah so we have like this like very cool group and cluster which is like based on tpus right like nvidia tpus which is really fortunate for us because it means that we can easily do simulation um and like training and inference right they are quite like the Rubens specifically, they have this architecture where like one rack can basically consider like one machine. There's like a very high bandwidth between the nodes and one rack, which is very important for reinforcement learning where you have like really fast iteration cycles compared to LLMs that are relatively speaking quite slow. So that's quite fortunate for us. I think like some of the things we did like eight, 10 years ago at Google, where We basically had abundant CPUs at Google that doesn't scale anymore. So we really need to make sure that everything we do is on the GPUs and we don't need millions of [55:00] CPUs beyond these GPUs. So our simulations can run on the GPU inference and training and so on. So we have some good architectures for that. I think like the LM companies now have kind of like the problems that we had many years ago where the need like sandboxes running on CPUs, assimilating computers and so on. And we're trying to not put ourselves in that situation. All the computers, is it roughly fungible between the different things that you're doing? I imagine and And then you're kind of doing these big rollouts where the agents are... are actually learning? Is that the rough buckets of what computers are being used for? Yeah, you could split it up into... training, inference, and simulation. Is the same compute equally good at training, inference, and simulation? No. So I think it's pretty clear what has happened with GPUs over the last many years, or like since deep learning took off, is that they've been more and more tailors towards training. And maybe in the last few years, also inference, but less about simulation. this amount of compute and experiences, then we get a certain amount of quality out of that. So I guess if GPUs aren't great for scaling simulation, then what's the solution? Right, so there are many different solutions here, right? So there's like model-based reinforcement learning where like the model becomes much more expensive compared to the simulation. So there's many different tricks you can do. There's various approaches for like replay buffers, or like reusing data many times. So yeah, there are ways around that. And other than that, we do spend quite some effort on just making the simulations as fast as possible and make the best use of the GPUs. Are you seeing scaling laws yet or is it too early to say? So for reinforcement learning, like scaling is all also like slightly different. So like scaling loss is the component of how much compute you put in, how much data you put in and the model size. Well, reinforcement is a little bit different because the problem is like non-stationary. So as you progress in the problem, the data changes, the data distribution changes. So they are a little bit different. It's like a very interesting research direction. I understand the difference, but are you seeing some sort of scaling law where more compute in gets you closer to achieving the goals, or is it hard to say right now? Yeah, but it's something we are actively pushing towards. And especially as we transition from a smaller cluster to the bigger cluster, we want to be ready for that. Yeah. Lese, at the beginning, when David introduced you, he mentioned that you had done some very foundational work. to compute some algorithms for reinforcement learning, upon which the field really relies on today. Maybe talk about some of that work, and in particular, your work around Impala and Seed. What are those and how do they relate to the work that you're doing at Ineffable? Right, so when I joined the deep learning team at DeepMind, most people were running single machine CPU-based reinforcement learning. So the scale was quite small. So what I did together with colleagues was to make an agent reinforcement learning agent that really utilized GPUs that could scale linear with compute. I think partially because it was very efficient, it was very good at scaling. And the third thing was it was very easy to extend to it being the most popular reinforcement agent at the company. And I think at one point, most our team used it or like derivatives of it. So that was quite successful and quite a nice project to work on. Then like later on, like we got TVUs, right? We got like racks of these TVUs. So we need to scale beyond that. So some of my work at Google Brain, after I left DeepMind to found the Google Brain Amsterdam office was to scale, to like racks of TVUs and that product was called like seed RL. So that was like really about utilizing the TV used both for inference and training in the best possible way, while keeping the environments as flexible as you want it to. It could be like anything from go to very expensive physics simulations. And how's that being used at Ineffable? Yeah, so we actually implemented a few of those algorithms, also derivatives of these algorithms. So that is what we are using today. depending on the environments, we use different [60:00] Algorithms, right? There are pros and cons between them. So yeah, we talked about some very simple, fast to simulate environments and also some that are more expensive. So for those who are working on large scale RL architectures today and haven't read those original papers, it's definitely worth a look because I think you'll discover that, you know, many of the ideas that are currently in use, like kind of trace back to those early work that Lasse kind of did and figured out, which was, yeah, a little bit prescient, I would say. like eight years ago, maybe. Like the LLM community is also using reinforcement learning, in particular, last two years. So and like a lot of the things that they're using are, you know, being rediscovered approaches that we used many years ago. So it's kind of like interesting to see with a lot of novelty as well on top of it, right? But yeah. Can you talk a little bit about your team? How big is the team right now? We have intentionally kept the team relatively small. So we're not in a hurry to build a giant organization. I think it's, you know, you can do an awful lot nowadays with a small number of incredibly talented people. So we have hired 20 people in total. And yeah, the people we've hired, it's about 20. finding some of the exceptionally talented people. But it's not just about talent for us. It's also about bringing together people who are kind, collaborative human beings. That's really important to the kind of culture we're trying to build and people who are really well aligned with the kind of mission that we're following and people that sort of fit the needs that we have. So we try and build the team with the right. set of skills as well. So all of those four things are really important. And we kind of have very high standards and very high expectations of the people that come in. So we're very lucky to get some amazing, really amazing people. one person. So it's really also to keep the compute to person ratio quite high. So we're not like fighting for resources from day one. I think three of the co-founders worked together on AlphaStar. Many of you did work together. Are you assembling the Avengers from, especially given the recent changes from DeepMind, are you assembling the Avengers of folks that you've worked with in the past? Have you hired new people? And if so, kind of what are the attributes that make somebody new to you? likely to be a fit and ineffable. So it's a mixture. So, I mean, AlphaStar was an exceptional team. It was sort of intentionally assembled within DeepMind by choosing the kind of best of the best to work on that project. for startups and we hire people who are brilliant um but also kind and nice people to have around um who are going to work together well who are aligned with the mission and who uh you know fit the things that we that we need so um but in terms of finding those people we put a lot of effort into getting the right people so as uncomfortable as it is i've had to say no to a lot of people i've worked with in the past just because our bar is so super high yeah i've worked i've been fortunate to work with some amazing people and so you know I've been choosing amongst like a a very exceptional crop and bringing in yeah what I consider to be a yeah superstar team um and so yeah it's it's um it's it's not easy to get into ineffable but uh the people who we have are wonderful human beings and yeah very very proud of the people we brought in yeah I can see why people are drawn to working the biggest... clusters, the biggest labs. are kind of They're locked in this... prisoner's dilemma, race, and there's a lot of dollars at stake, and so... decisions get made differently. I can imagine for a set of people that especially that really, really deeply believes in the scientific mission of what you want to accomplish, this must feel like a breath of fresh air. Yeah. [65:00] right and we've tried to establish a research-led culture so it's it's not a company that's led by you know building an incremental next product v 17.1 on the thing that's come before it's a company that's built on researchers pursuing their passion and trying to produce the breakthroughs that they believe are possible and we center the company around that from the beginning and from the outset to create like a haven for research, you know, where we push back on product pressure, because we believe that if we make these breakthroughs, it will lead to, you know, far bigger products downstream and far bigger successes than if we kind of start off from day one on like, you know, here's the product we're doing from the beginning. So yeah, so it's research led, we're creating, I think, a really special environment. And yeah, people are loving it. I'm loving it. Wonderful. I'd love to close with some predictions, maybe especially since you're such a super forecaster. What's your guess for when you'll have the first strong evidence that the approach is working? And then what's your forecast for when we'll achieve the mission that you've set out to achieve, this first contact with superintelligence? Yeah, so these are fair questions. So there are going to be some reasonable bars of uncertainty on this Yeah but you being recorded so we going to hold you to it Exactly yeah So I think we will see some extremely strong evidence that we on the right track within a year And I think we will achieve something which is a true superintelligence, according to the definition I gave earlier, of something which discovers things which are far beyond what humans can achieve across everything from science, mathematics, technology, languages, et cetera, et cetera. I originally thought within three to five years, my timelines have accelerated a little bit. So I'm going to say three years because of what can be done with coding agents to accelerate things. I think, yeah, I think things are going to move fast. Buckle up. It's been your journey. Wonderful. Let's say, David, this is a wonderful, wonderful conversation. I'm so inspired by what you're building and thank you for taking the time to sit down and share with our audience what you're building today. Thank you, Sonia. Thank you. It's been a pleasure. Thank you.