Meta's LeCun Warns LLM Scaling Will Not Yield Human-Level AI
Alex Kantrowitz
Language models cannot make scientific discoveries or reason because discrete token prediction lacks an internal world model, requiring AI development to shift toward sensory-trained architectures.
AI-assisted summary of Why Can't AI Make Its Own Discoveries? — With Yann LeCun — original source. Check the original for full context.
Source published:
Key ideas
- Yann LeCun argues that LLMs will never achieve human-level intelligence or make original scientific discoveries because they lack internal world models, operate solely on discrete token retrieval, and face severe diminishing returns after exhausting natural public text. (0:00)
- True intelligence requires four foundational pillars absent in current language models: understanding the physical world, persistent memory, reasoning beyond token-space generation, and the ability to plan action sequences toward goals. (29:37)
- Pixel-level video generation models like Sora cannot serve as real world models due to the mathematical intractability of predicting high-dimensional continuous distributions, necessitating non-generative architectures like Joint Embedding Predictive Architecture (JEPA). (40:44)
- Enterprise generative AI adoption faces a severe last-mile reliability barrier, with only 10% to 20% of proof-of-concept projects advancing to production due to a persistent 5% hallucination rate and high inference costs. (21:13)
- Open-source research fundamentally outpaces closed labs by recruiting global talent and decentralizing innovation, exemplified by foundational breakthroughs like ResNet originating in Beijing and the initial LLaMA model being developed by a 12-person team in Paris. (54:33)
Detailed notes
## Background This episode of the Big Technology Podcast, hosted by Alex Kantrowitz, was published on YouTube on March 19, 2025. It features Meta's Chief AI Scientist Yann LeCun in an in-depth conversation exploring why generative AI models struggle to make original scientific discoveries despite ingesting the world's text. According to the show description, the discussion covers the limits of large language models, the need for abstract world models and non-generative architectures like JEPA, the risks of investor disappointment, and the strategic importance of open-source AI after DeepSeek.
## TL;DR Yann LeCun argues that large language models are fundamentally incapable of making authentic scientific discoveries or achieving human-level intelligence because they operate strictly on statistical token prediction rather than internal mental models of the physical world. While industry scaling of autoregressive text and pixel generation faces diminishing returns and fails to solve basic logic or planning, a four-year-old child acquires vast intuitive physics through rich perceptual data with far less computational overhead. LeCun advocates for non-generative architectures like the Joint Embedding Predictive Architecture (JEPA) that predict actions in abstract latent space to enable genuine reasoning, verification, and goal-directed planning. Finally, he stresses that progress cannot be bottlenecked within secretive proprietary labs, defending globally distributed, open-source scientific collaboration as essential for long-term technological advancement.
## Atmosphere & style The conversation carries an intellectually grounding, collegial, and refreshingly pragmatic tone that cuts sharply through prevailing Silicon Valley hype. As long-time interlocutors with a decade of rapport, Kantrowitz plays the engaged, probing surrogate for public and market anxieties, while LeCun delivers candid, authoritative, and occasionally sardonic dismissals of overfunded AI claims. Listening to the dialogue feels like sitting in an advanced computer science seminar where theoretical rigor meets historical perspective, moving briskly from foundational neuroscience and classical AI dilemmas to quantitative data comparisons. The emotional arc transitions from an unsparing deconstruction of current LLM limitations into an energizing, scientifically optimistic defense of open research and abstract world models.
## Bulletin - [00:00](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=0s) Yann LeCun argues that LLMs will never achieve human-level intelligence or make original scientific discoveries because they lack internal world models, operate solely on discrete token retrieval, and face severe diminishing returns after exhausting natural public text. - [29:37](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=1777s) True intelligence requires four foundational pillars absent in current language models: understanding the physical world, persistent memory, reasoning beyond token-space generation, and the ability to plan action sequences toward goals. - [40:44](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=2444s) Pixel-level video generation models like Sora cannot serve as real world models due to the mathematical intractability of predicting high-dimensional continuous distributions, necessitating non-generative architectures like Joint Embedding Predictive Architecture (JEPA). - [21:13](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=1273s) Enterprise generative AI adoption faces a severe last-mile reliability barrier, with only 10% to 20% of proof-of-concept projects advancing to production due to a persistent 5% hallucination rate and high inference costs. - [54:33](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=3273s) Open-source research fundamentally outpaces closed labs by recruiting global talent and decentralizing innovation, exemplified by foundational breakthroughs like ResNet originating in Beijing and the initial LLaMA model being developed by a 12-person team in Paris.
## Key takeaways - [00:00](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=0s) Society has conflated artificial intelligence with LLMs, which are merely statistical token retrieval systems incapable of formulating new questions or inventing novel scientific concepts. - [01:55](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=115s) Human thought does not occur in language; biological language centers like Broca's area function as modular output layers for underlying mental representations and world simulations. - [05:40](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=340s) Techniques like Chain of Thought merely bolt superficial patches onto autoregressive models rather than performing genuine reasoning through structured solution spaces. - [10:40](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=640s) LLM scaling faces diminishing returns as public internet text (10¹³ to 10¹⁴ tokens) is exhausted, driving up data generation costs while failing to fix basic logical deficits. - [13:30](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=810s) Current systems lack Kahneman's System 2 deliberate planning, preventing them from solving novel, zero-shot real-world problems the way humans and animals naturally do. - [16:29](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=989s) Massive capital expenditure into current LLM architectures will not produce human-level AI, though enterprise and consumer inference demand protects foundational infrastructure from immediate obsolescence. - [23:26](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=1406s) The gap between impressive lab demos and reliable production systems mirrors historical pitfalls like IBM Watson and the 1980s expert systems collapse. - [29:37](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=1777s) Four essential capabilities define real intelligence: physical world understanding, persistent memory, authentic reasoning, and goal-directed planning. - [31:58](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=1918s) Artificial General Intelligence will not emerge overnight from an isolated startup, but will require years of iterative, open scientific discovery across the global research ecosystem. - [33:19](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=1999s) Fixing common physical reasoning errors in models via supervised post-training is brittle and fails against the infinite long tail of real-world physical dynamics. - [36:28](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=2188s) While frontier models ingest 10¹⁴ bytes of text across 400,000 human reading years, a four-year-old child processes the same byte volume of rich sensory data in just 16,000 waking hours. - [40:44](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=2444s) Video generation systems do not comprehend physics; attempting to predict future frames at the pixel level forces models to model chaotic noise rather than underlying causal reality. - [43:33](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=2613s) Autoregressive self-supervised learning works on text due to discrete token vocabularies, but becomes mathematically intractable when applied to high-dimensional continuous video spaces. - [49:00](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=2940s) The Joint Embedding Predictive Architecture (JEPA) avoids pixel generation by predicting abstract latent representations, discarding unpredictable noise while capturing fundamental scene dynamics. - [53:06](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=3186s) Action-conditioned JEPA models establish true planning and reasoning by chaining abstract future state predictions to search for optimal action sequences. - [54:33](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=3273s) Open-source development moves faster than proprietary silos because no single entity holds a monopoly on breakthrough ideas or can centralize all global research talent. - [56:39](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=3399s) Historic deep learning milestones like ResNet from Beijing and LLaMA from Paris illustrate that cutting-edge AI breakthroughs are decentralized and not exclusive to Silicon Valley.
## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=0s) Introduction and the Mystery of Missing Scientific Discoveries
Alex Kantrowitz opens the show by asking why generative AI models, despite having absorbed the entirety of written human knowledge, have yet to produce an original scientific discovery of their own, and whether these systems are finally learning how the physical world works. He welcomes Turing Award laureate and Meta Chief AI Scientist Yann LeCun to discuss the fundamental limits of the current AI paradigm.
Kantrowitz presents a challenge first raised by Dwarkesh Patel: if generative AI models have memorized basically the entire corpus of human knowledge, why haven't they made even a single novel connection leading to a scientific breakthrough? A moderately intelligent human with that volume of memorized facts would easily notice cross-domain patterns—such as observing shared symptoms across disparate conditions to deduce a medical cure.
```mermaid flowchart TD accTitle: LLM Limitations vs True Scientific Discovery accDescr: Diagram illustrating how current LLMs process text through memorization and retrieval compared to the human scientific discovery process.
subgraph LLMApproach["Current LLM Architecture"] A["Text Corpus Memorization"] --> B["Statistical Pattern Matching"] B --> C["Token Retrieval & Output Generation"] C -.->|"Fails at"| D["Original Hypotheses & New Insights"] end
subgraph HumanDiscovery["Human Scientific Discovery"] E["Abstract Mental Models"] --> F["Questioning Axioms & Assumptions"] F --> G["Formulating Problems & Equations"] G --> H["Novel Discoveries"] end ```
LeCun responds immediately that while we should expect breakthroughs from AI in the broader sense, we should not expect them from large language models (LLMs). Society has conflated the entirety of artificial intelligence with chat-driven LLMs. In reality, LLMs are trained strictly on text to regurgitate, retrieve, and output sequences that match the statistical properties of their training data. While useful, they also hallucinate false facts and are fundamentally unsuited for original invention:
> “Well, from AI yes, from large language models no.” — Yann LeCun > > “They really in their purest form, they are incapable of inventing new things.” — Yann LeCun
[01:55](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=115s) Why LLMs Cannot Become "Einstein in a Data Center"
Kantrowitz brings up an argument shared on LinkedIn by Thomas Wolf of Hugging Face, who noted that building an "Einstein in a data center" requires far more than knowing existing answers. It requires a system that questions consensus, writes "what if everyone is wrong about this?", and asks queries nobody else has dared to articulate when textbooks and experts dictate otherwise. Kantrowitz asks if an LLM can be taught to operate this way, to which LeCun answers with an emphatic "no."
Future architectures that achieve this will not be LLMs, though an LLM might serve as a modular subcomponent. LeCun draws an anatomical parallel to human cognition: in the human brain, language production is handled by Broca's area, a localized patch of tissue. That is our biological LLM. Yet humans do not actually think in language; thought occurs via underlying mental representations and internal world models that exist independently of speech. Even people unable to speak can think.
LeCun explains that research demands distinct, non-overlapping cognitive skills: - **Retrieval & Memorization:** The ability to retain and recall vast libraries of existing information (what LLMs excel at, akin to recalling an obscure equation solved a century ago). - **Problem Solving & Execution:** Taking a well-defined prompt or set of constraints and solving it mathematically or logically (the skill tested in school exams). - **Framing & Questioning:** The hardest aspect of scientific discovery, which involves questioning assumptions, seeing novel conceptual angles, and deciding which questions are actually worth asking.
Current language models perform only the first task. They cannot solve fundamentally new problems outside their training distribution, and they cannot formulate the right questions to challenge established paradigms:
> “To create an Einstein in a data center, we don't just need a system that knows all the answers, but rather one that can ask questions nobody else has thought or dared to ask, one that writes 'what if everything everyone is wrong about this' when all textbooks, experts, and common knowledge suggest otherwise.” — Thomas Wolf (quoted by Alex Kantrowitz) > > “In the human brain, that's done by a tiny little brain area right here called the Broca's area... that's our LLM, okay, but we don't think in language, we think in mental representations of a situation.” — Yann LeCun
[05:40](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=340s) The Flaws of Reasoning via Chain of Thought and Token Space
The discussion shifts to modern "reasoning models" that use step-by-step thinking. Kantrowitz asks whether one could program counterintuitive, heretical thinking simply by instructing a reasoning model to question its directives. LeCun clarifies what reasoning actually entails and dismisses the notion that current approaches represent a true reasoning breakthrough. Rather than shifting paradigms, the field is merely bolting crude patches onto autoregressive LLMs.
Techniques like Chain of Thought prompt the model to generate superfluous tokens so it spends more compute on an answer. While this produces superficial improvements, LeCun emphasizes that it is not authentic reasoning. In classical computer science and AI, reasoning entails searching through a structured space of candidate solutions against an explicit verification criteria. Generating a stream of text tokens and using an external verifier to pick the best output is as inefficient as writing software by randomly stringing together grammatically valid code instructions and hoping one executes properly.
```mermaid flowchart LR accTitle: Token Space Generation vs Abstract Mental Planning accDescr: Contrast between how LLMs attempt reasoning via text tokens versus how humans manipulate continuous mental models.
subgraph TokenSpaceApproach["LLM Token-Space Search"] direction TB T1["Generate Candidate Text Sequences"] --> T2["External Verifier Checks Tokens"] T2 --> T3["Select Best Guess"] end
subgraph AbstractModelApproach["Human Abstract Reasoning"] direction TB M1["Internal World Model"] --> M2["Continuous Mental Simulation"] M2 --> M3["Direct Plan Formulation"] end ```
Crucially, neither humans nor animals reason in "token space." When a person plans how to assemble a table, they do not churn through competing linguistic descriptions; they manipulate continuous spatial representations in their mind. LeCun offers a concrete visual thought experiment: imagine a cube hovering in the air and rotate it 90 degrees around its vertical axis. A person instantly knows the resulting shape looks identical to the original cube without using words or generating text. Because LLMs lack internal world models and operate purely on discrete text tokens, they cannot replicate this basic cognitive feat:
> “Can you bolt a couple widgets on top of an LLM so that you kind of have some primitive reasoning function? And that's essentially what a lot of reasoning systems are doing.” — Yann LeCun > > “Reasoning at least in classical AI is in many domains involves a search through a space of potential solutions... there is no mechanism at all in LLMs for this search mechanism.” — Yann LeCun > > “When humans or animals reason, we don't do it in token space... We reason internally, right? We have a mental model of the situation, and we manipulate it in our head, and we find kind of a good solution.” — Yann LeCun
Kantrowitz notes that when DeepSeek's reasoning model was released, users were captivated by viral screenshots of the model generating philosophically deep reflections on the human condition. Yet, as LeCun's critique illustrates, this is another clever illusion produced by autoregressive statistical association rather than genuine introspection or conceptual understanding.
[10:40](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=640s) Diminishing Returns and the Need for a New AI Paradigm
Alex Kantrowitz reflects on reading through chain-of-thought outputs, remarking that the reasoning often feels less like genuine thought and more like the model simply read Yuval Noah Harari's *Sapiens* alongside a few other books and regurgitated the text. Yann LeCun agrees with that characterization, saying, “pretty much yeah I mean yeah a lot of it is regurgitation.” Moving into a core question, Kantrowitz asks whether training conventional large language models is hitting a wall—shifting away from the historical predictability where pumping in more compute and more data reliably yielded better performance.
```mermaid flowchart TD accTitle: Limits of LLM Scaling vs New Architecture Needs accDescr: Diagram contrasting current LLM scaling challenges with the requirements of a next-generation architecture.
subgraph LLMLimits["Current LLM Paradigm: Diminishing Returns"] A["10¹³ – 10¹⁴ Internet Tokens<br>(Public text exhausted)"] --> B["Synthetic Data & Human Annotators<br>(Ballooning costs)"] B --> C["Core Weakness: Superficial Pattern-Matching<br>(No real logic or novel planning)"] end
subgraph NewArchitecture["Required New Paradigm"] D["World Model & Commonsense Physics"] --> E["Zero-Shot Problem Solving & Goal Search"] E --> F["True Agentic Systems<br>(Internal action planning & verification)"] end
C -.->|"Paradigm shift required"| D ```
LeCun clarifies that while he might not label it a hard wall, the approach is unmistakably facing diminishing returns. The industry has effectively run out of natural human text: current LLMs are already trained on $10^{13}$ to $10^{14}$ tokens, representing practically the entire publicly available internet. Companies are now turning to licensed private content, synthetic data generation, and hiring thousands of subject-matter experts, PhDs, and professors to manually plug holes in models' knowledge bases. Yet LeCun emphasizes that these stopgaps tackle the wrong issue, because the systems fundamentally fail to grasp basic logic. As LeCun puts it:
> “The costs are ballooning of generating that data and the the returns are not that great so we need a new paradigm.”
To build truly agentic systems, AI cannot merely regurgitate pre-existing plans; it requires a fundamentally different architecture built around searching for candidate solutions, verifying their validity, and planning multistep actions to reach explicit goals. LeCun points out that while everyone in tech hypes agentic workflows, nobody currently knows how to build them beyond recycling sequences already present in the training set.
---
[13:30](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=810s) Zero-Shot Problem Solving and Kahneman's System 1 vs. System 2
Contrasting current models with biological intelligence, LeCun observes that both humans and animals routinely encounter completely unfamiliar situations and resolve them either zero-shot or after minimal practice, drawing solely on their intuitive grasp of how the physical world operates. For instance, a human learns to drive a car with only a couple dozen hours of practice. LeCun links this to Daniel Kahneman's dual-process framework:
- **System 2 (deliberative and focused):** During a novice's first driving sessions, full mental bandwidth is required. The driver consciously models hazards, maps maneuvers, and imagines worst-case scenarios. - **System 1 (automatic and subconscious):** After roughly 20 to 30 hours behind the wheel, driving becomes second nature. A driver can comfortably hold a conversation because the task has transitioned into an automatic, subconscious reflex.
To illustrate how expertise transforms deliberation into reflex, LeCun shares a personal memory from an academic workshop:
> “I have this vivid memory of once being in a workshop where one of the participants was a chess grandmaster and he played a simultaneous game against like 50 of us right, you know going from from one person to another, you know I got wiped out in 10 turns or something, I'm terrible at chess right, but so he would come, you know come to my my table and, you know, had time to think about this cuz he, you know, he was playing the other 50 tables or something, so I make my move in front of him, he goes like 'what?' and then immediately plays, so he doesn't have to think about it. I was not a challenging enough opponent that he had to actually call his System 2; his System 1 was sufficient to beat me.”
LeCun points out that every task humans master initially demands the full, deliberate power of our minds before being compiled down into subconscious routines. The defining capability of human and animal cognition is the ability to confront an unprecedented scenario, reason through a sequence of novel actions, and satisfy a goal using common-sense world knowledge—precisely what modern AI systems lack.
---
[16:29](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=989s) "A PhD Next to You" vs. Capital Expenditure Realities
Kantrowitz presses LeCun on the profound financial stakes and timeline mismatch facing the industry. Enormous venture sums have poured into LLM-first companies—$6.6 billion into OpenAI, multiple rounds totaling over $7.5 billion into Anthropic, and vast resources into Elon Musk's xAI for Grok. If scaling existing models is yielding diminishing returns while the research for a replacement paradigm remains unsolved and years away, Kantrowitz asks whether the tech sector is marching straight into an investment crisis.
LeCun responds unequivocally on the limits of current approaches:
> “The first thing you can say is we are not going to get to human-level AI by just scaling up LLMs, this is just not going to happen.”
LeCun dismisses aggressive forecasts made by certain peers, calling the notion that industry will deliver "a country of geniuses in a data center" within two years "complete BS" and stating there is "absolutely no way in hell." While massive parameter counts and vast datasets will produce systems capable of answering virtually any common query—creating the illusion of having a PhD at one's side—LeCun cautions that this illusion is merely an extraordinary retrieval engine over a gigantic memory bank. True doctoral-level intelligence requires inventing brand-new solutions to unsolved problems, an ability absent in pure autoregressive models (referencing a point highlighted by researcher Tom Walsh).
Defending Meta's heavy infrastructure spending, LeCun explains that their immense capital expenditure is aimed at inference operations rather than pure speculative model training:
- Meta's objective is to serve over 1 billion users interacting with Meta AI by year-end across smart glasses, standalone applications, and integrated social products. - Powering persistent daily use for hundreds of millions of users requires immense compute clusters and data center investments that must be constructed months or years in advance. - Even if a breakthrough to a post-LLM paradigm does not arrive within the next three years, the demand to run consumer-scale inference across Meta, Microsoft, Google, and Amazon will fully absorb this computational capacity, preventing the infrastructure buildout from being rendered obsolete.
[21:13](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=1273s) Consumer Scale vs. the Enterprise Reliability Barrier
Alex Kantrowitz highlights the current tension between massive capital expenditure and real-world utility. Setting up data centers requires years of forward-looking planning, forcing tech giants to invest heavily because they simply cannot afford to miss out if progress remains continuous. Kantrowitz contrasts consumer adoption with enterprise realities: OpenAI boasts 400 million users for ChatGPT, while Meta commands an install base across its family of apps in the billions, converting roughly 600 million users to Meta AI. While Meta AI's engagement may not yet match the intensity of ChatGPT, getting toward a billion consumer users is plausible.
```mermaid flowchart LR accTitle: The Enterprise Adoption Funnel accDescr: Proof of concept stages fail to graduate into production deployments due to accuracy and cost constraints.
subgraph Labs["Exploration Stage"] A["Enterprise POCs (100%)"] end
subgraph Roadblock["Deployment Barriers"] B["Hallucinations (5% error rate)"] C["High Compute & Run Costs"] end
subgraph RealWorld["Production"] D["Shipped to Production (10–20%)"] end
A --> B A --> C B --> D C --> D ```
However, Kantrowitz points out that the vast majority of AI investments depend on enterprise adoption, where current systems stumble. Referencing an argument from Benedict Evans regarding tools like Deep Research, he notes that an AI system achieving 95% accuracy leaves a fatal 5% hallucination rate. In a 100-page business report, an executive cannot afford to guess which 5% is fabricated. Consequently, across the corporate world, only about 10% to 20% of enterprise proof-of-concept projects graduate into live production; they are simply too fallible or too expensive to operate. Kantrowitz presses Yann LeCun on what happens if the industry hits a performance ceiling when so much capital was deployed expecting unbroken upward trajectories.
- Meta leverages a user base of over 3 billion mobile users to capture roughly 600 million Meta AI users. - OpenAI has reached an estimated 400 million ChatGPT users with deeper per-user session intensity. - Only an estimated 10% to 20% of enterprise generative AI proofs of concept make it into production environments. - Deep research tools delivering 95% accuracy present major operational risk because users cannot identify which 5% contains errors.
---
[23:26](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=1406s) The Last Mile Problem and the Ghost of IBM Watson
Yann LeCun contextualizes the enterprise struggle as an enduring problem in computing history, emphasizing that moving beyond an impressive lab demonstration to a dependable production system is where AI projects historically break down. He draws a direct parallel to autonomous vehicles: despite dazzling demos a decade ago, fully autonomous Level 5 self-driving cars remain unrealized because closing the "last mile"—the final fractions of a percent required for true safety and reliability—proves exponentially harder than the initial progress.
To illustrate how corporate AI hype can unravel, LeCun points to IBM Watson. IBM positioned Watson to generate billions in revenue by having it ingest the medical literature to serve as an automated diagnostician in every hospital. Instead, Watson struggled in complex clinical settings, failed to mesh with existing workflows, ran up against the natural conservatism of the medical workforce, cost IBM vast sums of money, contributed to executive turnover, and was ultimately dismantled and sold off for parts. LeCun stresses that this cycle of enterprise friction is not a novel symptom of generative AI:
> “The process we're seeing now with the difficulty of deploying a system is not new; it's happened absolutely at all times.”
He notes that current LLMs still provide clear, practical value—especially for internal retrieval-augmented generation (RAG) tasks across corporate intranets. Within Meta, employees rely daily on an internal knowledge-base retrieval tool called Metamate, which indexes company documentation so staff can query organizational data.
---
[25:37](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=1537s) Expert Systems, AI Winters, and Timeline Mismatches
LeCun recalls the 1980s boom in expert systems to warn about recurring industry patterns. Back then, "knowledge engineer" was hailed as the definitive career of the future. The job required sitting beside human experts, converting their heuristics into structured symbolic rules and facts, and feeding them into inference engines. The hype reached geopolitical levels, prompting the Japanese government to pour vast funding into its Fifth Generation Computer Systems project to build specialized hardware architectures. Yet by the mid-1990s, the entire wave collapsed into an AI winter. The economic feasibility was restricted to a handful of narrow, rule-bound problems, failing completely to deliver wide-ranging societal transformation.
Kantrowitz picks up on these historical cycles, noting that he and LeCun first spoke nine years ago about the long journey LeCun experienced alongside Yoshua Bengio, Geoffrey Hinton, and others. For years, connectionist neural network ideas were rejected and shut out from mainstream academic conferences until sufficient compute caught up to validate the architecture. Kantrowitz asks whether the combination of massive capital expenditures, diminishing returns from pure LLM scaling, and broader stock market volatility threatens to plunge the tech sector into another major AI backlash and winter.
- The 1980s expert system boom drove the Japanese Fifth Generation Computer project before collapsing in the mid-1990s. - Knowledge engineers spent years manually interviewing domain specialists to build rigid rule-based inference engines. - Connectionist architectures developed by pioneers like Yann LeCun, Yoshua Bengio, and Geoffrey Hinton were largely ignored by mainstream computer science until modern compute made deep learning viable.
---
[29:37](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=1777s) The Four Missing Pillars of True Intelligence
Addressing investor expectations, LeCun is blunt about the peril of banking solely on existing paradigms. He argues that anyone expecting current transformer architectures to reach human-level cognition purely by ingesting more tokens and applying minor tweaks is operating under a profound delusion:
> “If you're an investor and you invested in a company that told you we're going to get to human-level AI and PhD level by just, you know, training on more data and with a few tricks, I don't know if you're going to lose your shirt, but that was probably not a good idea.”
Real cognitive intelligence, LeCun insists, requires four foundational pillars present in nearly every animal but utterly absent in current large language models:
1. **Understanding the physical world** 2. **Persistent memory** 3. **The ability to reason** 4. **The ability to plan actions toward goals**
```mermaid flowchart TD accTitle: The Four Pillars of True Intelligence accDescr: LeCun's four fundamental requirements for autonomous intelligent systems.
subgraph CorePillars["Essential Cognitive Capabilities"] P1["1. Physical World Understanding<br>(Sensory grounding via video)"] P2["2. Persistent Memory<br>(State retention over time)"] P3["3. Reasoning<br>(Logic beyond token prediction)"] P4["4. Planning<br>(Mental simulations to achieve goals)"] end
CorePillars --> CommonSense["Common Sense & Autonomous Agency"] ```
Acquiring common sense demands training on raw perceptual streams—such as video—rather than exclusively on human-authored text. LeCun notes that research teams across Meta, Google DeepMind, and top academic institutions are actively pursuing systems that build internal mental models of physics to simulate and plan actions toward explicit goals. While early experimental results are beginning to surface in academic papers, scaling these world-model architectures into reliable, general-purpose systems will not happen overnight. He estimates that translating these proofs of concept into scalable, practical architectures capable of solving novel problems will take between three to five years rather than occurring in the immediate future.
[31:58](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=1918s) The Illusion of an AGI "Magic Bullet" and the Open Research Community
Yann LeCun contextualizes the ongoing surge of artificial intelligence investment, cautioning that true general intelligence will not arrive overnight through some proprietary breakthrough. He stresses that artificial general intelligence is not going to be a singular, binary threshold where one day it does not exist and the next day it does. Instead, advancing AI requires steady, continuous conceptual breakthroughs developed by the global scientific community over years, scaling up ideas that prove sound while abandoning dead ends. LeCun argues that open collaboration will inevitably outpace closed silos, warning investors against being seduced by overhyped claims from secretive small teams.
Alex Kantrowitz reflects on how much he values LeCun’s perspective as an empirical grounding point against industry hype, noting that past conversations have consistently held up over time. Setting up the next leg of their discussion, Kantrowitz points out two critical threads emerging from LeCun’s worldview: building an actual understanding of physical reality and the necessity of open-source research.
> “And so if you think that there is some startup somewhere with five people who has discovered the secret of AGI and you should invest five billion in them, you're making a huge mistake.” — Yann LeCun
- Progress in AI relies on distributed scientific contributions across the global research ecosystem rather than proprietary discovery by a single company. - Entities that openly publish and share their findings advance faster than closed development environments. - Investors pouring billions into small startups under the premise that an isolated team holds the singular key to AGI are fundamentally misjudging how science progresses.
---
[33:19](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=1999s) The Paper Test: Post-Training Patches vs. True Physical Intuition
Kantrowitz revisits a memorable test from their podcast two years prior: asking ChatGPT what happens if you hold a piece of paper horizontally with both hands and let go with your left hand. Back then, ChatGPT confidently hallucinated that the paper would magically float toward the left hand, and Kantrowitz had read the erroneous answer aloud before LeCun pointed out the obvious physics failure. Testing the prompt again two years later, Kantrowitz finds the model now answers correctly, explaining that gravity pulls the left side down while the right side creates a pivot effect.
LeCun is unimpressed by the model's new response, explaining that this fix is almost certainly the result of post-training and supervised fine-tuning rather than genuine physical reasoning. An engineer or annotator at OpenAI likely encountered that exact prompt, fed the system the correct answer, and fine-tuned it into the weights. Because the real world contains an infinite variety of physical interactions, brute-force fine-tuning will inevitably fail along the long tail of human experience. Post-training becomes exorbitant and brittle when developers try to manually patch every blind spot.
> “You cannot cover the set of all possible questions in the training set.” — Yann LeCun
- **The original failure:** Two years earlier, ChatGPT asserted that letting go of a horizontally held sheet of paper with the left hand would cause the paper to float toward that hand. - **The updated response:** Modern versions correctly state that gravity causes the left side to drop while pivoting around the right hand. - **Post-training limitations:** Manually patching common failure modes fails to address the infinite long tail of physical scenarios that cannot be covered in training data.
---
[36:28](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=2188s) Video Generation and the Data Reality: Toddlers vs. Trillion-Token Models
Kantrowitz asks whether modern video generation models like OpenAI's Sora demonstrate an emergent grasp of physics, noting that generating a prompt of a man kicking his legs while seated on a chair yields anatomically consistent legs that bend at the joints without hallucinating extra limbs. LeCun dismisses this as an optical illusion of comprehension. Video generators still routinely produce physics anomalies, such as spilled wine floating upward, flying off, or vanishing into thin air. Adding massive datasets to patch isolated video scenarios merely compensates for an absence of real understanding.
```mermaid flowchart TD accTitle: Training Data Volume Comparison accDescr: Comparison of sensory byte volume between Large Language Models, four-year-old children, and infant learning milestones.
subgraph LLM["Large Language Models"] T["30 Trillion Tokens"] -->|"~3 bytes per token"| B1["~10¹⁴ Bytes of Text"] B1 --> R["Requires ~400,000 years for a human reading 12 hrs/day"] end
subgraph Child["Human Sensory Intake"] V["16,000 Awake Hours (Age 4)"] -->|"2 MB/s via optic nerves"| B2["~10¹⁴ Bytes of Vision/Touch"] V2["9-Month-Old Infant"] -->|"~10¹³ Bytes of Vision"| B3["Intuitive Physics Acquired"] end ```
To demonstrate why text alone can never achieve human-level intelligence, LeCun presents a quantitative comparison between large language models and human sensory development. While an LLM ingests an enormous amount of text that would take a human 400,000 years to read, a four-year-old child processes an equivalent volume of sensory data in just 16,000 waking hours. Furthermore, a nine-month-old infant masters core intuitive physics with an order of magnitude less data than that.
> “In four years a child has seen through vision or touch for that matter as much data as the biggest LLMs, and it tells you clearly that we're not going to get to human level by just training on text.” — Yann LeCun
- **LLM data scale:** A typical frontier model trained on 30 trillion tokens processes roughly $10^{14}$ bytes of text, representing 400,000 years of reading at 12 hours per day. - **Child sensory bandwidth:** The human optic nerves carry roughly 2 megabytes per second (1 MB/s per nerve). Over 16,000 waking hours in a child's first four years, the visual cortex ingests approximately $10^{14}$ bytes. - **Infant physical mastery:** In just nine months (roughly $10^{13}$ bytes of sensory input), an infant learns foundational concepts of intuitive physics, including gravity, momentum conservation, and object permanence. - **Video context:** The 16,000 hours of video a child sees in four years equates to just 30 minutes of total video uploaded to YouTube.
---
[40:44](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=2444s) Why Generative Pixel Prediction Fails to Build a World Model
LeCun argues forcefully against the prevailing industry trend of using video generation to create world models. The objective of an intelligent system is not to generate high-fidelity, pixel-by-pixel simulations of the future, but to understand what actions are possible and predict outcomes at an appropriate level of abstraction. Predicting every minute detail of the physical world is both impossible and unnecessary for functional intelligence.
Using the everyday example of spilling a cup of water, LeCun points out that no human can calculate the precise trajectory of every water droplet, the exact splatter shape on the table, or the specific acoustic frequency produced. However, human abstract reasoning immediately anticipates the essential consequence: the water will spill, ruin documents, and soak a nearby smartphone. To plan actions toward goals, an AI architecture must abandon pixel-level generative objectives and instead learn abstract world representations that discard irrelevant noise.
> “The problem of understanding the world does not go through the solution to generating video at the pixel level.” — Yann LeCun
- **Generative limitations:** Predicting individual pixels forces models to waste computational capacity on chaotic, unpredictable details rather than underlying reality. - **Abstract world models:** True comprehension requires predicting state changes at an abstract level to evaluate cause and effect. - **Non-generative architectures:** Architectures designed to comprehend physical reality and plan actions must be non-generative, focusing entirely on structural and causal representations.
[42:29](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=2549s) Why Video Generation Fails at Physics
Alex Kantrowitz recounts an experiment he ran using a state-of-the-art video generation model to test its grasp of real-world physics. First, he asked it to depict a man sitting on a chair kicking his legs out. To Kantrowitz's surprise, the legs stayed attached to the torso throughout the motion, making him think the technology had made a real leap forward. But when he tested an edge case—prompting for a car driving into a haystack—the illusion completely fell apart: two stationary bales of hay appeared, and suddenly a haystack magically emerged straight out of the car's hood. Kantrowitz laughs that his immediate reaction was, “okay Yann, Yann wins again.”
Yann LeCun explains that this discrepancy stems directly from the training data distribution. AI labs fine-tune video generators with massive amounts of footage of humans performing everyday actions because human prompts represent the vast majority of consumer requests. The models perform passably when recreating human bodies simply because they have memorized an abundance of examples. However, as soon as a user introduces an unfamiliar physical interaction that the creators never anticipated—like a vehicle plowing into farm bales—the system hallucinates nonsensical physics because it possesses no underlying world model.
- Video models succeed on human motion primarily due to heavy fine-tuning on vast datasets of people. - Kantrowitz’s haystack test demonstrates how generative video models break down when facing unrepresented physical interactions. - “The thing is those systems have been fine-tuned with a huge amount of data for humans because, you know, that's what people are asking most videos they ask for.” — Yann LeCun
---
[43:33](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=2613s) Self-Supervised Learning: Why Generative Prediction Works for Text but Fails for Video
Kantrowitz asks how non-generative architectures like V-JEPA (Video JEPA) and I-JEPA (Image JEPA) can solve the problem of abstract representation. LeCun steps back to explain self-supervised learning, the foundational paradigm powering both modern LLMs and computer vision breakthroughs. In self-supervised learning, an unlabelled input—whether text, an image, or a video—is partially corrupted or masked, and a large neural network is trained to recover the original or predict the missing piece. LLMs represent a specialized, causal variant of this concept: by restricting the neural network so that it can only look at tokens to its left, the model learns to predict the next word in an autoregressive sequence without needing manual human annotations.
```mermaid flowchart LR accTitle: Causal Autoregressive Text Generation accDescr: Diagram showing how a causal language model predicts the next token from past tokens and feeds it back into the input. subgraph InputTokens["Input Tokens"] T1["Token 1"] --> T2["Token 2"] T2 --> T3["Token 3"] end subgraph Model["Causal Architecture"] T3 --> LLM["LLM Layer"] end subgraph Prediction["Next Token"] LLM --> T4["Token 4 (Argmax/Sample)"] T4 -.->|"Autoregressive Loop"| InputTokens end ```
While autoregressive reconstruction works remarkably well for text, LeCun emphasizes that applying the same reconstruction strategy to computer vision—such as Masked Autoencoders (MAE) for static images or generative frame prediction for video—hits a hard mathematical wall. In natural language processing, the space of possibilities at any given step is finite and discrete; an LLM simply outputs a probability distribution over a vocabulary of roughly 100,000 distinct tokens. In video, however, predicting the exact raw pixels of future frames requires modeling probability distributions across continuous, infinitely dimensional spaces. Because real-world futures contain endless irreducible details, generative video models are mathematically ill-equipped to serve as world models.
- Self-supervised learning masks or distorts raw input data and trains networks to reconstruct the uncorrupted state without manual labels. - LLMs are causal self-supervised models constrained to look only at past tokens to predict the subsequent token autoregressively. - Vocabulary distributions in language models are simple categorical vectors summing to 1 across roughly 100,000 discrete tokens. - “In fact, it's mathematically intractable to represent distributions in high-dimensional continuous spaces.” — Yann LeCun - “This is why, you know, trying to train a video generation system as a way to get a system to understand how the world works—that's why it can't succeed.” — Yann LeCun
---
[49:00](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=2940s) Inside JEPA: Learning Abstract World Models via Joint Embeddings
To bypass the pitfalls of pixel generation, LeCun advocates for the Joint Embedding Predictive Architecture (JEPA). Instead of trying to reconstruct missing pixels, JEPA extracts representations of both the original input and the corrupted input using neural encoders. A predictive model then operates entirely within this abstract latent space, attempting to predict the feature representation of the complete input from the representation of the partial input. By optimizing in embedding space rather than pixel space, the network naturally learns to discard unpredictable visual noise while retaining the fundamental invariants of the scene.
```mermaid flowchart TD accTitle: JEPA Architecture Flow accDescr: Diagram showing how JEPA processes full and corrupted inputs through encoders and predicts representations in latent space. subgraph Processing["Encoder Pathways"] Full["Full Video / Image (x)"] --> Enc1["Encoder"] Enc1 --> RepFull["Latent Representation: s(x)"] Corr["Corrupted / Masked Input (y)"] --> Enc2["Identical / EMA Encoder"] Enc2 --> RepCorr["Latent Representation: s(y)"] end subgraph LatentPrediction["Abstract Prediction"] RepCorr --> Pred["Predictor"] Pred --> PredState["Predicted State: s'(x)"] PredState <-->|"Loss Minimization (Prediction Error)"| RepFull end ```
LeCun illustrates this principle using a vertical pen balanced on a desk: when released, the exact direction it tips over is physically chaotic and impossible to forecast down to the exact pixel, yet at an abstract level, one can easily predict with certainty that the pen is going to fall. Meta's research team is exploring several non-contrastive training algorithms for these joint embeddings, including DINO, VICReg (Variance-Invariance-Covariance Regularization), and I-JEPA. When applied to video (V-JEPA), the architecture excels at identifying high-level semantics—such as recognizing specific human actions or detecting impossible physical events generated by game engines, such as objects vanishing or rolling balls disappearing behind barriers, simply by monitoring the spike in the system's latent prediction error.
- JEPA compares embeddings rather than raw sensory inputs, preventing models from wasting capacity on chaotic, unpredictable details. - Training methods being explored for JEPA include DINO, VICReg, and distillation architectures like I-JEPA. - V-JEPA evaluates physical plausibility by measuring prediction error when objects abruptly vanish, pass through solid matter, or mutate unnaturally. - “It's not generative because the system is not trying to regenerate the part of the input; it's trying to generate an abstract representation of the input.” — Yann LeCun - “The pen that you put on the table vertically, and when you let it go you cannot predict in which direction it's going to fall, but at some abstract level you can say that the pen is going to fall.” — Yann LeCun
[53:06](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=3186s) Action-Conditioned JEPA and Genuine Planning
When a self-supervised world model is trained on natural, physically plausible video, it builds an internal baseline for how reality behaves. If something physically impossible occurs in a test video, the system's prediction error spikes immediately, providing a measurable signal that the model has internalized physical constraints and acquired rudimentary common sense. Yann LeCun explains that Meta has extended these architectures into "action-conditioned" systems. In this framework, the model takes a representation of the world's state at time $t$ along with a proposed action—such as an instruction to actuate a robotic arm—and predicts the resulting state at time $t+1$ purely inside an abstract representation space rather than predicting raw, pixel-level frames.
```mermaid flowchart LR accTitle: Action-Conditioned JEPA Planning Architecture accDescr: Diagram illustrating how an action-conditioned JEPA world model predicts future abstract states and searches for action sequences to reach a goal. subgraph Input["Current State & Action"] S0["State at time t"] A0["Action"] end subgraph Model["JEPA World Model"] WM["Abstract Predictor"] end subgraph Prediction["Future State & Evaluation"] S1["Predicted State at time t+1"] Plan["Search & Goal Evaluation"] end S0 --> WM A0 --> WM WM --> S1 S1 --> Plan ```
By chaining these abstract predictions together, the model can simulate the future and mentally test hypotheses. An operator can provide a target state—specifying what the world should look like at the end of a task—and the model can conduct a search over possible action sequences to determine which steps will successfully achieve that target. LeCun emphasizes that this capability represents the foundational architecture for authentic machine intelligence, contrasting it with simple text extrapolation: “That's planning that's reasoning that's actual reasoning and actual planning.”
[54:33](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=3273s) Open Source, DeepSeek, and Global AI Innovation
With time running out on the interview, Alex Kantrowitz asks LeCun for a rapid-fire reaction to DeepSeek and whether open-source AI has officially overtaken proprietary frontier models. LeCun hesitates to use the word "overtaken," pointing out that proprietary AI labs immediately absorb and benefit from whatever the open-source community publishes. However, he stresses that the pace of innovation within open source is inherently faster because proprietary labs cannot match the collective brainpower of the broader research ecosystem. Pointing to DeepSeek's recent triumphs, LeCun observes: “Nobody has a monopoly on good ideas and so the magic efficiency of the open source world is that it recruits talents from all over the world.”
When given a long leash and minimal bureaucratic constraints, small, agile research teams can rethink standard practices and invent model architectures that legacy frontier labs overlook. Once those insights are shared publicly, the entire global scientific community leaps forward together. LeCun also highlights the economic reality driving enterprise adoption: while corporate partners frequently experiment and prototype using proprietary closed APIs, they routinely shift production deployments to open-source foundations like Llama. Open weights offer drastically lower inference costs, tighter data security, granular customization, and the indispensable ability to run systems on-premises.
- Progress moves faster across open source because no single corporation, regardless of size or budget, can assemble all global talent under one roof. - Proprietary software developers routinely benefit from open-source breakthroughs since all published techniques and model architectures are accessible to them. - Enterprise customers typically use proprietary cloud APIs for early-stage prototyping, but pivot to open-source models like Llama for production rollouts to reduce operating costs and retain operational control.
[56:39](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=3399s) Shattering Geopolitical Myths: ResNet, Llama, and the Global Nature of Science
Addressing critics who fear that open-source AI will inadvertently help geopolitical rivals like China, LeCun dismisses the notion that overseas researchers depend on Western handouts. Top-tier scientific talent is distributed worldwide, and Chinese researchers have long produced seminal contributions to foundational AI. To illustrate this, LeCun cites ResNet (Residual Networks), published in 2015 out of Microsoft Research Beijing, which stands as the single most cited scientific paper in the history of science. ResNet revolutionized deep learning by introducing identity shortcut connections—allowing layers to learn deviations from an identity mapping rather than fitting full transformations from scratch—which enabled researchers to successfully train neural networks containing dozens or even hundreds of layers.
LeCun recalls hiring ResNet's lead author, Kaiming He, to join the FAIR (Fundamental AI Research) lab in California, where He worked for about eight years before taking a professorship at MIT. LeCun notes that breakthrough AI development is fundamentally decentralized: the original LLaMA model did not come out of Silicon Valley either, but was built by a tight-knit team of just twelve researchers at FAIR's lab in Paris. Kantrowitz and LeCun wrap up the discussion reflecting on their nine years of conversations cutting through industry hype. LeCun concludes with a defense of open international research: “You have to take advantage of the diversity of ideas backgrounds creative juices of the entire world if you want Science and Technology to progress fast and that's enabled by open source.”
- ResNet, published in 2015 by Kaiming He and colleagues at Microsoft Research Beijing, holds the title of the single most cited paper in all of science. - ResNet introduced skip connections that compute identity deviations, resolving gradient degradation and enabling stable training across networks exceeding 100 layers. - Kaiming He was recruited by LeCun to FAIR in California, conducted research there for roughly eight years, and subsequently joined the faculty at MIT. - The original LLaMA model was developed in Paris by a small FAIR team comprising only 12 researchers, demonstrating that cutting-edge AI breakthroughs do not originate exclusively within Silicon Valley.
## Notable quotes > [16:29](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=989s) The first thing you can say is we are not going to get to human-level AI by just scaling up LLMs, this is just not going to happen. — Yann LeCun
> [40:44](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=2444s) The problem of understanding the world does not go through the solution to generating video at the pixel level. — Yann LeCun
> [36:28](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=2188s) In four years a child has seen through vision or touch for that matter as much data as the biggest LLMs, and it tells you clearly that we're not going to get to human level by just training on text. — Yann LeCun
> [31:58](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=1918s) if you think that there is some startup somewhere with five people who has discovered the secret of AGI and you should invest five billion in them, you're making a huge mistake. — Yann LeCun
> [54:33](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=3273s) Nobody has a monopoly on good ideas and so the magic efficiency of the open source world is that it recruits talents from all over the world. — Yann LeCun
> [43:33](https://www.youtube.com/watch?v=qvNCVYkHKfg&t=2613s) In fact, it's mathematically intractable to represent distributions in high-dimensional continuous spaces. — Yann LeCun
## Mentioned Yann LeCun, Alex Kantrowitz, Dwarkesh Patel, Thomas Wolf, Hugging Face, DeepSeek, Yuval Noah Harari, Daniel Kahneman, OpenAI, ChatGPT, Anthropic, Elon Musk, xAI, Grok, Tom Walsh, Meta, Meta AI, Microsoft, Google, Amazon, Benedict Evans, Deep Research, IBM Watson, Metamate, Yoshua Bengio, Geoffrey Hinton, Google DeepMind, Sora, YouTube, V-JEPA, I-JEPA, DINO, VICReg, Llama, ResNet, Kaiming He, FAIR, MIT
## Video description Yann LeCun is the chief AI scientist at Meta. He joins Big Technology Podcast to discuss the strengths and limitations of current AI models, weighing in on why they've been unable to invent new things despite possessing almost all the world's written knowledge. LeCun digs deep into AI science, explaining why AI systems must build an abstract knowledge of the way the world operates to truly advance. We also cover whether AI research will hit a wall, whether investors in AI will be disappointed, and the value of open source after DeepSeek. Tune in for a fascinating conversation with one of the world's leading AI pioneers.
Chapters:
00:00 Introduction to Jan LeCun and AI's limitations 01:12 Why LLMs can't make scientific discoveries 05:40 Reasoning in AI systems: limitations of chain of thought 10:13 LLMs approaching diminishing returns and the need for a new paradigm 16:29 "A PhD next to you" vs. actual intelligent systems 21:36 Consumer AI adoption vs. enterprise implementation challenges 25:37 Historical parallels: expert systems and the risk of another AI winter 29:37 Four critical capabilities AI needs for true understanding 33:19 Testing AI's physics understanding with the paper test 37:24 Why video generation systems don't equal real comprehension 43:33 Self-supervised learning and its limitations for understanding 51:10 JEPA: Building abstract representations for reasoning and planning 54:33 Open source vs. proprietary AI development 58:57 Conclusion
[Source](https://www.youtube.com/watch?v=qvNCVYkHKfg)