LeCun Team Finds Sparse Latents Lift World Model Planning by 57 Points

LpWM: A Case for Sparse Representations in World Models — Xiaol.x

Xiaol.x

Enforcing exact zeros in latent representations linearizes physical transitions across dynamic regimes, allowing compact three-million-parameter predictors to succeed where standard dense representations fail.

AI-assisted summary of LpWM: A Case for Sparse Representations in World Models — original source. Check the original for full context.

Source published:

Key ideas

  • Yann LeCun's group introduces LpWM, showing that replacing standard Gaussian latent targets in JEPA world models with rectified Laplace distributions yields 30–65% exact zeros and drastically simplifies the learned dynamics. (0:00)
  • On the PushT benchmark, intermediate-capacity predictors using sparse latents outperform dense baselines by up to 57 percentage points, with a compact 3M-parameter MLP ∘ LTI predictor achieving ~70% planning success on sparse codes versus ~7% on dense ones. (6:03)
  • Linear probes confirm a mechanistic phenomenon termed 'mode factoring': the latent vector's binary zero-support decodes discrete physical force regimes with 97% accuracy, while continuous coordinate magnitudes encode position within each regime. (9:11)
  • While vanilla frame-level sparsity primarily tracks end-effector motion (r = 0.87), introducing an auxiliary temporal Jaccard loss penalizing support flips successfully realigns the zero pattern to track physical contact (r = 0.61) without degrading planning performance. (12:27)
  • The performance advantage of sparse representations is restricted to intermediate predictor capacities; when scaled up to high-capacity Diffusion Transformers (DiT), sparse and dense representations achieve identical performance. (13:34)

Detailed notes

## Background Published on YouTube by Xiaol.x on September 11, 2026, this video presents Episode 02 of the podcast 'Loop State'. The episode features hosts Mara and Theo breaking down an August 2026 paper from Yann LeCun's research group (Kuang et al.) that challenges standard Gaussian latent geometry in JEPA world models in favor of sparse representations.

## TL;DR The hosts examine the paper 'LpWM: A Case for Sparse Representations in World Models', which shows that enforcing exact zeros in latent representations makes environment dynamics dramatically simpler to learn. Theoretical intuition rooted in one-hot state partitions motivates RDMReg, a regularizer steering features toward a rectified Laplace distribution to produce 30–65% exact zeros without codebooks. On the PushT benchmark, sparse latents allow intermediate, low-capacity predictors (such as an MLP over linear dynamics) to outperform dense counterparts by up to 57 percentage points, while both representations tie at the deep transformer level. Analysis on the Piecewise and OGBench-Cube benchmarks reveals that active coordinate supports cleanly factor physical modes, though tracking physical contact requires an auxiliary temporal Jaccard loss.

## Atmosphere & style The discussion carries a lively, intellectually curious podcast dynamic where Mara and Theo bounce technical ideas off each other with sharp clarity and approachable analogies. It moves from playful conceptual thought experiments—like calling one-hot physics a lookup table in a linear algebra costume—into precise mathematical and empirical deconstruction. The tone is enthusiastic yet refreshingly skeptical, celebrating the efficiency gains of sparse world models while meticulously probing empirical limitations, missing baselines, and theoretical caveats.

## Bulletin - [00:00](https://www.youtube.com/watch?v=FVpfCJfV110&t=0s) Yann LeCun's group introduces LpWM, showing that replacing standard Gaussian latent targets in JEPA world models with rectified Laplace distributions yields 30–65% exact zeros and drastically simplifies the learned dynamics. - [06:03](https://www.youtube.com/watch?v=FVpfCJfV110&t=363s) On the PushT benchmark, intermediate-capacity predictors using sparse latents outperform dense baselines by up to 57 percentage points, with a compact 3M-parameter MLP ∘ LTI predictor achieving ~70% planning success on sparse codes versus ~7% on dense ones. - [09:11](https://www.youtube.com/watch?v=FVpfCJfV110&t=551s) Linear probes confirm a mechanistic phenomenon termed 'mode factoring': the latent vector's binary zero-support decodes discrete physical force regimes with 97% accuracy, while continuous coordinate magnitudes encode position within each regime. - [12:27](https://www.youtube.com/watch?v=FVpfCJfV110&t=747s) While vanilla frame-level sparsity primarily tracks end-effector motion (r = 0.87), introducing an auxiliary temporal Jaccard loss penalizing support flips successfully realigns the zero pattern to track physical contact (r = 0.61) without degrading planning performance. - [13:34](https://www.youtube.com/watch?v=FVpfCJfV110&t=814s) The performance advantage of sparse representations is restricted to intermediate predictor capacities; when scaled up to high-capacity Diffusion Transformers (DiT), sparse and dense representations achieve identical performance.

## Key takeaways - [00:00](https://www.youtube.com/watch?v=FVpfCJfV110&t=0s) Conventional JEPA world models enforce dense, maximum-entropy Gaussian latents to avoid representation collapse, inadvertently making downstream dynamics unnecessarily complex to learn. - [01:21](https://www.youtube.com/watch?v=FVpfCJfV110&t=81s) JEPA world models conduct planning entirely in abstract latent space without decoding pixels, using predictors as latent physics engines combined with the Cross-Entropy Method. - [02:51](https://www.youtube.com/watch?v=FVpfCJfV110&t=171s) Proposition 1 proves that one-hot latent codes linearize any Lipschitz continuous dynamic system into state-transition permutation matrices, though doing so exactly requires exponentially many dimensions. - [04:30](https://www.youtube.com/watch?v=FVpfCJfV110&t=270s) LpWM relaxes one-hot coding into distributed continuous sparsity using RDMReg, matching 1D random batch projections against a rectified Laplace distribution to force 30% to 65% of coordinates to exact zero. - [04:30](https://www.youtube.com/watch?v=FVpfCJfV110&t=270s) To maintain exact zeros while preserving gradient flow during backpropagation, LpWM's projector uses a standard ReLU forward pass combined with a GELU gradient on the backward pass. - [06:03](https://www.youtube.com/watch?v=FVpfCJfV110&t=363s) The authors establish a predictor complexity ladder spanning from lightweight 3M-to-34M parameter linear models up to 822M-parameter Diffusion Transformers to quantitatively benchmark dynamics complexity. - [06:03](https://www.youtube.com/watch?v=FVpfCJfV110&t=363s) On the PushT benchmark, sparse LpWM outperforms dense LeWM by 24 to 57 points across intermediate predictor rungs like MLP ∘ LTI and by 36 to 45 points on MLP ∘ LTV. - [06:03](https://www.youtube.com/watch?v=FVpfCJfV110&t=363s) Both sparse and dense representations fail completely on single-step linear predictors (LTI(1)) and tie at top-tier Diffusion Transformers, locating the entire sparse advantage within intermediate model capacities. - [09:11](https://www.youtube.com/watch?v=FVpfCJfV110&t=551s) Sparse representations maintain their performance advantage over covariance-penalized VICReg baselines, outperforming VICReg even at the deep transformer tier. - [09:11](https://www.youtube.com/watch?v=FVpfCJfV110&t=551s) In a piecewise-affine room, latent binary supports capture discrete dynamical zones with 97% accuracy even when visual zone boundaries are completely absent, proving sensitivity to physical dynamics over visual textures. - [12:27](https://www.youtube.com/watch?v=FVpfCJfV110&t=747s) Vanilla marginal RDMReg possesses no temporal awareness, causing the zero support pattern out-of-the-box to correlate strongly with arm motion (r = 0.87) rather than physical contact (r = 0.05). - [12:27](https://www.youtube.com/watch?v=FVpfCJfV110&t=747s) An auxiliary temporal Jaccard regularization penalizes support changes across consecutive frames, raising physical contact correlation to 0.61 and cube motion correlation to 0.80 on OGBench-Cube. - [13:34](https://www.youtube.com/watch?v=FVpfCJfV110&t=814s) The empirical implementation operates far from the theoretical one-hot ideal, retaining 30% to 65% active coordinates rather than a strict single active dimension. - [13:34](https://www.youtube.com/watch?v=FVpfCJfV110&t=814s) The empirical study is limited to four simulated planar environments, uses a single training seed with three evaluation seeds, and omits direct empirical comparisons to discrete categorical latents like DreamerV2. - [15:06](https://www.youtube.com/watch?v=FVpfCJfV110&t=906s) LpWM challenges default self-supervised learning assumptions, framing latent geometry as an intentional design variable that can reduce the compute requirements of world model predictors.

## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=FVpfCJfV110&t=0s) Rethinking the Gaussian Latent in World Models

Hosts Mara and Theo kick off the second episode of *Loop State* by diving into an August 2026 paper from Yann LeCun's research group: *"LpWM: A Case for Sparse Representations in World Models"* by Kuang et al. Mara introduces the core premise as a fundamental question she had never thought to ask: why does practically every self-supervised world model push its internal representations toward a dense Gaussian blob? Models like LeWM and VICReg enforce dense geometries by design, but the authors pitch a provocative alternative. Dense representations might simply be the wrong shape.

The core thesis of the paper is not just an aesthetic critique, but a functional one: if latent codes are made sparse—meaning a substantial fraction of their coordinates sit at exact zeros—the underlying dynamics that the model must learn become far simpler. In fact, they become so simple that a nearly linear predictor can successfully plan actions where a dense latent representation would demand an expressive, compute-heavy transformer. The headline finding highlights this drastic difference: on the PushT manipulation task, sparse codes beat dense ones by up to 57 percentage points in planning success at intermediate predictor capacity.

- Paper: *"LpWM: A Case for Sparse Representations in World Models"* (Kuang, Dagade, Le Lidec, Maes, Balestriero, LeCun; August 2026). - Headline result: Up to a 57-point improvement in closed-loop planning success over dense representations when using intermediate-capacity predictors.

[01:21](https://www.youtube.com/watch?v=FVpfCJfV110&t=81s) How JEPA Predicts Without Pixels

To unpack why representations matter, Theo breaks down the machinery of a Joint Embedding Predictive Architecture (JEPA) for listeners who have heard the acronym but never opened the foundational papers. A JEPA world model consists of three core components: an encoder $F$ that maps an observation image into a latent vector $z$, a predictor $G$ that takes the current latent $z_t$ alongside an action $a_t$ to forecast the next latent state $\hat{z}_{t+1}$, and a loss function measuring the distance between $\hat{z}_{t+1}$ and the encoded ground-truth target frame $z_{t+1}$. Because the model predicts strictly within its abstract latent space and never decodes images back into pixels, it bypasses the massive overhead of generative pixel modeling.

```mermaid flowchart LR accTitle: JEPA World Model Architecture accDescr: Visual flowchart tracing the JEPA pipeline from observations through latent encoding, prediction, and loss computation against target frames. subgraph Input["Observations"] O_t["Image (t)"] O_next["Image (t+1)"] end subgraph Encoding["Encoder F"] O_t --> Z_t["Latent z_t"] O_next --> Z_next["Latent z_{t+1}"] end subgraph Dynamics["Predictor G"] Z_t --> G_pred{"Predictor G"} A_t["Action a_t"] --> G_pred G_pred --> Z_hat["Predicted ẑ_{t+1}"] end subgraph Objective["Latent Loss"] Z_hat -.-> Loss(["Distance Loss"]) Z_next -.-> Loss end ```

However, predicting directly in latent space introduces a critical failure mode: representation collapse. Without constraints, the encoder can trivially minimize loss by outputting an identical constant vector for every frame, yielding perfect but useless predictions. To prevent this, architects use regularization terms to force latents toward high-entropy distributions: - **LeWM** applies SIGReg to align latents with an isotropic Gaussian. - **VICReg** penalizes covariance to drive the variance-covariance matrix toward the identity.

Because both targets are maximum-entropy distributions, they guarantee dense representations where every single coordinate is almost surely nonzero. At test time, agents execute planning via the Cross-Entropy Method (CEM), searching for action sequences that steer predicted latents toward a target goal latent. In this paradigm, the predictor serves as the entire physics engine, raising the central question: how complex must that learned physics engine actually be?

[02:51](https://www.youtube.com/watch?v=FVpfCJfV110&t=171s) Proposition 1: One-Hot Codes and Linear Physics

Theo presents an illuminating thought experiment that anchors the mathematical intuition: what happens to dynamical complexity if the latent representation is strictly one-hot, where exactly one coordinate is active and every other coordinate is zero? Under this extreme condition, the system's dynamics become mathematically linear.

The paper formalizes this intuition in Proposition 1. For any compact state space with Lipschitz-continuous dynamics, one can partition the state space into $N$ distinct cells of radius $\epsilon$ and represent each state as an $N$-dimensional one-hot indicator of its cell. Under this mapping, an action $a$ is represented by a transition permutation matrix $P(a)$ that routes the active coordinate to the destination cell dictated by the physical dynamics:

$$z_{t+1} = P(a_t) z_t$$

The decoded transition error is bounded tightly by $(L + 1)\epsilon$, where $L$ is the Lipschitz constant. As Mara observes, complex nonlinear physics is essentially converted into "a lookup table wearing a linear algebra costume." A direct corollary proves that across a $d$-dimensional unit cube partitioned into $N$ cells, the error shrinks at the rate of $N^{-1/d}$, approaching zero as the code widens. However, this power exposes the curse of dimensionality: an exact one-hot code requires an exponential number of dimensions ($N \propto \epsilon^{-d}$). Rather than proposing unusable discrete lookups, the authors suggest relaxing the constraint: distributed sparse representations, where a significant fraction of coordinates are set to exact zeros, could capture a meaningful fraction of that mathematical linearity at a computationally feasible dimensionality.

[04:30](https://www.youtube.com/watch?v=FVpfCJfV110&t=270s) Inducing Exact Sparsity with RDMReg

Achieving distributed sparsity in a JEPA without falling back on discrete codebooks, vector quantization (VQ), or brittle discrete bottlenecks requires reshaping the regularization objective. While LeWM matches features against a standard Gaussian, LpWM steers them toward a rectified generalized Gaussian—specifically, a zero-mean Laplace distribution passed through a ReLU activation.

Geometrically, a rectified Laplace distribution exhibits sharp exponential decay along the positive axis paired with a concentrated Dirac delta spike of probability mass sitting precisely at zero, representing all negative mass clipped by the rectification. The authors enforce this geometry using **RDMReg**, a regularizer that projects latent batches onto random unit directions and minimizes the 2-Wasserstein distance between empirical batch projections and the rectified target distribution.

To ensure coordinates reach exact zeros rather than merely vanishingly small values, the encoder's projector ends with a reparameterized ReLU: - **Forward pass:** Evaluates a strict standard ReLU, guaranteeing exact zeros. - **Backward pass:** Employs the gradient of a Gaussian Error Linear Unit (GELU) to prevent dead neurons and keep gradient flow healthy.

Depending on latent width, LpWM yields representations where 30% to 65% of coordinates are active, compared to 100% active density in LeWM. Crucially, reverting the target distribution back to a standard Gaussian without the rectification cleanly recovers LeWM. Because the encoder, predictor capacity, and CEM planner remain identical, this single distribution switch isolates sparsity as an independent variable.

[06:03](https://www.youtube.com/watch?v=FVpfCJfV110&t=363s) The Predictor Ladder and PushT Dynamics

To convert the theoretical claim of "simpler dynamics" into rigorous empirical metrics, the authors construct a ladder of predictors ranging from primitive linear models to deep transformers. By evaluating closed-loop planning success across this ladder, the specific rung where sparse latents succeed and dense latents fail serves as a direct gauge of dynamical simplicity.

```mermaid flowchart TD accTitle: Predictor Complexity Ladder accDescr: Hierarchy of predictor models evaluated in the paper, ranging from simple Linear Time-Invariant baselines to six-layer Diffusion Transformers. subgraph AttentionFree["Attention-Free Rungs (~3M to 85M Params)"] R1["LTI(1): Pure linear single-step (z_{t+1} = Az_t + Ba_t)"] R2["LTI(k): Linear with 3 frames of history"] R3["MLP ∘ LTI: Single ReLU layer over linear dynamics"] R4["MLP ∘ LTV: Linear Time-Varying (State-dependent matrices)"] end subgraph TransformerRungs["Transformer Predictors (Up to 822M Params)"] R5["1-Layer Transformer"] R6["6-Layer DiT (Standard LeWM predictor)"] end R1 --> R2 --> R3 --> R4 --> R5 --> R6 ```

The predictor ladder spans a massive scale: at a feature width of 4,096, the top-rung DiT contains 822 million parameters, while the bottom-rung LTI(1) uses only 34 million parameters, and the middle rungs hover between 70 and 85 million parameters. Architectures below the transformer level are entirely attention-free. Models are evaluated across five latent widths (from 384 to 4,096) utilizing Maximal Update Parameterization ($\mu\mathrm{P}$) to prevent hyperparameter drift across widths.

The experimental tests span two environments: 1. **Wall:** A simple 2D navigation task where an agent must locate a doorway. Because the underlying dynamics are trivial, even the bottom-rung LTI(1) predictor achieves nearly 100% closed-loop success across both sparse and dense representations, validating Wall as an intentionally designed null result. 2. **PushT:** A challenging benchmark requiring an agent to push a T-shaped block into a designated goal pose, involving complex contact mechanics, friction, and rotational discontinuities.

On PushT, both representations fail completely at the lowest LTI(1) rung, and both match high performance at the top transformer rungs. The entire distinction lives in the middle tiers: - **MLP $\circ$ LTI:** Sparse LpWM outperforms dense LeWM by **24 to 57 points** of planning success across widths. - **MLP $\circ$ LTV:** Sparse leads by **36 to 45 points**. - **LTI($k$):** Sparse leads by **11 to 23 points**.

Theo highlights a standout result from Figure 1B: at a width of 768 using the lightweight 3-million-parameter MLP $\circ$ LTI predictor, closed-loop planning achieves roughly **70% success on sparse codes** compared to a miserable **7% success on dense codes**. As Mara summarizes, the core value proposition is not necessarily building a marginally higher-scoring world model, but creating a radically cheaper, low-capacity world model that actually works.

[09:11](https://www.youtube.com/watch?v=FVpfCJfV110&t=551s) Ruling Out SIGReg and Validating Latent Support

Addressing potential skepticism, Mara questions whether the performance gap might simply reflect an idiosyncrasy or weakness of LeWM’s SIGReg regularizer rather than a fundamental flaw of dense codes. To test this, the authors evaluated a second dense baseline by substituting SIGReg with VICReg within the exact same setup.

When evaluated across the PushT ladder, sparse LpWM once again dominated across intermediate rungs like MLP $\circ$ LTV and MLP $\circ$ LTI. Intriguingly, against VICReg, LpWM even outperformed the dense baseline at the top-rung deep transformer. The authors hypothesize that because VICReg constrains only first- and second-order moments (variance and covariance), its target distribution remains underspecified, generating a geometrically degraded latent space for dynamics modeling; by contrast, SIGReg constrains the full distribution shape, making it a stronger dense competitor.

``` +------------------+---------------------+---------------------+ | Predictor Rung | Sparse LpWM vs LeWM | Sparse LpWM vs VICReg | +------------------+---------------------+---------------------+ | LTI(1) | Tie (Both fail) | Tie (Both fail) | | LTI(k) | Sparse +11 to +23 | Sparse wins | | MLP ∘ LTI | Sparse +24 to +57 | Sparse wins | | MLP ∘ LTV | Sparse +36 to +45 | Sparse wins | | Deep Transformer | Tie (Both succeed) | Sparse wins | +------------------+---------------------+---------------------+ ```

The discussion then turns to the mechanistic interpretation of the latent space: what are the inactive coordinates actually doing? To probe this, the authors engineered the **Piecewise** environment—an open room partitioned into hidden spatial zones, where each zone exerts a distinct constant directional force on the agent. The dynamics are locally linear within each region and globally piecewise-affine.

The authors evaluated the agent across a $20 \times 20$ spatial grid, binarizing each latent vector into its mathematical support (the binary mask of nonzero entries) and calculating the Jaccard similarity index (intersection over union) across spatial coordinates: - The binary support overlaps heavily among positions within the same dynamics zone and drops precipitously at regime boundaries. - Crucially, this zone-clustering behavior persists even when the visual markers of the zones are completely removed from the environment rendering, proving that the latent support latches onto the underlying physics rather than cosmetic visual textures. - Linear probe decoders reveal a striking separation of concerns, termed **mode factoring**: the binary support decodes the discrete physical zone with ~97% accuracy (matching the full continuous vector), while continuous coordinate positions within a zone are decoded primarily from coordinate magnitudes. - In planning evaluations over random goals within Piecewise, sparse models attained **84.7% success** compared to **65.3% for dense models**, maintaining their performance edge across extended planning horizons.

Turning finally to contact-rich robotic manipulation in the OGBench-Cube environment—where discrete dynamical transitions represent physical contact between the gripper and the cube—the hosts note the paper's transparency regarding the limits of vanilla sparsity and support stability.

[12:27](https://www.youtube.com/watch?v=FVpfCJfV110&t=747s) Does It Still Work When Things Touch? Support Dynamics on OGBench-Cube

When testing whether the sparsity pattern naturally tracks physical contact on the OGBench-Cube manipulation task, the initial results show an immediate limitation of pure marginal regularization. Out of the box, the latent support shifts whenever the robotic arm moves across the workspace. The support correlates at 0.87 with end-effector motion, while its correlation with actual object contact hovers at an insignificant 0.05. As the hosts observe, without additional inductive biases, "the zeros are just a motion detector." Because RDMReg exclusively penalizes and shapes the marginal feature distribution frame by frame, it possesses no temporal awareness across transitions.

```mermaid flowchart TD accTitle: Temporal Jaccard Loss Effect on Latent Support accDescr: Diagram illustrating how adding temporal Jaccard loss redirects the latent zero support from tracking arm motion to tracking physical contact and cube motion. subgraph Raw["Vanilla RDMReg (Marginal Only)"] direction TB A1["Frame-level Laplace Prior"] --> B1["Latent Support Pattern"] B1 --> C1["End-Effector Motion (r = 0.87)"] B1 -.->|Weak signal| D1["Object Contact (r = 0.05)"] end subgraph Regularized["RDMReg + Temporal Jaccard Prior"] direction TB A2["Temporal Jaccard Loss"] --> B2["Penalize Inter-Frame Support Flips"] B2 --> C2["Latent Support Stabilizes"] C2 --> D2["Cube Motion (r = 0.80)"] C2 --> E2["Physical Contact (r = 0.61)"] end ```

To bridge this gap, the authors introduce an optional temporal Jaccard loss that explicitly penalizes support flips between adjacent time steps. Adding this temporal prior completely realigns what the zero patterns encode without degrading planning performance:

- Correlation with end-effector motion plummets from 0.87 down to 0.40. - Correlation with cube motion climbs significantly up to 0.80. - Correlation with physical contact jumps from 0.05 to 0.61. - Planning success rate remains stable throughout tuning, proving that reallocating support semantics does not harm task execution. - Raster plots reveal that the active support becomes quiet during free space navigation and sharply reorganizes when entering the contact interaction window. - The fundamental mechanism is summarized neatly: "Sparsity gives you a slot for the mode. A temporal prior decides what goes in the slot."

---

[13:34](https://www.youtube.com/watch?v=FVpfCJfV110&t=814s) What Does This Paper Not Show? Theoretical and Empirical Limitations

Turning to the boundaries of the paper, the hosts scrutinize the exact failure modes, omissions, and assumptions underpinning LpWM. The most significant empirical asterisk is that sparse representations do not beat dense representations when paired with high-capacity predictors: on PushT, when using a Diffusion Transformer (DiT) backbone, sparse and dense representations achieve parity. The massive performance deltas only materialize at intermediate predictor capacity.

The gap between the mathematical theory and practical implementation is equally sharp. The paper's formal linearization proof (Proposition 1) relies on strictly one-hot latent codes, which demand exponentially many dimensions relative to state space complexity. In contrast, the trained networks produce codes with 30% to 65% active coordinates, operating far from true one-hot regimes. The mathematical proof serves as an inspirational motivation for local linearity rather than an exact description of the empirical model.

The experimental design also leaves several practical questions open:

- **Benchmark Diversity:** Evaluated across four environments, all of which are simulated; three are purely two-dimensional navigation or planar pushing benchmarks, with zero physical robot validation. - **Statistical Rigor:** Models were trained across only a single training seed and evaluated on three test seeds, reporting numbers from the best-performing tuned hyperparameter cells. - **Interpretability Prerequisite:** Support interpretation is completely unsupervised only when physical modes are purely spatial (as in the Piecewise domain); on contact benchmarks like Cube, it requires a hand-tuned, optional auxiliary temporal loss. - **Missing Categorical Baselines:** The study does not include direct head-to-head empirical comparisons against discrete latent frameworks like DreamerV2's categorical distributions, despite referencing them in related work. - **Author Aspirations vs. Present Evidence:** The paper positions latent Linear Quadratic Regulator (LQR) control and the hypothesis that sparsity benefits will scale up to heavy transformer backbones under harder dynamics strictly as future work rather than demonstrated results.

---

[15:06](https://www.youtube.com/watch?v=FVpfCJfV110&t=906s) Key Takeaways and Open Questions

In their closing reflections, the hosts highlight the broader paradigm shift implied by the paper. The geometry of a world model's latent space should not default to an isotropic Gaussian blob simply because self-supervised learning conventions inherited from computer vision dictate it. Latent geometry is an explicit system design choice. Introducing true zeros into the representation simplifies the learning burden on the physics engine, providing discrete structural slots that signal which physical regime the system occupies.

The episode leaves listeners with a core open challenge for model-based reinforcement learning: at what threshold of dynamical complexity does even a heavy transformer like a DiT begin to fail on dense geometries and actively demand sparse representations? While the current experiments stop just short of proving that transition point, the paper lays the architectural foundation for testing it.

## Notable quotes > [00:00](https://www.youtube.com/watch?v=FVpfCJfV110&t=0s) Why does every world model squeeze its features into a Gaussian blob?

> [02:51](https://www.youtube.com/watch?v=FVpfCJfV110&t=171s) What if the code were one-hot?

> [02:51](https://www.youtube.com/watch?v=FVpfCJfV110&t=171s) a lookup table wearing a linear algebra costume. — Mara

> [04:30](https://www.youtube.com/watch?v=FVpfCJfV110&t=270s) How do you learn "almost one-hot" without a codebook?

> [06:03](https://www.youtube.com/watch?v=FVpfCJfV110&t=363s) How do you measure "simpler dynamics"?

> [09:11](https://www.youtube.com/watch?v=FVpfCJfV110&t=551s) Is this just SIGReg being weak? — Mara

> [09:11](https://www.youtube.com/watch?v=FVpfCJfV110&t=551s) What do the zeros mean?

> [12:27](https://www.youtube.com/watch?v=FVpfCJfV110&t=747s) Does it still work when things touch?

> [12:27](https://www.youtube.com/watch?v=FVpfCJfV110&t=747s) the zeros are just a motion detector.

> [12:27](https://www.youtube.com/watch?v=FVpfCJfV110&t=747s) Sparsity gives you a slot for the mode. A temporal prior decides what goes in the slot.

> [13:34](https://www.youtube.com/watch?v=FVpfCJfV110&t=814s) What does this paper not show?

## Mentioned Yann LeCun, Mara, Theo, Loop State, LpWM, LeWM, VICReg, SIGReg, RDMReg, JEPA, PushT, OGBench-Cube, DreamerV2, Lean 4, Claude Fable 5.1

## Video description Should a world model's latent code be dense or sparse? Every JEPA world model pushes its features toward a Gaussian blob. This paper from Yann LeCun's group (Kuang, Dagade, Le Lidec, Maes, Balestriero, LeCun, August 2026) asks whether that is the wrong shape, and shows that codes with exact zeros make the dynamics simpler to learn: a 3M-parameter predictor plans at ~70% on sparse codes where it gets ~7% on dense ones.

Loop State EP. 02. Two hosts, Mara and Theo, walk the paper one question at a time.

Chapters 0:00 Why does every world model squeeze its features into a Gaussian blob? 1:20 What is a JEPA world model actually predicting? 2:51 What if the code were one-hot? 4:30 How do you learn "almost one-hot" without a codebook? 6:03 How do you measure "simpler dynamics"? 7:37 Where does sparsity actually win? 9:11 Is this just SIGReg being weak? 10:22 What do the zeros mean? 12:14 Does it still work when things touch? 13:41 What does this paper not show? 15:12 Sign-off

What the paper shows - RDMReg swaps LeWM's Gaussian target for a Rectified Laplace, so 30–65% of latent coordinates are exactly zero. Everything else (encoder, predictor, CEM planner) is shared. - Proposition 1: one-hot codes turn any Lipschitz controlled dynamics into exactly linear latent dynamics, at the price of exponentially many dimensions. - On PushT, sparse beats dense by 24–57 points (MLP∘LTI), 36–45 (MLP∘LTV), 11–23 (LTI(k)) at intermediate predictor capacity. Ties at the DiT rungs. Both fail at LTI(1). - The sparse advantage survives a second dense baseline (VICReg). - The zeros are interpretable: on Piecewise, the binary support decodes the dynamics zone (0.97) even with no visual cues, and magnitudes decode position. On OGBench-Cube, the support only tracks contact once a temporal-Jaccard prior is added.

What it does not show No win over dense codes with a strong predictor, four simulated environments and no real robot, one training seed with three evaluation seeds, and no head-to-head with discrete latents like DreamerV2's categoricals.

Paper: https://arxiv.org/abs/2608.22764 Figures are rasterized from the arXiv e-print. Concept diagrams and pixel-art hosts were authored for this episode. Voices: MiniMax speech-2.8-hd. Board rendered with Leanim (Lean 4). Produced with Claude Fable 5.1.

#WorldModels #JEPA #SparseRepresentations #SelfSupervisedLearning #Robotics #MachineLearning #YannLeCun #AIResearch

The chapter timestamps are computed from the frozen narration manifest. Verify them against the upload once, since YouTube requires the first stamp to be 0:00 and at least three stamps to show chapter markers, both of which this list satisfies.

[Source](https://www.youtube.com/watch?v=FVpfCJfV110)

中文简报

LeCun团队发现稀疏隐表征可将世界模型规划性能提升57个百分点 在隐表征中引入精确零值能够使跨动力学机制的物理转换线性化，从而让仅有300万参数的紧凑预测器在标准稠密表征失效的场景中取得成功。 主讲人探讨了论文《LpWM: A Case for Sparse Representations in World Models》，该研究表明在隐表征中引入精确零值能显著降低环境动力学的学习难度。基于One-hot状态分区的理论直觉促成了RDMReg正则化项的提出，该方法引导特征趋向于修正拉普拉斯分布，无需码本即可产生30%至65%的精确零值。在PushT基准测试中，稀疏隐表征使得中等低容量预测器（如基于线性动力学的MLP）的性能比稠密表征高出多达57个百分点，而两者在深度Transformer层级上表现持平。在Piecewise和OGBench-Cube基准上的分析表明，激活坐标支撑集能够清晰分解物理模式，不过追踪物理接触仍需引入辅助的时间Jaccard损失。 Yann LeCun团队推出了LpWM，表明在JEPA世界模型中用修正拉普拉斯分布替代标准高斯隐变量目标可产生30%–65%的精确零值，并大幅简化所学得的动力学。 (0:00) 在PushT基准测试中，使用稀疏隐变量的中等容量预测器性能比稠密基线高出多达57个百分点，其中参数量仅300万的紧凑MLP ∘ LTI预测器在稀疏编码上实现了约70%的规划成功率，而在稠密编码上仅为约7%。 (6:03) 线性探针证实了一种被称为“模式分解”（mode factoring）的机制现象：隐向量的二元零值支撑集能够以97%的准确率解码离散物理受力机制，而连续坐标幅度则编码了各机制内的位置。 (9:11) 尽管朴素的帧级稀疏性主要追踪末端执行器的运动（r = 0.87），但引入惩罚支撑集翻转的辅助时间Jaccard损失，成功使零值模式重新对齐以追踪物理接触（r = 0.61），且未降低规划性能。 (12:27) 稀疏表征的性能优势仅局限于中等预测器容量；当扩展到高容量的Diffusion Transformer（DiT）时，稀疏表征与稠密表征取得了相同的表现。 (13:34)

Reading summary

In an August 2026 paper, Kuang et al. present LpWM, arguing that enforcing sparse latent representations in self-supervised world models radically simplifies transition dynamics compared to standard dense Gaussian representations. By inducing coordinates with exact zeros, the authors demonstrate that lightweight, nearly linear predictors can plan effectively where dense models require expressive transformers. Theoretical Linearization via Sparse Representations (2:51) To justify sparsity mathematically, the authors introduce Proposition 1, analyzing an extreme case where latent states are strictly one-hot vectors. Under this formulation, partitioning a compact state space into N discrete cells converts Lipschitz-continuous nonlinear dynamics into linear transition permutation matrices, bounded by an error rate of N^(-1/d). While an exact one-hot code suffers from the curse of dimensionality by demanding exponential dimensions, it illustrates how sparsity can linearize transition mechanics. LpWM relaxes this discrete requirement into distributed continuous sparsity. Instead of storing discrete lookup tables, the model retains 30% to 65% active coordinates, aiming to capture partial dynamical linearity while operating within a computationally tractable dimensional budget. Inducing Exact Zeros with RDMReg (4:30) Standard Joint Embedding Predictive Architectures (JEPAs) like LeWM and VICReg prevent representation collapse by maximizing latent entropy toward isotropic Gaussians or identity covariance matrices, which guarantees fully dense representations. LpWM replaces these targets with a rectified Laplace distribution, characterized by exponential decay along the positive axis and a concentrated probability mass at zero. To align latent distributions without discrete codebooks, the authors introduce RDMReg, which minimizes the 2-Wasserstein distance between projected batch features and the rectified prior across random unit directions. The encoder projector applies a reparameterized activation that evaluates a strict ReLU on the forward pass to produce exact zeros, while routing gradients through a Gaussian Error Linear Unit (GELU) backward pass to maintain gradient flow. Empirical Gains Across Predictor Capacities (6:03) Evaluating planning on the contact-heavy PushT benchmark across a spectrum of predictors—ranging from simple Linear Time-Invariant (LTI) systems to a 6-layer Diffusion Transformer (DiT)—reveals that sparsity primarily benefits resource-constrained architectures. On intermediate predictors such as MLP ∘ LTI and MLP ∘ LTV, LpWM outperforms dense LeWM by 24 to 57 percentage points in closed-loop planning success. For example, using a 3-million-parameter MLP ∘ LTI predictor at a width of 768, sparse codes reached roughly 70% success, whereas dense codes attained 7%. When tested against VICReg, LpWM maintained superior performance across all intermediate tiers and also outperformed it at the deep transformer level. However, against the stronger LeWM baseline, sparse and dense representations achieved parity when paired with the full 822-million-parameter DiT predictor, indicating that the immediate advantage of sparse codes lies in enabling radically smaller physics predictors. Mode Factoring and Support Tracking Limitations (9:11) Analysis in the synthetic Piecewise environment reveals a structural division of labor termed mode factoring. Linear probes show that the binary mathematical support (the pattern of non-zero entries) decodes discrete dynamical regimes with approximately 97% accuracy, while continuous coordinate magnitudes encode position within each regime. This physical mode segmentation occurs even when visual regime markers are eliminated from the environment. Conversely, tests on the OGBench-Cube manipulation benchmark show that pure marginal RDMReg regularization fails to naturally track object interactions; out of the box, support patterns correlate heavily with robotic end-effector motion (0.87) but barely with cube contact (0.05). The authors address this by adding an auxiliary temporal Jaccard loss to penalize rapid support transitions, which shifts support correlation toward cube motion (0.80) and physical contact (0.61) without harming planning success. The authors emphasize that latent geometry is an explicit architectural choice rather than an automatic default to Gaussian distributions. Significant qualifications remain: LpWM has only been tested in simulated, predominantly planar domains using single training seeds, does not directly compare against discrete categorical models like DreamerV2, and achieves parity rather than superiority over dense baselines when using high-capacity transformer predictors.

Source quotes

[0:00] Why does every world model squeeze its features into a Gaussian blob? [2:51] What if the code were one-hot? [2:51] Mara: a lookup table wearing a linear algebra costume. [4:30] How do you learn "almost one-hot" without a codebook? [6:03] How do you measure "simpler dynamics"? [9:11] Mara: Is this just SIGReg being weak? [9:11] What do the zeros mean? [12:27] Does it still work when things touch? [12:27] the zeros are just a motion detector. [12:27] Sparsity gives you a slot for the mode. A temporal prior decides what goes in the slot. [13:34] What does this paper not show?

Original transcript

[0:00] [snorts] Welcome to Loop State. I'm [0:02] Mara. Today's paper comes out of um Yan [0:07] Lun's group and it starts with a [0:10] question I had never thought to ask [0:11] Theo. What's the question? [0:13] >> Why does every self-supervised world [0:15] model push its features toward a dense [0:17] Gaussian blob? Lew does it. Vikra style [0:20] models do it. The paper is called LPWM, [0:24] a case for sparse representations in [0:26] world models by Kuang and colleagues, [0:28] August 2026. [0:31] >> And the pitch is that dense is the wrong [0:34] shape. [0:35] >> The pitch is narrower and more [0:37] interesting. If the latent code is [0:39] sparse, meaning a good fraction of its [0:41] coordinates are exactly zero, then the [0:44] dynamics you have to learn on top of it [0:46] get simpler. Simple enough that a nearly [0:48] linear predictor can plan where a dense [0:51] code needs a transformer. [0:53] >> So the driving question for today is [0:56] should a world model's memory of the [0:58] world be dense or sparse? [1:01] >> And the headline number on the push task [1:04] sparse codes be dense ones by up to 57 [1:08] points of planning success at [1:10] intermediate predictor capacity. We'll [1:12] earn that number over the next 10 [1:14] minutes. Let's start with what a world [1:18] model is even predicting. [1:21] Theo for the person who has heard the [1:23] word jeepa but never opened the paper. [1:26] What's the machine? [1:28] >> Three parts. An encoder F takes an image [1:30] and produces a latent vector Z. A [1:33] predictor G takes ZT and the action A T [1:36] and outputs a guess for the next latent [1:38] Zhat T +1. The training loss is just the [1:42] distance between that guess and the [1:44] encoding of the real next frame. [1:46] >> Um, so it never draws pixels. It [1:50] predicts in its own latent space, [1:53] >> right? And that's where the trap lives. [1:55] If nothing stops it, the encoder learns [1:57] to output a constant. Every prediction [1:59] is perfect and useless. That's collapse. [2:03] >> And the fix is the regularizer. [2:06] The fix is to force the latent [2:08] distribution to look like something [2:10] specific. Lilim uses sigereg to match an [2:13] isotropic gaussian. Vigrag matches the [2:15] variance and covariance to the identity. [2:18] Both are maximum entropy targets. So [2:21] both produce dense codes. Every [2:23] coordinate is nonzero almost surely [2:26] >> and planning. [2:28] >> At test time, you encode the current [2:30] frame and the goal frame. then search [2:32] over action sequences with the cross [2:34] entropy method until the predicted roll [2:36] out lands near the goal latent. The [2:39] predictor is the whole physics engine. [2:42] So the question becomes how complicated [2:44] does that engine have to be? [2:46] >> Here's the thought experiment that [2:48] hooked me. Suppose the latent code were [2:51] not dense at all. Suppose it were one [2:53] hot exactly one coordinate on everything [2:58] else zero. What happens to the dynamics? [3:02] >> They become linear. Exactly linear. The [3:06] paper proves it as proposition one. Take [3:09] any compact state space and any lipshits [3:12] dynamics. Chop the state space into n [3:14] cells. Encode a state as the one hot [3:17] vector of its cell [3:19] >> and an action. [3:21] >> An action becomes a matrix P of A. It [3:24] moves the hot entry from this cell to [3:26] the cell the dynamics would land in. So, [3:29] Zt + 1 equals P of A T * ZT. The decoded [3:35] prediction is off by atmost L +1 * [3:39] epsilon, the cell radius. [3:42] >> So, nonlinear physics became a lookup [3:46] table wearing a linear algebra costume. [3:50] >> That's exactly what it is. And the [3:52] corollary says on the unit cube in [3:54] dimensions with n cells the error [3:57] shrinks like nus1 /d it goes to zero as [4:00] the code gets wider [4:02] >> but that exponent is a curse 1 /d [4:08] >> the authors say so themselves exact one [4:10] hot codes need exponentially many [4:12] dimensions that's not the proposal the [4:15] proposal is the relaxation a distributed [4:18] sparse code where a chunk of the [4:20] coordinates are zero might buy you a [4:22] fraction of that linearity at a price [4:24] you can pay. [4:26] >> So how do you get sparity out of a jeepa [4:30] without a codebook without vector [4:31] quantization without any discrete [4:33] bottleneck? [4:34] >> You change the target distribution. Lew [4:37] matches the latent to a Gaussian. LPWM [4:40] matches it to a rectified generalized [4:42] Gaussian. Concretely a lelass [4:44] distribution with mean zero passed [4:46] through a relu. [4:47] >> Rectified lelass. Draw me the picture. [4:51] >> A sharp exponential decay on the [4:53] positive side and a spike of probability [4:56] mass sitting exactly at zero because [4:59] everything negative got clipped there. [5:01] The regularizer called RDMREG projects [5:04] the batch onto random unit directions [5:06] and pushes each projection to that [5:08] target with a two Waserstein distance. [5:11] >> And the zeros are exact, not just small. [5:15] >> Exact. The projector ends in a [5:17] reparameterized relu. Forward pass is a [5:20] plain relu so you get true zeros. [5:22] Backward pass borrows the gu gradient so [5:25] neurons don't die. They measure the code [5:28] 30 to 65% of coordinates active [5:31] depending on width versus 100% for dense [5:34] loom. [5:36] >> And if I set the target back to a plain [5:40] Gaussian with no relu, [5:43] >> you recover luma. Exactly. That's what [5:46] makes the comparison clean. One knob on [5:48] the target distribution separates dense [5:50] from sparse. Everything else, encoder, [5:53] predictor, planner, is shared. [5:57] >> Uh, okay. The claim is that sparse codes [6:00] have simpler dynamics. That's not a [6:03] number. How do you turn it into one? [6:05] >> You build a ladder of predictors from [6:08] dumb to smart. And you ask at each rung, [6:11] can this predictor plan? The rung where [6:13] sparse succeeds and dense fails is your [6:17] measurement of simplicity. [6:19] >> What's on the ladder? [6:20] >> Bottom rung LTI1. Next latent equals A * [6:24] Z + B * A pure linear one frame. Then [6:29] LTI K with three frames of history. Then [6:32] MLP LTI1 relu layer on top. Then MLP LTV [6:38] where the matrices depend on the state. [6:41] Then a one layer transformer. Top rung [6:45] the six layer DIT predictor LUM ships [6:48] with. [6:49] >> And how different are they in size? [6:52] >> At width 4,96 [6:55] the top rung has 822 million parameters. [6:58] The bottom rung has 34 million. The [7:01] middle rungs sit around 70 to 85 [7:03] million. Everything below the [7:05] transformers is attentionfree [7:07] >> and both sparse and dense get the same [7:13] encoder and the same planner. [7:15] >> Same everything five widths from 384 to [7:18] 496 [7:20] with maximal update parameterization. So [7:22] width scaling doesn't wreck the [7:24] hyperparameters. Two environments wall [7:27] which is easy and push t which is not. [7:31] So where on the ladder does sparsity [7:34] win? [7:35] >> Start with where it doesn't. On wall, a [7:38] 2D agent finding a doorway. The bottom [7:40] rung already gets near 100% closed loop [7:43] success for both dense and sparse. The [7:47] dynamics are linear enough that there's [7:49] nothing to simplify. [7:51] >> So [7:53] um wall is a null result by design. [7:56] >> Correct. And the paper says so. Now push [7:59] T. Push a T-shaped block to a target [8:02] pose. Contact, rotation, friction. At [8:06] the bottom rung, LTI1, both fail. At the [8:09] top rungs, the transformers both succeed [8:12] about equally. [8:14] >> In the middle, [8:15] >> the middle is the whole story. With MLP [8:18] LTI, sparse LPWM beats dense LWM by 24 [8:23] to 57 points of success rate across [8:25] widths. with MLP LTV 36 to 45 points [8:30] with plain LTI K 11 to 23 points. [8:34] >> Read that off the figure for me. [8:37] >> Figure 1B predictors down the side [8:40] widths across and every arrow in the [8:42] middle block points up. My favorite cell [8:45] at width 768 MLP LTI is a 3 million [8:49] parameter predictor. Closed loop. It [8:52] plans at about 70% on sparse codes and [8:55] about 7% on dense. [8:59] >> So the payoff isn't a better world [9:01] model. It's a cheaper one that works. [9:04] Devil's advocate. Maybe dense isn't the [9:07] problem. Maybe low whims particular [9:10] Gaussian regularizer is um did they [9:15] check another dense baseline? [9:17] >> They did. They swapped sigreg for [9:19] vicreg. same architecture and reran the [9:22] pusht ladder. Sparse LPWM wins again on [9:26] MLP LTV and MLP LTI and this time it [9:30] also wins at the top rung the deep [9:32] transformer. [9:33] >> Wait, why would it beat Vicreg even with [9:35] a strong predictor when it only tied LOM [9:38] there? [9:38] >> Their hypothesis Vik only constrains [9:41] second moments variance and covariance [9:44] that leaves the target distribution [9:46] underspecified. So the dense code it [9:48] produces is a worse geometry to predict [9:50] in sigreg pins the full distribution. So [9:53] it's a stronger dense baseline that's [9:56] tagged as a hypothesis in the paper not [9:59] a proof. [10:00] >> So the sparse advantage survives two [10:03] different dense baselines [10:05] >> and it survives their hyperparameter [10:08] sweeps which are in the appendix for [10:10] both methods. Now the question I [10:13] actually care about what are the zeros [10:15] doing [10:16] >> right if 30 to 60% of the code is [10:20] switched off which coordinates are off [10:22] and does it mean anything [10:24] >> to answer that they built an environment [10:26] where the answer is known peacewise an [10:29] open room cut into zones and each zone [10:32] applies a different constant push to the [10:35] agent locally linear globally peacewise [10:38] aphine the zone is the hidden [10:41] and they compare the codes across the [10:45] room. [10:46] >> They put the agent on a 20x 20 grid, [10:49] encode every position, and binarize each [10:52] code into its support, which coordinates [10:54] are non zero. Then they measure overlap [10:57] with a jackard index intersection over a [11:00] union of the two supports. [11:02] >> And the heat map, [11:04] >> pick one reference cell. Its support [11:07] overlaps heavily with every cell in the [11:09] same zone and drops sharply at the zone [11:12] boundary. Here's the part that got me. [11:15] It still does that when the zones are [11:17] not rendered. No colors, no visual cue. [11:21] The code found the physics, not the [11:23] paint. [11:24] >> So the support is the discrete state [11:29] >> and the magnitudes are the continuous [11:31] state. Linear probes confirm it. The [11:34] binary support decodes the zone at about [11:36] 97% exactly as well as the full vector [11:39] does. Position inside the zone decodes [11:42] better from the magnitudes than from the [11:44] support. They call this mode factored [11:47] >> and planning in peacewise. [11:50] >> Random goals 84.7% for sparse against [11:54] 65.3 for dense. And the sparse models [11:57] stay ahead at every planning horizon [11:59] they tested, which is what you'd expect [12:02] if the code knows which aphine regime [12:04] it's in. [12:05] >> Peacewise is a toy where the regimes are [12:08] painted on the floor. What about [12:11] contactrich manipulation where the [12:14] regime change is the gripper touched the [12:17] cube? [12:18] >> This is where the paper is most honest. [12:20] On OG Bench Cube, they measure support [12:22] and stability. One minus the jakard [12:25] between consecutive frames. Plain LPWM [12:27] support changes whenever the arm moves. [12:30] Correlation with aector motion 0.87. [12:34] Correlation with contact 0.05. [12:37] >> Um, so without help, the zeros are just [12:42] a motion detector. [12:44] >> Their words almost exactly. RDMREG only [12:47] shapes each frames marginal. It says [12:49] nothing about time. So they add an [12:51] optional temporal jakard loss that [12:53] penalizes support changes between [12:55] adjacent frames. [12:57] >> And that flips it. [13:00] >> As the weight goes up, correlation drops [13:03] from 87 to 40. Cube motion correlation [13:07] rises to 80. Contact rises from 05 to [13:13] 61. Planning success doesn't move. The [13:15] raster plot shows the support going [13:17] quiet and then reorganizing inside the [13:20] contact window. [13:21] >> So the support can track physical [13:24] events, but you have to tell it that [13:26] time matters. [13:28] >> Sparsity gives you a slot for the mode. [13:30] A temporal prior decides what goes in [13:32] the slot [13:34] >> boundary time. Uh what does this paper [13:37] not show? It does not show sparse codes [13:41] beating dense ones with a strong [13:42] predictor on push t with the dit they [13:45] tie. The win is at intermediate [13:48] capacity. Full stop. [13:50] >> The theory [13:51] >> the theory is about exact one hot codes [13:54] which need exponentially many [13:56] dimensions. The learned codes are 30 to [13:59] 65% active nowhere near one hot. The [14:02] linearization result motivates the [14:04] method. It does not describe the trained [14:07] model [14:08] >> environments [14:10] >> four all simulated three of them [14:12] two-dimensional navigation or pushing no [14:15] real robot one training seed three [14:18] evaluation seeds and results reported at [14:20] the tuned best hyperparameter cell for [14:23] each method [14:24] >> and the interpretability claim [14:28] >> holds cleanly where the modes are [14:29] spatial on cube it needs an extra loss [14:32] to mean anything and that loss is [14:35] optional and hand weighted. Also, no [14:37] head-to-head with discrete latent like [14:40] Dreamer v2's categoricals, even though [14:42] the related work points there. [14:44] >> What do the authors themselves flag as [14:47] the hope rather than the result? [14:50] >> That as dynamics get harder, the sparse [14:53] advantage will climb the ladder to [14:55] stronger predictors and that sparse [14:57] codes might one day make classical [14:59] control like LQR usable in latent space. [15:03] Both are stated as future work. [15:06] >> My takeaway, the shape of a latent is a [15:09] design choice, not a law of nature. [15:12] Zeros in the code buy you a simpler [15:14] physics engine, and the pattern of zeros [15:17] tells you which regime you're in. [15:19] >> My open question, at what point does the [15:22] dynamics get hard enough that even the [15:25] transformer wants the zeros? That's the [15:28] experiment this paper sets up and [15:30] doesn't run yet. Thanks for listening to [15:32] loop