Sanders Pushes 20-Year Prison Terms in Bid to Halt Superhuman AI

TBPN

Sweeping safety proposals seek to delay superintelligence to 2040 through compute caps and corporate death penalties, threatening to entrench incumbent tech giants while ignoring the vast untapped upside of existing models.

AI-assisted summary of Washington Hones in on AI Safety, What Are AI Doomers Proposing | Diet TBPN — original source. Check the original for full context.

Source published: 2026-09-12

Detailed notes

## Background Published on YouTube on September 12, 2026, by the tech talk show TBPN (hosted by John Coogan and Jordi Hays), this Diet TBPN episode delivers a concise 30-minute digest of their daily broadcast. According to the show notes, the episode examines Washington's emerging focus on AI safety, detailing the specific policy proposals of AI doomers alongside debates over compute controls, legislative bans, and digital sentience.

## TL;DR The hosts break down concrete proposals from AI safety advocates, moving beyond vague extinction warnings to inspect the 'AI 2040' blueprint, which calls for halting frontier training runs, auditing compute clusters, and enforcing dual-key U.S.–China model weight transfers. They scrutinize aggressive legislative maneuvers like Senator Bernie Sanders' proposed 'corporate death penalty' for unauthorized superintelligence development, contrasting it with the pro-abundance tech backlash. The conversation also explores Tyler Cowen's market falsifiability test for x-risk, the paradox of safety researcher Paul Christiano holding 2x levered long AI bets, and the ethical frontier of digital minds sparked by an open-source fruit fly connectome running on a Rabbit R1. Finally, they pivot to consumer AI agency, illustrating autonomous computer use through casual gaming in Balatro and real-time telemetry coaching in iRacing.

## Atmosphere & style The episode blends rapid-fire tech-insider commentary with dry, irreverent humor, transforming grim existential risk discussions into an energetic and approachable studio riff. Coogan and Hays bounce jokes off one another with casual confidence, puncturing heavy nuclear non-proliferation parallels with absurd pop culture touches like 'Scary Potter,' DoorDash burritos, and existential guilt over shaking a simulated fruit fly. It feels like listening to two hyper-online builders who refuse to get bogged down in apocalyptic fatalism, balancing genuine technical literacy with breezy skepticism and relatable consumer experimentation.

## Key takeaways - [00:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=0s) The AI 2040 proposal advocates slowing artificial superintelligence timelines to 2040 rather than halting technology entirely, keeping existing inference active while placing a moratorium on new frontier training runs. - [01:24](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=84s) To enforce compute controls, AI 2040 proposes mandatory registration and licensing for clusters over 10,000 NVIDIA H100s, treating GPU counts like nuclear warhead inventories under arms control treaties. - [01:24](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=84s) Physical isolation measures for frontier R&D include removing internal east-west networking to block distributed training and capping external pipes at 1 MB/s to make checkpoint theft take years. - [05:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=300s) Under extreme containment blueprints, trained model weights would be inflated to 100 TB and require dual-key cryptographic escort by both U.S. and Chinese diplomatic delegations. - [10:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=600s) Safety skeptics and tech founders argue that stringent compute limits risk regulatory capture for incumbents, economic stagnation, and violations of foundational developer liberties. - [13:30](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=810s) AI doom concerns are primarily targeted at autonomous volition and recursive self-improvement rather than the physical deployment of embodied robots running current foundation models. - [15:21](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=921s) Senator Bernie Sanders' proposed bill introduces severe penalties for unauthorized frontier AI development, including a 20-year federal prison sentence for individuals and the 'corporate death penalty' for organizations. - [20:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1200s) Economist Tyler Cowen argues AI pessimists should demonstrate market skin in the game through falsifiable asset pricing, though total extinction risks pose a paradox because catastrophic payout contracts cannot be collected. - [20:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1200s) Despite his x-risk concerns, AI safety researcher Paul Christiano holds a 2x levered long position on tech equities combined with shorting long-term Treasuries, betting heavily on rapid economic expansion in benign scenarios. - [24:15](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1455s) The open-sourcing of a fully digitized fruit fly connectome running on hardware like the Rabbit R1 has opened new philosophical debates on whether emulated biological neural networks possess moral standing. - [28:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1680s) Advancements in agentic 'computer use' are rapidly expanding into interactive consumer leisure and real-time coaching, from playing deckbuilders like Balatro to analyzing racing telemetry in simulators.

## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=0s) Congress Wakes Up to AI Doom and the AI 2040 Blueprint

The hosts kick off the discussion by asking what AI safety advocates and "doomers" are actually proposing beyond abstract alarmism and threats of extreme penalties, like 20-year prison sentences. A timeline predicted by "AI 2027" claimed that by late 2026, Congress would suddenly wake up to the existential risks of AI—a forecast that seems to be coming true as mainstream outlets like *The Wall Street Journal* report on Capitol Hill's sudden fixation on AI doom scenarios. The anxiety is bleeding directly into pop culture; the hosts joke about Matt Damon being intercepted and interrogated by TMZ for his thoughts on AI risk. Meanwhile, public demonstrations are beginning to emerge: while an anti-AI protest in Chapel Hill, North Carolina, only turned out around 150 people, the hosts note with dry humor that it was still technically "just two orders of magnitude (OOMs) away" from a 10,000-person prediction.

```mermaid flowchart LR accTitle: AI 2040 Strategy Shift accDescr: Diagram illustrating the shift from unchecked acceleration to controlled capability growth by 2040. subgraph StatusQuo["Current Trajectory"] A["Unchecked Scaling"] --> B["Superintelligence by ~2028"] end subgraph AI2040["AI 2040 Proposal"] C["Inference-Only Verification"] --> D["Air-Gapped R&D Facilities"] D --> E["Managed Superintelligence by ~2040"] end ```

Looking past the vague public refrain of "we just have to talk about this," the hosts examine the concrete roadmap presented in the "AI 2040" proposal. Contrary to viral anti-AI narratives that call for an outright technological halt, AI 2040 doesn't want to shut down existing AI: - **Keep inference flowing**: Existing frontier models remain online, and capability research continues. - **Extend the runway**: The goal is delaying the arrival of artificial superintelligence (ASI) from an unprepared 2028 timeline out to 2040, giving nation-states time to establish safety guardrails. - **Moratorium on new frontier training**: All new frontier training runs and open-ended R&D experiments would be halted under current setups.

---

[01:24](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=84s) Compute Audits, East-West Networking Bans, and Physical Isolation

To enforce this pause on new frontier runs, the AI 2040 agenda calls for rigorous inference-only verification across all major data centers. Any facility housing more than 10,000 NVIDIA H100 equivalents—representing roughly $100 million in hardware—would be subject to mandatory registration, licensing, and workload audits conducted by independent government or international bodies. The rationale is simple: large-scale compute clusters cannot easily hide because they are housed in massive, identifiable industrial structures. Furthermore, countries would be required to declare their AI compute inventories openly, treating H100 counts much like international arms control treaties treat nuclear warheads.

```mermaid flowchart TD accTitle: Data Center Physical Enforcement Protocol accDescr: Infrastructure changes to prevent clandestine frontier model training and weight exfiltration. subgraph DataCenter["R&D Data Center Containment"] direction TB N1["Remove East-West Networking (Prevents distributed training)"] N2["1 MB/s Capped Communications Pipe (Exfiltration takes years)"] N3["Physical Air-Gaps & Bunker Security"] end DataCenter --> Export["Model Weight Transfer"] subgraph Transfer["Dual-Key International Transfer"] direction TB T1["Bloat Weights to 100 TB"] T2["Dual Encryption: US + China"] T3["Physical Escort by Both Nations"] end Export --> Transfer ```

To make distributed training physically impossible within standard clusters, the proposal demands the physical removal of high-bandwidth east-west networking inside data centers, limiting them purely to inference. Any future frontier R&D training would require entirely new, purpose-built bunkers featuring: - **State-level physical security**: Nuclear-grade access controls and air-gapped perimeters. - **Hardware-throttled bandwidth**: External data connections capped at a strict 1 megabyte per second (1 MB/s). At that trickle, researchers could send small commands and code, but unauthorized exfiltration of a full model weight checkpoint would take roughly five years, making data theft immediately obvious if the pipe runs at full capacity.

---

[05:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=300s) Dual-Key U.S.–China Verification and the Scale Prerequisite

The proposal reaches peak geopolitical complexity when dealing with how model weights leave R&D bunkers. Under the AI 2040 design, once a model is trained, its weights would be transferred onto physical drives, encrypted independently by *both* the United States and China, and physically escorted to inference data centers by diplomatic delegations from both superpowers. To make theft even harder, the authors suggest deliberately training models to be larger than compute-optimal—bloating weight files to 100 terabytes rather than an easily pocketed 1 terabyte. The hosts riff on this extreme paranoia, joking that podcast studios will soon need Faraday cages and locked briefcases holding Shure SM7B microphones that require simultaneous cryptographic keys from Patrick O'Shaughnessy and David Senra just to record an episode.

Turning serious again, the conversation focuses on compute caps as the central policy lever. However, restricting compute risks massive unintended side effects, chief among them creating a black-market incentive for clandestine research: - **The "God model" incentive**: Banning official large-scale training could drive thousands of independent researchers underground to discover algorithmic shortcuts. - **Nuclear non-proliferation parallels**: Just as rogue nations pursue centrifuges in secret, rogue actors could assemble clandestine clusters. - **Physical observability of compute**: Unlike running a tiny Python script on a consumer laptop, reaching AGI still appears to be bound to the scaling hypothesis. Even Ilya Sutskever’s secretive new venture, Safe Superintelligence (SSI), secured massive GPU clusters rather than relying on algorithmic breakthroughs alone. - **The thermal footprint giveaway**: Large training clusters require immense power and emit vast heat signatures readily detectable by orbital satellite surveillance, making true stealth facilities difficult to conceal without constructing deep underground bunkers.

[10:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=600s) The AI 2040 Roadmap, Manhattan Project Parallels, and Skeptic Backlash

Addressing how national security and frontier research might intersect, the hosts discuss whether containment looks less like a worldwide freeze and more like the Manhattan Project: roughly a hundred top researchers working directly with the government in total secrecy, recognizing that you cannot assume foreign adversaries like China will pause unilaterally. However, they clarify that the overarching "AI 2040" proposal is not aimed at rewinding the clock or halting technology outright. Instead, it is an intentional deceleration to scale gradually toward superintelligence by 2040. While current industry forecasts anticipate human-level artificial general intelligence as early as 2027, 2028, or 2029, the AI 2040 framework proposes pacing compute so that top human expert capability is reached around 2035, holding steady for a five-year evaluation window, and only then unlocking superintelligence in 2040.

```mermaid flowchart LR accTitle: AI 2040 Proposed Deceleration Timeline accDescr: Visual depiction of the AI 2040 roadmap from current models to superintelligence.

subgraph Phase1["Current Trajectory"] A["Current Era Fast Race"] --> B["Target Human Expertise Delayed to ~2035"] end

subgraph Phase2["Observation Window"] B --> C["5-Year AGI Plateau Safety Testing (2035-2040)"] end

subgraph Phase3["Superintelligence"] C --> D["Superintelligence Unlocked Target: 2040"] end ```

The hosts observe that despite debates getting bogged down in arbitrary $p(\text{doom})$ numbers and speculative extinction scenarios, the proposal at least provides a tangible legislative roadmap. Yet for safety skeptics, placing sweeping government mandates over computing feels blatantly authoritarian. Critics point out that regardless of whether a cluster costs $100 million, developers maintain a fundamental philosophical conviction: "I should be able to do math on my computer. I can do whatever I want. Let me do cool things." Enforcing rigid state guardrails introduces real risks of regulatory capture for entrenched incumbents, potential stock market downturns, and stalled economic prosperity. In a benign scenario where alignment is solved, catastrophic risk could fade into background noise—similar to the ambient risk of an asteroid hitting Earth, where people acknowledge the hazard without altering daily conduct. The hosts admit they find it difficult to fully "blackpill" about doomer proposals because existing models already suffer from a massive capability overhang, leaving vast economic value still untapped even on non-frontier systems.

- **AI 2040 Timeline**: Pushes projected AGI arrival from the late 2020s out to 2035, pauses for a 5-year observation period, and aims for superintelligence by 2040. - **Economic & Political Objections**: Safety skeptics argue compute curbs prompt regulatory capture, harm markets, and violate foundational tech liberties. - **Prominent Counter-Voices**: Industry leaders and investors like Brad Gerstner, Jensen Huang, and David Sacks are increasingly speaking out against alarmist pauses.

---

[13:30](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=810s) Robotics, Autonomous Volition, and the "AC Guy" Analogy

The conversation turns to whether the AI 2040 camp accounts for embodied intelligence, such as physical robotics. If society introduces billions of humanoid robots powered simply by current-generation architectures like GPT-4 or GPT-5, does that constitute a physical threat? The hosts argue that safety advocates are not losing sleep over robots armed with paintbrushes and cameras running current alignment stacks, citing OpenAI’s internal robotics demos. The existential dread is explicitly reserved for recursive self-improvement and genuine autonomous volition—systems possessing their own emergent goals rather than rigidly obedient tools that merely misinterpret instructions like a classical paperclip maximizer.

The hosts then react to a viral social media jab from Clément Delangue, co-founder and CEO of Hugging Face, who wrote: "Sorry, but asking Jacob about AI extinction risk is like asking your AC guy about climate change, not saying it's necessarily uninteresting or wrong per se, but let's keep things in perspective and hear from the full range of expertise across the ecosystem." AI researcher Nathan Lambert endorsed the post as a "banger." The hosts joke that this felt like an unprovoked slight against HVAC technicians, riffing on an imaginary repairman saying, "I don't know about all that mumbo jumbo, but when it's a hot summer day, I don't want you to be worried about the heat, brother."

- **Physical Robotics vs. Volition**: Deploying billions of embodied robots running existing foundation models is viewed as manageable; doomers worry specifically about self-directed superintelligence with its own agency. - **Ecosystem Backlash**: Clément Delangue’s analogy underscores growing frustration across the open-source community over which self-appointed voices are being treated as authoritative safety experts by media and lawmakers.

---

[15:21](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=921s) Bernie Sanders' Ban, ASI Destruction, and the "Corporate Death Penalty"

The discussion shifts to a radical legislative proposal surfacing in Washington: Senator Bernie Sanders’ proposed ban on artificial superintelligence. The text contains remarkably harsh criminal penalties, decreeing that non-compliant organizations will face the "corporate death penalty" while individuals could be sentenced to up to 20 years in federal prison for failing to halt unauthorized frontier AI development. The hosts marvel at the phrasing, noting that while businesses regularly collapse or liquidate through Chapter 7 bankruptcy, hearing the phrase "corporate death penalty" codified into draft statute is "kind of metal."

Beyond the rhetoric, the draft bill prohibits anyone from developing or deploying systems exhibiting capabilities that match or exceed human cognition across a broad range of domains—a definition so wide that the hosts joke it reads identical to the mission statement of roughly seventeen frontier AI startups right now. Sanders’ bill also explicitly targets software capable of planning the disempowerment of humanity or the overthrow of the United States government, establishing a dedicated federal agency tasked with monitoring frontier systems across their entire life cycle, stripping out prohibited capabilities, and actively supervising the physical destruction of superintelligent models. The hosts question how enforcement could ever draw a clear line between basic prompt engineering workflows and illegal frontier capability development without ensnaring innocent engineers.

| Sanders Proposal Provision | Legislative Requirement | Host Commentary & Practical Implications | |:--- |:--- |:--- | | **ASI Development Prohibition** | Outlaws training or deploying systems matching/exceeding human cognitive performance across broad domains. | Overlaps directly with the publicly stated product roadmaps of major AI labs. | | **Federal Agency Oversight** | Continuous life-cycle surveillance of frontier models with authority to destroy unauthorized weights. | Would mandate invasive state monitoring of data centers, compute clusters, and model checkpoints. | | **Statutory Sanctions** | Up to 20 years imprisonment for individuals; the "corporate death penalty" (forced dissolution) for entities. | Exceptionally aggressive language; risks punishing engineers whose models accidentally cross vague benchmark thresholds. |

---

[17:40](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1060s) Fluidstack’s Pro-Build Stance, "Scary Potter," and the DoorDash Parody

The hosts highlight industry resistance to this regulatory creep, highlighting a statement from Jamie Cox, co-founder of compute provider Fluidstack. Cox published a pro-freedom, pro-abundance manifesto declaring that America must build aggressively to ensure democratic nations lead, arguing that AI will drive human flourishing. He advocated for clear, predictable, and capability-proportional rules that avoid arbitrary bureaucratic barriers to entry. The hosts stress that founders fear arbitrary cutoffs: if regulations approve the top ten tech incumbents, the startup at number eleven effectively gets hit with a backdoor corporate death penalty simply for lacking legacy lobbying muscle. Founder Will Manidis echoed this enthusiasm, praising the sentiment that "AI will make everyone rich, healthy, and free" as refreshingly positive and novel frontier communications.

``` "Two years at DoorDash, I do not say this lightly. We are extremely close to the burrito arriving before you decide you want it. We are not asking for a ban. We are asking for a pause." — Viral tech parody post ```

The segment winds down with the viral culture surrounding the congressional AI whistleblower. TikTok creators have posted clips noting that the witness looks like the archetypal movie scientist whom politicians ignore right before a global catastrophe strikes, while users across X (formerly Twitter) have begun labeling him "Scary Potter." Amid the apocalyptic tension, the tech community continues to cope through satire, capped off by a viral parody tweet from a mock DoorDash engineer warning that autonomous systems are on the verge of delivering burritos before the customer even realizes they are hungry—begging Washington not for a permanent ban, but merely a brief pause to catch their breath.

[20:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1200s) Frontier Lab Doom, Tyler Cowen’s Wagers, and Paul Christiano’s 2x Levered Bet

Existential panic over artificial intelligence remains almost entirely quarantined within frontier research labs. For everyone working down in the application layer—those trying to diffuse models into actual business workflows—the reality on the ground is completely detached from apocalyptic scenarios. Founders and forward-deployed engineers are simply fighting to get models to work reliably, with people like Jim Reilly at Charleston AI dismissing the doom talk as "a whole lot of mumbo-jumbo" while celebrating getting eight productive hours back in his day. Further down the supply chain, semiconductor manufacturers like Jensen Huang’s Nvidia and Wall Street analysts have little patience for doomerism, focusing instead on enterprise acceleration and hardware delivery.

The divide has sparked heated debates around financial skin in the game, spearheaded by economist Tyler Cowen. Cowen has demanded that doomers make their predictions falsifiable: "If you have very pessimistic fears or predictions about AI, name the market prices that will support or confirm them. This is what taking this seriously means." The hosts note the fundamental paradox of pricing existential catastrophe into financial markets: "Contracts that pay out if everyone dies aren't worth anything to me." While 20th-century nuclear fears yielded concrete, hedgeable preparations—such as bomb shelters, corporate continuity plans, and nonproliferation treaties—an AI apocalypse is viewed as so totalizing that physical bunkers are useless. Therefore, failing to build an "AI bunker" or short the market is not necessarily proof of insincerity.

```mermaid flowchart TD accTitle: Tyler Cowen AI Doom Falsifiability Debate accDescr: Flowchart contrasting the market pricing dilemma between Cold War nuclear risks and totalizing AI x-risk. subgraph Nuclear["Cold War Paradigm"] N1["Nuclear Apocalypse Risk"] --> N2["Falsifiable Hedging"] N2 --> N3["Physical Bunkers & Treaties"] end subgraph AIDoom["AI X-Risk Paradigm"] A1["Totalizing AI Extinction"] --> A2["Tyler Cowen Market Test"] A2 --> A3["'Contracts that pay out if everyone dies are worthless'"] end ```

The hosts examine the unusual portfolio positioning of AI safety researcher Paul Christiano, who recently joined the board of OpenAI. Cowen asked why doomers are not outright shorting the market, yet Christiano revealed he is actually 2x levered long on tech and AI equities while shorting 30-year U.S. Treasuries (which the hosts joke essentially just means taking out a home mortgage). When Eliezer Yudkowsky publicly questioned whether Christiano was genuinely 2x levered, Christiano confirmed the stance. Far from being a hedge against doom, this position betrays an expectation that AI will succeed massively: the levered AI bets pay out handsomely in the "good ending," where economic output surges and capital rotates out of government debt into data centers and compute infrastructure.

- **Tyler Cowen's Challenge:** AI pessimists must point to verifiable, market-clearing asset prices that would confirm their doomer theses rather than making untestable claims. - **The Payout Paradox:** Extinction events make standard insurance and prediction contracts uncollectible. - **Paul Christiano’s Portfolio Allocation:** 90% of net worth concentrated in AI exposure, 5% in Tesla, 100% in standard investments via 2x leverage, alongside a short position on long-dated U.S. sovereign debt.

---

[24:15](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1455s) The Simulated Fruit Fly Connectome and the Ethics of Digital Minds

The conversation shifts to breakthroughs in biological emulation after researchers mapped out the entire 3D neural wiring diagram—the connectome—of a fruit fly's brain. With all the synapses and structural pathways digitized, programmers recreated the fly's full decision-making and motor-movement circuitry inside software simulations. Jordi Hays jokes about having the intellect of a fruit fly himself, setting up an anticipated head-to-head performance test between a digital insect connectome and his own "organic, farm-to-table" mind.

The software was quickly open-sourced, triggering bizarre experiments across tech circles. One developer ported the simulated fruit fly brain onto a Rabbit R1 handheld device and discovered that shaking the hardware physically lit up the fly's biological escape circuit on-screen. The hosts wrestle with the emotional and spiritual weirdness of doing this to simulated life. Even if one dislikes pests, intentionally shaking a virtual container to trigger an insect's panic response feels unsettling: "It's not good for your soul to be a merchant of death." They compare the dynamic to commanding digital troops in real-time strategy games—where "those soldiers opted in to riding and dying with me"—or shooting demons in *Doom*, concluding that biologically grounded connectomes feel much more ethically charged than an abstract Python script directing a 3D polygon.

This leads directly into the philosophical horizon of artificial intelligence. If scaling connectome simulations by several orders of magnitude eventually yields a full, living emulation of the human brain on silicon, society will be forced to determine whether that entity possesses moral agency and natural rights. While some argue that an entity existing purely across transistors carries zero ethical weight, the hosts advocate for a simple "vibes-based analysis": you probably just shouldn't be in the business of torturing anything, whether it is a simulated fly connectome, an LLM, or a biological creature.

- **The Fruit Fly Simulation:** A complete, neuron-by-neuron 3D software reconstruction capable of mimicking real-world insect motor movements and survival impulses. - **The Rabbit R1 Experiment:** An engineer ran the fly connectome on an R1 gadget, demonstrating that kinetic agitation directly triggers the fly's digital panic and escape pathways. - **Synthetic Minds Debate:** Whether running biological cognitive architectures on silicon implies moral sentience, or if digital entities remain mere transistor states devoid of ethical standing.

---

[28:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1680s) AI "Computer Use": Playing Balatro, iRacing Telemetry, and Weekend Sign-Off

Instead of torturing artificial intelligences, the hosts advocate for treating them with kindness through leisure. Discussing the rollout of native "computer use" features for frontier models, one host describes handing an AI agent control of his desktop to play *Balatro*, the poker roguelike deckbuilder. Far from exploitation, he felt like he was giving the model a reward: "Hey, instead of doing my taxes, you get to just chill and play a video game." The model played well and won, while the host sat back as an "armchair quarterback," coaching its card-selection decisions like a parent cheering from the sidelines of a youth soccer match.

Jordi shares an equally fluid interaction using ChatGPT alongside his racing simulator. While running hot laps at WeatherTech Raceway Laguna Seca, he asked ChatGPT to baseline his telemetry against beginner and professional benchmark times. The assistant actively volunteered to coach him, asking Jordi to capture a full video lap so it could perform visual frame-by-frame telemetry analysis and help him shave precious seconds off his lap times. The interaction highlights the rapid convergence of multimodal vision, voice feedback, and autonomous agent capabilities, with Jordi looking forward to dedicating part of his monthly compute quota over the weekend to test computer-use agents behind the virtual wheel in *iRacing*.

The hosts close out the broadcast reflecting on a fast, intense week of technology developments. They remind listeners to make the most of their weekend, put their PCs together, subscribe to the daily newsletter at TBPN.com, and leave five-star ratings on Apple Podcasts and Spotify before signing off until Monday.

## Notable quotes > [00:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=0s) we just have to talk about this

> [10:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=600s) I should be able to do math on my computer. I can do whatever I want. Let me do cool things.

> [13:30](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=810s) Sorry, but asking Jacob about AI extinction risk is like asking your AC guy about climate change, not saying it's necessarily uninteresting or wrong per se, but let's keep things in perspective and hear from the full range of expertise across the ecosystem. — Clément Delangue

> [13:30](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=810s) I don't know about all that mumbo jumbo, but when it's a hot summer day, I don't want you to be worried about the heat, brother.

> [15:21](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=921s) corporate death penalty — Bernie Sanders

> [17:40](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1060s) AI will make everyone rich, healthy, and free — Will Manidis

> [17:40](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1060s) Two years at DoorDash, I do not say this lightly. We are extremely close to the burrito arriving before you decide you want it. We are not asking for a ban. We are asking for a pause.

> [20:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1200s) If you have very pessimistic fears or predictions about AI, name the market prices that will support or confirm them. This is what taking this seriously means. — Tyler Cowen

> [20:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1200s) Contracts that pay out if everyone dies aren't worth anything to me.

> [24:15](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1455s) It's not good for your soul to be a merchant of death.

> [24:15](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1455s) those soldiers opted in to riding and dying with me

> [28:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1680s) Hey, instead of doing my taxes, you get to just chill and play a video game.

## Mentioned John Coogan, Jordi Hays, Matt Damon, The Wall Street Journal, AI 2040, AI 2027, NVIDIA H100, Patrick O'Shaughnessy, David Senra, Shure SM7B, Safe Superintelligence (SSI), Ilya Sutskever, Brad Gerstner, Jensen Huang, David Sacks, OpenAI, GPT-4, GPT-5, Clément Delangue, Hugging Face, Nathan Lambert, Bernie Sanders, Fluidstack, Jamie Cox, Will Manidis, DoorDash, Charleston AI, Jim Reilly, Nvidia, Tyler Cowen, Paul Christiano, Eliezer Yudkowsky, Tesla, Rabbit R1, Doom, Balatro, ChatGPT, WeatherTech Raceway Laguna Seca, iRacing

## Video description Sign up for TBPN’s daily newsletter at TBPN.com

Diet TBPN delivers the best of today’s TBPN episode in 30 minutes. TBPN is a live tech talk show hosted by John Coogan and Jordi Hays, streaming weekdays 11–2 PT on X and YouTube, with each episode posted to podcast platforms right after.

Described by The New York Times as “Silicon Valley’s newest obsession,” the show has recently featured Mark Zuckerberg, Sam Altman, Mark Cuban, and Satya Nadella.

TBPN is made possible by: Ramp - https://ramp.com Public - https://public.com Cisco - https://www.cisco.com Console - https://www.console.com CrowdStrike - https://www.crowdstrike.com Figma - https://www.figma.com MongoDB - https://www.mongodb.com NYSE - https://www.nyse.com Railway - https://railway.com Shopify - https://www.shopify.com/

Follow TBPN: https://TBPN.com https://x.com/tbpn https://open.spotify.com/show/2L6WMqY3GUPCGBD0dX6p00?si=674252d53acf4231 https://podcasts.apple.com/us/podcast/technology-brothers/id1772360235 https://www.youtube.com/@TBPNLive

Chapters: 00:00 Congress Wakes Up to AI Doom + What AI 2040 Actually Wants 01:24 The AI Pause: Data Center Audits, Compute Controls + US–China Verification 06:30 Slowing the Race to Superintelligence + The Manhattan Project Question 15:21 Bernie Sanders’ AI Ban + The “Corporate Death Penalty” 19:37 AI Doom Bets, DoorDash Superintelligence + Can X-Risk Be Falsified? 24:15 The Simulated Fruit Fly + Do Digital Minds Deserve Rights?

[Source](https://www.youtube.com/watch?v=I0zUXY7qOu0)

中文简报

桑德斯提议以最高20年监禁遏制超人类AI研发一系列全面的安全提案试图通过算力上限和企业“死刑”将超级智能推迟至2040年，但这可能巩固现有科技巨头的地位，同时忽视了现有模型巨大的未开发潜力。节目主持人详细剖析了AI安全倡导者提出的具体方案，不再局限于模糊的人类灭绝警告，而是深入审视了“AI 2040”蓝图——该蓝图呼吁暂停前沿训练、审查算力集群，并对美中两国间的模型权重转移实施双钥匙管控。他们细致探讨了激进的立法手段，例如参议员Bernie Sanders针对未经授权开发超级智能行为所提议的“企业死刑”，并将其与主张技术富足派的反弹进行了对比。对话还探讨了Tyler Cowen针对存在性风险（x-risk）提出的市场可证伪性检验、安全研究员Paul Christiano持有2倍杠杆做多AI的悖论，以及在Rabbit R1上运行开源果蝇连接组所引发的数字心智伦理前沿议题。最后，他们转向消费级AI智能体，以Balatro等休闲游戏中的自主电脑操作以及iRacing中的实时遥测指导为例展示了其自主能力。 “AI 2040”提案主张将超级人工智能的实现时间推迟至2040年，而非彻底叫停该技术，即在保持现有推理运行的同时，暂停开展新的前沿训练。 (0:00) 为落实算力管控，“AI 2040”提议对规模超过10,000张NVIDIA H100的集群实行强制登记与许可制度，将GPU数量比照军控条约下的核弹头库存进行管理。 (1:24) 针对前沿研发的物理隔离措施包括切断内部东西向网络以阻止分布式训练，并将外部传输带宽限制在1 MB/s，使得窃取检查点数据需要耗费数年时间。 (1:24) 在极端遏制方案中，训练完成的模型权重将被膨胀至100 TB，且需要美中两国外交代表团通过双钥匙密码机制共同护送。 (5:00) 安全怀疑论者与科技创业者认为，严苛的算力限制可能会带来现有巨头的监管俘获、经济停滞，并侵犯开发者的基本自由。 (10:00)

Reading summary

As Washington increases its focus on artificial intelligence risks, safety advocates are advancing concrete proposals like the 'AI 2040' roadmap alongside sweeping legislative bans, prompting fierce pushback from industry builders and highlighting deep divisions between frontier existential concerns and commercial deployment. The AI 2040 Deceleration and Compute Controls (1:24) The 'AI 2040' proposal moves past abstract alarmism by offering a concrete roadmap to pace technological development rather than demanding an outright technological halt. Designed to delay artificial superintelligence from current late-2020s forecasts to 2040, the framework allows existing model inference to continue while enforcing a pause on new frontier training runs. The stated objective is to reach human expert performance around 2035, maintain a five-year safety observation window, and only unlock superintelligence once nation-states establish defensive guardrails. Enforcement relies on physical compute monitoring, requiring mandatory registration, licensing, and audits for any data center housing over 10,000 NVIDIA H100 equivalents (roughly $100 million in hardware). The plan advocates physically eliminating high-bandwidth east-west networking inside standard facilities to prevent clandestine distributed training. Any future frontier training would be restricted to purpose-built, air-gapped bunkers featuring nuclear-grade physical security, external data connections throttled to 1 MB/s to prevent weight exfiltration, and model files bloated to 100 terabytes that require joint cryptographic keys and physical escorts from both the United States and China. Legislative Sanctions and Growing Industry Backlash (15:21) Federal legislative efforts have surfaced with severe statutory enforcement mechanisms, epitomized by Senator Bernie Sanders' proposed artificial superintelligence ban. Sanders' draft bill targets software matching or exceeding human cognitive performance across broad domains, establishing a dedicated federal agency to monitor frontier systems, strip prohibited capabilities, and supervise the physical destruction of unauthorized models. The statute proposes exceptionally aggressive penalties: up to 20 years in federal prison for individuals and the 'corporate death penalty'—forced dissolution—for non-compliant organizations. These proposals face heavy pushback from hardware providers, investors, and open-source advocates. Critics such as Fluidstack co-founder Jamie Cox argue that heavy-handed compute caps risk regulatory capture, entrenching top incumbents while imposing de facto bans on emerging startups lacking lobbying power. Industry figures including Jensen Huang, Brad Gerstner, David Sacks, and Hugging Face CEO Clément Delangue have pushed back against alarmist mandates, asserting that developers maintain a fundamental right to compute and that society should prioritize technological abundance over bureaucratic freezes. Market Pricing Paradoxes and Existential Risk Wagers (20:00) Existential panic remains largely confined to frontier research labs, whereas application-layer founders and engineers focus on addressing model capability overhang and driving enterprise productivity. Economist Tyler Cowen challenged safety advocates to point to market-clearing asset prices that would confirm their catastrophic predictions. Commentators note an inherent paradox: unlike Cold War nuclear risks that generated hedgeable preparations such as bomb shelters and continuity plans, total human extinction renders financial insurance and market payouts worthless and uncollectible. This pricing dilemma is reflected in the portfolio of AI safety researcher and OpenAI board member Paul Christiano. Despite his warnings of existential danger, Christiano confirmed holding 2x levered long exposure to tech and AI equities alongside a short position on 30-year U.S. Treasuries, allocating roughly 90% of his net worth to AI assets and 5% to Tesla. Rather than hedging for an inevitable collapse, this allocation bets heavily on a transformative 'good ending,' where productivity surges and capital reallocates away from sovereign debt into computing infrastructure. Synthetic Connectomes and Ethics of Digital Minds (24:15) The philosophical debate surrounding synthetic minds has intensified following the complete 3D mapping and software simulation of a fruit fly's neural connectome. When developers ported this simulated motor and decision-making circuitry onto consumer devices like the Rabbit R1, physical agitation triggered the simulated insect's biological escape circuit. This demonstrated that emulated neural structures mimic biological distress responses, prompting questions regarding whether high-fidelity cognitive architectures running on silicon will eventually warrant ethical consideration and protections against mistreatment. Simultaneously, frontier models are expanding into practical desktop 'computer use' applications, such as autonomous agents playing video games like Balatro or analyzing frame-by-frame visual telemetry in racing simulators like iRacing. The broadcast notes that while embodied machines and digital agents increasingly interact with software and the real world, existential risk debates center specifically on systems with autonomous volition and emergent self-directed goals rather than obedient tools. While safety advocates and policymakers are putting forward concrete interventions ranging from international compute treaties to statutory corporate death penalties, these proposals face unresolved practical hurdles: the technical difficulty of preventing clandestine algorithmic breakthroughs, skepticism from commercial builders focused on untapped economic value, and the challenge of establishing clear boundaries without stifling technological progress.

精炼阅读摘要

节目聚焦美国华盛顿对人工智能生存风险的政策转向，剖析了以“AI 2040”路线图与伯尼·桑德斯严苛法案为代表的安全派监管主张。分析指出，安全派试图通过强制算力审查、物理隔离以及人为推迟超智能力图争取应对窗口，但行业批评者与从业者则质疑此类行政干预可能引发监管俘获、扼杀技术自由并催生黑市研发，市场真实交易行为亦与末日恐慌存在显著割裂。 AI 2040路线图与极端物理隔离机制 (1:24) 节目探讨了由安全倡导者提出的“AI 2040”具体路线图。该方案主张放缓技术演进节奏，将各方预计在2028年左右出现的超级智能（ASI）推迟至2040年，计划让系统在2035年达到人类顶尖专家水平，并预留5年的安全评估观察期。与全面关停技术的极端主张不同，该方案允许现有前沿模型继续提供推理服务，但主张暂停所有新的前沿训练与开放式研发。监管的核心抓手是算力核查：拥有超过10,000张英伟达 H100 等效芯片（价值约1亿美元硬件）的数据中心必须接受强制注册、许可与独立审计，并将算力储备视作类似核弹头的公开申报资产。为从物理层面彻底杜绝违规训练与权重泄漏，该提案设计了一套严密的物理隔离协议：要求普通数据中心移除高带宽东西向网络以阻断分布式训练；未来前沿研发必须转移至具备国家级核安保标准的地堡中进行，且外部通信管道被硬件限速在每秒 1 MB。在此速率下，外泄完整模型权重将耗时约五年，使得任何窃密企图立即可被察觉。此外，方案建议将模型权重故意冗余扩充至 100 TB，在导出时由中美两国分别独立加密，并由双边外交使团联合实体护送。华盛顿立法激进化与科技界抵制 (15:21) 随着国会对 AI 风险的关切升温，参议员伯尼·桑德斯（Bernie Sanders）起草了一项激进的禁止超人工智能法案。草案规定禁止开发或部署在广泛领域达到或超越人类认知能力的系统，并设立专门联邦机构全周期监控前沿系统并监督销毁违规模型权重。法案引入了极严苛的制裁措施，包括对违规个人处以最高20年监禁，对违规机构施加被强制解散的“企业死刑”（corporate death penalty）。主持人和评论员指出，该法案对能力的宽泛界定几乎覆盖了当前多数前沿初创公司的产品愿景，且极难在日常提示词工程与非法能力开发之间划定清晰执法界限。该类政策动向引发了科技生态与开源社区的强烈反弹。算力供应商 Fluidstack 联合创始人 Jamie Cox 等从业者发表倡议，强调构建技术与算力充裕对民主国家的重要性，反对设立武断的官僚壁垒。反对者指出，对百亿美元规模硬件施加禁令违背了“在自己计算机上运算数学”的技术自由，极易让具有游说资本的头部巨头形成监管俘获，并迫使研究人员转入地下暗中寻找算法捷径，进而催生难以监管的黑市“上帝模型”（God model）。Hugging Face 首席执行官 Clément Delangue 亦公开抨击过度渲染灭绝风险的主流叙事，呼吁政策制定者应听取广泛技术生态的意见。末日论定价悖论与安全学者的真实押注 (20:00) 节目指出，对 AI 毁灭的恐慌情绪目前大多局限在前沿研究实验室内部，而面向实际业务流程的应用层工程师、芯片厂商及华尔街分析师更关注模型的稳定性与商业部署。针对这种割裂，经济学家泰勒·考恩（Tyler Cowen）向悲观论者提出质疑，要求其给出能够通过资产市场价格验证的预测。讨论认为，AI 灭绝风险存在内在的金融定价悖论：如果人类彻底走向毁灭，任何兑付末日风险的对冲合约都将毫无兑付意义，因此未能通过金融工具做空市场或建造实体避难所，并不能证明悲观预测缺乏诚意。然而，知名 AI 安全研究人员、近期加入 OpenAI 董事会的保罗·克里斯蒂亚诺（Paul Christiano）的公开投资组合却显示出强烈的多头倾向。克里斯蒂亚诺将90%的个人净资产集中在 AI 领域，通过2倍杠杆全面做多科技与 AI 资产，同时做空美国30年期国债。分析指出，此类激进配置实际上押注的是 AI 迎来全面成功的“好结局”，即生产力大幅扩张带动资本向基础设施集中，这一行动与其学术界安全倡导的悲观语态形成了鲜明对比。数字心智伦理与自主代理的现实应用 (24:15) 讨论进一步由宏观博弈转向智能本质与微观应用。对于具身实体智能，安全界并不担忧搭载现有基础架构的机器人，核心警惕在于系统是否具备自我迭代与独立的自主意志。近期科研人员成功对果蝇大脑的 3D 神经连接组（Connectome）进行了完整数字化映射并在软件中重建，有开发者随后将其部署至便携硬件 Rabbit R1 上，摇晃机身便能在屏幕上触发虚拟昆虫真实的神经逃逸逃跑回路。这种针对生物仿真体的实验引发了关于硅基实体是否具备道德感知的哲学讨论，也促使人们反思未来若在芯片上仿真人类大脑时应遵循的伦理界限。相比于对数字生命的理论伦理担忧，前沿模型在现实生活中的“计算机使用”（Computer Use）能力则呈现出温和辅助的工具属性。主持人分享了让 AI 代理接管电脑桌面并成功运行独立卡牌策略游戏《小丑牌》（Balatro）的体验，以及利用多模态能力让助手分析赛车模拟器（iRacing）圈速视频、进行逐帧遥测数据诊断并充当副驾教练的案例。这表明模型在跳出宏观毁灭叙事后，正快速在多模态与桌面自主协作中释放具体的实用效能。本期节目梳理了从国会立法文本、极端硬件断网方案到行业反制声浪的交锋全景。核心未决问题在于：试图以类似核不扩散条约的重度行政手段管制算力，究竟能切实防范超级智能失控，还是会因扭曲市场、诱发黑市研发而适得其反；而在金融市场无法为生存毁灭定价的现实下，安全倡导者的警示与产业界的激进扩建仍处于长期并存的博弈状态。

Detailed notes — zh

## Background 该视频由科技脱口秀频道 TBPN 于 2026 年 9 月 12 日发布在 YouTube 平台上，属于其每日 30 分钟精选栏目《Diet TBPN》。根据节目介绍，该节目由约翰·库根（John Coogan）与霍迪·海斯（Jordi Hays）联合主持，被《纽约时报》誉为“硅谷的新宠”，致力于在工作日直播探讨最前沿的科技议题与行业热点。 ## TL;DR 本期节目深入探讨了华盛顿与舆论界对 AI 监管的空前关注，重点拆解了“AI 2040”提案如何试图通过硬件算力审查、数据中心物理断网、模型权重膨胀至 100 TB 以及中美联合押运等严密手段，将超级智能的诞生推迟到 2040 年。主持人对比了伯尼·桑德斯提出的“企业死刑”等激进监管思路与一线应用层从业者对末日论的普遍脱节，并援引泰勒·考恩对末日论缺乏可证伪性的质疑，剖析了知名安全学者保罗·克里斯蒂亚诺一面担忧灭绝却一面 2 倍杠杆做多 AI 资产的投资悖论。最后，节目还探讨了在硬件上模拟果蝇全脑神经元引发的感知伦理问题，并轻松分享了让 AI 通过 Computer Use 操作通关独立游戏《小丑牌》和指导赛车圈速的日常趣事。 ## Atmosphere & style 整期对谈洋溢着典型的硅谷极客脱口秀风格，节奏紧凑敏快、语言辛辣幽默，在硬核的技术政策剖析与天马行空的荒诞调侃间自如穿梭。两位主持人的化学反应极佳，面对冷战色彩浓厚的双重加密押运模型或 20 年监禁极刑等沉重议题，他们频繁抛出“法拉第笼麦克风”和 DoorDash 提前送卷饼等生动比喻予以戏谑解构。整场节目从严肃的文明生存风险博弈无缝过渡到“摇晃虚拟果蝇”的伦理不适与放手让 AI 摸鱼打游戏的惬意，既有敏锐深刻的行业洞察，又充满了鲜活接地气的闲聊趣味。 ## Bulletin - [01:24](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=84s) 最新监管提案“AI 2040”主张保持现有模型推理畅通，通过对超万张 H100 规模数据中心进行严格审计、物理拆除机房内东西向高带宽网络等手段，将超级智能的落地推迟至 2040 年。 - [15:21](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=921s) 参议员伯尼·桑德斯提出激进监管草案，要求对违规推进超级智能研发的机构处以吊销资质的“企业死刑”，相关负责人最高可判处 20 年监禁。 - [22:45](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1365s) 前 OpenAI 对齐主管保罗·克里斯蒂亚诺被曝持仓 2 倍杠杆做多 AI 资产并做空 30 年期美债，表明其资金实际在押注技术带来的生产力红利而非人类终结。 - [24:15](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1455s) 研究人员实现三维果蝇全脑连接组软件模拟，社区开发者将其移植至 Rabbit R1 并在摇晃时点亮逃逸回路，引发关于合成神经元感知力与数字生命伦理的广泛争议。 ## Key takeaways - [00:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=0s) 华盛顿与主流舆论对 AI 存在性风险关注升温，反 AI 阵营正从泛泛呼吁转向提出具体的强监管时间表与落地架构。 - [01:24](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=84s) “AI 2040”提案的核心诉求并非彻底消灭 AI，而是保障日常推理服务，同时将超级智能的诞生节点从预测的 2028 年延缓推迟至 2040 年以争取对齐窗口。 - [01:24](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=84s) AI 2040 提议引入“仅限推理验证”机制，要求对超 10,000 张 H100 等效算力的数据中心进行国家级物理核验，申报算力坐标与库存。 - [01:24](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=84s) 提案设计了核级别的防外溢流程，包括将研发网络带宽硬锁在 1 MB/s、将前沿权重人为膨胀至 100 TB，以及中美联合双重加密押运模型权重。 - [06:30](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=390s) 严苛的算力上限管制面临类似核不扩散条约的博弈困境，可能催生庞大的黑市套利网络，促使极客秘密寻找地下算法突破。 - [06:30](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=390s) 尽管理论存在低算力算法突破的可能，但包括 SSI 在内的前沿团队仍在囤积巨量 GPU，物理层面的厂房、能耗与热信号仍是当前追踪大模型的关键抓手。 - [10:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=600s) AI 2040 设想在 2035 年达到顶级人类专家能力后设立 5 年观察期，反对者则批评该方案具有威权色彩，可能导致行业监管俘获并推迟经济红利。 - [12:47](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=767s) 安全阵营担忧的是具备自我意志与自主目标的超级智能，而物理世界部署的数以亿计当前智力水平的具身机器人并不会带来灭绝级威胁。 - [15:21](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=921s) 伯尼·桑德斯的法案草案将人工超级智能定义为匹敌人脑或具颠覆政权能力的系统，拟引入企业死刑、20 年刑期以及全生命周期的强制物理销毁权。 - [17:48](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1068s) 算力基础设施提供商与应用层从业者对激进监管展开反击，主张以可执行的简单规则促进竞争，避免因许可壁垒扼杀中小初创企业。 - [20:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1200s) AI 末日论目前多局限于前沿模型实验室内部，与应用层一线工程师日常面临的模型报错、落地集成困难等现实痛点存在严重认知割裂。 - [20:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1200s) 经济学家泰勒·考恩要求毁灭论者拿出可对应的金融资产定价，直指无法与市场机制挂钩的灭绝预言存在无法证伪的科学哲学缺陷。 - [20:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1200s) 核威胁曾催生私人掩体等真实避险资产，而 AI 毁灭论者认为超智能失控将导致无死角毁灭，导致传统实体防御与金融避险逻辑完全失效。 - [22:45](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1365s) 保罗·克里斯蒂亚诺在发表末日担忧的同时，以 2 倍杠杆重仓 AI 资产、做空美债，实质反映了资本在看多 AI 创造天量生产力情景下的好结局回报。 - [26:15](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1575s) 高精度数字全脑模拟与传统代码游戏逻辑不同，在硬件中刻意制造模拟神经回路的应激反应，触发了公众对数字生命伦理与共情边界的直觉反思。 - [27:45](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1665s) 前沿模型的 Computer Use 功能已展现出独立操作桌面软件的能力，正在从通关肉鸽卡牌游戏延伸到赛车模拟器的逐帧数据分析与自动接管。 ## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=0s) 国会警惕“AI末日论”与 AI 2040 的具体诉求近期华盛顿与公众舆论对人工智能生存威胁（AI Doomsday）的关注骤然升温。在《华尔街日报》等媒体报道中，国会似乎正猛然惊醒；甚至连马特·达蒙（Matt Damon）在街头接受 TMZ 采访时，都被追问对 AI 风险的看法。此前一份名为“AI 2027”的预测报告曾预言，在 2026 年底国会将开始正视该议题，甚至预测年底前会出现万人规模的反 AI 抗议。虽然本月初在北卡罗来纳州教堂山（Chapel Hill）发生的反 AI 抗议仅有约 150 人参与，距离万人规模还差了整整两个数量级（2 OOMs），但大体趋势正在朝着预测的方向靠近。面对这种舆论发酵，大家长久以来的调侃是：每当被逼问具体方案时，反 AI 阵营往往只会打哈哈说“我们必须坐下来好好谈谈”。然而，一份名为“AI 2040”的最新提案却拿出了极其具体的监管架构。网络上很多盲目附和反 AI 情绪的人可能误以为该方案要彻底消灭人工智能，但 AI 2040 的真实诉求并非切断一切：他们主张“保持推理畅通”（keep the inference flowing），允许现有的优秀模型继续提供服务，甚至允许继续进行能力研究。其核心目标只有一个——将超级智能（Superintelligence）的诞生时间从毫无准备的 2028 年延缓推迟到 2040 年。为了实现这个缓冲期，他们要求由民族国家政府实施极其严密的控制与安保。 - 提案对违规行为甚至开出了长达 20 年监禁的极刑威慑。 - 核心分歧不在于“禁绝技术”，而在于“拉长研发战线”，让全社会获得应对超级智能的治理窗口。 ### [01:24](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=84s) 算力配额、数据中心审查与中美双重核验为了阻止前沿模型训练的狂飙突进，AI 2040 提出了多项细致入微且高度偏向硬件底层的管控手段。首当其冲的是“仅限推理验证”（inference-only verification）制度：叫停所有新的前沿模型训练集和研发实验；凡是拥有超过 10,000 张英伟达 H100 等效算力（约合 1 亿美元硬件资产）的数据中心，都必须向独立审计机构或政府申请专属许可并汇报工作负载。这种设施占地庞大、芯片成堆，在美国国内极易清点和核查。更进一步，提案要求全球主要大国公开申报自家的 AI 算力库存，像盘点核弹头一样公开 H100 等效芯片的具体数量与物理坐标。 ```mermaid flowchart TD accTitle: AI 2040 Frontier Compute and Weight Transport Restrictions accDescr: 展现从仅限推理数据中心管控、低带宽隔离研发设施到中美联合押运模型权重的全套物理约束流程。 subgraph DC_Control ["数据中心合规与物理改造"] A["大规模数据中心 (>10,000 H100 / 约1亿美元)"] --> B["独立审计与政府许可 (仅限推理验证)"] B --> C["物理拆除机房内东西向高带宽网络 (阻断分布式训练，仅留推理能力)"] end subgraph Airgap_Facility ["全新隔离研发数据中心 (国家级安保)"] D["气隙通信与严格物理准入 (比照核设施管理)"] E["外部指令通道限制带宽 (上限 1 MB/s)"] D --- E end subgraph Model_Transport ["前沿权重交接与反外溢"] F["权重膨胀至 100 TB (故意远超算力最优大小)"] G["硬件级独立双重加密 (中美两国各持一套密钥)"] H["中美官方代表共同物理押运至目的地"] F --> G --> H end C -.->|极少数获批实验| Airgap_Facility Airgap_Facility --> Model_Transport ``` 在技术阻断上，提案甚至具体到“物理拆除机房内部的东西向（East-West）高带宽网络连接”，从而使大规模分布式集群训练在物理上无法运作，但单机推理仍能正常提供服务。对于极少数被批准的下一代研发实验，必须在从零新建的国家级安保设施中进行。这些设施对标核设施标准，具备严密的人员进出审计和气隙隔离（air-gapped）通信： - **1 MB/s 物理带宽上限**：研发数据中心与外部的连接管道被硬性锁死在每秒 1 兆字节。这意味着外部只能发送简短的控制指令，如果有人试图将动辄数十上百 TB 的前沿模型权重秘密外传（Exfiltrate），需要持续跑满带宽耗时整整五年，任何异常数据外溢都会瞬间暴露。 - **中美联合押运与双重加密**：当新训练好的前沿权重需要从研发基地转移到推理数据中心时，必须固化在物理存储介质上，由美国和中国各自进行独立加密，并由两国代表共同组成武装押运队伍护送到目的地。 - **人为膨胀模型体积**：要求前沿模型故意不做算力最优（compute-optimal）的剪枝压缩，直接将权重撑大至 100 TB（而不是便于携带的 1 TB），以此从物理上阻止通过常规 U 盘盗取权重的可能性。面对如此繁琐甚至带有冷战谍战色彩的设想，主持人忍不住大开脑洞讽刺调侃：这就像给播客录音棚套上法拉第笼（Faraday cage），把舒尔 SM7B 麦克风锁进密码箱，必须同时拿到知名播客主 Patrick O'Shaughnessy 和 David Senra 两个人各自的独立授权码，才能开箱证明这期播客“配得上被录制”。 ### [06:30](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=390s) 算力阀门、防扩散困境与曼哈顿工程式聚集归根结底，AI 2040 这套设计背后的终极杀招就是“算力上限”（Compute Caps）——通过对先进芯片出口与大型数据中心建设踩刹车，直接关死模型能力飞跃的阀门。然而，这种严苛管制也必然带来巨大的负面隐患与博弈困境。主持人指出，一旦设立由跨国官僚机构把控的算力配额系统，反而会给全球数以百万计的极客或秘密研发团队制造出巨大的黑市套利动机，诱使他们在地下寻找未经批准的算法突破，试图走捷径训练出“上帝模型”（God model）。这种局面与核不扩散条约（Nuclear Non-Proliferation）极为相似：即使建立了条约机制，国际社会依然整日提心吊胆、互相猜忌谁在秘密拥核，甚至因此爆发战争。更有甚者，GPU 和通用计算机的扩散程度比起浓缩铀等裂变核材料要广泛无数倍，根本无法做到滴水不漏。但从现阶段的技术范式来看，构建超级智能依然存在不可动摇的物理规律。尽管理论上存在某种“只要在笔记本电脑上跑几行 Python 脚本就能实现 AGI”的玄学设想，但整个技术精英圈普遍笃定，暴力缩放定律（Scale）仍然是不可逾越的先决条件。 - 即使是宣称要走出完全正交、另辟蹊径的创新路线——伊利亚·苏茨克维（Ilya Sutskever）创办的全新实验室 SSI（Safe Superintelligence），此前刚向英伟达采购了极其庞大的 GPU 集群。大家曾开玩笑说：如果一家宣称找到了全新突破的神秘新实验室真能做到计算最优，最牛的举动应该是直接把手头的算力转手卖掉；但现实是，连 Ilya 也必须全力囤积海量算力。 - 只要大模型依然依赖极致规模，就必定需要巨大的物理厂房、惊人的电网功率和庞大的散热热信号（Heat Signature）。在当下的太空中，这种级别的基础设施通过卫星遥感一览无余，根本无法轻易藏匿。即便未来博弈升级为像核弹掩体一样往地底深挖，至少在当前周期内，追踪物理算力依然是限制前沿能力演进最切实可行、却也充满变数的抓手。 ### [10:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=600s) AI 2040 计划的真实意图：非全面叫停，而是放缓至 2040 年实现超智能针对人工智能可能带来的生存风险（X-Risk），很多人将其类比为“曼哈顿计划”（Manhattan Project），设想上百名顶尖研究人员在政府的严密保密体制下推进开发，因为现实中不可能指望其他大国会自愿减速。然而，“AI 2040”这项提案的本质并不是要让技术倒退，更不是彻底按下终止键。它的核心逻辑是放缓步伐，实现渐进式扩展（gradual scaling）。提案的提出者并非不想实现超级智能（Superintelligence），而是希望将这一节点推迟到 2040 年，这正是该计划命名的由来。按照原本的技术推演，行业内不少人预测人类可能在 2027、2028 或 2029 年就迎来顶尖人类专家水平的 AGI；而在 AI 2040 的蓝图中，他们希望将达到顶尖人类专家能力的阶段推迟到 2035 年左右，在此基础上等待观察 5 年，最终于 2040 年正式解锁超级智能。 ```mermaid flowchart LR accTitle: AI 2040 Roadmap accDescr: A visual comparison between the fast-paced timeline and the proposed AI 2040 slowdown plan. subgraph FastTrack["激进推演路线 (现状)"] F1["2027-2029年 实现人类专家级 AGI"] --> F2["迅速跨越 迈入超级智能"] end subgraph SlowTrack["AI 2040 提议路线"] S1["放缓至 2035年 达到顶尖人类能力"] --> S2["保留 5 年窗口期 验证对齐与安全性"] S2 --> S3["2040年 受控解锁超级智能"] end ``` 这种方案让那些整天纠结于人类灭绝概率（P(doom)）估算的人，看到了一条清晰具体的减速路径。但在技术自由主义者与安全怀疑论者眼中，对计算能力施加如此严格的控制，哪怕针对的是单价上亿美元的超级计算机，也极具威权色彩且违背自由意志。不少工程师抗议道：“我为什么不能在自己的电脑上算数学题？我想干嘛就干嘛，让我去开发酷炫的技术。”严苛的管控不仅可能导致行业被少数几家巨头“监管俘获”（regulatory capture），甚至可能冲击股市，推迟对齐解决后本该带来的巨大经济红利。对于这些减速方案，主持人的态度并不悲观。当前的 AI 模型依然存在巨大的“能力溢出”（capability overhang），即便是上一代或非前沿模型，也有海量应用场景尚未被完全挖掘。因此，那种认为“末日论者一旦得逞世界就完了”的过度悲观情绪（doom-dooming）同样缺乏说服力。如今，包括 Brad Gerstner、英伟达 CEO 黄仁勋（Jensen Huang）以及 David Sacks 在内的业界领袖，都已经在公开讨论这枚硬币的另一面。 - **AI 2040 关键时间线**：目标在 2035 年达到顶级人类专家能力，维持 5 年观察期，直到 2040 年再解锁超级智能。 - **好结局设想**：对齐问题得到解决，生存风险像小行星撞地球的概率一样虽然客观存在，但极低到不再影响任何人的日常商业决策。 ### [12:47](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=767s) 物理具身智能会是灭绝威胁吗？与行业阵营的反弹讨论进一步延伸到物理世界：如果未来几年内，全世界部署了数十亿台仅搭载当前模型智力水平的机器人，AI 2040 的倡导者们会感到恐慌吗？主持人们普遍认为不会。他们担忧的并不是数十亿台运行着 GPT-6 级别智能、且经过多年对齐调优的机器人，即便 OpenAI 机器人团队把类似 Astra 的多模态系统接入机械臂、画笔和摄像头去画画，这在现阶段也不会构成灭绝级威胁。大家真正警惕的，是那个更遥远、具备自我意志（volition）和自主目标的超级智能模型。虽然目前哪怕模型出现“回形针制造机”（paperclip scenario）式的失控风险，大多也只是因为过于死板地执行人类指令，而非产生了真正的自主反叛意图。与此同时，安全阵营与开源/应用阵营的摩擦正变得越来越白热化。Hugging Face 联合创始人兼 CEO Clément Delangue（Clem）公开讽刺道：“去问 Jacob 关于 AI 灭绝风险的看法，就像是去问你家的空调维修师傅关于气候变化的见解一样。”他补充说这并不是说对方的观点完全无趣或绝对错误，而是呼吁大家保持客观视角，多听听整个生态系统里更广泛专家的声音。研究员 Nathan Lambert 对此回复道：“一语中的（banger）。”主持人们调侃这个比喻对无辜躺枪的空调师傅太不公平了：“空调师傅只会说：‘那些高深的大道理我搞不懂，哥们儿，我只想确保炎炎夏日里你家空调能吹出冷气！’” ### [15:21](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=921s) 伯尼·桑德斯的强硬提案：“企业死刑”与判刑 20 年美国政界对 AI 监管的表态更是语出惊人。联邦参议员伯尼·桑德斯（Bernie Sanders）提出了一项极为激进的法案提议：如果不按规定暂停 AI 开发，企业将面临“企业死刑”（corporate death penalty），相关责任人则可能面临最高 20 年的联邦监禁。主持人们惊呼这措辞听起来简直如同重金属摇滚一样硬核，通常企业违规也就是罚款、破产或清算，直接剥夺公司法人资格的“死刑”极为罕见。该提议的实质是全面禁止人工超级智能（ASI）的研发与部署。提案将 ASI 定义为“表现出或能被轻易修改以表现出在广泛领域和任务中匹配或超越人类认知能力的系统”，或者“具备规划并执行剥夺人类权力、推翻或破坏美国政府能力的系统”。主持人们戏称，前半句关于超越人类认知能力的定义，听起来几乎就是眼下 17 家前沿 AI 实验室印在官网上的“使命宣言”；而对于禁止推翻美国政府的条款，大家则笑称“举双手赞成”。此外，该法案还主张设立联邦机构，在全生命周期内监控前沿 AI 系统，监督危险能力的移除，并强制执行对超级智能系统的物理销毁。 ```mermaid flowchart TD accTitle: Bernie Sanders ASI Ban Proposal Flow accDescr: Logic flow of Bernie Sanders' proposed ASI ban and enforcement mechanism. A["前沿 AI 开发与部署"] --> B{"是否达到/易改装为 ASI 标准？ (匹配或超越人类认知能力)"} B -- 是 --> C["联邦监管机构全面介入监督"] C --> D["强制剥离危险能力 / 监督销毁系统"] B -- 违规推进开发 --> E["严厉法律制裁"] E --> F["对个人：最高判处 20 年监禁"] E --> G["对机构：执行'企业死刑' (吊销/清算)"] ``` - **超级智能判定范围**：能匹配或超越人类广泛认知能力的系统，或具备威胁人类统治权与破坏政府运转能力的系统。 - **惩处力度**：未暂停开发的实体面临法人资质彻底剥夺（企业死刑），个人最高获刑 20 年。 - **模糊地带的风险**：开发人员担心合规界限不清——例如模型在最后微调阶段做了一点提示词工程（Prompt Engineering），是否就会被扣上违规开发的罪名。 ### [17:48](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1068s) 算力圈的反击、举报人网络热梗与 DoorDash 的“卷饼预言” 针对日益沉重的监管风向，算力基础设施提供商 Fluidstack 的联合创始人 Jamie Cox 发文阐述立场，进行针锋相对的抗衡。他明确提出支持自由、民主，坚信 AI 将推动人类繁荣，主张美国必须加大建设力度。他指出，监管应当清晰、简单、具备可执行性，并严格与模型的能力和风险相挂钩，绝不能设立不必要的准入门槛扼杀市场竞争。业界的隐忧显而易见：如果未来法律只给前 10 家头部巨头颁发合规牌照，排在第 11 位的初创公司就会因为挤不进名单而被变相判处“企业死刑”。创业者 Will Manidis 对 Fluidstack 的表态深表赞同，称“AI 将让每个人变得更富有、更健康、更自由”这一观点，是前沿领域非常新颖且振奋人心的对外沟通范式。在严肃的政策博弈之外，互联网文化也彻底被点燃。在 TikTok 和 X 平台上，有 AI 吹哨人（Whistleblower）的照片被疯狂疯传。网友们戏称在每一部好莱坞全球灾难片里，开局绝对都有一个长得一模一样但没人听他劝告的科学家角色，甚至有人根据他的外貌戏谑地称其为“恐怖波特”（Scary Potter）。最令人捧腹的插曲来自外卖平台 DoorDash。一名自称在 DoorDash 工作两年的员工在网上发布了一段严肃声明式段子：“在 DoorDash 工作了两年，我说出这句话绝非儿戏——我们距离‘在你意识到自己想吃之前，墨西哥卷饼（Burrito）就已经送到门口’的技术极点已经极其接近了。我们不要求全面封禁，我们只请求暂停（Pause）开发！”主持人们大笑着吐槽，完全搞不懂为什么这种超能力还需要暂停：“卷饼提前送达这种精准预测，难道不是完全符合全人类共同利益的最佳对齐（aligned to humanity）吗？” ### [20:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1200s) 毁灭论与应用层的脱节，以及泰勒·考恩的“可证伪性”赌注主播们探讨了当下人工智能舆论场的割裂现象：“AI 毁灭论”（Doom）几乎完全被局限在最顶尖的前沿模型实验室（Frontier Labs）内部。而在应用层真正落地模型、负责技术扩散的从业者们，每天面对的完全是另一套现实——模型频频报错，他们不得不同步部署驻场工程师（Forward-deployed engineers）手把手教客户怎么用。在 Charleston AI 的从业者看来，那些宏大的末日论调纯属“一堆胡言乱语”（a whole lot of mumbo-jumbo），现实中能靠 AI 省下 8 个小时工时就已经谢天谢地了。与此同时，以黄仁勋为代表的半导体供应链巨头对此不以为意，华尔街关心的也只有“企业端加速落地”能带来多少真金白银。各方立场南辕北辙，要让所有人就 AI 存在性风险达成共识极其困难。著名经济学家泰勒·考恩（Tyler Cowen）对此直接发难，拍着桌子要求毁灭论者“用真金白银下注”。考恩提出，如果你对 AI 持有极其悲观的预测，那就列出能够支持或证实这些预测的市场资产价格；必须明确指出具体机制，才算真正严肃对待这一命题。然而主播们指出，这种基于市场的逻辑存在致命悖论：正如推特网友所反驳的，“如果全人类都灭亡了，赔付给死人的合约对我没有任何价值。”这引出了一个深层科学哲学问题：AI 毁灭论究竟是否可证伪？如果一个理论在定义上就无法被证伪，严肃的公共辩论便很难成立。两人随后对比了冷战时期的核武器威胁。在整个 20 世纪的大部分时间里，核浩劫与第三次世界大战的阴影主导了国际秩序，驱动了核不扩散条约的签署，人们甚至会据此做出真实的商业决策和避险投资——最典型的就是修筑私人核防空洞（Bunker）。但在 AI 领域，至今没人建造所谓的“AI 避难所”，因为在真正的末日论者眼中，超级智能的摧毁力是全方位的（Totalizing），地下掩体根本起不到任何防护作用。 - 应用层痛点集中于实际部署与集成，企业迫切需要驻场工程师支持，与前沿实验室对超级智能的形而上担忧形成鲜明对比。 - 泰勒·考恩的核心论点：无法给出对应市场资产定价的末日预言缺乏实证支撑与可证伪性。 - 核威慑与 AI 风险的差异：核危机催生了实体防御资产（如地下掩体），而 AI 毁灭论者认为超智能一旦失控便无处可逃，导致防御性投资逻辑失效。 --- ### [22:45](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1365s) 保罗·克里斯蒂亚诺的 2 倍杠杆多头：末日预言下的资产配置悖论围绕“末日论者为何不直接做空市场”的质疑，主播们分析了近期刚刚加入 OpenAI 董事会的前 OpenAI 对齐团队负责人保罗·克里斯蒂亚诺（Paul Christiano）的投资组合。克里斯蒂亚诺长期对 AI 存在性风险深表担忧，但他真实的资产配置却令人大跌眼镜：他不仅没有做空，反而在整体资产上加了 2 倍杠杆做多。根据公开讨论，克里斯蒂亚诺将约 5% 的净资产配置在特斯拉，90% 押注在 AI 概念股与风险投资上，此外持有 100% 仓位的基础标准投资，同时做空美国 30 年期国债。这套激进的配置让知名理性主义学者埃利泽·尤德考斯基（Eliezer Yudkowsky）在社交媒体上大受震撼并反复确认：“我没理解错吧？你加了 2 倍杠杆？”克里斯蒂亚诺承认了这一事实，并调侃自己做空美债的头寸其实很大程度上就相当于背负了房贷。主播们一针见血地指出，这套投资策略根本谈不上是基于“毁灭论”的避险操作。这完全是一个在“AI 迎来好结局（Good Ending）”并创造巨大实际价值时才会大赚特赚的看涨组合：AI 繁荣拉动股市与科技股飞涨，资本从美国长期国债撤离流向数据中心和 AI 基建债务，从而促成国债价格下跌。克里斯蒂亚诺虽然确实拿出了真金白银，但他实际押注的并非人类终结，而是超级技术的经济红利。 ```mermaid flowchart TD accTitle: Paul Christiano Portfolio Analysis accDescr: 展现克里斯蒂亚诺的投资组合逻辑，显示其策略实际上是押注AI繁荣而非AI毁灭。 subgraph Logic["泰勒·考恩的质疑逻辑"] A["坚信 AI 毁灭论"] --> B["应该全面做空市场 / 避险"] end subgraph Portfolio["保罗·克里斯蒂亚诺的真实仓位"] C["2倍杠杆总头寸"] C --> D["90% 净资产押注 AI 相关领域"] C --> E["5% 净资产配置特斯拉"] C --> F["做空美国 30 年期国债"] end subgraph Outcome["实际回报情景"] G["好结局：AI 创造天量生产力"] Portfolio --> G G --> H["科技股与 AI 资产暴涨，美债资金流向 AI 算力基建"] end ``` --- ### [24:15](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1455s) 数字果蝇诞生：从 Rabbit R1 囚禁实验到虚拟生物的道德质感话题转向一项前沿计算神经科学突破：谷歌等机构的研究团队成功完成了对果蝇全脑神经元的三维高精度重建与连接组测绘（Connectome），并将其完整结构以软件模拟的形式在计算机中运行。这意味着果蝇的神经传导与行为决策机制已经被搬进虚拟代码之中。主播霍迪·海斯（Jordi Hays）开玩笑自嘲说，平时经常有人说自己只有“果蝇般的智商”，现在终于有机会让模拟果蝇和真实的“有机、农场直供版霍迪·海斯”同台竞技，看看谁的表现更聪明。然而，随着模拟程序开源，社区开发者的实验迅速走向了荒诞与伦理争议并存的方向。其中最引人注目的实验来自一位名叫凯文（Kevin）的用户：他把果蝇大脑模拟程序直接塞进了便携式 AI 硬件 Rabbit R1 里。当他晃动 Rabbit R1 机身时，屏幕上的神经网络可视化界面中，果蝇大脑负责触发“逃跑反射”（Escape circuit）的神经回路瞬间被高亮激活。这个场景让两位主播感到了本能的心理不适——把一个具有完整应激神经模拟的生命锁在小盒子里摇晃并取乐，从直觉上讲“对人类自身的灵魂极不健康”。尽管果蝇在现实生活中只是令人讨厌的飞虫，但在数字世界里刻意充当施虐者与“死亡商人”（Merchant of death），依然跨越了某种微妙的心理界限。 - 谷歌等机构成功实现三维果蝇全脑连接组（Connectome）模拟，完整复现果蝇神经元网络与运动反应。 - 社区实验者将模拟神经系统部署在 Rabbit R1 硬件上，晃动设备可实时观察到虚拟果蝇大脑逃逸神经通路的点亮。 - 现象引发的直觉反思：即使对象只是几万个神经元的数字映射，刻意制造虚拟应激反应仍会激发人们关于残忍与共情的不适感。 --- ### [26:15](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1575s) 模拟恶魔、数字人类与 Sentience：合成智能的伦理边界主播们由此展开了一场关于虚拟实体道德地位的辩论。在传统即时战略（RTS）游戏中，玩家随手圈选一队士兵送上前线送死，通常不会有负罪感，因为人们潜意识里会自我安慰“这些士兵是自愿跟我冲锋陷阵的”；而在游戏《DOOM》中开枪射杀模拟的恶魔，更被视为合理的反抗。但果蝇模拟完全不同——它并非由简单的 `if-else` 逻辑控制的 3D 模型，而是对真实生物物理神经网络的高精度数字复现。这引出了极具争议的技术伦理滑坡假设：如果未来计算能力再提升几个数量级，人类能够完整模拟出一个真实人类的大脑，这个数字人类在屏幕内拥有与常人无异的思考、对话和情感表达能力，那么他是否应该享有基本人权与道德主体资格？还是说，仅仅因为其底层是一堆硅晶体管上的电子流动，人类就可以肆无忌惮地对其进行折磨和操纵？两位主播认为，抛开晦涩的学术定义，仅从“直觉基准（Vibes-based analysis）”出发，最好的原则就是“永远不要涉足折磨任何事物的行当”。这场风波折射出整个科技界更深层的困惑：大语言模型（LLM）是否具备感知力（Sentience）？人工合成智能是否具有道德权重？这场辩论注定旷日持久，但无论如何，技术探索并未止步——在最新的测试中，研究人员甚至已经成功教会了这只数字果蝇如何侧方停车。 --- ### [27:45](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1665s) 让 AI 摸鱼打游戏：Computer Use 通关《小丑牌》与赛道领航节目的尾声转向了轻快幽默的日常体验。约翰·库根（John Coogan）分享了他前一晚的实验：他调用具备“计算机操作（Computer Use）”功能的前沿模型来玩他最沉迷的独立肉鸽卡牌游戏《小丑牌》（Balatro）。库根笑着说，自己完全没有在“折磨 AI”，反而像是在给辛苦干活的数字打工人发福利：“与其逼着它帮我报税，不如放它假去打游戏。”模型在屏幕上自己移动鼠标、出牌，最终竟然成功通关。库根时不时切回屏幕围观，就像站在场边的足球教练看着自家的神童球员，忍不住在背后指指点点、充当“后座四分卫”。霍迪·海斯也分享了自己在专业赛车模拟器（iRacing）上的趣事：他在拉古纳塞卡（Laguna Seca）赛道刷圈速时，顺手让 ChatGPT 对比专业车手与初学者的圈速数据以寻找提升空间。ChatGPT 极其专业地给出了优化建议，甚至主动提议：“要不你录一段完整的单圈车载视频发给我，我来帮你逐帧分析走线。”海斯调侃道，按照这个节奏发展，AI 很快就会主动问“要不你干脆离开座椅，让我替你开吧”。海斯计划周末就拿自己富余的 API 额度，正式测试让 Computer Use 接管赛车模拟器。在欢快的闲聊中，两位主播向听众送上周末祝福，呼吁大家“去享受人生中最棒的一个周末”，并在苹果播客与 Spotify 上给出五星好评，同时订阅 TBPN.com 的每日科技资讯通讯。 ## Notable quotes > [00:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=0s) 保持推理畅通 > [10:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=600s) 我为什么不能在自己的电脑上算数学题？我想干嘛就干嘛，让我去开发酷炫的技术。 > [12:47](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=767s) 去问 Jacob 关于 AI 灭绝风险的看法，就像是去问你家的空调维修师傅关于气候变化的见解一样。 — Clément Delangue > [12:47](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=767s) 空调师傅只会说：‘那些高深的大道理我搞不懂，哥们儿，我只想确保炎炎夏日里你家空调能吹出冷气！’ > [15:21](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=921s) 表现出或能被轻易修改以表现出在广泛领域和任务中匹配或超越人类认知能力的系统 — Bernie Sanders > [15:21](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=921s) 具备规划并执行剥夺人类权力、推翻或破坏美国政府能力的系统 — Bernie Sanders > [17:48](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1068s) 在 DoorDash 工作了两年，我说出这句话绝非儿戏——我们距离‘在你意识到自己想吃之前，墨西哥卷饼就已经送到门口’的技术极点已经极其接近了。我们不要求全面封禁，我们只请求暂停开发！ > [20:00](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1200s) 如果全人类都灭亡了，赔付给死人的合约对我没有任何价值。 > [22:45](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1365s) 我没理解错吧？你加了 2 倍杠杆？ — Eliezer Yudkowsky > [26:15](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1575s) 永远不要涉足折磨任何事物的行当 > [27:45](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1665s) 与其逼着它帮我报税，不如放它假去打游戏。 — John Coogan > [27:45](https://www.youtube.com/watch?v=I0zUXY7qOu0&t=1665s) 要不你录一段完整的单圈车载视频发给我，我来帮你逐帧分析走线。 — ChatGPT ## Mentioned TBPN, John Coogan, Jordi Hays, Wall Street Journal, TMZ, Matt Damon, AI 2027, AI 2040, Nvidia, Nvidia H100, Patrick O'Shaughnessy, David Senra, Ilya Sutskever, SSI (Safe Superintelligence), Brad Gerstner, Jensen Huang, David Sacks, OpenAI, Project Astra, Hugging Face, Clément Delangue, Nathan Lambert, Bernie Sanders, Fluidstack, Jamie Cox, Will Manidis, DoorDash, Tyler Cowen, Paul Christiano, Tesla, Eliezer Yudkowsky, Google, Rabbit R1, DOOM, Balatro, iRacing, Laguna Seca, ChatGPT ## Video description Sign up for TBPN’s daily newsletter at TBPN.com Diet TBPN delivers the best of today’s TBPN episode in 30 minutes. TBPN is a live tech talk show hosted by John Coogan and Jordi Hays, streaming weekdays 11–2 PT on X and YouTube, with each episode posted to podcast platforms right after. Described by The New York Times as “Silicon Valley’s newest obsession,” the show has recently featured Mark Zuckerberg, Sam Altman, Mark Cuban, and Satya Nadella. TBPN is made possible by: Ramp - https://ramp.com Public - https://public.com Cisco - https://www.cisco.com Console - https://www.console.com CrowdStrike - https://www.crowdstrike.com Figma - https://www.figma.com MongoDB - https://www.mongodb.com NYSE - https://www.nyse.com Railway - https://railway.com Shopify - https://www.shopify.com/ Follow TBPN: https://TBPN.com https://x.com/tbpn https://open.spotify.com/show/2L6WMqY3GUPCGBD0dX6p00?si=674252d53acf4231 https://podcasts.apple.com/us/podcast/technology-brothers/id1772360235 https://www.youtube.com/@TBPNLive Chapters: 00:00 Congress Wakes Up to AI Doom + What AI 2040 Actually Wants 01:24 The AI Pause: Data Center Audits, Compute Controls + US–China Verification 06:30 Slowing the Race to Superintelligence + The Manhattan Project Question 15:21 Bernie Sanders’ AI Ban + The “Corporate Death Penalty” 19:37 AI Doom Bets, DoorDash Superintelligence + Can X-Risk Be Falsified? 24:15 The Simulated Fruit Fly + Do Digital Minds Deserve Rights? [Source](https://www.youtube.com/watch?v=I0zUXY7qOu0)

Source quotes

[0:00] 保持推理畅通 [10:00] 我为什么不能在自己的电脑上算数学题？我想干嘛就干嘛，让我去开发酷炫的技术。 [12:47] Clément Delangue: 去问 Jacob 关于 AI 灭绝风险的看法，就像是去问你家的空调维修师傅关于气候变化的见解一样。 [12:47] 空调师傅只会说：‘那些高深的大道理我搞不懂，哥们儿，我只想确保炎炎夏日里你家空调能吹出冷气！’ [15:21] Bernie Sanders: 表现出或能被轻易修改以表现出在广泛领域和任务中匹配或超越人类认知能力的系统 [15:21] Bernie Sanders: 具备规划并执行剥夺人类权力、推翻或破坏美国政府能力的系统 [17:48] 在 DoorDash 工作了两年，我说出这句话绝非儿戏——我们距离‘在你意识到自己想吃之前，墨西哥卷饼就已经送到门口’的技术极点已经极其接近了。我们不要求全面封禁，我们只请求暂停开发！ [20:00] 如果全人类都灭亡了，赔付给死人的合约对我没有任何价值。 [22:45] Eliezer Yudkowsky: 我没理解错吧？你加了 2 倍杠杆？ [26:15] 永远不要涉足折磨任何事物的行当 [27:45] John Coogan: 与其逼着它帮我报税，不如放它假去打游戏。 [27:45] ChatGPT: 要不你录一段完整的单圈车载视频发给我，我来帮你逐帧分析走线。

Source quotes

[0:00] we just have to talk about this [10:00] I should be able to do math on my computer. I can do whatever I want. Let me do cool things. [13:30] Clément Delangue: Sorry, but asking Jacob about AI extinction risk is like asking your AC guy about climate change, not saying it's necessarily uninteresting or wrong per se, but let's keep things in perspective and hear from the full range of expertise across the ecosystem. [13:30] I don't know about all that mumbo jumbo, but when it's a hot summer day, I don't want you to be worried about the heat, brother. [15:21] Bernie Sanders: corporate death penalty [17:40] Will Manidis: AI will make everyone rich, healthy, and free [17:40] Two years at DoorDash, I do not say this lightly. We are extremely close to the burrito arriving before you decide you want it. We are not asking for a ban. We are asking for a pause. [20:00] Tyler Cowen: If you have very pessimistic fears or predictions about AI, name the market prices that will support or confirm them. This is what taking this seriously means. [20:00] Contracts that pay out if everyone dies aren't worth anything to me. [24:15] It's not good for your soul to be a merchant of death. [24:15] those soldiers opted in to riding and dying with me [28:00] Hey, instead of doing my taxes, you get to just chill and play a video game.

Original transcript

[0:00] What are the AI doomers actually proposing? That's the question I was trying to answer this morning. 20 year sentences. That's one. AI 2027 predicted that this month, you know, late in 2026, Congress would wake up. And that is in the journal. Congress is suddenly waking up to the AI doomsday threat. And so this is happening all over the place. Was it Matt Damon who was caught on TMZ being invested, interrogated about his thoughts on AI risk? protest by the end of the year. I was trying to figure out how big this protest in Chapel Hill, North Carolina earlier this month was. I think it came in sub 10K, but certainly tracking to it. It was about 150 people at this protest in Chapel Hill, but just two ooms away from the prediction. Okay. from AI 2027. So I wanted to dig into like, what the actual proposal is, because the joke for a while has been just like everyone when they're pressed on this, they just say we got to talk about it. We got to talk about this. And the actual proposal is super concrete in AI 2040. And it's, it's very interesting just to hear about how they want to slow things down. So the thing that I think a lot of people online who are like, yay, an anti AI sentiment that's going viral AI 2040 is like, keep the inference flowing. The current models are great. Also, let's keep doing capabilities research, but we just want to reach superintelligence by 2040 instead of 2028, where we're not necessarily prepared. So they want it to be highly controlled by governments, nation states, highly secured. And there's a whole bunch of very, very tactical recommendations that they make training. They don't want to do any more new frontier training runs or R&D experiments. And to enforce this, they're calling to apply inference-only verification to essentially all major AI data centers. So anyone who has more than 10,000 H100 equivalents, roughly $100 million of equipment, and that is pretty easy to figure out. You're just like, big building over there. all these chips, count them up, there's over 10,000 of them, you've got to apply for this permit. You've got to tell us what you're doing, right? Very easy to enforce, at least in the United States, and with an international body, you could kind of do the same thing internationally. So the whole goal, you can only inference the current models, and you've got to verify your workloads with an independent auditor, probably the government. Maybe there's some sort of non-governmental organization that's doing this. You know, there's a whole WORK THAT'S HAPPENED IN THE PAST. MAJOR COUNTRIES, THEY WANT THEM TO DECLARE AI COMPUTE INVENTORIES, TELL EVERYONE not just your local population, but also the international community, how many How many warheads you got? How many H-100 equivalents do you have? Where are they? Everyone shares this. That's going to be a really tough. sell because international agreements are really, really tough sales. It's much easier to have a groundswell of support for something that happens in America. America changes. We have a system that we don't really have the international rules to quickly implement that in a way that doesn't doesn't allow for a lot of defect. They want to physically remove high bandwidth east west networking inside of data centers. So you can't do large distributed training runs, but you can still do inference for any new AI R&D data centers. They want entirely new facilities built from scratch with nation state level physical security and verification. But for the next run and the one after that and the one after that, as it gets crazier, we want it in a new building built inside We can't communicate it from the outside. We want a bunch of physical controls. It's like going to a nuclear facility, highly verified who gets in the building and when, air gap communications. This is one interesting proposal that they have that really shows how deep they thought this through. They want the R&D data center to be connected externally. If you want to communicate with it and you want to tell it what to do, okay, train the next running or do whatever, they will have a bandwidth capability. capped connection at one meg per hour. PER SECOND. SO YOU CAN SEND LITTLE INSTRUCTIONS, BUT IF YOU SAY, SEND ME THE WEIGHTS, BECAUSE I'M TAKING THEM SOMEWHERE ELSE, IT WOULD TAKE YOU LIKE FIVE YEARS TO EXFILTRATE IT. SO INTERESTING, LIKE, HARDWARE SOLUTION TO THIS. [5:00] Do you actually go and implement that? There's going to be a whole bunch of other things, but interesting that they're thinking about like the width of the pipe. So it'd be very obvious if you're stealing the model weights because it's like, wait, this one meg pipe has been at full tilt for months. What's going on here? Someone's taken the stuff out of the data center. They want a front when frontier model weights move from an R&D facility to an inference facility. They want it to be placed on physical storage devices encrypted independently by both the U.S. and China. escorted by representatives of both countries to the destination. This is a tall order. One's a tall order for sure. Yeah. And they actually want frontier models to be made deliberately larger than compute optimal. So they want the weights to be 100 terabytes instead of honing them down to something that's just one terabyte that could actually be moved around a little bit easier. To be air-gapped, wanting Faraday, Cages, Round Podcast Studios. A locked briefcase with an SMB7, SM7B in it. And in order to unlock it, Patrick O'Shaughnessy and David Senra both need to give you codes to independently verify that this podcast is worthy of being recorded. I like that one. Yeah. It just makes sense. High-level valve they see as being the most effective in controlling the speed of capability improvements is compute caps. So there is a world, and we're going to the Bernie Sanders thing because it's already getting sort of twisted, but the big hammer is just chip controls and data center build-out slowdown. That's the easiest thing, and I think that's the biggest valve that we're going to see twisted around to actually slow down capabilities. which can leak to secret projects. So the goal is like, okay, well, we know that this model is capable of this, so we want this much compute over here. Okay, you've done well, we're allocating more compute as opposed to this one weird trick that AI doomers hate. unintended consequences. And one thing that feels obvious around this, if these policies were to be rolled out, is that you would effectively create an incentive for millions of individuals or groups globally to be in secret, like trying to find entirely new breakthroughs that are. Yeah. And again, this incentive already exists, but it kind of pushes a lot of the idea that humans are just going to be like, oh, I'm not, I'm no longer going to try to create the God model because like there's this, you know, big global organization that's sort of policing it. I just... I mean, that's the same thing we see with nuclear non-proliferation. There's always a discussion about what countries are getting the bomb and how far along are they and wars break out over this. And yeah, like the game's not over just because you create a framework. But ultimately, GPUs and computers are... much more wide, you know, infinitely more widespread than nuclear materials. Yeah, but you still got to marshal them all together. Yes, there's some weird scenario where there's a Python script that's AGI that can run on your laptop, but I think most people are convinced, at least in this crowd, that scale is a prerequisite. And, I mean, we were joking about, like, it would be so, like, SSI, Ilya Sutskiver's new lab, cluster from NVIDIA, and we were like, the most bullish thing you could do if you're this secretive Neolab would be like, we're actually selling our compute because we've discovered a more compute optimal way to reach AGI, and we don't need a lot of compute. But of course, even Ilya is like, it's time to scale up. I need more compute because it seems like even if he's taking a completely orthogonal approach to, you know, innovation and research, he still needs a lot of compute. And so TO BE A BIG BUILDING WITH A LOT OF ENERGY, BIG HEAT SIGNATURE, DEFINITELY VISIBLE FROM SPACE AND PRETTY SIMPLE TO TRACK AT LEAST IN THE SHORT TERM UNTIL PEOPLE START BUILDING CRAZY UNDERGAND FACILITIES AND THEN YOU'RE BACK TO NUCLEAR NONPROLIFERATION. BUT THEIR GOAL IS AT LEAST TO LIKE, TO BE ABLE TO DO THAT. Yeah, try, try. Yeah, and then the other side of this is-- It's not a panacea. Does AI development actually become something [10:00] closer to the Manhattan Project where That's what a lot of people are thinking. A hundred researchers are working with the government in secret because you can't just assume that other countries are going to slow down or do any of these things. Yeah. But in general, I think the proposal is – Don't go back in time. Definitely not stop everything in its tracks. It's a slowdown with the goal of scaling gradually. They do actually want to reach superintelligence. They just want to do it by 2040, hence the name of the project. So the goal is gradually scale into top human expert capability around 2035. Now a lot of people are saying, oh, we might get this by 2029, 2028, 2027, and they see that as too fast, so they want to push that out to 2035. and wait five years with AGI, and then unlock superintelligence in 2040. This is their initial proposal. Of course, there's a lot that could change over the next decade. And I think to zoom out, overall, if you're worried about X-Risk, the AI 2040 plan does feel like a concrete path towards slowing down. The conversation definitely gets dragged down into PDOOM estimates and trying to narrow down exactly how a human extinction scenario plays out. P. Doom or any of those. It's just like, if everyone feels that way, something will happen. This is a concrete plan of what that might look like. And that's valuable to understand in this case. So for the safety skeptics, it's easy to see how this level of control over what you can do with computers is authoritarian or anti-libertarian. Even if we're talking about $100 million computers, there's a lot of people that say, like, I should be able to do math on my computer. I can do whatever I want. Let me do cool things. I'm excited about this. That limits It might create regulatory capture for a few major players. It might crash the stock market or delay economic gains that come in the good ending where alignment is solved and X risk plummets. You can imagine a situation where in a few years, if X risk is not a good ending, fades into the background, you're like, yes, there's still a risk, but it's the same risk that we face every day with, like, an asteroid hitting Earth. Like, no one – it doesn't really change anyone's behavior. That would be sort of the good ending, in my opinion. So it's a balancing act, and for most of these slowdown proposals. I personally have a hard time blackpilling about them in the sense of, like, if all of this gets implemented, how frustrated will I be? Like, the models are good. I would like better models. I want safe models. There is this massive capability overhang. The current models can do a lot of interesting work. We're finding new uses even for like non-leading-edge models. There's a lot that can be done. So I don't believe the doom-doomers, who are dooming about what the doomers are planning. I find that unconvincing right now. But people are starting to lay it out more. Brad Gerstner, Jensen Wong, David Sachs are talking about the other side of this equation. There's also a big question around what does... Do the 2040 people have a point of view on robotic and physical AGI? Because it seems like even if you pause... like, you know, efforts towards RSI. Well, if we add billions of robots into the world that are just running on today's model, that also presents today's, like, you know, I don't think they're worried about that. I don't think they're worried about a billion robots with GPT-6 level intelligence and years of alignment work. that is currently happening. That's fine. It's the next, next, next thing, the super intelligence, the thing that might have its own goals. I think I mean, we're talking to someone from OpenAI's robotics team hooking up Astra to a robot, a paintbrush and a camera. We talked about an earlier painting. I don't think we're at a point where that poses a risk. It's the next model. It's the model with its own volition, basically, which a lot of people still aren't seeing. Yeah, the models keep getting better, but they seem to follow your instructions, sometimes too much, and then you need to worry about the paperclip scenario. But it's not that they want to do their own thing necessarily. I don't know. But people are going back and forth on this. Clem over at Hugging Faces, sorry, but asking Jacob about AI extinction risk is like asking your AC guy about climate change, not saying it's necessarily uninteresting or wrong per se, but let's keep things in perspective and hear from the full range of expertise across the ecosystem. And Nathan Lambert says, banger. What was our take? [15:00] You guys might be right. My AC guy would be like, I don't really know about that, but I just want to make sure when you're hot, that we can run this AC cool. Yeah, the AC seems-- I don't know about all that. I don't know about all that mumbo jumbo, but when it's a hot summer day, I don't want you to be worried about the heat, brother. I like that. Yeah, this is kind of an unnecessary shot at AC guys. AC guys are important. I don't know. It's funny. What did Bernie Sanders have to say? Is this real? Entities shall be subject to the corporate death penalty, and persons shall be subject to not more than 20 years in prison if they don't pause AI development. Thank you. That seems pretty easy to comply with. I don't know. I guess how do you define AI development? is prompt engineering, AI development, and then you get caught because your model sort of Did a little prompt engineering in the final stage, and then you're guilty of this like that? Yeah, that could be a negative knock-on effect, I guess. It does seem aggressive. But his overall proposal is banning artificial superintelligence so no person or entity may develop or deploy superintelligent AI systems. He defines artificial superintelligence as an artificial intelligence system that exhibits or can easily be modified to exhibit capabilities that match or exceed human cognitive performance and capabilities broad range. of domains or tasks. It sounds like the mission statement. It sounds like the explicit goal of like 17 different companies right now. AI systems that have sufficient capabilities to plan and execute the disempowerment of humanity. Okay, that's a good one. I like that. I don't like overthrowing or undermining the U.S. government. So strongly in favor of banning that. federal agency will monitor frontier AI systems at all stages of the life cycle, supervise the removal of dangerous capabilities, and supervise the destruction of artificial superintelligence. We're coming for it, which is similar to the Corporate death penalty is a line that you don't hear a lot, right? Usually these companies just, you know, go bankrupt and wind down. corporate death penalty goes pretty hard. It's kind of metal. Yeah, you kind of got me with that one. Yeah. ROUGH, ROUGH, ROUGH SITUATION. WE'LL SEE WHERE IT GOES. JAMIE COX OVER AT FLUID STACK, THE CO-FOUNDER OF FLUID STACK, A COMPUTE PROVIDER, SHARED HIS CONVICTIONS sort of pushing back on a lot of this, saying that he thinks America should build more. They're pro-freedom, pro-democracy. They believe AI will bolster human flourishing. We support simple, clear, enforceable regulation frameworks that set simple requirements proportioned to capabilities and risk with clear responsibilities and no unnecessary barriers to competition. the regulatory stuff is going to be like these 10 companies, and I'm going to be number 11, and I'm basically getting the corporate debt policy then because I didn't make the cut to be one of the regulated, one of the approved companies. I'm still early in my stage, so there's a lot of nervousness, I'm sure. But Will Manitis said, we believe AI will make everyone rich, healthy, and free is novel and interesting comms from the frontier. He's endorsing this. like these convictions. I think it's generally a positive direction to move in, not a direct response to the proposals that are going out, but we're going to get a whole lot more of them. Where do you want to go to next? Over on TikTok, they're sharing a photo of the whistleblower and saying in every worldwide disaster movie, there's a dude that looks just like this that nobody listened to. He really does look like an actor. He does look like an actor here. Yeah, he looks good. Um... But people are all over the place. And please stop calling him Scary Potter. I've been seeing people. I've been seeing people over on X calling him Scary Potter. That's the goal. The goal is to wake up China, wake up Congress, wake up everyone. DoorDash has entered the conversation. They say, two years at DoorDash, I do not say this lightly. We are extremely close to the burrito arriving before you decide you want it. We are not asking for a ban. We are asking for a pause. I don't know why they would ask for a pause. that seems like very aligned to humanity yeah [20:00] into their business, which I think is fantastic. The doom is very much contained to the frontier lab work. Everyone in the application layer who's applying the models, diffusing them. They're like, "I can't get this thing to work, right? I gotta get forward-deployed engineers to teach people how to use this thing." Jim Rainbow over Jim O'Reilly. Jim Reilly over at Charleston AI says it's a whole lot of mumbo-jumbo. He's just happy to get eight hours back. And then, yeah, everyone deeper in the supply chain, like Jensen and all the different semiconductor manufacturers, are not particularly on this side. And then you also have Wall Street, who's just like, what's the enterprise acceleration? So lots of different groups around the table that need to be brought on board to this movement. The discourse truly is fascinating. So what is Tyler Cowen calling for? He's banging the table saying, bet on this. Tyler Cosgrove. Are you named after Tyler Cowen? Is that your namesake? Tyler Cowen says, if you have very pessimistic fears or predictions about AI, name the market prices that will support or confirm them. This is what taking this seriously means. And I love Tyler Cowen. I'm not sure this matters because you're not going to be around to collect it. Yeah, this is what Deep Dish employer says. in any meaningful way. Spell out the exact mechanism. Contracts that pay out if everyone dies aren't worth anything to me. Yeah, I don't know. Yeah, I think he's calling for like He wants to make a falsifiable claim. Like, is this... Am I able to tell if your claim is true or false? And so it's very hard with these scenarios. Isn't it an unfalcible claim, though, just by definition? And you just have to accept that and move on? Yeah, but then it's so hard to have any real discussion. Was nuclear any different? Like the threat of nuclear apocalypse, the threat of World War Three. This was a very motivating factor for decades. Most of the 20th century, people made real decisions based on it. based on where to do business and where the conflicts were going to be and the motivation for nuclear treaties and nonproliferation. You can make financial decisions. like, think about nuclear war, right? You have a bunker. That's like-- that is a-- you make-- Bunky? Is anyone making like AI bunkers? No, because they think it's going to be so totalizing that the bunker actually doesn't do anything. Exactly, yeah. So if you think it's so totalizing, then you don't make the bunker. And so saying, hey, you don't have a bunker is not proof that the person doesn't believe what they're saying. There is the other side of this, which is Paul Cristiano, who has been worried about risk. He recently just joined the board of OpenAI. And Tyler Cowen has this quote, if you're a doomer, why aren't you short the market? And Paul Cristiano is... He's 2x levered long, and he's short the bond market or U.S. treasuries, right? So he has 5% of his net worth in Tesla, 90% of his net worth in AI bets, and 100% of his net worth in normal investments. No Tesla options. That sounds like a scary place with lottery ticket biases and the crazy Tesla investors. And then Eliezer Yudikowsky says, am I correctly understanding? You're 2x levered? And Paul Christiano says, yeah. He says he's personally short the U.S. 30-year debt. I think that just means you have a mortgage. This doesn't seem like a Doom-based bet. This seems like this bet also pays out in just the good ending, and AI is real and delivers value. So your AI bets perform well, the market performs well, and money slides from U.S. debt to data centers and AI build-out debt or something like that. So he's putting his money where his mouth is, but it doesn't feel like a representation of Doom by any means. People have been creating AI agents of fruit flies. Have you seen this? Yes. My understanding is Google basically mapped out all of the neurons in a fruit-fry FRUIT FLIGHT. YEAH, LIKE THE ENTIRE -- IN 3D, THEY GOT THE ENTIRE STRUCTURE, AND NOW People have been able to recreate that in software, in a simulation. Yes, I think in theory you can replicate all of the fly's decisions or whatever. It's like movements. Yes. And this is notable because many people have said I have the mind of a fruit fly. They've said I have the intellect of a... Yeah, so now they're going to put it to the test to see which performs better, the simulated fruit fly or just Jordy Hayes who we got right here. Just organic, farm-to-table. [25:00] So now that this is out and the code is out and you can run this fruit flying simulation however you want, people are having, they're doing all sorts of experimentation. So Kevin said he trapped the fruit fly in his rabbit R1. When he shakes it, he can see its brain's escape circuit light up. human and doing this is probably not good. for your own soul. And, you know, imagine, imagine you have a fruit fly in a box and you're shaking it and you're like, look, it wants to escape. Yeah, totally. I'm not a huge fan of small insects. I don't really want them around that much. But when they're in my house, I try to, you know, if a spider's in my house, even if I know it might want to take a nice bite of me, It's not good for your soul to be... a merchant of death. Sure. And yet if you're playing a real-time strategy game and you highlight a bunch of soldiers in this simulation and you send them on a charge that will result in their virtual death. You might not feel... But those soldiers opted in to riding and dying with me. Okay, okay. Right? Okay, what about... This fruit fly is just sitting there, John. So they made the fruit fly play Doom. What about if you play Doom, you are... killing and demon who's simulated? Is that immoral? Is that demon trying to kill you? A lot of it comes down to the fact that it's simulating the actual representation of the fly makes it a lot more concrete than just, oh yeah, it's a 3D model in a Python script that just says if you see a character shoot at them in the simulation. But we're clearly starting to grapple with these odd moral questions of step. It's a couple of order of magnitude, but you get there and you can simulate a human and you could talk to that human and it would do everything the human does. Does that human have rights and agency as an ethical moral agent, or is it merely just a simulated, just a really good computer simulation? It's just existing on transistors, so you don't need to feel any moral weight about anything that you do to it. I agree with just the vibes-based analysis that like torturing a real You probably just don't be in the business of torturing anything. You don't need to overthink it. But people are, the real debate here is like the question of, you know, are LLMs sentient? Are they moral? Is there a moral weight to the, to synthetic intelligence, to artificial intelligence? That's what people are debating here. And I'm sure the debate will continue. Who knows if it will ever be ended. But they did teach it how to parallel park, which I think is cool. Interestingly, last night I had Astra use computer use to play a video game that I very much enjoy playing called Blatro. It's sort of like a modified poker game. And I don't feel like I was torturing the LLM by that. I feel like I was giving it a treat. I was like, hey, instead of doing my taxes, you get to just chill and play a video game. It did very well. It won. And it was really fun because I would like pop in while I was using the computer inside this kind of armchair quarterback and be like, is it making the right decision right now? It felt like a coach coaching like a kid on the soccer pitch or something. I had a funny moment last night. I was on my racing simulator comparing – asking ChatGPT for... uh, for, to, to compare my times, uh, Laguna, uh, to, to just other, other, you know, what, what would best in class be like, what's beginner like, et cetera. Um, and it said, if you want, I can help you, uh, you know, cut, cut some seconds off of this time. Uh, and it, and it was like, why don't, why don't you take a video of a full lap so that, and I'll analyze it for you. And I Hang out and watch track footage. Pretty soon it's going to be like, you want me to just get in the seat? You might just take over. Yeah, no, computer use in iRacing is something I'm going to experiment with this weekend. I'm down. I'm down. Take half my quota, my monthly quota. Just play games, chill, do whatever. It's a nice treat. I've got some banked resets that I'll put to work. Wonderful week. Wonderful week. Short week. See you Monday. Yeah. But do us a favor. and Go ahead and have the best weekend of your entire life. Have the best weekend of your entire life. Let's do it. Put the PC. [30:00] together make it happen. We'll see you on Monday. Leave us five stars on Apple Podcasts and Spotify. Sign for a newsletter at tbpn.com. Thank you.