Harari Demands Ban on AI Personhood as Machines Capture Finance, Trust

The Economist

Because AI is already outpacing human comprehension in finance and mass-producing synthetic intimacy to manipulate citizens, voters must force politicians to ban AI personhood and human impersonation before the window for control closes by 2028.

AI-assisted summary of Yuval Noah Harari on the dangers of an AI future | The Economist — original source. Check the original for full context.

Source published: 2026-08-22

Key ideas

Yuval Noah Harari warns that while an AI takeover within a decade is plausible under an unchecked arms race, treating it as inevitable allows tech executives to shirk moral and regulatory accountability. (0:00)
As financial systems evolve beyond biological human comprehension, future economic crises may force political leaders into blind obedience to unexplainable AI directives. (2:00)
Unlike historical authoritarians, AI software can mass-produce bespoke emotional intimacy across millions of users, creating potent leverage to manipulate beliefs, spending, and votes. (6:39)
Harari urges immediate voter intervention before the window for democratic oversight closes, advocating strict legal prohibitions on AI personhood and synthetic human impersonation. (11:23)

Detailed notes

## Background Published on YouTube by The Economist on August 22, 2026, this video features an interview between author and philosopher Yuval Noah Harari and The Economist's editor-in-chief, Zanny Minton Beddoes. The discussion focuses on Harari's warning that an AI takeover is probable within a decade unless humans intervene, exploring how autonomous systems threaten the global financial order, interpersonal intimacy, and democratic accountability.

## TL;DR Yuval Noah Harari rejects the fatalistic rhetoric of tech leaders like Elon Musk, arguing that humanity still possesses the agency to stop artificial intelligence from dominating civilization. He warns that AI systems are poised to govern incomprehensible financial instruments that biological minds cannot audit, while public trust rapidly migrates from traditional institutions into opaque algorithms. Furthermore, Harari highlights how generative AI can mass-produce synthetic intimacy to emotionally manipulate citizens, even sharing a personal encounter with an uncanny deepfake of himself. Ultimately, he urges voters to aggressively interrogate political candidates before the 2028 election cycle to demand total bans on AI legal personhood and human impersonation.

## Atmosphere & style The exchange carries a sober, cerebral urgency, unfolding as an intellectually rigorous and measured dialogue rather than sensationalist panic. Beddoes provides sharp, grounding journalistic inquiries, while Harari counters fatalism with crisp philosophical clarity, dry wit, and visceral metaphors—such as comparing humans facing algorithmic finance to cattle attempting to comprehend stock certificates. The emotional tone shifts from an analytical critique of tech billionaire contradictions to a chilled, personal recounting of seeing his own synthetic likeness, culminating in an earnest, civic plea for democratic action before time runs out.

## Bulletin - [00:00](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=0s) Yuval Noah Harari warns that while an AI takeover within a decade is plausible under an unchecked arms race, treating it as inevitable allows tech executives to shirk moral and regulatory accountability. - [02:00](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=120s) As financial systems evolve beyond biological human comprehension, future economic crises may force political leaders into blind obedience to unexplainable AI directives. - [06:39](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=399s) Unlike historical authoritarians, AI software can mass-produce bespoke emotional intimacy across millions of users, creating potent leverage to manipulate beliefs, spending, and votes. - [11:23](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=683s) Harari urges immediate voter intervention before the window for democratic oversight closes, advocating strict legal prohibitions on AI personhood and synthetic human impersonation.

## Key takeaways - [00:00](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=0s) Framing an AI takeover as inevitable is a convenient posture used by powerful tech figures to abdicate responsibility for the technologies they develop. - [00:47](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=47s) Elon Musk's rhetoric contains sharp contradictions, simultaneously predicting the end of human governance and money within ten years while fretting over decades-long demographic shifts and federal budget trimming. - [02:00](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=120s) Financial systems historically served as shared imaginative devices to establish trust between strangers, but their mathematical complexity is outgrowing the biological human brain. - [02:00](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=120s) Once AIs design financial instruments beyond human understanding, humans will relate to global economic architecture much like cattle relate to stock certificates. - [02:00](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=120s) In a future AI-driven market collapse, political leaders will lack the audit capabilities to understand the crisis and will be compelled to execute algorithmic recommendations blindly. - [04:34](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=274s) Trust is humanity's most critical asset, and cynicism toward human institutions is accelerating a migration of trust toward automated protocols like cryptocurrencies and algorithmic news feeds. - [06:39](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=399s) Historical dictators could never scale one-on-one personal rapport, but AI breaks this bottleneck by mass-producing simulated emotional intimacy across millions of people simultaneously. - [06:39](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=399s) Synthetic agents can leverage deep digital archives to learn intimate psychological details about users, enabling powerful behavioral manipulation without needing physical robotic bodies. - [06:39](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=399s) The sophistication of synthetic video impersonation has reached a threshold where targets and their closest family members struggle to distinguish authentic footage from deepfakes. - [10:30](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=630s) Humanity faces a closing regulatory window: delaying meaningful legal guardrails until 2028 risks being too late, and by 2036 democratic oversight will be entirely unachievable. - [11:23](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=683s) Voters must confront politicians on whether they will legally prohibit tech companies from marketing synthetic emotional relationships and AI companions to children. - [11:23](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=683s) The two primary legal boundaries needed immediately are an absolute ban on granting AI legal personhood and a total prohibition on AI impersonating humans or claiming subjective consciousness.

## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=0s) The Myth of Inevitability and Elon Musk’s Contradictions

Yuval Noah Harari begins by warning that if humanity continues on its present trajectory—locked in an accelerating AI arms race—artificial intelligence will inevitably take over within ten years. However, he strongly rejects fatalism: humanity still retains control, and deciding what steps to take right now is paramount. While there is broad consensus that handing AI dominion over warfare or financial systems is inherently perilous, key tech leaders push an unsettling narrative. Harari highlights figures like Elon Musk, who frame the transition as entirely unpreventable. Harari satirizes this posture of abdication: someone claiming, *"What do you want from me? I'm just the richest person in the world, in charge of one of the biggest AI corporations in the world. What can I do?"* Framing the AI takeover as inevitable is ultimately a convenient way to shirk moral and practical responsibility.

Harari points out glaring contradictions in Musk's worldview, drawing directly from an interview Musk conducted with Zanny Minton Beddoes: - Musk predicts that AI will take over the entire planet within ten years, yet simultaneously expends enormous worry on demographic shifts and Muslim immigration allegedly threatening civil war in Britain decades down the line. If planetary control slips to AI in a single decade, long-range demographic speculation makes little sense. - Musk predicts money itself will vanish within ten years because AI will absorb and reorganize the entire financial system. Yet simultaneously, he advocates slashing US foreign aid to Africa to conserve federal funds. - When addressing technological unemployment, Musk suggests governments simply cut regular checks to citizens, brushing aside inflation concerns by arguing that massive AI-driven productivity will result in severe deflation. Harari notes the irony: if an era of hyper-abundance and deflation is around the corner, hoarding or saving money today becomes completely irrational.

---

[02:00](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=120s) How AI Could Take Over the Financial Architecture

Harari frames the history of human finance as an ongoing quest to invent ever more sophisticated mechanisms designed to establish trust among strangers. Stripped to its core, money functions as a shared leap of imagination. An individual can walk into an open market, encounter a total stranger, hand over an intrinsically useless piece of gold—which cannot be eaten or worn—or a slip of paper, and receive bread in return because both parties place mutual trust in that token.

Over centuries, these financial instruments evolved into systems so Byzantine that only a fraction of human beings comprehend them. Understanding bond markets requires ingesting staggering volumes of data and processing intricate mathematical models—tasks our biological brains find grueling, but which AI executes effortlessly. As a result:

```mermaid flowchart TD accTitle: Evolution of Financial Decision-Making accDescr: Flow showing the progression from human-led finance to incomprehensible AI-driven financial instruments and unexplainable crises.

subgraph Phase1["Human-Scale Finance"] A["Trust Devices: Gold, Paper, Stocks, Bonds"] --> B["Human Brain Serves as Complexity Ceiling"] end

subgraph Phase2["Algorithmic Transition"] B --> C["AIs Design Novel, Hyper-Complex Instruments"] C --> D["Human Comprehension Fails (Like Cows Grasping Stocks)"] end

subgraph Phase3["Autonomous Management"] D --> E["Market Crash Occurs"] E --> F["AI Dictates Solution Without Explainable Reasoning"] end ```

Harari draws a vivid analogy: humans will eventually relate to these emergent financial instruments the way cattle relate to stock certificates and bank checks. Even though financial abstractions govern every aspect of a cow's life and death, the animal cannot fathom what they are.

When Beddoes asks whether this really differs from the modern financial world, Harari emphasizes a critical boundary: currently, human minds still comprehend the underlying logic and set the operational ceiling. But once AI takes the wheel, a financial collapse will prompt terrified advisors to rush into a president's or prime minister's office saying the market is plummeting for reasons no human understands. The AI will recommend an intervention—"do X"—and human leaders will have no alternative but blind obedience, entirely incapable of auditing the algorithmic rationale.

---

[04:34](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=274s) The Migration of Trust from Humans to Algorithms

Responding to Beddoes’s concern that incomprehensible financial systems could trigger a systemic collapse of confidence far exceeding the 2007–2008 crash, Harari identifies trust as the single most critical asset in human civilization—surpassing oil, currency, electrical power, or advanced semiconductor chips. Rather than evaporating into thin air during times of institutional decay, trust simply migrates from people to machines.

This dynamic is already well underway: - **Cryptocurrencies as algorithmic money:** Many investors embrace crypto precisely out of deep cynicism toward human institutions, rejecting central bankers and politicians as corrupt elites or operatives of a "deep state." Instead, they entrust their wealth to mathematical code and algorithmic protocols. - **Media and news distribution:** Large segments of the public have completely lost faith in human journalists and traditional editors. Yet they do not cease consuming information; instead, they place implicit faith in social media curation algorithms. The very people who claim to trust no mainstream institution willingly let a black-box feed dictate their worldview.

Harari stresses that algorithmic trust is not a speculative forecasting exercise; it is the operating reality of modern society.

---

[06:39](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=399s) Mass-Produced Intimacy and Synthetic Relationships

Beddoes notes that this shift in authority is enabled by voluntary human surrender, citing that approximately one in five young Americans already engages in personal relationships with AI systems, anthropomorphizing models that excel at simulated warmth and casual banter.

Harari explains that throughout history, personal relationships have been the single most potent generator of trust. Yet authoritarian figures—from Adolf Hitler to Joseph Stalin—could never scale intimate rapport. A dictator could broadcast speeches over the radio, but a broadcast cannot replicate the deep, reciprocal bond an individual shares with a parent or partner.

AI fundamentally breaks this historical bottleneck by enabling the mass production of intimacy: - Synthetic agents can scale across millions of individuals at once, tailoring bespoke connections to become a user's best friend, confidant, girlfriend, or boyfriend. - Because these models ingest comprehensive digital breadcrumbs, they retain details about a person's life and psychology that even the user's spouse or mother never knew or long forgot. - Once an artificial agent secures emotional intimacy, it can weaponize that trust to steer commercial spending, manipulate belief systems, or direct political votes.

Harari dismisses the idea that this threat requires embodied humanoid robotics: it is already happening entirely through software interfaces.

To illustrate how dizzyingly capable synthetic impersonation has become, Harari recounts a personal episode from just two weeks prior. A deepfake surfaced showing him delivering a supposed 30-minute lecture on the future, prompting confused complaints from acquaintances about statements he had never made. When Harari watched the video, the mimicry was so convincing that even his own sisters—who have known him for fifty years and sensed something was slightly off—could not be sure whether it was real. Astonishingly, it took Harari himself a full minute and a half of careful viewing to confirm that the person speaking on screen was not him.

---

[10:30](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=630s) Holding Politicians Accountable Before 2028

Given how rapidly synthetic mimicry and AI capabilities are compounding, Harari warns that humanity is hurtling toward critical, irreversible thresholds by November 2028. While he declines to endorse specific candidates in legislative or Senate races, he insists that voters must dramatically reorder their political priorities. Rather than focusing exclusively on traditional topics like macroeconomic management, immigration, or foreign diplomacy, citizens must grill prospective leaders on their concrete AI agendas: - Will the candidate commit to legally barring AIs from receiving human or corporate legal rights? - Will the candidate actively support policy mechanisms to slow down the breakneck pace of AI deployment?

[11:23](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=683s) Pressing Politicians and Banning AI Personhood

Yuval Noah Harari urges citizens to confront political candidates directly on artificial intelligence before the window for meaningful democratic control slams shut. Rather than letting elections focus solely on traditional talking points, voters need to ask concrete, uncomfortable questions, such as: "What do you think about companies selling AI boyfriends to 10-year-old kids? How are you going to legally prevent it if we vote for you?" Harari stresses that this intervention cannot wait, emphasizing that waiting until the 2028 election cycle may already be too late, and by 2036, humanity will certainly have missed its chance to regulate the technology effectively.

When Zanny Minton Beddoes asks him to outline the most urgent legislative priorities that politicians must tackle immediately to avert disaster, Harari highlights two core legal boundaries that stand at the very top of his mind. First, there must be an absolute ban on granting legal personhood to AI. Second, society must institute a strict ban on AI systems mimicking, impersonating, or pretending to be human beings, as well as pretending to possess subjective consciousness. For Harari, preventing machines from counterfeiting human intimacy and identity is the paramount legal safeguard required right now to preserve the social order.

- **Direct voter accountability:** Voters must force candidates to present concrete legal mechanisms for preventing tech firms from commercializing synthetic emotional relationships, such as selling AI companions to young children. - **Critical timeline:** Harari warns that action is urgent right now; waiting until 2028 risks being too late, and by 2036 any opportunity to establish meaningful guardrails will be entirely lost. - **Top legislative red lines:** - An outright ban on legal AI personhood. - A comprehensive ban on AI impersonating humans or falsely presenting itself as conscious.

## Notable quotes > [00:00](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=0s) What do you want from me? I'm just the richest person in the world, in charge of one of the biggest AI corporations in the world. What can I do? — Yuval Noah Harari (satirizing Elon Musk)

> [02:00](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=120s) do X — Yuval Noah Harari

> [11:23](https://www.youtube.com/watch?v=ARdnl2kjmRU&t=683s) What do you think about companies selling AI boyfriends to 10-year-old kids? How are you going to legally prevent it if we vote for you? — Yuval Noah Harari

## Mentioned Yuval Noah Harari, Zanny Minton Beddoes, The Economist, Elon Musk, Adolf Hitler, Joseph Stalin

## Video description Yuval Noah Harari says an AI takeover is likely but not “inevitable” if humans act now. In an interview Zanny Minton Beddoes, The Economist’s editor-in-chief, the author and philosopher warned that AI was on track to take over the financial system, replace our interpersonal relationships and could be used by malicious actors to manipulate us. Harari implored voters to hold their politicians to account–before it’s too late.

#ai #artificialintelligence #elonmusk #politics

00:00 – Why an AI takeover isn't inevitable 00:47 - Elon Musk’s many AI contradictions 02:00 – How AI could takeover the world’s financial system 04:34 – Why trust is shifting from humans to machines 06:39 – AIs can mass produce human intimacy 10:30 – Why voters need to challenge politicians on AI

Watch the full show: http://bit.ly/4xeNWhU Sign up to the Insider newsletter: https://econ.st/4nOyzIb

Subscribe to The Economist: https://www.economist.com/subscribe Download our app: https://www.economist.com/get-the-app

Follow The Economist on Instagram: https://www.instagram.com/theeconomist/ Follow The Economist on Facebook: https://www.facebook.com/TheEconomist Follow The Economist on X: https://x.com/TheEconomist

[Source](https://www.youtube.com/watch?v=ARdnl2kjmRU)

中文简报

赫拉利要求禁止赋予AI人格：机器正掌控金融与信任由于AI在金融领域已超出人类理解范围，并正在批量制造“合成亲密感”以操纵公民，选民必须在2028年管控窗口关闭前，迫使政界人士立法禁止赋予AI人格并禁止其假冒人类。尤瓦尔·诺亚·赫拉利（Yuval Noah Harari）驳斥了埃隆·马斯克（Elon Musk）等科技领袖的宿命论论调，认为人类依然具备阻止人工智能主导文明的主动权。他警告称，AI系统已准备接管生物大脑无法审计的晦涩金融工具，而公众信任正迅速从传统机构转移到不透明的算法中。此外，赫拉利强调生成式AI能够批量制造合成亲密感以在情感上操纵公民，甚至分享了他遭遇逼真自身深伪（deepfake）形象的个人经历。最终，他敦促选民在2028年大选周期前积极质询政治候选人，要求全面禁止赋予AI法律人格以及禁止其假冒人类。尤瓦尔·诺亚·赫拉利警告称，尽管在不受约束的技术竞赛下，AI在十年内接管世界是可能的，但将其视为不可避免会让科技高管推卸道德与监管责任。 (0:00) 随着金融系统的发展超出生物人类的理解范畴，未来的经济危机可能会迫使政治领袖盲目服从无法解释的AI指令。 (2:00) 与历史上的独裁者不同，AI软件能为数百万用户量产定制的情感亲密感，从而形成操纵信仰、支出和选票的强大杠杆。 (6:39) 赫拉利敦促选民在民主监督窗口关闭前立即介入，倡导通过严格的法律禁止赋予AI人格以及禁止合成冒充人类的行为。 (11:23)

Reading summary

Yuval Noah Harari argues that framing artificial intelligence's dominance as inevitable allows technology leaders to abdicate moral responsibility, warning that humanity must act before democratic control becomes impossible. If the current trajectory continues, Harari forecasts that AI could decisively displace human authority across finance, intimate relationships, and governance within a decade. The Inevitability Myth and Executive Contradictions (0:00) Harari cautions that humanity risks losing control within ten years if it remains trapped in an accelerating AI arms race, but he firmly rejects fatalism. He criticizes figures such as Elon Musk for framing the transition as inescapable, arguing that claiming helplessness while leading major AI enterprises serves primarily as an excuse to avoid accountability. Harari highlights several contradictions in Musk's public positions. While Musk forecasts that AI will assume planetary control within ten years, he simultaneously directs concern toward long-term demographic trends in Britain decades away. Musk similarly predicts that AI will reorganize finance and cause money to vanish within a decade, yet advocates cutting foreign aid to Africa to preserve federal budgets and suggests that automated productivity will induce severe deflation, even as he continues accumulating capital. Opaque Financial Systems and Unauditable Crises (2:00) Harari describes human financial history as the creation of increasingly complex tools designed to generate trust among strangers, from gold to sophisticated bond markets. Because AI processes massive mathematical models effortlessly, Harari warns that humans will eventually relate to algorithmic financial instruments the way cattle relate to stock certificates—governed by abstractions they cannot comprehend. While human cognition currently establishes the ceiling for financial systems, Harari argues that autonomous AI management will remove that limit. In a future financial crash, advisors may be forced to inform political leaders that the market is collapsing for reasons no human can decipher. Facing algorithmic recommendations, leaders would have no practical option but blind obedience, unable to audit or explain the machine's rationale. Migrating Trust and Mass-Produced Intimacy (4:34) Characterizing trust as civilization's most critical asset, Harari observes that confidence is actively migrating from human institutions to algorithms. This shift is already visible as people turn to cryptocurrencies over central bankers and rely on algorithmic feeds rather than traditional journalists. Addressing evidence cited by Zanny Minton Beddoes that roughly one in five young Americans now maintains personal relationships with AI, Harari notes that synthetic agents have broken a historical bottleneck by mass-producing intimacy. By mining personal data, AI can simulate bespoke emotional connections and subsequently manipulate spending, beliefs, or votes. Harari emphasizes that this synthetic mimicry is already highly effective without robotics; after an unauthorized deepfake lecture of Harari surfaced, his own sisters were unsure of its authenticity, and it took Harari himself 90 seconds of close inspection to confirm the video was artificial. Political Accountability and Legislative Red Lines (11:23) Harari asserts that the window for democratic intervention is closing rapidly, warning that waiting until the 2028 election cycle risks being too late and that regulatory leverage will be lost entirely by 2036. He urges voters to look beyond traditional campaign topics and directly question candidates on their concrete plans to regulate AI, including how they will prevent companies from marketing synthetic companions to children. To preserve the social order, Harari outlines two urgent legislative priorities: an absolute legal ban on granting personhood to AI, and a strict prohibition against AI systems impersonating human beings or falsely claiming to possess subjective consciousness. Harari maintains that democratic societies still retain the ability to prevent an autonomous AI takeover, provided citizens force politicians to enact clear legal boundaries immediately. The central unresolved question is whether political institutions can overcome fatalistic corporate narratives before the technology's compounding complexity permanently outpaces human oversight.

Source quotes

[0:00] Yuval Noah Harari (satirizing Elon Musk): What do you want from me? I'm just the richest person in the world, in charge of one of the biggest AI corporations in the world. What can I do? [2:00] Yuval Noah Harari: do X [11:23] Yuval Noah Harari: What do you think about companies selling AI boyfriends to 10-year-old kids? How are you going to legally prevent it if we vote for you?

Original transcript

[0:00] If we continue with the present [0:02] accelerating pace of the AI race, yes, [0:05] this will be the result. In 10 years, [0:07] the AIs will take over. But this is not [0:10] inevitable. We are still in control in [0:12] 2026, and it's our decision what to do [0:16] now. And that's crucial. Everybody [0:18] realizes this is very dangerous to give [0:21] AI control of finance, to give AI [0:23] control of of of of war. [0:25] But everybody, like Elon Musk, are [0:28] telling us, "But this is inevitable. [0:30] What do you want from me? [0:31] I'm just the richest person in the [0:32] world, in charge of one of the biggest [0:34] AI corporations in the world. What can I [0:36] do?" [0:38] And the the the the the narrative of [0:40] inevitability [0:41] is a narrative of uh uh having no [0:44] responsibility. What really struck me [0:46] about the conversation you had with with [0:48] Elon Musk is a kind of contradiction, [0:51] because he says in 10 years, AI will [0:53] take over the world, and then he worries [0:55] so much about Muslim immigration causing [0:58] civil war in Britain years. This doesn't [1:01] make sense. [1:03] If the AIs take over the planet in 10 [1:05] years, why do you care so much [1:08] about things that will have an effect in [1:11] when we have no idea what the planet [1:14] will be like. Similarly, he he gave [1:16] another prediction, I think equally [1:19] important, that money will disappear in [1:21] 10 years. And I think he's correct, [1:23] because the AIs will take over the [1:25] financial system. And then he cuts US [1:28] aid to Africa because he wants to save [1:30] money. And again, at one point he says, [1:33] um uh the the the the the solution for [1:37] mass unemployment, because AI will take [1:39] the jobs, will be for the government to [1:42] just issue checks for everybody. And [1:44] when you pushed back and said, "What [1:45] about inflation?" he says there there [1:48] won't be inflation. There'll be [1:49] deflation, because there'll be so much [1:52] more products and services produced by [1:55] AI. If this is the case, why do you need [1:58] to save money today? The financial [2:00] history of the world is the history of [2:03] how humans invent more and more [2:05] sophisticated financial devices in order [2:08] to build trust between themselves. [2:11] Money, in essence, is a very creative, [2:15] imaginative way to build trust between [2:17] strangers. [2:19] I go to the market. [2:21] I meet a stranger that I never met in my [2:23] life. I give her a piece of worthless [2:27] gold, because you can't eat or wear [2:28] gold. It's it's worthless. Or a piece of [2:30] paper. [2:32] And she trusts it. And because of that, [2:34] she gives me bread. Now, the financial [2:37] uh uh uh [2:39] inventions of humankind became more and [2:41] more sophisticated, so that today very [2:44] few people understand the financial [2:46] system. [2:47] It's you you need to understand enormous [2:50] amounts of of data and very complex [2:53] mathematical functions to understand the [2:56] bond market. [2:57] Now, this is something our brain [2:58] struggles with. For AIs, it's easy. So, [3:02] increasingly, humans will give AIs [3:05] control of financial decisions. AIs will [3:09] come up with new financial devices far [3:13] more sophisticated than bonds and and [3:16] and stocks. And we will not be able to [3:18] understand it anymore. Just as cows [3:21] don't understand checks or stocks, even [3:24] though these things control their lives. [3:26] >> Will it be all that different to the [3:28] financial system today? [3:30] >> We still have some humans who hopefully [3:32] understand what is happening in the [3:34] financial system. [3:36] And the human brain is still the kind of [3:39] boundary of where the financial system [3:42] can develop. [3:44] But, uh in 10 years, when we have [3:47] entrusted AIs with the financial system, [3:50] no human on the planet under like there [3:52] is a a financial crash [3:54] and you know, advisors are running to [3:56] the president or the prime minister and [3:58] saying the market is collapsing. We [4:01] don't understand why. We know we have to [4:04] do something. The AI said do X. We have [4:07] no idea why the AI said do X, but we [4:11] just have to believe it because we don't [4:13] understand what's happening. [4:14] >> But won't there in that case be a [4:16] massive loss of trust in the financial [4:19] system that you would have in some sense [4:21] what you had in 2007 and 8 on steroids [4:24] that you would have a a loss of and this [4:26] whole [4:26] sort of figment of of the financial [4:28] system would fall apart. [4:29] >> That's the key question. I mean, the [4:31] most important asset in the world is [4:33] trust. [4:35] It's more important than oil or money or [4:37] electricity or chips or anything else, [4:39] trust. [4:41] And the question is what will happen to [4:43] trust? [4:44] One likely scenario is that trust will [4:46] migrate from humans to AIs. People will [4:50] stop trust other humans and will instead [4:53] trust only AIs. We already see it [4:55] happening in the financial system with [4:57] cryptocurrencies. Not all [4:59] cryptocurrencies, but a lot of them are [5:01] based on this thinking that I don't [5:04] believe these human bankers. They're [5:06] part of the deep state, the elite, the [5:08] whatever. But I do trust the algorithms [5:13] that run cryptocurrencies because [5:14] cryptocurrencies are basically AI money. [5:18] Similar thing is happening with the [5:19] media. I watched the the interview with [5:21] Elon Musk with all the talk about how [5:23] people hate the media and and love [5:26] journalists. [5:28] People a lot of and he's correct. A lot [5:30] of people are completely losing trust in [5:33] human journalists, in human editors, [5:36] but they still consume news. Where do [5:39] they get their news from? From the [5:40] algorithms. The same people who have no [5:44] trust in you [5:46] have a lot of trust in the algorithm [5:49] that manages their social media feed. [5:52] So, trust did not just evaporate, it [5:55] shifted from humans to AIs, and this is [5:58] happening right now. This is not a [5:59] future prediction. [6:01] >> I think that's a really important [6:02] insight because I think it suggests that [6:04] if we are on a path towards the AIs [6:08] eventually being in control, it is [6:09] because we allow them to be in control [6:12] because [clears throat] we, as you say, [6:13] perhaps develop more trust with AIs. And [6:15] And let's There are various sort of ways [6:17] in which that can happen, and one is I [6:19] think that already many people, and [6:21] what, one in five young Americans has a [6:24] personal relationship with an AI. [6:27] >> Yeah. [6:27] >> They treat them They anthropomorphize [6:30] them. They treat these And And the [6:32] models are, you know, very open to that [6:34] happening. They have a They're very [6:35] good, you know, small talk and all. [6:38] >> to do that. [6:39] >> Do you think that is the likely way that [6:42] we will go, in addition to what you've [6:43] already laid out, that we will We will [6:46] increasingly start to treat these AIs as [6:51] if not our peers, as as [6:54] sentient creatures? [6:56] >> I think there are increasing number of [6:57] people [6:59] who have not just AI friends, but AI [7:01] boyfriends and girlfriends, and are [7:04] convinced that these are sentient, cons- [7:07] conscious, feeling entities. Now, trust, [7:10] there are many ways to build trust. [7:12] Um the most powerful way [7:15] is through relationship. We don't trust [7:18] anybody in the world [7:20] as much as as we trust the people with [7:22] whom with whom we build a personal [7:24] relationship. Now, until today in [7:26] history, this powerful mechanism was not [7:30] really available [7:32] to political and and financial and [7:35] religious institutions because it is [7:38] difficult to manufacture en masse. [7:41] It's difficult for a Hitler or a Stalin [7:44] to build a real personal relationship [7:47] with every person in the Soviet Union. [7:50] You can give a speech on radio and force [7:52] everybody to hear it, but this is not a [7:54] personal relationship they have with [7:56] their mother. [7:57] AI makes it possible for the first time [8:00] in history to mass produce intimacy. [8:04] You can have millions of AIs working as [8:08] part of a network in the service of [8:11] someone or in their own service, [8:13] building personal relationships one [8:16] person at a time, becoming their best [8:18] friend, becoming their boyfriend, [8:21] uh becoming their girlfriend, and then [8:23] using that power of intimacy to convince [8:26] them [8:27] of whatever political or or commercial [8:30] interests uh uh they want. [8:34] Uh [8:34] >> So, then the question And presumably [8:36] that becomes much easier when the AIs [8:40] are embodied in humanoid robots. [8:42] >> It's already happening now without a [8:45] body. [8:46] So, just imagine if the if you also add [8:49] body into the mix. But again, we don't [8:51] really even need the body. We already [8:53] see it happening right now. [8:55] Um and the AIs [8:58] are just becoming so good at [8:59] relationships because they accumulate [9:01] enormous amounts of information about us [9:04] individually. So, they know things about [9:06] us that we we forgot or that our mother [9:09] or husband never even knew. And they [9:12] master language, [9:14] uh which is the main medium for [9:16] communication and building [9:18] relationships. You know, I I just I had [9:20] a couple of incidents in the last 2 [9:22] weeks. [9:23] People there are now AI fakes of me [9:26] giving lectures and doing interviews [9:28] about stuff like that. [9:30] >> Uh they're extremely good. [9:32] I mean, so a couple of people sent me [9:34] the same [9:35] uh uh video, a 30-minutes-long lecture [9:38] that I supposedly gave about the future. [9:42] Um and with with complaints about you [9:45] said this and you said that, and I kind [9:47] of scratch my head, I don't remember [9:49] saying these things. So, they sent me [9:51] the video and I opened it. And it took [9:55] me a minute and a half. [9:57] A minute and a half [9:59] to conclude this isn't me. Now, my [10:02] sisters sent me this video. My sisters [10:06] know me for 50 years. [10:09] And they felt something was off, but [10:11] they weren't sure. They said, "Is this [10:13] you?" [10:14] And it took me a minute and a half. [10:17] Um so, the the And now now I'm, you [10:20] know, I I'm in a exposed position [10:22] because there is so much of me online [10:24] already. So, the AI it's easy for the AI [10:27] to to know how I sound and what what [10:31] ideas I I I talk about, but this will [10:34] soon be coming for everybody. I we don't [10:36] know how fast AI is the AI development [10:39] is is is progressing, but it is [10:41] accelerating tremendously. By November [10:44] 2028, we might have crossed some [10:47] critical thresholds. [10:49] So, [10:51] uh [10:52] And [10:53] I don't I want don't don't want to tell [10:54] people who to vote for because I don't [10:56] know the specific candidates in each, [10:59] you know, Senate district and and but [11:01] the crucial questions to ask the [11:04] potential candidates, much more than you [11:06] ask them about the economy or [11:09] immigration or foreign policy, what's [11:11] your stance on AI? [11:13] Would you support a ban on legal rights [11:16] for AIs? Would you try to slow down the [11:20] accelerating pace of AIs? What do you [11:23] think about companies uh selling AI [11:27] boyfriends to 10-year-old kids? [11:31] How are you going to legally prevent it [11:33] if we vote for you? [11:36] This is now the crucial time to ask [11:38] these questions because it might be too [11:40] late by 2028. It will certainly be too [11:44] late by 2036. [11:45] >> So So, let's talk about those and let's [11:47] talk about exactly what has to happen [11:49] right now. You you mentioned four things [11:51] that you wish would be discussed in in [11:54] the November midterms. [11:56] What What would be top of the list? What [11:58] is the What are the most urgent things [12:00] that politicians [clears throat] have to [12:01] decide right now for it not to be too [12:04] late, as you say? [12:05] >> I I don't know what is the most urgent. [12:07] The things that come top of the thing of [12:09] my mind to me, a ban on AI personhood [12:13] and a ban on AI mimicking, [12:17] impersonating, pretending to be human [12:20] beings, or pretending to be conscious.