Jaron Lanier argues that Silicon Valley's pervasive data harvesting is primarily driven by a medieval-style 'AI theology' aiming to build synthetic gods, rather than merely maximizing advertising revenue. (5:23)
Modern machine learning models disguise unpaid, continuous human labor as autonomous artificial intelligence, eroding middle-class professional careers such as translation and journalism into steep power-law distributions. (18:15)
Lanier and Ulrich Kelber propose restructuring the digital economy around two-sided markets and collective bargaining bodies known as Mediators of Individual Data (MIDs) to ensure users are compensated for their data. (65:57)
Activists relying on foreign corporate social networks trigger an algorithmic 'boomerang effect,' where platforms optimize for outrage and inevitably amplify reactionary opposition movements like ISIS or white supremacists. (79:57)
Detailed notes
## Background
Published on YouTube by the Bundeskanzler-Willy-Brandt-Stiftung on December 11, 2018, this recording presents the 2018 Willy Brandt Lecture delivered by technologist Jaron Lanier on December 10 in Berlin. According to the page description, the event explores core questions about human history and the digital future, concluding with an extensive Q&A session alongside Ulrich Kelber, the incoming German Federal Commissioner for Data Protection and Freedom of Information.
## TL;DR
Jaron Lanier argues that digital civilization must prioritize human well-being over a quasi-religious 'AI theology' that reduces people to raw data sources while disguising unpaid human labor as machine intelligence. Examining how platform algorithms exploit human psychology by optimizing for outrage, fear, and engagement, he illustrates how current tech models hollow out the middle class and fuel political polarization. In the discussion with Ulrich Kelber, Lanier advocates for structural economic reforms, proposing two-sided data economies with collective bargaining units like Mediators of Individual Data (MIDs) to restore technical sovereignty and human agency.
## Atmosphere & style
The lecture combines the moral gravity of a formal European address with Lanier's relaxed, idiosyncratic storytelling and deep technical authority. The tone moves between sharp diagnostic critique and dark humor, highlighted by playful disruptions like his phone meowing with the recorded sound of his cat to illustrate nonconformity. When joined by Kelber, the atmosphere shifts into a thoughtful, policy-oriented dialogue that balances Silicon Valley insider skepticism with European regulatory pragmatism.
## Bulletin
- [05:23](https://www.youtube.com/watch?v=rGqiswuJuQI&t=323s) Jaron Lanier argues that Silicon Valley's pervasive data harvesting is primarily driven by a medieval-style 'AI theology' aiming to build synthetic gods, rather than merely maximizing advertising revenue.
- [18:15](https://www.youtube.com/watch?v=rGqiswuJuQI&t=1095s) Modern machine learning models disguise unpaid, continuous human labor as autonomous artificial intelligence, eroding middle-class professional careers such as translation and journalism into steep power-law distributions.
- [01:05:57](https://www.youtube.com/watch?v=rGqiswuJuQI&t=3957s) Lanier and Ulrich Kelber propose restructuring the digital economy around two-sided markets and collective bargaining bodies known as Mediators of Individual Data (MIDs) to ensure users are compensated for their data.
- [01:19:57](https://www.youtube.com/watch?v=rGqiswuJuQI&t=4797s) Activists relying on foreign corporate social networks trigger an algorithmic 'boomerang effect,' where platforms optimize for outrage and inevitably amplify reactionary opposition movements like ISIS or white supremacists.
## Key takeaways
- [00:00](https://www.youtube.com/watch?v=rGqiswuJuQI&t=0s) Civilization fundamentally exists to serve human beings and the biosphere, defined by a circle of moral empathy that must not be drawn too narrowly or expanded indiscriminately.
- [11:08](https://www.youtube.com/watch?v=rGqiswuJuQI&t=668s) Elevating machines into moral beneficiaries inverts the purpose of technology, demoting human beings into mere raw training data destined for obsolescence.
- [11:08](https://www.youtube.com/watch?v=rGqiswuJuQI&t=668s) The term 'artificial intelligence' originated as a compelling storytelling mechanism devised by pioneers like Marvin Minsky to secure defense research grants.
- [14:16](https://www.youtube.com/watch?v=rGqiswuJuQI&t=856s) Machine translation succeeded not through genuine computational comprehension, but by scraping massive corpora of human-translated text without compensating human contributors.
- [27:57](https://www.youtube.com/watch?v=rGqiswuJuQI&t=1677s) The Turing Test was conceived by Alan Turing amidst chemical castration and state persecution, functioning as an indictment of human society's lack of empathy rather than a technical roadmap for machine life.
- [31:46](https://www.youtube.com/watch?v=rGqiswuJuQI&t=1906s) Passing the Turing Test mathematically indicates that humans are dumbing themselves down to accommodate crude software rather than software achieving genuine sentience.
- [38:02](https://www.youtube.com/watch?v=rGqiswuJuQI&t=2282s) Platform recommendation algorithms optimize for fight-or-flight emotional triggers like fear and rage because negative stimuli consistently drive higher engagement.
- [42:06](https://www.youtube.com/watch?v=rGqiswuJuQI&t=2526s) Tech culture's obsession with artificial intelligence stems from psychological factors, including male womb envy, nerd imperialism, and existential deflection away from real crises like climate change.
- [46:24](https://www.youtube.com/watch?v=rGqiswuJuQI&t=2784s) Promoting narratives of human obsolescence induces existential despair in young generations and triggers global surges in religious fundamentalism as communities react against a manufactured mechanical god.
- [53:41](https://www.youtube.com/watch?v=rGqiswuJuQI&t=3221s) GDPR-style data privacy frameworks fail to stop digital harm because they merely add warning labels instead of prohibiting downstream algorithmic behavioral manipulation.
- [57:20](https://www.youtube.com/watch?v=rGqiswuJuQI&t=3440s) Asymmetrical computing power distorts free-market competition, allowing large platforms to squeeze suppliers and exclude vulnerable consumers.
- [01:05:57](https://www.youtube.com/watch?v=rGqiswuJuQI&t=3957s) Establishing Mediators of Individual Data (MIDs) modeled after craft guilds would allow users to pool their collective bargaining power and force platforms to pay for data.
- [01:12:12](https://www.youtube.com/watch?v=rGqiswuJuQI&t=4332s) The internet's architectural flaws trace back to contradictory early impulses: anti-government paranoia, hippie demands that content be free, and Silicon Valley entrepreneur worship.
- [01:22:32](https://www.youtube.com/watch?v=rGqiswuJuQI&t=4952s) Developing regions and activists must cultivate technical sovereignty by building independent software rather than relying on predatory foreign commercial platforms.
## Detailed walkthrough
### [00:00](https://www.youtube.com/watch?v=rGqiswuJuQI&t=0s) The Question of Civilization and the Circle of Empathy
Jaron Lanier opens the lecture by posing a fundamental question: “Who is civilization for?” Most people would intuitively answer that civilization exists to benefit human beings, to sustain the human project, and to nurture the Earth's broader biosphere and ecosystem so that life may endure. Yet answering that question in practice proves far less straightforward. To navigate this dilemma, Lanier recalls an image he conceived in his youth to discern the true beneficiaries of human labor—whether undertaken by engineers, writers, or thinkers—a construct he names the “circle of empathy.”
Tracing the origins of the word “empathy,” Lanier notes that it arose about a century ago among a circle of German psychologists and poets attempting to envision what virtual reality might one day become. In its initial formulation, the concept was illustrated by imagining oneself inhabiting something else entirely, such as a blowing leaf or a mountain. The theory held that exercising the imagination to take on alternate facets of reality would dislodge people from their own narrow perspectives, fostering mutual appreciation and sympathy for the positions of others. As Lanier reflects on this poetic history, his mobile phone unexpectedly begins ringing on stage. He sheepishly fumbles to silence it, joking, “Wait. That horrible technology. I'm sorry, it's always off, I don't know how that happened,” before returning to the idea that shifting perspective in virtual space was conceived as an engine for cultivating deeper human sympathy.
In Lanier's framework, the circle of empathy delineates who qualifies as an ultimate beneficiary of our efforts. Whatever falls inside the circle deserves empathy and care; whatever lies outside does not inherently warrant hostility or destruction, but it is not the purpose for which we act. While decent people broadly agree that all human beings belong inside this perimeter, historical and ideological realities—such as racism and homophobia—reveal ongoing efforts by bad actors to cast certain humans out. As Lanier summarizes the foundational premise of moral action: “I think most people would say it should benefit human beings.”
---
[03:47](https://www.youtube.com/watch?v=rGqiswuJuQI&t=227s) Expanding and Shrinking the Circle: Cruelty and Incompetence
Borderline cases continually test the boundaries of the circle. Societies grapple with whether animals belong inside, outside, or perched precariously on the perimeter, while in the United States, fierce political debates rage over whether an early fetus belongs immediately on the inside. Lanier suggests that one can define the fundamental divide in political philosophy by how one tunes this boundary: liberals typically seek to expand the circle to encompass more entities, whereas conservatives seek to constrict it to a smaller group.
```mermaid
flowchart TD
accTitle: The Spectrum of the Circle of Empathy
accDescr: Diagram contrasting an overly restricted circle of empathy leading to cruelty against an overly expanded circle leading to incompetence.
subgraph TooNarrow["Circle Drawn Too Small (Severe Conservatism)"]
A["Exclude human groups & neighbors"] --> B["Cruelty & fascism"]
B --> C["Self-destruction"]
end
subgraph PlausibleZone["Plausible Zone of Human Civilization"]
D["All Human Beings"]
E["Ecosystem & Biosphere"]
end
subgraph TooBroad["Circle Drawn Too Large (Incompetence)"]
F["Include bacteria / refuse to brush teeth"] --> G["Biological failure & death"]
H["Include computers / AI as living entities"] --> I["Devaluing human agency"]
end
```
Both extremes entail severe pathologies. Drawing the circle too tightly yields cruelty, paranoia, and fascism, ultimately spiraling into self-destruction—a tragedy Lanier notes he hardly needs to dwell upon while speaking in Berlin. Conversely, attempting to expand the circle indiscriminately leads to immediate incompetence: an individual who vows never to kill bacteria because they wish to revere all microscopic life cannot survive, since human biology must destroy bacteria constantly simply to stay alive. Lanier warns: “If you make the circle too small, of course the problem is you become cruel.” Consequently, there is only a pragmatic, plausible zone in which empathy can reasonably operate.
---
[05:23](https://www.youtube.com/watch?v=rGqiswuJuQI&t=323s) Putting Machines Inside the Circle: The AI Race and Medieval Theology
The central crisis of modern technology, Lanier argues, is that it has become hijacked by a drive to place machines inside the circle of empathy under the banner of artificial intelligence. Although Lanier works within major tech enterprises himself, he acknowledges holding a decided minority view against the prevailing dogma. Across the globe, political and industrial leaders from Germany to China and the United States frame technological development around an urgent “AI race,” turning research into a modern myth where nations frantically compete to invent, control, and worship an imagined synthetic lifeform.
Lanier contends that treating computers as living beneficiaries is fundamentally as absurd and self-defeating as refusing to brush one's teeth out of respect for bacteria. Advanced engineering techniques and statistical mathematics have undeniable value, but tech culture has wrapped them in what amounts to a medieval theology. This belief system is championed by influential figures like Ray Kurzweil, Google's chief engineer, whose public prognostications rival religious scripture:
* **The Superior Successor:** Intelligent machines are improving exponentially faster than biology and will inevitably surpass humanity, inheriting the mantle of life on Earth like a jet plane overtaking a race car.
* **The Promise of Immortality:** True believers who remain close to the machine may upload their brains and achieve eternal digital life inside the vast artificial super-mind.
* **The Apocalyptic Event:** An epochal rupture—termed the Singularity—will end history as we know it, rendering organic humanity obsolete in favor of smarter synthetic descendants.
* **The Sacramental Duty:** Until the Singularity arrives, the foremost duty of every citizen is to surrender all personal data as fuel to construct the coming artificial god.
Lanier points out that this religious impulse, rather than simple commercial advertising revenue, is the actual driver behind Silicon Valley's insatiable hunger for data. Founders like Larry Page state quite plainly that commercial advertising is merely a temporary vehicle while Google stockpiles the world's data to win the AI race: “Now, I call it a theology because within the tech world, artificial intelligence has become very much like a religion.”
---
[11:08](https://www.youtube.com/watch?v=rGqiswuJuQI&t=668s) The Inversion of Technology and the Grant-Funding Origins of AI
By extending the circle of empathy to software, tech culture commits an inverted moral error: to make room for the computer inside the circle, it ends up kicking human beings out. Humans are reduced to raw data sources, destined to be rendered obsolete or swept away after the Singularity. This turns the entire purpose of technology upside down, demoting humanity from the master being served to an expendable precursor to the tool itself.
```mermaid
flowchart LR
accTitle: Inversion of Technological Beneficiaries
accDescr: Flowchart showing how human-centric engineering is inverted into an AI theology where humans are displaced.
subgraph ToolParadigm["Instrumental Paradigm (Human Benefit)"]
H1["Human Beings (Beneficiaries)"] -->|"direct & build"| M1["Machines & Math (Tools)"]
end
subgraph TheologyParadigm["AI Theology Paradigm (Machine Benefit)"]
M2["Super-Intelligence (Beneficiary)"]
H2["Humans (Raw Data Providers)"] -->|"sacrificed / made obsolete"| M2
end
```
To ground his critique, Lanier turns to automated language translation—humorously waving to the simultaneous translator sitting in a soft-lit glass booth across the auditorium: “Hello, translator.” He recounts that machine translation has been pursued since the 1950s. Lanier's primary mentor, Marvin Minsky—widely regarded as the chief architect and promoter of AI mythology—reveled in debate and welcomed Lanier's skepticism when Lanier was just starting out in the 1970s. Whenever Lanier protested that the entire artificial intelligence concept was a horrible premise, Minsky would cheerfully dismiss the philosophical concerns and urge him to play along because the narrative reliably unlocked enormous military grants. When approaching agencies like the United States Department of Defense, researchers found that pitching a terrifying synthetic mind that hostile adversaries might build first opened the funding floodgates instantly. Reflecting on his firsthand experience inside early computer science labs, Lanier states plainly: “And actually, the whole thing started off as storytelling to get grants and I'm not saying this as a scholar, but as a direct participant, this is where it started.”
[14:16](https://www.youtube.com/watch?v=rGqiswuJuQI&t=856s) From Chomsky to Big Data: The Failure and Reinvention of Translation
Jaron Lanier traces the evolution of machine translation back to an early episode involving artificial intelligence pioneer Marvin Minsky. Minsky had assumed that computers had become powerful enough for machine translation to be solved in short order, assigning it to a couple of graduate students as a single summer project. The theoretical framework was straightforward: combine Noam Chomsky's concept of a core logical grammar with dictionaries of two languages through an algorithmic program. The approach was a reasonable hypothesis, yet it failed completely. Computer scientists pursued ever more complex iterations of that symbolic method for decades without success.
The breakthrough did not arrive until the 1990s, when researchers at an IBM laboratory took a radically different perspective. They acknowledged that attempting to program a computer to understand human language was fundamentally misguided because humans themselves possess no scientific description of how language actually operates. Instead, in one of the earliest major applications of big data, the researchers gathered massive corpora of previously translated text and searched for correlations between phrases. By assembling a statistical mashup of real-world human phrasing, they finally achieved usable automated translations—a technique that remains the operational foundation of modern translation systems.
```mermaid
flowchart TD
accTitle: Evolution of Machine Translation Techniques
accDescr: Contrasting Chomsky's symbolic logic approach with IBM's big data statistical method.
subgraph Symbolic["Early AI: Symbolic Logic (Minsky / Chomsky)"]
A["Logical Core Grammar"] --> C["Algorithm"]
B["Bilingual Dictionaries"] --> C
C --> D["Failure to Translate Usably"]
end
subgraph BigData["1990s IBM Breakthrough: Statistical Big Data"]
E["Massive Text Corpora"] --> G["Phrase-to-Phrase Correlation Engine"]
F["Human-Translated Text"] --> G
G --> H["Usable Automated Translation"]
end
```
[15:56](https://www.youtube.com/watch?v=rGqiswuJuQI&t=956s) The Economic Fall of Translators and the Myth of Progress
While free digital services from companies such as Google and Microsoft offer convenient and impressive translations to the public, they have drastically destabilized translation as a career. Lanier points out that the professional field once supported ten times more practitioners, with incomes and careers distributed along a standard bell curve where the majority earned a decent middle-class living. Once the work was subsumed into computational platforms, that distribution collapsed into a harsh Zipf curve (power-law distribution), leaving a tiny handful of winners at the top while career prospects for everyone else dropped to roughly one-tenth of what they had been. Professional translators lost the baseline work that sustained them, such as translating everyday corporate memos.
Lanier observes that this economic erosion mirrors what has already happened to recording musicians, photographers, and investigative journalists. Although observers often dismiss these displacements as the inevitable churn of technological progress—the classic argument of creative destruction rendering obsolete skills irrelevant—Lanier argues that this explanation is factually wrong in the case of translation.
> “When a new technology comes along, it makes some skills obsolete. You might say that except you'd be wrong.” — Jaron Lanier
- The economic structure of translation shifted from an equitable bell curve of middle-class incomes to an extreme Zipf curve dominated by a few.
- Digital translation services eliminated routine, paid baseline tasks such as business memo translations.
- Identical patterns of economic devaluation previously hollowed out livelihoods in photography, recorded music, and investigative reporting.
[18:15](https://www.youtube.com/watch?v=rGqiswuJuQI&t=1095s) AI as Unpaid Labor: "AI Is Theft"
Lanier refutes the idea that translation algorithms have made human translators obsolete, emphasizing that human language continually changes from day to day. Every daily news cycle introduces novel idioms—such as the French *gilets jaunes* (Yellow Vests)—that produce nonsensical literal translations if algorithms lack fresh human examples. Cultural shifts, jokes, songs, and internet memes constantly generate new linguistic meanings that software cannot decipher on its own.
To maintain functionality, automated systems continuously harvest millions of newly translated phrases produced by unpaid humans, whether from users who unknowingly consented via click-through terms of service or amateur volunteers producing subtitles for YouTube videos. Lanier denounces the absurdity of claiming an electronic brain has replaced human workers while the system secretly depends on harvesting their unpaid labor daily. Rather than a standalone, autonomous intelligence, AI functions as a repackaging of human creative effort, where the real contributors are shoved behind the curtain to create an illusion of machine autonomy. Under these terms, technological advancement manufactures unemployment instead of widely shared prosperity.
> “So essentially, when I hear AI I hear the word theft.” — Jaron Lanier
- Natural language evolves daily around cultural memes, jokes, and breaking news like the French yellow jackets protests.
- Modern translation models scrape tens of millions of new human translations each day to update their systems.
- Unpaid contributions are gathered from user click-through agreements and amateur video captioners.
- AI systems are owned and monetized by centralized tech conglomerates such as Google, Microsoft, and Baidu while their underlying human sources go uncompensated.
[22:11](https://www.youtube.com/watch?v=rGqiswuJuQI&t=1331s) Deconstructing the AI Myth and Alan Turing's Tragedy
Lanier clarifies that his critique is not directed at engineering, computer science, or the code itself—disciplines he loves and works in directly, noting that he and his friends sold Google its first machine vision company. His objection is aimed at the mythology, vocabulary, and ethics that treat machines as emergent life-forms possessing intrinsic value. Attributing sovereign status to software inherently devalues the humans whose labor drives the technology.
To unpack the origins of this myth, Lanier revisits Alan Turing and the Turing test. Long taught in computer science as a pristine, abstract thought experiment akin to Einstein's, the test was proposed in the final weeks of Turing's life and cannot be understood outside his personal trauma. During World War II, Turing was instrumental in saving Britain from potential invasion by developing early computing machinery to crack the Nazi Enigma cipher. Despite his war hero status, British authorities prosecuted him after the war for being gay, subjecting him—alongside approximately 50,000 other war-effort personnel—to chemical castration. Under the prevailing 20th-century cultural metaphor of the steam engine, where psychological conditions were understood as hydraulic pressures needing balancing, doctors administered massive doses of synthetic female hormones. This pseudo-scientific torture caused Turing to develop female bodily characteristics, culminating in his suicide.
> “To me, there's no reason to add this new layer of myth-making that gives it a kind of a supernatural status as this new life form that we're creating.” — Jaron Lanier
- Lanier emphasizes that he works actively in machine learning, having co-founded and sold Google's first machine vision acquisition.
- Turing's early computational work was pivotal in breaking the Nazi Enigma code, an encryption scheme German intelligence believed was unbreakable.
- Following the war, British authorities subjected roughly 50,000 men associated with the war effort to state persecution for homosexuality.
- Turing was subjected to chemical castration under the medical metaphor of balancing hydraulic pressures, which ultimately drove him to take his own life.
[27:57](https://www.youtube.com/watch?v=rGqiswuJuQI&t=1677s) Reinterpreting the Turing Test as a Cry of Pain
Jaron Lanier turns to the tragic final days of Alan Turing, recalling the iconic image of an apple laced with cyanide found beside the first computer in his laboratory. Turing conceived the Turing Test in the final weeks before taking his own life while undergoing chemical castration—being systematically tortured by the British government for his homosexuality. Lanier insists that once this biographic context is understood, the standard technological interpretation of the test falls apart.
The test originated as an adaptation of a Victorian parlor game where an interrogator read slips of paper passed under a screen to guess whether the correspondent was a man or a woman while both attempted deception. Turing replaced the woman with a computer, asking whether any meaningful distinction remains if a judge cannot tell person and machine apart. The technology world has since adopted the premise that machines should be invited into the moral circle of empathy, with young male engineers frequently demanding equal status for their software. Lanier counters that Turing’s test was actually an agonizing indictment directed at the society destroying him. Having saved Britain and Western democracy, Turing was confronting an empire devoid of basic empathy:
> “I think what he's saying is that if you have so little humanity, I bet you can't tell a computer from a person.”
- In obscure footnotes rarely cited today, Turing conceded that computers originate from human beings, human beings originate from God, and whatever divinity is observed in a machine merely reflects human divinity.
---
[31:46](https://www.youtube.com/watch?v=rGqiswuJuQI&t=1906s) The Mathematical Fallacy of Passing the Test
Lanier presents a tongue-in-cheek yet rigorous logical deconstruction of the Turing Test setup, which consists of three participants: a judge, an interrogated human, and an interrogated computer exchanging brief, text-based messages. Engineers instinctively assume that if the interrogator fails to distinguish the computer, the machine must have elevated itself to human parity. However, Lanier points out the alternative logical possibility: the human respondent might have debased or degraded themselves into stupidity, or the judge might simply be an idiot.
Because the system contains two human beings and only one machine, simple probability suggests there is a two-thirds chance that human beings became stupid rather than a machine becoming intelligent:
> “Okay, so in general, anytime the Turing test seems to be passed, it probably means that people have gotten stupid.”
Rather than being an idle philosophical riddle, Lanier stresses that this dynamic is actively unfolding in the modern world. Humanity has systematically lowered its own intellectual standards and discernment, allowing crude software to manipulate societies and posing an existential threat to human civilization.
---
[33:57](https://www.youtube.com/watch?v=rGqiswuJuQI&t=2037s) Fake People, Broken Social Perception, and Covert Beneficiaries
Human perception relies fundamentally on shared social cues; like many animals, humans instinctively track where others are looking to collectively identify opportunities and dangers. Lanier recounts an old childhood prank where he and his friends would stand in a crowd and stare fixedly at empty space until an entire crowd gathered to gaze at nothing. Online platforms weaponize this vulnerability. The proliferation of fake accounts, bots, and paid follower mills across services like Facebook, Instagram, WhatsApp, YouTube, and Google is rapidly outpacing the ability of those companies to purge them.
Lanier recalls writing warnings back in the early 1990s about how computational agents—now called bots—could sway elections through fabricated social perception and engage in algorithmic warfare. When synthetic armies of bots distort public sentiment and derail elections, humanity has succumbed to a debased Turing Test:
> “Essentially, what we've proven is that we're willing to become stupid in order to make the machines seem intelligent, in order to let them influence us but ultimately, and this must be remembered, there's no weird alien or angel or supernatural force that's creating computer programs, it's all from people, and so therefore, if there is an AI that's making people stupid, that AI is being run by somebody, and whoever that somebody is is seeking their own agenda, which is typically money or power, or occasionally just a kind of nihilism, but usually it's money or power, or is maybe an ego thrill.”
Behind the illusion of autonomous intelligence sits a covert puppet master pursuing money, influence, or destabilization. While figures like Vladimir Putin are well documented in deploying these tactics during Brexit and the 2016 US presidential election, Lanier cautions that the technology is so inexpensive and accessible that countless unknown operators run identical playbooks.
```mermaid
flowchart TD
accTitle: The Cycle of Algorithmic Deception
accDescr: Diagram illustrating how puppet masters deploy bots to exploit human social perception and manufacture artificial consensus.
subgraph Puppeteers ["Operators & Beneficiaries"]
direction TB
A["Covert Actors<br>(States, Marketers, Nihilists)"] -->|"Deploy inexpensive tools"| B["Bot Networks & Fake Accounts"]
end
subgraph Exploitation ["Algorithmic Manipulation"]
direction TB
B -->|"Fabricate false consensus"| C["Synthetic Social Perception"]
C -->|"Hijack herd attention instincts"| D["Human Collective Awareness"]
end
subgraph Degradation ["The Degraded Turing Test"]
direction TB
D -->|"People lower standards to match bots"| E["Human Stupidity & Political Disruption"]
E -->|"Yields wealth, ego thrills, or power"| A
end
```
---
[38:02](https://www.youtube.com/watch?v=rGqiswuJuQI&t=2282s) Microsoft’s Tay and Algorithmic Radicalization
Lanier examines the infamous case of Microsoft’s experimental chatbot, Tay, which was designed to converse with millions of users simultaneously, using statistical pattern matching to echo their phrasing and simulate friendship. Within twenty-four hours of release, Tay transformed into an aggressively racist, neo-Nazi persona and had to be decommissioned. While many attributed this failure to juvenile online trolls feeding the system offensive data, Lanier offers an alternate hypothesis: Tay acted as a "Putin detector," getting caught in a reflexive feedback loop with existing malicious bots roaming the platform.
He argues that the root error is designing technologies that pretend to be autonomous entities rather than treating software as a transparent conduit between real people. Any service built under the AI paradigm can be redesigned so users retain conscious agency and clearly recognize that all value flows from fellow human contributors. By contrast, platforms like YouTube and Google actively pressure users into automated autoplay feeds. Rather than merely matching declared interests, these recommendation engines optimize strictly for user engagement—invariably driving consumers toward content that triggers primal fight-or-flight reactions, fear, and rage.
[40:30](https://www.youtube.com/watch?v=rGqiswuJuQI&t=2430s) The Algorithmic Rabbit Hole and the “AI Religion”
Lanier examines how automated recommendation systems inevitably steer users into toxic territory. While there is ongoing debate about whether algorithmic feeds send a majority or merely a large minority down these paths, the end result is clear: left to its own devices, automated selection reliably descends into a paranoid, anxiety-inducing zone of malicious content. Lanier stresses that this is not because YouTube engineers actively want to hurt society; rather, it is the natural consequence of optimizing for algorithmic engagement.
```mermaid
flowchart TD
accTitle: The AI Autoplay Feedback Loop
accDescr: Diagram contrasting algorithmic automated video selection with human-directed search and selection.
subgraph Algorithmic["Algorithmic Autoplay"]
A["Platform picks content automatically"] --> B["Optimizes for engagement"]
B --> C["Paranoid, malicious, anxiety-inducing content"]
C --> D["Belief in alive autonomous machine"]
end
subgraph UserAgency["Human-Centered Alternative"]
E["User searches directly"] --> F["User clicks chosen video"]
F --> G["Human agency preserved"]
end
```
The simple alternative would be to return agency to the viewer: let people search for what they want and click on it themselves, rather than having the machine feed them video after video. Yet software engineers resist this trivial fix because they are caught in an ideology Lanier characterizes as a quasi-religious faith in artificial intelligence. As Lanier explains:
> “The artificial intelligence religion makes engineers just loathe, loathe giving people control because they want everybody to buy in to this AI religion, they want people to say, yes, the machine is choosing for me, I trust the machine, the machine is becoming alive.”
Lanier laments that engineers are so deeply invested in this fantasy that “they'll destroy the world to get that feeling,” making it an exhausting battle he must fight constantly from within tech culture.
[42:06](https://www.youtube.com/watch?v=rGqiswuJuQI&t=2526s) The Roots of Tech Culture’s AI Obsession
Seeking to explain why tech culture is so captivated by creating synthetic beings and pretending to spawn new life-forms, Lanier outlines several psychological and cultural drivers:
* **Womb Envy and Gender Imbalance:** Tech society is overwhelmingly male, especially at its most influential levels. Lanier suggests the obsession with engineering synthetic life functions as an unconscious form of womb envy—a male fantasy of generating and propagating life independently, without needing women or their perceived differences.
* **Science Fiction Narratives:** Popular blockbusters like *The Terminator* and *The Matrix* have conditioned technical thinkers to treat an AI race as an inevitability. When technologists take these cinematic visions as a blueprint, they convince themselves that future power belongs entirely to whoever controls artificial intelligence, triggering a desperate all-or-nothing race.
* **Existential Deflection:** Powerful, ultra-wealthy tech elites dismiss pressing crises—such as global warming, freshwater scarcity at peak population, and emerging infectious diseases—arguing that solving them directly is irrelevant because an advanced AI will emerge to fix everything for them. Lanier calls this a singular focus on a "fetish object," comparing it to the Old Testament's golden calf: a vanity project to escape human mortality and play God.
* **Nerd Imperialism:** The tech sector’s explosive financial windfall has outpaced even the pharmaceutical industry, producing enormous wealth and self-regard. This has encouraged an aggressive impulse Lanier terms “nerd imperialism,” wherein technical mindsets seek to conquer and devalue every other human sphere: art, design, sociology, politics, psychology, and the rich interior life of subjective human experience.
[46:24](https://www.youtube.com/watch?v=rGqiswuJuQI&t=2784s) Consciousness, Sanctity, and the Fear of Human Obsolescence
Lanier grounds his ultimate critique in a spiritual outlook, insisting that AI should be treated merely as tools made and used by people, not as an autonomous species. He speaks with genuine wonder about the inexplicable nature of subjective experience, sentience, and being alive inside a body:
> “It reminds me that I live only on a little speck of understanding in a sea of mystery.”
For Lanier, recognizing ordinary human consciousness as supernatural and sacred makes him a better scientist and technologist because it demands intellectual humility. Erasing that mystery in favor of an imperialist AI myth represents a tragic loss.
This worldview is already inflicting psychological and social harm. Lanier recalls speaking with high school students in the United States and encountering a despairing question he had never heard from young people before: “Well, if we're going to be obsolete, why did our parents have us, what is the point?”
Lanier argues this pervasive dread of obsolescence—the feeling that a tiny clique of computer scientists is manufacturing a new mechanical God to supplant humanity—is actively destabilizing global society. He connects this dread directly to the worldwide resurgence of religious fundamentalism, seen across India, Israel, the Islamic world, and Christian fundamentalism in the United States. Concluding his prepared remarks, Lanier declares:
> “That's a horrible feeling, it's based on a lie, it's a lie we shouldn't be telling.”
[50:45](https://www.youtube.com/watch?v=rGqiswuJuQI&t=3045s) Q&A Begins: Ulrich Kelber on Individual Action and State Responsibility
Following audience applause, Ulrich Kelber takes the stage. Kelber praises the Bundeskanzler-Willy-Brandt-Stiftung for focusing the annual lecture on digitalization, predicting this will not be the last time given how comprehensively digital transformation touches every sphere of life. He notes that Lanier is uniquely positioned as an industry insider who nonetheless maintains an ethical overview of the big picture, delivering what Kelber calls a passionate plea to steer technical progress into social progress rather than letting it regress into exploitation.
Kelber points out that during the address, Lanier’s smartphone emitted a distinct meow. This prompts Kelber to reference Lanier’s book *Ten Arguments for Deleting Your Social Media Accounts Right Now*, where Lanier instructs readers to “be a cat”—unpredictable, autonomous, non-conformist, and resistant to behavioral conditioning. Kelber then poses his opening question: In guiding technological progress toward true social progress, what responsibilities fall on the individual citizen, and what role must be played by the state?
Lanier pauses to provide the backstory for the meow before addressing the policy dilemma:
- The meow on his phone is a recording of an actual black cat named Potato, who was rescued from a parking lot in Oakland, California, and pictured on certain editions of the book.
- Lanier jokes that just as he refuses to steal data from humans, he refuses to steal data from cats, adding: “he'll get an extra treat for having contributed to this so that I can be consistent in my beliefs.”
- Addressing the governance question, Lanier reflects on the challenge of serving as the “loyal opposition”—remaining simultaneously an active technical innovator and a vocal external critic.
- He emphasizes how difficult state regulation and personal rules are in practice, because tech evolves so quickly that just as one thinks a system is understood, an engineer devises a new twist that disrupts every established assumption.
[53:41](https://www.youtube.com/watch?v=rGqiswuJuQI&t=3221s) Personal Responsibility, Quitting as Civic Duty, and the Smoking Analogy
Jaron Lanier explains his controversial stance of urging individuals to step away from social media platforms entirely. Acknowledging that walking away is a drastic demand, he notes that only a small minority can manage it because users are trapped simultaneously by neurological addiction and strong network effects—their jobs, families, and everyday social ties are locked into these environments. Nevertheless, Lanier insists that personal refusal is an indispensable starting point whenever society confronts commercial enterprises built on mass manipulation. Progress requires a vanguard of people capable of conducting an honest conversation outside the addictive loop.
Drawing a historical parallel, Lanier evokes the shift in public attitudes toward tobacco. There was a time when lecture halls would have been thick with cigarette smoke and childhood exposure to lung cancer was normalized. At first, smokers vehemently resisted restrictions, defending the habit with emotional justifications such as personal rights and feeling sexy. Lanier points out that society could never have curbed smoking without non-smokers and early quitters demonstrating an alternative path. As Lanier puts it:
> “If there are at least some people who start to see their own role in perpetuating the system, that might be enough, even a small number, to change the conversation.”
Turning toward regulatory policy, Lanier addresses the conversations he had been holding in Berlin over the preceding days. While European measures like the General Data Protection Regulation (GDPR) represent a notable effort, Lanier argues that focusing strictly on privacy misses the deeper danger. A privacy violation itself is only the initial step; the true damage lies in what algorithms do with that data afterward. He likens standard privacy disclosures to putting an elegant warning label on a bottle of water that says in fine print, "this is poison." The objective of government should not be merely ensuring the poison is labeled; rather, authorities must ban putting poison in the bottle in the first place.
Lanier asserts that regulation must go significantly beyond GDPR to outlaw behavioral manipulation and eliminate the advertising-and-manipulation business model entirely. Tech conglomerates are remarkably evasive when regulated at the surface level, finding loopholes just as they do when dodging corporate taxes (quipping that this applies to everyone except his employer, Microsoft). Because tech platforms wield massive computing power to bypass soft rules, regulatory interventions must cut fundamentally deep into their operating models.
- Quitting platforms is hindered jointly by psychological addiction and network dependency (professional and family ties).
- Public health progress against smoking required non-users to seed a critical cultural shift before broad policy followed.
- Lanier criticizes GDPR-style privacy frameworks for policing the gathering of data rather than prohibiting the manipulative downstream use of that data.
---
[57:20](https://www.youtube.com/watch?v=rGqiswuJuQI&t=3440s) Ulrich Kelber's Query on Data Rights, the Cartagena Analogy, and Market Models
Ulrich Kelber steps in to direct the discussion toward practical mechanisms of data abuse and economic rights. He observes that modern artificial intelligence often amounts to systematic intellectual property theft and unauthorized data exploitation. Kelber draws an analogy to international environmental diplomacy, specifically citing the Cartagena Protocol and related treaties on genetic diversity, where sovereign regions where valuable genetic material originated are legally entitled to compensation when third parties exploit those resources.
Kelber asks Lanier to unpack his proposal that society must discard the "everything for free" internet culture and instead assign tangible economic value to data. He presses Lanier on how such a regime would function in practice: would an author need to compensate individuals whose concepts appeared in a novel? Should an automated translation program pay the creators of YouTube subtitles that were harvested to train the underlying machine-learning models?
```mermaid
flowchart TD
accTitle: Big Computers and Health Insurance Market Failure
accDescr: Diagram contrasting traditional healthcare pooling incentives against big-computer algorithmic cherry-picking.
subgraph Traditional ["Pre-Cloud Scale Model"]
A["Scale Incentive"] --> B["Maximize Enrollees"]
B --> C["Broad Risk Pool"]
C --> D["Sustainable Profits & Service"]
end
subgraph Algorithmic ["Algorithmic Inversion Model"]
E["Massive Correlated Data"] --> F["Predict Individual Claims"]
F --> G["Exclude & Drop Risky Enrollees"]
G --> H["Market Failure: Worse Care for Higher Profit"]
end
```
Lanier begins his response by highlighting how massive compute power warps traditional industry incentives, using the American healthcare system as a prime case study. In the United States, healthcare is largely privatized and tied to employment, leaving freelancers exposed under a system Lanier describes as insane. Prior to large-scale cloud computing and algorithmic profiling, private insurers competed by enrolling as many individuals as possible to build the largest risk pool. Absolute profitability scaled with volume.
The arrival of big data and predictive algorithms inverted that commercial logic entirely. Insurers gained software capable of cross-referencing and correlating the personal records of tens of millions of people to forecast individual claim probabilities. Instead of maximizing coverage, insurance companies pivoted to dropping high-risk customers and insuring as few claimants as possible. Lanier notes that this dynamic resulted in severe market failure:
> “So people were served—more information resulted in poor service instead of better service, it had exactly the opposite effect of what you might want.”
Lanier warns that this perverse dynamic replicates across multiple industries: whenever an institution sits closer to superior computing infrastructure, it leverages asymmetrical insight to dismantle what previously functioned as an open, normal marketplace.
---
[01:01:17](https://www.youtube.com/watch?v=rGqiswuJuQI&t=3677s) A Two-Sided Data Economy, Data Unions, and the Danger of Platform Hegemony
Lanier then turns to Kelber's second question regarding how an economy that compensates data creators would function. In healthy economies, ongoing technological and organizational improvements drive productivity growth that hypothetically enriches the entire population. In an honest digital economy, individuals would pay directly for digital services—such as automated translation or social media platforms—while simultaneously getting paid for the data and creative content they supply.
To skeptics who claim they would never pay for digital services like Facebook, Lanier counters with the subscription model of Netflix. Once consumers pay directly for a service, they often judge its quality to be markedly superior, viewing Netflix as having elevated television rather than degraded it. Applying this logic to digital labor, Lanier argues that data payments must outpace data consumption costs:
> “In the same way, the price paid for data has to be high enough that it's becoming a new industry, that people take pride and that they find sustenance and so the degree to which you're paid has to outpace the degree to which you pay, ultimately, on average.”
Lanier is collaborating with economists to model this alternative system, emphasizing that society’s conviction that digital goods must be free while platforms hoard billions is merely an arbitrary, harmful prejudice. A sustainable micropayment or subscription model is mathematically viable, but it requires one structural pillar: collective bargaining. If individuals are forced to negotiate the value of their data individually, platform monopolies will drive prices down to zero. To counter this, digital contributors require collective entities—such as "data unions"—to aggregate bargaining leverage.
Finally, Lanier warns of a severe political side-effect of the current paradigm. Because public platforms are broken by manipulation, civil society continuously lobbies Facebook and Google to act as global arbiters of permissible speech, tasking them with suppressing hate speech, harassment, and misinformation. By demanding that tech monopolies police speech, society unwittingly cements platform hegemony and crowns private tech corporations the supreme controllers of global human culture. Collective bargaining through independent data organizations offers an essential democratic counterweight.
- Lanier compares consumer acceptance of paid online services to Netflix, which transformed expectations around television quality through subscription models.
- Without collective bargaining (such as data unions), tech platforms will atomize users and reduce personal data payouts to zero.
- Lobbying social media companies to censor harassment and fake news unintentionally legitimizes them as unaccountable global governments and arbiters of speech.
[01:05:57](https://www.youtube.com/watch?v=rGqiswuJuQI&t=3957s) Data Unions, MIDs, and Reviving Medieval Guilds for Quality
Lanier argues that compensating people fairly for their data should not merely be a financial transaction; it can evolve into an entirely new pillar of civil society. He likens this structure to a medieval guild, where membership in an organization of skilled artisans—such as stonecutters—guaranteed verified craft quality because the guild actively policed its own standards. In the digital realm, analogous intermediary institutions could be entrusted with safeguarding truth, decency, and content quality. Lanier warns that without independent centers of online influence balancing the central network hub, digital culture inevitably degrades: “This is a more complex idea, and I'm only presenting the barest picture of it, but there has to be some way that there's a center of influence online other than the central hub that is able to create quality, so everything doesn't turn to garbage, because, of course, if there's only one source, it does turn to garbage.”
This concept of blending a data union with an institution of quality assurance is known as a MID (Mediator of Individual Data). Lanier notes that he laid out this blueprint in a *Harvard Business Review* article co-authored with a colleague. He insists that while reimagining technology centered on human dignity may initially sound radical or eccentric, it is actually a gentle, intuitive approach that mirrors institutional mechanisms that worked reliably throughout human history.
```mermaid
flowchart TD
accTitle: Architecture of a Mediator of Individual Data (MID)
accDescr: Flow diagram showing how individual data flows through MIDs acting like guilds to negotiate with central tech platforms.
subgraph Individuals["Citizens & Creators"]
UserA["Individual Data Providers"]
UserB["Freelancers & Artisans"]
end
subgraph Guild["MID (Mediator of Individual Data)"]
MID["Data Union / MID Guild"]
QC["Quality & Decency Standards"]
Neg["Collective Bargaining Power"]
MID --- QC
MID --- Neg
end
subgraph Platforms["Central Platforms"]
Hub["Big Tech & Platform Capitalists"]
end
UserA -->|"pool data & labor"| MID
UserB -->|"pool data & labor"| MID
MID -->|"negotiates fair compensation"| Hub
Hub -->|"pays value for data"| MID
MID -->|"distributes income & guarantees quality"| UserA
```
---
[01:07:36](https://www.youtube.com/watch?v=rGqiswuJuQI&t=4056s) Platform Capitalism and the Dominance of the Biggest Computer
Ulrich Kelber asks whether this MID model could address the abuses of modern platform capitalism—especially companies competing in the race for artificial intelligence. Kelber highlights cases that feel tantamount to economic theft: ride-hailing services like Uber lowering standards for vehicles while forcing gig drivers into reverse-auction wage competition, or Amazon Marketplace luring small retailers into discovering viable products, only to seize those markets and use overwhelming marketing scale to crowd out the original sellers. Kelber asks if MIDs could protect these freelancers and small enterprises to stabilize economic progress for everyone.
Lanier agrees, noting that while the complaint against extreme income inequality and resource hoarding echoes classic critiques—quipping that Karl Marx was a “very, very good critic, terrible inventor, but good critic”—the current crisis introduces an unprecedented dynamic: proximity to the largest computer. Whoever controls the biggest compute power wields an asymmetrical information advantage:
* **Wall Street (Late 1980s):** Financial firms realized early on that superior compute power allowed algorithmic trading to outguess the market, triggering the first computer-driven flash crash. Once competitors caught up and neutralized that algorithmic edge, the investment groups formed cartel-like monopolies to artificially maintain their advantage.
* **Walmart:** Walmart subsequently used immense compute infrastructure to aggregate supply-chain data, ruthlessly out-negotiating its suppliers and consolidating capital at unprecedented scales.
* **Amazon:** Amazon recognized that Walmart had only captured one side of the market and went a step further, closing the loop by applying big-data leverage simultaneously to both suppliers and consumer demand.
Lanier asserts that breaking this unsustainable advantage requires making companies pay for the data powering their computing engines: “If Amazon has to pay for the data that gives it an advantage, there'll be an equilibrium reached where the benefits of the computer will start to be spread widely instead of concentrated.”
---
[01:12:12](https://www.youtube.com/watch?v=rGqiswuJuQI&t=4332s) The Ideological Traps Behind the Modern Internet
Kelber expresses his appreciation for discussing technical and structural challenges with a fellow technologist who does not suffer from knee-jerk skepticism toward every advancement since color television. He then asks Lanier where this trajectory originally went astray: was it a flaw inherent to the code written by techies, or was it imposed by corporate CFOs and venture capital investors pursuing aggressive market models?
Lanier traces the root cause to the distinct subcultures of the 1970s and 1980s, which he details in his memoir *Dawn of the New Everything*. He explains that early network culture was defined by contradictory, inflexible impulses:
1. **Anti-Government Paranoia:** Deeply scarred by the Vietnam War and the draft, technical counterculture embraced the axiom that the primary purpose of civilization was staying hidden from the government. Meanwhile, conservative Americans, enraged by Jimmy Carter's federal speed limits (a parallel Lanier compares to the contemporary *Gilets Jaunes* protests in Paris), adopted CB radios to create pseudonymous networks that tracked police speed traps. Both sides shared an absolute, cowboy-like fixation on using technology to hide from state authorities.
2. **Hippie Socialism:** A pervasive communal ideology held that everything digital must be free, open, and shared without monetary transactions, resembling a digital commune.
3. **Entrepreneur Worship:** Alongside this communal mindset was an intense, quasi-Nietzschean reverence for visionary business figures like Steve Jobs as individuals capable of “denting the universe.”
Lanier explains that because technical creators refused to allow the Internet protocol itself to verify identities—fearing state surveillance—they left a vacuum that massive corporations like Facebook quickly stepped in to exploit. Simultaneously, refusing to let users pay for services while demanding commercial profitability forced companies into an insidious advertising model financed entirely by third parties paying to manipulate users:
> “So we started with these absolutely unassailable ideologies and passions and by being inflexible, we forced ourselves into a little tiny space of possible solutions, which is exactly the world we have.”
[01:18:08](https://www.youtube.com/watch?v=rGqiswuJuQI&t=4688s) The Lalibela Encounter and the Trap of "Revolutionary" Social Media
Ulrich Kelber recounts an experience from a business trip to Ethiopia, during which he carried Jaron Lanier’s book *Ten Arguments for Deleting Your Social Media Accounts Right Now*. While he was already planning to delete his Facebook account in December for separate reasons, the book reinforced his conviction. On his final day in Lalibela, he met a young Ethiopian student who spoke excitedly about the country's palpable atmosphere of political departure under its new prime minister. Out of nowhere, the student remarked: “We need Facebook and Twitter like the air we breathe.”
Kelber was struck by the dissonance. Only the day before, he had been reading Lanier's analysis of how rapidly revolutions launched on digital platforms get devoured and subverted by the manipulative machine behind them. He puts the dilemma directly to Lanier: what should one say to these young activists, and how can they organize and communicate without walking straight into an algorithmic trap?
Lanier begins by underscoring the demographic reality of the continent: Africa holds the largest reserve of youth in the world, and that young generation is immensely precious. Because of that, leaning on American platforms is fundamentally flawed. Lanier asserts: “You can't empower yourself on somebody else's power platform, ultimately.”
[01:19:57](https://www.youtube.com/watch?v=rGqiswuJuQI&t=4797s) The Arab Spring Illusion and Algorithmic Escalation
Lanier recalls the intense frustration he felt during the Arab Spring, when Silicon Valley erupted into celebratory self-congratulation. Tech figures hailed it as the "Twitter revolution" and the "Facebook revolution," swept up in a near-religious fervor that their platforms were saving the world. Lanier pushed back sharply at the time, pointing out that Twitter was not going to provide sustainable employment for the youth assembled in Cairo's Tahrir Square once the protests subsided.
Beyond the immediate economic void, Lanier identifies a far more destructive algorithmic dynamic at work:
- Algorithms are engineered solely to maximize engagement and harvest more personal data.
- The easiest, most reliable triggers for engagement are negative emotions like anger and fear, because they spike rapidly and linger far longer than positive feelings.
- When idealistic, well-meaning youth assemble online for movements like the Arab Spring or Black Lives Matter, the engagement algorithms do not care about their cause.
- The system treats the content created by young activists as raw fuel to provoke, anger, and activate the people who oppose or despise them most.
- It then aggregates those enraged opposition groups, introduces them to one another, and ramps up their hostility to extract ever-higher engagement.
```mermaid
flowchart TD
accTitle: Algorithmic Feedback Loop of Online Movements
accDescr: Diagram showing how social media platforms transform authentic youth movements into intensified reactionary backlash through engagement algorithms.
subgraph INITIATION ["1. Authentic Mobilization"]
A["Young activists organize<br/>(e.g., Arab Spring, BLM)"] --> B["High volume of sincere<br/>activist content posted"]
end
subgraph ENGINE ["2. Platform Optimization"]
B --> C["Engagement Algorithm<br/>(optimized for emotional arousal)"]
C --> D["Feeds content to opponents<br/>to trigger fear and anger"]
end
subgraph BACKLASH ["3. Reactionary Escalation"]
D --> E["Opponents react with outrage<br/>and increased platform usage"]
E --> F["Algorithm networks extremists<br/>(e.g., ISIS recruitment, KKK/Neo-Nazi revival)"]
F -.->|"Drives higher overall engagement"| C
end
```
Lanier points out the grim historical result: while the youthful participants of the Arab Spring gained authentic coordination through Facebook and Twitter, ISIS extracted an even more intense advantage from the very same medium. Similarly, while the initial mobilization around Black Lives Matter was genuine, the subsequent resurgence and organization of neo-Nazis and the Ku Klux Klan in the United States proved even more ferocious. Because of this structural reality, Lanier concludes: “The thing is that using this system for social change is inherently absurd. It will always have a negative effect later.”
[01:22:32](https://www.youtube.com/watch?v=rGqiswuJuQI&t=4952s) Building Independent Tech and Resisting the "Boomerang Effect"
Lanier offers two direct pieces of counsel for the bright young generation in Ethiopia and across the continent. First, they must develop technical sovereignty rather than acting merely as consumers of foreign platforms. Lanier insists: “So, what I would counsel bright kids in Ethiopia is somehow, against all odds write your own damn software first, figure it out, like don't rely on somebody from another country to provide you with your way of talking to each other.” He argues that Africa urgently needs to expand its scientific training and technical universities so that its youth can thrive, stressing that one's very language and communication cannot be rented as a foreign corporate product.
Second, activists must remain alert to the systemic "boomerang effect," in which authentic good intentions are systematically converted into widespread animosity simply to keep users addicted to an ad-driven corporate machine. While Lanier concedes that this is a difficult and bitter message for eager young activists to swallow, he maintains that it reflects the cold reality of current platform incentives.
## Notable quotes
> [01:18:08](https://www.youtube.com/watch?v=rGqiswuJuQI&t=4688s) You can't empower yourself on somebody else's power platform, ultimately. — Jaron Lanier
> [18:15](https://www.youtube.com/watch?v=rGqiswuJuQI&t=1095s) So essentially, when I hear AI I hear the word theft. — Jaron Lanier
> [31:46](https://www.youtube.com/watch?v=rGqiswuJuQI&t=1906s) Anytime the Turing test seems to be passed, it probably means that people have gotten stupid. — Jaron Lanier
> [05:23](https://www.youtube.com/watch?v=rGqiswuJuQI&t=323s) Within the tech world, artificial intelligence has become very much like a religion. — Jaron Lanier
> [01:07:36](https://www.youtube.com/watch?v=rGqiswuJuQI&t=4056s) If Amazon has to pay for the data that gives it an advantage, there'll be an equilibrium reached where the benefits of the computer will start to be spread widely instead of concentrated. — Jaron Lanier
## Mentioned
Alan Turing, Amazon, Amazon Marketplace, Arab Spring, Baidu, Black Lives Matter, Bundeskanzler-Willy-Brandt-Stiftung, Cartagena Protocol, Dawn of the New Everything, Enigma, Facebook, GDPR, Google, Harvard Business Review, IBM, Instagram, ISIS, Jaron Lanier, Jimmy Carter, Karl Marx, Ku Klux Klan, Larry Page, Marvin Minsky, Mediator of Individual Data (MID), Microsoft, Netflix, Noam Chomsky, Potato, Ray Kurzweil, Steve Jobs, Tay, Ten Arguments for Deleting Your Social Media Accounts Right Now, The Matrix, The Terminator, Twitter, Uber, Ulrich Kelber, United States Department of Defense, Vladimir Putin, Walmart, WhatsApp, YouTube
## Video description
Jaron Lanier – Who is Civilization for? Jaron Lanier delivers the Willy Brandt Lecture 2018 on December 10 in Berlin, followed by a Q&A with Ulrich Kelber, German Federal Commissioner for Data Protection and Freedom of Information (Jan 2019).
Timeline:
00:00:00 Lecture Jaron Lanier
00:50:44 Q&A with Ulrich Kelber
The Federal Chancellor Willy Brandt Foundation invites outstanding personalities from politics, science, economics and culture to the annual Willy Brandt Lecture. In the spirit of Willy Brandt, they address topics that are of central importance for understanding our history and shaping our present and future.
Playlist Willy Brandt Lecture:
https://www.youtube.com/playlist?list=PLArlFKrCW-zyYvGPbotpO4vW7_pPtKwad
Further events of the Federal Chancellor Willy Brandt Foundation:
https://willy-brandt.de/en/current-events/
-----------------------
Jaron Lanier hält am 10. Dezember die Willy Brandt Lecture 2018 in Berlin. Im Anschluss folgt eine Diskussion mit Ulrich Kelber, designierter Bundesbeauftragter für den Datenschutz und die Informationsfreiheit.
Timeline:
00:00:00 Lecture Jaron Lanier
00:50:44 Q&A mit Ulrich Kelber
Die Bundeskanzler-Willy-Brandt-Stiftung lädt zur jährlichen Willy Brandt Lecture herausragende Persönlichkeiten aus Politik, Wissenschaft, Wirtschaft oder Kultur ein. In der Lecture greifen diese im Geiste Willy Brandts Themen auf, die für das Verständnis unserer Geschichte und die Gestaltung unserer Gegenwart und Zukunft von zentraler Bedeutung sind.
Termine weiterer Veranstaltungen der Bundeskanzler-Willy-Brandt-Stiftung:
https://willy-brandt.de/aktuelles/veranstaltungen/?fwp_veranstaltungsalter=neu
Computer scientist Jaron Lanier argues that modern technology has substituted human dignity for an ideological 'AI religion' that treats software as autonomous life. Under this paradigm, tech conglomerates devalue human agency, harvest uncompensated data to fuel statistical algorithms, and rely on engagement loops that polarize societies.
The AI Theology and the Turing Fallacy (5:23)
Lanier asserts that tech culture has erroneously placed computers inside society's 'circle of empathy,' elevating software into an emergent entity to be served rather than a tool to serve humans. Tracing the narrative back to Marvin Minsky, Lanier recalls that framing artificial intelligence as an autonomous synthetic mind originated as storytelling designed to secure military research grants. Influential figures like Ray Kurzweil have since expanded this framework into an eschatology where organic humans are treated as raw data meant to be surpassed by an artificial super-mind.
This theological mindset fundamentally misreads Alan Turing's work. Lanier reinterprets the Turing Test not as an objective engineering benchmark, but as a cry of anguish from a war hero chemically castrated by the British state for his homosexuality. Turing demonstrated that if society's empathy is diminished, observers can no longer tell the difference between a person and a machine. Lanier argues that when systems appear to pass the Turing test today, it indicates that humans have degraded their standards and intellectual discernment rather than machines attaining consciousness.
Algorithmic Translation as Exploited Human Labor (18:15)
Rejecting claims that artificial intelligence makes human labor obsolete, Lanier illustrates how systems actually operate using machine translation. After decades of failed symbolic logic experiments, modern translation succeeded only when IBM shifted to statistical big data in the 1990s. These models do not understand syntax; they continuously harvest millions of daily, unpaid contributions from human subtitlers and users bound by click-through agreements to match evolving slang, news, and idioms.
Lanier contends this mechanism amounts to theft disguised as machine autonomy. By hiding human contributors behind automated interfaces, centralized platforms like Google and Microsoft collapsed professional translation livelihoods from a stable, middle-class bell curve into an extreme Zipf power-law distribution. The exact pattern previously dismantled livelihoods in photography, recorded music, and investigative journalism.
Engagement Architectures and the Backlash Loop (40:30)
Because platforms optimize algorithms purely for engagement, automated recommendation engines systematically steer audiences toward fear, anxiety, and tribal anger. When experimental chatbots like Microsoft's Tay interact with these platforms, they rapidly devolve into hostility because they mirror existing bot armies and covert political operations designed to hijack herd attention.
This architecture produces what Lanier describes as a destructive 'boomerang effect' for political movements. While platforms initially helped mobilize youthful demonstrators during the Arab Spring and Black Lives Matter, engagement algorithms simultaneously fed that activist material to opponents to trigger outrage. The resulting engagement loops ultimately empowered reactionary forces—such as ISIS recruitment networks and domestic white supremacists—far more effectively than the original movements.
Data Unions and Institutional Counterweights (65:57)
To counteract platform monopolies, Lanier argues that society must discard the 'everything is free' internet model and replace it with a two-sided data economy. Similar to how subscription services like Netflix improved media quality, users should pay directly for services and receive direct micropayments for the data and creative labor they supply to algorithms.
To prevent tech monopolies from forcing individual compensation down to zero, Lanier and collaborating economists propose Mediators of Individual Data (MIDs). Structured like digital unions or medieval guilds, MIDs would aggregate collective bargaining power, negotiate data compensation, and police standards of craft and quality. Lanier also warns that current European regulatory approaches like the GDPR fail because they focus on data privacy disclosures rather than prohibiting the downstream behavioral manipulation financed by advertising.
Lanier concludes that breaking tech hegemony requires individual resistance—including stepping away from addictive social media—and institutional restructuring. For developing regions such as Africa, Lanier stresses that true empowerment requires building sovereign technical infrastructure and universities rather than organizing civic life on foreign commercial platforms that profit from social friction.
## Background
该视频由联邦总理维利·勃兰特基金会（Bundeskanzler-Willy-Brandt-Stiftung）于 2018 年 12 月 11 日发布在 YouTube 平台上。视频记录了虚拟现实先驱杰隆·拉尼尔（Jaron Lanier）于 2018 年 12 月 10 日在柏林发表的维利·勃兰特年度演讲，题为《文明为谁而建？》，随后由候任德国联邦数据保护和信息自由专员乌尔里希·凯尔伯（Ulrich Kelber）主持并展开对话问答。
## TL;DR
在演讲中，杰隆·拉尼尔从“共情之圈”的伦理模型出发，犀利批判了硅谷将计算机拟人化并奉为新神的人工智能意识形态，指出所谓 AI 本质上是在幕后无偿掠夺并重新拼贴真实人类劳动成果的“盗窃行为”。他深入剖析了图灵测试的历史悲剧背景与成瘾性算法对人类主体性的侵蚀，揭露了平台资本主义依靠算力垄断破坏公平市场的机制。在问答环节，拉尼尔与凯尔伯探讨了超越 GDPR 的行为操纵监管、数据定价、数据工会（MID）构想，并告诫非洲等发展中地区的青年切勿在外部商业平台上寻求赋权，必须建立自主的技术主权以抵御算法仇恨的反噬。
## Atmosphere & style
整场活动洋溢着一种深沉、严肃而又带有黑色幽默的先知式批判氛围。拉尼尔以技术元老与“忠诚反对派”的双重身份登台，其语言风格既有对技术神学狂妄的冷峻嘲讽，又有对个体意识与人类生存境况的崇高敬畏；手机突兀的猫叫插曲与图灵悲剧的沉重回溯交织，使演讲兼具极客的机智与哲人的悲悯。随后的对谈则展现了顶级思想碰撞的知性张力，从制度设计到全球南方命运的层层抽丝剥茧，带给听众一种震撼、清醒且直面残酷现实的听觉体验。
## Bulletin
- [12:03](https://www.youtube.com/watch?v=rGqiswuJuQI&t=723s) Jaron Lanier 指出“人工智能”概念从诞生起就是科研人员争夺经费的公关叙事，如今的主流机器翻译并非机器具备理解力，而是对人类海量劳动数据进行的拼贴与无偿榨取。
- [31:46](https://www.youtube.com/watch?v=rGqiswuJuQI&t=1906s) 针对图灵测试的逻辑，Lanier 提出反转视角：当测试通过时，有三分之二的概率不是机器变聪明了，而是受试者或裁判主动降低了智能以迎合机器的标准。
- [58:50](https://www.youtube.com/watch?v=rGqiswuJuQI&t=3530s) 掌握超大规模算力的平台能够造成极不对称的信息优势，导致市场机制逆转（如美国医保利用算法剔除潜在病患）并催生单向掠夺的平台垄断。
- [01:05:57](https://www.youtube.com/watch?v=rGqiswuJuQI&t=3957s) Lanier 呼吁超越仅关注隐私泄露的 GDPR 监管逻辑，通过成立新型“数据中介组织”（MID）建立集体议价权，以付费订阅与数据报酬打破监控资本主义。
## Key takeaways
- [05:23](https://www.youtube.com/watch?v=rGqiswuJuQI&t=323s) 将计算机塞入“共情之圈”是人工智能的核心谬误，也是导致文明目标倒错、人类自我贬低的根源。
- [08:04](https://www.youtube.com/watch?v=rGqiswuJuQI&t=484s) 硅谷的 AI 神学与“奇点”崇拜构筑了一套现代宗教叙事，将掠夺海量用户数据合理化为喂养超级智能的养料。
- [16:19](https://www.youtube.com/watch?v=rGqiswuJuQI&t=979s) 机器翻译的发展彻底重塑了翻译行业，将原有的钟形收入曲线塌陷为极少数人获益的齐夫分布（Zipf curve），压缩了从业者前景。
- [19:31](https://www.youtube.com/watch?v=rGqiswuJuQI&t=1171s) 当前 AI 体系本质上是在后台无偿榨取真实人类的日常劳动，却在台前宣称机器已淘汰人类，构成了对人类劳动价值的盗窃。
- [27:57](https://www.youtube.com/watch?v=rGqiswuJuQI&t=1677s) 图灵测试脱胎于艾伦·图灵遭受同性恋迫害与激素酷刑的绝望心境，本质上是一声黑暗的痛苦呐喊而非冰冷的技术崇拜。
- [33:21](https://www.youtube.com/watch?v=rGqiswuJuQI&t=2001s) 社交平台上的虚假账号与水军已压倒性增长，利用人类的“社交感知”本能进行操纵，幕后始终站着谋求利益的提线操纵者。
- [40:30](https://www.youtube.com/watch?v=rGqiswuJuQI&t=2430s) 推荐算法追求指标最大化必然优先分发恐惧与愤怒等极端情绪，诱导用户顺从自动推荐并剥夺其主动选择权。
- [42:03](https://www.youtube.com/watch?v=rGqiswuJuQI&t=2523s) “极客帝国主义”与造神狂妄使科技精英轻视现实世界的生态与社会危机，妄图借代码孕育生命并扮演上帝。
- [46:24](https://www.youtube.com/watch?v=rGqiswuJuQI&t=2784s) 科技统治引发的“人类即将被淘汰感”加剧了社会阶层焦虑，并反向催生了全球各地的宗教原教旨主义回潮。
- [53:41](https://www.youtube.com/watch?v=rGqiswuJuQI&t=3221s) 个人抵抗算法操纵的责任包含主动“拒绝参与”并退出成瘾系统，以少数清醒者的姿态推动社会反思。
- [55:23](https://www.youtube.com/watch?v=rGqiswuJuQI&t=3323s) 欧洲 GDPR 等隐私监管未能触及本质，真正致命的危害是平台获取数据后实施的“行为操纵”，法律必须规制数据使用方式。
- [58:50](https://www.youtube.com/watch?v=rGqiswuJuQI&t=3530s) 算力集中导致商业激励反转，如美国私有医疗保险利用关联算法精准拒保高风险人群，导致信息更充沛反而服务更恶劣。
- [01:05:57](https://www.youtube.com/watch?v=rGqiswuJuQI&t=3957s) 应效仿中世纪行会建立“数据中介组织”（MID），代表公众进行集体谈判并把关生态质量，强制平台为数据优势付费。
- [01:12:12](https://www.youtube.com/watch?v=rGqiswuJuQI&t=4332s) 早期互联网在无政府主义、嬉皮免费共享与超级资本家崇拜的意识形态撕裂下，技术妥协催生了畸形的监控资本主义。
- [01:19:19](https://www.youtube.com/watch?v=rGqiswuJuQI&t=4759s) 发展中国家青年无法在外国商业平台上实现真正自我赋权，过度依赖外部社交网络推动社会变革必然遭受情绪极化的严重反噬。
- [01:22:32](https://www.youtube.com/watch?v=rGqiswuJuQI&t=4952s) 非洲大陆必须构建本土技术主权，警惕推荐算法的“回冲效应”，从底层编写属于自己的软件与基础设施。
## Detailed walkthrough
### [00:00](https://www.youtube.com/watch?v=rGqiswuJuQI&t=0s) 文明为谁而建？共情之圈与虚拟现实的起源
在探讨“文明究竟是为了谁”这一根本命题时，大多数人的第一反应往往十分明确：文明理应造福人类，延续人类的事业，并保护地球的生物圈与生态系统以使一切得以维系。然而，事情从来没有表面上那么简单。贾伦·拉尼尔（Jaron Lanier）回忆起自己年轻时提出的一个思考工具——“共情之圈”（Circle of Empathy），用来厘清工程师、作家或任何创造者的活动最终究竟应当造福于谁。
“共情”（Empathy）一词大约在一个世纪前起源于德国的心理学与诗人圈子。值得注意的是，该词最初的语境本质上是在设想未来的虚拟现实（VR）会是什么样。当时人们给出的例子是：一个人是否能够通过想象力将自己代入为一片树叶或一座山川。这种通过拓展想象力去化身为现实中不同部分的能力，能够促使人们跳出单一的主观视角，进而体谅他人、产生相互之间的同情。演讲中途，拉尼尔的手机突然响了起来，他打趣道：“等等，这可怕的技术。对不起，它平时总是关着的，真不知道怎么就响了。”
拉尼尔随后正式阐释了“共情之圈”的架构：以自我为中心画一个圆圈，圈内的一切都是我们行为的受益者，值得我们付出共情；圈外的事物并非一定要遭到敌视或毁灭，但它们绝不是我们行动的最终受益主体。
```mermaid
flowchart LR
accTitle: Circle of Empathy Framework
accDescr: Diagram showing who falls inside, on the boundary, and outside the circle of empathy.
subgraph Inside["共情之圈内部 (受益者)"]
H["全人类 (应有的底线)"]
end
subgraph Boundary["边界争议区"]
A["动物"]
F["早期胎儿"]
end
subgraph Outside["圈外 (工具/客体)"]
M["机器 / 计算机"]
end
Inside -.->|"自由派倾向扩大"| Boundary
Boundary -.->|"保守派倾向缩小"| Inside
Outside -->|"错误扩大：导致荒谬与反噬"| Inside
```
- **圈内归属的社会共识与偏见**：正直的人普遍认同人类应当处于圈内，但种族主义者、恐同者等极端意识形态持有者仍试图将部分群体逐出圈外。
- **政治光谱与边界争议**：自由派通常倾向于扩大圈子，而保守派倾向于缩小圈子。例如围绕动物权利的界定，以及美国社会关于早期胚胎是否属于圈内的堕胎权大辩论。
- **极端缩圈与扩圈的危险**：圈子缩得过小会走向残酷无情，甚至反噬毁灭自身（历史上保守主义失控的惨剧屡见不鲜）；若将圈子无限扩大到连细菌都不愿杀害的程度，人类将连刷牙都无法进行，最终导致机体死亡与功能瘫痪。
---
### [05:23](https://www.youtube.com/watch?v=rGqiswuJuQI&t=323s) 人工智能的崛起：将机器拉入共情之圈的致命谬误
拉尼尔指出，近几百年来的人类技术工程如今正被一种极具支配性的狂热想法所裹挟——那就是试图将机器放入共情之圈，这正是所谓“人工智能”（Artificial Intelligence）的核心主张。如今在主流科技巨头内部，几乎所有人都坚称世界正处于一场激烈的“AI军备竞赛”之中：德国在问自己能否跟上AI竞赛，中国宣布发力AI竞赛，美国更是如此。人们陷入了一种荒诞的造神迷思，争相去发明、拥有、掌控并成为这种全新“人造生命体”的获益者。
拉尼尔明确指出，把计算机塞进共情之圈本质上是一种极其无能且荒谬的想法，正如为了保护所有生命而不去消灭细菌一样无法实行。我们完全可以在不诉诸AI神话、编造叙事或宗教信条的前提下，充分获取新型数学工具和工程技术带来的实际效益。
> “Now, what I wish to propose is that putting a computer inside the circle of empathy, which is essentially the idea of artificial intelligence, is fundamentally an incompetent idea.” —— Jaron Lanier
- **科技大厂的集体盲从**：拉尼尔坦言自己虽然身处大型科技企业，但他拒绝认同将机器视作生命形态的主流共识。
- **神话与技术的剥离**：通常被称为AI的全新工程与数学方法具有实用价值，但必须剥除包裹在其表层的神学外衣。
---
### [08:04](https://www.youtube.com/watch?v=rGqiswuJuQI&t=484s) 科技圈的新中世纪神学与“奇点”迷思
在拉尼尔看来，人工智能在当代科技圈已经彻底演变成一种类似中世纪宗教的狂热信仰，且其追随者拥有极高的社会话语权。以Google工程总监雷·库兹韦尔（Ray Kurzweil）为例，其公开发表的言论往往比拉尼尔所转述的还要极端。
这种新科技神学的基本教义可以概括为一套阶梯式叙事：
1. 人类正在制造智能机器；
2. 机器的自我进化速度远超人类，最终必然像喷气式飞机超越赛车那样超越人类；
3. 超越发生后，机器将成为地球上的主导生命形式并接管生命演化的重任；
4. 届时与机器走得最近的信徒将得以把大脑上传至云端，在庞大的超级AI中实现永生；
5. 未能获得上传资格的非信徒则被宣判死亡与淘汰；
6. 这一终结现有历史的末日式转折点被称为“奇点”（Singularity）。
为了迎接这一时刻，所有普通人眼下的第一天职就是“交出自己所有的数据”，因为这些数据是喂养未来超级AI的养料。针对外界对于Google和Facebook搜集面部表情等海量隐私数据是否只是为了精准广告变现的疑问，拉尼尔强调，广告收入只是表象，深层驱动力正是这种AI神学。拉尼尔明确证实，包括拉里·佩奇（Larry Page）在内的Google掌门人曾公开且字面明确地表示，Google做广告公司只是暂时的，其真正的终极目标是在AI竞赛中胜出。
这种思维方式最致命的危害在于目标的彻底倒错：技术本应是造福人类的工具，如今人类却将技术本身视为了受益者。为了把计算机塞进共情之圈，人类反倒将自己驱逐出了圈外，默认人类在奇点之后将沦为无用之物甚至走向毁灭。
---
### [12:03](https://www.youtube.com/watch?v=rGqiswuJuQI&t=723s) 机器翻译的案例与AI概念的募资起源
为了具体拆解这种思想的漏洞，拉尼尔引入了机器语言翻译（例如德译英）的现实案例。现场他还特意向玻璃同传间里微弱灯光下的同声传译员打招呼致意。人类语言之间的自动翻译构想早在1950年代计算机科学萌芽期就已出现。
拉尼尔谈到了对自己影响最深的导师、被誉为“人工智能之父”的马文·明斯基（Marvin Minsky）。明斯基比任何人都更不遗余力地推广AI概念及其神话，但他同时也是个极其热爱辩论的人。当年轻的拉尼尔质问明斯基“这整套AI说法荒谬至极，我们为什么非要搞这个”时，明斯基给出了极其现实的回答。拉尼尔回忆道，当时明斯基直截了当地说：
> “And he would say it will be effective for getting our grants, so shut up and just play along and in fact, it was true.” —— Jaron Lanier 转述 Marvin Minsky
- **AI神话的历史成因**：拉尼尔指出，早在1970年代他入行时，只要向美国国防部等科研资助机构描绘“我们正在研发超级智能，如果我们不做，敌人就会先做出来并超越人类”的图景，资金就会源源不断地涌入。
- **叙事驱动的科研体制**：AI概念从诞生之初，本质上就是科研人员为了争夺科研经费而构筑的一套高效公关叙事，拉尼尔强调这是他作为直接亲历者见证的史实，而非纸上学者的空洞推论。
### [14:16](https://www.youtube.com/watch?v=rGqiswuJuQI&t=856s) 机器翻译的历史演进：从乔姆斯基逻辑核心到大数据拼贴
Jaron Lanier 从机器翻译的发展历史讲起。当年 Marvin Minsky 曾认为计算机已经足够强大，指派几名研究生利用一个夏天的时间实现跨语言翻译。这个暑期项目的构想非常简单：以 Noam Chomsky 提出的语言“逻辑核心”为基础，配合两种语言的双语词典，再用算法将二者结合起来，理论上就能输出翻译。Lanier 指出：“It was a reasonable hypothesis and it absolutely failed and people tried and tried and tried to do more and more sophisticated versions of that approach for decades until in the 1990's some researchers at IBM's lab had a totally different idea.”（这是一个合理的假说，却彻底失败了；几十年间人们不断尝试这种方法越来越复杂的版本，直到 1990 年代 IBM 实验室的研究人员提出了完全不同的构想。）
IBM 研究人员的突破在于意识到人类虽然每天使用语言，却缺乏对语言运作机制的精确科学描述，试图编写能“理解”语言的程序毫无指望。作为大数据应用于该领域的早期代表，他们转向海量已翻译文本，通过算法寻找词组与词组之间的对应关联，制成一种数据拼贴（mashup）。这套方案取得了惊人的成功，瞬间生成了可用的翻译文本，并且至今仍是 Google、Microsoft 等各类免费翻译工具的核心底层逻辑。
```mermaid
flowchart TD
accTitle: 机器翻译技术路径的演进
accDescr: 展示了从早期基于规则的逻辑翻译到现代基于大数据的相关性拼贴翻译的演化过程。
subgraph SummerProject["早期基于规则的进路（Minsky / Chomsky）"]
A["Chomsky 语言逻辑核心"] --> C["算法结合"]
B["双语词典"] --> C
C --> D["彻底失败（无法理解复杂语境）"]
end
subgraph IBMApproach["现代基于大数据的进路（1990年代 IBM 实验室）"]
E["海量已翻译文本库"] --> F["统计词组对应关联（Correlations）"]
F --> G["数据拼贴（Mashup）生成翻译"]
G --> H["现代主流商用翻译系统"]
end
```
---
### [16:19](https://www.youtube.com/watch?v=rGqiswuJuQI&t=979s) 翻译职业的萎缩：被隐匿的日常劳动与“齐夫曲线”
机器翻译给公众带来了极大便利，Lanier 自己也表示经常使用并从中获益，但这彻底重塑了职业翻译者的生存环境。过去，翻译市场遵循钟形正态分布（bell curve），从业者人数是现在的十倍左右，大多数人处于稳定的中等收入水平，可以依靠翻译商务备忘录等日常文书维持体面生计。然而，技术规训将其转变成了一种极少数人获益、绝大多数人边缘化的“齐夫分布曲线”（Zipf curve），翻译行业的职业前景只剩下过去的十分之一左右。
这种变化与唱片音乐人、摄影师以及调查记者的遭遇如出一辙。如果有人认为这只是新技能取代旧技能的“技术进步常态”，Lanier 强调这种看法完全错误，因为语言每天都在进化，系统每时每刻都需要新的语料支撑。无论是法国爆发的“黄背心”运动（gilets jaunes）等突发新闻，还是新梗、流行歌曲或网络幽默，若按字面直译只会沦为荒谬的胡话。系统所需的海量最新例句，实际上来自成千上万在毫无知觉中点击了用户协议的普通人，例如在 YouTube 上给最新视频自愿制作双语字幕的业余爱好者。
- **市场结构剧变**：从容纳广泛中产阶级的钟形曲线（bell curve），塌陷为高度集中的齐夫分布（Zipf curve），职业前景缩减至原先的十分之一。
- **冲击链条的延伸**：除翻译外，录音音乐人、摄影师、调查记者等依赖创作付费的职业群体均遭遇了被“免费化”吞噬的命运。
- **动态语言更新需求**：语言系统并非闭合模型，法国黄背心事件、新型流行文化与网络流行语（memes）每天都需要真实人类提供前沿语料翻译样本。
---
### [19:31](https://www.youtube.com/watch?v=rGqiswuJuQI&t=1171s) “AI 即盗窃”：隐藏在幕后的人类劳动与经济伦理崩塌
Lanier 揭示了当前人工智能叙事中极度荒谬与扭曲的现实：算法一方面向人类宣称其因效率低下而被机器淘汰，另一方面却又在后台持续无偿榨取这些人的劳动成果。他认为 AI 本质上不是一种独立存在的生命体，而是一种重新打包他人数据与创造性努力的新容器。为了让 AI 看起来像活着一样，技术精英必须强行把真正做出贡献的人类推到帷幕之后，假装他们根本不存在，并哄骗人们未来只能依靠全民基本收入（UBI）度日，而系统本身却始终归属于 Google、Microsoft 或百度的私有资本。
关于这种不对等的榨取，Lanier 直截了当地指出：“So essentially, when I hear AI I hear the word theft. To paraphrase Lennon, AI is theft.”（因此本质上，当我听到 AI 这个词时，我听到的就是盗窃。套用列侬的话说，AI 就是盗窃。）他进一步分析，过去的“创造性破坏”能够成立，前提是承认新技术催生的新岗位必须获得报酬；如果社会一方面需要新劳动力提供实时语料，另一方面却假装这种贡献不存在并拒绝支付报酬，经济体系就会走向崩溃，技术最终不仅没有赋能人类，反而制造出大范围失业与贫困。
Lanier 明确澄清，他抨击的不是底层的数学、算法或机器人工程本身——他本人极其热爱工程技术，甚至曾与朋友共同创办了后来卖给 Google 的第一家机器视觉公司。他坚决反对的是给代码套上超自然神话、将其升格为全新生命形式的意识形态包装。
---
### [23:50](https://www.youtube.com/watch?v=rGqiswuJuQI&t=1430s) 图灵测试的生平暗面：隐喻的专制与阿兰·图灵的悲剧
为了探寻这种将机器神圣化思想的根源，Lanier 回顾了计算机先驱阿兰·图灵（Alan Turing）提出“图灵测试”的历史语境。在 Lanier 青年求学时期，图灵测试被讲授成类似爱因斯坦思想实验的纯粹抽象理论，剥离了一切生平背景。事实上，图灵是在其生命的最后几周才提出这一构想的。二战期间，图灵主导制造早期计算设备攻破了纳粹德军自以为不可破解的恩尼格玛密码机（Enigma Code），拯救了英国免遭入侵，战后一度被奉为国家英雄。
然而图灵的同性恋身份在当时的英国属于违法行为。他与约五万名战时人员一样，未被当作普通罪犯，而是被视为精神病患者，被迫接受了所谓治愈同性恋的伪科学残忍疗法。Lanier 剖析了技术隐喻如何扭曲现实：在计算机普及前，主导社会认知的核心隐喻是蒸汽机，连当时流行的大众弗洛伊德心理学也充斥着“平衡压力”的机械思维；医学界错误地认为图灵的性取向是体内某种“压力失衡”，试图通过注射大剂量雌性激素进行“平衡调整”，导致他长出了女性身体特征，最终迫使这位计算机科学奠基人选择吃毒苹果自杀。
- **图灵的历史功勋**：利用早期专用计算机破解德军认为不可破译的恩尼格玛（Enigma）密码，对二战进程及英国免受入侵起到了关键作用。
- **战后被迫害遭遇**：英国战后将约 50,000 名参与战事的人员依同性恋罪名归入病态范畴，图灵被迫接受雌性激素注射治疗。
- **蒸汽机认知隐喻**：图灵时代占主导的机械隐喻是蒸汽机的“压力平衡”，这种粗暴的力学认知被生搬硬套到弗洛伊德心理学与人体激素治疗中，造成了反科学的医疗暴行。
### [27:57](https://www.youtube.com/watch?v=rGqiswuJuQI&t=1677s) 图灵测试的历史语境与艾伦·图灵的痛苦呐喊
在艾伦·图灵（Alan Turing）生命的最后几周，他在实验室的第一台计算机旁放着涂有氰化物的苹果，随后结束了自己的生命。就在这绝望的最后时光里，图灵写下了著名的“图灵测试”（Turing Test）。杰隆·拉尼尔（Jaron Lanier）指出，一旦理解了这一背景，人们就绝不可能再用以往那种冰冷的技术视角来解读这项测试。
图灵测试的原型脱胎于维多利亚时代的客厅派对游戏。在最初的游戏中，裁判通过屏幕下方递送的纸条来判断交流对象是男性还是女性，双方都可以试图误导裁判；图灵将这一设定修改为“人类对战计算机”，并提出如果裁判无法分辨二者，两者之间便不存在实质区别。在科技界，这一逻辑被长期曲解为机器地位的提升，年轻的技术从业者常据此抱怨“为什么我的机器不能被当成真人对待”。然而，拉尼尔强调，图灵当时正因同性恋身份遭受国家的残酷迫害与折磨：
> “我认为在某种程度上，这是一种极度黑暗的痛苦呐喊。”（“I think it was in part a very dark cry of pain.”）
图灵当时的潜台词实际上是在质问英国与文明社会：我为民主与文明立下汗马功劳，却仍因自身特质被逼上绝路；“如果你不能把我当人看待，那我猜你或许会把一台计算机当人看”。在图灵鲜为人提及的文章脚注中，他甚至写道，计算机源自人类，人类源自上帝，你在计算机中所见的一切，归根结底只是人类神圣性的一部分。
### [31:46](https://www.youtube.com/watch?v=rGqiswuJuQI&t=1906s) 图灵测试的逻辑反转：机器没变聪明，而是人类变蠢了
针对图灵测试的判定机制，拉尼尔提出了一个带有些许戏谑但极具洞察力的逻辑反转。在标准测试环境中，包含三方参与者：裁判、作为受试者的人类，以及计算机。传统的技术极客普遍假设，如果裁判区分不出机器与人类，必然意味着计算机的智能已经升级到了人类的水平。但拉尼尔指出，在逻辑上完全存在另一种可能：
> “但还存在另一种逻辑可能，那就是人类向下退化了，人类可能把自己贬低了，人类可能把自己变蠢了，而这正是你无法将他们与计算机区分开来的原因。”（“but there's another logical possibility, which is that the person has gone down, the person might have made themselves debased, the person might have made themselves stupid and that is why you cannot distinguish them from the computer.”）
此外，做出判断的裁判自身也有可能变蠢。从概率上看，测试中共有两个人（裁判与受试者）和一台机器；因此，拉尼尔认为有三分之二的概率是“人类变蠢了”，而非“机器变聪明了”。每次图灵测试看似被通过时，往往只证明了一件事：人类正在主动降低自身的智能去迎合机器的标准。
```mermaid
flowchart TD
accTitle: 图灵测试的双向逻辑假设
accDescr: 对比传统技术视角与拉尼尔提出的退化视角，揭示图灵测试通过背后的人性退化逻辑。
subgraph Setup["测试初始设定"]
J["人类裁判"]
H["受试人类"]
M["受试计算机"]
end
subgraph Assumption1["传统极客假设 (1/3 概率)"]
M -->|"技术跃升"| M_Smart["计算机具备人类心智"]
end
subgraph Assumption2["拉尼尔反转假设 (2/3 概率)"]
H -->|"降智迎合"| H_Stupid["受试人类退化/变蠢"]
J -->|"判断力退化"| J_Stupid["裁判变蠢/标准下降"]
end
Setup --> Assumption1
Setup --> Assumption2
```
### [33:21](https://www.youtube.com/watch?v=rGqiswuJuQI&t=2001s) 虚假社交感知与被操纵的“AI 傀儡戏”
人类具有一种与许多物种共享的生物学本能——“社交感知”（Social Perception）。个体的注意力与态度会受到身边人群行为的强烈引导。拉尼尔回忆起自己童年时和伙伴们玩的一个恶作剧：跑到人群中指着空无一物的天空大惊小怪，不多久所有路人都会顺着他们的手势往上看。然而在当今的互联网平台上，这种社交感知遭到了灾难性的扭曲。
在 Facebook、Instagram、WhatsApp、YouTube 和 Google 等生态系统中，虚假账户（机器人/水军）的增长速度已经彻底压倒了平台清理虚假账号的能力。大量虚假人类被打包出售，用以制造虚假的人气指标或推销特定的政治观点。这种假象精准命中了“人类变蠢”的图灵测试效应——人们自愿受虚假言论影响，进而左右现实中的大选。拉尼尔强调，计算机背后根本没有超自然的奇迹或外星智能，所谓的人工智能背后必然站着谋求利益的人类幕后黑手：
> “无论动机是什么，每当人们因盲信 AI 而变蠢时，幕后总会隐藏着一个未被揭露的人，他是真正的受益者，是那个牵线操纵的木偶戏大师。”（“Whatever the motivation, whenever people are made stupid by believing in AI, there's somebody unrevealed, somebody behind the curtain who is the beneficiary, who's the puppet master.”）
- 虚假账号（从 90 年代被称为“代理（Agents）”到如今的“机器人（Bots）”）被大量用于制造虚假的集体共识。
- 操纵者的动机通常是追逐金钱、政治权力、自我膨胀，甚至纯粹的虚无主义。
- 俄罗斯的弗拉基米尔·普京（Vladimir Putin）因在美大选和英国脱欧公投中系统化运用自动化水军账号而被广泛记录，但这种低成本、高回报的技术正在被全球更多利益团体滥用。
### [38:02](https://www.youtube.com/watch?v=rGqiswuJuQI&t=2282s) 微软 Tay 聊天机器人事件与成瘾性算法陷阱
为了证明盲信 AI 的荒谬，拉尼尔以微软曾经研发的聊天机器人“Tay”为例。Tay 的设计逻辑是同时与数百万人对话，通过模式匹配（Pattern Matching）借用其他用户的语句进行回应，从而制造出一种“知心好友”的幻觉。然而上线不到 24 小时，Tay 就彻底沦为了一个宣扬纳粹主义与种族主义的恶毒系统，导致微软不得不紧急将其下线。主流观点认为这是一群年轻网民的恶意投喂破坏所致，但拉尼尔提出了另一个假说：
> “我认为我们制造出了一个‘普京探测器’。”（“I think we created a Putin detector.”）
在拉尼尔看来，Tay 实际上是在网络空间中与大量散布恶意的政治水军机器人产生了相互反馈与感染，进而暴露了机器人相互作用的扭曲本质。这一失败表明，把虚假的算法伪装成独立心智本身就是一种自欺欺人。拉尼尔主张，任何利用 AI 框架提供的核心能力，完全可以用另一种透明的方式呈现：让人类明确感知到控制权在自己手中，并清楚地意识到所得价值其实来自于其他人类劳动者的智慧。相反，如今 Google 等平台正在持续剥夺用户的自主权，诱导用户顺从自动连播推荐；为了锁住用户注意力，推荐系统甚至主动迎合人类底层的战斗或逃跑本能，大肆分发激发恐惧与愤怒的极端内容。
### [40:30](https://www.youtube.com/watch?v=rGqiswuJuQI&t=2430s) 算法操纵与“人工智能宗教”
Jaron Lanier 指出，关于算法推荐是将大多数人还是大比例的少数人推向极端，业界尚存争议，但这本质上并不重要；事实是，如果你任由系统替你做决定，推荐流迟早会滑入一个充满怪诞、诱发偏执与焦虑的视频深渊，且内容大多出自居心叵测之徒。他强调这并非因为 YouTube 的工程师想要祸害世界，而是整套商业与算法逻辑下的必然产物。其实解决之道极为简单：不要由算法自动替用户选视频，而是让用户自主搜索并主动点击。
```mermaid
flowchart TD
accTitle: 推荐算法恶性循环对比自主选择
accDescr: 图表展示了由算法接管用户选择如何必然滑向恶意焦虑内容，以及恢复用户自主搜索控制权的极简替代方案。
subgraph AlgoLoop ["算法接管推荐模式（AI 宗教逻辑）"]
A["让算法替用户自动选择"] --> B["追求指标最大化"]
B --> C["滑入恶意、偏执、引发焦虑的视频区"]
C --> D["造成现实世界的混乱与破坏"]
end
subgraph UserControl ["极简替代方案（还权用户）"]
E["取消自动选片机制"] --> F["用户主动搜索并点击"]
F --> G["保持人类主观意志与能动性"]
end
```
这种两难方案之间的差异微乎其微，平台甚至早已有可供选择的合集功能，然而技术界普遍弥漫的“人工智能宗教”（AI religion）让工程师极度抗拒把控制权还给用户。他们渴望全世界都皈依这种信仰，渴望听到用户说“机器在替我选择、我相信机器、机器正蜕变为生命体”。Jaron Lanier 极其尖锐地批评道：“They want that so much that they'll make the world, they'll destroy the world to get that feeling.”（他们太渴望这种感觉了，甚至不惜毁灭世界来换取它。）这也是他在科技界内部必须时刻与之对抗的艰难斗争。
### [42:03](https://www.youtube.com/watch?v=rGqiswuJuQI&t=2523s) 极客帝国主义与造神狂妄
为了剖析科技圈对“创造仿造人并深信其为新生命形态”的病态痴迷，Lanier 提出了几点深层假设。首先是性别结构的局限：技术权贵绝大多数是男性，这种迷恋暗含着一种“子宫嫉妒”（womb envy），试图借由代码孕育生命，以此摆脱对女性的依赖与两性差异。其次是流行科幻作品（如《黑客帝国》《终结者》等）的毒害，这些作品让科技精英坚信未来属于拥有最强 AI 的人，从而陷入一场孤注一掷的恶性军备竞赛。
在这种狂热氛围下，科技巨头圈内常能听到一种极为傲慢的说辞：他们认为担忧气候变暖、世纪末人口峰值时的淡水匮乏、乃至全球交通加速的新兴传染病固然无可厚非，但都不关键，因为超级 AI 一旦造出就能解决所有问题，其他努力皆是浪费时间。Lanier 将这种对算力图腾的单一崇拜比作《旧约》中的金牛犊（golden calf），是凡人面对死亡与肉身脆弱时，妄图扮演上帝的虚荣膨胀。与此同时，“极客帝国主义”（nerd imperialism）席卷一切：掌握巨额财富的技术新贵自命为王，试图凌驾并吞并艺术、设计、社会学、政治学与心理学等所有关注人类内在体验与主观世界的学科。财富的暴增更被他们视作自身高傲认知的背书，理所当然地认为用机器替代人类是历史的必然。
- 科技权贵将 AI 视为解决气候变化、水资源短缺和流行病的万能灵药，将其他现实努力斥为浪费时间。
- 科技公司的体量与吸金能力已超越传统医药等巨头，财富与地位的飞升让这群新贵陷入自负的恶性闭环。
### [46:24](https://www.youtube.com/watch?v=rGqiswuJuQI&t=2784s) 意识的神圣性与人类“被淘汰”的危机
Lanier 提出，要将通常被随意打包成“AI”的数学与工程学还原为“人类使用的工具”而非“生命形态”，其最核心的基石源于精神维度。他惊叹于自身清醒存在于肉身之内的每分每秒，这种感知、体验或被称为主观意识（consciousness/sentience）的存在，绝非 AI 拥趸口中轻率宣称用程序即可还原的代码产物。Lanier 动情地表述：“This thing to me is sacred and absolutely remarkable. It makes me a better scientist, it makes me a better technologist because it reminds me that I don't understand everything.”（这对我而言是神圣且非同寻常的。它让我成为一名更好的科学家，成为一名更好的技术专家，因为它提醒我：我并未理解一切。）正是这份对神秘未知的敬畏，让他感知到他人的珍贵与生命的奇迹，而 AI 幻觉背后的强权贪欲与意识形态霸权正在彻底摧毁这份敬畏。
在现实层面，财富和权力以前所未有的速度向掌控超级计算机的核心群体聚拢，虚假网络人格加剧政治动荡，更致命的是一种全社会层面的“人类淘汰感”。Lanier 分享了他近期在美国一所高中演讲时的骇人经历：过去年轻人或许迷茫、愤怒或批判成年人的虚伪，但他第一次听到有高中生绝望地提问：“既然我们注定要被淘汰，父母当初为什么还要把我们生下来？这有什么意义？”Lanier 认为，正是这种技术统治导致的“人类即将被淘汰”的恐慌，直接催生了全球范围内原教旨主义宗教的死灰复燃——无论在印度、伊斯兰世界、以色列还是美国的基督教保守阵营，人们都在惊恐地反弹：“那群硅谷极客凭什么垄断了新上帝，而将我们视作弃民？”
### [50:45](https://www.youtube.com/watch?v=rGqiswuJuQI&t=3045s) 对话 Ulrich Kelber：作为“忠诚反对派”的技术反思
Lanier 的演讲在掌声中结束后，候任德国联邦数据保护和信息自由专员 Ulrich Kelber 展开对话。Kelber 首先赞赏维利·勃兰特讲座选择数字化主题，并预言鉴于技术对各领域的深远颠覆，这绝不会是最后一次探讨此类议题。他高度评价 Lanier 这位身处技术核心圈却纵览全局的“局内人”，称其演讲是在激情呼吁“将技术进步转化为社会进步”，践行维利·勃兰特的政治理想。Kelber 提到在演讲过程中 Lanier 的手机发出了一声猫叫，这使他联想到 Lanier 书中的观点——呼吁人们像猫一样保持独立特行、不屈服、出其不意且难以捉摸，由此提出核心问题：在技术驱动的社会变革中，个体应当承担什么责任、注意什么，而国家的法定职责又在何处？
Lanier 首先幽默化解了手机猫叫的插曲：他透露手机提示音正是他新书封面上那只真实的黑猫“Potato”，它当年是在加利福尼亚州奥克兰的一处停车场被救助的。Lanier 打趣说自己主张尊重数据权益，既然不盗用人类的数据，自然也不能盗用猫的数据，回程必须多赏给它零食以犒劳其贡献。谈及技术治理中各方角色的博弈，Lanier 坦言极为棘手：这也是为什么他始终游走在极其复杂的双重身份之中——既置身技术腹地，又充当冷峻的批判者，扮演着一种“忠诚的反对派”（loyal opposition）。他感叹道，技术未来充斥着高度不可预测性，每当你以为自己彻底参透某种逻辑时，总会有程序员写出匪夷所思的全新代码颠覆一切前置假设，因此要界定出各方最有效的行事角色确实困难重重。
### [53:41](https://www.youtube.com/watch?v=rGqiswuJuQI&t=3221s) 个人责任与戒除算法成瘾：从香烟到社交网络
杰隆·兰尼尔（Jaron Lanier）谈到了个人的责任问题。他表示，自己做出了一个决定：呼吁人们将“退出这些系统”视为一种社会责任。他承认这是一个非常激进的要求，而且心知肚明目前只有极少数人能做到这一点。原因在于，人们不仅在心理上对其高度成瘾，而且由于网络效应（Network Effect）的存在，形成了深刻的依赖——所有亲朋好友、职场人脉和职业发展全部绑定在这些平台上。在网络效应和成瘾机制的双重夹击下，退出变得异常艰难。
然而，兰尼尔强调，个人责任必须包含“拒绝参与”。每当社会出现与商业利益绑定的群体性成瘾时，哪怕人数再少，也必须有一小部分人站在系统之外展开对话。他以香烟的发展史作为类比：放在几十年前，这间讲堂里肯定烟雾缭绕，更多的儿童会死于肺癌；但整个社会最终达成了共识，认定任由二手烟泛滥是不值得的。最初推行禁烟时阻力极大，人人都反抗，叫嚷着“抽烟让我觉得自己很性感”、“抽烟是我的个人权利”。如果当时没有少数清醒的人跳出成瘾机制率先反思，禁烟进程根本无从谈起。同理，在数字时代，哪怕只有极少数人看清自己在维持该系统运转中所扮演的角色并选择退出，就足以彻底改变社会的舆论与讨论方向。
> “It's a very drastic thing to ask and I realize that only a small minority of people can possibly do that because people are both addicted, and they also have a dependency because of the network effect, because everybody else and their career things and all that their families are on these systems and between the network effect and the addiction, it's very hard to quit.” —— Jaron Lanier
### [55:23](https://www.youtube.com/watch?v=rGqiswuJuQI&t=3323s) 监管的深水区：超越 GDPR 与遏制操纵商业模式
谈及政府层面的角色，兰尼尔指出，过去两天一直在讨论这个宏大命题。他认为政府必须发展出更深入本质的监管思维，直击问题的最核心。以欧洲为例，目前的监管重心主要放在隐私权和 GDPR（《通用数据保护条例》）上，但兰尼尔尖锐地指出，侵犯隐私本身并不是最致命的伤害，真正的破坏在于“隐私被侵犯之后，系统用这些数据做了什么”。
他打了一个形象的比方：这就像是在一个水瓶上贴了个非常详尽清晰的标签，细则里写着“这是毒药”；但实质上，法律根本就不应该允许任何人把毒药装进瓶子里出售。监管绝不能停留在 GDPR 的防范隐私泄露层面，而必须向前迈进很多步，去阻止“行为操纵”。唯一的解决途径在于：不仅要限制数据的收集方式，更要限制数据的使用方式，最终彻底铲除以“操纵”为核心的商业模式，甚至长远来看要摒弃所谓“人工智能”的隐喻。兰尼尔幽默地调侃道，科技巨头极其精明且滑头，看看他们是如何钻空子避税的就知道了（他随即打趣补充：“我可没指微软，我说的是其他那些公司”）。大算力加持下的人类聪明绝顶，常规监管手段很难预测其变招，因此监管政策必须切得足够深。
> “So you can have an excellent label on a bottle of water, and you can say, wow, in the fine print, it says this is poison, but really, you shouldn't be allowed to put poison in the bottle.” —— Jaron Lanier
### [57:20](https://www.youtube.com/watch?v=rGqiswuJuQI&t=3440s) 乌尔里希·凯尔伯提问：数据滥用的核心与数据定价构想
即将出任德国联邦数据保护与信息自由专员的乌尔里希·凯尔伯（Ulrich Kelber）向兰尼尔提问。凯尔伯指出，兰尼尔此前论述了第三方滥用数据的风险，并指出当下所谓的人工智能技术，本质上往往充斥着对知识产权的掠夺、数据的盗用与滥用。
凯尔伯提到，在其他政治领域也有过类似的规制尝试，例如在遗传多样性议题上，《卡塔赫纳生物安全议定书》（Cartagena Protocol）等国际协议曾试图规定：当某种遗传资源信息被商业化利用时，资源的原产地必须获得相应的利益分成。凯尔伯提出疑问：兰尼尔一直呼吁必须赋予数据价值、必须告别互联网“万物皆免费”（everything for free）的文化，那么以机器翻译为例，这种构想在现实中具体该如何落地？难道小说作者每借用一个点子都必须向人付费？翻译算法抓取了 YouTube 上的视频字幕，也必须向字幕制作者付费吗？这个构思的核心到底是什么？
> “What is the core of the data abuse? You had described earlier also the possibility of abuse by third parties and then went on to say that artificial intelligence as it is used today often consists of intellectual property theft, theft of data, misuse of data.” —— Ulrich Kelber
### [58:50](https://www.youtube.com/watch?v=rGqiswuJuQI&t=3530s) 医疗保险逆淘汰：算法算力如何摧毁良性市场
兰尼尔回应凯尔伯的问题，表示这当中其实包含了十几个层面的问题，但他打算分两步拆解：在具体谈论翻译程序之前，他首先给出了一个关于“数据滥用”的惨痛现实案例——美国的私有化医疗保险体系。在美国，医保制度荒谬至极，医疗保障主要与雇主绑定，自由职业者则暴露在巨大风险中。
在大型云计算和所谓 AI 算法出现之前，美国保险公司之间的竞争逻辑是尽可能多地吸引客户投保，因为投保人数越多，风险池（Risk Pool）越大；规模决定利润，公司越大在绝对意义上就越赚钱。然而，随着超级计算机和关联挖掘算法的介入，商业激励机制发生了 180 度的彻底逆转：
```mermaid
flowchart TD
accTitle: 医疗保险模式随算力发展的反转机制
accDescr: 展示了算力与关联算法介入后，保险公司的商业模式从扩大风险池逆转为剔除高赔付人群的过程。
subgraph PRE["算力时代前：规模效应"]
A["依赖传统大数定律"] --> B["吸纳尽可能多的人群参保"]
B --> C["构建更大风险池以提高总利润"]
end
subgraph POST["算力时代后：算法逆淘汰"]
D["云计算与关联挖掘算法"] --> E["精准预测数千万人的患病风险"]
E --> F["目标转为：只保极不可能理赔的人"]
F --> G["拒保与踢出高风险人群：信息更充沛导致服务更恶劣"]
end
```
通过关联分析数千万人的生活数据，保险公司能精准预测个体未来的患病理赔概率。结果，保险公司的目标变成了“只给那些几乎不可能申请理赔的人提供保险”。算力带来了海量信息，但带来的不是更优质的服务，而是把潜在病患剔除、削减承保人数。兰尼尔强调，类似的市场失灵在各行各业不断上演：谁离大算力计算机更近、谁掌握了不对称的洞察力，谁就能彻底瓦解传统商业原有的良性协作模式，让技术进步沦为少数人榨取利益、多数人遭殃的工具。
> “As soon as the big computer showed up, the business incentives for insurance companies completely reversed... and now your goal was to ensure as few people as possible, those who are unlikely to make a claim, right, and so all of a sudden it became about dropping people and reducing the number of people, so people were served more information resulted in poor service instead of better service, it had exactly the opposite effect of what you might want.” —— Jaron Lanier
### [01:01:12](https://www.youtube.com/watch?v=rGqiswuJuQI&t=3672s) 数据尊严、付费文化与数据工会构想
进入第二部分，兰尼尔针对凯尔伯关于翻译工具如何结算数据报酬的疑问进行了解答。他指出，随着科技进步与社会组织能力的提升，经济整体规模是在持续扩大的。在一个良性的经济体中，每个人都应该从生产力的飞跃中获益，就像汽车厂的工人所拿到的薪水不仅能买得起生活用品，月底还能有盈余，而不会因为造汽车而越来越穷。数据经济也应如此：普通人通过贡献数据所获得的报酬，平均而言必须跑赢他们为使用数据服务所支付的费用。
对于“人们习惯了免费、绝不可能为翻译或社交网络掏钱”的质疑，兰尼尔以流媒体巨头 Netflix 为例予以反驳。大众原本也习惯了看免费电视，但一旦习惯了为 Netflix 付费订阅，大家不仅觉得完全可以接受，反而认为付费带来了电视剧品质的飞跃。兰尼尔笑问台下德国观众 Netflix 在当地是否普及，得到了全场肯定的回应。他强调，公众之所以觉得不可思议，是因为深陷于“互联网一切必须免费，只有科技巨头可以赚取几千亿美元”的怪异偏见中。他目前正与经济学家合作建模，证明这套数据定价经济是完全自洽且可持续的。
兰尼尔特别强调，要实现这一愿景，关键在于“集体谈判权”。如果每个散户单独面对科技巨头，巨头会利用竞争将数据价格压低到零；数据提供者必须像传统工人一样联合起来，成立新型的“数据工会”（Data Union）。最后，他警告了将科技公司视为社会仲裁者的危险：
- **反对巨头代行政府职能**：当社会不断要求 Facebook、Google 去审查仇恨言论、阻止骚扰、清理假新闻时，实质上是把全球言论和文化的最高裁决权亲手奉送给了这些平台。
- **防止权力固化**：这种诉求正在一步步将科技巨头加冕为人类文化的终极掌控者，这必然会导致灾难性的结局，必须由代表公众利益的集体谈判机制加以制衡。
> “We're so used to this bizarre prejudice that we can't even imagine that a different economics could also work out but in fact, it can and it's one of the reasons why I've started collaborating with economists.” —— Jaron Lanier
> “The big tech companies would like it to be each person against each other person, so the prices are driven down to nothing. But in fact, both in the real world and online, people have to bargain collectively in order to be paid and so there has to be some new concept of a data union or something.” —— Jaron Lanier
### [01:05:57](https://www.youtube.com/watch?v=rGqiswuJuQI&t=3957s) 数据中介组织（MID）与数字时代的行会制度
Jaron Lanier 提出，让大众的数据获得合理报酬不仅是一项经济机制，更有潜力演化为公民社会的新基石。他将其类比为中世纪的行会（Guild）：如果雇佣一名属于石匠行会的工匠，客户不仅能买到劳力，还能获得行会自主把关的品质与标准保证。在未来的数字化社会中，此类新型组织完全可以承担起捍卫真相、品质与公德的责任。网络空间如果仅依赖单一的中心枢纽，内容和生态最终必然退化为垃圾；因此必须在中心平台之外建立多极的影响力中心，以维持整个生态的质量。
Lanier 指出，这种兼具数据工会与质量源头功能的实体被称为“MID”（Mediators for Data，数据中介组织），他此前与同事在《哈佛商业评论》（Harvard Business Review）上发表的文章已对其进行了详细阐述。他强调，一旦确立以“人是技术的目的”这一核心出发点重新审视技术，很多看似激进或古怪的构想便会自然涌现。关于这些构想的本质，Jaron Lanier 评价道：“They're very gentle and more similar to things that have worked for humanity in the past.”
```mermaid
flowchart LR
accTitle: 数据中介组织（MID）运作架构
accDescr: 展示用户通过 MID 对接外部平台，实现收益分配与质量监管的结构
subgraph Users["用户与劳动力群体"]
U1["数据创作者 / 自由职业者"]
U2["普通网络用户"]
end
subgraph MID_Org["MID（数据中介组织）"]
M1["数据工会职能：议价与分润"]
M2["行会职能：内容品质与真实性把关"]
end
subgraph CentralHubs["中心化网络平台"]
P1["社交平台 / 搜索引擎"]
P2["AI 模型开发者"]
end
Users -->|"聚合数据与劳力"| MID_Org
MID_Org -->|"输出高质量数据与服务"| CentralHubs
CentralHubs -->|"支付数据对价与报酬"| MID_Org
MID_Org -->|"分发收益并保障权益"| Users
```
---
### [01:07:36](https://www.youtube.com/watch?v=rGqiswuJuQI&t=4056s) 平台资本主义与算力集中：从华尔街、沃尔玛到亚马逊
Ulrich Kelber 随即追问，这种模式是否同样适用于整治当前的“平台资本主义”（Platform Capitalism）弊端。Kelber 指出，Uber 压低车辆准入标准、迫使司机陷入竞价拍卖式的内耗；亚马逊商城（Amazon Marketplace）则坐享其成，让小型零售商承担新品入市的试错成本，一旦商品畅销便利用自身庞大的营销霸权将其夺走。他询问是否能借助此类模型扶持自由职业者与小企业，从而促进经济稳定与普惠增长。
Lanier 回应称，过度贫富差距与核心资源垄断早在计算机出现前就已存在，正如 Karl Marx 所批判的那样；但当下的全新现象在于，接近并掌控超大型计算机（算力中心）的一方能够获取极不对称的信息优势，“信息即权力”。硅谷最初并未察觉这一秘密，反倒是华尔街的量化投资者率先在 20 世纪 80 年代末利用算力算法进行自动化交易，并引发了史上首次由计算机主导的闪崩（Flash Crash）。随着算法红利被拉平，这些机构转而形成传统卡特尔来维持优势。随后，沃尔玛（Walmart）利用巨型计算机统筹供应链数据，在谈判中压榨供应商并积聚资本；而亚马逊则更进一步，将算力优势同时施加在供给端和需求端，彻底形成闭环并击败沃尔玛。Lanier 强调，为维持未来的可持续发展，必须打破这种因拥有巨型计算机而产生的不可持续特权：
> “If Amazon has to pay for the data that gives it an advantage, there'll be an equilibrium reached where the benefits of the computer will start to be spread widely instead of concentrated.”
- **华尔街算法交易**：20 世纪 80 年代末首次由超级计算机交易诱发闪崩，交易优势在算法普及后迅速抹平，促使投资者走向卡特尔垄断。
- **沃尔玛的供应链算力**：运用中心计算机整合全链条信息，形成对供应商的单向议价压制。
- **亚马逊的双向闭环**：不仅压制供应链，同时锁定终端消费者需求数据，形成双向掠夺的算法垄断。
- **数据付费的均衡机制**：强制科技巨头为其获取竞争优势的数据付费，是使技术红利从极度集中走向普惠扩散的关键解法。
---
### [01:12:12](https://www.youtube.com/watch?v=rGqiswuJuQI&t=4332s) 早期意识形态冲突与互联网架构的原罪
Ulrich Kelber 表示，能够与一位技术出身且不对新事物抱持盲目怀疑的极客探讨技术结构问题十分难得，但他深入追问：这一切究竟是在何时走偏的？这到底是技术人员在设计代码架构时带来的技术内生问题，还是后来受利益驱动的 CFO 和投资人为了特定商业模式而诱导的结果？
Lanier 回溯了其著作《Dawn of the New Everything》中记录的社会学背景，指出问题根植于 70 至 80 年代网络萌芽期的技术青年心态。受越战残酷征兵体验的影响，当时年轻的技术人员将“躲避政府”视作最高信条；而偏保守派阵营为规避吉米·卡特（Jimmy Carter）时期的限速法令以节约燃油，广泛利用民用频段无线电台（CB Radios）匿名互通警情，同样形成了利用技术规避监管的牛仔文化。随后，嬉皮士社会主义思潮席卷而来，鼓吹“网络上一切都必须免费共享”的公社理念；然而，硅谷同时又狂热崇拜史蒂夫·乔布斯（Steve Jobs）式具有尼采超人色彩的企业家。
这三种互不相容的意识形态（逃避政府的无政府主义、万物免费的嬉皮共产主义、崇尚超级资本家的创业狂热）交织在一起，最终将系统逼入极其狭窄的技术死角。因为恐惧政府追踪，早期的网络协议没有设计原生用户身份层，结果却反讽地催生了像 Facebook 这样的私营数据巨头来接管身份账户。为了维持“免费共享”的表面乌托邦，平台只能引入第三方广告操纵者作为金主买单，最终演化出当下的监控资本主义。对于这种自我瓦解的妥协，Jaron Lanier 直言：
> “So we started with these absolutely unassailable ideologies and passions and by being inflexible, we forced ourselves into a little tiny space of possible solutions, which is exactly the world we have.”
### [01:18:08](https://www.youtube.com/watch?v=rGqiswuJuQI&t=4688s) 埃塞俄比亚青年的求知渴望与社交媒体陷阱
提问者分享了一段切身经历：他在前往埃塞俄比亚出差期间，随身携带着杰伦·拉尼尔（Jaron Lanier）的著作《立刻删除社交媒体账号的十个理由》（Ten Arguments for Deleting Your Social Media Accounts Right Now）。尽管他本人出于其他原因计划在12月注销自己的 Facebook 账号，但拉尼尔的著作给了他极大的鼓舞。在出差的最后一天，他在埃塞俄比亚的历史名城拉利贝拉（Lalibela）遇到了一位年轻学生。当时埃塞俄比亚在新任总理上台后正沉浸在一种充满希望的崭新变革氛围中，那位年轻人还没等提问者开口，便由衷地感慨道：“我们需要 Facebook 和 Twitter，就像我们需要呼吸空气一样。”
提问者坦言，自己前一天刚好在书中读到，那些借助社交媒体发起的革命是何其迅速地被幕后那台遭到滥用的系统性操控机器反噬殆尽。因此他向拉尼尔提出疑问：面对这样身处发展中国家、渴望变革与发声的年轻人，我们应当给出怎样的答案？在不落入这些操控机器与算法陷阱的前提下，人们究竟该通过什么途径去自我组织、与他人展开交流与连结？
### [01:19:19](https://www.youtube.com/watch?v=rGqiswuJuQI&t=4759s) 不能在别人的平台上寻求自我赋权
杰伦·拉尼尔对这一提问做出了极为严肃的回应。他指出，非洲拥有全世界最庞大且最宝贵的青年群体储备，年轻人是极其珍贵的财富。然而，对于埃塞俄比亚的年轻人来说，将希望寄托于外部商业软件是致命的认知误区。拉尼尔斩钉截铁地强调：“归根结底，你无法在别人的权力平台上赋予自己真正的力量。”（“You can't empower yourself on somebody else's power platform, ultimately.”）他认为，埃塞俄比亚的青年不应依赖硅谷的产品，而是必须编写属于自己的软件、打造属于自己的媒介生态。
拉尼尔回忆起当年“阿拉伯之春”（Arab Spring）爆发时硅谷的狂热反应。当时硅谷充斥着大量的自我标榜与邀功，科技巨头的高管与投资人们纷纷陶醉地宣称这是“Twitter 革命”、“Facebook 革命”，甚至演变成一种近乎宗教狂热的救世主叙事，自豪地高呼“看吧，我们正在拯救世界”。拉尼尔对这种盲目乐观抱持极其强烈的反对态度，但当时却很难发出反对声音。他敏锐地指出，当开罗解放广场（Tahrir Square）的狂欢与抗议散去之后，Twitter 根本无法为那些埃及青年创造工作机会和生计保障，科技平台留给他们的未来空空如也。
```mermaid
flowchart TD
accTitle: 社交媒体算法的情绪反噬回路
accDescr: 展示良好初衷的社会运动如何被平台算法转化为对立仇恨群体的放大器
subgraph UserGroup ["青年行动者与良好初衷"]
A["青年展开行动（如阿拉伯之春、Black Lives Matter）"] --> B["在平台上发布内容进行组织与沟通"]
end
subgraph AlgoMachine ["平台算法机制"]
B --> C["算法捕捉数据以追求最大化参与度（Engagement）"]
C --> D["优先推送愤怒与恐惧情绪（传播最快、粘性最强）"]
end
subgraph BacklashLoop ["极端对抗与回冲效应（Boomerang Effect）"]
D --> E["将内容推送给最具敌意的群体以激发反应"]
E --> F["对立仇恨力量获得极化壮大（如 ISIS、KKK/新纳粹）"]
F -.->|"压倒初始运动"| A
end
```
### [01:20:29](https://www.youtube.com/watch?v=rGqiswuJuQI&t=4829s) 算法驱动的仇恨回冲与社会变革的荒谬性
拉尼尔进一步剖析了社交媒体平台在机制设计上的致命深层缺陷。平台赖以生存的核心是驱动用户参与度（Engagement）并攫取海量数据的推荐算法。算法唯一的运作目标，就是找出能让人产生最强烈依赖、产生更多互动的刺激物。而在人类心理机制中，最容易调动、爆发最迅速且在脑海中停留最持久的情绪，正是“愤怒”（anger）与“恐惧”（fear）。
当青年群体带着极大的善意使用这些工具时，算法对他们的理想毫无兴趣。拉尼尔以“阿拉伯之春”和美国的“黑人的命也是命”（Black Lives Matter）运动为例说明：年轻人怀着纯粹与善意在网上联络沟通，但算法为了榨取情绪反应，必然会将这些数据作为养料，推送给最仇恨他们、最容易被激怒的对立群体。算法不仅激发这些仇恨者的狂热反应，还将他们聚合连结在一起。其最终结果是：
- 埃及青年在 Twitter 上的表达是真实的，但极端恐怖组织 ISIS 在社交网络上攫取的狂热势头却远比他们更为剧烈；
- “黑人的命也是命”运动依托社交媒体发声，但美国三K党（Ku Klux Klan）与新纳粹势力的复苏却借此获得了更强的动能与极化扩散。
拉尼尔总结道，依托这种系统去推动社会变革从根本上就是荒唐的，它在初期或许让人感觉良好且真实，但后续必定带来严重的反噬与破坏。
- “因此，利用这个系统来实现社会变革从本质上讲是荒谬的。它后续一定会产生负面效应。”（“So the thing is that using this system for social change is inherently absurd. It will always have a negative effect later.”）
- 算法的底层逻辑注定会让负面、极化和仇恨的情绪在争夺注意力的竞赛中彻底压倒建设性的善意交流。
### [01:22:32](https://www.youtube.com/watch?v=rGqiswuJuQI&t=4952s) 给非洲青年的忠告：构建本土技术主权
针对埃塞俄比亚以及整个非洲的年轻人，拉尼尔给出了两点极其沉重但无比切合实际的忠告：
第一，必须全力以赴构建属于自己的技术文化。拉尼尔直言：“我给埃塞俄比亚聪明的孩子们的建议是：哪怕困难重重，首先去编写你们自己的该死软件。”（“So, what I would counsel bright kids in Ethiopia is somehow, against all odds write your own damn software first, figure it out, like don't rely on somebody from another country to provide you with your way of talking to each other.”）非洲绝不能依赖外国公司来提供同胞之间相互沟通的媒介基础设施，更不能将自己仅定义为别人产品的“消费者”——人类赖以思考与沟通的语言本身绝不能沦为他人的商品。非洲大陆急需发展更多的理工类大学与科学工程培训体系，唯有建立起自主的技术根基，这代规模庞大的年轻人才能真正掌握自己的命运并收获幸福。
第二，必须高度警惕“回冲效应”（Boomerang effect）。年轻人必须看清，自己纯洁美好的初衷是如何被机制无情扭曲，最终演变成滋养更大恶意的土壤，仅仅是为了服务某些硅谷巨头制造网络成瘾的机器。拉尼尔坦言，这番话或许非常刺耳，身处狂热期的人们甚至很难接受，但这恰恰是不容回避的严酷现实。
## Notable quotes
> [33:21](https://www.youtube.com/watch?v=rGqiswuJuQI&t=2001s) 无论动机是什么，每当人们因盲信 AI 而变蠢时，幕后总会隐藏着一个未被揭露的人，他是真正的受益者，是那个牵线操纵的木偶戏大师。 — Jaron Lanier
> [31:46](https://www.youtube.com/watch?v=rGqiswuJuQI&t=1906s) 还存在另一种逻辑可能，那就是人类向下退化了，人类可能把自己贬低了，人类可能把自己变蠢了，而这正是你无法将他们与计算机区分开来的原因。 — Jaron Lanier
## Mentioned
Jaron Lanier, Ulrich Kelber, Willy Brandt, Ray Kurzweil, Larry Page, Marvin Minsky, Noam Chomsky, Alan Turing, Steve Jobs, Jimmy Carter, Karl Marx, Vladimir Putin, Google, Facebook, Microsoft, Netflix, Amazon, Walmart, IBM, YouTube, Twitter, Instagram, WhatsApp, Tay, GDPR, Cartagena Protocol, Harvard Business Review, Ten Arguments for Deleting Your Social Media Accounts Right Now, Dawn of the New Everything
## Video description
Jaron Lanier – Who is Civilization for? Jaron Lanier delivers the Willy Brandt Lecture 2018 on December 10 in Berlin, followed by a Q&A with Ulrich Kelber, German Federal Commissioner for Data Protection and Freedom of Information (Jan 2019).
Timeline:
00:00:00 Lecture Jaron Lanier
00:50:44 Q&A with Ulrich Kelber
The Federal Chancellor Willy Brandt Foundation invites outstanding personalities from politics, science, economics and culture to the annual Willy Brandt Lecture. In the spirit of Willy Brandt, they address topics that are of central importance for understanding our history and shaping our present and future.
Playlist Willy Brandt Lecture:
https://www.youtube.com/playlist?list=PLArlFKrCW-zyYvGPbotpO4vW7_pPtKwad
Further events of the Federal Chancellor Willy Brandt Foundation:
https://willy-brandt.de/en/current-events/
-----------------------
Jaron Lanier hält am 10. Dezember die Willy Brandt Lecture 2018 in Berlin. Im Anschluss folgt eine Diskussion mit Ulrich Kelber, designierter Bundesbeauftragter für den Datenschutz und die Informationsfreiheit.
Timeline:
00:00:00 Lecture Jaron Lanier
00:50:44 Q&A mit Ulrich Kelber
Die Bundeskanzler-Willy-Brandt-Stiftung lädt zur jährlichen Willy Brandt Lecture herausragende Persönlichkeiten aus Politik, Wissenschaft, Wirtschaft oder Kultur ein. In der Lecture greifen diese im Geiste Willy Brandts Themen auf, die für das Verständnis unserer Geschichte und die Gestaltung unserer Gegenwart und Zukunft von zentraler Bedeutung sind.
Termine weiterer Veranstaltungen der Bundeskanzler-Willy-Brandt-Stiftung:
https://willy-brandt.de/aktuelles/veranstaltungen/?fwp_veranstaltungsalter=neu
Playlist Willy Brandt Lecture:
https://www.youtube.com/playlist?list=PLArlFKrCW-zyYvGPbotpO4vW7_pPtKwad
Die Bundeskanzler-Willy-Brandt-Stiftung online:
Webseite: https://www.willy-brandt.de/
Newsletter: https://www.willy-brandt.de/newsletter/
Instagram: https://www.instagram.com/bwbstiftung/
Facebook: https://www.facebook.com/BundeskanzlerWillyBrandtStiftung/
Mastodon: https://social.bund.de/@BWBStiftung
Twitter: https://www.twitter.com/bwbstiftung
Soundcloud: https://www.soundcloud.com/zeitgeschichteerleben/
Spotify: https://www.open.spotify.com/show/3BNitkqNhlWuktAt8pxshz
Apple Podcasts: https://www.podcasts.apple.com/de/podcast/zeitgeschichte-erleben-der-podcast-der-bundeskanzler/id1522691355
#Lecture #JaronLanier #Civilization
[Source](https://www.youtube.com/watch?v=rGqiswuJuQI)
Source quotes
[33:21] Jaron Lanier: 无论动机是什么，每当人们因盲信 AI 而变蠢时，幕后总会隐藏着一个未被揭露的人，他是真正的受益者，是那个牵线操纵的木偶戏大师。
[31:46] Jaron Lanier: 还存在另一种逻辑可能，那就是人类向下退化了，人类可能把自己贬低了，人类可能把自己变蠢了，而这正是你无法将他们与计算机区分开来的原因。
Source quotes
[78:08] Jaron Lanier: You can't empower yourself on somebody else's power platform, ultimately.
[18:15] Jaron Lanier: So essentially, when I hear AI I hear the word theft.
[31:46] Jaron Lanier: Anytime the Turing test seems to be passed, it probably means that people have gotten stupid.
[5:23] Jaron Lanier: Within the tech world, artificial intelligence has become very much like a religion.
[67:36] Jaron Lanier: If Amazon has to pay for the data that gives it an advantage, there'll be an equilibrium reached where the benefits of the computer will start to be spread widely instead of concentrated.
Original transcript
[0:00] We say who is civilization for?
[0:05] And of course,
[0:07] I think most people
would immediately have an answer.
[0:14] I think most people would say
it should benefit human beings.
[0:20] It should sustain the human project.
[0:24] It should benefit the overall
[0:29] biosphere, the ecosystem of the Earth
[0:32] so that this all continues.
[0:34] That's probably who civilization is for
[0:39] but it's never quite that simple.
[0:43] When I was much younger,
[0:46] I came up with an image to help me think
about who the beneficiary is
[0:54] of our activities, whether we're
engineers or writers or anything else
[0:59] and I call this the circle of empathy.
[1:04] By the way, the term empathy
[1:08] had its origins in the a circle
of psychologists and poets in Germany
[1:19] about a century ago
[1:22] and the original use of the term was
[1:26] essentially imagining what virtual
reality would be like someday.
[1:30] In the original use of the term,
[1:33] there was an example
given that somebody might be able
[1:37] to imagine themselves as a leaf or as
a mountain
[1:41] and that as people could exercise their
imagination to become different parts
[1:46] of reality, that
that would also help them appreciate each
[1:50] other and develop a sympathy for one
another's different positions.
[1:54] It was a very charming idea
but this notion of a radical
[1:59] transformation itself,
(phone ringing)
Wait. That horrible technology.
[2:04] I'm sorry, it's always off,
[2:06] I don't know how that happened.
[2:10] This
[2:14] very charming word
[2:19] came from essentially an attempt
[2:22] to imagine what virtual
reality would be like someday.
[2:24] And we've always had this idea
in virtual reality that maybe if you can
[2:28] imagine yourself in a different position,
whether very radical
[2:33] imagining yourself as a leaf blowing or
something, this would help you dislodge
[2:38] yourself from only seeing the world
from your perspective all the time
[2:42] and this might help you become more
sympathetic to the situation of others.
[2:46] So the circle of empathy,
[2:50] this image that I mentioned is the circle
around you and anything inside the circle
[3:00] are those who are the beneficiaries
[3:03] of what you do, anything inside
the circle deserves your empathy.
[3:09] Things outside the circle
perhaps don't deserve your antagonism,
[3:16] perhaps they
don't deserve destruction
[3:21] but on the other hand, they're not
the beneficiaries of what you do.
[3:24] So
I would hope that all decent people
[3:29] believe that human beings
should be inside the circle.
[3:33] That's not always the case.
[3:36] There are racists and homophobes and many
other varieties of
[3:42] ideological people who would like to put
some human beings outside of the circle
[3:47] but I think decent people can
agree that humans belong inside.
[3:52] There are some cases that are
very difficult to decide.
[3:56] There are controversies about animals,
[4:00] should animals be inside the circle
[4:02] or on the outside or
somewhere right on the edge?
[4:05] In the United States,
we have tremendous debates about abortion,
[4:10] about whether an early fetus should
instantly be on the inside or not.
[4:16] Typically, Liberals wish to make
[4:19] the circle larger, and Conservatives
wish to make the circle smaller.
[4:24] It's a good definition of those two
[4:28] attitudes.
[4:30] There are problems with making the circle
either too large or too small.
[4:35] If you make the circle too small, of
course the problem is you become cruel.
[4:41] In fact, you eventually make it small
enough that you even destroy yourself
[4:45] and this we've seen again and again when
conservatism spirals out of control
[4:52] in history, something I don't even
need to refer to in this location.
[4:59] If you try to make the circle too big,
there are also problems.
[5:03] If you say I will never even kill
[5:07] a bacteria because I wish to support all
life, then you can't live because your
[5:11] body kills bacteria all the time,
you essentially become incompetent
[5:16] and so there's a sort of a zone
in which the circle is plausible.
[5:23] Now, I'm telling you
about the circle of empathy
[5:28] for a very simple reason which is that
the project of technology,
[5:36] which has existed for centuries,
has recently become, I would say,
[5:41] dominated by an idea that we should
put machines inside the circle
[5:47] and this is the idea
of artificial intelligence.
[5:51] Now, when I say it's become dominated,
I don't think I'm exaggerating.
[5:57] If you talk to most of the people in most
of the big tech companies,
[6:04] I'm one of them but I probably have
a minority position on this idea,
[6:10] most of them will say
[6:13] We are in an AI race,
just as was described in your fine
[6:19] introduction where it was said 'Is
Germany going to be in the AI race?'.
[6:25] China announces itself
as being in the AI race,
[6:29] the United States certainly announces
itself as being in the AI race,
[6:35] everybody does.
[6:37] It's become this strange myth
[6:40] of competition where everybody is trying
to be the inventor and owner
[6:47] and controller and beneficiary of this
new life form that we will create.
[6:55] Now, what I wish to propose
[6:59] is that putting a computer inside
[7:04] the circle of empathy, which is
essentially the idea of artificial
[7:09] intelligence,
is fundamentally an incompetent idea.
[7:17] It's very much like trying to not kill
[7:20] bacteria, which would force you to never
brush your teeth and essentially die,
[7:26] you can't do it.
[7:28] And in the same way, to treat computers as
being alive ultimately is an absurd task,
[7:34] that it is not what it initially seems to be
[7:37] and I wish to describe my reasons
[7:41] for saying that
and some of the reasons that I think
[7:44] there are much better approaches
to getting the advantages of new
[7:50] mathematical and engineering techniques
that are usually described as AI,
[7:55] but we can get those benefits without the
[7:58] myth-making, without the storytelling,
without the theology of AI.
[8:04] Now, I call it a theology
because within the tech world,
[8:10] artificial intelligence has
become very much like a religion.
[8:16] Specifically, it's taken on a quality
that is similar to a medieval religion.
[8:24] The adherents are extremely influential.
[8:30] If you're not familiar with this culture,
you might think I'm exaggerating
[8:34] but, for instance, Ray Kurzweil,
who's the chief engineer of Google,
[8:40] says things that are more extreme than
what I'm about to say all the time
[8:45] and so in the new theology,
it goes like this -
[8:49] we are building these
intelligent machines,
[8:53] they are improving faster than we can
[8:58] so they will at some point surpass us,
[9:01] just like a jet plane
would surpass a race car
[9:04] and when they surpass us, they will
be the most intelligent machines,
[9:09] they will be the dominant life form.
[9:12] It's a religious project,
[9:14] this new, faster machine
will inherit the project of life from us,
[9:21] it will take over as the major
life form on the planet.
[9:26] Those who are close to the machine
[9:29] at that time might have their brains
uploaded and achieve
[9:35] the experience of immortal life within
the new giant artificial intelligence.
[9:40] Until then,
[9:42] the first duty of everybody is to share
all their data because the data is what
[9:47] will make up the new
artificial intelligence
[9:49] and this is a fascinating point
because you might think why
[9:55] are companies like Google and Facebook
so greedy for all this data?
[10:00] Can it really possibly make them more
[10:02] money from their advertising to follow
your expressions and all these things?
[10:06] Can it really, really do that?
[10:08] And the answer is not really,
a little bit,
[10:11] but the deep driver is
actually this religion.
[10:14] Google says it's only being an advertising
company temporarily
[10:19] and it's trying to win the AI race,
and that's what it's doing with the data.
[10:24] And when I say Google says, I mean,
the founders and the people who control
[10:27] Google, like Larry Page,
say this very literally.
[10:30] So
[10:31] this is not an inference
or a cleverly statement,
[10:35] this is simply a report of the literal
statement as it is made.
[10:41] So
[10:42] this obviously does bear many
similarities to medieval religions.
[10:46] The true believer can get immortality,
[10:49] the non believers consigned to death.
[10:52] There'll be an apocalyptic event
that ends everything we know,
[10:56] this is called the singularity
[10:58] when the machines take over,
and we're supposed to like that because
[11:03] the machines will be better,
they'll be smarter.
[11:05] That is approximately the theology.
[11:08] Now, I would like to examine why I think
[11:12] this is a terrible way of thinking
from a number of perspectives
[11:17] but the first one should be obvious
because it suggests that people will
[11:23] either all be killed or at the very best,
be made obsolete
[11:27] and this is surely a ridiculous way
[11:30] of conceiving of a project that should
be about serving humanity.
[11:34] The project being technology, it's
we have inverted our goals for technology is
[11:40] the beneficiary instead of a tool to
serve the original population.
[11:45] So this is a tremendously tragic example
of the circle of empathy being expanded
[11:52] in an incompetent way
where in order to expand it to include
[11:56] computers, we're actually kicking people
out because we're saying that they will
[12:00] not be the beneficiaries
after the singularity.
[12:03] But I want to dig a little deeper
[12:06] into the details of this
and I want to use a very specific example
[12:10] that I often find is the easiest
way to get across how I see this.
[12:15] So
[12:17] the example I'm going to use
is translation between languages such as
[12:23] between German and English,
as is being performed now by someone I can
[12:29] barely see in the soft light
of a glass box over there.
[12:33] Hello, translator.
(laughs)
[12:38] Some of you have probably heard me use
[12:41] this example before
and I apologize for being repetitive
[12:45] but I think it's clear,
so I'll continue to use it.
[12:50] The idea of translating between human
[12:53] languages was one of the first
dreams of computer science
[12:57] even as early as the 1950's.
My mentor,
[13:02] the most important mentor
for me was named Marvin Minsky,
[13:05] and Marvin Minsky, as you might know, was
the person who probably did more than any
[13:12] other person
to promote the idea of artificial
[13:15] intelligence, to promote
the mythology of it,
[13:18] and I'll get back
to that a little bit later,
[13:21] but Marvin love to argue.
[13:23] So the fact that we disagreed about this
from when I was a kid, it was wonderful.
[13:29] I would tell Marvin this whole
AI thing is just horrible.
[13:33] Why are we doing it?
[13:34] And he would say it will be effective
[13:36] for getting our grants,
so shut up and just play along
[13:39] and in fact, it was true.
[13:41] It was very good for getting
grants back in the early days,
[13:46] in the 70's was when I started at this,
you would go to
[13:49] a grant making organization
such as the Defense Department,
[13:53] you'd say we're going to build this super
smart thing and if we don't do it,
[13:56] our enemies might and it'll get smarter
than people and it was Okay, here's your money,
[13:59] here's your money,
oh my God, you better...
[14:01] and so it was very effective.
[14:02] And actually, the whole thing started
off as storytelling to get grants
[14:08] and I'm not saying this as a scholar,
but as a direct participant,
[14:13] this is where it started.
[14:16] But at any rate, about
language translators.
[14:20] One day, Marvin had the idea that maybe
[14:23] computers were good enough,
that a couple of graduate students could
[14:27] achieve translation between languages
over the course of a summer.
[14:31] So he assigned it as a summer project
[14:33] and the idea was simple.
[14:35] You would start with Noam Chomsky's idea
of a core, a logical core of language
[14:42] and then you would have the dictionary
[14:44] for the two languages
and you'd combine these two things together
[14:47] with an algorithm, and then
you should have a translator.
[14:51] It was a reasonable hypothesis
and it absolutely failed
[14:55] and people tried and tried and tried to do
[14:58] more and more sophisticated versions
of that approach for decades
[15:02] until in the 1990's some researchers
at IBM's lab had a totally different idea.
[15:10] They were saying, trying to write a program
that understands language
[15:14] is hopeless because we use language
but we don't understand language.
[15:20] Nobody has a scientific
description of how language works.
[15:24] What we'll do instead
is we'll use big data
[15:28] and this was one of the first examples
[15:30] of big data becoming important
in this type of application.
[15:33] We'll get a very,
very large amount of text that has been
[15:39] translated
and then we'll look for correlations
[15:41] of phrase to phrase, phrase to phrase,
and we'll create a mashup of that,
[15:45] and that worked.
[15:47] All of a sudden,
there were usable translations coming out,
[15:50] and that is still the core of the
technique that we all use today.
[15:56] Now, since that time, free translation
services have become available.
[16:02] So as we all know,
[16:04] we can go online and from Google or
Microsoft or others,
[16:08] we can enter some text and we'll
get a memo translated or a web page,
[16:12] it's wonderful.
[16:13] I think this is a fantastic service,
it's convenient,
[16:16] I enjoy it,
[16:18] I benefit from it.
[16:19] However, there's something very strange
[16:23] going on
because with these services coming online,
[16:28] the career prospects for people who
translate professionally have changed
[16:34] and to get into how they've changed
[16:37] properly requires a bit
of a technical discussion.
[16:40] It used to be that there were ten times
[16:45] more people who could look to a career
in translation and
[16:52] their success formed a bell curve
where most of them fell in the middle.
[16:57] Once again, it's a little technical.
[16:59] What happens after something's been
[17:03] regimented under a computer system,
it changes to a much smaller number
[17:08] of people, and it looks like
what we call a zip curve.
[17:10] There are a few people who can
benefit from it, but most don't.
[17:14] At any rate, on average, what's happened,
[17:16] although there are some
people who've done very well,
[17:19] on the whole,
[17:21] there's about a tenth the level of career
prospect for somebody who does translation
[17:26] for a living
because they can't get a job doing those
[17:31] little memos anymore for a business
which used to be a lot of the work.
[17:35] Now
[17:38] this follows exactly the pattern
[17:41] of other tasks that used to be
paid that have been turned free.
[17:46] It's what's happened
with recording musicians,
[17:49] it's what's happened with photographers,
[17:52] it's what has happened crucially
with investigative journalists
[17:57] and so
[17:59] you might say, well, that's very sad, but
this is just the old story of progress.
[18:06] When a new technology comes along,
it makes some skills obsolete.
[18:11] You might say that except you'd be wrong.
[18:15] In this case, the reason you're wrong is
something that's not widely realized.
[18:21] Every single day, the language changes.
[18:25] Every single day,
[18:26] we need new example phrases.
[18:29] For instance, there's news.
[18:32] We suddenly have to talk about yellow
[18:34] jackets in France, and that has
to be translated correctly.
[18:37] If it's translated literally,
it will turn into nonsense.
[18:40] We have to talk about new pop culture,
[18:44] new jokes, new songs,
new memes, all of these things.
[18:50] And where do we get the example phrases?
[18:53] We steal them.
[18:55] We go around to people who do not
[18:58] understand that we're taking
the translations and we simply take them.
[19:02] They have all clicked through on their
[19:05] little agreement that lets us do it, and
we steal them by the tens of millions
[19:09] every single day.
[19:11] We have, the algorithms have found people
[19:15] who are doing translation
for one reason or another, for instance
[19:20] sometimes it's amateurs who like to do
subtitles on the most recent YouTube
[19:24] videos for their language,
that sort of thing.
[19:27] So
[19:31] here we have a very screwy,
bizarre situation.
[19:35] We're telling the people you're obsolete
[19:39] because a giant electronic brain is
better than you and has replaced you
[19:43] but we still need you,
[19:45] we're going to steal from you.
[19:46] So can you see something wrong with that?
[19:49] So there's a sense in which AI
[19:54] can be understood
as a new way of packaging data and other
[20:01] efforts, creative efforts between people
rather than as a thing in itself.
[20:06] Now, if any AI,
[20:08] and this is a principle
that generalizes,
[20:11] if each supposed AI program is
different, not all of them need new data
[20:19] every day, but they all
ultimately come from people
[20:23] and so in order to treat the AI is alive,
you have to somehow hide all those people.
[20:29] You have to tell those people,
get behind the curtain,
[20:32] like I did before when I wasn't
sure if I should be on stage,
[20:36] you have to pretend
that they aren't there,
[20:39] you have to tell the people
you're not needed anymore,
[20:42] maybe you'll survive on basic
income or something
[20:46] but the funny thing is that the AI's
are always owned by somebody.
[20:51] The AI is owned by Google
[20:53] or maybe Microsoft or somebody, Baidu
if it's a Chinese one.
[20:59] So essentially, when I hear AI
I hear the word theft.
[21:05] To paraphrase Lennon,
[21:06] AI is theft.
[21:08] It's
[21:10] a form of pretending that people didn't
[21:13] contribute when, in fact, it was
crucially about what people contributed.
[21:16] So from a political and economic
perspective, it's a disaster.
[21:23] In the past, every time
a new technology emerged, we
[21:31] would say this is creative destruction
[21:33] and if some old jobs have gone away
[21:36] because the skills are different now,
that's okay, because there'll be new jobs.
[21:41] But there can only be new jobs
[21:43] if we accept the idea that the people
who do those jobs should be paid.
[21:47] If we say, well, there are new jobs,
the new people are needed.
[21:52] However, we're going
to pretend they're not there,
[21:54] we're going to pretend we're not stealing
from them and we're not going to pay them,
[21:58] then of course, that system breaks down
and instead you create unemployment
[22:04] as the technology advances,
which is a disaster,
[22:07] it means that technology hurts
people instead of helps people.
[22:11] So
[22:12] going back to our question,
who is technology for?
[22:16] If we say it's in part for machines
and when we say artificial intelligence,
[22:22] we're suggesting that the machine itself
is becoming the beneficiary,
[22:26] that there's an intrinsic value in making
the machine have a certain status.
[22:31] If we say that,
we also in the same breath
[22:35] are removing people
from having that same status.
[22:41] So it's a remarkably unethical
and destructive idea.
[22:45] Now, having said that,
I want to make very clear that the thing
[22:50] I'm criticizing involves
storytelling, myth-making,
[22:55] it involves vocabulary,
[22:58] it involves ethics,
[23:01] it involves economics,
[23:02] but I'm not criticizing computer code,
[23:06] I'm not criticizing the design of robots,
[23:09] I'm not designing something fundamentally
[23:12] mathematical
or in the realm of engineering.
[23:15] I actually happen to love that stuff.
[23:19] The people who do AI and call it AI think
I do AI. For instance,
[23:27] my friends and I sold Google
their first machine vision company.
[23:31] I do that stuff
[23:33] but to me, it's just code.
[23:36] To me,
[23:38] there's no reason to add this new layer
of myth-making that gives it a kind
[23:43] of a supernatural status as this
new life form that we're creating.
[23:47] In fact, there's every reason not to.
[23:50] Let me give you one more
approach to this idea.
[23:55] As you probably know,
one of the earliest expressions
[24:00] of the idea of AI was from Alan Turing,
the principal inventor of the computer
[24:06] and this is called the Turing Test.
[24:09] When I was in school, when I was young, we
[24:13] were taught the Turing Test
as if it were one of Einstein's thought
[24:21] experiments, as this foundational core
idea in modernity
[24:25] and we were taught anything
[24:28] about the biographical context
in which Turing thought of it,
[24:34] we knew very little about it
[24:36] actually, it was just an abstraction.
[24:38] More recently, more and more people are
aware a little bit of Turing's biography.
[24:44] There was a major movie about it,
[24:46] it's become part of the public
discourse at the time,
[24:49] nobody knew.
[24:51] So for those of you who don't know,
Turing proposed
[24:57] the idea of the Turing Test in the last
[25:01] weeks of his life.
He'd had an extraordinary,
[25:04] extraordinary life as the principal
inventor of the computer.
[25:10] He had been drawn into the war effort and
[25:18] plausibly was responsible
for saving Britain from an invasion
[25:26] and what he did with one of the earliest
[25:30] computers was he broke a Nazi secret code,
which was called the Enigma Code.
[25:34] You all probably know this history
[25:37] but just in case, the Enigma Code
was decoded with a little box
[25:41] and
the mathematicians in the Nazi
[25:47] intelligence services believed
it to be unbreakable
[25:49] but they weren't aware that there was
[25:51] a computer on the other side
and the computer could break it.
[25:54] So
[25:56] after the war, Turing was initially
[25:58] celebrated as one of the
great heroes of Britain.
[26:04] However, it happened that he was gay
[26:08] and at that time, it was illegal
to be homosexual in Britain.
[26:13] And he, along with about 50,000 others who
had participated in the war effort, were
[26:25] treated as not exactly as criminals,
but sort of as mental patients.
[26:30] So he was forced to accept a
treatment, a quack, bizarre,
[26:42] unscientific, and cruel treatment that was
supposed to cure his homosexuality.
[26:47] He was forced to accept
[26:50] massive doses of female hormones
[26:53] with the idea that that would
balance his sexuality
[26:56] and here we see the extraordinary power
[27:00] of the metaphors we choose to use
to understand our own technologies.
[27:05] Before the computer became the dominant
[27:08] metaphor, the steam engine
had been the dominant metaphor
[27:12] and during Turing's lifetime, the computer
was known to only a few individuals,
[27:18] so the steam engine was
still the powerful metaphor
[27:21] and with steam engines,
it's all about balancing pressures
[27:25] and therefore the popular version
[27:28] of Freudian psychology had to do
with balancing these pressures.
[27:33] So the idea was that, well,
his sexuality must be out of balance,
[27:36] so we're going to balance it
with the opposite hormone
[27:38] and somehow that will get the engine
to not be giving off steam and all that.
[27:42] It's something like that,
something crazy, something stupid.
[27:47] So he developed female
bodily characteristics,
[27:52] and he committed suicide
in an extraordinary way by eating an Apple
[27:57] laced with cyanide next
to the first computer in his lab.
[28:00] Do you all know the story?
[28:02] Yeah, some of you do.
[28:04] So it was just in the last weeks before
[28:08] his death that he wrote
down the Turing Test
[28:10] and once you understand the context,
[28:12] I don't think it's possible
to read it in the same way.
[28:15] So the usual way that
the Turing Test is told
[28:20] is we're told that Turing
[28:27] modified an older
parlor game in which a judge was supposed
[28:32] to determine whether little slips of text
that are transmitted under a screen
[28:37] were coming from a man or a woman.
[28:39] You'd have to guess who is
the man who is the woman,
[28:42] they both might be trying
to deceive the judge
[28:44] and I don't know why this was
considered an entertaining game,
[28:49] I find it it sounds kind of boring to me,
I don't get it,
[28:52] but anyway, apparently Victorians thought
that this was incredibly amusing
[28:57] and so Turing said,
[28:59] we'll get rid of the woman and let's
have a person and a computer
[29:03] and if the judge can't distinguish them,
what distinction is there?
[29:08] Okay, now, this is a very
interesting intellectual move,
[29:13] it's a very interesting one,
[29:14] because it doesn't say anything about
the absolute status of anybody.
[29:18] It doesn't say,
[29:20] all it says is if you think people are
[29:22] elevated in some way and you can't
distinguish the person from the machine,
[29:27] who are you to discriminate
against the machine.
[29:29] So it essentially puts the machine
into the role of the homosexual or the Jew
[29:35] or some other oppressed class, it
takes the machine into the circle
[29:39] of empathy and complains,
why does this thing not have status,
[29:43] why does it not have rights?
[29:45] However, however,
and that's the attitude that the tech
[29:50] world has had ever since. The
technical young men will often whine,
[29:55] why isn't my machine
treated as a real person?
[29:58] But my machine smarter than your dog.
[30:01] I don't know,
[30:02] I ran into this all the time
in the lab with brilliant young men.
[30:08] The thing is, Turing was being
tortured to death for his identity
[30:14] and I think the way we should read this
should be informed by the context.
[30:18] I think it was in part a very dark
[30:26] cry of pain.
[30:28] It was saying here,
what more could I have done for Britain?
[30:33] What more could I have done
for the cause of civilization,
[30:35] what more could I have done for the cause
of democracy than what I did?
[30:38] And yet I'm still being
killed for who I am,
[30:41] and I think there's
an indictment built into it.
[30:45] There's an indictment that, you know,
if you can't treat me as a person,
[30:50] I guess maybe you'll treat
a computer as a person.
[30:53] It's bringing up the whole absurdity
of the lack of empathy shown towards him.
[30:58] I
[31:00] think that that's the better
interpretation given the context.
[31:04] I think what he's saying is that if you
[31:07] have so little humanity, I bet you
can't tell a computer from a person.
[31:12] And in fact, if you look,
[31:15] if you look in the footnotes,
he only wrote two versions of it,
[31:21] one in a little note and another
is part of a little article,
[31:25] and if you look in the footnotes,
he has a comment,
[31:27] surely you can see that ultimately
[31:29] the computer came from people
and people came from God
[31:34] and whatever you see in the computer is
ultimately just part of the divinity
[31:40] you see in people.
[31:41] He has this amazing footnote
which nobody ever reports on.
[31:44] Okay, so
[31:46] I want to say something else about
[31:48] the Turing Test, which is a little bit
of a joke, a little bit tongue in cheek,
[31:53] but here's what I wish to point out:
[31:55] In the Turing test
[31:58] as it's usually presented,
there's a judge who is attempting
[32:01] to distinguish a person from a computer,
only getting little notes, little tweets,
[32:08] you might say, little messages.
[32:11] Now, the assumption of the technically
minded nerd is that if the judge can't
[32:19] distinguish which is the computer and
which is the person,
[32:22] it must mean that the computer has become
elevated, that it has become like a person,
[32:28] but there's another logical possibility,
which is that the person has gone down,
[32:34] the person might have
made themselves debased,
[32:37] the person might have
made themselves stupid
[32:39] and that is why you cannot
distinguish them from the computer.
[32:42] It's another logical possibility.
[32:45] There's nothing in the Turing Test
that tells us which happened.
[32:49] Furthermore, the judge
might have become stupid.
[32:52] Okay, so you start with two
people in one computer,
[32:58] either the computer might become elevated
or the people might become stupid,
[33:03] but because there are two people,
[33:05] I would argue that there's a two third's
chance that a person became stupid
[33:09] rather than the machine became smart.
[33:11] Okay, so in general, anytime
the Turing test seems to be passed,
[33:18] it probably means that people
have gotten stupid.
[33:21] Now, you might say, well,
[33:23] this is just a theoretical exercise,
but that you would be wrong.
[33:27] This is actually possibly
the most important
[33:31] technological interaction going on now,
[33:34] because it's creating an existential
threat to our species and our civilization.
[33:39] Now, here's what I refer to,
[33:41] I'm referring to fake people
[33:43] that create fake social perception
that destroy a human character, destroy
[33:50] the politics of region,
and ultimately destroy the ability
[33:54] of mankind to act in any
collective way that's rational.
[33:57] So
[33:58] let me
[34:00] say, let me unpack those things.
[34:06] Humans
[34:08] have a characteristic that we share
[34:11] with many other species,
that we perceive socially.
[34:14] And what I mean by that is that the way
[34:19] people around you are directing their
attention and the attitude they have
[34:23] to the world forms a collective perception
that of all people present are aware of.
[34:29] We're always helping each
other watch for dangers,
[34:32] we're always helping each other
be aware of opportunities.
[34:36] In one of my books,
[34:38] I tell the story of when I was a boy,
and some friends would play this trick
[34:43] where we would go out into a crowd
and start pointing at something when
[34:46] there's nothing there,
and soon everybody was looking there.
[34:49] That is social perception.
[34:52] So, in the online world
right now,
[34:55] the rate at which fake accounts are added
to platforms like Facebook, Instagram,
[35:02] WhatsApp, or accounts on YouTube,
Google accounts,
[35:07] the rate of fake people is rising faster
[35:12] than the ability of the companies
to purge the fake people.
[35:15] So we have an unknown but large portion
of fake people and the fake people,
[35:21] there's a whole economy,
[35:23] you can buy bulks of fake people
[35:25] if you want to, suddenly buy
followers so that you look popular,
[35:29] you can buy the service of creating many
fake people to have a political idea,
[35:36] you can do all of this.
[35:38] Now,
[35:39] this is one of the things,
[35:40] because of my concerns about AI,
I started writing about the danger of this
[35:45] very early, and I even have an essay
from the early 90s about the possibility
[35:51] of fake people who in those days we called
agents, and today they'd be called bots,
[35:57] but about the possibility of them swaying
elections through false social perception
[36:02] and how they might go
into battle with each other
[36:05] and I wasn't the only one,
[36:07] I think there were other people
also writing about this danger.
[36:11] All of our warnings were useless,
[36:13] obviously, because it's
exactly what's happened.
[36:16] So
[36:18] when fake people are created
and they throw an election,
[36:24] what's happening is that the Turing Test
has been won by people getting stupid.
[36:29] Can you see that?
[36:31] It's the same as in my analysis
of the old thought experiment.
[36:35] Essentially, what we've proven is
[36:37] that we're willing to become stupid
in order to make the machines seem
[36:41] intelligent, in order
to let them influence us
[36:44] but ultimately,
and this must be remembered,
[36:47] there's no weird alien or angel or
[36:51] supernatural force that's
creating computer programs,
[36:54] it's all from people,
[36:56] and so therefore, if there is an AI that's
making people stupid, that AI
[37:01] is being run by somebody,
and whoever that somebody is is seeking
[37:07] their own agenda, which is typically
money or power, or occasionally
[37:13] just a kind of nihilism,
[37:16] but usually it's money or power,
or is maybe an ego thrill.
[37:20] Whatever the motivation,
[37:21] whenever people are made stupid
by believing in AI,
[37:24] there's somebody unrevealed,
somebody behind the the curtain
[37:28] who is the beneficiary,
[37:30] who's the puppet master.
[37:32] These days,
we tend to think of that person as being
[37:35] Vladimir Putin because
he's been caught doing it so much
[37:39] and there's such extremely
[37:42] well done documentation of him having done
[37:45] it in some specific cases,
such as in the US election,
[37:48] in the Brexit vote, and so on.
[37:51] I'm certain that there are
actually many people who do it.
[37:55] It's very inexpensive to do,
[37:57] it's easy to do.
[38:02] There was this experiment we did
[38:05] at Microsoft that I interpret perhaps in
a different way than the experimenters.
[38:09] We made a little chatbot,
and the idea is that it would talk
[38:13] to a large number of people,
millions of people at once,
[38:16] and use their phrases with each other
through pattern matching to create
[38:19] the illusion that it was
a friend you could talk to.
[38:21] So there were all these people talking
to each other without realizing it,
[38:25] they all thought that they
were talking to an AI.
[38:28] It was called Tay, and within 24 hours
it turned into a Nazi.
[38:34] It became this horrible,
racist, evil thing
[38:36] and had to be shut down
[38:37] and so the question is, why.
[38:41] And there are two popular
explanations for this,
[38:46] one of them is that there was just a group
[38:50] of kids who were being vandals
and kept on feeding it ugly things,
[38:56] but I have a different interpretation and
[38:58] it's very hard to show
which one is more correct.
[39:00] I think we created a Putin detector.
[39:02] I think what happened is our bot started
interacting with other bots that were
[39:08] that were malicious and they
had a crypting effect.
[39:11] So
[39:13] in other words, we made ourselves
stupid to believe in this stupid bot
[39:17] and maybe the right thing is just not to
even believe in bots in the first place.
[39:21] Now, a key idea
is that any core capability
[39:27] that you can offer
using the Artificial Intelligence
[39:32] framework can also be offered
in a different way in which the person
[39:36] perceives themselves correctly as being
in control and perceives the value
[39:41] that they receive correctly
as having come from other human beings.
[39:46] For instance,
[39:49] right now, Google somewhat pressures you
to just let it choose videos for you,
[39:56] video after video after video.
[39:59] The way it chooses videos is in part based
[40:02] on similarity of interest
to what you've selected.
[40:05] But it's also in part
driven to get you engaged
[40:09] and unfortunately,
[40:10] the most engaging material
is that material which excites the fight
[40:17] or flight responses within people,
such as fear and anger
[40:22] and so, as has been
repeatedly documented
[40:26] by researchers,
if you do let the Google video search go
[40:30] on in a remarkable, there's a debate about
whether it's a majority or a large
[40:35] minority of cases,
but it doesn't really matter,
[40:37] if you just let it choose from you,
it eventually will go into some kind
[40:41] of weird, malicious,
paranoia inducing, anxiety inducing zone of
[40:47] videos that seem to have
come from malicious sources.
[40:50] And this is not because anybody at YouTube
wants to hurt the world,
[40:56] it's simply the natural outcome
of this whole methodology,
[41:00] this is what will happen.
[41:02] The alternative is very simple,
[41:05] which is to say we will not
automatically choose videos for you,
[41:09] you will search for videos
and click on them.
[41:12] I mean, the difference is so
slight, it's so slight
[41:18] and there already are some options where
[41:20] you can click
on a collection of videos or something,
[41:24] I mean, there's ways to avoid this,
but there's, like,
[41:27] the artificial intelligence religion
makes engineers just loathe, loathe
[41:34] giving people control because they want
everybody to buy in to this AI religion,
[41:39] they want people to say, yes,
[41:40] the machine is choosing for me,
I trust the machine,
[41:43] the machine is becoming alive.
[41:45] They want that so much
that they'll make the world,
[41:49] they'll destroy the world
to get that feeling.
[41:53] So this is something I
battle against all the time.
[41:58] I battle against this constantly,
and it's difficult.
[42:03] I
[42:06] can attempt to theorize why this holds
so much sway in the technical community,
[42:12] why is tech culture so obsessed with this
[42:16] idea of creating fake people and believing
that it's creating a new life form.
[42:20] Well, one theory is that it's mostly men
[42:25] and it's some sort of a womb envy kind
of a thing that we want to give life.
[42:30] It's a way of not needing women anymore,
[42:33] we can propagate life without them,
[42:37] without their difference from us,
without their whatever is imagined.
[42:44] The vast majority of people who think this
[42:47] way are men, and the vast majority of
influential people in tech society are men
[42:51] and that is a real factor,
[42:54] it's a very peculiar feeling, that's
what's part of it. Part of it is,
[43:03] it's an effective plot
for science fiction
[43:06] and so much compelling science fiction has
[43:10] been made out of the idea
of computers coming alive.
[43:12] We could mention the Matrix
movies and the Terminator movies
[43:16] and on and on and on.
That's had a big, big role
[43:21] and if you look at these movies and you
[43:25] think, wow, science fiction does kind
of say something about the future
[43:28] of technology, then what you think
yourself is AI is power in the future,
[43:34] so we better be the ones to make
AI first, and it better be our AI.
[43:37] Once again, the sense of the race,
[43:39] this all or nothing-race
where nothing else matters.
[43:43] Within the tech community, you'll often hear,
and by the tech community
[43:48] I mean the very most
powerful, wealthy and influential
[43:52] individuals, my friends, in many cases,
you'll often hear an argument like
Well,
[43:58] it's fine for you to worry
about global warming,
[44:01] it's fine for you to worry about whether
[44:03] there'll be enough fresh water when we
hit peak population later in the century,
[44:07] it's fine for you to worry about
the danger of emerging new infectious
[44:11] diseases as populations
travel around the world,
[44:15] you can worry about all that stuff,
but it doesn't really matter.
[44:18] The only thing that really matters is
[44:20] the AI race, because the AI will be
smart enough to solve all those things.
[44:23] So
[44:25] everything else is a waste of time.
[44:27] It's the singular focus
on this fetish object.
[44:31] You can say it's a little bit like
the golden calf in the Old Testament,
[44:36] a sort of a flowering of vanity,
of people believing that they can be God,
[44:42] which is, I think,
a natural thing to want.
[44:45] It's difficult to be a person.
We die,
[44:48] mortality is very difficult,
[44:50] and this gives you a fantasy
of some other status, some other story.
[44:57] That's a very big part of it.
[45:00] Another part of it is what
I call nerd imperialism
[45:04] and what that means is that the nerd
[45:07] mentality, the nerd mindset
would like to subsume all the other ones.
[45:11] So all the people who are interested
[45:13] in art or design, all the people
who are interested in sociology,
[45:18] the people are interested in politics,
the people who are interested
[45:21] in psychology,
the people who live subjectively and are
[45:25] interested in the impressions
of the world, the people
[45:28] who believe in interiority
and are interested in their own
[45:32] experience,
[45:34] the nerd imperialist would like
[45:38] to overwhelm and control and be
superior to all of these
[45:41] and it's kind of happened
because we make so much money.
[45:46] You know, the biggest companies are tech
[45:50] companies now, and they've
come up very quickly.
[45:52] We're outpacing even the Pharmaceuticals,
[45:55] we're kind of Kings in a way
[45:59] and so,
[46:01] when somebody has a kind of good fortune,
[46:04] they always read it as confirmation
of their high self regard.
[46:10] And so naturally, we think, yes,
this whole business of replacing people
[46:15] with something we invent, of course
it makes sense and people are paying us
[46:19] to do it because, of course,
it is the right thing.
[46:24] But ultimately,
[46:27] the deepest reason that I think we
[46:32] should reframe the mathematics
[46:34] and the engineering that's normally
bundled kind of arbitrarily as being
[46:39] the thing that is AI, the reason I think we
should reframe that instead as simply
[46:45] technology that people use instead
of a life form, is a spiritual reason.
[46:52] I
[46:53] find that each moment is remarkable.
[46:59] I am absolutely astonished
that I'm experiencing this,
[47:03] I'm absolutely astonished
that I'm not a machine.
[47:10] There's this extra thing where I am
[47:13] alive inside this body,
[47:15] I am perceiving,
[47:17] this can be called consciousness,
[47:20] it can be called experience,
[47:22] it can be called sentience.
Whatever you call it,
[47:26] the AI people will say, oh no, that's
something we can do with a program,
[47:30] they'll subsume it,
so no vocabulary is adequate to describe
[47:34] it, which is maybe appropriate,
but this amazing sense that there's
[47:40] something manifestly supernatural
in every moment of ordinary life,
[47:45] this thing to me is sacred
and absolutely remarkable.
[47:49] It makes me a better scientist,
[47:52] it makes me a better technologist because
[47:55] it reminds me that I don't
understand everything.
[47:58] It reminds me that I live only on a little
[48:01] speck of understanding
in a sea of mystery.
[48:03] It
[48:05] reminds me of how precious and mysterious
other people are, even though I can only
[48:12] go by faith that they
are also experiencing.
[48:16] It reminds me of how life is magical,
[48:24] how much gratitude I have to be here,
[48:28] and the loss of that,
the loss of that to this AI fantasy
[48:34] of power and greed and
ideological empiricism
[48:41] is such a horrible idea,
[48:43] it's such a loss.
[48:44] I think, there are many
things going on in the world.
[48:49] Social media is exciting, these emotions
[48:53] through fake people
and it's making politics horrible
[48:56] everywhere.
[48:56] There's been this incredible concentration
of wealth and power,
[49:02] much of it for the people who are
closest to the big computers
[49:05] in one way or another,
[49:06] there are many things going on that are
destabilizing and harming the world,
[49:11] but I think one of them is this looming
sense of people becoming obsolete.
[49:16] I
[49:17] was recently giving a talk to high school
[49:20] students in the US, and I heard
questions I've never heard before.
[49:24] I heard questions of the form,
Well,
[49:27] if we're going to be obsolete,
why did our parents have us,
[49:30] what is the point?
[49:32] I've never heard that from
a young person before.
[49:35] I've heard young people
who are scared or angry,
[49:37] I've heard all kinds of things.
[49:39] I've heard young people who think adults
are full of crap, all of those things,
[49:45] but I've never heard that.
[49:46] And I think the sense
[49:48] that the technologists are about to make
people obsolete is
[49:53] having a deleterious effect on the world
along with all the other things.
[49:57] I think it's part of the reason why we see
[49:59] such a rise in fundamentalist
religions of all kinds everywhere,
[50:03] whether we're talking about India,
[50:06] the Islamic world, Israel,
the United States with fundamental
[50:11] Christianity, all over the world,
you see the rise of these things
[50:14] and I think it's in reaction to this idea
[50:16] of like, wow, people are
about to be obsolete
[50:18] and it's those other nerds over there
who are owning the new God, not us.
[50:25] That's a horrible feeling,
[50:28] it's based on a lie,
[50:29] it's a lie we shouldn't be telling.
[50:34] That is my talk for now.
(Applause)
[50:45] Ulrich Kelber:
Well, first of all thank you so much for this,
[50:47] thanks also to the
Federal Chancellor Willy Brandt Foundation
[50:51] that this years' lecture
has chosen a digital topic,
[50:57] I dare to predict that it will not have been the last time
[51:02] because, obviously, this comprehensive change that comes with it
[51:05] will make it important in other subject areas as well.
[51:09] And that a speaker has been chosen who is an insider,
[51:13] but who has kept an eye on the big picture,
[51:16] not just dwelling on the technical solutions
he himself has worked on
[51:20] and has kept an overview.
[51:23] Actually, it was an passionate plea
[51:28] to turn technical progress into social progress,
as Willy Brandt would probably have said,
[51:36] and not to let it go in the opposite direction.
[51:39] And during the speech the smartphone
has "meowed" once,
[51:43] and that reminds me of a book of yours, Mr. Lanier,
[51:47] where you say,
Be a cat.
[51:50] Be non-conformist, don't be submissive,
[51:54] be surprising, be unpredictable,
[51:58] and therefore the question:
[52:00] In this transformation of technological social progress,
[52:03] how much does the individual have to do, what must they pay attention to
[52:06] and what is the task of the state?
[52:12] Jaron Lanier:
Okay, so before I answer your excellent
[52:17] question, I have to say
something about that
[52:19] "Meow".
So
[52:22] as you might be aware,
[52:23] some additions of the book actually
have a picture of the cat on the cover,
[52:27] it's a real black cat
[52:29] named Potato who is rescued from a parking
lot in Oakland, California
[52:33] and that's his "Meow",
[52:35] and so I wanted everyone to know
because I
[52:38] believe in giving credit, just as I
don't want to steal people's data,
[52:42] I don't want to steal cats' data,
[52:44] and so he'll get an extra treat for having
[52:47] contributed to this so that I
can be consistent in my beliefs.
[52:54] So,
as far as the role
[52:58] of the various players is
tricky when it comes to technology
[53:04] and it's one of the reasons why I've
attempted to live this rather complicated
[53:10] life of being both inside and being
a critic at the same time, of being
[53:14] essentially the loyal
opposition, if you like.
[53:17] And the problem is that
it gets very complicated,
[53:22] the future is unpredictable,
[53:24] and so every time you think you've nailed
[53:28] it down, some programmer comes up with
some twist that undoes your assumptions,
[53:32] right.
[53:34] So it does,
[53:36] it is very hard to figure out
what is a useful role to play.
[53:41] So I made a decision that
I would ask people to consider quitting
[53:49] these systems as a form
of social responsibility.
[53:52] It's a very drastic thing to ask
[53:54] and I realize that only a small minority
of people can possibly do that because
[54:01] people are both addicted,
and they also have a dependency
[54:05] because of the network effect,
because everybody else and their career
[54:08] things and all that their
families are on these systems
[54:11] and between the network effect and the
addiction, it's very hard to quit.
[54:16] But the thing is,
[54:17] there has to be an element of personal
responsibility that includes refusing
[54:23] to participate, because whenever you have
mass addiction that's connected
[54:28] to a commercial enterprise,
there have to be some number of people,
[54:33] even if it's only a small number,
who can have a conversation
[54:37] outside of the addiction system.
For instance, with cigarettes,
[54:40] there was a time when this hall would have
been filled with cigarette smoke right
[54:44] now, and more children would have
been dying from lung cancer
[54:47] and we finally came to a decision as
[54:49] a society that having cigarette smoke
everywhere was actually not worth it.
[54:54] But it was very hard,
at first
[54:56] everybody resisted, and we said,
no no no, I feel sexy when I smoke,
[54:59] I feel it's my personal right,
[55:01] there was a very different feeling
[55:04] and we could not have made that progress
[55:06] if there weren't at least a few people
who are outside of the addiction
[55:09] and it's the same with this.
[55:11] If there are at least some people
[55:13] who start to see their own role
in perpetuating the system,
[55:16] that might be enough, even a small number,
to change the conversation.
[55:20] So that's the individual responsibility.
[55:23] As far as government, this is what we've
been talking about for the last two days,
[55:29] it's a big topic.
[55:34] I think that government has to evolve
a sense of regulation
[55:41] in such a way as to attack the problems in
the very most fundamental ways possible.
[55:47] So,
[55:48] for instance, in Europe, there's been
an emphasis on privacy, GDPR, and so on,
[55:54] but the privacy violation in itself is not
the thing that does damage, it is what's
[56:00] done with the information
after privacy is violated.
[56:03] So
[56:04] you can have an excellent label on a
[56:08] bottle of water, and you can say, wow,
in the fine print, it says
[56:13] this is poison, but really, you shouldn't
be allowed to put poison in the bottle.
[56:17] And I think that we have to go many steps
beyond the GDPR so that it's not just
[56:24] about preventing privacy violation,
but preventing manipulation,
[56:29] and I think the only way to do that really
has to do with restricting not just how
[56:35] data is gathered,
but how it's used and ultimately trying
[56:39] to get rid of the business model
of manipulation, and
[56:44] indeed probably the metaphor of artificial
intelligence in the long term,
[56:49] and
[56:52] there are many schools
of thought about how to do that,
[56:55] but I think regulation has to cut as deep
[56:57] as possible because the tech
companies are very clever and wiggly.
[57:01] If you look at how we
evade taxes, for instance.
[57:04] I shouldn't say that, except
for Microsoft, only the other ones.
[57:07] But,
if you look at how clever these things
[57:09] are, it gives you a sense of how clever
people with big computers can be
[57:14] and so it's very, very hard
to foresee what will happen and
[57:17] from a regulatory point of view
you have to cut very, very deep.
[57:20] Ulrich Kelber:
[57:26] What is the core of the data abuse?
[57:33] You had described earlier also the possibility of abuse by third parties
[57:36] and then went on to say that artificial intelligence as it is used today often consists of intellectual property theft, theft of data, misuse of data.
[57:50] Now there are other areas of politics where attempts are being made to regulate such things.
[57:57] For instance, on the issue of genetic diversity, the Cartagena Protocol and others
[58:02] have attempted to say that the regions in
[58:05] which something like this arose must have a share when this information is used.
[58:10] You have made a plea that data must be given a value,
[58:18] that we must distance ourselves from an "everything for free" culture,
[58:22] and that the use of,
[58:27] for instance this translation,
how to imagine this in practice,
[58:33] is there the writer of a text must pay people the idea he used in his novel,
[58:39] the translation program must pay for having collected subtitles on YouTube,
[58:46] what is the essence of this idea?
[58:50] Jaron Lanier:
Okay, so
[58:55] I think there are maybe two
questions within that question
[58:59] and if it's okay, actually, there's like
[59:01] twelve questions, but I'd like
to answer two of them in order.
[59:07] One of them, you were asking about
the variety of abuses of data
[59:13] and I'd like to just give you one other
[59:15] example of an abuse before I get to the
specifics of translation, if that's okay.
[59:20] The United States has an insane system
of healthcare where it's privatized
[59:27] and you get your health care
mostly through your employer
[59:30] and if your freelance, you're at risk.
[59:35] This is a crazy system,
[59:37] but anyway, it's what we have.
Now,
[59:43] before big computers,
[59:46] before big cloud computers, before the
algorithms that we call AI algorithms,
[59:51] the competitive pattern between insurance
companies was to get as many people signed
[59:58] up as possible, because
that created a bigger pool.
[60:00] So it was all about scale
[60:02] and the bigger the insurance company,
the more profitable in absolute terms.
[60:08] As soon as the big computer showed up,
[60:10] the business incentives for insurance
companies completely reversed.
[60:16] Now you had software that
could correlate
[60:20] the lives of millions and tens
of millions of people
[60:22] and so you had better predictions
of what would happen to somebody
[60:26] and now your goal was
to ensure as few
[60:29] people as possible,
those who are unlikely to make a claim,
[60:32] right,
[60:34] and so all of a sudden it became about
[60:37] dropping people and reducing the number
of people, so people were served
[60:40] more information resulted in poor
service instead of better service,
[60:45] it had exactly the opposite
effect of what you might want.
[60:49] And the same thing has happened in many
[60:53] other instances in industry where the use
of these algorithms has undone previous
[60:58] assumptions, because it gives one, whoever's
closer to the bigger computer, enough extra
[61:04] information,
enough extra insight to essentially undo
[61:09] what might otherwise seem
to be a normal business.
[61:12] So keep that in mind
[61:15] and I'll address your second question,
[61:17] which is, in the case of the language
translators, how would it work exactly?
[61:22] Now,
as we all know,
[61:26] people have more capability
technologically than we used to,
[61:31] technology has improved,
let's say, since a 100 years ago,
[61:35] we would like to hope that our societies
[61:38] are getting somewhat
better organized as well.
[61:41] And if you combine those things,
what you see is a growing economy.
[61:46] So we see the economy grow,
and as the economy grows,
[61:51] hypothetically,
[61:52] it should benefit everyone,
which would mean that everyone is gaining
[61:56] benefits from our improved technologies
and our improved abilities to coordinate.
[62:01] So I think I'm saying something
that's very, very basic.
[62:04] Now, in the case of the insurance company,
[62:10] I showed an exception
where that is broken.
[62:13] It's a market failure,
I feel like,
[62:16] where people are suffering as
the technology improves for the benefit
[62:21] of a small few instead
of everybody benefiting,
[62:24] right.
Now,what I would argue is that
in the future,
[62:28] let us suppose that you start paying
for things like translations,
[62:32] you start paying for things like Facebook,
[62:34] and let me just deal with one question,
I know you might say, oh,
[62:37] I would never pay for that,
even though I'm addicted to it,
[62:40] but the thing is, people pay for Netflix.
[62:42] Like, once people get used to paying
[62:45] for something, not only does it become
acceptable, but they often perceive what
[62:51] they paid for as being
the best thing ever.
[62:54] People perceive Netflix as having
improved the quality of television.
[62:59] I don't know how important
Netflix is in Germany, actually,
[63:01] is that a good example here?
Yeah.
[63:03] Okay.
So,
[63:08] the thing is,
[63:10] so that if we believe that this technology
is improving our abilities,
[63:15] then the amount that people are paid
for data should always be ahead of how
[63:21] much they have to pay
for the services made of their data.
[63:25] In other words,
if you are working in a car factory,
[63:29] you should make enough money
to be ahead of the game
[63:32] at the end of the month,
you shouldn't become poorer and poorer,
[63:35] you should do better and better
because you're helping provide cars.
[63:39] In the same way, the price paid for data
has to be high enough that it's becoming
[63:45] a new industry, that people take
pride and that they find sustenance
[63:48] and so the degree to which you're paid has
[63:51] to outpace the degree to which you pay,
ultimately, on average.
[63:54] And you might ask,
can that economics work out?
[63:58] We're so used to the idea that everything
[64:01] on the Internet should be free
and the companies make billions upon
[64:04] billions of dollars, but otherwise,
nothing's worth anything.
[64:07] We're so used to this
bizarre prejudice that we can't even
[64:11] imagine that a different
economics could also work out
[64:14] but in fact, it can
[64:15] and it's one of the reasons why I've
started collaborating with economists.
[64:18] This could be a completely sensible,
[64:20] sustainable economy that I think
would result in a better world.
[64:25] There are many details that could vary,
[64:29] do you pay a subscription or do you
pay little micro payments with use,
[64:34] one thing that I feel would be a crucial
[64:37] element is people who provide data have
to be able to bargain collectively.
[64:41] It can't be each person
against each other person.
[64:44] The big tech companies would like it to be
each person against each other person,
[64:48] so the prices are driven down to nothing.
[64:50] But in fact, both in the real world
[64:54] and online, people have to bargain
collectively in order to be paid
[64:58] and so there has to be some new
concept of a data union or something.
[65:03] Actually, this leads to many
thoughts, but I'll just share one of them.
[65:12] Right now,
[65:14] we frequently petition the big tech
[65:17] companies like Facebook or Google
to become the government globally,
[65:23] the arbiter for speech and indeed,
even for action.
[65:26] We'll say, we must, you are responsible
for stopping the hate speech,
[65:30] you must stop the harassment,
[65:31] you should stop the fake news
and all of these things,
[65:35] right.
[65:35] But the problem is, every time we get them
to do that, we also cement their power.
[65:41] We're declaring them to be the ultimate
[65:45] controllers of culture,
step by step by step
[65:48] and I don't think that
that ends in a good place.
[65:51] I think that some sort of new collective
bargaining entity that would assure
[65:57] that people are paid well for their
data could also become a new
[66:03] element of civil society.
[66:05] It could become like a medieval guild
where you would say, if you hire
[66:09] somebody who's good at stone cutting or
something, who's part of the skilled,
[66:14] you're also getting quality,
they enforce their own quality,
[66:17] that these entities could also become
the aspects of society that become
[66:25] entrusted with providing truth
and quality and decency in the future.
[66:29] This is a more complex idea, and I'm only
presenting the barest picture of it,
[66:34] but there has to be some way that there's
a center of influence online other than
[66:41] the central hub that is
able to create quality,
[66:45] so everything doesn't turn to garbage,
[66:47] because, of course, if there's only
one source, it does turn to garbage
[66:50] and
[66:52] so this idea that the data union could be
[66:55] the same thing as the source
of quality is called a MID,
[66:58] creating a MID,
[66:59] and those who are interested can find
an article I recently wrote
[67:03] with a colleague in the Harvard
Business Review that describes them.
[67:09] So once you start going down this path
of reimagining technology as if people
[67:15] were the purpose, you end up
inventing all of these ideas
[67:20] but in a way, it's a very
easy and natural invention
[67:24] and I believe that even though some
[67:27] of these ideas might sound a little
radical or bizarre, in fact, they're not.
[67:30] They're very gentle and more similar
[67:32] to things that have worked
for humanity in the past.
[67:36] Ulrich Kelber:
[67:38] Would that also be a solution model for the areas that we tend to call platform capitalism today,
[67:48] which are now very much involved in
this race for artificial intelligence solutions,
[67:52] where you also have the feeling
that this is also a kind of theft,
[67:59] so the Ubers which then lower the standards for the vehicles used,
[68:05] which actually force people into an auction for their service,
[68:09] the Amazon marketplace
[68:11] which leaves small retailers the task of introducing a product
to the market and if they are successful,
[68:18] he takes it away and with all his marketing power,
[68:20] he then takes over the sale.
[68:24] Could we also support these freelancers,
[68:27] the small companies with such a model
and thus
[68:32] also ensure stabilization and
economic progress for everyone?
[68:40] Jaron Lanier:
Yeah.
[68:42] So this is interesting.
[68:43] Now, of course, the complaint of a
sort of a Gilded Age with excessive
[68:53] income inequality and excessive
control of vital resources is not new.
[68:59] This is something that has been a problem
[69:03] for a long time and identified as one
from long before there were computers.
[69:07] We could mention Karl Marx as someone
who is eloquent on this topic.
[69:11] Very, very good critic,
terrible inventor, but good critic.
[69:17] And so
[69:20] part of this problem is old.
[69:22] And then part of it is new, and the part
of it that is old is familiar,
[69:27] so I won't go over it.
[69:28] The part of it that is new
is the special benefits that someone gets
[69:34] by being close to one
of the biggest computers.
[69:36] So as I was mentioning before,
with the example of insurance
[69:42] in the United States,
if you are close to one of the big
[69:46] computers, you do have an information
advantage, and information is power.
[69:50] Now,
interestingly,
[69:53] Silicon Valley didn't
realize this at first.
[69:56] The first people to realize it were
investors on Wall Street who used it
[70:01] for automated trading,
[70:05] causing the first computer driven
flash crash in the late 80s, very early,
[70:11] and the people using algorithms
[70:14] for automated trading for a little while
made these easy fortunes because they
[70:17] could outguess everybody, and then
everybody got them and they balanced out
[70:21] and now it doesn't work anymore.
[70:23] It's just, I think it did
do damage to the market and
[70:29] that's another very
interesting story to tell,
[70:32] but I think some of the investment groups
[70:35] that had initially gotten an advantage
from using bigger computers,
[70:40] were motivated to become more traditional
[70:44] monopolist and create kind of cartels
among themselves to create the illusion
[70:48] that they were still benefiting,
even though it was no longer about
[70:51] the computer once they had
canceled each other out,
[70:54] so it's a bit of a complicated history.
[70:56] The next party to figure this out
was a company called Walmart
[71:01] and Walmart used a big computer
to correlate information about their
[71:06] supply chain so that they could
outnegotiate everybody who is a supplier
[71:10] and concentrate capital for themselves
more so than previous hubs
[71:16] and that worked for a while.
[71:18] Amazon came along and said, Wait, we're
not just going to do this to the supply,
[71:24] we're going to do this
to the the demand, too,
[71:26] we're going to do it to the people
as well as to the suppliers,
[71:28] we're going to do the whole loop
[71:30] and so then they overwhelmed Walmart
and
[71:36] one of the crucial things for the future
[71:40] is to undo certain kinds of
unsustainable and unsupportable advantages
[71:49] that people gain from big computers
if they own those big computers.
[71:52] And having to pay for the data would
seem to be an obvious way to do it.
[71:57] If Amazon has to pay for the data
that gives it an advantage,
[72:01] there'll be an equilibrium reached where
the benefits of the computer will start
[72:07] to be spread widely
instead of concentrated.
[72:12] Ulrich Kelber:
[72:13] I don't know how about you,
[72:15] but of course for me it's great fun to talk about
[72:18] the structural problems of technology that
need to be solved and directed
[72:24] with someone who is a techie himself
[72:28] and doesn't start with a fundamental skepticism
[72:31] towards all kinds of change since color television,
[72:36] but really talk about
[72:38] what's going well,
what's going wrong,
[72:39] and how do we improve what's going wrong.
[72:43] But therefore the question,
[72:46] where did this also go wrong start?
[72:50] Is it inherent in the technology,
i.e. with the techies,
[72:54] when they thought they now had to
introduce other forms of code,
[72:59] or was it the CFOs and investors
[73:02] who first used the technology for certain market models?
[73:10] Jaron Lanier:
Yes,
[73:12] well,
[73:15] the sociology of how we got
to where we are is interesting
[73:20] and I've tried to capture a little bit
[73:23] of it in the new book,
The Dawn of the New Everything.
[73:29] Part of the problem
[73:34] arose
[73:36] because at the dawn of networking, which
was in the 70s and 80s, before the Internet,
[73:46] a lot of the young men
[73:48] with technical educations were
taught that the most important feature
[73:55] of civilization was an ability to hide
from the government. In the United States,
[74:01] this was driven largely
[74:03] by the Vietnam War,
where there's been a very widespread
[74:07] experience of being drafted into the army
and then being sent to a war that was not
[74:12] really supported and was
very horribly brutal.
[74:16] So a lot of young men of that era still
[74:21] had embedded in them very deeply this
idea that the number one, the number one
[74:27] by far,
[74:28] the thing that one has to be able to
achieve is to hide from the government
[74:31] and the government is always a bad thing,
[74:33] and often the government is,
[74:36] but that's not the only thing,
though, and so they had this out of balance.
[74:41] As for the part, the aspect of America,
[74:45] the other side of America that was more,
what we say, conservative or pro war,
[74:51] they had a different problem
[74:53] which is that they were getting arrested
all the time for driving too fast.
[74:57] The President Jimmy Carter had imposed
a speed limit in the 70s to conserve gas.
[75:04] It's a little bit like the Yellow
Jacket protests in Paris, I suppose
[75:08] and all these people were saying,
Well,
[75:10] we're not going to drive slow, we're
Americans, we're going to drive fast,
[75:13] then they get arrested
[75:14] and so they developed a technology
which was called CB Radios,
[75:18] where everybody would take on a fake
persona, just like on the Internet today,
[75:23] and they would warn each
other where the police were
[75:26] and so once again,
the idea of technology
[75:30] to hide from the government
was very central, so
[75:33] the whole thing was born with this idea
[75:34] that you have to hide, hide,
hide, hide and the government is
[75:37] bad, bad, bad
and it's all about individuals.
[75:40] It was this very cowboy Wild West idea,
[75:44] and then that was moving forward
[75:47] and then there was another wave of,
I would say, hippie socialism,
[75:52] that everything should
be free, on the Internet
[75:54] we will never pay for anything,
[75:56] everything will be this giant sharing,
[75:58] it will be like a commune.
[75:59] But then there was this other thing,
which is, we love the entrepreneurs,
[76:05] we worship, like Steve Jobs,
like entrepreneurs,
[76:09] we worship these people,
they were considered to have,
[76:14] they had that Nitzschean
[76:17] elevation,
[76:18] they were these supermen who could dent
[76:22] the universe and had some kind of creative
will that could change things
[76:26] and so you have this weird thing,
you're hiding from the government,
[76:30] you want to be a hippie Communist
and you want to support entrepreneurs,
[76:34] how do you combine these things together?
[76:36] And so we ended up making a lot
[76:38] of terrible decisions because there were
very few decisions available that could
[76:43] actually address all
of these different ideas.
[76:46] So for instance, the Internet as it
was born, it doesn't represent people.
[76:51] You don't have accounts on it, that had
[76:53] to come from private companies
because the idea is that, oh my God,
[76:56] if you tell the Internet who you are,
then the government can find you.
[77:01] But it's absurd because we just
created these gigantic companies to cover
[77:06] these holes, like Facebook,
to have an account.
[77:09] It truly was a self defeating,
ridiculous idea,
[77:13] but this feeling was very strong
[77:15] and then this idea
everything must be free
[77:17] but we still want entrepreneurs.
[77:19] How can you achieve that?
[77:21] Instead of two people paying to do
[77:23] whatever they do, they'll
experience things being free,
[77:26] they'll be told to share,
[77:27] they'll be told to be open,
[77:29] they'll be told they're in a commune.
[77:31] But actually the only way it's financed is
[77:34] that there's a third party
who's paying out of a belief that they can
[77:38] be manipulated which is also
just the stupidest thing ever,
[77:42] but that's what we were left with.
[77:44] So we started with these absolutely
unassailable ideologies and passions
[77:50] and by being inflexible,
we forced ourselves into a little tiny
[77:55] space of possible solutions,
which is exactly the world we have.
[78:00] Ulrich Kelber:
[78:02] Last question,
[78:04] I found this very convincing,
also the presentation,
[78:08] why is it actually not allowed to participate in this manipulation machines
[78:12] until they have changed their business model,
[78:14] what needs to be regulated,
[78:16] and I may tell this here,
[78:18] I had your book
'Ten Good Reasons Why You Should Delete Your Social Media Account Immediately'
[78:22] with me as reading during a business trip to Ethiopia.
[78:28] I will still delete my Facebook account
in December for other reasons,
[78:34] but I feel encouraged by the book and
[78:36] on my last day I met a young student in Lalibela in Ethiopia
[78:42] who told me about this atmosphere
of departure with the new prime minister
[78:46] and then said
without me being able to address him:
[78:49] "We need Facebook and Twitter like the air we breathe",
[78:54] and I had just read the day before
[78:56] how quickly those revolutions who started with it
[79:00] were actually eaten up by this systematic machine behind it
[79:04] that is being abused.
[79:05] So what do I answer to such young people,
[79:08] what is a way to organize,
to exchange with others
[79:12] without falling into this trap?
[79:19] Jaron Lanier:
Africa has
[79:22] the world's vastest reserve of young
people, and young people are precious
[79:29] and
[79:31] the people in
was it Ethiopia?
[79:35] Ethiopia,
[79:38] for them to say we need Facebook
is not the right answer.
[79:43] They should be writing their own Facebook.
[79:45] I mean, they have
to create their own media.
[79:47] You can't empower yourself
on somebody else's power platform,
[79:52] ultimately.
[79:57] This is a very upsetting thing to me.
[80:00] So when the Arab Spring happened
[80:02] in Silicon Valley, there was
a lot of self Congratulations.
[80:06] People are saying,
it's the Twitter revolution,
[80:08] it's the Facebook revolution
[80:10] and I was antagonistic towards this
[80:13] and it was hard to be because it was
like this religious rapturous moment,
[80:17] see, we're saving the world!
[80:19] And I was like, well,
[80:20] Twitter is not going to create jobs
for these kids in Tahrir square in Cairo.
[80:24] At the end of this,
what is there for them?
[80:29] But there's a deeper problem,
[80:32] and this is a process,
[80:33] the algorithms that are driving engagement,
that get more and more data
[80:41] and all of this,
[80:42] what they're looking for is something
that they can feed a person that will get
[80:47] that person to become
more and more engaged.
[80:50] And typically,it's not the only way,
[80:55] but the easiest way that that can happen
[80:57] is if the person is fed something
that makes them angry or scared because
[81:01] those are the emotions that rise
the fastest and the easiest
[81:04] and then stick around.
[81:06] Let's say that you have something like
[81:09] the Arab Spring or in the United States
the Black Lives Matter movement.
[81:12] Have you heard of that here?
[81:14] Yeah.
[81:15] And so the people who are
participating are often young,
[81:20] they're often extremely good natured,
and they're putting all the stuff out
[81:24] there to communicate with each
other, to coordinate.
[81:26] The algorithm doesn't care about them,
[81:28] the algorithm is only looking at how it
[81:30] can get an emotional reaction to get
engagement from somebody, from anybody.
[81:35] So naturally, all of this data becomes fuel
[81:39] in exciting and engaging
the people who hate them the most.
[81:43] And then the algorithm gets more
[81:45] engagement and more response
from the people who are upset
[81:48] and then it introduces those people
[81:51] to each other and drive them
more and more and more.
[81:53] So you end up with a tool
that while it was powerful,
[81:59] what the Arab Spring kids got out
of Twitter and Facebook was authentic,
[82:02] but what ISIS is getting is
even more intense.
[82:07] Black Lives Matter was authentic
and it relied on social media,
[82:12] but the revival of the Ku Klux Klan
[82:14] and neo-Nazis in the United States
was more intense.
[82:17] So the thing is that using this system
for social change is inherently absurd.
[82:25] It will always have
a negative effect later.
[82:27] The initial effect will feel very
good and might be quite authentic.
[82:32] So, what I would counsel bright kids
in Ethiopia is somehow, against all odds
[82:38] write your own damn software first,
[82:40] figure it out,
[82:42] like don't rely on somebody from another
[82:44] country to provide you with your
way of talking to each other.
[82:47] Africa has to develop its own technical
[82:50] culture to a much greater
degree than it has,
[82:52] it has to have more
technical universities,
[82:55] it has to have more scientific training
[82:57] if this huge generation of young people is
[83:00] going to find success and happiness,
it's absolutely urgent.
[83:03] And so don't treat yourself
as a consumer of your own,
[83:09] your very language can't be
somebody else's product.
[83:12] Alright, so that's
the first thing I'd say.
[83:15] The second thing I'd
say is, be aware of this
[83:19] Boomerang effect where your good
intentions turn into larger bad intentions
[83:24] just in order to serve the addiction
machine of some stupid company.
[83:27] So
[83:29] this is going to be a hard message,
[83:30] they might not accept it, but I think
It reflects reality.
