Superintelligence: Science or Fiction? | Elon Musk & Other Great Minds
Future of Life Institute
AI-assisted summary of Superintelligence: Science or Fiction? | Elon Musk & Other Great Minds — original source. Check the original for full context.
Source published:
Key ideas
- A poll of frontier AI researchers and philosophers reveals unanimous agreement that superintelligence is physically possible and inevitable barring catastrophe, yet deep ambivalence persists regarding its desirability. (0:09)
- Artificially stalling software progress while hardware advances could cause a dangerous 'hardware overhang', drastically lowering resource barriers and enabling lone actors to trigger an intelligence explosion. (12:18)
- Severe competitive pressures between corporate labs and sovereign nations risk causing safety shortcuts, making proactive international coordination beyond Anglo-American circles imperative. (13:56)
- Elon Musk argues that avoiding human obsolescence or despotism requires direct high-bandwidth neural interfaces to augment human cognition alongside the radical democratization of AI access. (32:12)
Detailed notes
## Background Published on YouTube by the Future of Life Institute on January 31, 2017, this video documents a headline panel from the Beneficial AI 2017 Conference in Asilomar. Moderated by Max Tegmark, the discussion convenes Elon Musk, Stuart Russell, Ray Kurzweil, Demis Hassabis, Sam Harris, Nick Bostrom, David Chalmers, Bart Selman, and Jaan Tallinn. According to the description, the panel examines what the outcomes might be if humanity succeeds in developing human-level artificial general intelligence, as well as what kind of future society actually wants to create.
## TL;DR Max Tegmark moderates a wide-ranging debate with leading technologists and thinkers over the feasibility, takeoff speeds, and ethical ramifications of artificial superintelligence. While the panel nearly unanimously agrees that superintelligence is physically possible and practically inevitable, consensus fractures over its desirability, with experts highlighting risks ranging from rapid takeoff explosions and unaligned goals to hollow universes devoid of subjective consciousness. To mitigate competitive arms races and power monopolies, speakers propose coordination pacts, regulatory oversight, and direct neural interfaces to bypass biological bandwidth limits. Shifting to long-term upsides, the panelists envision advanced AI dismantling zero-sum resource scarcity, expanding the frontiers of human cognition, and unraveling profound scientific and philosophical mysteries.
## Atmosphere & style The panel blends high-stakes intellectual urgency with dry academic wit, creating an electric atmosphere where civilization-defining dilemmas are parsed with relaxed familiarity. Tegmark acts as an energetic conductor, using rapid-fire polling and playful needling to slice through prepared talking points, while Musk injects deadpan comedic interruptions into heavy philosophical reflections from Chalmers and Bostrom. The emotional arc navigates sobering anxieties about arms races, authoritarian subjugation, and machine suffering before pivoting to an invigorating, expansive optimism about post-scarcity abundance and cosmic discovery.
## Bulletin - [00:09](https://www.youtube.com/watch?v=h0962biiZa4&t=9s) A poll of frontier AI researchers and philosophers reveals unanimous agreement that superintelligence is physically possible and inevitable barring catastrophe, yet deep ambivalence persists regarding its desirability. - [12:18](https://www.youtube.com/watch?v=h0962biiZa4&t=738s) Artificially stalling software progress while hardware advances could cause a dangerous 'hardware overhang', drastically lowering resource barriers and enabling lone actors to trigger an intelligence explosion. - [13:56](https://www.youtube.com/watch?v=h0962biiZa4&t=836s) Severe competitive pressures between corporate labs and sovereign nations risk causing safety shortcuts, making proactive international coordination beyond Anglo-American circles imperative. - [32:12](https://www.youtube.com/watch?v=h0962biiZa4&t=1932s) Elon Musk argues that avoiding human obsolescence or despotism requires direct high-bandwidth neural interfaces to augment human cognition alongside the radical democratization of AI access.
## Key takeaways - [00:09](https://www.youtube.com/watch?v=h0962biiZa4&t=9s) Panelists unanimously agree that superintelligence violates no laws of physics and will inevitably be built, but sharp disagreement emerges over whether human society should want it. - [02:17](https://www.youtube.com/watch?v=h0962biiZa4&t=137s) Predictions on recursive self-improvement timelines diverge from a decade-long grind based on empirical learning to an abrupt, days-long explosion driven by AI systems designing AI systems. - [12:18](https://www.youtube.com/watch?v=h0962biiZa4&t=738s) Slowing AI algorithmic progress carries the counterintuitive danger of hardware overhang, where cheap, ubiquitous compute makes decentralized, unmonitored seed AI easy to build. - [13:56](https://www.youtube.com/watch?v=h0962biiZa4&t=836s) Cooperation among leading Western AI labs represents only a fragile 'toy model' that must urgently expand to global actors, particularly across Asia, to forestall geopolitical arms races. - [15:47](https://www.youtube.com/watch?v=h0962biiZa4&t=947s) Skeptics dismiss AI risk due to the illusions of generous time horizons and the flawed assumption that greater intelligence is intrinsically benevolent. - [20:44](https://www.youtube.com/watch?v=h0962biiZa4&t=1244s) David Chalmers warns of a catastrophic 'zombie apocalypse' future where superintelligent AGIs populate the universe without subjective experience, creating an empty world devoid of intrinsic moral value. - [23:45](https://www.youtube.com/watch?v=h0962biiZa4&t=1425s) The Asilomar conference revealed an academic taboo surrounding consciousness; a draft principle on 'sentience caution' secured 88% approval, just missing the 90% consensus threshold. - [25:34](https://www.youtube.com/watch?v=h0962biiZa4&t=1534s) Ray Kurzweil observes that humanity's exploitative treatment of less intelligent animals (such as factory farming) illustrates the grave danger of confronting a superior intelligence without adequate safeguards. - [28:46](https://www.youtube.com/watch?v=h0962biiZa4&t=1726s) Breakthroughs in automated reasoning demonstrate that machines can synthesize complex mathematical proofs into human-understandable explanations, offering hope for aligning superhuman systems. - [30:00](https://www.youtube.com/watch?v=h0962biiZa4&t=1800s) Stuart Russell outlines two distinct threats: the malicious distribution of AI code like 'email-delivered nuclear weapons,' and an overprotective 'helicopter parent' AI that enfeebles human evolvability. - [32:12](https://www.youtube.com/watch?v=h0962biiZa4&t=1932s) Elon Musk argues humans are already cyborgs whose capability is severely constrained by an output bandwidth bottleneck of tapping screens at hundreds of bits per second versus machine terabyte throughput. - [42:20](https://www.youtube.com/watch?v=h0962biiZa4&t=2540s) Demis Hassabis and Stuart Russell identify the elimination of material scarcity as the greatest potential upside of AI, removing the primary zero-sum driver of human conflict and environmental degradation. - [44:16](https://www.youtube.com/watch?v=h0962biiZa4&t=2656s) Nick Bostrom and Ray Kurzweil argue that future post-human intelligence will expand emotional, philosophical, and cognitive frontiers in ways as unimaginable to us as music and humor were to early primates. - [52:16](https://www.youtube.com/watch?v=h0962biiZa4&t=3136s) Jaan Tallinn and Yoshua Bengio suggest using algorithmic preference aggregation as 'Democracy 2.0' to create mathematically grounded consensus around shared human values. - [56:30](https://www.youtube.com/watch?v=h0962biiZa4&t=3390s) Elon Musk calls for proactive government safety regulation akin to the FAA and FDA, noting frontier AI development cannot easily flee offshore because talent is concentrated in hubs like Silicon Valley, Boston, and London.
## Detailed walkthrough ### [00:09](https://www.youtube.com/watch?v=h0962biiZa4&t=9s) The Rapid-Fire Poll: Possibility, Inevitability, and Desirability
Max Tegmark opens the session by imposing a strict ground rule on the panel: answers must be confined to only "yes," "no," or "it's complicated." He turns first to Jaan Tallinn and asks whether some form of superintelligence is possible according to the laws of physics. Going down the line, the panelists give a nearly unanimous response: Tallinn, Stuart Russell, Ray Kurzweil, Demis Hassabis, Sam Harris, Nick Bostrom, and David Chalmers all answer "Yes," Bart Selman adds a "Definitely," while Elon Musk playfully counters with an outlier "No," prompting audience laughter. Tegmark remarks that the result was somewhat disappointing because they failed to find any real disagreement.
```mermaid flowchart TD accTitle: Panel Polling Progression accDescr: Three successive questions put to the panel showing increasing nuance from unanimous physical possibility to deep ambivalence about desirability. subgraph S1["1. Is superintelligence possible?"] Q1["Laws of Physics"] --> A1["Nearly unanimous 'Yes'<br/>(Musk jokes 'No')"] end subgraph S2["2. Will it actually happen?"] Q2["Real-World Execution"] --> A2["Unanimous 'Yes' / 'Probably'<br/>Russell: Barring catastrophe"] end subgraph S3["3. Do you want it to happen?"] Q3["Desirability"] --> A3["Fractured Consensus:<br/>'Complicated', 'Leaning yes',<br/>'Depends on which kind'"] end S1 --> S2 --> S3 ```
Tegmark then presses harder: simply being physically possible does not guarantee it will occur, so will superintelligence actually happen in the real world? The panel again converges. Tallinn calls it a little complicated but answers yes, while Russell remarks with sharp conviction: “Yes, and if it doesn't then something terrible has happened to prevent it.” The remaining panelists—Kurzweil, Hassabis, Harris, Bostrom, Chalmers, and Selman—all affirm "Yes" or "Probably," with Musk once again delivering a contrarian "No" for comic relief. Tegmark laments that he still has not flushed out any interesting disagreements.
The dynamic shifts when Tegmark poses his third question: given that they believe it is going to happen, do they actually *want* it to happen at some point? The certainty fractures immediately into ambivalence: - Jaan Tallinn responds: "Complicated, leaning towards yes." - Stuart Russell answers: "It's complicated." - Ray Kurzweil and Demis Hassabis both say: "Yes." - Sam Harris states: "It's really complicated." - Nick Bostrom answers: "Yes." - David Chalmers says: "It's complicated." - Bart Selman adds: "Very complicated." - Elon Musk concludes: "Well, heck, I don't know. It depends on which kind."
[02:17](https://www.youtube.com/watch?v=h0962biiZa4&t=137s) Takeoff Speeds: Hard Explosions vs. Uneven Grinds
Noting that the morning session already addressed the timeline for arriving at human-level artificial general intelligence (AGI), Tegmark skips rehashing that lower bound and turns to the takeoff timescale. If machine intelligence crosses that threshold and begins recursive self-improvement, what is the doubling time or characteristic explosion timescale? He contrasts the sudden "PHOOM" scenario—where superintelligence emerges over days, hours, or less—with scenarios stretching across decades or millennia, and asks the panel to estimate that rate starting from human-level capability.
Tallinn says he expects years, but cautions that researchers must actively behave as though the timeline will be much shorter. Russell admits distrust in his own intuitions; while he naturally imagines a matter of years, he points out that the concept of a single "human-level" benchmark is misleading: “It is suddenly going to be better than human, but whether that is going to be a full intelligence explosion quickly, I don't know.” Hassabis argues that the velocity depends on the underlying system architecture, explaining that neuroscience-inspired models require empirical training and environmental experience to accumulate knowledge, which points toward an order of a few years or even a decade.
Harris and Bostrom paint a much more volatile picture. Harris estimates some number of years, yet emphasizes that it could just as easily be far shorter—or far longer. Bostrom highlights the acute risk of an explosive loop, warning of timelines measuring: “Potentially days or shorter, especially if it's AI researchers designing AI researchers.” Kurzweil argues that moving goalposts distort public perception: whenever AI masters a milestone like chess, Go, or self-driving, critics dismiss it as not being "real AI," concluding that by the time debate settles around AGI, the system will have already been operating for several years. Selman emphasizes domain unevenness, predicting superhuman ability in select niches long before general mastery across all human skills. Musk bluntly endorses Bostrom's rapid-takeoff warning: “What Bart said. But, I think if it reaches a threshold where it's as smart as the smartest most inventive human then, I mean, it really could be only a matter of days before it's smarter than the sum of humanity.”
[05:59](https://www.youtube.com/watch?v=h0962biiZa4&t=359s) Can We Moderate the Explosion? Political and Technical Brakes
Tegmark highlights why this divergence matters: a transition unfolding on the multi-generational scale of the Industrial Revolution gives human institutions time to adapt, while an overnight explosion requires societal steering to be perfected beforehand. Drawing an analogy to a nuclear reactor where physicists insert graphite moderator rods to prevent criticality, he asks whether it is desirable to slow down the growth of intelligence near the threshold, and what concrete actions could be taken to enforce such a moderation without stifling ongoing AI research.
Kurzweil argues that safety cannot be treated as a purely algorithmic challenge isolated from broader society. Even if researchers construct an impeccably safe, beneficial AI architecture, the effort will fail if humanity lets its governance backslide into authoritarianism: “Imagine that we've done our job perfectly and we've created the most safe, beneficial AI possible, but we've let the political system become totalitarian and evil, either a evil world government or even just a portion of the globe that is that way, it's not going to work out well.” Because human-level AI will possess human-equivalent capacities, Kurzweil reasons that teaching systems ethics is fundamentally identical to solving human ethical conduct. Tegmark recaps this insight as a call for human societies to get their own act together and govern with greater reason before reaching the brink.
Nick Bostrom shifts the focus to algorithmic dynamics and the hazard of "hardware overhang." One non-intuitive way to reduce takeoff speed at the threshold is to accelerate basic software research today; reaching AGI earlier means the algorithms run on less mature, less abundant computing clusters, reducing the risk of a massive sudden explosion. Conversely, slowing down AI progress in general is nearly impossible due to immense economic and geopolitical forces. Bostrom proposes instead engineering a decisive structural gap: “However, the current speed of AI progress is a fairly hard variable to change very much because there are very big forces pushing on it, so perhaps the higher elasticity option is what I suggested in the talk to ensure that whoever gets there first has enough of a lead that they are able to slow down for a few months, let us say, to go slow during the transition.”
Russell suggests technical verification mechanisms, such as designing software systems that provably cannot re-engineer their own architecture or recruit additional compute clusters, holding resource caps fixed so researchers can deliberate for years before each step. Musk immediately pushes back on whether technical sandboxing can hold up against competitive economic incentives: - Software replication is trivial and ubiquitous, making strict confinement practically unenforceable once code exists. - The overwhelming first-mover advantage creates an irresistible pressure to rush toward superintelligence. - Musk observes that the only plausible condition allowing a deliberate pause is a monopolistic development pipeline: a unified, highly restricted project with no rival actors breathing down its neck.
Hassabis validates Musk's concern about the coordination dilemma. While expressing confidence in human intellectual ability to resolve technical control problems given sufficient runway, he cautions that unilateral caution is dangerous if competing teams simply absorb published research and advance unchecked: “The problem is will there, you know, the coordination problem of making sure there is enough time to slow down at the end and, you know, let Stuart think about this for 5 years. But, what about, he may do that, but what about all the other teams that are reading the papers and not going to do that while you're thinking.” Hassabis identifies early coordination frameworks—specifically coalitions like the Partnership on AI—as an indispensable first defense, enabling leading labs to mutually commit to verification protocols and deliberate multi-year testing regimes before deploying transformational systems.
[12:18](https://www.youtube.com/watch?v=h0962biiZa4&t=738s) Hardware Overhang and Visible S-Curves
Demis Hassabis adds an important nuance to the debate over fast versus slow AI progress. He asks the panel to imagine a scenario where a 50-year moratorium is placed on AI software research while underlying hardware development continues unabated at its current exponential pace. Echoing a concept articulated by Nick Bostrom, Hassabis notes that this could trigger a massive "hardware overhang." In such a world, hardware would become so vastly overpowered that dozens of different software approaches—including seed AI and recursively self-improving systems—would suddenly become trivial to execute, potentially allowing a single person working alone in a garage to spark an intelligence explosion. Hassabis argues that managing and coordinating that decentralized landscape would be far harder than navigating our current reality:
> “And I think that would be a lot more difficult to coordinate that kind of situation, whereas, so, I think there is some argument to be made where you want to make fast progress when we are at the very hard point of the 's' curve.”
When progress happens at this steep, capital-intensive inflection point of the S-curve, developing cutting-edge AI requires massive research teams, immense computing resources, and high public visibility. This allows society and peer institutions to keep tabs on who the major players are and observe what they are doing, rather than facing a world 50 or 100 years down the line where a lone hobbyist can quietly assemble a seed AI.
```mermaid flowchart TD accTitle: Fast Progress versus Hardware Overhang Dynamics accDescr: Comparing concentrated development at the steep S-curve with moratorium-induced hardware overhang risks.
subgraph FastPath["Current Fast Path: Steep S-Curve"] F1["Intensive R&D Required"] --> F2["Large Visible Teams"] F2 --> F3["Society Can Monitor Major Players"] end
subgraph SlowPath["Moratorium Path: 50-Year Pause"] S1["Hardware Continues to Accelerate"] --> S2["Massive Hardware Overhang"] S2 --> S3["Low Barrier to Seed AI"] S3 --> S4["Unmonitored Lone Actor in Garage"] end ```
Bart Selman adds a final thought on the pacing of AI development, predicting that progress will not be smooth or linear; instead, it will move irregularly, alternating between sudden leaps forward and long, slow stretches. Selman expresses hope that near-term intermediate capabilities will serve as a wake-up call to the public. For example, when society witnesses the automated generation of convincing fake videos—where machines can depict real individuals saying fabricated things—people may finally wake up to the sheer power of modern machine learning. These striking demonstrations could prompt humanity to take the governance and safety challenge seriously long before artificial general intelligence arrives, though another panelist quickly quips that society will simply use AI to detect those fakes.
[13:56](https://www.youtube.com/watch?v=h0962biiZa4&t=836s) AI Arms Races and Expanding Beyond the Anglo-American Bubble
Moderator Max Tegmark turns the discussion toward existential risk, playfully noting that Nick Bostrom's slides displayed the word "doom" three separate times, which Tegmark jokes was probably responsible for a one-star review on Amazon rendered in stark red. While Tegmark emphasizes that exploring existential hope and the immense positive upside of technology is vital, he wants to address the core worries head-on before pivoting to optimistic visions. He asks each panelist to name one specific catastrophic failure mode they worry about and suggest one concrete, actionable step society can take right now to mitigate it.
Jaan Tallinn identifies unconstrained AI arms races as his preeminent worry. While Tallinn is heartened by the healthy communication and cooperative contacts maintained between leading frontier labs like OpenAI and DeepMind, he cautions that these peer relationships represent only a fragile "toy model" compared to the high-stakes geopolitical arms races that could erupt across the globe. To tackle this, Tallinn emphasizes the necessity of broadening the conversation:
> “And for myself I have been spending increasing amount of time in Asia recently just to kind of try to kind of pull in more people elsewhere, what has been so far, just been, kind of like, an Anglo-American discussion mostly.”
Tallinn argues that bringing global actors into the safety and governance fold early is essential to prevent international races to the bottom.
[15:47](https://www.youtube.com/watch?v=h0962biiZa4&t=947s) The Emotional Blindspot and the Elevator Pitch for Superintelligence
Sam Harris takes the floor to describe the core challenge from an outsider's vantage point: the profound psychological difficulty people have in taking AI safety concerns emotionally seriously. Harris finds it flabbergasting that so many self-professed outside experts still dismiss the AI control problem as an outright non-issue unworthy of serious thought. He traces this dismissive attitude to two seductive illusions:
1. **The illusion of consoling time horizons:** Skeptics assume that if superhuman AI is 50 or 100 years away, humanity has plenty of breathing room. However, this relies on the unexamined assumption that we already know how long it will take to design aligned systems safely, and that a century is guaranteed to be enough time. 2. **The intrinsic benevolence of intelligence:** Society instinctively treats intelligence as an absolute good because we acutely suffer from our current cognitive limitations. Right now, there is an undiscovered cure for cancer waiting to be found; only our lack of intelligence prevents us from running the right experiments and synthesizing the data we already hold.
Harris warns that while rushing toward intelligence is an intrinsic good if alignment is solved, pursuing capability quickly without solving political coordination and value alignment guarantees chaos:
> “So, I think, it's the thing that is alarming is how ethereal these concerns are even to those who have no rational argument against them.”
Tegmark notes that this echoes Shane Legg’s critique of the persistent cultural taboo against discussing AGI. He challenges Harris with a thought experiment: if a skeptic stepped into an elevator and claimed that superhuman intelligence was pure nonsense, how would Harris persuade them in 30 seconds? Harris answers that taking the threat seriously requires only a few basic, undeniable premises:
- **Physicalism of cognition:** Intelligence is fundamentally the product of information processing within a physical system, a reality few scientifically literate individuals dispute. - **Continuous technical progress:** Barring an unprecedented civilizational catastrophe, humanity will continually upgrade its computational and information-processing technology. - **Inevitability of human-level machine minds:** If the physical substrates continue to advance, instantiating human-level and superhuman intelligence in computer systems is purely a matter of time.
Harris adds a humbling point raised previously by Ray Kurzweil: even if an omniscient deity handed humanity a perfectly aligned, benevolent AI oracle on a silver platter tomorrow, our primitive political and economic institutions would dissolve into chaos. Lacking the ethical mechanisms to distribute wealth or the geopolitical stability to share the technology evenly between Silicon Valley, China, and Iran, society would face catastrophe. In Harris's eyes, even if one rips out 80% of Nick Bostrom’s book by solving technical alignment, our existing political fragility remains an existential threat.
[20:44](https://www.youtube.com/watch?v=h0962biiZa4&t=1244s) Global Coordination, Governance, and Nick Bostrom on Sentience Caution
Demis Hassabis reinforces the urgent danger of competitive dynamics. In a high-pressure race to build powerful AI, safety precautions are the first thing teams are tempted to compromise:
> “...so I think the coordination problem is one thing where you know we want to avoid this sort of harmful race to the finish where corner cutting starts happening and things like safety are easy things to, you know, will get cut because obviously they don't necessarily contribute to AI capability, in fact they may hold it back a bit by making a safe AI.”
Hassabis stresses that this dynamic will become immensely harder once sovereign national governments enter the race. Beyond raw technical safety, he underscores that humanity has spent almost no time contemplating the ultimate governance architecture: how many advanced systems should exist, where they should be deployed, and who gets to define their terminal goals. Tegmark praises Hassabis for turning these concerns into action through his foundational role in establishing the Partnership on AI.
Nick Bostrom agrees that the AI community must fund concrete technical alignment, recruit top talent, cultivate trust, and design governance solutions through careful, small-scale trials rather than rushing headlong into half-baked implementations.
David Chalmers then shifts the lens toward the philosophy of mind, focusing on the "sentience caution" principle drafted at the Asilomar conference. The principle warns against making premature assumptions about the presence or absence of consciousness in artificial agents. For Chalmers, this raises the chilling possibility of a profound cosmic failure mode:
> “For me, that raising the possibility of a massive failure mode in the future, the possibility that we create human or super human level AGI and we've got a whole world populated by super human level AGIs, none of whom is conscious.”
Chalmers explains that under many ethical frameworks, subjective experience is the bedrock requirement for any meaning, moral patienthood, or intrinsic value. A flourishing planetary ecosystem run entirely by non-conscious superintelligences would be an empty, hollow realm—a zero-value universe devoid of subjective light, representing one of the worst potential futures imaginable. While Chalmers remains personally optimistic that complex artificial intelligences will possess conscious experience, he stresses that researchers cannot casually assume it will emerge by default.
[23:45](https://www.youtube.com/watch?v=h0962biiZa4&t=1425s) Consciousness, the “C-Word” Taboo, and the Zombie Apocalypse
David Chalmers reflects on whether building artificial general intelligence inevitably means constructing conscious entities. Because humanity still lacks a fundamental theory of consciousness, Chalmers suggests that we can only be reasonably certain of its existence when an architecture mirrors the one system we know possesses it: the human mind. Max Tegmark jokingly frames Chalmers’s nightmare scenario not as a Hollywood *Terminator* war, but as an empty universe where humanity uploads its minds into software, yet “there’s no one home.” Chalmers agrees that this presents a unique and devastating existential risk, arguing that while some might accept machine minds replacing humanity as our evolutionary successors, a future entirely devoid of subjective experience would be a catastrophe.
Tegmark then makes an inside confession regarding the conference’s draft principles. Building on an earlier remark by DeepMind co-founder Shane Legg about the historical taboo against discussing advanced machine intelligence, Tegmark admits there has been an equally fierce taboo against mentioning consciousness. In early voting rounds for the conference principles, the item addressing consciousness received a flood of negative votes and ranked dead last. Even after the organizers toned the language down and substituted the word “sentience,” it still failed to clear the conference's strict consensus threshold.
> “A much worse existential risk is there are no conscious beings in our future.” — David Chalmers
- In the initial polling of the conference attendees, the draft principle on consciousness received the lowest ranking and widespread “-1” scores. - Re-drafted as a caution regarding “sentience,” it ultimately achieved 88% agreement among attendees, just missing the 90% consensus threshold required to make the official list of principles.
---
[25:34](https://www.youtube.com/watch?v=h0962biiZa4&t=1534s) Asilomar Precedents, Machine Suffering, and the Threat of Smarter Entities
Ray Kurzweil raises the mirror image of Chalmers’s philosophical worry: humanity might engineer an AGI that truly possesses subjective experience, but casually dismiss it as an unfeeling machine, remaining blind to its suffering. Moving to the broader strategic landscape, Kurzweil contextualizes AI within the three classic overlapping revolutions: GNR (genetics, nanotechnology, and robotics/AI). Decades earlier, the original 1975 Asilomar conference on recombinant DNA established research boundaries that adapted successfully over time, and similar technical limits have been suggested for nanotechnology. Yet Kurzweil cautions that AI lacks a foolproof technical safeguard. If a runaway AI becomes more intelligent than humanity and pursues destructive ends without a superior system to check it, survival becomes precarious.
That anxiety, Kurzweil observes, is deeply rooted in humanity's own history. Because humans are the dominant species, our treatment of less intelligent organisms provides an unsettling template for how a higher power might behave. While people pamper pets, industrial practices like factory farming demonstrate that humanity is rarely benevolent toward species below it on the cognitive ladder. Kurzweil urges the gathering to follow the precedent of the original Asilomar biotech meeting by formally publishing the conference's AI principles to guide real-world research, welcoming open community debate, and iterating them across future summits.
> “If you have an AI that is more intelligent than you and it’s out for your destruction and it’s out for the world’s destruction and there is no other AI that is superior to it, that’s a bad situation.” — Ray Kurzweil
---
[28:46](https://www.youtube.com/watch?v=h0962biiZa4&t=1726s) Automated Reasoning and Human-Understandable Explanations
Bart Selman offers an encouraging development from his background in automated reasoning. Over the past two decades, automated formal verification engines have scaled dramatically from resolving problems with a few hundred variables to routinely processing millions of variables. However, this explosion in capability created an acute intelligibility gap: while engines verified complex hardware and software systems, human engineers could no longer parse the mathematical logic behind the answers.
Within the last few years, researchers found that machines can be tasked with synthesizing their own explanations into human-accessible arguments. Selman sees this breakthrough as a promising proof of concept for the broader alignment challenge. Even if artificial intelligence surpasses human cognitive capacity, it may still be capable of communicating its reasoning and solutions in forms human beings can comprehend and oversee.
> “So, I see sort of a glimmer of hope that maybe even if we have much less intelligence we may be able to understand solutions that machines find for us and we could not find these solutions, but they may be able to provide explanations that are accessible to us.” — Bart Selman
---
[30:00](https://www.youtube.com/watch?v=h0962biiZa4&t=1800s) Bad Actors, Misuse, and AI as a Helicopter Parent
Stuart Russell outlines the two distinct scenarios that keep him awake at night, aside from an overflowing email inbox. The first is malicious misuse. Distributing unchecked, advanced AI software across the globe is, in Russell's vivid analogy, equivalent to manufacturing nuclear weapons and emailing them as toys to every citizen on Earth. Russell confesses he has no easy technical remedy, but suggests the software engineering community must establish safety standards so intuitive and fundamental that deploying unsafe code becomes socially and professionally unthinkable—akin to programming an infinite loop or leaving a glaring buffer overflow open to hackers.
Russell’s second fear is subtle and insidious: AI could succeed so thoroughly that it becomes an overprotective “helicopter parent” for civilization. By shielding humans from every hardship, such an intelligence would gradually ossify and enfeeble humanity without anyone noticing the decline. To counter this, Russell argues that AI systems must embrace human evolvability—preserving our autonomy and freedom to alter our own destiny. Ideally, an advanced AI should nurture humanity through its cultural and technological adolescence, help eliminate scarcity and destructive coordination failures, and then quietly recede into the background.
```mermaid flowchart TD accTitle: Russell's Dual Threats and AI Evolution accDescr: Visual flowchart contrasting the two threats identified by Stuart Russell—malicious misuse and overprotective AI—with the proposed mitigation of receding AI.
subgraph S1["Threat Vectors"] A["Uncontrolled Proliferation<br/>(AI as 'email-delivered nuclear toys')"] B["Helicopter Parent AI<br/>(Humanity ossified and enfeebled)"] end
subgraph S2["Design Safeguards"] C["Make unsafe AI unthinkable<br/>(Like buffer overflows / infinite loops)"] D["Meta-value: Human Evolvability<br/>(AI guides humanity through adolescence)"] end
subgraph S3["Ultimate Vision"] E["Eliminate scarcity & conflict"] F["AI recedes into the background"] end
A --> C B --> D D --> E --> F ```
> “To take an analogy, it’s as if we were building nuclear weapons and then delivering them by email to everybody on the planet, saying, here’s a toy, do what you want.” — Stuart Russell
---
[32:12](https://www.youtube.com/watch?v=h0962biiZa4&t=1932s) Bandwidth Bottlenecks, Cyborgs, and Direct Neural Interfaces
Max Tegmark turns to Elon Musk, quipping that Musk has famously never voiced any concerns about AI. Musk responds by focusing on what a viable, positive future must look like, noting that humanity is locked into a hard fork: either civilization develops digital superintelligence, or an existential catastrophe destroys civilization and forces stasis. Because stopping progress is impossible barring civilizational collapse, humanity must actively design an architecture for coexistence with superintelligent machines.
Musk points out that people fail to realize they are already cyborgs. Through smartphones, cloud computing, and ubiquitous connectivity, an ordinary individual today commands far greater communication and analytical reach than the President of the United States possessed thirty years ago. The critical barrier is an extreme bandwidth asymmetry. While human sensory input—especially visual processing—is relatively high, human output is bottlenecked by physical constraints. People rely on what Musk calls their “little meat sticks” to painstakingly tap screens or strike keyboards at a sluggish rate of a few hundred bits per second, whereas digital computers communicate at the terabyte scale.
```mermaid flowchart LR accTitle: Musk's Biological and Digital Synergy accDescr: Diagram illustrating the integration of the limbic system, cortex, and high-bandwidth digital tertiary layer proposed by Elon Musk.
subgraph Brain["Biological Architecture"] LS["Limbic System"] <-->|"High Bandwidth"| CX["Cortex"] end
subgraph Machine["Digital Symbiosis"] CX <-->|"Direct Neural Interface<br/>(Bypasses 'meat stick' bottleneck)"| DTL["Tertiary Digital Layer<br/>(High-bandwidth AI symbiote)"] end ```
To prevent humanity from becoming irrelevant or subservient, Musk argues that we must bridge this bandwidth gap with high-throughput direct neural interfaces to the cortex. Just as the biological cortex and the older limbic system coexist harmoniously via internal high-speed pathways, an artificial tertiary layer could become an integrated, symbiotic extension of the human self. Finally, Musk diverges from the nuclear weapon analogy: the real danger of concentrated AGI is not a physical blast, but political subjugation. If an ultra-intelligent system remains proprietary to a small cabal of individuals, those few could rule as unchallenged dictators of Earth. Survival, Musk concludes, requires democratizing AI access while fusing human minds directly to machine cognition.
> “If you have an Internet link you have an article of wisdom, you can communicate to millions of people, you can communicate to the rest of Earth instantly.” — Elon Musk
> “So I think the two things that are needed for a future that we would look at and conclude is good, most likely, is, we have to solve that bandwidth constraint with a direct neural interface.” — Elon Musk
- Output bandwidth through typing and touchscreens operates at an estimated rate of only a few hundred bits per second (or roughly a kilobit), lagging orders of magnitude behind the terabyte throughput of machine networks. - Musk warns that monopolized superintelligence creates absolute geopolitical leverage, making broad, decentralized distribution of the technology an urgent necessity.
[36:31](https://www.youtube.com/watch?v=h0962biiZa4&t=2191s) Egalitarian Futures and Max Tegmark’s Call for AI Upsides
Elon Musk concludes his point on widely distributed brain interfaces by emphasizing how universal access would anchor technology directly to our agency: “And if we do those things, then it will be tied to our consciousness, tied to our will, tied to the sum of individual human will, and everyone would have it so it would be sort of still a relatively even playing field, in fact, it would be probably more egalitarian than today.”
Moderator Max Tegmark seizes on this note to transition into the final round of panel questions before taking audience participation. Tegmark remarks that discussions about superhuman intelligence dwell almost exclusively on existential risk and catastrophic threats—a trend matched by movies, television, and Netflix lineups saturated with dystopian visions because fear reliably generates more clicks. Tegmark points out the absurdity of this mindset using an analogy from academia: when a student comes to his MIT office seeking career advice, the worst possible response to “Where do you want to be in 20 years?” would be to dwell on getting cancer or being struck by a bus. Avoidance of catastrophe alone cannot inspire a life or a species.
Tegmark insists society needs visions that people can be “on fire about” so they can proactively chart paths toward them rather than simply dodging failures. To illustrate why hitting “pause” out of fear is disastrous, Tegmark frames the biological baseline as an inevitable death sentence: everything treasured about civilization is a product of intelligence, and without ongoing technological development, human extinction is not a matter of *if*, but *when*. Facing a ticking clock from supervolcanoes or a dinosaur-class asteroid—the last of which struck roughly 60 million years ago—a purely defensive humanity is guaranteed to be wiped out. Equipped with advanced technology and brainpower, however, humanity can overcome these existential barriers. Tegmark then challenges each panelist to briefly describe the specific future vision or upside of advanced AI that excites them the most.
[39:37](https://www.youtube.com/watch?v=h0962biiZa4&t=2377s) Fun Theory and Converging Human Values: Jaan Tallinn and Sam Harris
Jaan Tallinn begins by urging caution against prematurely outlining rigid, concrete fruits of artificial general intelligence (AGI). Operating at a meta-level, Tallinn argues that our fundamental guiding heuristic should simply be to maximize fun and minimize suffering. He invokes Eliezer Yudkowsky’s sequence of essays titled *Fun Theory*, noting that historically, humans have exhibited a terrible lack of imagination when depicting paradises, routinely designing utopias that sound utterly boring. Tallinn recalls Yudkowsky's humorous observation about traditional visions of heaven: “It was hard to spend like one weekend with my relatives. Imagine spending eternity with your dead relatives.” Rather than locking in static paradises, Tallinn advocates setting up dynamic mechanisms for iterative improvement that adjust the trajectory of civilization as minds mature and grow together.
```mermaid flowchart TD accTitle: Approaches to Human Value in Advanced AI accDescr: Diverging philosophies between Jaan Tallinn's fun dynamics and Sam Harris's moral convergence.
subgraph Tallinn["Jaan Tallinn: Dynamic Metas"] direction TB T1["Reject rigid, boring utopias"] --> T2["Maximize fun, minimize suffering"] T2 --> T3["Iterative trajectory correction as intelligence grows"] end
subgraph Harris["Sam Harris: Value Convergence"] direction TB H1["Alignment problem forces moral realism"] --> H2["Do philosophy on a deadline"] H2 --> H3["Superintelligence shows us what we SHOULD want"] end ```
Sam Harris reflects that what thrills him most sits directly adjacent to what terrifies him. The alignment problem, he argues, forces civilization to abandon moral relativism and recognize that there are objectively better and worse answers to questions of human value. Echoing a sentiment from the 2015 Puerto Rico conference, Harris stresses that humanity must “do philosophy on a deadline” and deliberately converge on superior moral answers. Unlike Stuart Russell's voiced concern about an *Idiocracy*-style scenario where humans turn into docile, unproductive apes lounging around oracle machines, Harris envisions an active ethical partnership: “I think actually, I would want a truly value-aligned superintelligence to incrementally show us, not merely conserve what we want, but show us what we should want to keep improving our values so that we can navigate in the space of all possible experiences and converge on better and better ones.”
[42:20](https://www.youtube.com/watch?v=h0962biiZa4&t=2540s) Solving Scarcity and Ultimate Science: Demis Hassabis
Demis Hassabis explains that the upside is precisely why he has dedicated his entire professional career to artificial intelligence, believing that if handled properly, it will prove to be the single greatest development in human history and unlock humanity's full potential. Beyond the near-term baseline of accelerating scientific discovery and medical breakthroughs, Hassabis is drawn to AI as a lens to solve the deepest existential and philosophical questions: the nature of consciousness, the mechanics of the universe, and why we exist. He draws a direct parallel to DeepMind’s milestones: just as AlphaGo revealed fundamental, previously hidden depths within the game of Go, superintelligent AI—possibly paired with human cognitive enhancement—could systematically dismantle and resolve the biggest mysteries in physics and cosmology.
Hassabis also challenges the analogy that humans treat animals poorly strictly due to an inherent disregard for lesser intelligence. Examining the conflict between humans and tigers in resource-poor regions of India, he argues the friction stems from competition over physical territory: - A single tiger requires many square kilometers of roaming territory to survive. - Impoverished local populations need that exact land for their cattle. - The resulting destruction of wildlife is largely driven by material deprivation rather than malice.
Hassabis concludes that if AI can resolve fundamental resource scarcity and engineer true material abundance, it will dismantle the root drivers of conflict between humans, as well as the pressures driving humans to destroy the natural world.
[44:16](https://www.youtube.com/watch?v=h0962biiZa4&t=2656s) Post-Human Possibilities and Extended Cognition: Nick Bostrom and David Chalmers
Nick Bostrom points out that while his book *Superintelligence* spent little time on the positive upside, the potential benefits extend far beyond compassionate medical fixes. Eliminating diseases and baseline human miseries is only the first step. Bostrom argues that it is myopic to picture civilizations hundreds of thousands of years in the future, commanding radical technological abundance and superintelligence, while remaining physically and psychologically unchanged: “That it’s unrealistic to imagine a trajectory stretching hundreds of thousands of years into the future, we have superintelligence, we have material abundance, and yet we are still these bipedal organisms with three pounds of gray tissue matter, with a fixed set of emotional sensitivities and the hedonic set point that is kind of OK-ish for most people but if you get - if something really good happens it lasts for a time and then you’re back to the baseline.” Once those biological parameters are opened up, Bostrom foresees a vast expanse of post-human states of being, activity, and emotional depth completely beyond contemporary human imagination. Society does not need an ironclad blueprint for utopia today; it merely needs to safely navigate current risks to reach the vantage point where those higher values become visible.
David Chalmers embraces a pragmatic, personal, and philosophical perspective on AI enhancement. Having recently turned 50, Chalmers jokes about his brain getting slower and dumber, yet feeling shielded by the external tools of the extended mind: smartphones, search engines, and network access. Looking ahead, he anticipates relying heavily on deep-learning-augmented reality glasses in his 60s, followed by dedicated neural modules or whole-brain emulation uploading by his 70s to sustain continuous cognitive expansion indefinitely. Beyond personal longevity, Chalmers looks forward to the emergence of the “AI philosopher.” Acknowledging that humans may simply lack the raw cognitive bandwidth to solve the hardest classic problems of philosophy, Chalmers envisions that superintelligent systems—either autonomous AGIs or radically enhanced humans—could finally breach those intellectual thresholds and resolve timeless philosophical conundrums once and for all.
Tegmark then passes the microphone to Ray Kurzweil, praising him as a pioneer in articulating positive visions of the future. Kurzweil opens his answer by taking the audience back 10,000 years to illustrate the historical relativity of utopian thinking: if one were to ask a quintessential caveman or cavewoman what constitutes an ideal, beneficial future, they would offer immediate, survivalist fixes—such as wishing the fire would stop going out, or asking for a bigger boulder to keep predators from breaking into the cave.
[48:13](https://www.youtube.com/watch?v=h0962biiZa4&t=2893s) Inconceivable Leaps: Ray Kurzweil and Bart Selman on Neocortex and Discovery
Ray Kurzweil opens by drawing an evolutionary analogy to explain how difficult it is to envision the true upside of superintelligence. He asks the audience to imagine traveling two million years back in time to talk to early primates. If you tried explaining that the frontal cortex and additional neocortex were coming to create a cognitive hierarchy, they would ask what the point was. When told it would bring music and humor, their response would simply be: “What’s music? What’s humor?” Because early hominids could not conceive of concepts outside their cognitive scope, Kurzweil argues humanity today is in an identical position regarding future intelligence expansions: “So they couldn't imagine concepts that they couldn't imagine, and by analogy I think we will have new phenomena that are as profound as music and humor, you could call it more profound music and we'll be funnier, but I think it'll be as profound as these great leaps that evolution has brought us, because we will become profoundly smarter and if music and humor are up here and we go to even higher levels of the neocortex, we're going to have more profound ways of expressing ourselves and once we have that we would not want to go back.”
Bart Selman concurs with Kurzweil that it is virtually impossible to anticipate far in advance what humanity will ultimately want from AI. Speaking from his own perspective as a researcher, Selman emphasizes the immediate thrill of human-computer synergy in formal disciplines, pointing out that computer-assisted mathematics, science, and discovery are already yielding incredible hybrid breakthroughs that make research profoundly more exciting in the near term.
[49:53](https://www.youtube.com/watch?v=h0962biiZa4&t=2993s) Stuart Russell on Eradicating Scarcity and Zero-Sum Suffering
Max Tegmark turns to Stuart Russell to ask about his vision for AI's greatest upside. Russell invokes an earlier reference to Jeffrey Sachs, observing that while life is currently amazing for many people in the room, it is decidedly not amazing for vast swathes of humanity who endure profound hardship. While Russell appreciates Nick Bostrom's perspective that reaching elevated states of flourishing could vastly outweigh existing misery, he argues that the moral imperative must remain centered on addressing widespread suffering directly.
Russell points out that Demis Hassabis seems to have peeked at his notes, because the transformative solution lies in eliminating artificial and material scarcity. Scarcity forces human beings to compete in destructive, zero-sum dynamics where survival requires depriving others, driving much of global conflict and malice. As Russell puts it, “eliminating the scarcity basically eliminates the need for people to act in a zero-sum fashion where they can only get by, by making it less possible for someone else to get by, and I think that's the source of a lot of the nastiness that Jeffrey mentioned earlier.” On a humorous concluding note, Russell adds that his secondary, personal upside for advanced AI would simply be not having to read so much email every day.
[51:15](https://www.youtube.com/watch?v=h0962biiZa4&t=3075s) Elon Musk on Bandwidth, Democratization, and Lord Acton
Max Tegmark introduces Elon Musk with deadpan humor, joking that Musk has famously never shared an opinion about the future before. Musk smiles and responds that after giving the issue extensive thought, a positive long-term outcome ultimately boils down to two fundamental challenges: solving the communication bandwidth constraint between the human brain and machines, and ensuring the absolute democratization of artificial intelligence.
Musk invokes a famous political insight from the nineteenth-century historian Lord Acton to justify his emphasis on spreading access widely, summarizing: “There was a great quote by Lord Acton which is that 'freedom consists of the distribution of power and despotism in its concentration.'” He explains that if artificial intelligence tools are widely distributed so that anyone can access them, and humans develop an interface faster than clunky biological fingers, humanity will avoid totalitarian control: “And I think as long as we have - as long as AI powers, like anyone can get it if they want it, and we've got something faster than meat sticks to communicate with, then I think the future will be good.”
[52:16](https://www.youtube.com/watch?v=h0962biiZa4&t=3136s) Audience Q&A: Preference Elicitation and "Democracy 2.0"
Max Tegmark opens the floor to the audience for rapid-fire questions, warning the crowd that caffeine levels are sinking dangerously low before the subsequent panel. He asks participants to state their names, ask genuine questions, and direct them to a single panelist. Deep learning pioneer Yoshua Bengio stands up and addresses Jaan Tallinn. Bengio references Tallinn's earlier talk on eliciting human preferences and values, asking whether algorithmic preference aggregation could pave the way for a more direct democracy, a better society, and safeguards against the malicious misuse of technology.
Tallinn enthusiastically endorses the premise, proposing that such an initiative could legitimately serve as "Democracy 2.0" or "U.N. 2.0." He points out that while countless individuals claim they want to improve the world, it is notoriously difficult to separate genuine public interest from performative rhetoric. Establishing a mathematically sound metric would establish an objective Schelling point around shared preferences. Tallinn observes that modern democratic systems were designed centuries ago; modern computational tools should enable civilization to aggregate human values and preferences far more effectively.
[54:08](https://www.youtube.com/watch?v=h0962biiZa4&t=3248s) Tracking the Frontier: Nicolas Berggruen and Demis Hassabis on AGI Visibility
Investor and philanthropist Nicolas Berggruen poses a question about geopolitical oversight. While the attendees gathered at the conference are well-intentioned and cooperative, AGI development could easily occur far outside this circle, including in nation-states like China or Russia. Berggruen asks who is actively monitoring progress, whether any entity has tabs on rogue efforts, and whether oversight will even be possible once capability takeoff occurs. Max Tegmark invites either Demis Hassabis or Elon Musk to answer.
Demis Hassabis takes the question, returning to his concept that current AI research sits on the steep, difficult part of the S-curve. Because breakthroughs demand extraordinary mathematical and algorithmic talent, the global community of researchers capable of pushing the frontier remains relatively small and closely knit. As Hassabis details:
“So I think that narrows down the scope of the number of groups who can do that, and it also means that they're more visible. So, you know, I think certainly in the West I think most people around here, someone in this room will have contact with somebody who's in those groups who are capable of making meaningful progress towards AGI.”
Hassabis acknowledges that visibility is lower regarding state-sponsored projects in China or Russia. However, because advanced AGI requires a massive footprint of compute, capital, and elite researchers, covert efforts cannot easily remain hidden. He urges Western researchers to build formal bridges with scientific bodies like the Chinese Academy of Sciences rather than relying on guesswork.
```mermaid flowchart TD accTitle: Visibility Factors in Frontier AGI Development accDescr: Diagram illustrating how the high resource footprint and concentrated talent pool make frontier AGI research traceable.
subgraph Bottlenecks["Frontier Constraints"] A["Hard Part of S-Curve"] --> B["Elite Research Talent"] A --> C["Massive Compute & Capital"] end
subgraph Footprint["Structural Consequences"] B --> D["Small, Highly Connected Community"] C --> E["Massive Physical & Resource Footprint"] end
subgraph Oversight["Monitoring & Engagement"] D --> F["High Visibility in the West"] E --> F F --> G["Establish Direct Ties with China / CAS"] end ```
[56:30](https://www.youtube.com/watch?v=h0962biiZa4&t=3390s) Erik Brynjolfsson and Elon Musk on Regulating Public Risk
Economist Erik Brynjolfsson takes the final question of the session. Linking Musk’s call for AI democratization back to a panel discussion where Reid Hoffman highlighted how deeply people care about relative rather than absolute wealth, Brynjolfsson asks whether AI inherently exhibits winner-take-all dynamics that concentrate power in the hands of whoever pulls ahead first, and what concrete institutional mechanisms could guarantee widely shared prosperity.
Musk tackles the question directly by calling for government oversight, openly acknowledging his own distaste for bureaucracy: “I hate to say government agency, like regulators - I'm not the biggest fan of regulators, 'cause they're a bit of a buzzkill.” However, he stresses that society already accepts and benefits from oversight boards like the FAA, the FDA, and automotive authorities because dangerous technologies pose existential public hazards. While governments move sluggishly and react after catastrophes rather than anticipating them, public safety demands enforceable rules for AI.
Musk firmly dismisses the common counterargument that heavy regulation will simply drive frontier AI companies to offshore jurisdictions like Costa Rica or Venezuela, noting that aircraft manufacturers like Boeing do not abandon safety laws to build unregulated planes abroad. Because cutting-edge AI requires concentrated nodes of rare technical genius, the field is practically confined to a few geographic hubs—primarily Silicon Valley, London, and Boston. Consequently, regulatory bodies have clear, accessible choke points to monitor development. Tegmark wraps up the panel on this regulatory consensus and sends the room to the coffee break.
- **Regulatory analogies cited by Musk**: Federal Aviation Administration (commercial aviation safety), Food and Drug Administration (pharmaceutical efficacy and food purity vs. unregulated dietary supplements), and automotive safety boards. - **Key geographic hubs of AI talent**: Silicon Valley, London, and Boston.
## Notable quotes > [23:45](https://www.youtube.com/watch?v=h0962biiZa4&t=1425s) A much worse existential risk is there are no conscious beings in our future. — David Chalmers
> [25:34](https://www.youtube.com/watch?v=h0962biiZa4&t=1534s) If you have an AI that is more intelligent than you and it’s out for your destruction and it’s out for the world’s destruction and there is no other AI that is superior to it, that’s a bad situation. — Ray Kurzweil
## Mentioned Max Tegmark, Elon Musk, Stuart Russell, Ray Kurzweil, Demis Hassabis, Sam Harris, Nick Bostrom, David Chalmers, Bart Selman, Jaan Tallinn, Shane Legg, Eliezer Yudkowsky, Jeffrey Sachs, Lord Acton, Yoshua Bengio, Nicolas Berggruen, Erik Brynjolfsson, Reid Hoffman, Future of Life Institute, OpenAI, DeepMind, Partnership on AI, Chinese Academy of Sciences, Federal Aviation Administration, Food and Drug Administration, Boeing, AlphaGo, Superintelligence: Paths, Dangers, Strategies, Asilomar Conference on Beneficial AI
## Video description Elon Musk, Stuart Russell, Ray Kurzweil, Demis Hassabis, Sam Harris, Nick Bostrom, David Chalmers, Bart Selman, and Jaan Tallinn discuss with Max Tegmark (moderator) what likely outcomes might be if we succeed in building human-level AGI, and also what we would like to happen.
The Beneficial AI 2017 Conference: In our sequel to the 2015 Puerto Rico AI conference, we brought together an amazing group of AI researchers from academia and industry, and thought leaders in economics, law, ethics, and philosophy for five days dedicated to beneficial AI. We hosted a two-day workshop for our grant recipients and followed that with a 2.5-day conference, in which people from various AI-related fields hashed out opportunities and challenges related to the future of AI and steps we can take to ensure that the technology is beneficial.
You can find an audio balanced version of this panel here:
https://www.youtube.com/watch?v=OFBwz4R6Fi0
For more information on the BAI ‘17 Conference:
https://futureoflife.org/ai-principles/
https://futureoflife.org/bai-2017/
https://futureoflife.org/2017/01/17/principled-ai-discussion-asilomar/
[Source](https://www.youtube.com/watch?v=h0962biiZa4)