Sam Altman & OpenAI | 2023 Hawking Fellow | Cambridge Union
Cambridge Union
AI-assisted summary of Sam Altman & OpenAI | 2023 Hawking Fellow | Cambridge Union — original source. Check the original for full context.
Source published:
Key ideas
- Sam Altman argues that merely scaling current large language models on human text will fail to achieve superintelligence, asserting that a distinct algorithmic breakthrough is required to generate novel scientific discoveries. (61:45)
- AGI development is projected to become the most expensive engineering project in human history, with scaling constraints expected to transition from energy to semiconductor manufacturing and packaging bottlenecks. (53:47)
- Altman advocates for an international regulatory agency akin to the IAEA to license and monitor frontier AI models operating above extreme compute thresholds to prevent catastrophic misuse. (24:02)
- Tech executives observe that younger programmers are beginning to outperform veteran engineers because deep fluency with AI coding assistants is outpacing decades of traditional experience. (20:00)
Detailed notes
## Background Published by the Cambridge Union on YouTube on November 15, 2023, this video documents OpenAI being awarded the 2023 Professor Stephen Hawking Fellowship in the Cambridge Union Debating Chamber on November 1, 2023. The event features an address and extended audience Q&A with OpenAI CEO and co-founder Sam Altman, who accepted the honor on behalf of his team to discuss artificial general intelligence, safety, and societal transformation.
## TL;DR In his fellowship lecture and subsequent discussion, Sam Altman reflects on Stephen Hawking's prescient warnings and optimism regarding AI, framing cognitive amplification as humanity's greatest lever for scientific progress. He unpacks OpenAI's evolution from an unfashionable non-profit research lab to a capped-profit entity structured to sustain massive compute demands while prioritizing alignment. Addressing questions from Cambridge students and leadership, Altman discusses the outcomes of the Bletchley Park AI Safety Summit, the necessity of an IAEA-style global regulatory body, and the vital role of empirical, real-world deployment over theoretical safety models. He concludes that while existing language models have advanced dramatically, achieving genuine superintelligence will require entirely new algorithmic breakthroughs capable of generating novel scientific knowledge.
## Atmosphere & style The setting within the historic, wood-paneled Cambridge Union Debating Chamber creates a striking juxtaposition between centuries-old academic tradition and rapid technological disruption. Altman delivers his remarks in a characteristically quiet, measured, and pragmatic cadence, deflating dramatic doomer rhetoric while treating catastrophic risks with serious, analytical sobriety. The dynamic between the student audience and Altman is intellectually rigorous and direct, blending deep philosophical inquiries about consciousness and human agency with candid assessments of geopolitical governance and capital constraints.
## Bulletin - [01:01:45](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=3705s) Sam Altman argues that merely scaling current large language models on human text will fail to achieve superintelligence, asserting that a distinct algorithmic breakthrough is required to generate novel scientific discoveries. - [53:47](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=3227s) AGI development is projected to become the most expensive engineering project in human history, with scaling constraints expected to transition from energy to semiconductor manufacturing and packaging bottlenecks. - [24:02](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=1442s) Altman advocates for an international regulatory agency akin to the IAEA to license and monitor frontier AI models operating above extreme compute thresholds to prevent catastrophic misuse. - [20:00](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=1200s) Tech executives observe that younger programmers are beginning to outperform veteran engineers because deep fluency with AI coding assistants is outpacing decades of traditional experience.
## Key takeaways - [05:00](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=300s) OpenAI was launched in 2015 when AGI was an academic taboo, later adopting a capped-profit subsidiary under a non-profit board to raise historic amounts of compute capital without compromising safety incentives. - [06:50](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=410s) Instilling human values into frontier models through empirical alignment research proved substantially easier in early practice than researchers had anticipated. - [08:30](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=510s) Altman defines his primary personal motivation for developing AGI as accelerating foundational scientific discovery by ten to one hundred times, aiming for breakthroughs like a grand unified theory of physics. - [32:50](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=1970s) The most acute near-term catastrophic risk from advanced AI stems from deliberate human misuse—such as engineered bioweapons or mass cyberattacks—rather than rogue autonomous systems. - [21:45](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=1305s) Universities must avoid banning AI tools and instead integrate them into curricula, as foundational learning and generating novel insights will remain the core human value. - [24:02](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=1442s) Global governance will eventually require an international regulatory body analogous to the IAEA to inspect compute clusters and prevent dangerous frontier proliferation. - [27:47](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=1667s) Elite startup founders require a high tolerance for being misunderstood for extended periods and should build deep expertise at the intersection of two distinct disciplines. - [35:07](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=2107s) Deciding whose values AI should represent is a harder problem than technical alignment, requiring a framework that balances democratic consensus against fundamental human rights protections. - [42:36](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=2556s) Model safety alignment and capability development are not opposing vectors, but rather a single unified research track advancing deep learning. - [47:30](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=2850s) Open-sourcing cutting-edge frontier models carries dangerous, irreversible societal tail risks, distinguishing them from smaller, easily auditable open-source models. - [49:02](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=2942s) OpenAI may never build consumer hardware, anticipating instead a hybrid future split between edge devices for low-latency tasks and massive centralized cloud infrastructure for heavy reasoning. - [55:05](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=3305s) AI safety cannot be solved in theoretical isolation; it requires empirical deployment and iterative real-world interaction across hundreds of millions of users. - [01:01:45](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=3705s) Achieving true superintelligence capable of expanding human knowledge beyond existing text will require a novel algorithmic leap beyond predictive language modeling.
## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=0s) Awarding of the 2023 Professor Stephen Hawking Fellowship
The evening opened at the Cambridge Union debating chamber with an introduction recounting the origins of the Professor Stephen Hawking Fellowship, which was established six years earlier in 2017 in partnership with Professor Hawking. The fellowship was conceived to bring world-leading figures in STEM fields to Cambridge to engage directly with students. Following Hawking's death shortly thereafter, the award grew into a living memorial to his legacy and his enduring relationship with the university. Hawking's family members, including Jane Hawking and Lucy Hawking, were recognized in the audience.
For 2023, the Fellowship Committee took the unprecedented step of conferring the award on a collective rather than an individual: the team behind OpenAI. The host emphasized that products such as DALL-E and ChatGPT had given the general public its first tangible grasp of the sweeping changes artificial intelligence would bring. OpenAI was selected not solely for technical prowess, but for articulating a positive and responsible approach toward artificial general intelligence (AGI), expanding public access, and reshaping global discourse. As the speaker highlighted, OpenAI’s commitment has “successfully shifted the Overton window on AI and AGI in a way that little else has previously been able to do.” Sam Altman was then called forward to accept the honor on behalf of the company and deliver his address.
[02:45](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=165s) Address by Sam Altman: The Prescience of Stephen Hawking
Taking the rostrum, Sam Altman thanked Lucy Hawking and the Hawking family for their hospitality, noting that he intended to keep his prepared remarks brief so as to spend the bulk of the session answering student questions. He described speaking at Cambridge as deeply humbling, recounting how Stephen Hawking’s insights into physics and the future of computation had personally inspired him. Altman stressed that Hawking had been extraordinarily prescient regarding artificial intelligence, and that OpenAI's researchers had drawn significant direction from his worldview.
Altman drew direct links between OpenAI’s philosophy and Hawking’s balanced vision of technology, quoting Hawking's observation: “We cannot predict what we might achieve when our own minds are amplified by AI.” Altman noted that cognitive amplification remains in its earliest infancy and will soon astound the public. He cited Hawking's further hope that advanced tools might repair the environmental and societal harms wrought by previous revolutions: “Perhaps with the tools of this technological revolution, we will be able to undo some of the damage done to the world by the last one, industrialization. We will aim to finally eradicate disease and poverty. Every aspect of our lives will be transformed.” To realize that potential, Altman maintained, deployment must be paired with safety protocols, proactive governance, and societal alignment.
[05:00](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=300s) From an Un-Airconditioned Apartment to Scaling Laws
Altman turned to the founding story of OpenAI to offer encouragement to students facing failure in audacious projects. Founded in 2015 as a non-profit, OpenAI committed itself to building artificial general intelligence that benefits humanity. At the time, the term "AGI" was treated as an academic taboo:
> “At the time, AGI was considered a bad word to say, and we were sort of laughed at by all serious professors in the field.”
He recalled working out of an apartment with no air conditioning, hitting roadblocks, and struggling through periods where virtually nothing worked. A turning point came in 2017 when Altman watched Hawking’s keynote at the Web Summit conference, where Hawking stated: “I'm an optimist, and I believe we can create AI for the good of the world, that it can work in harmony with us. We simply need to be aware of the dangers, identify them, employ the best possible practice in management, and prepare for its consequences well in advance.”
Invigorated by that optimism, the lab released GPT-1 in 2018. Altman candidly admitted that GPT-1 could barely do anything, earned little attention, and was rejected by academic conferences. Yet internal research convinced the team that system performance scaled predictably with compute and dataset size, leading to the formalization of scaling laws.
```mermaid flowchart TD accTitle: Evolution of OpenAI Scaling and Governance accDescr: Flowchart showing OpenAI's timeline from non-profit founding to capped-profit structure and model releases. subgraph Governance["Structure & Capital"] NP["2015: Non-Profit Founded"] CP["2019: Capped-Profit Subsidiary<br/>(Returns Capped, Mission First)"] NP --> CP end subgraph Releases["Iterative Model Releases"] G1["2018: GPT-1<br/>(Rejected by conferences)"] G2["2019: GPT-2"] G3["2020: GPT-3"] CG["2022: ChatGPT<br/>(Low-key preview)"] G4["2023: GPT-4"] G1 --> G2 --> G3 --> CG --> G4 end CP -.->|"Funds Compute & Talent"| G2 ```
Realizing that training frontier models would require unprecedented capital, OpenAI restructured: - Created a capped-profit subsidiary governed strictly by the original non-profit board. - Imposed hard limits on returns to investors and employees, channeling surplus proceeds back to the public charity. - Kept the non-profit charter primary to prevent financial incentives from superseding the core mission of safe deployment.
[06:50](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=410s) The ChatGPT Turning Point and Alignment Realities
The scaling hypothesis yielded rapid successive milestones: GPT-2 in 2019, GPT-3 in 2020, ChatGPT in late 2022, and GPT-4 in early 2023. Altman remarked on the unexpected public reaction to ChatGPT, admitting the team initially viewed it internally as an unassuming experiment: “We called it a low-key research preview, and that's what most of us believed would happen.” Instead, within a year it had accumulated hundreds of millions of users worldwide and reshaped society's concept of intelligence.
Alongside raw capabilities, Altman emphasized the necessity of model alignment. OpenAI uses iterative public releases to gather societal feedback and evaluate safety baselines. He noted an unexpected positive surprise from their empirical work: “Through our alignment and safety techniques, we've been able to instill human values into our models. This is something that many people, myself included, thought might be much harder than it has so far turned out to be.” However, the company continues to run extensive red-teaming, external audits, and real-time monitoring to suppress misuse.
[08:30](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=510s) Managing Existential Downside to Unlock Scientific Breakthroughs
Altman balanced his optimism by invoking Hawking's stark 2017 warning: “AI could be the worst invention in the history of our civilization.” In frontier AI, Altman argued, severe existential hazards and unprecedented upsides are inherently linked; one cannot capture the benefits without methodically confronting the catastrophic risks.
Explaining why OpenAI pushes forward despite these hazards, Altman pointed to the potential transformation of global living standards: - **Global Healthcare:** AI can assist in regions facing severe doctor shortages and, over decades, potentially solve complex pathologies and diseases. - **Democratized Education:** Personalized, interactive AI tutoring adapted to the individual cognitive pace and interests of each student. - **Scientific Acceleration:** Altman identified the acceleration of foundational discovery as his primary motivation, concluding: “We on our own, maybe we can accomplish a certain rate; with AI, maybe we can make that rate ten times or a hundred times faster. And I think all real human progress comes from deepening our scientific understanding of the world.”
[10:00](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=600s) A Scientific Approach to AI Risk and Hawking's Call to Action
Altman opens by emphasizing that society must take the full spectrum of AI safety risks seriously, spanning from present-day tools to future systems that will inevitably be classified as superintelligence. OpenAI is focusing on building a quantitative, empirical framework rather than indulging in doom-laden predictions. Altman criticizes purely fatalistic attitudes toward technological disruption: “The sort of current rhetoric where we just say it's all going to be horrible and there's nothing that can be done is not very productive and certainly not very scientific.” OpenAI is dedicated to solving the alignment of superintelligent systems over the coming years while collaborating directly with governments and civil society.
```mermaid flowchart TD accTitle: Transition from Current Systems to Superintelligence accDescr: Flowchart illustrating the path from current risks to safe superintelligence through evidence-based research and collaboration. subgraph S1["Current Landscape"] A["Current AI Systems"] --> B["Superintelligent AI"] end subgraph S2["Risk Mitigation Strategy"] B --> C["Quantitative, Evidence-Based Methodology"] C --> D["Evaluate, Forecast & Protect"] end subgraph S3["Global Collaboration"] D --> E["Governments & Civil Society"] D --> F["Interdisciplinary Academic Research"] E & F --> G["Safe & Equitable AGI"] end ```
Addressing the Cambridge audience directly, Altman expresses admiration for the university's intellectual legacy, admitting a touch of envy that the United States lacks an exact equivalent. He appeals to students and researchers across disciplines to contribute via research, policy advocacy, or beneficial applications. Concluding his prepared remarks, Altman invokes the words of Professor Stephen Hawking, urging the audience to push past theoretical discourse into concrete planning, before formally accepting the 2023 Hawking Fellowship Award amid applause.
- Altman stresses that building safe Artificial General Intelligence (AGI) must be a border-transcending, collective endeavor. - Both unlocking the capabilities of AI and guaranteeing its equitable distribution are described as fundamental "moral imperatives." - Hawking’s quoted charge challenges the room to stand on the “threshold of a brave new world” as pioneers willing to think big.
---
[12:27](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=747s) The Capped-Profit Structure and Capital Requirements of Scale
The conversation transitions to a moderated Q&A format. The moderator inquires about the foundational philosophy differentiating OpenAI from its peers, particularly how its corporate model supports its mission. Altman recounts the evolution of OpenAI's structure, reflecting on their early uncertainty regarding where AI technology was headed and how much leverage they could muster. Initially, the founders evaluated three potential organizational setups: a non-profit, which risked not getting enough done; a standard for-profit company, which risked driving forward too aggressively without restraint; and a government entity, which risked stagnation followed by an uncontrolled surge. Choosing the non-profit model seemed prudent at the outset, but they rapidly recognized a major miscalculation: they fundamentally underestimated just how much scale mattered.
Realizing that raising tens or hundreds of billions of dollars within a conventional non-profit structure was impossible, OpenAI devised its hybrid "capped-profit" model. This setup preserved non-profit governance—retaining the mandate to hand the technology over to humanity when appropriate—while leveraging capitalist markets to finance astronomical computing costs. Altman projects that OpenAI’s push toward AGI will likely become “the most expensive engineering project in the history by the time it's done.”
Reflecting on corporate behavior, Altman notes that previous tech giants did not falter because their founders were ill-intentioned, but rather because bad incentive structures inevitably corrupt positive goals: “In fact, so much so that any time you can spend thinking about how to correctly set the incentives on a person, a team, an organization, whatever, that's like probably the highest leverage thing you can do.” Capping financial returns eliminates the systemic pressure to pursue profit maximization at the expense of safety and public good.
---
[15:00](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=900s) Aligning Incentives and Relations with Elon Musk
Altman expands on the moral dimension of technological progress, arguing that the public conversation often focuses so heavily on risks that the transformative upsides are obscured. He highlights the profound gains in global quality of life that could occur if access to high-grade intelligence grew tenfold year after year. Fulfilling that vision requires strict governance rules so teams avoid being pushed into dangerous shortcuts by market incentives.
The moderator asks about OpenAI co-founder Elon Musk, who departed the organization and publicly advocated for a pause in the training of cutting-edge frontier models. Altman shares that he had just crossed paths with Musk earlier that day at the UK AI Safety Summit held at Bletchley Park. Downplaying interpersonal friction, Altman offers a brief and diplomatic response: “I think we share much more in common than we think differently about this.” He expresses enduring respect for Musk and affirms his desire for constructive future collaboration between their initiatives.
---
[17:19](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=1039s) Progress Toward AGI and Defining Success for the Next Decade
When prompted to reflect on the rollout of systems like ChatGPT and DALL-E, Altman firmly clarifies that true AGI has not yet arrived. Nonetheless, he points out that showing GPT-4 to researchers five years prior would have exceeded their expectations of how close machines could come to generalized reasoning. Even if current models exhibit only nascent, fragile understanding, their capacity to synthesize complex concepts and parse human language constitutes a monumental milestone.
Altman admits that internal teams at OpenAI find the current iteration of GPT-4 almost embarrassing because they are intimately familiar with its blind spots, hallucinations, and structural limitations. Even so, the model delivers measurable utility to hundreds of millions of users worldwide. When the moderator asks what OpenAI considers success over a ten-year horizon, Altman describes the company's core identity as that of a "tool builder": “And if the tools are so good that they are enabling us to live much better lives, to if someone can go discover the grand theory of all of physics in ten years using our tools, that would be pretty awesome.”
- The rapid adoption of GPT-4 proved that deeply useful AI arrived much faster than common industry projections suggested. - Success in ten years is framed not around self-serving corporate metrics, but around external human breakthroughs unlocked by AI tooling, such as unifying fundamental physics.
[20:00](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=1200s) Advice to the Next Generation and How AI Is Changing the Workforce
Asked how young people can adapt to an AI revolution whose impact they will feel most directly, Sam Altman points out that the youth naturally adapt best to every technological shift. In fact, he recounts a striking trend recently reported to him by CEOs of major tech corporations. Historically, software engineering managers could reliably track programming productivity: fresh 22-year-old college graduates would enter a company and improve predictably by about 30% each year until hitting their late twenties, at which point career paths diversified. Now, however, those same executives report an unprecedented reversal:
> “For the first time, these CEOs have observed that the younger people are outperforming the older people as programmers.”
What junior engineers lack in raw years of experience, Altman explains, they more than compensate for through intuitive mastery of modern AI tools. He underscores how profound this shift is:
> “What they lack in experience, they make up for with familiarity with AI tools. And this is like, I think, a bigger deal than it sounds like on the surface.”
Looking at the broader historical picture, Altman hopes this technological disruption breaks a recent period of stagnation. He notes that while younger generations have historically driven aggressive innovation and social change, that dynamic had slowed down over the past decade. Because technological revolutions shake established foundations, he expresses hope that young people will leverage this moment to unseat entrenched problems and aggressively lead the future.
---
[21:45](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=1305s) How Universities Must Adapt to AI
Addressing an audience at the University of Cambridge—an institution more than eight centuries old—the moderator asks how higher education needs to evolve when its fundamental methods of teaching and learning have remained largely consistent for hundreds of years. Altman reflects on his own academic path, asserting that the primary value he gained from school boiled down to two essential meta-skills: first, the meta-skill of learning how to learn new things, and second, learning how to produce novel, creative thoughts that could not simply be looked up. The specific domain facts were merely useful training data for that core cognitive engine, and he insists that dynamic will remain completely untouched by AI.
Altman draws a historical parallel to the emergence of search engines when he was a student. Older peers and graduates from five years before him faced school administrators who tried to ban Google outright, demanding students sign pledges never to use it on the premise that effortless fact-finding would destroy education. Altman views the early reactionary bans against ChatGPT as an identical, short-lived panic:
> “Now people have decided that they're gonna embrace it and learn how to use it and that, I think, is a much better idea.”
He warns that educational institutions do a disservice if they shield students from modern tooling, because the tools of education and professional life will continuously evolve:
> “But you have got to teach people to fully use all of the tools that are available. Otherwise, they will just underperform everybody else.”
---
[24:02](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=1442s) Progress at the Bletchley Park AI Safety Summit and Global Governance
Having arrived directly from the UK AI Safety Summit at Bletchley Park, Altman describes an unexpected degree of international consensus regarding the governance of advanced models. Across world leaders, international delegates, and frontline AI labs, he sees an unprecedented alignment on the imperative to regulate powerful systems—beginning with existing capabilities and scaling in rigor as system autonomy and risk levels grow. He anticipates that reasonable baseline policies are already being drafted across various jurisdictions, but predicts that humanity will eventually require a unified international framework:
> “I think we will at some point need a global regulatory body which we don't get often, but we had one for atomic energy. I think we'll decide we need something like that as we get closer to AGI.”
```mermaid flowchart TD accTitle: Trajectory of AI Safety and Regulation accDescr: Flowchart illustrating the evolution of AI oversight from current models to AGI-level international governance. subgraph CurrentPhase["Near-Term Oversight"] A["Existing Frontier Models"] --> B["National Legislation & Sensible Guardrails"] end subgraph EscalationPhase["Scaling Capabilities"] B --> C["Increasing System Risk & Capabilities"] C --> D["Multilateral Alignment Among Nations & Labs"] end subgraph LongTermGovernance["High-Capability / AGI Regime"] D --> E["Global Regulatory Agency (Modeled on IAEA for Atomic Energy)"] end ```
Noticing protest flyers or papers dropped in the hall during the event, Altman acknowledges the fear behind calls to halt development altogether. He expresses genuine empathy for those anxieties, noting that every major leap in technology evokes understandable panic. However, he rejects calls for an outright moratorium on AI progress as deeply naive:
> “The default state of the world is that things decay and get worse and worse. And the only way that not only do we make life better, which again I think is a moral obligation for all of us, but even keep push against that decay and all like kill each other, destroy each other, is with progress and increasing quality of life, and the only way we get that is through technology.”
Halting development is not only unworkable in practice, but forfeits the massive benefits required to elevate society. Instead, Altman insists on finding a pragmatic middle path that couples vigilant safety and responsibility with steady technological expansion.
---
[26:27](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=1587s) The British Government's Regulatory Approach
When pressed on what the UK government is getting right and wrong in its handling of AI safety, Altman jokes that he almost hesitated to voice his opinion out of fear of causing offense, before realizing the comment was actually a compliment. He praises the UK’s defining cultural temperament:
> “One of the things that I really admire about British culture is I think British people, British society, British institutions. They're very reasonable.”
He contrasts the British approach with two unhelpful global extremes: - Certain nations that advocate doing nothing, adopting a laissez-faire stance that governments should never intervene or slow technology down. - Other jurisdictions that admit to having little technical competence but jump straight into heavy-handed bans and preemptive shutdowns simply because they know how to draft restrictions.
By avoiding both inaction and blanket bans, Altman argues, the British government has charted a nuanced, thoughtful path that serves as a positive model for the rest of the world.
---
[27:47](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=1667s) Essential Founder Qualities and Entrepreneurship
Drawing on his years as president of Y Combinator, Altman looks past standard venture capital clichés—such as general intelligence or raw idea quality—to pinpoint the less obvious traits that determine startup success. While acknowledging that fierce determination is mandatory, he emphasizes a founder's psychological stamina:
> “Willingness to be misunderstood for a long period of time. The best companies are not the ones that are the most fashionable.”
Because the best ventures often look unappealing at inception, Altman observes that starting an AI application company today might paradoxically be unideal purely because it is fashionable, though the sheer scale of the AI platform shift might prove to be an exception.
Beyond defying conventional opinion, Altman highlights three intertwined capabilities that set elite founders apart: - **Internally Driven Commitment:** The fortitude to persist for years without positive feedback, accolades, or external validation. - **Deep Problem Obsession:** An unrelenting, intrinsic impulse to grind through structural setbacks out of genuine fascination with the problem space. - **Mastery of Evangelism and Clear Communication:** The ability to articulate ideas compellingly, which Altman notes is vital for hiring top personnel, convincing investors, and explaining the company's ultimate mission to the public.
[30:14](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=1814s) Website Crawlers and Accessibility for People with Disabilities
The question-and-answer session begins with an audience question from Amy, the head of social media at the University of Cambridge. She explains that over the summer, her team developed guidelines for using ChatGPT to draft campaign ideas, student emails, and social media quizzes. However, she seeks Altman’s direct advice on a major policy dilemma: should Cambridge leave its university website open to AI web crawlers from OpenAI and other developers, or should it block access, following the path taken by publications like *The New York Times* and *The Guardian*? Altman answers that the decision ultimately depends on what Cambridge wants future AI systems to know:
> “I think that really depends on how you want future versions of GPT to work? Like, do you... Do you prefer that in GPT-5 and 6 is all the information on that website or not?”
He points out that organizations operate on differing philosophies: some companies adopt business models requiring payment before their content can be accessed, whereas many other content creators actively prefer their knowledge to remain broadly accessible.
```mermaid flowchart TD accTitle: Data Access Models for AI Web Crawlers accDescr: Decision pathway for institutions deciding whether to allow AI crawlers on their websites based on business and distribution goals.
A["Content Owner Strategy"] --> B{"Primary Objective?"} B -->|"Monetization / IP Control"| C["Block Crawlers<br/>(e.g., The New York Times, The Guardian)"] B -->|"Public Knowledge & Reach"| D["Open Access Crawlers<br/>(Include in GPT-5, GPT-6)"] ```
The next question comes from an audience member who asks what concrete steps OpenAI is taking to ensure that AI remains genuinely accessible, specifically to people with disabilities. Altman states that OpenAI approaches accessibility from two distinct angles: economic affordability and multi-sensory capability. On the financial side, he emphasizes OpenAI's commitment to maintaining a robust free version of ChatGPT for anyone who wants it, alongside an active effort to lower API inference costs:
> “So, you know, the number one developer request on us is just, can you make GPT-4 cheaper?”
Regarding physical accessibility, Altman points out that the software has evolved beyond raw text to integrate multimodality—introducing image comprehension and two-way voice/audio interfaces, with more input types planned. He envisions the technology developing into a tool people can interact with passively throughout their daily lives in whichever modality suits their physical needs, noting that the model itself will eventually help optimize its own accessibility design.
- **Developer Priorities**: The single most frequent request from third-party builders is making GPT-4 cheaper so that international developers and cost-constrained projects can afford to deploy it. - **Multimodal Accessibility**: Expanding beyond text to bidirectional audio and vision processing allows people with physical or visual impairments to navigate the service intuitively.
---
[32:50](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=1970s) Catastrophic Risks, Misuse, and Global Compute Governance
A student at the back poses a time-travel thought experiment: imagine a future version of Altman walks into the chamber 15 years from now and reveals that, despite all of OpenAI's rigorous safety protocols, an AI catastrophic failure resulted in the loss of roughly one billion human lives. The student asks Altman what he believes would be the most likely cause of such a catastrophe and what marginal intervention today could prevent it. Altman responds that malicious human intent represents the greatest danger:
> “I think the most likely way this... something like that happens is intentional misuse. So a person who is using the tools to intentionally cause harm.”
While accidental catastrophes during training runs or subtle model-driven persuasion remain serious safety concerns, Altman states that it requires "not too many deductive leaps" to see how an actor weaponizing AI for a mass bioweapon or a catastrophic cyber attack could inflict immediate, vast devastation.
To mitigate that scenario, Altman advocates for an international oversight framework. He calls for a global regulatory authority tasked with inspecting and licensing frontier models operating above an established compute and power threshold:
> “And if it turns out that because of algorithmic progress, it's not, then we're definitely going to have some challenges as a world.”
He stresses that such regulations must be crafted carefully so they do not strangle open-source developments or small-scale hobbyist innovation. The entire strategy rests on the hope that the physical computing threshold required to unlock dangerous capabilities remains high enough to monitor; if novel algorithmic efficiencies make dangerous capabilities attainable on low-compute platforms, controlling misuse will become drastically harder for the international community.
---
[35:07](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=2107s) Pluralistic Value Alignment and Collective Moral Preferences
An audience member asks about the philosophical foundations of value alignment: when building systems intended to benefit all of humanity, how does OpenAI decide which values to encode, how does it navigate ethical dilemmas with conflicting moral hierarchies, and how can a single model faithfully represent a diverse, pluralistic world? Altman acknowledges that solving the core mathematics of alignment is only the preliminary hurdle:
> “I think part one of our challenge is to solve the technical alignment problem. And that's what everybody focuses on. But part two is to whose values do you align the system once you're capable of doing that? And that may turn out to be an even harder problem.”
Altman shares an exploratory, working idea that he notes he might later revise as he learns more: utilizing the interactive nature of AI itself to crowdsource human ethics. Because ChatGPT can communicate with hundreds of millions of people individually, it could actively poll users—or humanity as a whole—to establish societal consensus on thorny ethical boundaries.
```mermaid flowchart TD accTitle: Interactive Value Alignment Framework accDescr: OpenAI's proposed process for establishing model boundaries and ethics through large-scale user dialogue and democratic preference setting.
subgraph Consensus["Broad Democratic Input"] A["ChatGPT Dialogue with Global Users"] --> B["Establish Default Societal Baseline"] end
subgraph Boundaries["Boundaries & Customization"] B --> C["Hard Outside Edges<br/>(Absolute Bans, e.g., Violence)"] B --> D["User Customization Spectrum<br/>(Personalized Nudges & Styles)"] end
subgraph Model["Frontier System"] C --> E["Aligned AGI System"] D --> E end ```
He suggests using this interactive model to determine the default baseline behavior, delineate the outer guardrails of unacceptable outputs (such as absolute bans on causing physical harm), and define how much room individual users have to customize their system's persona. He also wonders whether conversational systems could serve an educational role by exposing users to alternative viewpoints and showing them the externalities of their choices, or even nudging humanity toward moral progress:
> “And the idea that AGI reflects the collective moral preferences of everyone on Earth seems to me like at least a good baseline to start with.”
---
[37:28](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=2248s) Scientific Discovery: Academia vs. Startups and Intersecting Skills
A PhD student follows up on Altman’s vision of using AI to accelerate the scientific frontier. The student confesses that doing research within academia can feel like watching a slow-moving glacier, whereas industry feels like the place where breakthrough progress is happening, and asks where young scientists should direct their careers.
Altman addresses his own past with a grin and self-deprecating candor. He admits that as a kid he had planned on becoming an academic before pivoting to industry—a detour that left him exceptionally happy. However, he warns the student not to follow his trajectory uncritically:
> “I'm biased because... I thought when I was a kid that I wanted to be an academic, and then I decided to go work in industry, and I've been super happy about it. That reroute. So you shouldn't listen too much to me because it worked this way for me and it works this other way for others, but I'm certainly biased to thinking that you can do amazing knowledge advancement work in industry and in startups in particular. But I think you've got to discount that close to 100% coming from me.”
Regardless of whether one chooses an academic lab or a startup incubator, Altman shares a broader rule for professional leverage: young researchers should avoid the trap of becoming narrow, hyper-specialized single-domain experts. Instead, he advises combining deep rigor in a scientific domain with native fluency in modern AI tooling:
> “I'm a big believer that if you can get really good at two things with an important intersection, that's when you can massively contribute to the world.”
He observes that narrow disciplines are fiercely competitive, but researchers who master the cross-section of advanced domain knowledge and applied machine learning unlock outsized leverage and career advantages.
---
[39:20](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=2360s) Critical Thinking, Automation, and Self-Inflicted Intellectual Atrophy
An attendee transitions to a philosophical concern regarding the downstream cognitive consequences of AI. Congratulating Altman on OpenAI's technological milestones, the questioner clarifies that his greatest fear is not malice inflicted on people by others, but self-inflicted cognitive decay. He argues that people have an overwhelming economic and convenience incentive to offload their own critical thinking to AI and AGI. Citing the Roman emperor and Stoic philosopher Marcus Aurelius, the attendee cautions that when human intelligence and reasoning wither, society will be reduced to little more than "dead bodies walking around," and begins pressing Altman on how OpenAI intends to mitigate this creeping atrophy of human intellect as systems advance.
[40:00](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=2400s) Cognitive Capacity, Tools, and Historical Moral Panics
Sam Altman rejects the notion that artificial intelligence will erode human cognitive abilities or make people dumber, arguing instead that superior tools allow humans to operate at progressively higher levels of abstraction. When provided with more capable tooling, individuals are able to hold larger problems in their heads and free up mental bandwidth for harder tasks. While acknowledging that some subset of the population will always choose to spend their time taking drugs or playing video games all day, Altman stresses that the vast majority of people are driven by an inherent desire to create, contribute to society, feel useful, and satisfy their curiosity. AI accommodates both choices: it enables people who wish to do less to do less, while empowering those who want to achieve more to do exponentially more.
Reflecting on past technological revolutions, Altman notes that every major shift triggers a predictable moral panic claiming society is losing something irreplaceable and that human life is doomed to get worse. In reality, modern humans achieve vastly more than people did centuries ago because of accumulated societal scaffolding and tools like computers and the internet. Previous generations labored intensely to invent systems whose ultimate fruits they would never live to see, driven by the conviction that future generations would stand on their shoulders, think harder, and live healthier, more capable lives. Looking ahead, Altman envisions future generations achieving feats so profound that contemporaries might feel a touch of regret for not growing up in an era of superintelligence. Altman summarizes his stance directly: “I don't think AI makes us dumber. I think it's going to turn out to make us much, much smarter.”
[42:36](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=2556s) Capabilities and Alignment as a Single Vector, and Government Talent
An audience member asks how society can ensure that top computer scientists, engineers, and AI researchers are evenly distributed across core model development, AI alignment, and government policy when the overwhelming share of private capital flows straight into capabilities development.
Altman responds by admitting he used to share the view that capabilities research and safety alignment were orthogonal, competing vectors. Over time, his perspective shifted as empirical progress revealed that the underlying deep learning mechanics are fundamentally shared. Major breakthroughs originally pursued under the banner of alignment turned out to yield the biggest capability advances, and vice versa:
```mermaid flowchart LR accTitle: Unified Deep Learning Progress accDescr: Diagram showing how deep learning research drives both capability leaps and alignment techniques together. subgraph Progress["Deep Learning Mastery"] DL["Pushing Back Veil of Ignorance"] end subgraph Synergies["Dual Outcomes"] CAP["Capabilities Leaps"] ALN["Alignment Techniques"] end DL --> CAP DL --> ALN CAP <-->|"Techniques & Insights"| ALN ```
As Altman describes it: “And so I think of it as like a sort of surprisingly one-dimensional vector of progress, which is how good are we at pushing back the veil of ignorance on deep learning, and then we can use that to make a successful system, which is a combination of capabilities and alignment.”
Addressing the state capacity side of the question, Altman rejects the standard governmental excuse that premier technical talent only wants to work in the private tech sector. In his experience, brilliant and thoughtful technologists possess a genuine appetite to work on AI governance inside public institutions; governments simply have to muster the administrative will to create those roles and recruit them aggressively.
[44:15](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=2655s) Challenging Tech Utopianism, Social Media, and the Threat to Democracy
A student challenges Altman's optimistic narrative, drawing a parallel to the early utopian promises of social media as an engine for the democratization of knowledge. Instead of democratizing society, the student argues, massive corporations monopolized information flows, monetized them aggressively, and sold attention to the highest bidder—paving the way for AI to automate mass disinformation and micro-targeted propaganda. Pointing to OpenAI's backing by wealthy private investors, the questioner asks Altman point-blank whether he worries he has “sacrificed democracy on an altar of utopianism.”
Altman gives an emphatic answer: “No.” While he acknowledges the severe negative consequences of social media—specifically its harm to young people's mental health and its role in accelerating political radicalization—he states that he would still not push a hypothetical button to undo social media's existence. In his view, social media arrived faster than institutions and regulatory bodies could formulate guardrails, representing a failure of institutional governance rather than an inherently evil technology.
Turning to the student's criticism, Altman sharply rejects the paternalistic premise that an elite class should decide what information is legitimate for the public: - He argues that decentralizing and democratizing access to information is fundamentally a net good for the world. - He criticizes the presumption among people at elite universities who believe they “just know better” than ordinary citizens about what information should be consumed. - He agrees with the audience member on the dangers of ad-based business models, stating that advertising creates an intrinsic misalignment between a platform's incentives and its users' well-being. - He condemns the attention-hacking mechanics pioneered by consumer tech platforms. - While OpenAI will inevitably encounter novel, unforeseen failure modes, Altman insists the objective must be to learn from past corporate mistakes rather than duplicate the misaligned incentives of social media companies.
[47:30](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=2850s) Open Source Models and Calibrating Societal Risk
An audience member highlights criticisms raised by prominent researchers Andrew Ng and Yann LeCun, who argue that apocalyptic warnings about artificial general intelligence (AGI) are used opportunistically to stifle open-source development and concentrate power inside a handful of incumbent tech corporations. The questioner asks Altman for his stance on open-source AI models.
Altman affirms that open-source software is vital and positive for individual empowerment, endorsing the release of open models at reasonable scales where empirical testing ensures developers are not committing an irreversible error. However, he strongly breaks with the absolutist position that all cutting-edge frontier models should be made open source immediately upon training:
- He points out that while society typically self-corrects through trial and error, frontier AI introduces unique tail risks that demand precaution. - Referencing Andrew Ng directly, Altman remarks: “Andrew, he was my like undergrad professor, I respect for him, but I like I also respectfully disagree that we should just like instantly open source every model we train.” - Characterizing unrestricted frontier model releases as “insanely reckless,” Altman states that OpenAI will refuse to adopt such an approach, framing the disagreement as a question of where one calibrates acceptable risk when gambling with societal safety.
[49:02](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=2942s) Consumer Hardware, Chips, and Physical-World AI
A finance student named Pantitra asks when OpenAI can be expected to release hardware. Altman responds with candor: “Possibly never, unfortunately.”
Seeking clarification on whether the audience member means consumer devices, in-house silicon chips, or robotics, Altman notes that although it would feel “a little bit sad” if OpenAI's technologies never interacted directly with the physical world, pure cognition remains the company's paramount objective and the higher-order bit of the AI transition.
[50:00](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=3000s) AI Hardware Form Factors and Ilya Sutskever’s Test for Consciousness
Altman addresses the open question of hardware in the era of artificial intelligence. He maintains that whenever a major technological shift occurs, humanity gets an opportunity to rethink computing platforms from first principles. Because computers can now genuinely understand natural language and execute intricate workflows with minimal human input, failing to pioneer an entirely new category of physical device would feel like a missed opportunity. At the same time, he acknowledges the sheer difficulty of competing with the smartphone, pointing out that modern phones are remarkably capable and versatile general-purpose machines, setting an exceptionally high bar for any contender.
The conversation then turns to the nature of machine consciousness. When asked by an audience member for his definition of consciousness and pressed on whether OpenAI might already possess a conscious system in secret, Altman quickly dismisses the secrecy premise. Turning to the difficulty of testing for subjective awareness, he describes a thought experiment formulated by OpenAI co-founder and Chief Scientist Ilya Sutskever. Because measuring internal experience directly is so fraught, the experiment relies on rigorous data sanitization:
```mermaid flowchart TD accTitle: Ilya Sutskever Consciousness Thought Experiment accDescr: Diagram showing the process of training a model on data stripped of consciousness concepts and testing whether it recognizes the subjective experience when described.
subgraph DataPrep ["1. Data Curation"] A["Standard Training Corpus"] --> B["Strip All Mentions of Consciousness & Subjective Experience"] B --> C["Exclude an Unrelated Baseline Concept as Control"] end
subgraph TrainingEval ["2. Model Training & Baseline Probe"] C --> D["Train Frontier Model on Redacted Data"] D --> E["Query Model on Control Concept"] E --> F["Model Confirms: 'I know nothing about that'"] end
subgraph TestPhase ["3. Consciousness Evaluation"] F --> G["Introduce Concept of Consciousness & Describe Subjective Human Experience"] G --> H{"Model Reaction"} H -->|"Aha Moment: 'I recognize that feeling, but never had the word' "| I["Potential Indicator of Genuine Consciousness"] H -->|"Blank Slate: 'I have no concept of this' "| J["Non-Conscious System"] end ```
Altman notes that if a model trained without any textual artifacts of subjective experience can immediately recognize and map an explained human internal state to its own operations, that result would demand serious scientific attention.
- As Sam Altman promised regarding secret advances: “We would definitely tell the world if we had one we thought was conscious.” - Sutskever's proposed test requires expunging not just direct mentions of consciousness, but even adjacent concepts describing subjective experience, alongside a separate control concept to verify whether omission prevents recognition.
---
[52:53](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=3173s) On-Device Edge Models versus Scaled Cloud Compute
An audience member asks about the proliferation of smaller, localized open models and whether the future of artificial intelligence will live primarily on personal devices like phones and iPads without internet access, or if cutting-edge cognitive work will stay tethered to the cloud. Altman outlines a bifurcated ecosystem rather than a winner-take-all scenario. While localized models operating on consumer hardware are essential and worth supporting, the heaviest computational loads and premier reasoning capabilities will inevitably demand the resources of large-scale cloud clusters.
- On the structural split between local hardware and centralized compute, Sam Altman states: “I think the future will be a hybrid.” - He emphasizes that while on-device models handle latency-sensitive and offline tasks, the high-value cognitive labor users desire will remain centered in massive cloud infrastructure.
---
[53:47](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=3227s) Compute Bottlenecks and the Coming Abundant Energy Revolution
The debate moves to the intersection of artificial intelligence and energy limits. A student asks whether the dramatic computational and energy demands of AI models could prove incompatible with safe development, especially if energy becomes drastically cheaper. Altman reveals that AI and energy are the two domains occupying almost all of his strategic focus. He anticipates a dramatic shift in energy economics in the near future, driven by rapid progress in nuclear fusion alongside continuing gains in solar generation and storage technology.
Even under optimistic projections for limitless clean power, Altman argues that energy will not remain the primary bottleneck for scaling AI. Instead, the ultimate constraint will rest in hardware manufacturing and the broader semiconductor supply chain. Fabricating advanced chips, moving raw materials, and packaging specialized silicon involve complex physical logistics that even zero-cost electricity cannot immediately resolve.
- Sam Altman anticipates rapid breakthroughs in power generation: “And I think we are just a few years from like very cheap, very abundant fusion, to say nothing of what's almost certain to happen with solar and storage.” - On the enduring structural limits of AI hardware, he stresses: “Even if our most wild predictions about what the economics of fusion can look like all come true, we will still be limited by the chips we can make and the entire supply chain there for a long time, but it does not change the physics of this.”
---
[55:05](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=3305s) Responsible Scaling, Preparedness Frameworks, and the UK AI Summit
A student studying the ethics of AI cites a fresh evaluation by Cambridge’s Centre for the Future of Intelligence (CFI) analyzing compliance with UK government safety recommendations. The study ranked Anthropic first at 82%, OpenAI second at 74%, and Meta last at 48%. The questioner asks whether OpenAI will commit to publishing its risk thresholds and whether those thresholds will be shaped by democratic consensus or left solely to internal technical teams. Altman nods to his colleague Aleksander Madry, who was present in the chamber and heads OpenAI’s Preparedness team. Altman commits to publishing the framework, though he emphasizes that its value lies in ongoing revisions rather than any single static document.
```mermaid flowchart LR accTitle: AI Safety and Governance Pipeline accDescr: Flowchart illustrating the movement from internal company frameworks to empirical deployment and international societal governance.
A["Frontier AI Labs (OpenAI Preparedness Team)"] -->|"Publish Living Frameworks & Risk Thresholds"| B["Real-World Deployment & Empirical Contact"] B -->|"Iterative Safety Feedback from Millions of Users"| C["Global Oversight Body (IAEA-Style Agency)"] C -->|"Societal Agreement on Safety Rules"| D["Binding International Standards"] ```
Altman argues that private corporations cannot be the final arbiters of safety policy. Drawing an analogy to the International Atomic Energy Agency (IAEA), he insists that an international body must ultimately establish binding baselines for society. He cites this necessity as the exact reason he attended the UK AI Safety Summit at Bletchley Park despite logistical friction. Furthermore, he challenges the view that pure lab caution equates to real safety, arguing that true system robustness can only be discovered through real-world deployment with millions of active users.
- Emphasizing the limits of corporate self-policing, Sam Altman argues: “Fundamentally, companies shouldn't regulate themselves here.” - On the role of global oversight, he adds: “Other companies are going to contribute what they think. But eventually, this global IAEA-like body, or whatever we decide on, is got to say, ‘Okay, you all have these drafts, that's great, thank you for doing that, but here's what we're going to agree to as a society.’” - Contrasting armchair analysis with live operational feedback, Altman insists: “Safety in theory is easy. The hard thing is to make a system that is really useful to hundreds of millions of people and is safe.” - Criticizing labs that minimize real-world releases, he remarks: “It is easy to be safe by not releasing things, but you actually know nothing about how to make the world safe that way. This kind of work can only do it with contact with reality, and that's the hard part.”
---
[57:39](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=3459s) Investing in Longevity Science and Cellular Reprogramming
The topic shifts toward Altman’s personal venture investments in biotechnology and longevity therapeutics. When asked what role government policy should play in accelerating the industry, Altman clarifies that he is not a domain expert and treats the sector as a high-risk, high-reward frontier. Despite that caveat, he sees immense promise in cellular rejuvenation, singling out partial reprogramming as an underappreciated breakthrough that warrants deeper academic and financial support.
Given that age remains the predominant risk factor across modern chronic illnesses, Altman argues that treating biological aging directly would deliver massive macroeconomic dividends, slashing national healthcare costs and extending human healthspan. He characterizes his financial backing of anti-aging biotech as an asymmetric lever for global health.
- Sam Altman frames longevity research as an unproven frontier: “I view this as still very much in the moonshot category. It's still very risky science.” - Highlighting the scientific promise of epigenetic reset, he notes: “I think partial reprogramming is one of the coolest things I've seen in bio in the last decade, and it does not seem to be getting as much attention and study as I think it deserves.”
---
[59:08](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=3548s) Human Rights Frameworks versus Descriptive Majoritarian Ethics
The final question of the exchange revisits Altman’s earlier thoughts on establishing a values hierarchy for artificial intelligence through broad user feedback. The audience member argues that relying on majoritarian opinions risks entrenching societal biases like sexism and racism, pointing to instances where models reflexively assume high-status professions like doctors must be men. The questioner asks whether AI systems should integrate codified international human rights frameworks or effective altruist principles, pressing Altman on whether models should be fundamentally descriptive of the status quo or prescriptive of human ideals.
Altman agrees that raw majoritarian feedback is insufficient on its own. He explains that any balanced alignment architecture must explicitly build in universal human rights frameworks rather than just averaging human inputs, ensuring that foundational liberties and non-discrimination principles override prevailing biases.
[01:00:00](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=3600s) Democratic Design and Avoiding Paternalism in AI Values
Continuing his thoughts on how an AI governance framework should reflect human rights and user preferences, Sam Altman reflects on the inherent difficulty of building an equitable system when different demographics participate at vastly different rates. While acknowledging that any governance structure must be aware of and compensate for these disparities in a way that people broadly accept, Altman adds a frank disclaimer about the sheer complexity of the task, admitting he might end up disavowing any specific idea because getting the architecture of such a system right is extraordinarily difficult. Yet, he insists on one guiding principle: the system must ultimately reflect the genuine will of the people who interact with it and bear its consequences, rather than conforming to a centralized, elite-imposed vision.
Altman challenges a subtle paternalism he often detects among well-meaning intellectuals and policymakers who question broader public input. While he recognizes that those asking how to safeguard values are acting in good faith, he warns against assuming that experts inherently understand what everyday people desire better than the people themselves. Altman sees technology as part of the broader trajectory of human progress, where systems ought to achieve an accurate, high-fidelity understanding of what affected individuals actually want, rather than having external arbiters dictate the answers. Over time, an intelligent system could also serve an educational role, helping individuals deliberate and converge on truth as societal perspectives naturally evolve.
- Altman emphasizes the core principle: “But as a principle, the idea that the system represents the will of the people who are going to use it and be impacted by it, I think that's quite important.” - On the patronizing tendencies in governance discussions, Altman observes: “But there always is this tinge of like we know better what other people want than they do.” - Rather than elite prescription, Altman advocates for fidelity to genuine human desires: “Like, one thing that I hope for with this is that as has been this sort of part of the story of human progress, that we can get closer to an accurate fidelity of what the people who are going to be impacted by the technology actually want, not other people saying, well, they don't really know what they want.”
---
[01:01:45](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=3705s) Scaling LLMs vs. The Next Breakthrough to Reach AGI
```mermaid flowchart TD accTitle: Path Beyond Large Language Models to Superintelligence accDescr: Diagram illustrating current LLM hill-climbing versus the new breakthrough required to discover novel physics and achieve AGI.
subgraph CurrentPath ["Current Paradigm: Large Language Models"] A["Pre-training on Human Text"] --> B["Behavior Cloning"] B --> C["Climbing the Current Hill (Scale & Optimization)"] end
subgraph NeededBreakthrough ["The Next Quest: True AGI / Superintelligence"] D["New Algorithmic Breakthrough"] C -.->|"Insufficient for novel discovery"| D D --> E["Independent Reasoning & Scientific Inquiry"] E --> F["Discovery of Novel Physics"] end ```
For the final question of the evening, a student in the chamber asks Altman about the fundamental trajectory of artificial general intelligence. Noting that while large language models like ChatGPT have represented the biggest leap forward over the past decade, they still exhibit obvious shortcomings that demonstrate they are not yet AGI. The student asks whether the path to general intelligence is simply a matter of continuing to "min-max" existing large language models, or if the field still requires a completely undiscovered conceptual breakthrough.
Altman answers unequivocally that scaling the current paradigm will not be enough on its own: “I think we need another breakthrough.” While he affirms that OpenAI can and should continue pushing language models—likening their ongoing efforts to climbing a hill whose peak remains far off—he expresses deep skepticism that simply training models on existing human-generated text will ever produce genuine superintelligence. Returning to his earlier touchstone about scientific advancement, Altman points out that an intelligence incapable of generating novel physics cannot truly be deemed a superintelligence. Because training on human data fundamentally teaches models to clone human behavior and text, discovering new fundamental truths about the universe will demand an additional paradigm beyond pure language modeling—framing this search as OpenAI’s next great quest.
- In evaluating whether language models alone can reach AGI, Altman states: “Within reason, I mean, if you push that super, super far, maybe all this other stuff emerges, but within reason, I don't think that will do something that… I view as critical to an AGI.” - Outlining the benchmark for superintelligence, Altman stresses: “I don't think it's a super intelligence. And training on the data of what teaching it to clone the behavior of humans and human text I don't think that's going to get there.” - The event concludes with the chamber thanking Altman for spending his evening in Cambridge and officially accepting the 2023 Professor Stephen Hawking Fellowship on behalf of the OpenAI team.
## Notable quotes > [55:05](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=3305s) Safety in theory is easy. The hard thing is to make a system that is really useful to hundreds of millions of people and is safe. — Sam Altman
> [40:00](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=2400s) I don't think AI makes us dumber. I think it's going to turn out to make us much, much smarter. — Sam Altman
> [24:02](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=1442s) I think we will at some point need a global regulatory body which we don't get often, but we had one for atomic energy. I think we'll decide we need something like that as we get closer to AGI. — Sam Altman
> [10:00](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=600s) The sort of current rhetoric where we just say it's all going to be horrible and there's nothing that can be done is not very productive and certainly not very scientific. — Sam Altman
> [37:28](https://www.youtube.com/watch?v=NjpNG0CJRMM&t=2248s) I'm a big believer that if you can get really good at two things with an important intersection, that's when you can massively contribute to the world. — Sam Altman
## Mentioned Sam Altman, Stephen Hawking, Lucy Hawking, Jane Hawking, Elon Musk, Ilya Sutskever, Aleksander Madry, Andrew Ng, Yann LeCun, Marcus Aurelius, OpenAI, Cambridge Union, University of Cambridge, Bletchley Park AI Safety Summit, International Atomic Energy Agency (IAEA), Centre for the Future of Intelligence (CFI), Anthropic, Meta, Google, The New York Times, The Guardian, Y Combinator, ChatGPT, DALL-E, GPT-1, GPT-2, GPT-3, GPT-4, GPT-5, GPT-6
## Video description Sam Altman accepts the 2023 Hawking Fellowship Award on behalf of the team behind OpenAI on Wednesday 1st November 2023 at 7:15pm in The Debating Chamber.
The Union and the Professor Hawking Fellowship Committee are delighted to announce that the team behind OpenAI, the creators of Chat-GPT and DALL-E, have been jointly awarded the 2023 Professor Hawking Fellowship.
The Fellowship Lecture was delivered by Mr Sam Altman, CEO and Co-Founder of OpenAI, and took place in the Union chamber on the evening of Wednesday 1st November 2023.
The Hawking family commented the following: "We are very excited that the Hawking Fellowship at the Cambridge Union has been awarded to OpenAI this year. Our father Stephen was deeply engaged with issues around AI and its potential impact on all aspects of society in the future.
Many of his early comments on the new technology were extremely prescient and have been often quoted over the past year. We believe he would have been strongly in support of this award which brings the founders of OpenAI to Cambridge to debate their work and share their insights with students."
............................................................................................................................
Awarding of the Fellowship: 00:00 Address by Sam Altman: 02:45 What differentiates OpenAI from other companies: 12:27 What differentiates Elon Musk's approach to yours: 16:17 Can you reflect on the progress towards AGI so far: 17:19 In 10 years time, what does success look like: 19:14 What is your advice to young people in the face of an AI revolution: 19:52 How do universities need to adapt: 21:45 What was happening at Bletchley (UK AI Summit): 24:02 What is the British Government getting right and wrong: 26:27 What qualities do you look for in founders of companies: 27:47 Should we keep open the University website to AI crawlers: 30:14 What steps are you taking to make sure AI remains accessible to people with disabilities: 31:28 What do we do to prevent a worst-case scenario: 32:50 How do you decide which values make AI ethical: 35:07 Where would you advise young people to go to make scientific discoveries: 37:28 How can you mitigate against losing critical thinking: 39:20 How can we ensure experts are distributed across development, alignment and policy: 42:36 Do you worry you've sacrificed democracy: 44:15 What is your take on open source models: 47:30 When can we expect OpenAI to launch hardware: 49:02 What is your definition of consciousness and how far are we from a conscious model: 50:44 What is your view of smaller models: 52:53 Do you think the safety of AI is compatible with more abundant energy: 53:47 What is your approach to responsible scaling in the coming months: 55:05 Interest in longevity science: 57:39 Do you want AI to be descriptive or prescriptive with respect to human rights: 59:08 Is there another breakthrough that's needed to reach AGI: 1:01:45 ............................................................................................................................
The Professor Stephen Hawking Fellowship was founded in 2017 by the Union in conjunction with the Hawking family to honour his legacy and celebrate his relationship with the students of Cambridge.
The fellowship is awarded to an individual or team who has made significant impact on science, technology and its communication, with an emphasis on impacts affecting young people.
The fellowship is presented at this event in our debating chamber during Michaelmas term, during which the fellow gives an address and to which we also welcome the scientific advisory panel and members of the Hawking family.
............................................................................................................................ Thumbnail Photographer: Nordin Catic ............................................................................................................................
SUBSCRIBE for more speakers: https://www.youtube.com/channel/UClOuePXlJnw8-br7SXiuHqg ............................................................................................................................
Connect with us on:
Facebook: https://www.facebook.com/TheCambridgeUnion
Instagram: https://www.instagram.com/cambridgeunion
Twitter: https://twitter.com/cambridgeunion
LinkedIn: https://www.linkedin.com/company/cambridge-union-society
[Source](https://www.youtube.com/watch?v=NjpNG0CJRMM)