reddit.com Interviews Peter Norvig

AI-assisted summary of reddit.com Interviews Peter Norvig — original source. Check the original for full context.

Source published: 2010-02-25

Key ideas

Machine learning with linear separators proved unexpectedly powerful by projecting complex datasets into higher-dimensional polynomial feature spaces, demonstrating that pushing simple representations can outperform overly complex models. (0:04)
Programming language choice matters far less at scale than developer ability and social communication overhead, explaining why Google relies on C++ and mainstream languages rather than Lisp. (3:01)
Google mitigates many-core hardware bottlenecks by treating entire data centers as the fundamental computational unit, using functional programming abstractions like MapReduce rather than manual thread-and-lock orchestration. (14:14)
Research and production pipelines are integrated at Google because massive data sizes force researchers to build prototypes against production datacenter APIs from day one, avoiding the need to rewrite code from scratch. (24:13)

Detailed notes

## Background Published by Reddit on YouTube on February 25, 2010, this video features Peter Norvig responding to the top-voted questions submitted by the site's community. According to the video description, Norvig was serving as Google's Director of Research (formerly Director of Search Quality) and co-author of the landmark textbook Artificial Intelligence: A Modern Approach.

## TL;DR In this wide-ranging Q&A, Peter Norvig reflects on breakthroughs and stalled frontiers in artificial intelligence, contrasting the unexpected power of linear separators against the persistent challenges of inductive logic programming. He details Google's engineering philosophy, explaining why organizational communication and existing toolchains favored C++ over Lisp, and how distributed data center architectures necessitate declarative functional abstractions like MapReduce. Norvig also defends software engineering's track record against civil engineering benchmarks, demystifies AGI through an analogy to space flight components, and outlines how Google embeds researchers directly alongside production engineers. He wraps up the session with a playful lightning round covering books and frisbee before questioning whether Reddit's voting system truly surfaces what users want.

## Atmosphere & style The interview has an unpretentious, contemplative, and intellectually generous feel, anchored by Norvig's calm, measured cadence and dry humor. Rather than indulging in corporate polish or lofty theoretical hype, he demystifies complex topics with everyday analogies—comparing computers thinking to submarines swimming, or NASA's modular thruster research to artificial general intelligence. The exchange flows naturally between rigorous computer science discourse and self-effacing admissions about problem-solving limits, making the viewer feel like they are sitting in on an informal, candid office-hours discussion with a master practitioner.

## Bulletin - [00:04](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=4s) Machine learning with linear separators proved unexpectedly powerful by projecting complex datasets into higher-dimensional polynomial feature spaces, demonstrating that pushing simple representations can outperform overly complex models. - [03:01](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=181s) Programming language choice matters far less at scale than developer ability and social communication overhead, explaining why Google relies on C++ and mainstream languages rather than Lisp. - [14:14](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=854s) Google mitigates many-core hardware bottlenecks by treating entire data centers as the fundamental computational unit, using functional programming abstractions like MapReduce rather than manual thread-and-lock orchestration. - [24:13](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=1453s) Research and production pipelines are integrated at Google because massive data sizes force researchers to build prototypes against production datacenter APIs from day one, avoiding the need to rewrite code from scratch.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=4s) Linear separators surpassed early expectations because mapping raw inputs into polynomial feature spaces enables simple linear models to isolate non-linear boundaries. - [01:41](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=101s) Inductive logic programming has stalled relative to statistical machine learning because learning arbitrary computer programs over expressive languages is far harder than estimating parameters for simple equations. - [03:01](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=181s) Lisp fell out of favor for large-scale enterprise development as modern languages reached feature parity and the engineering bottleneck shifted from rapid individual refactoring to coordinating cross-team communication among hundreds of developers. - [05:38](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=338s) Educational search requires an entirely different technical paradigm than transactional search engines, demanding continuous learning models that span semesters and tools to synthesize disparate materials into full courses. - [07:25](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=445s) The metaphysical debate between weak and strong AI is secondary to practical engineering; artificial general intelligence should be approached like space travel by mastering prerequisite modular components first. - [10:47](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=647s) Software engineering is fundamentally harder than civil engineering because bridge builders construct tailored solutions for single, pre-measured sites, while software engineers deploy single binaries to hundreds of millions of unpredictable, shifting runtime environments. - [14:14](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=854s) Managing parallel computation across massive data centers necessitates functional abstractions that express commutativity and associativity, rather than fragile thread locking. - [17:13](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=1033s) The benefits of ingesting additional training data eventually asymptote, signaling engineers to either identify orthogonal types of data or fundamentally rethink their algorithmic architecture. - [18:25](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=1105s) Google internal workplace satisfaction surveys revealed a paradox: an engineer's favorite perk is brilliant colleagues, yet their biggest daily frustration is being interrupted by colleagues asking questions. - [21:55](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=1315s) Adopting checklists in software engineering, analogous to medical protocols described in 'The Checklist Manifesto,' can dramatically reduce routine errors and improve test discipline. - [24:13](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=1453s) Google bridges the gap between research and production by having researchers write code directly against datacenter-scale APIs and embedding production engineers within research units to maintain live products like Google Translate and Speech Recognition. - [26:59](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=1619s) Norvig questions whether basic upvoting and downvoting mechanisms on community platforms like Reddit cast an appropriate net or require updated algorithmic rules to deliver what users truly want.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=4s) Machine Learning: The Surprising Power of Linear Separators

Peter Norvig opens by greeting the Reddit community and expressing his appreciation for the extensive list of submitted questions. The first question, asked by user Khun John, inquires which field of artificial intelligence has most surpassed his expectations and surprised him. Norvig points directly to machine learning via linear separators, noting that years prior, researchers dismissed linear separation as an overly simplistic technique, only for it to prove remarkably potent.

Gesturing toward a board on set that the Reddit crew had scribbled on, Norvig illustrates what linear separators actually do in practice. When classifying data plotted on an $x$-$y$ axis into positive and negative examples—represented visually as X's and O's—the most straightforward method is to draw a straight line separating the two classes, placing all X's on one side and all O's on the other. While that approach initially seems too primitive for complex datasets where points are intermingled, Norvig explains the breakthrough insight: while an elliptical boundary cannot be drawn as a straight line in raw $x$ and $y$ coordinates, mapping the data into polynomial feature spaces (such as $x^2$, $x$, and $y^2$) transforms the problem, allowing linear separators to isolate non-linear boundaries. Summarizing the broader takeaway from this development, Peter Norvig observes: “And so that was surprising to me and I think that the overall lesson is if you keep with a simple representation and push it hard you can go farther than you thought.”

[01:41](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=101s) Disappointments and Inductive Logic Programming

Turning to the opposing perspective, a user asks what Norvig considers an AI failure or an area that should have progressed much further than it has. While hesitant to label any single endeavor an outright failure, Norvig points to inductive logic programming (ILP) as the area that has stalled the most. He notes a clear bibliographical indicator from his own work: when comparing the second and third editions of his textbook *Artificial Intelligence: A Modern Approach* (co-authored with Stuart Russell), the chapter on inductive logic programming remained the least changed.

Norvig explains the fundamental contrast between standard statistical machine learning and inductive logic programming: in linear separation, a function is modeled simply as $y = mx + b$, requiring the algorithm merely to estimate the parameters $m$ and $b$. In inductive logic programming, however, $y$ is treated as an arbitrary function of $x$ that can be represented by any full computer program written in an expressive programming language. Formulating representations this broad has proven extremely difficult to automate. He jokes that this difficulty might be comforting to professional software engineers because it proves their work is hard for machines to learn automatically. Nonetheless, Norvig maintains that inductive logic programming remains an essential field that deserves renewed effort, emphasizing: “The simple representation seems to have done better but I think it's important field and I think we should be doing more with it it definitely has been some advances in recent years and I've been trying to follow that but that's something we should push harder.”

[03:01](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=181s) Why Google Doesn't Extensively Use Lisp

Users cerebrum and alexandros ask why Lisp—a language Norvig famously championed—is not used extensively at Google. Norvig frames the issue primarily around developer productivity, engineering culture, and corporate inertia. When Google was founded, the earliest core engineers were already deeply proficient in C++, were exceptionally productive in it, and built an engineering culture around that momentum.

To illustrate why language dogmatism can be misleading, Norvig cites the experience of Erann Gat, an early Googler who had previously been a dedicated Lisp programmer at NASA's Jet Propulsion Laboratory (JPL). Gat had consistently outperformed colleagues at JPL and credited his edge to Lisp; however, upon joining Google, Gat was stunned to discover C++ programmers who were significantly more productive than he was, eventually concluding that raw developer capability often matters far more than language choice. Norvig adds several reasons why the programming landscape has shifted: - **Feature parity**: Twenty years prior, mainstream programming languages were relatively impoverished compared to Lisp, but modern languages have largely absorbed its most compelling features. - **Team scale and social communication**: Lisp excels when an individual or small team conducts exploratory development or executes a weekend refactoring. Once an organization scales to hundreds or thousands of engineers, the bottleneck is no longer how quickly a programmer can alter code, but the social challenge of communicating cross-cutting changes across the organization. - **Ecosystem and libraries**: Lisp developed web protocol implementations and external libraries far more slowly than rival languages, naturally pulling developers toward richer ecosystems.

Reflecting on language decisions across large organizations, Peter Norvig argues: “I think we're getting to a point where language choice is less important 20 years ago I think it was more important because they were languages we're more were impoverished and list pet quite an advantage now the features are tending to even out and I guess the other aspect is I think Lisp is really optimized for optimized for a small group a single program or a small group of programmers doing exploratory work and that's great and if you want to refactor something in a weekend I'd rather be doing it in lisp than anything else but by the time you get up to hundreds of programmers the problem of doing a complete refactor ization is not so much that the language is stopping you from making the changes the problems are more social of how do you communicate with the other 100 or 200 or 2000 programmers and say I want to make this change does that change affect you and so again a kind of language choice becomes less important.”

[05:38](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=338s) Current Personal Projects and Daily Life at Google

Responding to a question from lizard regarding his active projects and a typical workday as Google's Director of Research, Norvig describes his personal technical focus: search and discovery systems for educational materials. Having recently wrapped up the latest edition of his textbook, he has become fascinated by how search technology can improve education across multiple domains, including university courses, K-12 schooling, and informal self-directed learning for makers and hobbyists.

Norvig emphasizes that educational search demands an entirely different paradigm from standard search engine mechanics. Instead of resolving isolated, episodic queries where a user seeks a fast, transactional answer, education requires models that support continuous learning spanning entire semesters, months, or years. His work explores finding disparate pedagogical assets across the web, building organizational tools so authors and educators can aggregate those resources into cohesive courses, and adapting search interactions to protracted educational journeys. Regarding his daily routine, Norvig explains that his time is divided between hands-on work on this educational project, attending reviews with research teams he oversees to connect complementary efforts, and clearing an endless stream of email.

```mermaid flowchart TD accTitle: Educational Search Framework accDescr: Pipeline showing how disparate educational materials are found, structured, and delivered over extended timeframes.

subgraph Discovery["1. Discovery & Content Ingestion"] A["Raw Web Educational Content\n(College, K-12, Hobbyist/Maker)"] B["Discovery & Search Algorithms"] A --> B end

subgraph Structuring["2. Course Composition Tools"] C["Curator / Teacher Tooling"] D["Integrated Course Modules"] B --> C C --> D end

subgraph Experience["3. Long-Term Engagement Model"] E["Semester-Long Learning Journeys\n(Extended Multi-Month Search)"] D --> E end ```

[07:25](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=445s) Strong AI, General Intelligence, and the Space Travel Analogy

In response to crypto_z's inquiry about whether Google is working on "strong AI," Norvig clarifies what the terminology historically meant versus how it is used today. Philosopher John Searle (whom Norvig encountered at UC Berkeley) originally framed the distinction: weak AI refers to machines that merely act or appear intelligent, conscious, or thinking, whereas strong AI asserts that the machine actually possesses genuine consciousness and understanding. Norvig dismisses that philosophical debate as irrelevant to practical engineering, quoting computer scientist Edsger Dijkstra's famous dictum that asking whether a computer can think is like asking whether a submarine can swim. Just as a naval engineer cares only that a submarine maneuvers through water, an AI practitioner cares about functionality rather than metaphysical validation.

Norvig notes that the community has since repurposed "strong AI" to denote Artificial General Intelligence (AGI)—the capacity to solve any general intellectual task at human level, rather than narrow tasks like playing chess or recognizing faces in photographs. While Norvig finds AGI research valuable, he argues the field still lacks the foundational understanding required to achieve it directly. To illustrate the state of the discipline, he draws an analogy to space exploration: asking whether an organization is building general AI is like asking whether NASA is working on interplanetary travel. While NASA has no immediate interplanetary expeditions launched, it actively builds the prerequisite modular components, such as ion thrusters and solar sails. Norvig states that AI is in the exact same phase: researchers must focus on mastering the underlying building blocks before anyone can meaningfully assemble them into a complete general intelligence.

[10:00](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=600s) Tackling Difficult Programming Problems

Addressing a question from p_estrada on how he approaches challenging software engineering problems, Norvig gives a candid, self-deprecating response: “How do you approach a difficult programming problem and the answer is I don't know yeah you just go ahead and do it and hope that you get the answer.”

He explains that problem solving relies on a flexible repertoire of heuristics rather than a rigid formula: - **Prior art**: Investigating whether a similar problem has already been solved by someone else or encountered in his own past work. - **Decomposition and simplification**: Isolating a smaller, simpler piece of the problem, solving it first, and using the resulting insights to tackle the larger challenge. - **Generalization**: Paradoxically attempting an even harder, generalized version of the problem to see if proving properties about the general case reveals an attack vector for the specific instance. - **Collaboration**: Brainstorming with peers to gather alternative perspectives and technical ideas.

[10:47](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=647s) Problem-Solving Heuristics and Why Software Engineering Is Harder Than Bridge Building

Peter Norvig wraps up his thoughts on problem-solving strategies when developers hit a wall. He notes that when you are stuck, there are several viable maneuvers: you can gather more data to simplify the problem or make it vanish entirely, finesse the situation by delegating, or build a social community to handle the portions you cannot accomplish alone. He reflects that many of these mindset-shifting tactics mirror George Pólya's classic book *How to Solve It*—which, although written about mathematics rather than programming, directly tackles what to do to jog your mind and explore an entirely new angle when stuck. He concedes that the industry still lacks a definitive answer for the single best way to achieve that breakthrough.

Turning to a question submitted by Reddit user "obsessed with ammas" asking why the industry remains so bad at software development, Norvig politely pushes back against the premise. In his view, software developers are actually quite skilled, but the underlying challenge is uniquely difficult. Evaluating success requires choosing a benchmark; people often draw comparisons to civil engineering and point out that bridges rarely collapse while software programs crash constantly. Norvig explains that bridge builders operate under far simpler customized constraints: they construct a single bridge in one physical location, measuring the exact water current, wind speeds, and subterranean soil dynamics beforehand. By contrast, a software engineer creates a single system and distributes it to a hundred million users across completely unseen, varying runtime environments.

In real-world deployment, software faces conflicting DLLs or silently breaks years down the road when an external networking standard evolves. Norvig maintains that no other discipline handles that scale of variability better because they tackle simpler, isolated domains. If software could match civil engineering reliability, it would need the proactive adaptability of an on-site engineer: the software would assess its deployment environment, identify shifting conditions or delicate local interactions, and autonomously reconfigure itself to eliminate the friction without demanding individualized human tailoring.

As Norvig sums up the comparison: > “I think we're actually pretty good at software development and I think it's just really hard so to know if we're good or bad you have to compare it to something and you know one thing we commonly gets compared to civil engineering and bridges don't fall down that often but they sometimes do programs seem to crash a lot more than bridges do but but if you look at the relative efforts and the customization then the bridge designers have it a lot easier because they're building one bridge at a time in a specific place and they know exactly how much current is going by and how much wind and what the soil is like underneath and so on and they build that one solution whereas software engineers we build a solution and then we ship it to a hundred million people and they're all in slightly different environments that we don't get to look at ahead of time…”

```mermaid flowchart TD accTitle: Comparison of Engineering Disciplines accDescr: Civil engineering solves a bespoke problem for a single known site, whereas software engineering distributes a single binary to 100 million unknown environments.

subgraph CivilEng["Civil Engineering Model"] CE_Site["Specific Site Assessment<br/>(Wind, Current, Soil)"] --> CE_Build["Bespoke Single Solution"] CE_Build --> CE_Result["High Reliability at Single Site"] end

subgraph SoftwareEng["Software Engineering Model"] SE_Build["Single Static Solution"] --> SE_Deploy["Deploy to 100M+ Users"] SE_Deploy --> SE_Issues["Diverse Runtime Contexts<br/>(DLL Conflicts, Network Shifts)"] SE_Issues --> SE_Goal["Ideal Future State:<br/>Proactive Self-Reconfiguration"] end ```

[14:14](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=854s) Scaling Beyond Many-Core: From Data Centers to Functional Abstractions

User "run time" asks how programming languages will adapt to many-core processors and whether existing paradigms—such as manual thread locking, Software Transactional Memory (STM), pure functional programming, the actor model, or SIMD GPU parallelism—will scale to meet the challenge. Norvig responds by shifting the framing entirely. For engineers working at Google's scale, multi-core chips do not represent the primary architectural crisis because the foundational computational unit is already an entire data center housing thousands of machines. Whether an individual motherboard holds a single CPU core, 8 cores, or 128 cores does not fundamentally alter the complexity envelope; the system must already be distributed and parallel across a massive fleet.

While physical distance matters at every tier—a local cache miss, a jump to an adjacent core on the same die, crossing to another board on the rack, or traversing network switches to reach a different rack altogether—Norvig treats them as instances of the same overarching challenge governed by different latency constants. Because manual thread and lock orchestration collapses when coordinating computations across hundreds of thousands of parallel workers, the solution points back toward functional programming paradigms built around high-level abstractions.

Norvig points to Google's MapReduce framework as a prime example of this philosophy. When Google originally introduced MapReduce, the team viewed it as merely the first among perhaps a dozen functional primitives designed to split problems into distributed chunks and assemble the results back together. Although MapReduce proved surprisingly versatile and succeeded far beyond initial internal expectations, Norvig argues it is time for the field to develop the remaining abstractions. Developers need declarative constructs that capture the commutativity and associativity of problems, giving the execution runtime the semantic clarity needed to partition and recombine workloads efficiently across massive clusters.

Addressing the scaling limit of traditional multithreading, Norvig remarks: > “And certainly if you're thinking at the level of the thousands or hundreds of thousands of parallel computations then probably this manual keeping track of threads with locks is not going to scale very well so what we really need is closer to the functional type approach something where we have the right abstractions to make this all work…”

- **Google MapReduce expectation vs. reality:** The engineering team initially assumed MapReduce would be one of roughly a dozen parallel patterns; it ended up carrying far more computational weight across Google than originally anticipated. - **Latency hierarchy:** System design must factor in cache misses, intra-board core hops, inter-board rack hops, and multi-switch inter-rack hops as varying distance constants within the same parallel model. - **Declarative algebra:** The next generation of distributed primitives must express commutativity and associativity so runtimes can automate splitting and recombining data pipelines. - **Domain limitation:** The MapReduce approach excels for web-scale workloads running identical transformations across vast data sets before summarizing; other computation domains will necessitate alternative core abstractions.

[17:13](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=1033s) Diminishing Returns on Big Data vs. Smarter Algorithms

Reddit user "Khafre" asks about Google's famous philosophy prioritizing vast data over algorithmic complexity: is there an observable point of diminishing returns where accumulating additional data stops helping, making it worthwhile to focus once again on engineering clever algorithms, and how does one detect that tipping point?

Norvig notes that diagnosing this limit is straightforward in practice: you simply plot system performance against the volume of training data collected. Initially, ingesting more data drives steep performance gains. Eventually, the learning curve asymptotes into a flat plateau. Once performance flattens out, continuing to feed the exact same kind of data yields negligible benefits while adding unnecessary processing overhead that slows the pipeline down.

When that asymptote arrives, an engineer faces two distinct choices: 1. Search for new, orthogonal types of data that the system currently lacks—which Norvig notes is functionally equivalent to inventing a new algorithmic representation. 2. Step back and question the fundamental premises of the model to determine whether a radically different algorithmic architecture is required to break through the ceiling.

Regarding the moment data stops driving progress, Norvig observes: > “And at some point it begins to asymptote out and then you say well probably I don't want any more of the same kind of data it's just starting to slow me down and not gaining me much…”

[18:25](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=1105s) Engineering Environments: Hardware, Cubicles, and Team Dynamics

User "gesharm" asks what Google’s internal research reveals about creating an ideal programming work environment: whether open cubicles match closed private offices, whether an ultraportable 10-inch netbook can rival a high-end multi-core Mac Pro driving triple 30-inch displays in the hands of an experienced developer, and whether tech firms should invest more heavily in elite physical working conditions and top-tier equipment.

Norvig finds the topic critical and expresses personal enthusiasm for running empirical, in-house studies on developer environments, noting that tech companies now employ enough engineers to gather robust data. He laments that computer science has historically conducted far too few scientific studies evaluating workplace productivity, leaving companies to rely on subjective employee preferences instead of rigorous evidence. While external research does suggest that expansive screen real estate boosts developer productivity, Norvig jokingly questions whether those findings might have been sponsored by monitor manufacturers, adding that he prefers seeing independent replications before treating them as definitive truth.

On office layouts, Norvig observes that traditional workplaces usually end up with a mix of cubicles and offices, but Google framed the problem through the lens of team sizing. While companies like Microsoft historically gave every engineer a private closed office prioritizing deep, solitary concentration, Google deliberately prioritized communication by grouping whole engineering pods of two, three, or four people into shared rooms. The goal is that ambient overhearing keeps every developer plugged into relevant technical discussions without feeling like a jarring interruption.

Norvig recounts an internal annual Google satisfaction survey that polled employees on everything from cafeteria food quality and ergonomic chairs to compiler build speeds. The resulting data revealed a humorous human paradox: the single highest-rated perk across the entire company was the brilliant quality of one's colleagues, while the single most aggravating daily annoyance was being interrupted by a colleague asking a question.

Reflecting on the contradictory desires of software engineers, Norvig quips: > “So we've determined that the perfect place is one where I can ask a question of anybody else but nobody can ask a question of me but we haven't quite figured out how to implement that yet…”

In lieu of an impossible physical layout that satisfies that contradiction, Norvig notes that teams rely on pragmatic social protocols—such as putting on headphones—to clearly signal to nearby teammates when someone needs unbroken focus and quiet.

[21:27](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=1287s) Office Privacy, Headphones, and Workspace Tradeoffs

Discussing how engineers balance collaboration and concentration in open office setups, Peter Norvig observes that workers have adapted practical coping mechanisms. People put on headphones when they want to signal that they should not be interrupted, or they pick up their laptops and walk off to a private area when they need secluded focus. Similarly, when two colleagues want an in-depth one-on-one conversation, they simply grab each other and go sit somewhere else. Despite these informal adaptations, Norvig highlights that workplace dynamics remain unresolved: “I think we need a lot more research to know how you trade these things off.”

---

[21:55](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=1315s) The "Human Check" Lightning Round: Reading, Frisbee, and Google Wave

The interviewer shifts to a rapid-fire series of Reddit questions presented playfully as a "human check" to confirm Norvig is an actual person and not an artificial intelligence bot. The first question asks whether he still plays frisbee golf and what his typical score is. Norvig admits that while he used to compete seriously, he sadly no longer plays competitively, though he still enjoys going out into the backyard to throw discs around.

```mermaid flowchart TD accTitle: Peter Norvig Reading and Software Development Parallels accDescr: Flowchart illustrating how Atul Gawande's surgical checklist concepts inform software testing and engineering practices. subgraph Medicine["Medical Example (The Checklist Manifesto)"] M1["Doctors forget basic hand-washing"] --> M2["Introduce simple surgical checklists"] M2 --> M3["Infection rates drop dramatically"] end subgraph Software["Application to Software Engineering"] S1["Complex development environments"] --> S2["Implement structured checklists & tests"] S2 --> S3["Reduce bugs and improve release reliability"] end Medicine -.->|"Informs"| Software ```

Asked what magazines and books he reads, Norvig jokes that the prescribed answer is usually "all of them," before explaining that he primarily gravitates toward nonfiction and science periodicals like *Science*, *Science News*, *Discover*, and *New Scientist*. His most recent book at the time was *The Checklist Manifesto* by Atul Gawande, which investigates how implementing straightforward surgical checklists prevents post-operative infections simply by reminding doctors to wash their hands. Norvig reflects on how that idea applies to writing code: “And so it's interesting to sort of apply that are there simple things we can do in software development where we can learn from that and for the most part I think we do a good job we're probably better at doing tests than most people but that's something to look at.”

Turning to current technology, Norvig is asked if he has discovered a good use for Google Wave. He notes that while he has experimented with it, its biggest current friction is that not enough people are on it yet to make widespread communication seamless. However, he sees a major silver lining in adopting new platforms: “I think one of the killer apps for Google Wave is to discard all the legacy mailing lists that you're on that are no good anymore so anytime you move to a new platform you could just throw away all the old stuff and start over again so I think that's great and it's exciting to move to something new.”

To round out the rapid verification questions, Norvig confirms that ultimate frisbee remains his all-time favorite game, even if he cannot play as frequently as in the past, and answers the final mood check with a cheerful, “I feel good.”

- **Recent reading:** Atul Gawande's *The Checklist Manifesto*; regular reading includes *Science*, *Science News*, *Discover*, and *New Scientist*. - **Google Wave utility:** Acts as a fresh slate to prune outdated, cluttered legacy email threads and distribution lists. - **Favorite sport/game:** Ultimate Frisbee.

---

[24:13](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=1453s) Bridging the Gap: Research and Production Code at Google

Addressing a question from user "no_theory" about how research projects transition into live production code, Norvig explains that Google approaches research very differently from traditional corporations that enforce strict walls between researchers and software engineers. Despite having grown massively, Google retains the operational culture of a startup. Rather than segregating PhDs into an isolated corporate laboratory building, researchers and engineers work side by side, pitching in on whatever tasks need to be completed without jurisdictional friction.

A critical factor that prevents research from languishing on a shelf is the scale of Google's data infrastructure. In academia or conventional research labs, an investigator typically hacks together a prototype on a personal laptop, resulting in code that must be completely discarded and rewritten from scratch before it can handle production loads. At Google, researchers cannot use that workflow because the datasets simply do not fit on a single machine:

> “At Google part of the point of being here is that there is all this data available and in order to take advantage of it you can't load it onto your laptop it wouldn't fit so right from the start the researchers are writing code that use our main APIs that are using the data that everybody else uses if you want some web pages you use the full copy of the web if you want some link data you use the full link graph and so on and you use it in the data centers and so that makes it a lot easier to go from there to launch something.”

Even though a research prototype might initially run ten times too slowly to be deployed and demands significant re-engineering to reach production speed, building against the real APIs and datacenter backends ensures it is already substantially closer to launch.

```mermaid flowchart LR accTitle: Research to Production Pipeline at Google accDescr: Pipeline showing how Google researchers build on live cluster infrastructure and retain production ownership within the research team. subgraph Traditional["Traditional Lab Model"] T1["Prototype on Laptop"] --> T2["Hand off to Eng"] T2 --> T3["Complete Rewrite from Scratch"] end subgraph Google["Google Startup Model"] G1["Code against Main APIs & Full Web Graph"] --> G2["Re-engineer Prototype for Speed in Data Centers"] G2 --> G3["Integrate Production Engineers directly into Research"] G3 --> G4["Maintain Live System while Building Next-Gen"] end ```

Furthermore, Norvig highlights that Google often retains whole production systems within the research organization itself rather than handing them off to an external engineering team. Both Speech Recognition and Google Translate originated as research initiatives and remained inside the research division after launch. To support this model, Google embeds production engineers directly within the research teams. This prevents splitting the team into two disconnected factions—one maintaining the current live system while the other designs the next generation—allowing them to simultaneously keep the production service operational and build future capabilities together.

---

[26:59](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=1619s) Turning the Question Back on Reddit's Community

As the interview concludes, Norvig takes the opportunity to flip the dynamic and pose a question back to the Reddit community itself. He asks whether the site's fundamental upvoting and downvoting mechanism is truly delivering what the community hopes to achieve:

> “As a community do you feel that Reddit is getting what you want so is this experiment of saying we're just going to vote things up is this finding what you want is it getting you the things that are appropriate for you is it casting a net that's wide enough or too wide and if not is there a way to change the rules to make it better?”

## Notable quotes > [00:04](https://www.youtube.com/watch?v=hE7k0_9k0VA&t=4s) I think that the overall lesson is if you keep with a simple representation and push it hard you can go farther than you thought. — Peter Norvig

## Mentioned Peter Norvig, Stuart Russell, Artificial Intelligence: A Modern Approach, Erann Gat, John Searle, Edsger Dijkstra, George Pólya, Atul Gawande, The Checklist Manifesto, Google, Reddit, NASA, Jet Propulsion Laboratory (JPL), Microsoft, MapReduce, Google Wave, Google Translate, Speech Recognition, Lisp, C++

## Video description Peter Norvig answers the top questions from reddit.com.

See questions: http://bit.ly/9Hb3A5

Peter Norvig is currently the Director of Research (formerly Director of Search Quality) at Google. He is also the author with Stuart Russell of Artificial Intelligence: A Modern Approach - 3rd Edition.

Opening music taken from "Plume" by Silence http://www.jamendo.com/en/artist/silence

Creative Commons License Attribution-Noncommercial 3.0 United States

[Source](https://www.youtube.com/watch?v=hE7k0_9k0VA)

中文简报

Reddit 专访彼得·诺维格在这场内容广泛的问答中，彼得·诺维格（Peter Norvig）反思了人工智能领域的突破与停滞前沿，将线性分类器的出人意料的威力与归纳逻辑编程持续面临的挑战进行了对比。他详细阐述了 Google 的工程哲学，解释了为何组织沟通和现有工具链让 C++ 比 Lisp 更受青睐，以及分布式数据中心架构如何促使人们采用 MapReduce 等声明式函数式抽象。诺维格还参照土木工程的基准为软件工程的历史表现进行了辩护，通过航天组件的比喻为通用人工智能（AGI）去神秘化，并概述了 Google 如何将研究人员直接安排在生产工程师身边。在随后的轻松快问快答环节中，他聊到了书籍和飞盘，最后对 Reddit 的投票系统是否真能呈现用户所需提出了疑问。通过将复杂数据集映射到更高维度的多项式特征空间，利用线性分类器的机器学习展现出了出人意料的强大能力，这证明了挖掘简单表征的潜力能够超越过度复杂的模型。 (0:04) 在规模化运作下，编程语言的选择远没有开发人员的能力和社交沟通成本重要，这也解释了为何 Google 依赖 C++ 和主流语言而非 Lisp。 (3:01) Google 将整个数据中心视为基本计算单元，采用 MapReduce 等函数式编程抽象，而非手动编排线程和锁，以此缓解多核硬件瓶颈。 (14:14) Google 的研究与生产流程是紧密结合的，因为庞大的数据规模迫使研究人员从第一天起就基于生产数据中心的 API 构建原型，从而避免了从头重写代码的需要。 (24:13)

Reading summary

In an interview with Reddit, Google's Director of Research Peter Norvig argues that practical AI and software engineering advance by maximizing simple representations and high-level abstractions rather than chasing premature complexity or dogmatic tools. Drawing on industry experience, Norvig evaluates machine learning paradigms, software scalability, organizational design, and the discipline's distinctive operational hurdles. Power of Simple Representations Versus Expressive Automation (0:04) Norvig highlights machine learning with linear separators as the AI development that most exceeded his expectations. Once dismissed as overly primitive, linear separators proved highly effective when data is mapped into polynomial feature spaces, demonstrating that pushing a simple representation hard can solve complex, non-linear classification tasks. Conversely, Norvig points to inductive logic programming (ILP) as the field that has lagged furthest behind expectations. Whereas linear models estimate straightforward parameters like slope and intercept, ILP attempts to infer full, arbitrary computer programs from data. Because automating such broad expressiveness remains exceptionally difficult, the ILP chapter in Norvig and Stuart Russell's textbook saw the least revision across editions, though Norvig maintains the area still warrants continued research. Ecosystem and Team Scale Over Language Dogmatism (3:01) Addressing why Google does not rely heavily on Lisp, Norvig attributes the decision to developer productivity, team scaling, and language convergence rather than inherent language deficiencies. While early Lisp offered major advantages over its peers, modern languages have largely absorbed those capabilities. Norvig emphasizes that Lisp excels for solo developers or small groups executing rapid refactoring, but large enterprises with hundreds or thousands of engineers face organizational and social coordination bottlenecks where language syntax matters less. Furthermore, Google's foundational engineering culture was established by highly productive C++ engineers, and Lisp was slow to build out the libraries and networking protocols essential for large-scale web services. Distributed Functional Primitives Outweigh Multicore Locking (14:14) Regarding hardware scaling and many-core processors, Norvig argues that fine-grained multithreading and manual lock management fail when coordinating thousands of parallel workers. For large-scale web companies, multi-core chips do not redefine the engineering challenge because the baseline computational unit is already the distributed data center. System design must treat cache misses, intra-die core hops, and inter-rack network hops as points along a unified latency spectrum. To manage this scale, systems require declarative, functional abstractions that capture mathematical properties like commutativity and associativity. Norvig notes that while MapReduce proved far more durable and versatile than the single-pattern tool Google initially envisioned, the broader discipline still needs to define the remaining functional primitives to partition and recombine diverse data workflows. Deployment Variability and Empirical Engineering Environments (10:47) Norvig challenges the notion that software engineering is inherently inferior to civil engineering. Bridge designers build single bespoke solutions for known physical locations with measurable environmental forces, whereas software developers deploy single systems across a hundred million unseen, disparate runtime environments subject to evolving networking standards and conflicting libraries. Bridging that reliability gap would require software capable of autonomous local diagnosis and reconfiguration. On physical office setups, Norvig notes Google grouped developers into shared pods of two to four to preserve ambient communication, though internal surveys revealed the paradox that engineers value brilliant colleagues above all else while identifying colleague interruptions as their greatest frustration. Norvig calls for more rigorous, empirical studies on developer environments rather than relying on informal coping mechanisms like noise-canceling headphones. Integrating Research and Production via Live Datacenter Infrastructure (24:13) Google bridges the gap between basic research and production deployment by requiring researchers to work on live datacenter infrastructure from day one. Unlike traditional academic or corporate laboratories where prototypes are constructed on standalone laptops and later rewritten, Google researchers build directly against production APIs, the live link graph, and full web copies. This shared environment keeps experimental code much closer to launch readiness, even when prototypes initially require speed optimization. Additionally, research teams retain operational ownership of major live services—such as Speech Recognition and Google Translate—by embedding dedicated production engineers alongside research scientists rather than handing off maintenance to separate divisions. Norvig contextualizes current AI efforts by noting that Artificial General Intelligence remains in a foundational component-building phase analogous to space research developing modular thrusters rather than mounting direct interplanetary missions. At the conclusion of the interview, he poses an unresolved question back to the Reddit community: whether the platform's core upvoting and downvoting mechanism genuinely surfaces what users need, or whether alternative rules are required to curate relevant information.

精炼阅读摘要

在 Reddit 的专访中，时任 Google 研究总监的 Peter Norvig 探讨了人工智能演进、工程工具选择与系统规模化扩展的核心逻辑，指出推动简单表示法极限与构建高阶抽象往往优于盲目追求复杂机制。机器学习的简单表征与归纳逻辑编程的停滞 (0:04) Norvig 指出，线性分类器展现出的巨大威力远超早期研究界的预期。虽然直接在原始坐标下无法线性划分混合样本，但若将数据投射至多项式特征空间（如 x²、x、y²），线性工具便能巧妙分离复杂的非线性边界；这表明坚持简单表征并将其推向极限，能取得意想不到的突破。相比之下，归纳逻辑编程（ILP）在近年的进展最为缓慢。Norvig 提到，在他与 Stuart Russell 合著的《人工智能：一种现代方法》第二版与第三版中，该领域章节内容变动最小。统计机器学习只需拟合类似 y = mx + b 中的参数，而归纳逻辑编程则试图将函数表示为具备完整图灵完备语言的任意计算机程序，其自动化表征极其艰难，但 Norvig 强调该领域依然重要且值得投入更多探索。通用智能探索与编程语言生态的变迁 (3:01) 针对通用人工智能（AGI）的讨论，Norvig 借用 Edsger Dijkstra 的名言反驳哲学层面的意识争论——询问计算机能否思考就像询问潜艇能否游泳，工程实践的核心在于功能实现。他将当前的 AGI 研发比作美国宇航局（NASA）的星际旅行探索：正如 NASA 尚未启动星际飞船却在研发离子推力器和太阳帆等底层模块，AI 领域仍缺乏实现 AGI 的基础原理，当下重心应是攻破各类模块化构建块。在工具选择上，针对 Google 为何未大规模采用 Lisp，Norvig 解释道，现代主流语言已吸收了 Lisp 昔日的优势特性，且工程师的个体能力远比编程语言本身重要。当团队规模扩展至数百或数千名工程师时，系统的瓶颈已从代码重构速度演变为团队间的沟通协作与依赖管理，加之 Lisp 在外部协议与库生态建设上的滞后，大型组织自然会流向更广泛的生态体系。软件工程环境复杂性与多核分布式抽象 (10:47) Norvig 反驳了“软件工程行业水平低下”的论调。人们常以极少坍塌的桥梁工程对比容易崩溃的软件，但土木工程师是在明确已知的单一物理地点勘测水流、风速与地质后进行定制化施工；而软件工程师则是构建一套方案后将其交付给上亿用户，运行在完全未知且相互冲突的动态系统环境中。在硬件架构演进方面，面对多核处理器的普及，Norvig 认为在 Google 规模下，计算的基础单元是容纳数万台服务器的数据中心，单主板的核心数量并未改变系统底层的分布式本质。随着节点规模增长，基于锁的手工线程同步必然崩溃，行业必须转向函数式高阶抽象。Google 最初构想 MapReduce 时仅将其视作十余种分布式并行原语之一，未来的分布式模式需引入具备交换律与结合律的声明式代数结构，使底层运行时能够自动切分与聚合流水线任务。数据收益递减拐点与研发运维一体化体系 (17:13) 针对 Google 崇尚海量数据胜过复杂算法的理念，Norvig 强调数据带来的收益存在明确的渐近线拐点。当在图表上观测到系统表现曲线进入平缓期时，持续输入同质数据不仅收益微弱，还会增加处理开销；此时团队必须横向搜寻全新的正交数据，或从根本上质疑并重构算法架构。在工程组织模式上，Google 打破了传统企业将研发人员与生产工程师隔离的实验室壁垒。由于训练数据规模极其庞大、个人电脑无法容纳，研究人员从第一天起就必须基于真实 API 和集群完整数据（如全网网页与链接图）编写代码。此外，语音识别与 Google 翻译等系统孵化上线后均保留在研究部门内部，由嵌入其中的生产工程师共同运维，使前沿模型探索与线上服务维护形成紧密协同。在分析工程生产力时，Norvig 指出团队面临“既希望向任何人提问又不希望被任何人打扰”的人性矛盾，当前只能依赖佩戴耳机等社交礼仪作为缓冲。访谈最后，Norvig 将问题抛回给 Reddit 社区，公开求证基于简单赞踩投票的机制是否真的有效筛选出了社区用户所需的内容，抑或需要从底层重塑规则。

Source quotes

[0:04] Peter Norvig: I think that the overall lesson is if you keep with a simple representation and push it hard you can go farther than you thought.

Original transcript

[0:04] okay hello we're in it it's great to be [0:08] here thanks for the long list of [0:10] questions and we can't make it through [0:12] all of them but we'll see what we can do [0:13] our first question is from Khun John who [0:17] asked which AI field has surpassed your [0:20] expectations and surprised you the most [0:22] and I guess I got to say machine [0:25] learning by linear separators has gone [0:28] much farther than we ever thought it [0:30] could go years ago people were booing it [0:34] as a simple method but it's turned out [0:36] to be very powerful and so so what does [0:38] that mean for people who don't know well [0:40] we got this flight board here and the [0:42] Reta guys put a bunch of junk on it [0:43] that's just for show but what linear [0:49] separators means is you get some data [0:51] and you know you got some positive [0:52] examples on an XY plot and some negative [0:56] examples and you say let's separate them [0:58] out by just drawing a line and then [1:00] everything above the line is X's and [1:01] everything on the load lines of those [1:02] and that seems pretty simple because you [1:05] know what if the data was like this and [1:08] there were exes all over the place then [1:10] you can't draw a line separate out the [1:12] oils from the exes but you could draw an [1:15] ellipse and the trick was to see that if [1:18] you're just dealing in the x and y [1:20] coordinates there is no linear line that [1:23] looks like an ellipse but if you start [1:24] dealing with a x squared and the x and [1:26] the y squared coordinates then you can [1:28] do that and you can get a lot farther [1:30] than people ever thought you could [1:31] and so that was surprising to me and I [1:34] think that the overall lesson is if you [1:36] keep with a simple representation and [1:37] push it hard [1:38] you can go farther than you thought now [1:41] the next question D or something asks [1:43] I'd like to know the flip side of that [1:45] what do you consider a failure and [1:47] should we move further along than it is [1:50] and I guess you know I hate to call [1:53] anyone any one thing of failure but [1:55] related to this I can say that one thing [1:58] that hasn't moved this far and it's also [2:02] the thing that when I look at the the [2:04] difference between the second edition of [2:05] my textbook and the third edition it's [2:07] the chapter that has changed the least [2:09] is the chapter on inductive logic [2:12] programming and that's the idea [2:14] your representation for a function is [2:17] not a simple line you know the lines [2:19] represented y equals MX plus B and to [2:23] figure out which line you just figure [2:24] out what value of M what value of B with [2:28] inductive logic programming Y is a [2:31] function of X and can be any program [2:32] written in whatever programming language [2:34] you choose and that's proven hard to do [2:36] and you know maybe that's reassuring to [2:38] us programmers that we're doing [2:40] something that's difficult for computers [2:43] to learn automatically the simple [2:46] representation seems to have done better [2:47] but I think it's important field and I [2:49] think we should be doing more with it it [2:50] definitely has been some advances in [2:53] recent years and I've been trying to [2:55] follow that but that's something we [2:56] should push harder okay next question [3:01] cerebrum and also alexandra's were [3:04] interested in saying you were a big [3:06] advocate of Lisp why isn't it used [3:08] extensively at Google I guess the way I [3:11] look at it is that the best language to [3:14] use is the one that you and your friends [3:16] are most productive in and it just [3:19] turned out that when Google was started [3:21] the sort of core programmers that that [3:24] were there first were C++ programmers [3:27] and they were very effective with that [3:29] and they kept going with it and so part [3:31] of it is just I guess a little bit of [3:33] culture do you have the right culture [3:34] now given that of course you can make [3:37] mistakes I think thought I was very [3:39] telling to read what Iran got had to say [3:43] and so he was a an early Googler who had [3:47] been a list programmer at JPL he came [3:50] over to Google before I did and he said [3:53] that he was surprised that in his past [3:55] experience he had been a list programmer [3:57] and he has always been more productive [3:59] than anybody else that he was working [4:01] with and he attributed that largely to [4:03] his language choice and then he got to [4:05] Google and he saw C++ programmers that [4:08] were more productive to him and he was [4:11] surprised and eventually decided well [4:14] maybe it has more to do with programmers [4:16] themselves than with the language choice [4:18] I think we're getting to a point where [4:20] language choice is less important 20 [4:23] years ago I think it was more important [4:25] because they were languages we're more [4:27] were impoverished and list pet quite an [4:29] advantage now the features are tending [4:32] to even out and I guess the other aspect [4:34] is I think Lisp is really optimized for [4:38] optimized for a small group a single [4:42] program or a small group of programmers [4:43] doing exploratory work and that's great [4:48] and if you want to refactor something in [4:49] a weekend I'd rather be doing it in lisp [4:52] than anything else but by the time you [4:54] get up to hundreds of programmers the [4:56] problem of doing a complete refactor [4:59] ization is not so much that the language [5:01] is stopping you from making the changes [5:03] the problems are more social of how do [5:06] you communicate with the other 100 or [5:08] 200 or 2000 programmers and say I want [5:11] to make this change does that change [5:13] affect you and so again a kind of [5:15] language choice becomes less important [5:18] okay oh and I guess one other thing is [5:21] that for whatever reason the various [5:24] sets of libraries for web protocols and [5:27] other things have been slower to develop [5:29] and Lisp than they have in other [5:31] languages and so people go where the [5:33] rich libraries that's our next question [5:38] from lizard in which projects are you [5:42] personally strongly involved right now [5:43] and tying in to this can you describe [5:46] your individual typical day at google [5:48] for us with an emphasis and what kinds [5:51] of tasks you're mainly handling so my [5:53] main personal project right now is [5:55] around search for educational materials [5:58] and I just did a textbook and so I'm [6:02] interested in college-level courses how [6:05] do you find material related there's a [6:06] lot of great material out there we're [6:08] also interested in k-12 education and in [6:12] kind of personal ad hoc learning for [6:16] hobbyists or makers or whatever to learn [6:19] the specific tasks and we're interested [6:23] in this at the level of finding [6:25] materials that's already out there [6:27] organizing material to say I want to put [6:29] together a specific course can I find [6:31] the pieces and paste them together into [6:33] a course providing tools for authors [6:37] teachers to do that and then also with [6:41] this question of what is it like when [6:44] your search experience and your learning [6:46] experience lasts over a semester or [6:49] months or years rather than just a very [6:53] quick interaction that we're used to [6:54] where you do one search at a time and [6:56] get the answers so that's why I'm [6:58] personally interested in what do I do in [7:00] my typical day well part of it is is [7:02] bent on that project part of it is spent [7:06] in meetings with various teams that I [7:09] oversee trying to get up to date and [7:11] talk about what they're doing and maybe [7:13] make some connections for them with [7:15] other projects they might be working [7:17] with and of course like everyone else a [7:20] lot of my day is answering email next [7:25] question from crypto Z is Google working [7:28] on strong AI well so the first part is [7:31] to define what the question means and [7:33] john searle who's a philosopher he was [7:36] there at Berkeley when I was there [7:38] defined the term strong AI versus weak [7:41] AI and he said that we kiii is when [7:45] something looks like it's intelligent or [7:48] thinking or conscious and strong AI is [7:52] when it actually is and he wanted to [7:55] make that distinction and he says well [7:57] people actually are conscious or [7:59] intelligent but he thought that a [8:01] computer even if it looked like it did [8:03] probably wouldn't be really I have that [8:06] and so wouldn't be strong and my [8:08] reaction to that is who cares you know I [8:10] guess philosophers you got to have [8:11] something to do but I'm not going to [8:14] base my decision on whether they think [8:16] it's real or not [8:17] and Edgar Dykstra I had this line of [8:21] saying the questions of whether a [8:24] computer can think is equivalent to the [8:26] question of whether a submarine can swim [8:28] and if you're into a naval engineer you [8:32] don't really care what the philosophers [8:34] tell you that your submarines are [8:35] swimming or not you just want to make [8:37] them go so that definition of strong AI [8:41] I don't care about but now there's been [8:44] a new definition and people have taken [8:46] this perfectly good well-defined term [8:48] and redefined it to mean strong AI means [8:51] general human level intelligence [8:54] being able to solve any task at all that [8:57] level of a human does as opposed to the [9:00] very specific intelligence where you can [9:02] do one thing like play chess or [9:05] recognize faces in a photograph or [9:07] whatever and I think that that's [9:10] interesting and certainly we should have [9:11] some people working on that but I think [9:14] we're still at a phase where we don't [9:17] know how to do it and the analogy I like [9:20] to make is saying well is NASA working [9:23] on interplanetary travel and one answer [9:26] is no because there are no planned [9:28] interplanetary fights but the other [9:31] answer is well they're working on [9:32] specific components they're working on [9:34] solar sails and ion engines and other [9:37] things that may eventually get us to [9:40] interplanetary travel once we figure out [9:41] how it's possible I think it's the same [9:44] thing with AI is that we need more [9:46] pieces we need more components that we [9:47] know work and then eventually somebody [9:50] will be able to say let's put them all [9:51] together and so I'm glad that most of [9:53] the people are working on the components [9:55] and I'm also glad that there are some [9:57] people who are thinking about how to put [9:58] it all together but we're not there yet [10:00] to know how to do it okay peace Estrada [10:06] asks how do you approach a difficult [10:08] programming problem and the answer is I [10:12] don't know yeah you just go ahead and do [10:15] it and hope that you get the answer and [10:18] you can think about a lot of different [10:19] strategies so look to see if there's [10:22] similar problem has been solved before [10:24] by somebody else or by me you can try to [10:28] solve a simpler problem first take a [10:30] piece of it and see if that gives you [10:31] some ideas sometimes even try a more [10:35] difficult problem saying can you [10:36] generalize this and then prove something [10:39] about that generalization and attack [10:40] that problem you can talk to a bunch of [10:44] people see what ideas they have see if [10:47] they have something that helps you can [10:49] try to collect more data to make the [10:52] problem easier or make it go away [10:55] you can finesse the problem by getting [10:57] somebody else to do it you can build a [10:59] social community to do the work for you [11:03] for the parts of the work that you can't [11:04] do yourself I get I guess it occurs to [11:07] me that [11:08] a lot of these strategies you can see in [11:10] polio is how to solve it which is a book [11:13] about mathematics not about programming [11:14] problems but it's really a question of [11:16] when you get stuck what do you do to jog [11:19] your mind a little bit and get in a [11:20] different direction and I think we still [11:23] don't know the answer of what the best [11:25] way to do that is okay obsessed with [11:32] ammas or a mass or something like that [11:35] asks why are we still so bad at software [11:37] development I guess I would disagree I [11:41] think we're actually pretty good at [11:43] software development and I think it's [11:44] just really hard so to know if we're [11:48] good or bad you have to compare it to [11:50] something and you know one thing we [11:53] commonly gets compared to civil [11:55] engineering and bridges don't fall down [11:58] that often but they sometimes do [12:01] programs seem to crash a lot more than [12:04] bridges do but but if you look at the [12:07] relative efforts and the customization [12:09] then the bridge designers have it a lot [12:12] easier because they're building one [12:13] bridge at a time in a specific place and [12:17] they know exactly how much current is [12:19] going by and how much wind and what the [12:22] soil is like underneath and so on and [12:25] they build that one solution whereas [12:28] software engineers we build a solution [12:31] and then we ship it to a hundred million [12:33] people and they're all in slightly [12:35] different environments that we don't get [12:38] to look at ahead of time so I think it [12:39] for the level of difficulty of the [12:41] problem I think we're actually doing [12:43] pretty good it's not perfect and for [12:48] some of those 100 millions you'll ship [12:50] into an environment where it doesn't [12:51] work will be conflict with dll's or you [12:55] know it'll work overtime and then some [12:57] networking standard will get updated and [12:59] it will stop working so there's gonna be [13:01] lots of problems but I don't think any [13:03] other discipline knows how to do that [13:06] difficulty better because I think [13:08] they're solving simpler problems now [13:12] certainly if we could do if we could [13:15] have software that does what the civil [13:17] engineers do then we'd have much more [13:20] reliable system and [13:21] not there yet so what are the civil [13:23] engineers do well they say there's some [13:25] standard approach to building a bridge [13:27] but now on this specific site you know [13:29] what are the real problems to deal with [13:31] well maybe the soil is a little bit [13:34] shifty and so we have to build the tower [13:35] deeper into the ground than we would [13:37] normally a software doesn't do that very [13:40] well you have some capability to [13:42] configure software to your specific [13:44] machine but you really want it to be [13:46] proactive you want it to have the [13:48] discussion that the civil engineers [13:49] would have of saying hey I got deployed [13:51] in this environment what's the hard part [13:53] of this environment oh here's this [13:55] tricky interaction now I'm going to [13:57] reconfigure myself to solve that problem [13:59] and make it go away and so software can [14:02] do that then we'll really be as reliable [14:04] as all these other disciplines but still [14:08] won't need the customization okay next [14:13] question [14:14] run time asks how do you think languages [14:17] will evolve to tackle many core [14:19] processors and do you think any of the [14:21] current paradigms threading with locks [14:23] STM pure functional actor/model GPU [14:27] parallelism with SIMD of multi-threaded [14:30] development will scale to handle them I [14:32] guess I have a different approach to [14:35] looking at it so due to me many core is [14:38] not the problem because I'm used to [14:40] starting from the point of view that the [14:41] computational unit is a data center so [14:45] it's already got thousands of CPUs and [14:47] whether the individual boards have one [14:50] or eight or 128 cores on them that's not [14:54] going to add to the complexity too much [14:56] so we already need an approach that's [14:58] parallel at that very large scale and [15:01] and yes there's going to be a different [15:04] at every level there's a difference if [15:06] you miss the cache that is a difference [15:09] if you go to another core on board [15:13] it's the difference if you go to another [15:15] board on the rack or if you go to [15:17] another machine across racks and which [15:21] switches you're going across you have to [15:22] worry about all those levels of [15:24] distances but in another sense it's [15:27] solving the same problem with different [15:30] constants involved and certainly if [15:33] you're thinking at the level of the [15:35] thousands or hundreds of thousands of [15:37] parallel computations then probably this [15:41] manual keeping track of threads with [15:45] locks is not going to scale very well so [15:47] what we really need is closer to the [15:50] functional type approach something where [15:52] we have the right abstractions to make [15:53] this all work and you know Google is [15:56] well known for having this MapReduce [15:58] abstraction when we first introduced it [16:01] we said well here's one that we know is [16:03] really going to work really well and [16:05] there's probably gonna be a dozen [16:07] different abstractions but let's do this [16:08] one first and then I think it kind of [16:10] surprised us that it went a lot farther [16:12] and then we thought it should go but I [16:14] think it is time to look back and say [16:17] what are the next dozen abstractions [16:19] that are like that that allow you to [16:21] break problems up into pieces and [16:23] combine them back together again we have [16:26] a few ways of combining them together I [16:28] think there should be more and you want [16:30] to have ways where you the program kind [16:34] of declaratively understands the [16:36] community and associativity of the [16:38] problem and how it can be broken up into [16:40] pieces and how it can be assembled back [16:42] together we probably need a few better [16:44] abstractions for dealing with that and [16:46] then we'll be in good shape [16:48] now of course this is all from the point [16:51] of view of various different types of [16:53] computations this fits well for the type [16:55] for the web type of computations where [16:57] you have a lot of data and you're doing [16:59] the same thing mostly to different [17:01] pieces and then summarizing them and [17:04] combining them for other types of [17:05] computations you're going to need [17:06] different types of abstractions ok [17:13] Khafre asks with your big emphasis on [17:17] data over algorithms vastly successful [17:20] as it's been I have to wonder is there a [17:22] point of diminishing returns and [17:23] collecting data where it's once again [17:26] worthwhile to spend your time trying to [17:27] make clever algorithm instead how do you [17:29] recognize that point ok well that's [17:32] pretty easy one way to recognize it is [17:35] you just plot it you know you make a [17:38] graph here's the amount of data you [17:43] collect and here's the performance and [17:45] you get more data and the performance [17:48] goes up [17:48] and at some point it begins to asymptote [17:50] out and then you say well probably I [17:53] don't want any more of the same kind of [17:55] data it's just starting to slow me down [17:57] and not gaining me much of course you [17:59] could always go back and say is there [18:01] other data that I'm missing is there a [18:03] different type of data that I want and [18:05] in some sense that means that you're [18:07] inventing a new algorithm so you could [18:11] do that that's an easy approach and then [18:12] you can also say am i looking at the the [18:14] whole problem wrong is there a [18:16] completely different approach that I [18:17] should be taking and that's where you [18:19] have to start getting clever again okay [18:25] and geesh are more maybe GES harm asks [18:31] from your research at Google what have [18:34] you found to produce an environment most [18:36] conductive to programming our cubicles [18:38] as effective as closed offices is a [18:41] 10-inch netbook as effective in the [18:43] hands of a good programmer as fast [18:45] multi-core Mac Pro with three 30-inch [18:47] monitors in your view should employers [18:50] give more serious consideration to [18:52] working conditions and equipment of [18:54] increasingly well-paid programmers so I [18:56] think that's a great question and that's [18:57] that's actually one that I was [18:59] interested in doing studies for I think [19:01] we we have enough programmers now that [19:03] we can do that kind of experiment [19:05] in-house I see you're starting to see a [19:08] little bit of it I think as a field [19:11] there's been far too little of this we [19:13] really don't know [19:14] you know the programmers tell you what [19:16] they want but there haven't been these [19:18] good scientific studies to say what [19:20] works or what doesn't work I certainly [19:23] have seen studies that say that having [19:26] big monitors is better I don't know if [19:28] those studies were sponsored by the [19:31] monitor of manufacturers so I don't know [19:33] how much I trust them you bought you [19:34] want to see a couple of them before you [19:36] really trust them cubicles versus closed [19:43] offices well you know it's just sort of [19:46] the way buildings are designed you [19:47] normally end up with both I think Google [19:50] the question is more how many people are [19:53] in your office and some companies take [19:56] the point of view that for a programmer [19:58] concentration is the most important [19:59] thing and so every programmer gets a [20:01] private office [20:02] and Google has taken the point of view [20:05] that communication is more important and [20:07] so we try to put a whole team together [20:10] in an office so you're working with two [20:13] or three or four people and they're all [20:14] right there and hopefully the talk that [20:17] goes back and forth is talk that [20:20] everybody should know and so it's not so [20:22] much interrupting your concentration as [20:24] it is keeping you aware of what's going [20:26] on now there's always a trade-off there [20:28] I remember we we had a we do an annual [20:31] survey of things that people like and [20:34] don't like and you know everything from [20:36] it's a cafeteria food good - are the [20:39] chairs comfortable or your back hurting [20:41] you - are the compilers fast enough and [20:46] everything else and the the number one [20:50] thing that people appreciated most out [20:52] of all the fifty or hundred questions [20:54] was the quality of their colleagues and [20:56] so I thought that was a great answer and [20:59] the question that people thought was the [21:02] worst was being interrupted by somebody [21:05] asking a question so so we've determined [21:07] that the perfect place is one where I [21:09] can ask a question of anybody else but [21:11] nobody can ask a question of me but we [21:14] haven't quite figured out how to [21:15] implement that yet so you want to have [21:17] the right trade-off you want to have [21:19] ways of people to signal that they need [21:23] privacy and headphones seem to work [21:27] pretty well for that people put on the [21:29] headphones and then you know you don't [21:30] want to interrupt them sometimes people [21:33] take their laptops and go off someplace [21:35] so they can be private or if they want [21:37] to have a conversation just one-on-one [21:38] with some other person you go and grab [21:41] them and go sit someplace else but I [21:43] think we need we need a lot more [21:44] research to know how you trade these [21:46] things off okay human check to verify [21:55] you are a human and not an AI BOTS [21:58] please quickly answer the following do [22:01] you still play frisbee golf and if yes [22:04] what is your typical score and so sadly [22:07] I haven't really been playing very much [22:10] seriously I like to go out in the [22:12] backyard and throw things around but I [22:14] haven't been competing the way I didn't [22:17] what magazines and books do you read but [22:21] I'm told the best answer for that is all [22:22] of them all define magazines and books I [22:26] read well I guess the last book I read [22:29] was the check mark manifesto which is [22:33] about our checklist manifesto which is [22:35] about the idea of using checklists and [22:38] surgery and these doctor determined that [22:42] patients were getting infected because [22:44] doctors were spreading disease because [22:45] they were forgetting to wash their hands [22:46] and if you have a simple checklist that [22:47] says wash your hands then infections go [22:50] down and so it's interesting to sort of [22:52] apply that are there simple things we [22:54] can do and software development where we [22:58] can learn from that and for the most [23:01] part I think we do a good job we're [23:02] probably better at doing tests than most [23:04] people but that's something to look at [23:09] and so mostly I read kind of nonfiction [23:12] and science oriented science science [23:17] news discovery and a new scientist and [23:21] so on have you found a use for Google [23:23] Wave I played it a little bit I think [23:26] the problem right now it is that not [23:30] everyone is on it so it's harder to [23:32] communicate with I think one of the [23:34] killer apps for Google Wave is to [23:38] discard all the legacy mailing lists [23:41] that you're on that are no good anymore [23:42] so anytime you move to a new platform [23:44] you could just throw away all the old [23:46] stuff and start over again so I think [23:47] that's great and it's exciting to move [23:49] to something new [23:53] what's your favorite game I guess [23:56] ultimate frisbee still my still my [23:58] favorite even though I don't get a [24:00] chance to play as much as I used to and [24:01] finally how do you feel I feel good I [24:05] knew it I would let's see no Theory asks [24:13] what's the relationship between research [24:15] and production code at Google [24:16] how do research projects move into [24:18] production and that's pretty interesting [24:21] because I think it's different at Google [24:23] than it is in many organizations that [24:25] have a [24:27] more strict distinction between research [24:31] and engineering and Google despite the [24:34] size that it's at still operates mostly [24:38] as a start-up and you know and it [24:40] startup you you have a bunch of PhDs but [24:44] you don't stick them off in a separate [24:45] building and call them researchers this [24:47] sort of everybody is they're [24:48] contributing together and people do what [24:51] needs to be done without really [24:53] questioning it and Google still pretty [24:55] much has that feel so that there's not [24:57] that much of a sting shim between being [24:59] a researcher versus being an engineer [25:02] everybody pitches in and it's much [25:05] easier for a research project to make it [25:07] into production I think part of the [25:09] reason for that is because it starts out [25:11] so much closer to production you know in [25:15] academia and in some other research labs [25:17] you know a researcher gets an idea code [25:20] something up on their laptop and then [25:22] they've got a prototype but how are you [25:24] gonna harden that prototype and get it [25:26] into production well you really have to [25:27] throw it out and start over add Google [25:30] part of the point of being here is that [25:32] there is all this data available and in [25:35] order to take advantage of it you can't [25:37] load it onto your laptop it wouldn't fit [25:39] so right from the start the researchers [25:41] are writing code that use our main api's [25:45] that are using the data that everybody [25:48] else uses if you want some webpages you [25:51] use the full copy of the web if you want [25:54] some link data you use the full link [25:56] graph and so on and you use it in the [26:00] data centers and so that makes it a lot [26:02] easier to go from there to launch [26:04] something now you still may you know the [26:07] prototype may run ten times too slow to [26:09] be effective so there may be a big [26:11] reengineering job to get it to launch [26:13] but at least you're a couple steps [26:15] closer and and there certainly are [26:18] examples where Google has gone from a [26:21] research project to production code and [26:24] kept the whole thing in research and I [26:26] think that's kind of unique so we've [26:28] kept for example the speech recognition [26:30] and the machine translation were started [26:33] as research projects but then rather [26:35] than handing them off to an engineering [26:36] team when they were ready to go we said [26:39] we don't want to split up the team we [26:40] know [26:41] I have 1/2 that's running a production [26:43] system 1/2 that's trying to build the [26:45] next system we want to keep it together [26:46] and we'll keep it together inside [26:48] research we've brought production [26:50] engineers into research to help the [26:52] research team and they're simultaneously [26:53] maintaining the current system and [26:56] trying to update the next one [26:59] questions were reddit yes [27:03] as a community do you feel that reddit [27:07] is getting what you want [27:08] so is this experiment of saying we're [27:11] just going to both things up is this [27:13] finding what you want is it getting you [27:14] the things that are appropriate for you [27:16] is it casting a net that's wide enough [27:18] or too wide and if not is there a way to [27:23] change the rules to make it better