Is It Time to Rewrite the Operating System in Rust?

Is It Time to Rewrite the Operating System in Rust? — InfoQ

InfoQ

AI-assisted summary of Is It Time to Rewrite the Operating System in Rust? — original source. Check the original for full context.

Source published:

Key ideas

  • Rewriting entire operating systems from scratch in Rust risks repeating historical second-system failures and breaks critical binary compatibility with existing software. (54:53)
  • Rust delivers memory safety and high-level abstractions without a garbage collector or runtime overhead, making it the first viable C competitor for systems programming in thirty years. (49:14)
  • The immediate opportunity for Rust in systems infrastructure lies in hybrid modernization: replacing vulnerable firmware like OpenBMC, rewriting userland control daemons, and adding in-kernel drivers. (60:03)
  • Direct ports of C algorithms to Rust frequently fail because circular and intrusive pointer graphs clash directly with the borrow checker's strict single-ownership model. (41:00)

Detailed notes

## Background This talk was delivered by Bryan Cantrill at QCon San Francisco 2018 and published on YouTube by InfoQ on February 25, 2019. According to the video description, Cantrill examines why Rust has captured the imagination of systems software engineers and explores where it best fits within the deep operating system stack.

## TL;DR Bryan Cantrill assesses whether the operating system should be rewritten in Rust through the lens of systems programming history, tracing the lineage of Multics, Unix, C, and failed object-oriented rewrites like Sun Spring and Copland. While clean-slate de novo kernel rewrites fall victim to second-system syndrome and forfeit crucial legacy binary compatibility, Rust's compile-time ownership model, zero runtime overhead, and full-duplex C FFI make it the first viable language in decades to displace C. Cantrill argues against wholesale kernel rewrites, advocating instead for a pragmatic hybrid approach that integrates Rust into drivers, virtual file systems, complex userland system daemons, and historically vulnerable firmware environments like OpenBMC.

## Atmosphere & style Cantrill delivers a high-octane, theatrical performance packed with sharp comedic timing, booming laughter, and rapid-fire historical anecdotes. The presentation blends the gritty realism of a veteran kernel engineer with infectious evangelism, pivoting effortlessly from lamentations about the pitfalls of intrusive data structures and firmware 'sewers' to profound awe at Rust's compile-time safety guarantees. Watching it feels like a whirlwind masterclass in computing history delivered by a battle-scarred practitioner who is genuinely thrilled about the future of systems software.

## Bulletin - [54:53](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3293s) Rewriting entire operating systems from scratch in Rust risks repeating historical second-system failures and breaks critical binary compatibility with existing software. - [49:14](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2954s) Rust delivers memory safety and high-level abstractions without a garbage collector or runtime overhead, making it the first viable C competitor for systems programming in thirty years. - [01:00:03](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3603s) The immediate opportunity for Rust in systems infrastructure lies in hybrid modernization: replacing vulnerable firmware like OpenBMC, rewriting userland control daemons, and adding in-kernel drivers. - [41:00](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2460s) Direct ports of C algorithms to Rust frequently fail because circular and intrusive pointer graphs clash directly with the borrow checker's strict single-ownership model.

## Key takeaways - [49:14](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2954s) Operating systems must deliver raw execution speed, deterministic latency, and hardware abstraction; developer ergonomics alone cannot justify rewriting low-level software. - [05:45](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=345s) Historical second-system rewrites—such as Multics with PL/I or 1990s object-oriented projects like Apple Copland and Sun Spring—stalled or collapsed under excessive complexity. - [28:05](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=1685s) Managed languages with garbage collection remain ill-suited for core OS infrastructure due to unpredictable pause times and latency spikes. - [34:39](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2079s) Rust's compile-time ownership model enforces explicit resource lifetimes across library boundaries without relying on runtime garbage collection or developer vigilance. - [41:00](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2460s) Intrusive data structures common in C, such as doubly linked lists and AVL trees, require fundamental redesign into ownership-friendly structures like B-trees when moving to Rust. - [43:25](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2605s) Contiguous memory structures like B-trees often allow naive Rust implementations to outperform hand-tuned C code by minimizing CPU cache misses and pointer chasing. - [54:53](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3293s) Clean-slate kernel rewrites struggle in production because maintaining Linux binary compatibility presents an endless tail of edge cases and emulated syscall behaviors. - [50:32](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3032s) Production operating system kernels require custom allocation failure handling, as kernels cannot simply panic on out-of-memory errors like userland applications. - [45:56](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2756s) Seamless, full-duplex foreign function interface (FFI) support allows Rust to be integrated incrementally into existing C and assembly codebases. - [57:53](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3473s) Userland system daemons like systemd or SMF represent an ideal target for Rust adoption because managing distributed state and asynchronous failures is exceptionally complex. - [01:00:03](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3603s) Network-exposed, unobservable server firmware such as OpenBMC represents an urgent and practical frontier for complete rewrites in Rust. - [01:05:43](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3943s) Language enhancements such as non-lexical lifetimes and custom global allocators have matured Rust into a stable, production-ready foundation for permanent systems software.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=4s) Betteridge's Law and the Existential Crisis of Defining Software

Bryan Cantrill begins by confronting the talk's core question: is it time to rewrite the operating system in Rust? Invoking Betteridge's law of headlines—which dictates that any headline ending in a question mark can be answered with "no"—he gives an upfront spoiler: listeners will not get a flat "no," but rather a series of qualified yeses. Even so, the title remains a question mark rather than an exclamation point. Looking at the OS track lineup, Cantrill points out how absurd it is that multiple speakers still need an introductory slide asking what an operating system even is, despite the concept having been around for decades.

To demonstrate how deceptively slippery fundamental definitions are, Cantrill challenges the audience to define software. He points to Euclid's greatest common divisor (GCD) algorithm: if one argues it is not software simply because it predated hardware, constructing a consistent definition of software becomes nearly impossible. He recalls a visit to Cloudera in its earliest days, when only about six people worked in the office. Cantrill was having a weird day, deeply preoccupied with this question, and began pressing the Cloudera team. When they answered that software is simply something that runs on hardware, Cantrill began pulling that logic apart, pushing everyone into an existential crisis where ancient Sumerian procedures might qualify as software, provided software is viewed merely as executable knowledge regardless of Turing completeness. Seeing co-founder Jeff Hammerbacher staring at his hands in an existential daze, co-founder Mike Olson pulled Cantrill aside to ask if he was high or on drugs, demanding: “All right, can you just like talk about something else or whatever or get out of here?”

```mermaid flowchart TD accTitle: The Breakdown of Common Software and OS Definitions accDescr: Diagram illustrating how circular definitions of software, hardware, and operating systems unravel into edge cases like Euclid's algorithm and virtualization.

subgraph DEF_SW ["Defining 'Software'"] A["Attempt: 'Software is what runs on hardware'"] --> B{"Is Euclid's GCD software?"} B -->|Predates hardware| C["Knowledge execution / Sumerian algorithms"] B -->|Requires hardware| D["Circular crisis: What is hardware?"] end

subgraph DEF_OS ["Defining 'Operating System'"] E["Attempt: 'An OS abstracts hardware'"] --> F{"What about guest VMs?"} F -->|Abstracts hypervisor| G["Virtual hardware abstraction"] F -->|Abstracts silicon| H["Ring 0 vs. nested ring 0"] end ```

[02:21](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=141s) Defining the Operating System and Separating Kernel from OS

Moving from the ambiguity of software to operating systems, Cantrill notes that the classic definition—a system that abstracts hardware—breaks down under modern workloads. A guest operating system running on an AWS EC2 instance abstracts virtual hardware rather than bare metal, yet it remains unquestionably an operating system. Cantrill formulates a working baseline definition: an operating system is a program that abstracts hardware to allow for the execution of other programs.

Cantrill recounts catching up with his friend Mike Demmer, who had spent a couple of years writing kernel code at Tera (a high-performance computing company that later bought Cray and adopted the Cray name). Demmer remarked that an operating system is ultimately just a program—an asynchronous, interrupt-driven, complicated program, but a program nonetheless. Cantrill was initially offended in his pride as a kernel engineer, but eventually conceded the point. An operating system must abstract hardware and run other programs; furthermore, it defines their liveness: if the OS dies, every program executing beneath it dies as well.

Critically, Cantrill insists on separating the operating system from the operating system kernel: - **The Kernel:** The specific component running in the microprocessor's most privileged execution mode (such as Ring 0 on x86, though nested virtualization introduces additional virtual rings away from real silicon). - **The Operating System:** The broader ecosystem, encompassing userland libraries, system commands, background daemons, and management facilities.

Even when modern lightweight designs attempt to strip system bloat down to the bone, Cantrill emphasizes that substantial portions of the operating system necessarily execute outside the highest privilege level of the processor.

> “You know what? I realized that an operating system is really just a program. It's just a complicated program. It's an asynchronous program. It's an interrupt-driven program.” — Mike Demmer (recounted by Bryan Cantrill)

[05:45](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=345s) Early Operating System History: Executives, Burroughs, and Second System Syndrome

Cantrill dives into the history of systems software, observing that operating systems were originally termed “executives,” reflecting a mid-twentieth-century, post-World War II managerial hierarchy. Early executives were universally authored in raw machine code and assembly. A notable departure arrived in 1961 with the Burroughs B5000 mainframe and its operating system, the Master Control Program (MCP)—the namesake of the villainous system in the film *Tron* (which Cantrill pans, urging the audience to watch the 1983 film *WarGames* instead).

The Burroughs MCP was groundbreaking because it was implemented not in assembly, but in ESPOL (Executive Systems Problem Oriented Language), a custom language derivative of ALGOL. Cantrill quips that the name sounds like Esperanto or the Escuela Superior Politécnica del Litoral in Ecuador. Despite being decades ahead of its time in hardware-software co-design, Burroughs ultimately succumbed commercially, proving Cantrill's career observation: “being ahead of your time is not commercially fruitful.”

By 1964, efforts began on Multics to create an ambitious follow-up to MIT's CTSS (Compatible Time-Sharing System). In doing so, the project fell squarely into the trap of "second system syndrome"—the tendency described in *The Mythical Man-Month* where engineers, determined to rectify every flaw and limitation of a successful initial system, attempt to pack every conceivable idea and rewrite into the second iteration.

[08:38](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=518s) Second System Syndrome and the Multics Gamble on PL/I

Bryan Cantrill begins by outlining the classic trap that snares ambitious engineering projects: rewriting existing pieces, rethinking architectures, redesigning the database, and endlessly attempting to “finally do this the right way.” The inevitable result is an overgrown system that slips further and further behind schedule until it never ships at all. Cantrill identifies this dysfunction directly: “Um and that's the second system syndrome.” Project MAC at MIT suffered what was perhaps history's most catastrophic case of it with the Multics project.

Multics emerged in an awkward era for system software. Its architects knew they wanted to build the operating system in a higher-level language, but contemporary lore often claims they were constrained because their only alternatives were Fortran and COBOL. Cantrill rejects that framing as historically inaccurate, citing early computer scientist Jean Sammet and her 1963 landmark taxonomy book *Programming Languages*. That book was far from brief; as Cantrill puts it, the early 1960s were like a “Mesozoic era of programming languages, too much CO2 in the in the programming language atmosphere or something, you know, 80-ft tall ferns or whatever.” Multics's designers disliked all existing options and chose instead to standardize on PL/I—a language that had an elaborate specification but did not yet exist as an actual working implementation.

```mermaid flowchart TD accTitle: The Multics PL/I Waterfall Failure accDescr: Diagram illustrating how the Multics project adopted an unimplemented language, outsourced the compiler to a failing third party, and stalled for years without runnable code. subgraph SpecPhase["Design Phase"] A["Reject Existing Languages<br>(Fortran, COBOL, etc.)"] --> B["Adopt Unimplemented Language<br>(PL/I Specification)"] B --> C["Write Full Multics OS Spec<br>(Zero runnable code)"] end subgraph CompilerFailure["The Outsourcing Trap"] C --> D["Outsource 1st PL/I Compiler<br>(To unknown LA firm)"] D --> E["Firm fails after 1 year:<br>'We don't know what we're doing'"] E --> F["No compiler until 1966<br>(3-year stall)"] end subgraph Fallout["Coalition Collapse"] F --> G["Bell Labs abandons Multics (1969)"] G --> H["Ken Thompson implements Unix<br>on PDP-7 in Assembly"] end ```

[10:03](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=603s) Nightmare Waterfall and the Collapse of the Multics Triumvirate

Multics took waterfall engineering to a disastrous extreme. After writing an exhaustive specification for an operating system without a working compiler, project leadership decided to outsource the creation of the first-ever PL/I compiler to an obscure software shop in Los Angeles. A year later, Multics leadership checked in only to be told the contractor had made no progress and was incapable of building it. Multics was stranded with an immaculate system specification and zero ability to generate runnable machine code. Cantrill characterizes this as an “ultimate waterfall, like nightmare waterfall,” noting that Multics did not secure a working PL/I compiler until 1966.

The historical legacy of Multics remains deeply polarized: to some adherents it was an underappreciated masterpiece, while to critics it was an expensive failure. In 1971, Fernando Corbató published *Multics—The First Seven Years*, celebrating PL/I as an essential triumph of the architecture while glossing over the three-year delay caused by having no compiler. To Cantrill, this compiler vacuum directly destabilized the project's governing coalition—the “triumvirate” of MIT, General Electric, and Bell Labs. In an attempt to unblock themselves, some engineers created an interim dialect dubbed Early PL/I (EPL). Yet contemporary accounts diverge wildly like an operating system version of *Rashomon*: EPL developers claimed they saved the project, whereas Corbató later dismissed EPL as inconsequential and largely unused. Unable to tolerate the mounting delays, Bell Labs pulled out of the Multics project entirely in 1969.

- Multics insisted on formatting PL/I with a Roman numeral “I”; Cantrill warns audiences never to use an Arabic “1” on their slides. - Bell Labs abandoned the Multics consortium in 1969 after years of schedule slips and language tooling stalls. - The dialect EPL (Early PL/I) was built using TMG, an early compiler-compiler and macro processor.

[12:51](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=771s) Top-Down Specification vs. Bottom-Up Unix

Following the Bell Labs withdrawal, researcher Ken Thompson set out to build an alternative operating system. Free from Multics's bureaucratic overhead, he developed a nimble system on a spare PDP-7, later porting it to a PDP-11/20 and naming it Unix—a deliberate pun on the unwieldy complexity of Multics. Comparing the two projects, Cantrill highlights a profound philosophical clash between top-down and bottom-up engineering:

> “One of the things that I see, especially reading it side by side with the Multics history, is it was really top-down software versus bottom-up software.”

Multics relied on rigid, command-and-control planning: specify every detail in advance and hope the tools eventually appear. Unix, conversely, grew organically: get a small piece working, see what works, and then write a manual page documenting the code that actually exists. Cantrill views this bottom-up approach as matching the cultural moment of the late 1960s and early 1970s, though he jokes about his mother reminding him that as someone not yet born in 1968, he has no real concept of a society in upheaval.

[14:26](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=866s) The Myth of C: Assembly Roots, B, and the Road to Portable Assembly

Cantrill dispels a persistent industry myth: Unix was not initially written in C. The first versions for the PDP-7 and PDP-11 were authored purely in assembly. The high-level language available to them at the time was B, a typeless, interpreted derivative of BCPL created by Ken Thompson. B shared strange historical roots with Multics through TMG (the "transmogrifier" parsing system), which had been written by Bob Morris—father of Robert Tappan Morris, author of the 1988 Morris Internet worm.

In Unix, B was relegated to secondary utilities like parts of the assembler and an early version of `dc` (the desktop calculator). Cantrill digresses into the familiar developer experience of accidentally typing `dc` instead of `cd`: the shell launches a cryptic interactive prompt that silently swallows `Ctrl+C` and ignores requests to exit, a stubborn quirk preserving its B heritage. Thompson and Dennis Ritchie appreciated the abstraction of high-level programming, but B was interpreted, slow, and lacked byte addressability, making it entirely unsuitable for systems programming on PDP hardware.

To overcome B's limitations, Ritchie evolved the language into C. Contrary to popular assumptions, Unix and C were not conceived together: Unix is the older sibling that ultimately begat C. Cantrill defends C's common characterization as “portable assembly,” embracing it as a badge of honor rather than a critique:

> “I like the People call portable C portable assembly as a slur, but I I view it as praise. I love I mean, like, 'What? It's assembly and it's portable? That sounds great.'”

- B was an interpreted, word-oriented language derived from BCPL and lacked support for byte-level memory addressing. - TMG was authored by Bell Labs researcher Bob Morris, whose son later created the Morris worm. - Unix begat C out of necessity after assembly proved unportable and B proved too slow and structurally limited for system software.

[17:27](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=1047s) The Organic Evolution of C and the Missing Logical XOR

Bryan Cantrill contrasts early computing languages with the arrival of C, describing looking at C as "stepping into modernity suddenly." Prior programming languages seemed as though they were permanently shouted, as if the Caps Lock key were stuck. To illustrate this, Cantrill shares an anecdote about collecting vintage programming manuals. After buying an odd lot from someone in the UK that mostly contained useless material, he discovered a manual for "Language H," an obscure COBOL derivative created by NCR Elliott with `STERLING` as an actual keyword. In Language H, every program began with the keyword `CHAPTER 1`—treating software as if it were a book because computing was so young that literature served as the primary mental model—and every program concluded not with `EPILOGUE` or `END`, but with the command `OBEY`.

```mermaid flowchart TD accTitle: Evolution of C Features accDescr: Diagram illustrating how C evolved organically to meet the immediate implementation needs of the Unix kernel. subgraph Needs["Unix Kernel Requirements"] N1["Need complex data records"] N2["Need hardware bit access"] end subgraph Features["C Language Additions"] F1["Structures (added 1973)"] F2["Bit fields"] end N1 --> F1 N2 --> F2 F1 --> Kernel["Unix kernel rewritten in C"] F2 --> Kernel ```

Cantrill points out that unlike Multics, which pursued a top-down, command-and-control design, Unix and C grew organically from the bottom up, implementing features only when practical needs demanded them. The first version of C lacked structures entirely; they were introduced in 1973 specifically because Ken Thompson and Dennis Ritchie needed them to write the Unix kernel, refusing to implement the kernel in C until structures existed. Similarly, bit fields were added purely to handle hardware-level fields.

However, this organic growth had distinct downsides, leading Cantrill to air his long-standing grievance: the absence of a logical XOR operator in C. While C provides bitwise XOR (`^`), bitwise AND (`&`), bitwise OR (`|`), logical AND (`&&`), and logical OR (`||`), it omits a logical equivalent for asserting that exactly one of two conditions is true. Cantrill jokes that anyone reading his code can identify it by an assertion using bitwise XOR, followed by a multi-line comment: > “You'll know you're in my code because that code has that assertion, yes, done with bitwise XOR. I know how to do it, okay? With an extra line of code and then a block comment like cursing the ghost of Dennis Ritchie for not having logical XOR.”

When Cantrill complained about this omission to his colleague Roger Faulkner (who had worked alongside Ritchie), Faulkner explained that logical XOR was omitted because it cannot short-circuit. Cantrill pushed back, asking if short-circuiting was some biblical commandment for logical operators. When Faulkner asked Ritchie directly, Ritchie confirmed the reasoning: logical XOR was left out because it cannot short-circuit, so developers should just use bitwise operations instead.

[20:50](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=1250s) Preprocessors, C Dominance, and the Object-Oriented "Darkness"

Beyond basic operators, C developed powerful capabilities like macro processing through the preprocessor. While modern developers are often instructed to avoid the preprocessor entirely, Cantrill equates this advice to warnings given to teenagers:

> “Yes, I know you've been told never use the preprocessor and yeah, I get it. But like that's the same thing like we tell our kids about like you know, that they shouldn't you know, drink and they shouldn't do drugs and they shouldn't they shouldn't have sex. Those are all the fun stuff like fun stuff that like actually they should do when they get older and they can do so safely.”

Cantrill emphasizes that the C preprocessor was indispensable for major software milestones, including DTrace and ZFS, even though its design left much to be desired and subsequent standardization efforts arrived late and proved contentious. Throughout the 1980s, C dominated the industry and proved to be an exceptional fit for operating systems. Apart from rare exceptions—such as the classic Macintosh OS, which was written in Pascal, research microkernels, or MS-DOS remaining in assembly—virtually every production operating system was built in C.

In the 1990s, however, Cantrill recalls that "a great darkness came across the land." An aggressive object-oriented industrial complex emerged, dictating that every piece of software had to be object-oriented. C-based systems were suddenly dismissed as obsolete relics or the "walking dead," destined to be swept aside by grand architectural rewrites in C++ or Java.

[23:08](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=1388s) The Second-System Graveyard: Copland, Spring, Taligent, and JavaOS

The ambitious object-oriented operating system rewrites of the 1990s collapsed under the weight of Fred Brooks' famous "second-system effect." Cantrill highlights several prominent casualties of this era:

* **Apple Copland:** During Gil Amelio's tenure, Apple was struggling so severely to ship Copland that rival Silicon Valley firms were eyeing Apple primarily for its real estate rather than its technology. * **Sun Spring:** Written entirely in C++, Spring arrived with immense hype. When Cantrill tested it as an undergraduate, the operating system did little more than thrash the hard drive spindle. While Cantrill's test machine had what was then a high-end 32 megabytes of DRAM, the developers claimed it required 128 megabytes—an absurd demand equivalent to requiring two terabytes of DRAM today. Cantrill remarks that giving operating system developers an abundance of memory is dangerous because they will consume all of it, leaving the software unusable on mainstream hardware. * **Taligent:** A joint venture between Apple and IBM, Taligent was marketed as an ivory-tower effort where the industry's finest would produce the definitive future operating system. Cantrill notes with amusement that "Taligent" was an arrogant portmanteau of "talented" and "intelligent": > “Taligent, as it turns out, is a portmanteau of talented and intelligent. It's like, 'Woo boy.' It's like, 'How about how about that?' Like, that's call your company that. And as it turns out, they like nailed the worst of IBM meeting the worst of Apple in an effort that was a total disaster.” Taking a ground-up rewrite approach caused Taligent to act as a magnet for over-engineering, attracting "abstraction garbage like some sort of terrible abstraction fly strip." * **JavaOS:** Sun Microsystems envisioned a Java-everywhere future featuring dedicated Java microprocessors and JavaOS. The project foundered on fundamental systems realities; notably, the original Java specification lacked unsigned integer types, making direct communication with underlying hardware exceptionally difficult.

As all of these complex second-system projects crumbled under their own abstractions, Linux arrived to restore practical, C-based systems engineering to commodity x86 hardware.

[25:57](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=1557s) The Hegemony of C and the Rise of Dynamic Languages

Bryan Cantrill recounts how the massive resurgence of Unix led to C-based systems becoming deeply entrenched. While there were several efforts to build C++ operating systems, they universally withered away. Even when people cite counterexamples like Haiku, Cantrill points out its true scope:

> “I mean, I love Haiku, first of all, but Haiku was designed as like the hobbyist operating system to emulate BeOS.”

With C++ kernels largely failing to take hold, core systems remained written in C. Meanwhile, the broader programming landscape was dominated by an era of heavy Java and C++ hegemony, followed by the explosion of dynamic languages like Ruby, Python, and JavaScript. Cantrill recalls how JavaScript underwent a major shift: around 2006, developers realized that instead of just handling `<blink>` tags, the browser contained a real, functional language—essentially Lisp in disguise, but without the parentheses Cantrill dislikes. All of these dynamic languages heavily optimized for quick turnaround and developer speed. As Cantrill notes with typical bluntness:

> “I cannot bring myself to put developer velocity on a slide because it makes me want to throw up.”

Joking with an audience member named Steve about supposed "Ruby OS" or "Python OS" projects, Cantrill laughs that experimenting with such ideas belongs strictly to college days, where mistakes are age-appropriate, but that none constituted a serious foundation for actual operating systems.

- **C-based entrenchment**: Unix's comeback locked in C as the standard systems language, leaving C++ kernel projects abandoned. - **Dynamic wave**: The rise of Ruby, Python, and client-side JavaScript centered around developer velocity rather than low-level control. - **JavaScript re-evaluation**: Viewed initially as a toy for gimmicks, developers in the mid-2000s realized it was an expressive, functional language embedded in every browser.

---

[28:05](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=1685s) The 2010s Systems Dilemma: Node.js, Go, and the Problem with Garbage Collection

Entering the 2010s, systems programmers found themselves in a difficult bind. They craved modern programming conveniences—maps, hashes, string splitting, and regular expressions—without sacrificing the raw performance of C. The browser wars had supercharged runtimes like V8, which Joyent and others used on the server via Node.js. Yet Node.js exposed its own severe shortcomings. Cantrill remarks:

> “JavaScript is the failed state of programming languages.”

Its permissiveness caused significant issues, eventually forcing the ecosystem to adopt tools like TypeScript to regain sanity. Around the same time, the "Bell Labs refugees" at Google developed Go, solving many systems engineering problems, as showcased in talks like Aiden's presentation on gVisor. However, for core operating system infrastructure, Go and JavaScript both carry an insurmountable architectural obstacle: garbage collection.

```mermaid flowchart TD accTitle: The Memory Leak Dilemma in Garbage Collected Systems accDescr: Diagram illustrating how retain cycles prevent GC from freeing memory.

subgraph UserCode["User Application Code"] Closure["Accidental Closure Reference"] Root["Active Root Object"] end

subgraph HeapMemory["Heap Object Graph"] NodeA["Target Heap Object"] NodeB["Interconnected Subgraph"] end

Closure -->|"unintentional retain"| NodeA Root --> Closure NodeA --> NodeB

Collector["Garbage Collector / Heap Scanner"] -.->|"scans graph; cannot reclaim"| NodeB ```

Cantrill emphasizes that blaming the garbage collector for poor application health is misguided:

> “People like to blame the garbage collector... It's like, 'No, the garbage is ruining your process. Like, you're creating the garbage. It's not the GC hates you. It's like, stop creating crap and the GC won't have to do this.'”

In languages like JavaScript, accidental closures easily hold references into giant, interconnected object graphs, leaving developers baffled by memory leaks in managed code. In reality, the collector is acting not as a garbage collector, but as an exhaustive heap scanner that cannot reclaim memory because the programmer never severed the reference.

Cantrill points out that while garbage collection is wonderful for many types of applications, it fundamentally fails for low-level operating system infrastructure where latency must remain predictable and deterministic:

> “For system software, for the operating system, for that infrastructure, I think GC is a real problem.”

---

[32:20](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=1940s) Moving Past C++ and Discovering Rust

Having exhausted Node.js and ruled out GC-based languages for performance-critical systems, Cantrill faced a crossroads. He refused to return to C++, despite hearing constant pitches about how modern C++ had changed for the better. Having written 100,000 lines of C++ in college, he viewed the language as an irreconcilable past trauma:

> “That's great for C++ and I want C++ to have a happy and productive life, but C++ dragged all my [stuff] into the street and lit it on fire. So, like I am not going back to C++ ever and that's not a value judgment. That's just like that's self-preservation.”

Surveying the audience, Cantrill finds that while a good number had written some Rust, a larger group fell into the camp he had occupied earlier: being "seriously Rust-curious." He confesses that he spent time reading blog posts, trying to invent petty aesthetic grievances—such as nitpicking its whitespace—just to find a reason to dismiss it, before realizing he had to set trivialities aside.

Ultimately, Rust's core pitch resonated deeply with his philosophy. The language advertises itself clearly on its homepage as a systems programming language designed for blistering speed and memory safety without a garbage collector. For Cantrill, this aligns with his lifelong engineering objective: creating robust, permanent software built to outlast its creator.

- **C++ ruled out**: Cantrill considers modern C++ an absolute non-starter based on past software maintenance scars. - **Rust curiosity**: Moved from speculative reading and superficial syntax nitpicking to serious technical interest. - **Shared values**: Rust's foundational focus on systems-level performance, safety, and permanence directly matches the needs of long-lived OS infrastructure.

[34:39](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2079s) The Longevity of Software and Rust's Core Bargain

Bryan Cantrill reflects on the economics and beauty of working code. While software developers often complain that everything is perpetually broken, he argues that the inverse is profoundly true: software that works behaves like Euclid’s greatest common divisor algorithm from 2,000 years ago. When a program solves its intended problem correctly, it acts as a timeless mathematical machine that can operate indefinitely without needing an expensive rewrite. Cantrill expresses genuine gratitude for living in what he considers a golden age of engineering, suggesting that future descendants will look back in awe at this foundational era of computer science. However, because certain systems cannot survive long-term under current paradigms, they must be rewritten in technologies designed to last in perpetuity. For Cantrill, this industrial aspiration—building structures capable of withstanding the elements where oxidation is the primary adversary—is captured neatly in Graydon Hoare's naming of the Rust programming language.

```mermaid flowchart TD accTitle: The Rust Memory Management Bargain accDescr: Diagram illustrating the trade-off made between the developer and the Rust compiler regarding ownership and lifetime annotations.

subgraph CompilerContract["The Compiler's Bargain"] GC_Benefit["Garbage-Collected Safety<br/>(No manual free, no leaks)"] Manual_Perf["Manual Performance<br/>(Zero runtime overhead)"] end

subgraph DevCost["Developer Obligation"] Ownership["Clear Ownership Model"] Lifetimes["Lexical Lifetime Rules"] BorrowChecker["Pleasing the Borrow Checker"] end

DevCost -->|"developer proves correctness"| CompilerContract ```

What makes Rust singular among modern languages is its strict compile-time model of ownership. Rather than requiring developers to manually invoke memory management primitives like `free` or forcing an automatic runtime garbage collector, the compiler statically calculates the exact thread of execution that owns a given piece of memory. Cantrill describes this as a massive contract offered directly by the compiler:

> “I will give you this incredible superpower where you don't need to think about this, but you're going to need to help me out.”

Under this pact, the compiler demands that developers explicitly track how memory is loaned, borrowed, and returned using lexical lifetimes. Whenever the compiler becomes confused about reference lifetimes, it produces cryptic diagnostic errors, making developers feel like they are repeatedly being forced to fetch rocks before their code will compile. Yet the payoff of this arduous friction is unprecedented: developers obtain the full safety of a garbage-collected language married directly to the raw efficiency of manual memory management.

---

[37:31](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2251s) Beyond Personal Safety: The Problem of Software Composition

While memory safety is the banner feature of Rust for most engineers—preventing buffer overflows, miscalculated `memcpy` calls, uninitialized `bzero` buffers, and struct omissions—Cantrill notes that his own enthusiasm comes from a different angle. He admits he can write correct C code because decades of systems programming beat the discipline into him. Rust's strictness, such as requiring every struct field to be initialized up front rather than allowing uninitialized memory to slip through, initially prompts annoyance until the developer realizes the compiler is entirely right. Furthermore, while the borrow checker has earned a terrifying reputation for rejecting non-trivial code, the Rust compiler overcompensates by being extraordinarily helpful, displaying full-color ASCII diagnostic messages and offering suggestions for precisely what was intended.

Cantrill points out that while an expert developer might be able to craft perfectly safe C in isolation, safe C cannot scale when combining multiple components across an organizational boundary:

> “Even if you and I both know how to write memory safe C, it's very hard for us to have an interface boundary where we can agree about who does what.”

In C, two developers writing individually correct code will inevitably suffer miscommunications over ownership contracts: one assumes the caller frees a given resource handle on an error path, while the other assumes the callee takes ownership. C systems rely heavily on intrusive data structures—where an object embeds linked-list or tree nodes directly inside itself—to work around this friction, but those patterns present severe challenges for Rust. With Rust, composability is achieved natively because the borrow checker statically guarantees that ownership contracts across distinct library boundaries are respected at compile time without relying on human vigilance.

---

[41:00](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2460s) Rethinking Data Structures: Porting C to Rust

Turning to his own practical work, Cantrill recounts rewriting a critical production node service originally written in C and Node.js directly into Rust. He warns against generalizing his individual findings into sweeping internet flame wars about C versus Rust performance, sharing an anecdote about how an eight-paragraph disclaimer on his personal blog still yielded immediate comments claiming he thought Rust was always universally faster than C.

The biggest pitfall he encountered during the rewrite was attempting to translate C idioms straight into Rust. In traditional C systems programming, shared and circular ownership is ubiquitous:

> “So, when you a doubly linked list is actually two linked lists, right? And a given object is actually owned by two things. The thing your your next thing and your previous thing. They both think they own you. Rust does not want to do that.”

While there are famous deep-dive blog tutorials detailing how to write doubly linked lists in Rust, Cantrill notes that the language fundamentally demands rethinking problem domains from scratch. If an engineer structures algorithms around explicit, unambiguous single-ownership lines rather than complex intrusive graphs, Rust rewards them with exceptionally high-performance binaries. During a brief aside, Cantrill pauses in surprise after seeing a five-minute warning sign, only to humorously realize that the timekeeper was setting aside time for audience questions—prompting him to joke that he has no intention of taking questions and can use the entire remaining time to speak.

Reflecting on the psychological arc of his Rust implementation, Cantrill describes hitting an intense "bargaining phase" where the compiler beat him down so thoroughly that he gave up on writing hyper-optimized abstractions. Out of sheer fatigue, he abandoned trying to mirror his clever C architecture and opted for the most straightforward, naive, knuckleheaded approach imaginable just to get the program through the compiler, fully expecting that this dumbed-down implementation would exhibit abysmal runtime performance compared to the original C code.

[43:25](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2605s) Outperforming C: Cache Lines, AVL Trees, and Rust's B-Trees

Bryan Cantrill recounts a surprising benchmarking result from his own code. Having built what he felt was a clumsy, naive doubly linked list in Rust—using hash tables of UUIDs and timestamps to look up consecutive elements—he fully expected the implementation to crawl. Instead, it ran faster than his hand-tuned C code. Digging into the performance discrepancy revealed that the application spent almost all its execution time searching a sorted data structure. In C, Cantrill had used an AVL tree, a ubiquitous choice in systems software because it is simple, proven, and intrusive: an AVL tree node can be embedded directly into an outer C struct, allowing a single structure to reside on fifteen different trees simultaneously with zero extra allocations.

However, modern computer architecture has shifted the trade-offs. While an AVL tree is mathematically sound, pointer chasing across individually allocated tree nodes behaves poorly on modern CPU caches. Rust’s standard library binary tree collections (`BTreeSet` and `BTreeMap`) rely on B-trees, a structure traditionally associated with disk databases. Because B-trees group multiple keys together in contiguous nodes, they exploit cache lines and modern memory hierarchies far more efficiently than binary search trees. Cantrill notes that writing a generic B-tree in C is "gnarly" and dangerously intrusive; if a hand-rolled C B-tree corrupts memory, tracking down the bug is a nightmare. Thanks to Rust's type system and composability, developers get an ultra-optimized, memory-safe B-tree implementation out of the box without paying in developer sanity.

```mermaid flowchart TD accTitle: Data Structure Cache Efficiency Comparison accDescr: Diagram comparing C AVL tree pointer chasing with Rust B-Tree contiguous node layout in modern CPU caches.

subgraph C_Impl ["C Implementation: Intrusive AVL Tree"] AVL1["Node A (Heap pointer)"] --> AVL2["Node B (Heap pointer)"] AVL1 --> AVL3["Node C (Heap pointer)"] AVL_Note["Pointer chasing misses L1/L2 cache lines"] end

subgraph Rust_Impl ["Rust Implementation: Standard B-Tree"] BNode["B-Tree Node: [Key1 | Key2 | Key3 | Key4]"] BNode --> Child["Child Node: [Key5 | Key6 | Key7]"] B_Note["Contiguous chunks fit CPU cache lines perfectly"] end ```

- In C systems code, AVL trees remain standard because embedding intrusive tree nodes enables multi-tree membership without auxiliary dynamic allocation wrappers. - Modern hardware memory hierarchies heavily penalize pointer chasing across scattered heap allocations, making B-trees significantly faster in practice. - “I like a B-tree in C is gnarly. And cuz a B-tree like is just I mean, talk about intrusive. It's very intrusive. It's very kind of up in everything's underwear.”

[45:56](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2756s) Language Features Beyond Ownership: Enums, Macros, and Duplex FFI

Cantrill surveys the broader Rust feature set that won him over once he moved past the borrow checker. Algebraic data types stand out immediately for error handling. By providing first-class tagged unions via `enum` types such as `Result<T, E>`, Rust forces the programmer to explicitly unpack success values or inspect errors, eliminating entire classes of unhandled edge cases. He also highlights Rust's hygienic macros, which operate directly on the abstract syntax tree rather than performing blind C-preprocessor text substitution. Cantrill jokes about his C background, admitting he never even knew how to spell the word "hygienic" until he sat down to write a blog post about it two months prior.

Foreign Function Interface (FFI) capabilities are another critical differentiator, particularly when contrasted with garbage-collected runtimes. In GC environments, sharing live objects back and forth across a C boundary requires cumbersome pinning, marshalling, or manual lifetime tracking. Rust, by contrast, supports what Cantrill calls "full duplex" integration: C can call directly into Rust libraries, and Rust can seamlessly consume existing C interfaces without a runtime intermediary. For low-level edge cases where compiler safety rules become overly restrictive, the `unsafe` keyword provides a scoped escape hatch, allowing engineers to construct safe abstractions around raw hardware or memory structures without sacrificing the safety of surrounding code.

- Rust’s algebraic data types turn error handling into a compiler-enforced pattern matching exercise rather than easily ignored integer return codes. - First-class AST macros allow safe, powerful metaprogramming that avoids the accidental variable capture bugs endemic to C preprocessor macros. - Rust's zero-cost FFI enables drop-in bi-directional interoperability between C and Rust without bridge layers or garbage collector overhead. - “The foreign function interface in Rust is very well thought out, very full, and supports I I kind of invented this term. I'm sure there's a better one out there. Full duplex integration in that you can take C and drop it into Rust or Rust and drop it into C.”

[48:19](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2899s) The Culture and Values of the Rust Community

A technology's success in systems engineering is inextricably linked to the people maintaining it, and Cantrill highlights the Rust community as a genuine asset. Having spent decades participating in open-source development, he confesses that community interactions often induce a "throbbing headache" when sharp ideological divisions stall technical progress. While Rust cannot escape the universal friction of internet discourse, its community culture is anchored by clearly articulated shared values that keep discussions constructive and grounded.

Cantrill specifically calls out community leaders present in the audience (including Steve Klabnik) and notes that the project has sustained a rare cultural coherence. The community focuses on ergonomics, mechanical sympathy, and rigorous correctness rather than factional infighting. This culture gives developers confidence that the language will evolve sustainably rather than fracturing under competing corporate agendas.

- Systems software communities often suffer from vitriolic bike-shedding, making Rust's constructive governance style stand out. - The community's explicit focus on stability, tooling, and ergonomics helps anchor debates when technical trade-offs arise. - “I don't necessarily know that it's any less, you know, it's that it's any less difficult in Rust, but I think that the the values of Rust tend to drive conversations. That there there is it feels like there's a keel in the water.”

[49:14](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2954s) The Operating System Reality: Runtime Trumps Development Time

Turning to the core question of rewriting the operating system, Cantrill lays down an unyielding historical law: an operating system's runtime performance, correctness, and latency characteristics completely overshadow how pleasant or difficult it was to write. End users do not care how long developers spent wrestling with manual memory management, nor do they care if a language made engineers happy; they care solely whether the kernel runs at wire speed without panicking or dropping data. Structured languages like C only displaced assembly because optimizing compilers proved they could match raw assembly performance while delivering superior maintainability.

Because runtime characteristics reign supreme, any viable replacement for C in the kernel must meet the same zero-overhead bar. Rust is the first language in over thirty years to emerge as a realistic candidate because it matches C’s operational profile without relying on a managed runtime. Cantrill reminds the audience that every operating system still houses assembly for bootstrapping and context switching, but Rust gives systems architects an opportunity to write the rest of the deep stack with modern guarantees.

- Operating systems succeed or fail entirely on execution speed, deterministic latency, and rock-solid reliability. - Developer ergonomics cannot justify an operating system rewrite if the resulting kernel incurs runtime overhead or unpredictable pauses. - “Nobody cares how long it took you to develop it. And that it was easy to develop or hard to develop. It's like, no one cares. Um they care if it works and they care how fast it is.”

[50:32](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3032s) Early Kernel Experiments and the Allocation Failure Dilemma

Examining the pioneering attempts to construct kernels in Rust, Cantrill references Alex Light’s undergraduate thesis, which ported the Brown University pedagogical operating system Weenix to Rust. Light’s work laid bare a fundamental mismatch in early Rust: the standard library assumed that memory allocation could never fail. When an allocation failed in userland, early Rust simply aborted or panicked. In user applications, failing fast on out-of-memory (OOM) conditions is often acceptable, but an operating system kernel cannot simply give up and die.

```mermaid flowchart TD accTitle: Kernel Memory Allocation Strategies accDescr: Flowchart illustrating how a kernel handles sleeping versus non-sleeping memory allocations.

Req["Kernel Memory Allocation Request"] --> Check{"Can the caller sleep?"}

Check -->|"KM_SLEEP (Common case)"| Wait["Block/sleep until memory subsystem reclaims pages"] Wait --> Success["Allocation succeeds; execution resumes"]

Check -->|"KM_NOSLEEP (Interrupt/atomic context)"| TryAlloc{"Memory available right now?"} TryAlloc -->|"Yes"| Success TryAlloc -->|"No"| Fail["Return NULL/error; caller handles failure path"] ```

A kernel is the ultimate arbiter of system resources; when memory runs low, the kernel cannot abort—it must take active measures to reclaim pages, flush caches, or wait for resources to free up. Cantrill contrasts different OS philosophies: while Linux uses an aggressive OOM killer to reap running processes when pressured, operating systems like illumos and SmartOS are engineered to avoid running out of memory entirely. Kernel allocators distinguish between "sleeping allocations" (which block until memory becomes available and therefore cannot fail) and "non-sleeping allocations" (invoked in interrupt contexts where blocking is prohibited, requiring explicit failure handling). Because early Rust lacked fine-grained mechanisms for non-sleeping, fallible memory allocations, early OS authors found themselves fighting the language's core runtime assumptions.

- Early Rust assumed infallible heap allocations, immediately terminating execution on out-of-memory conditions. - Kernels cannot panic on allocation failure because they are responsible for paging, cache eviction, and managing the physical RAM itself. - Production operating systems bifurcate memory allocations into sleeping calls (which safely wait for memory to be reclaimed) and non-sleeping calls (which must return an explicit failure to avoid deadlocking interrupt handlers). - “In the operating system, we need to be able to know uh if this allocation is not going to work and if we are out of memory, I need to go do some work to free up memory.”

[52:10](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3130s) Allocation Failure and Rust RFCs

Handling memory allocation failure cleanly remains an essential requirement for operating system kernels, which must be able to accommodate out-of-memory conditions without abruptly terminating. In Rust, dealing with allocation failure has historically been tricky, though the landscape improved dramatically around Rust 1.26 or 1.27 with the stabilization of global allocators. Even so, handling allocation failures gracefully without panicking remains an unsettled, active area of design within the language community. Cantrill encourages anyone serious about Rust systems development to dive into the official Request for Comments (RFC) repository. Because much of the language and library ecosystem is relatively new, developers inevitably consult RFC discussions to understand the reasoning behind language decisions.

Cantrill finds these design debates not only informative but uplifting. As an example of community problem-solving, he points to the discussion surrounding raw string literals—specified as `r#"..."#` using an `r` followed by matching sequences of octothorpes, pound signs, or hashtags. Cantrill jokes about swearing he would never call the symbol a hashtag until his kids started using the term for everything, but notes how reading that RFC thread revealed thoughtful comparisons of how various systems tackle raw strings. In languages like C and JavaScript, the absence of proper raw string literals creates persistent friction, making Rust’s elegant syntax a genuine pleasure. While global allocators are largely settled, Cantrill notes that the exact semantics for out-of-memory handling in kernel environments are still being hammered out through these RFCs.

- Global allocators landed around Rust 1.26–1.28, unlocking custom allocation strategies for bare-metal targets. - Memory allocation failure recovery remains an open, active RFC topic in the Rust community. - Rust RFC design debates provide transparent rationale on foundational features, from allocator traits to syntax details like raw string literals.

[53:47](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3227s) The De Novo Rust OS Explosion

Since 2015, the systems software landscape has witnessed an explosion of small, experimental operating systems written in Rust. Cantrill lists several active projects that still compile, including Redox (the largest general-purpose microkernel project), Tifflin, Tock (focused on embedded IoT microcontrollers), Intermezzos, RustOS, and QuiltOS (a unikernel targeting Rust applications). He also highlights Philipp Oppermann’s renowned tutorial series, *Writing an OS in Rust*, affectionately pleading with Oppermann to give the project a formal moniker rather than leaving everyone to refer to it as "Philipp Oppermann's Blog OS." Cantrill stresses: “Philipp Oppermann, please give this thing a name.”

The common denominator across almost all these initiatives is that they are *de novo* efforts written completely from scratch. Cantrill praises these projects as terrific testbeds for language exploration, educational experiments, and teaching operating systems fundamentals. However, starting completely from scratch imposes immediate, massive operational trade-offs for production computing.

```mermaid flowchart TD accTitle: De Novo Rust Operating Systems Landscape accDescr: Categories of clean-slate Rust operating system projects emerging since 2015.

subgraph Projects["De Novo Rust OS Landscape"] direction TB General["Redox<br/>(General-purpose microkernel)"] Embedded["Tock<br/>(Embedded IoT & microcontrollers)"] Unikernel["QuiltOS<br/>(Rust-only unikernel)"] Teaching["Intermezzos &amp; Blog OS<br/>(Educational &amp; pedagogical kernels)"] end

General --> Tradeoffs["Trade-offs:<br/>No Linux ABI compatibility &amp; Second-system syndrome"] Embedded --> Tradeoffs Unikernel --> Tradeoffs Teaching --> Tradeoffs ```

- **Redox**: The most prominent and comprehensive general-purpose Rust microkernel effort. - **Tock**: Designed for secure, multi-tenant execution on low-power IoT microcontrollers. - **QuiltOS**: A unikernel architecture restricted exclusively to running Rust binaries. - **Intermezzos & Philipp Oppermann’s "Blog OS"**: Widely respected educational and teaching kernels built step-by-step in Rust.

[54:53](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3293s) The Pitfalls of Rewriting the Kernel from Scratch

Building an operating system entirely from scratch means forsaking Linux binary compatibility, rendering the vast majority of existing global software unusable. While Cantrill respects clean-slate purism, he insists that running existing software matters: “I like having the world's software work.” Emulating the Linux system call surface after the fact entails an agonizingly long, brutal tail of gray areas and corner cases. Drawing on his experience supporting Linux emulation in SmartOS—as well as talks by Justin Cormack and Aidan Hobson Sayers—Cantrill warns against falling into semantic quagmires like trying to reconcile `vfork` and `SIGCHLD` semantics, which he characterizes as an "unspeakable semantic Superfund site." Furthermore, de novo operating systems inevitably succumb to second-system syndrome: once engineers resolve to build a new kernel, they also resolve to invent a new file system, a new device driver architecture, and new process models, quickly stalling project momentum.

More fundamentally, the core Rust promise of memory safety fails to motivate kernel developers to discard battle-tested C code. In production kernels, memory bugs blow up immediately and violently, which has pushed teams to build robust internal tooling and invariants over decades. Because kernel internals rely on complex, multiply owned data structures that sit simultaneously across numerous linked lists, queues, and dispatch tables, Rust's strict ownership model struggles without liberal use of `unsafe`. As Cantrill puts it, “every time you use an unsafe in your Rust operating system, it's like you're losing a little bit of the advantage of Rust.” Finally, the thought of rewriting mission-critical subsystems like ZFS terrifies him: “I wake up in a cold sweat when I think about rewriting ZFS in Rust and go to like go look at the ZIO pipeline and tell me you want to rewrite that thing in Rust. It's just like, no thank you. I mean, the Rust compiler would slay me.”

- Emulating Linux binary compatibility requires handling deeply convoluted edge cases, such as the interaction between `vfork` and `SIGCHLD`. - Second-system syndrome frequently derails clean-slate OS projects by attempting to redesign device drivers, filesystems, and process scheduling all at once. - Operating system kernels are saturated with non-hierarchical, multiply owned data structures that fight Rust's borrow checker and demand widespread `unsafe` blocks.

[57:53](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3473s) Hybrid Approaches: In-Kernel Rust and Userland Components

Given the drawbacks of clean-slate rewrites, Cantrill argues strongly for a hybrid architectural approach. The first hybrid approach involves introducing Rust components directly inside existing C- and assembly-based operating system kernels. Rust’s zero-cost interoperability with C allows teams to preserve their battle-tested core while writing new drivers, virtual file systems, or experimental subsystems in Rust. Rather than attempting to rewrite working C code, this model provides a memory-safe sandbox that lowers the barrier to entry for new developers. FreeBSD has already explored prototype implementations of this concept, and Cantrill intends to bring Rust into the SmartOS (illumos) kernel down the line.

The second hybrid approach targets the operating system outside of the kernel. An OS is far more than ring 0: it encompasses daemons, background utilities, debuggers, device management frameworks, and service supervision systems like systemd or illumos’s Service Management Facility (SMF). Cantrill argues that if systemd or SMF were being engineered from scratch today, they should unquestionably be built in Rust. He challenges the widespread assumption that kernel engineering represents the pinnacle of difficulty, revealing an open secret among systems programmers: “It's harder to write user level software because more can go wrong.” Kernels operate in an environment with deterministic, tightly constrained assumptions where an administrator cannot simply delete a configuration file out from under the running program, whereas userland systems services must grapple with corrupt files, missing configurations, and distributed systems failures.

- **Hybrid Model 1 (In-kernel Rust)**: - Preserves the established C kernel foundation. - Enables incremental development of safe device drivers and file systems. - Pioneered in FreeBSD kernel module experiments and targeted for SmartOS. - **Hybrid Model 2 (Userland OS Infrastructure)**: - Focuses Rust adoption on critical userland daemons, debuggers, device management, and service supervisors. - A modern rewrite of `systemd` or illumos SMF would be prime candidates for implementation in Rust. - **Kernel vs. Userland Complexity**: - Kernels do not suffer from missing configuration files: if the kernel binary is missing, the system simply cannot boot. - Complex user-level services must continuously defend against corrupted configs, asynchronous filesystem mutations, and distributed failure modes.

[01:00:03](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3603s) Operating System Frontiers: The Cloud Control Plane and Firmware

Bryan Cantrill details how modern systems can avoid operational failures by leveraging the exact safety guarantees that Rust excels at providing. As software definitions evolve, the operating system is no longer confined to traditional kernel boundaries; it now expands outward to encompass entire cloud control planes. Beyond that, Cantrill introduces an area he is especially passionate about: an obscure, neglected operating system environment running underneath all modern computing that has stagnated technologically.

Cantrill pulls no punches when describing this tier of mission-critical software: “And that is the this absolute sewer of unobservable system critical, mission critical software called firmware.” While modern systems have evolved, firmware essentially remains glorified DOS, quietly powering the foundations of society while plagued with catastrophic flaws. Cantrill highlights the existential urgency of the issue, remarking, “As I'm fond of saying, like it is it is humanity versus firmware, and everyone needs to pick a side.”

```mermaid flowchart TD accTitle: Firmware and Kernel Rewrites in Rust accDescr: Diagram contrasting traditional kernel rewrite challenges with firmware Rust adoption.

subgraph Kernel["Traditional Kernel Challenges"] K1["Complex Multiple Ownership"] K2["Strict Binary Compatibility"] K3["Reinvent Massive Subsystems (e.g. ZFS)"] end

subgraph Firmware["Firmware / OpenBMC Domain"] F1["Simpler Ownership Models"] F2["Network/Socket Exposure Risks"] F3["High Safety Requirements (No C++)"] end

Kernel -.->|"Too Intractable"| Rust["Rust Systems Implementation"] Firmware ==>|"Ideal High-Impact Target"| Rust ```

The embedded and engineering hurdles of firmware are far more surmountable for Rust than rewriting a monolithic OS kernel. In firmware, developers do not face the messy multiple-ownership headaches found in an OS kernel, there are no stringent binary compatibility burdens to uphold, and nobody has to reinvent massive subsystems like ZFS. Pointing specifically to OpenBMC, Cantrill characterizes it as desperate for rescue from vulnerable C++ implementations: “I view OpenBMC as on its knees with tears streaming down its face begging for someone to please rewrite me in Rust.” If a baseboard management controller (BMC) must expose network sockets to the world, it must be authored in a memory-safe language rather than C++. Transitioning proprietary low-level systems into open firmware built on Rust stands as an imperative step toward securing global digital infrastructure.

- Cloud control planes and service fabrics represent clear targets for Rust adoption because avoiding complex operational failures is critical. - Firmware runs beneath the rest of software, yet remains largely unobservable, dangerous, and technologically archaic. - Unlike full operating system kernels, firmware projects are compact enough to make complete Rust implementations technically feasible. - Replacing C++ in network-exposed firmware like OpenBMC eliminates entire classes of remote memory-safety vulnerabilities.

---

[01:01:58](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3718s) The Hybrid Future of System Software and Kernel Extensions

Looking ahead, Cantrill expresses intense optimism about Rust’s trajectory across systems programming. The industry has spent decades waiting for a viable alternative capable of operating deep down the software stack. While practically rebuilding legacy operating systems from scratch remains nearly impossible due to historical momentum and complexity, Rust offers an ideal medium for extending kernels and creating new subsystems.

Its greatest operational advantage is seamless interoperability with legacy C-based systems. Cantrill notes, “The beauty of Rust is its ability to to interoperate, cooperate with native systems, with C-based systems allow for vistas to open up.” Instead of mandating clean-slate rewrites, Rust empowers developers to take pragmatic, hybrid approaches—grafting safe, performant components directly into existing platforms. For system engineers who have long lacked modern tooling at the lowest layer of the stack, this flexibility heralds an exhilarating new era.

---

[01:02:46](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3766s) Bargaining with the Borrow Checker: Advice for New Rustaceans

Opening the floor to questions, Cantrill responds humorously to an audience query about wrestling with the compiler. He insists that he harbors no hatred for the borrow checker: “I have only tried to bargain with the borrow checker. I have never—even when it is absolutely thrashing me, I know it is out of its own love for what it's trying to do.” For anyone learning the language, getting past that initial frustration is the defining milestone. He strongly cautions programmers against picking traditional starter projects when first experimenting with Rust: “The one thing I would advise if you have not implemented Rust, please, for the love of God, don't start with a doubly linked list.”

Trying to tackle a doubly linked list or porting familiar hobby routines like a Sudoku solver will rapidly lead to misery before one can appreciate what the borrow checker offers. A beginner will often enter an escalating battle with compiler diagnostics, where following suggested fixes—such as adding `mut` or switching to references—ends up cascading into dozens of new, baffling compilation errors until tears start welling up.

```mermaid flowchart LR accTitle: The Beginner Pitfall vs. Recommended Learning Path in Rust accDescr: Flow comparing the frustrating toy project path with the recommended book-driven typing path.

subgraph BadPath["The Frustrating Path"] B1["Write Linked List / Solver"] --> B2["Compiler Suggests 'mut'"] B2 --> B3["Compiler Suggests Reference"] B3 --> B4["Cascading Errors & Despair"] end

subgraph GoodPath["The Recommended Path"] G1["'The Rust Programming Language' (TRPL)"] G2["'Programming Rust' (Blandy & Orendorff)"] G3["Type Out Fully Worked Examples"] G1 & G2 --> G3 G3 --> G4["Gently Internalize Ownership Model"] end ```

Cantrill recommends buying and working through two foundational books side by side: 1. *The Rust Programming Language* (often called "TRPL") by Steve Klabnik, Carol Nichols, and the Rust community. 2. *Programming Rust* (the O'Reilly book by Jim Blandy and Jason Orendorff).

Cantrill shares how working through a fully worked example from *Programming Rust* mirrored his childhood experience of manually typing BASIC programs from the back pages of *Enter* magazine into a Commodore 64. Hand-typing code and fixing gentle syntax mistakes allows developers to observe the logic of ownership in an environment guaranteed to work, cementing why the compiler acts as a strict guardian rather than an adversary. Cantrill acknowledges that older, experienced engineers who are accustomed to “winging it” by just banging out code will find that approach fails in Rust; they must deliberately slow down, retreat to a quiet room, and study ownership concepts carefully before writing code.

---

[01:05:43](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3943s) Compiler Maturity, Compile Times, and Non-Lexical Lifetimes

An audience member asks about compiler parity: referencing Cantrill's earlier historical remark about Multics struggling for three years without a functioning PL/I compiler, the questioner asks whether the Rust compiler and optimizer have reached the mature performance and optimization level of GCC and Clang for C.

Cantrill acknowledges that he arrived late to Rust and thus avoided the early growing pains that drove users away years prior. Benefiting from recent milestones like incremental compilation, he finds Rust's build performance and generated binaries ready for enterprise demands: “To me, I would not hesitate to use it for production software.”

Asked whether engineering teams should wait another year before adopting Rust, Cantrill agrees that waiting offers tangible advantages as documentation expands, the ecosystem stabilizes, and language ergonomics improve. The primary upcoming milestone is non-lexical lifetimes (NLL). Under the older lexical lifetime rules, the borrow checker rejected valid code because it tied borrows strictly to enclosing lexical scopes, ignoring the fact that a variable was no longer being used. Cantrill points out how developers found themselves desperately arguing with the compiler: “There are times where like, 'No, seriously, Rust, I swear I'm done with this. Like, look—like, look at the code with me, Rust compiler. Like, we're done with this.'” With non-lexical lifetimes landing—promising to integrate by the December 6th release—the compiler understands dataflow rather than raw syntax scopes, resolving the borrow checker's most infuriating friction points and forever lowering the barrier for new programmers.

[01:08:24](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=4104s) The Longevity of Rust and the Evolution of the Borrow Checker

Bryan Cantrill reflects on the rapid improvements coming to the Rust compiler and developer experience—particularly around the borrow checker—noting that upcoming changes will make such an immense difference that future newcomers will barely understand what earlier developers struggled with. Developers joining the ecosystem will take the smoother ergonomics for granted, never quite grasping what all the early frustration was about. While pointing out that Rust consistently produces exceptionally high-quality software artifacts, Cantrill emphasizes that engineers should not feel an urgent pressure to jump in prematurely out of fear of missing out.

Instead, he offers a sweeping historical perspective on the language's permanence, comparing its arrival to a foundational civilizational milestone. “I honestly think Rust is going to be around forever. I really do. I think this is like you this is the formation of ancient Greek,” Cantrill observes, drawing laughs by telling the room: “So, I mean, there's no rush. You got thousands of years.” While the language and tooling will continue to mature, Cantrill concludes that for those who have been holding off because the ecosystem previously felt too turbulent or difficult, the wait is effectively over—there is no need to delay adopting Rust any longer. With that final thought, he thanks the host and concludes the presentation.

- Cantrill predicts that ergonomic improvements to the borrow checker will make such a massive difference that future programmers will not understand what earlier adopters found so difficult. - He characterizes Rust as a permanent fixture in computing history, comparing its emergence to the formation of ancient Greek and reassuring the audience that they have ample time to learn it. - For practitioners who delayed adoption waiting for the language and tooling to stabilize, Cantrill advises that they no longer need to wait.

## Notable quotes > [37:31](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2251s) Even if you and I both know how to write memory safe C, it's very hard for us to have an interface boundary where we can agree about who does what. — Bryan Cantrill

> [28:05](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=1685s) For system software, for the operating system, for that infrastructure, I think GC is a real problem. — Bryan Cantrill

> [57:53](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3473s) It's harder to write user level software because more can go wrong. — Bryan Cantrill

> [01:00:03](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3603s) I view OpenBMC as on its knees with tears streaming down its face begging for someone to please rewrite me in Rust. — Bryan Cantrill

> [01:08:24](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=4104s) I honestly think Rust is going to be around forever. — Bryan Cantrill

> The one thing I would advise if you have not implemented Rust, please, for the love of God, don't start with a doubly linked list. — Bryan Cantrill

## Mentioned Aidan Hobson Sayers, ALGOL, Alex Light, Apple Copland, B, BCPL, BeOS, Bell Labs, Bob Morris, Bryan Cantrill, Burroughs B5000, C, C++, Carol Nichols, Clang, Cloudera, Commodore 64, Cray, CTSS, Dennis Ritchie, DTrace, ESPOL, Enter, Fernando Corbató, FreeBSD, GCC, General Electric, Gil Amelio, Go, Graydon Hoare, Haiku, Intermezzos, Jason Orendorff, JavaOS, JavaScript, Jean Sammet, Jeff Hammerbacher, Jim Blandy, Justin Cormack, Ken Thompson, Linux, Master Control Program (MCP), Mike Demmer, Mike Olson, Multics, Node.js, OpenBMC, Philipp Oppermann, PL/I, Programming Rust, QuiltOS, Redox, Robert Tappan Morris, Roger Faulkner, Rust, RustOS, Service Management Facility (SMF), SmartOS, Steve Klabnik, Sun Spring, Taligent, Tera, The Rust Programming Language, Tifflin, TMG, Tock, Weenix, ZFS, gVisor, illumos, systemd

## Video description QCon San Francisco, the international software conference, returns November 17-21, 2025. Join senior software practitioners from early adopter companies as they share real-world insights and actionable advice to help you adopt the right technologies and practices.

Gain exposure to innovative approaches and fresh ideas in software development and engineering, designed to inspire and challenge you.

Don’t miss this chance to deepen your knowledge, sharpen your skills, and stay ahead in the ever-evolving world of technology.

Register now: https://bit.ly/3BYzfbe ---------------------------------------------------------------------------------------------------------------- Video with transcript included: https://bit.ly/2Nq7RW5

Bryan Cantrill explores Rust, explains why it has captured the imagination of so many systems software engineers, and outlines where it might best fit in the deep stack of operating system software.

This presentation was recorded at QCon San Francisco 2018: https://bit.ly/2uYyHLb

For more awesome presentations on innovator and early adopter topics check InfoQ’s selection of talks from conferences worldwide https://bit.ly/2tm9loz

#Rust #OperatingSystems #SystemProgramming #InfoQ #QConSanFrancisco

[Source](https://www.youtube.com/watch?v=HgtRAbE1nBM)

中文简报

是用Rust重写操作系统的时机了吗? Bryan Cantrill从系统编程历史的视角出发,回顾了Multics、Unix、C以及Sun Spring和Copland等失败的面向对象重写尝试的脉络,评估了是否应当用Rust重写操作系统。尽管从零全新重写内核往往会沦为第二系统综合征的受害者,并丧失关键的历史二进制兼容性,但Rust的编译时所有权模型、零运行时开销以及全双工C FFI,使其成为数十年来首个有望取代C的可行语言。Cantrill反对对内核进行全盘重写,转而主张采取务实的混合方案,即将Rust集成到驱动程序、虚拟文件系统、复杂的用户态系统守护进程,以及历史上脆弱的固件环境(如OpenBMC)中。 用Rust从零开始重写整个操作系统可能重蹈历史上第二系统失败的覆辙,并破坏与现有软件的关键二进制兼容性。 (54:53) Rust在无需垃圾回收器或运行时开销的情况下提供了内存安全和高级抽象,使其成为三十年来系统编程领域首个可行的C语言竞争者。 (49:14) Rust在系统基础设施中的切近机会在于混合现代化:替换OpenBMC等存在安全漏洞的固件、重写用户态控制守护进程,以及添加内核态驱动程序。 (60:03) 将C语言算法直接移植到Rust往往会失败,因为循环指针图和侵入式指针图与借用检查器严格的单所有权模型存在直接冲突。 (41:00)

Reading summary

Systems engineer Bryan Cantrill argues that while rewriting complete operating system kernels in Rust from scratch is an impractical trap, the language is primed to transform systems software through hybrid kernel extensions, userland services, and critical firmware. The Failures of Second-System Rewrites (8:38) History shows that ground-up rewrites in unproven or overly complex paradigms reliably fall victim to second-system syndrome. Multics stalled for years and fractured its consortium by relying on an unimplemented PL/I specification without a working compiler, whereas Unix succeeded through pragmatic, bottom-up design in C. The 1990s witnessed similar over-engineering catastrophes when object-oriented rewrites like Sun's Spring, Apple's Copland, Taligent, and JavaOS collapsed under heavy abstractions and excessive memory demands. Rust's Zero-Cost Safety and Modern Performance (34:39) Operating systems prioritize runtime efficiency, low latency, and deterministic execution over developer ergonomics. While modern garbage-collected runtimes like Node.js and Go introduce fatal stop-the-world and memory overhead hurdles for core systems, Rust delivers compile-time memory safety without a runtime collector. Rust also benefits from modern hardware dynamics: while traditional C systems rely on intrusive AVL trees that cause expensive CPU cache misses, Rust's standard B-tree implementations group keys into contiguous nodes, enabling safe data structures to outperform hand-tuned C code on modern memory hierarchies. The Intractability of Clean-Slate Kernel Rewrites (54:53) De novo Rust operating systems such as Redox, Tock, and QuiltOS serve as valuable educational testbeds, but rewriting established general-purpose kernels from scratch poses immense real-world barriers. Starting clean forfeits Linux binary compatibility, forcing teams into complex emulation edge cases like vfork semantics. Furthermore, production kernels depend heavily on intrusive, circular, and multiply-owned data structures that resist Rust's single-ownership model, compelling developers to rely on unsafe blocks that negate the language's safety advantages while attempting to duplicate decades-old subsystems like ZFS. The Hybrid Path: In-Kernel, Userland, and Firmware (57:53) Rather than full rewrites, Cantrill advocates for targeted hybrid adoption enabled by Rust's zero-cost, duplex Foreign Function Interface with C. Teams can preserve core kernel foundations while safely implementing isolated device drivers, file systems, and system supervisors like systemd or SMF in Rust. Outside the kernel, firmware represents the most critical frontier: legacy firmware remains unobservable and insecure, yet its execution environments feature simpler ownership models and higher network exposure, making it an ideal candidate for Rust implementations. Cantrill concludes that Rust has matured sufficiently for production infrastructure, bolstered by stabilizing features like global allocators and non-lexical lifetimes. However, handling memory allocation failures gracefully without panicking in low-memory kernel contexts remains an ongoing design challenge within the Rust RFC process.

精炼阅读摘要

Bryan Cantrill 认为,虽然将整个操作系统从零开始用 Rust 完全重写并不现实,但 Rust 具备与 C 语言相当的零运行时开销和卓越的编译期内存安全特性,是数十年来首次出现的系统级替代方案;他主张应采取渐进混合架构,将 Rust 应用于内核扩展、系统用户态服务以及固件等关键前沿领域。 历史警示:第二系统综合征与自顶向下的陷阱 (8:38) Cantrill 回顾了系统软件发展史,指出完全从零重写系统的工程往往容易陷入“第二系统综合征”。20世纪60年代的 Multics 项目试图采用尚未实现其实体编译器的 PL/I 语言,并采用死板的自顶向下瀑布式设计,结果因外包编译器开发失败延误数年,导致贝尔实验室在1969年退出该联盟。随后,Ken Thompson 和 Dennis Ritchie 在 PDP 硬件上以自底向上的有机方式构建了 Unix,并由 Unix 催生出 C 语言。C 语言在1973年引入结构体,完全是由于内核实现的现实工程需求所驱动。 到了20世纪90年代,面向对象浪潮引发了一系列雄心勃勃的重写项目,如苹果的 Copland、Sun 的 Spring、IBM 与苹果合资的 Taligent 以及 JavaOS。这些项目大多沦为过度抽象的第二系统牺牲品。例如 Sun 的 Spring 内核由于内存消耗过度,在当时高配的 32MB 内存机器上因频繁寻道而无法正常工作,开发人员甚至要求 128MB 内存。这表明脱离实际约束的大规模重写极易导致项目崩溃。 C语言的局限与现代运行时困境 (28:05) 在20世纪80年代后,C 语言成为生产级操作系统的绝对基石。尽管 C 语言常被称为“可移植汇编”,但系统开发人员在面对现代业务时,急需哈希表、字符串处理和正则表达式等现代化功能。然而,随后的技术路径各有难以克服的瓶颈:现代 C++ 带来了沉重的维护创伤与历史负担;而 Node.js 与 Go 等现代环境虽然开发速度快,却引入了垃圾回收机制(GC)。 对于操作系统核心层而言,垃圾回收机制具有不可调和的架构阻碍。Cantrill 指出,垃圾回收导致的性能停顿和无法显式控制的内存占用,破坏了系统内核对确定性延迟和硬件资源控制的要求。然而,纯 C 语言在跨模块和跨团队协作时,难以在接口边界保证内存所有权契约的一致性,系统极易因侵入式数据结构和人为疏忽出现内存安全漏洞。 Rust的核心契约:无GC的安全与现代硬件亲和性 (34:39) Rust 打破了系统级语言的僵局,其核心契约是通过编译期的所有权模型和借用检查器,在不引入垃圾回收器的情况下实现内存安全和手写代码级别的执行性能。虽然 Rust 强制要求单所有权,使得移植 C 语言常见的双向链表或环形引用变得极具挑战,但在适应其约束后,Rust 能在现代硬件上展现出更佳的性能表现。 Cantrill 以重写生产级节点的经历为例,他在 C 语言中习惯使用侵入式 AVL 树以避免额外内存分配,但由于现代 CPU 体系结构更注重缓存行利用率,AVL 树在离散节点指针寻道时缓存命中率极低。相比之下,Rust 标准库中的 BTreeSet 与 BTreeMap 采用连续内存块布局,天然契合现代缓存层级。结合无需运行时代价的双向 C 语言 FFI 互操作性、代数数据类型(Result/enum)和安全的 AST 卫生宏,Rust 为底层开发提供了现代工程工具。 内核重写的现实壁垒与从零构建的缺陷 (54:53) 在探讨是否要从零重写操作系统内核(如 Redox、Tock 等项目)时,Cantrill 指出完全从零开始存在巨大的实际缺陷。最关键的障碍在于丢失 Linux 二进制兼容性,这会导致全球现有庞大软件生态无法运行;而若要在全新内核上模拟 Linux 系统调用接口,将陷入极为繁杂的边缘语义泥潭(如 vfork 与 SIGCHLD 的复杂语义)。 此外,内核性能和确定性永远压倒开发阶段的人体工程学,最终用户只在乎系统是否能在高速运行时不丢包、不崩溃。现有成熟生产级内核中的复杂子系统(如 ZFS 及其 ZIO 管线)凝聚了数十年的工程防御机制与不变量,内部充斥着跨越多张链表和调度表的复杂多重所有权结构。若强行使用 Rust 重写,往往需要大量引入 unsafe 逃逸机制,从而削弱了 Rust 本身的安全优势。同时,内核内存分配失败必须得到显式恢复,而早期的 Rust 运行时甚至默认在分配失败时直接 panic,相关的错误恢复机制仍在 RFC 阶段持续演进。 务实的演进路径:内核混合、用户态与固件攻坚 (57:53) Cantrill 提出的可行演进路径是务实的混合架构模式。首先是在现有 C 语言内核中引入 Rust 模块(如 FreeBSD 的内核探索及 SmartOS 计划),利用 Rust 零成本 FFI 构建内存安全的设备驱动程序和虚拟文件系统,逐步降低新开发者的准入门槛并隔离风险。其次是将重点转向广义操作系统中的用户态核心服务,例如服务监督系统(如 systemd 或 SMF)、守护进程和调试工具;Cantrill 强调,编写需要处理不可控外部输入、损坏配置文件和分布式故障的用户态系统软件,在工程上往往比假定确定性硬件环境的内核更具挑战性。 另一个亟待攻克的高价值前沿是固件与云控制平面。Cantrill 将当前绝大多数服务器固件批评为缺乏可观测性、如同处于 DOS 时代的“绝对死水”,而固件直接暴露于网络攻击之下且涉及关键任务。固件环境的数据所有权模型相比传统宏内核更加简单清晰,对内存安全有着极高要求,是 Rust 能够避开多重所有权陷阱并立即产生重大安全变革的最佳落地战场。 Cantrill 总结指出,Rust 在编译期正确性与底层控制力上的平衡使其成为计算机系统领域的基石性里程碑。随着非词法作用域生命周期(NLL)等编译器人体工程学的成熟,开发者与借用检查器的摩擦大幅降低。尽管推倒并从零重写传统操作系统不可行,但采用混合架构、将 Rust 注入固件、驱动及控制平面的时机已经完全成熟。

Detailed notes — zh

## Background 本内容由技术大会频道 InfoQ 于 2019 年 2 月 25 日在 YouTube 平台发布,录制于 2018 年旧金山 QCon 大会。根据页面介绍,资深系统工程师 Bryan Cantrill 深入探讨了 Rust 为何能激发众多系统软件工程师的想象力,并详细分析了它在深层操作系统技术栈中最适合落地的位置。 ## TL;DR Bryan Cantrill 针对“是否该用 Rust 重写操作系统”给出了有条件的肯定回答,但他明确反对盲目从零推翻现有成熟内核(如 Linux 或 ZFS)。通过回顾从 Multics、Unix、C 语言到 90 年代面向对象狂热的系统演进史,他指出了垃圾回收机制在底层系统中的致命缺陷,并盛赞 Rust 在无 GC 条件下通过所有权模型与优秀组合性达成的内存安全。Cantrill 主张采取务实的混合演进策略:不要盲目追求重写成熟内核中复杂的多重所有权数据结构,而是应优先将 Rust 应用于内核扩展模块、复杂的系统级用户态守护进程(如服务管理器)以及亟待现代化的底层固件(如 OpenBMC)。 ## Atmosphere & style 整场演讲充满激情、机智幽默且极具感染力,带有 Bryan Cantrill 标志性的脱口秀式犀利叙事风格。他擅长将宏大的计算机发展史、荒诞的技术轶事与硬核的工程剖析融为一体,时而无情自嘲,时而对行业盲从展开辛辣讽刺。现场气氛轻松活跃而引人深思,观众既能体会到系统老兵面对复杂软件工程时的敬畏与审慎,也能强烈感受到他对 Rust 这门划时代语言由衷的狂热与历史性赞誉。 ## Bulletin - [00:04](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=4s) Bryan Cantrill 针对“是否该用 Rust 重写操作系统”给出了“有条件的肯定”:全面推翻重写并不现实且面临严重兼容性与第二系统综合征风险,但 Rust 在内核扩展、系统级用户态服务及开源固件中极具颠覆性价值。 - [28:05](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=1685s) 系统软件的核心铁律是运行时性能与正确性永远压倒开发速度,带垃圾回收(GC)的运行时因不可控的尾延迟和对象图残留,无法胜任严苛的操作系统与内核级基础设施开发。 - [43:25](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2605s) Rust 的所有权模型不仅带来了媲美手动管理的零运行时开销内存安全,其强类型和高可组合性更打破了 C 语言难以安全组合复杂数据结构(如 B 树替代 AVL 树)的底层困局。 - [01:00:03](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3603s) 底层的服务器固件(如 OpenBMC)长期停留在高脆弱且不可观测的类似“美化版 DOS”状态,因其网络暴露面和相对清晰的所有权关系,成为当下亟须用 Rust 重写的最关键领域。 ## Key takeaways - [02:21](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=141s) 操作系统的本质是一个抽象底层硬件(物理或虚拟)以运行其他程序的“超级程序”,且决定了其上所有进程的生死存续。 - [08:38](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=518s) 重写底层系统极易跌入“第二系统综合征”陷阱,Multics 因盲目追求未实现的 PL/I 规范与极端瀑布流开发导致数年无编译器可用并最终瓦解。 - [14:26](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=866s) Unix 秉持自底向上的实用主义哲学诞生,它催生了兼顾汇编级底层控制与跨平台便携性的“可移植汇编”——C 语言。 - [22:21](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=1341s) 20 世纪 90 年代面向对象工业复合体推行的 C++ 和 Java 重写 OS 浪潮(如 Copland、Spring、Taligent、JavaOS)因过度抽象与资源暴涨遭遇惨败。 - [28:05](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=1685s) 现代高级语言追求开发速度,但垃圾回收机制(GC)因无法避免的停顿延迟及将 GC 退化为堆扫描器的闭包引用,成为系统级软件的致命死穴。 - [35:51](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2151s) Rust 的所有权(Ownership)机制在编译期静态界定内存生命周期,达成垃圾回收语言级别的安全性与手动内存管理极致性能的统一。 - [39:03](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2343s) C 语言在模块协作时因边界职责不明极难实现高层组合;而 Rust 编译器强制契约,使复杂的抽象原语与现代高效数据结构得以无缝组合。 - [43:25](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2605s) 在系统编程中,Rust 的 B 树由于匹配现代 CPU 缓存行局部性在基准测试中超越了 C 语言惯用的 AVL 树,展现了高组合性带来的实际性能红利。 - [45:56](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2756s) 代数数据类型、卫生宏、全双工 FFI 以及受控的 unsafe 逃生舱,共同构成了 Rust 支撑底层工业级软件的核心能力基石。 - [49:14](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2954s) 系统内核的核心铁律是运行时正确性与吞吐性能永远高于开发耗时,早期 Rust 内核受限于内存分配失败即直接 Panic 的缺陷而难以胜任自愈调度。 - [53:47](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3227s) 从零从头重写操作系统(De Novo)面临丧失 Linux 二进制兼容性与陷入多重重构瘫痪的巨大工程风险。 - [56:11](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3371s) 成熟内核充斥着双向链表等密集的多重归属(Multiply Owned)数据结构,用单所有权的 Rust 硬套容易产生大量破坏安全保证的 unsafe 块。 - [57:53](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3473s) 务实的渐进路线一是在现有成熟 C 内核中通过 FFI 引入 Rust 编写的新驱动、文件系统及沙盒化功能模块。 - [58:57](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3537s) 务实的渐进路线二是用 Rust 重新构建用户态系统核心服务(如 systemd、SMF 及设备监控器),其错综复杂的错误处理与状态边界最能释放 Rust 的威力。 - [01:00:03](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3603s) 长期滞后的服务器固件基础设施(如 OpenBMC)直接暴露网络攻击面,所有权结构清晰,是目前最迫切、最适合用 Rust 彻底重写的领域。 - [01:05:43](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3943s) 非词法作用域生命周期(NLL)等编译器进化大幅削平了借用检查器的使用门槛,Rust 已经成熟到足以在底层生产环境中广泛落地的历史阶段。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=4s) 用 Rust 重写操作系统的提问与贝特里奇定律 演讲开场提出了核心问题:现在是否是用 Rust 重写操作系统的时候了?针对这个问题,Bryan Cantrill 提前给出了剧透。他提到了著名的“贝特里奇标题定律”(Betteridge's law of headlines),即任何以问号结尾的新闻标题,其答案通常都可以合理地归结为“不”。但他明确表示,听众今天不会从他这里得到一个简单的“不”,而是会得到一连串“有条件的肯定”(qualified yeses)。他强调标题之所以留有问号而非感叹号,背后存在着诸多复杂的考量。 接着,Cantrill 指出,即便身处操作系统的专业分论坛,关于“操作系统到底是什么”的界定依然模糊不清,甚至连他自己都不是当天第一个(甚至不是第二个)专门用一页幻灯片来探讨这一定义的演讲者。这看似不可思议,毕竟操作系统已经存在了这么多年,业界却连它究竟是什么都难以给出一个简洁公认的描述。Cantrill 直言:“定义操作系统远比想象中困难,就如同试图给‘什么是软件’下一个滴水不漏的定义一样。” ```mermaid flowchart LR accTitle: 核心命题与贝特里奇定律 accDescr: 展示演讲核心提问及其回答逻辑,从问题到贝特里奇定律,再到讲者的实际结论。 Q["核心问题:<br/>是否该用 Rust 重写 OS?"] --> B["贝特里奇定律预测:<br/>'不'"] Q --> A["Bryan Cantrill 的答案:<br/>'有条件的肯定 (Qualified Yeses)'"] A --> S["保持问号:<br/>多重制约与技术深思"] ``` ### [00:51](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=51s) 从“什么是软件”引发的存在主义危机 为了说明概念界定的困难,Cantrill 用“什么是软件”展开思想实验。如果有人给出一个定义,他就能轻易用后续问题把对方绕进死胡同,比如:“欧几里得的最大公约数(GCD)算法算不算软件?欧几里得写出 GCD 时是在写软件吗?”如果回答“不算,因为那时根本没有硬件”,那么随后的定义就很难自洽。 Cantrill 分享了一段在 Cloudera 成立初期的真实轶事。当时 Cloudera 办公室里只有大约六个人,创始人 Mike Olson 邀请 Cantrill 前去拜访。那天 Cantrill 状态有些异样,脑子里一直在死磕“到底什么是软件”这个哲学命题。当有人提出“软件就是在硬件上运行的东西”时,Cantrill 追问:“那什么是硬件?”随着这个概念被层层剥离拆解,整个讨论开始崩塌。 Cantrill 调侃说,当时办公室所有人距离陷入“存在主义危机”只剩 10 分钟:大家突然发现自己一无所知,意识到古苏美尔人其实也在运行软件,因为“软件仅仅是我们运行的知识,它甚至不必是图灵完备的”。眼看着像 Jeff Hammerbacher 这样早期核心成员都开始盯着自己的双手发呆、怀疑人生是否有意义时,Mike Olson 不得不把 Cantrill 拉到一旁私聊。 - Mike Olson 当时十分困惑且紧张地问他:“你怎么了?你是吸毒了吗?你现在嗨着呢吗?” - Bryan Cantrill 回应道:“不,Mike,我只是在想,软件到底是什么?” - Mike 只能无奈地求他换个话题或者干脆离开,以免整个团队陷入虚无。 - Bryan Cantrill 总结道:“定义操作系统也面临同样的困境:若定义为‘抽象硬件的程序’,那在 AWS 实例里运行的 Guest OS 抽象的只是虚拟硬件,难道它就不是操作系统了吗?” ### [02:21](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=141s) 操作系统的本质:抽象硬件以运行其他程序的“超级程序” 摆脱无休止的语义漂移后,Cantrill 给出了适合本场讨论的实用定义:操作系统是一个程序,它通过抽象硬件(无论是物理硬件还是虚拟硬件)来允许其他程序在其上执行。他强调,操作系统的本质仍旧只是一个程序,一种“超级程序”(super program)。 为了支持这一观点,Cantrill 提到了现场的好友 Mike Demmer。Demmer 曾在一家名为 Tera 的公司从事内核开发(Tera 后来收购了超级计算机巨头 Cray,反过来更名为 Cray,Cantrill 戏称这种被收购方反客为主保留名称的做法通常意味着收购方的品牌知名度不太行)。在做了几年内核开发后,Demmer 和 Cantrill 交流时感叹:“你知道吗,我意识到操作系统其实就是一个程序。它只是个复杂的程序,一个异步的、中断驱动的程序。”起初 Cantrill 还有点被冒犯,心想:“我可不只是普通程序员,我是内核程序员!别剥夺我的尊严。”但冷静下来后,他承认事实确实如此。 对于操作系统的边界,Cantrill 强调了两个关键准则: 1. **运行其他程序是底线**:如果一个程序不能运行其他程序,它即使对社会和人类再有价值,也不应被称为操作系统。 2. **定义所运行程序的生命周期(Liveness)**:操作系统的生死决定了其上所有进程的生死;操作系统一旦崩溃,其上运行的一切都会随之覆亡。 Cantrill 特别提醒,切勿将操作系统(OS)与操作系统内核(Kernel)混为一谈。内核仅仅是在微处理器的特权上下文(例如 x86 的 Ring 0)中运行的那部分组件。由于现代虚拟化技术的介入,Ring 0 的概念也变得极其复杂——嵌套虚拟化可能使代码运行在距离真实物理硬件好几个层级的虚拟 Ring 0 中。但操作系统的完整版图不仅包括内核,还包含系统库(Libraries)、命令(Commands)、守护进程(Daemons)以及各类系统基础设施(Facilities)。即使像 Justin Cormack 在之前的演讲中所主张的那样把系统过度臃肿的部分剥离掉(Justin 曾打趣地将膨胀归咎于 Sun Microsystems),那些不运行在最高特权级但属于操作系统的组件依然广泛存在。 > “The operating system is the program, because it is a program, that abstracts the hardware to allow for the execution of other programs.” —— Bryan Cantrill ```mermaid flowchart TD accTitle: 操作系统的层级与组成架构 accDescr: 展示操作系统在硬件/虚拟硬件之上的层次结构,区分特权级内核与非特权级系统组件。 subgraph HW["硬件抽象层 (Hardware / Virtual HW)"] PHW["物理或虚拟微处理器<br/>(Physical / Virtual Rings)"] end subgraph OS["完整操作系统 (Operating System)"] K["操作系统内核 (Kernel)<br/>运行于特权级 (Ring 0 / 虚拟 Ring 0)"] subgraph UserLand["系统级用户空间组件"] LIB["系统库 (Libraries)"] CMD["核心命令 (Commands)"] DAE["守护进程 (Daemons)"] FAC["系统基础设施 (Facilities)"] end end subgraph Apps["上层应用程序"] P1["应用进程 A"] P2["应用进程 B"] end PHW --> K K --> UserLand OS -->|"决定生命周期 (Liveness) & 提供运行环境"| Apps ``` ### [05:45](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=345s) 早期操作系统的历史演进与 Burroughs B5000 Cantrill 声明,听他的演讲绝对少不了历史课,谈论操作系统必须拨回历史的指针。早期操作系统其实被称为“执行程序”(Executives),这个命名带有鲜明的二战后层级化管理体制色彩,仿佛整个系统是一个严密的行政管理架构。后来,这一称谓才逐步演变为“操作系统”。 在早期,所有的系统软件都是用纯汇编语言或机器码手写的。计算机历史上一个极为亮眼的里程碑是 1961 年诞生的 Burroughs B5000 计算机。该机器搭载的操作系统被称为 MCP(Master Control Program,主控程序)。这正是电影《电子世界争霸战》(Tron)中同名反派程序名字的灵感来源(Cantrill 顺便吐槽 Tron 是一部奇差无比的烂片,建议大家去看 1983 年的《战争游戏》(WarGames)补课)。 Burroughs B5000 及其 MCP 的先驱性在于: - **高级语言编写系统**:MCP 并非使用汇编,而是用一种基于 ALGOL 的衍生语言 ESPOL 编写。ESPOL 常让人联想到世界语(Esperanto),或者厄瓜多尔的一所同名大学。 - **超越时代的架构**:Burroughs B5000 在多个维度上大幅领先于时代。但 Cantrill 感叹,作为一个多次在技术上领先于时代并因此承受不幸的技术人员,“领先于时代在商业上往往没有任何好果子吃”(除了能在 QCon 这种会议上无需自我介绍之外,毫无实际商业收益)。Burroughs 是一家极富创新精神的公司,造就了令人惊叹的机器,但最终走向了破产消亡。 紧接着在 1964 年,麻省理工学院等机构开始为兼容分时系统(CTSS)研发后续替代方案。团队抱着“这次一定要把所有事情都做对”的心态展开设计,从而跌入了软件工程史上臭名昭著的“第二系统综合征”(Second System Syndrome)。Cantrill 直言,每当他怀疑“第二系统综合征”是不是《人月神话》那代人虚构的 60 年代陈旧概念时,现实中总会冒出新的第二系统狠狠敲打他。 > “And I can tell you as a technologist who has had the misfortune of being ahead of my time several times over, being ahead of your time is not commercially fruitful.” —— Bryan Cantrill ### [08:38](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=518s) 第二系统综合征与 Multics 的编译器困境 重写系统往往会落入经典陷阱:人们总想借机重构一切,不仅要重写这里,还要重新设计那里、换掉底层数据库,力求“把事情彻底做对”。结果系统变得庞大臃肿,发布日期一推再推,最终根本无法交付。这就是典型的“第二系统综合征(Second System Syndrome)”。 麻省理工学院的 Project MAC 在开发 Multics 时,遭遇了历史上最为惨烈的第二系统综合征。Multics 团队希望能用高级语言来编写系统,虽然有传言称当时可选的语言只有 Fortran 和 COBOL,但早期计算机科学家 Jean Sammet 在 1963 年出版的名著《Programming Languages: History and Fundamentals》却展示了完全不同的图景:那时的编程语言极其繁盛,甚至多到了泛滥的地步,宛如富氧环境下巨型蕨类植物丛生的中生代。尽管选择繁多,Multics 团队却一个也看不上,执意选择了一种甚至尚未实现的全新语言规范——PL/I(注意这里的“I”是罗马数字,不是阿拉伯数字 1)。 他们写完了极其完备的 Multics 规格说明,却连一行能运行的代码甚至配套的编译器都没有。随后团队做出了一个令人匪夷所思的决定:将人类历史上首个 PL/I 编译器的开发工作外包给洛杉矶一家名不见经传的小团队。一年后 Multics 人员飞去视察进度,对方却两手一摊表示完全不知道自己在干什么、根本做不出来。 > “So this is like ultimate waterfall, like nightmare waterfall.” —— Bryan Cantrill - 规格说明详尽完备,但因缺乏编译器,根本无法编写实际代码。 - PL/I 编译器直到 1966 年才勉强交付,项目陷入了极端的瀑布流泥潭。 ```mermaid flowchart TD accTitle: Multics 编译器的瀑布流困境 accDescr: 展示 Multics 项目从过高规格到外包编译器失败并引发延误的过程 subgraph DesignPhase ["设计阶段"] A["拒绝现有语言 (Fortran/COBOL 等)"] --> B["选用未实现的新规范: PL/I"] B --> C["详尽完成 Multics 规范书"] end subgraph CompilerFiasco ["编译器外包危机"] C --> D["外包给洛杉矶未知团队"] D -->|"开发失败 / 无法交付"| E["长达数年缺乏可用编译器"] end subgraph Aftermath ["项目受挫"] E --> F["直至 1966 年才获得编译器"] F --> G["严重延误, 联盟瓦解"] end ``` ### [11:16](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=676s) 联盟瓦解与 Unix 的自底向上诞生 在计算机历史上,Multics 是个极具争议的项目:有人认为它是一场未被给予充分赞誉的巨大成功,也有人认为它是一场彻头彻尾的灾难。1971 年,Fernando Corbató 撰写了《Multics — The First Seven Years》,将 PL/I 描述为 Multics 的一大亮点,却对团队三年无编译器可用的尴尬事实轻描淡写。围绕早期应急开发的 EPL(Early PL/I)方言,历史记载也呈现出“罗生门”式的撕裂:EPL 的作者坚称自己拯救了世界,而 Corbató 却在文章中声称那套方言从未被真正使用、微不足道。 长期的编译器缺失加速了领导 Multics 的“三头同盟(Triumvirate)”走向瓦解。贝尔实验室(Bell Labs)认定该项目已不可救药,于 1969 年正式撤出。离开 Multics 项目的贝尔实验室研究员 Ken Thompson 随后转战 PDP-7,着手编写一个全新的操作系统,后来又将其移植到了 PDP-11/20 上。为了调侃 Multics 的繁复臃肿,这个新系统被戏谑地命名为 Unix。 > “And Multics was very top-down command and control, write the specification, maybe we'll get a compiler, maybe we won't kind of software. Unix was like, ‘Ah, let's just get something working and then I don't know, get some mouse working and then we'll write a man page for whatever we already got working.’” —— Bryan Cantrill Unix 代表了一种与 Multics 截然相反的哲学:它是自底向上(Bottom-up)、实用主义的产物,契合了 20 世纪 60 年代末至 70 年代初的社会与技术思潮。先让系统跑起来,再去补写 man page 文档。 - 贝尔实验室于 1969 年撤出 Multics 联合项目。 - Ken Thompson 先在 PDP-7 上实现 Unix 内核,随后移植到 PDP-11/20。 - Unix 名字本身就是对 Multics 复杂性的文字戏仿。 ### [14:26](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=866s) 汇编、B 语言与“可移植汇编” C 语言的诞生 关于 Unix 的开发历程存在一个广泛的误解,即许多人误以为 Unix 一开始就是用 C 语言编写的。事实上,最初在 PDP-7 上的 Unix 完全是用纯汇编语言写成的,随后移植到 PDP-11 上的版本依然是汇编。当时贝尔实验室确实存在源自 BCPL 的解释型语言——B 语言。 B 语言与 Multics 有着微妙的渊源:B 语言和前面提到的 EPL 方言都是基于 TMG(Transmogrifier)转译器构建的。TMG 由 Bob Morris(Robert Morris Sr.,即著名的莫里斯蠕虫作者之父)实现。然而,B 语言在 Unix 中仅处于极度边缘的辅助地位,仅用于构建汇编器以及早期的 `dc` 命令(就是当你想敲 `cd` 却经常手误敲出的那个没有退出提示、吞掉 Ctrl+C 的桌面计算器)。 B 语言的性能极其低下,而且完全缺乏字节寻址能力(Byte Addressability)。Dennis Ritchie 与 Ken Thompson 深刻意识到了用高级语言构建操作系统的巨大潜力,但 B 语言显然难以堪此重任。为了克服这些致命缺陷,他们基于 B 语言开发了一门全新语言——C 语言。 > “Unix begets C in a very important, I guess, biblical way. They they're siblings. They're not they're not twins. And Unix is very much the older sibling.” —— Bryan Cantrill C 语言并非与 Unix 同时诞生的“双胞胎”,而是 Unix 孕育出的产物,Unix 是更年长的兄长。尽管常有人贬称 C 语言不过是“可移植的汇编(Portable Assembly)”,但在那个年代,能够兼顾汇编级别的底层控制与跨硬件的便携性,正是 C 语言超越时代、极其伟大的核心优势。 - 初始阶段的 Unix 完全采用汇编编写,经历了 PDP-7 到 PDP-11 的汇编移植。 - B 语言是解释型语言,缺少字节寻址能力且性能低,无法支撑操作系统内核开发。 - Unix 催生了 C 语言的诞生,使其成为兼顾执行效率与系统抽象的“可移植汇编”。 ### [17:27](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=1047s) 从大写时代到现代 C 语言的有机演进与设计缺失 对比早期编程语言,转向 C 语言就像是瞬间迈入了现代世界。在 C 语言之前,几乎所有编程语言都像是在全大写地“尖叫”,仿佛当年的键盘连 Caps Lock 键都坏掉了一样。Bryan Cantrill 分享了他收藏老旧编程手册的趣事:他曾从英国买来一整箱杂七杂八的旧手册,其中包含一本鲜为人知的“Language H”手册。这是一种由 NCR Elliott 推出的 COBOL 衍生语言,甚至把“Sterling”(英镑)作为语言关键字。滑稽的是,Language H 的每个程序都必须以“Chapter 1”(第一章)开头,而且根本不存在“Chapter 2”;程序更以指令关键字“Obey”(服从)结束,完全没有后来通用的“End”或“Epilogue”,仿佛在向计算机下达军令。 相比之下,C 语言的革命是自底向上且有机演进的。如同当年 Unix 摒弃 Multics 自上而下的指令与控制方式一样,C 语言的功能完全是按需生长的。最初的 C 语言没有结构体(`struct`),直到 1973 年为了用 C 语言重写 Unix 内核才引入了结构体;位字段(bit fields)也是为了满足内核需求才加入的。然而,这种按需迭代也带来了弊端,Bryan 强调了令他非常恼火的一点——C 语言没有逻辑异或运算符(logical XOR)。在 C 语言中,既有按位与/或/异或(`&`、`|`、`^`),也有逻辑与和逻辑或(`&&`、`||`),偏偏遗漏了逻辑异或。当开发者需要断言“两个条件中有且仅有一个为真”时,往往只能使用按位异或来 hack,并在代码旁留下注释。为此,Bryan 曾向与 Dennis Ritchie 共事过的 Roger Faulkner 抱怨,Faulkner 辩称逻辑异或不存在是因为它无法实现短路求值。Bryan 反驳称并没有哪条天条规定逻辑运算必须短路,但 Faulkner 随后向 Dennis Ritchie 本人求证,Ritchie 确实回复道:“No, no, we don't have logical XOR because it can't short circuit and you should just use bitwise.” 虽然事实证明 Bryan 猜错了原委,但这生动反映了 C 语言设计的随意性与局限。 - C 语言最初并无结构体支持,1973 年为了重构 Unix 内核才紧急补全。 - Dennis Ritchie 确认 C 语言未引入逻辑异或(`^^` 之类)的核心技术原因在于其在语义上无法进行短路求值(Short-circuit evaluation)。 - C 语言具有自底向上(Bottom-up)的设计哲学,语言特性的增加高度绑定具体工程实践中暴露的即时需求。 --- ### [20:50](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=1250s) 宏预处理器的功过与 80 年代 C 语言的主导地位 C 语言有机生长的另一个关键产物是宏预处理器(Macro Preprocessor)。宏在 C 语言中至关重要,也是日后 Bryan Cantrill 深入 Rust 时反复回味反思的特性。尽管现代软件工程教育总是告诫初学者“千万不要使用宏预处理器”,但这就像教育年轻人远离烟酒与不良嗜好一样——虽然危险,但它所带来的能力和灵活性却是不可或缺的。无论是在 DTrace 还是 ZFS 等大型系统级软件的实现中,C 语言的宏预处理器都发挥了决定性的基石作用,尽管其机制本身也确实存在诸多严重缺陷。C 语言的标准化进程不仅姗姗来迟,而且充满了各方利益的博弈与争议。 贯穿整个 20 世纪 80 年代,C 语言在系统编程领域确立了绝对的主导地位。除了少数例外(如初代 Mac 操作系统使用了 Pascal 编写,或者某些仍在汇编上运行的 MS-DOS 系统及研究型 OS 之外),绝大多数主流生产级操作系统均完全采用 C 语言编写,C 语言与操作系统的系统栈达成了极佳的工程契合度。 ```mermaid flowchart TD accTitle: 20世纪80年代操作系统开发语言格局 accDescr: 展示80年代C语言如何在生产级操作系统中占据绝对统治地位,而Pascal和汇编仅占少数分支 subgraph Dominant["主流生产级系统栈"] C["C 语言\n(有机生长、强宏支持、高度贴合硬件)"] UnixOS["Unix 及主流生产级 OS"] C --> UnixOS end subgraph Exceptions["少数派与遗留系统"] Pascal["Pascal 语言"] --> Mac["初代 Macintosh"] Asm["纯汇编语言"] --> DOS["MS-DOS 等早期 PC 系统"] end ``` --- ### [22:21](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=1341s) 90 年代面向对象狂热、“第二系统综合征”与各大尝试的溃败 进入 20 世纪 90 年代,Bryan Cantrill 将这段时期形容为“一片漆黑降临大地的黑暗时代”。当时“面向对象工业复合体(Object-Oriented Industrial Complex)”甚嚣尘上,行业内充斥着盲目的教条,认定一切系统都必须面向对象化。基于 C 语言构建的系统被贬低为“行尸走肉(the walking dead)”和行将就木的遗产垃圾,而软件架构师们则宣称所有底层软件必将被基于 C++ 或 Java 的优美系统全面取代。然而,这场面向对象重构操作系统的浪潮最终以全军覆没收场。其核心败因在于这些项目全都陷入了典型的“第二系统综合征(Second-system syndrome)”,盲目堆砌过度设计与抽象垃圾。 Bryan 盘点了 90 年代几个代表性的失败案例: 首先是苹果公司的 Copland 操作系统。在 Gil Amelio 执掌苹果时期,由于 Copland 迟迟无法交付,硅谷其他公司甚至只是盯上了苹果的地产,苹果一度深陷生存危机。其次是 Sun Microsystems 使用纯 C++ 打造的 Spring 操作系统。Bryan 回忆他在大学本科时拿到过 Spring 的内部测试光盘,运行起来后系统一直在疯狂寻道、震动硬盘主轴(“rattling the spindle”)。当询问系统为何如此卡顿时,开发人员询问内存有多大;得知是当时的高配 32 MB DRAM 后,对方居然回答必须要有 128 MB 才能顺畅运行——这相当于在当代要求一台开发机拥有 2 TB 内存。Bryan 评价道:“To give you don't want to give your OS developers so much memory because they'll use it all and then it can't be used on a system that's got less memory.” 另一个著名的溃败项目是 IBM 与 Apple 联手打造的 Taligent。Taligent 的名字取自“Talented(天才的)”与“Intelligent(聪慧的)”的混成词,Bryan 调侃道:“As it turns out, they like nailed the worst of IBM meeting the worst of Apple in an effort that was a total disaster.” 这群自命不凡的顶级工程师躲在象牙塔中闭门造车,试图自顶向下交出一款无所不包的操作系统,结果就像吸尘贴一样吸满了可怕的抽象垃圾。与此同时,Sun 内部推行的 JavaOS 也彻底碰壁。Java 甚至没有原生无符号整数类型(unsigned type),这使得它直接与底层硬件寄存器打交道变得极其痛苦且不切实际。这些面向对象与虚拟机狂想在 90 年代末接连破灭,直到 Linux 凭借对实用主义和 x86 架构的聚焦正式崛起。 - **Apple Copland**:苹果旨在取代 System 7 的现代面向对象微内核系统,最终因延期与架构失控被彻底腰斩。 - **Sun Spring**:纯 C++ 编写的分布式对象操作系统,因极端消耗 DRAM 与存储资源而无法走向实用。 - **Taligent**:IBM 与 Apple 强强联手打造的面向对象操作系统公司,集两家企业官僚与过度设计之大成,最终耗资巨大却未成气候。 - **JavaOS**:缺乏硬件交互所需的无符号整型(unsigned types),在底层操作系统领域无法立足。 ### [25:57](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=1557s) C 语言内核的固化与“开发人员速度”时代的兴起 随着 Unix 系统的强劲复兴,基于 C 语言的系统根基变得无比深厚。在此期间虽然出现过一些采用 C++ 编写内核的尝试,但基本上都逐渐萎缩甚至夭折了。尽管存在像 Haiku 这样用 C++ 编写系统的罕见例外,但 Bryan Cantrill 调侃道,当你必须把 Haiku 当作最严肃的例证时,恰恰反向证明了 C++ 内核的普遍凋零——Haiku 本身十分优秀,但他更像是一个旨在模拟 BeOS 的爱好者操作系统。结果就是 C++ 内核在系统领域几乎全军覆没,操作系统底层几乎一律被 C 语言牢牢统治。 与此同时,软件行业经历了一段极其单一化的霸权时代,市场一度被 Java 和 C++ 笼罩,尤其是处处充斥着 Java。随后,Ruby 和 Python 相继登场。JavaScript 的发展历程则更为戏剧化:它起初只被当作在网页上实现闪烁标签(`<blink>`)之类花哨效果的玩具,经历过一段低谷;直到 2006 年前后,包括 Bryan Cantrill 在内的许多工程师猛然醒悟:浏览器端居然自带一门现成的编程语言,而且为了构建新型 Web 应用,开发者必须使用它。当人们真正深入了解它时才惊讶地发现,这绝非单纯的脚本玩具,而是一个“伪装在浏览器里、去掉了括号的 Lisp”(而 Cantrill 坦言自己向来反感 Lisp 那无穷无尽的括号,因此对 JavaScript 颇有好感)。 这些新兴的高级语言将核心全部押注在开发者体验与开发节奏上。尽管 Cantrill 极其厌恶“开发者速度(developer velocity)”这类陈词滥调,甚至直言“我实在没办法把‘开发者速度’写进 PPT 里,因为那会让我恶心想吐”,但他也不得不承认,整个行业当时的确都在疯狂追求让工程师写代码变得更快。台下的听众(如 Steve)开玩笑提到大学时期搞过的“Ruby 操作系统”或“Python 操作系统”,Cantrill 打趣那是年少轻狂时“符合年龄犯下的错误”,私下写写可以理解,但现实中绝不可能有人严肃地把 Ruby 或 Python 用来做真正的操作系统。 ### [28:05](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=1685s) 系统软件的困局:Node.js、Go 与垃圾回收(GC)的致命伤 步入 2010 年代后,系统程序员群体开始陷入焦虑与求索。他们既渴望 C 语言极致的执行性能,又在体验过高级动态语言后,由衷怀念哈希表、Map、字符串分割(split)以及正则表达式这些强大的高层抽象表达能力。在浏览器大战的推动下,以 V8 为代表的高性能引擎横空出世,促成了 Node.js 的爆发,甚至让服务端运行 JavaScript 成为现实。然而 Cantrill 坦言,这在系统底层依然难以长久,因为 JavaScript 的设计哲学就是包容一切混乱,以至于他尖锐地给出了定论:“JavaScript 就是编程语言界的失败国家(failed state)。” ```mermaid flowchart TD accTitle: 系统级软件对语言特性的诉求与现实冲突 accDescr: 展示高级语言带来的便利与垃圾回收机制在系统编程中引发的性能与延迟冲突。 subgraph HighLevel["现代高级语言的诱惑"] A["高级抽象数据结构<br>(Map, Hash, Regex)"] B["快速开发与原型落地能力"] end subgraph RuntimeCost["运行时代价 (Node.js / Go)"] C["垃圾回收机制 (GC) 介入"] D["复杂对象图与闭包导致堆引用残留"] end subgraph SystemsProblem["底层系统软件的死穴"] E["不可预测的尾延迟 (Latency)"] F["缺乏对内存布局与生命周期的强制思考"] G["无法满足内核级严苛性能要求"] end HighLevel --> RuntimeCost RuntimeCost --> SystemsProblem ``` 随后,源自贝尔实验室团队(Cantrill 称之为“贝尔实验室难民”,并强调这是最高的赞誉而非贬低)在 Google 开发出了 Go 语言。Go 确实解决了许多工程痛点,并在基础设施领域被广泛使用(例如 Aiden 曾精彩分享过的 gVisor 项目,展现了 Go 的潜力与隐患)。但无论是 Go 还是 Node.js,它们用于操作系统底层时都会撞上一堵无法逾越的高墙——垃圾回收(Garbage Collection, GC)。 Cantrill 详细剖析了系统软件为何与 GC 格格不入: - **错怪 GC 引擎**:开发者总是习惯抱怨“GC 毁了我的进程性能”,但真正的症结在于开发者自己在疯狂制造垃圾;GC 并非带着恶意针对谁,如果你停止制造海量废弃对象,GC 根本无需频繁介入。 - **内存泄漏与堆扫描器**:很多人误以为有 GC 就不会发生内存泄漏,然而复杂的对象图中,任何一个无意中残留的闭包引用都会导致整个庞大的对象树无法释放。此时 GC 根本找不到垃圾,它实质上退化成了一个极其耗费资源的“堆扫描器(heap scanner)”。 - **缺乏确定性与前期深思**:GC 带来的偶发性停顿和延迟是内核与底层基础设施无法承受的;更深层的原因在于,GC 免去了程序员在编码前期对内存拓扑与资源归属进行深入推敲的必要。Cantrill 强调自己并不反对 GC 在其他领域的价值,但对于底层系统级与性能关键型软件,“GC 是一个真正的严重问题”,他绝不可能选择带 GC 的语言来编写关键系统。 ### [32:20](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=1940s) 彻底告别 C++ 与对 Rust 的严肃探寻 在经历了与 Node.js 系统的“痛苦分手”后,Cantrill 面临着极为尴尬的技术选型困境。重回 C++ 是绝无可能的:他在大学时期曾硬生生写过 10 万行 C++ 代码,那段沉重的记忆让他彻底祛魅。面对外界“现代 C++ 已经彻底改头换面”的劝说,Cantrill 幽默而决绝地予以回绝:“那对 C++ 挺好的,我也祝愿 C++ 能过上幸福美满且富有成效的生活,但 C++ 曾经把我所有的家当拖到大街上付之一炬。所以我绝不会再回头用 C++,这无关道德评判,纯粹是为了自我保全。” 排除掉带有 GC 的运行时与 C++ 之后,Cantrill 开始把目光投向 Rust。他现场调查了与会观众对 Rust 的认知、向往与实操情况,发现台下许多人都和他此前的状态完全一样——处于“对 Rust 严重好奇(Rust curious)”的阶段:整天刷相关的博客技术帖,内心被其构想深深吸引,却又下意识地去代码细节里找毛病(比如挑剔代码格式或空格规范,借此掩饰内心的抗拒)。 但 Rust 所展示的核心价值观完全击中了 Cantrill 这样的系统老兵: - **不妥协的核心追求**:任何 Rust 官方资料都会开宗明义地强调:这是一门面向系统软件、运行速度极快且专为安全(Memory Safety)而设计的语言。 - **构建不朽软件的野心**:Cantrill 坦言自己追求编写能够“永久流传、寿命超过自己”的软件。他甚至提到,自己以前写过的某些底层软件(如 DTrace 等系统核心组件)绝对会在他死后依然在世间长久运转,而 Rust 所倡导的安全、零运行时开销与严谨性,恰恰是打造长久存续的底层软件最需要的特质。 ### [34:39](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2079s) 软件的永恒性与 Rust 追求持久的抱负 Bryan Cantrill 指出,某些现有软件之所以在经济上根本不值得重写,仅仅是因为它们运转良好。软件的美妙与令人惊叹之处在于,就像两千年前欧几里得的辗转相除法(GCD 算法)一样,当一段软件真正起作用时,它就能永远解决这个问题。大家往往习惯聚焦于软件千疮百孔、世界一团糟的阴暗面,但从本质上看,运作正常的软件就像一台永恒运转的数学机器。Cantrill 感慨现代软件开发者生活在一个前所未有的黄金时代,后代回望这段历史时,必将惊叹这是所有软件奠基写就的辉煌时期。然而,并非所有软件都能永世长存,许多系统终究需要使用能够真正经受住时间考验的技术来重写。 在 Cantrill 眼中,Rust 正是怀抱这种“永久存续”(in perpetuity)抱负的产物。尽管 Rust 的创造者 Graydon Hoare 给出了关于命名由来的诸多版本,但 Cantrill 最钟情的一个解释带有浓厚的工业质感:建造能够长久存续的事物,哪怕面对最严苛的自然侵蚀,氧气引起的化学锈蚀才是唯一的敌人。 > “And what I see from Rust is that aspiration for if in perpetuity.” —— Bryan Cantrill ### [35:51](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2151s) 所有权模型:编译器与程序员的宏大契约 Rust 最引人入胜且独一无二的特性是其静态“所有权”(Ownership)机制。编译器能够在编译期静态推导出谁拥有什么内存、当前的控制流线程何时拥有这些内存、以及何时不再拥有。当代码不再需要某块内存时,编译器便自动将其回收——Rust 语言中根本没有显式的 `free` 函数。Cantrill 形容这犹如编译器与开发者签下的一份宏大契约。 编译器向程序员承诺提供不必操心手动释放的超能力,但作为交换,程序员必须在编译器无法判定内存去向或所有权归属时提供协助。这套借用(Borrow)机制依赖词法生命周期(Lexical Lifetimes)来推导上下文,而一旦编译器被绕晕,就会抛出极其晦涩的报错信息,让开发者感觉像是在遭受刁难与折磨。但这层折磨的背后,换取的是既具备垃圾回收语言般的内存安全与开发体验,又享有手动内存管理极速性能的终极能力。 ```mermaid flowchart TD accTitle: Rust 所有权与借用检查契约 accDescr: 展示 Rust 编译器在编译期间如何通过借用规则换取自动内存管理与零开销性能 subgraph CompilerContract ["编译期契约 (Borrow Checker)"] A["分配内存 / 绑定所有者"] --> B{"内存流向分析"} B -->|"单所有者持有"| C["超出作用域自动释放 (无显式 free)"] B -->|"借用 / 引用传递"| D["借用生命周期追踪"] D -->|"生命周期冲突 / 悬垂"| E["报错拦截: 强制开发者明确所有权"] D -->|"生命周期合规"| C end subgraph RuntimeBenefits ["运行时收益"] C --> F["GC 语言级别的内存安全"] C --> G["C 语言级别的手动内存性能"] end ``` Cantrill 调侃道,Rust 编译器由于 Borrow Checker 喜怒无常而恶名昭彰,因此它在另一面表现出极端的“过度补偿”——它成了史上最友善的编译器。它会用漂亮的彩色 ASCII 字符精确指出错误位置,并温柔提示“你可能想写的是这个”,仿佛在讨好开发者:“请务必记住我现在有多热心,因为你最黑暗的绝望时刻即将来临。” > “I’m going to give you the power of a garbage collected language with the performance of manual memory management.” —— Bryan Cantrill - **严格的结构体初始化**:Rust 要求初始化结构体时必须为每个字段赋值,杜绝任何未初始化脏数据。 - **友善与严厉并存**:编译器的 diagnostic 极具辅助性,试图缓解类型推导和生命周期检查给程序员带来的心智打击。 ### [39:03](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2343s) 为什么 C 语言极难组合,而 Rust 实现了真正组合 Cantrill 澄清,尽管许多人推崇 Rust 是为了防止内存越界、错误的 `bzero` 或 `memcpy` 大小,但对他个人而言,纯粹的内存安全并不是转向 Rust 的最大动力。他坦言自己历经磨砺,完全有能力编写出正确、不泄露、不发生内存损坏的 C 代码(尽管算术溢出依然极难防范)。C 语言的核心痛点在于:即使一个人能写出完美的 C 代码,那也是因为他在自己的代码天地里掌控着一切;一旦涉及跨模块接口边界,C 代码就极其难以“组合”(Compose)。 当多位优秀的程序员协作或模块拼接时,接口契约往往充满歧义:在特定错误分支下,句柄到底是由调用方释放还是被调用方释放?极其容易因沟通偏差造成内存漏洞。C 语言社区通常采用“侵入式数据结构”(Intrusive Data Structures)来实现组合,将链表或树节点直接嵌入外部宿主结构中,但这套方案在 Rust 的所有权体系下极难推行。Rust 的颠覆性在于,Borrow Checker 从编译器层面强制保证了谁拥有什么、谁借用什么的契约被隐式恪守,从而让构建极其复杂的抽象原语并实现无缝组合成为可能。 > “Because even if you and I both know how to write memory safe C, it’s very hard for us to have an interface boundary where we can agree about who does what.” —— Bryan Cantrill ### [41:21](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2481s) 从 C 移植到 Rust 的思维转变与双向链表陷阱 Cantrill 分享了他将一套由 C 和 Node.js 构建的实际系统改用 Rust 重写的经历。在博客上对比两者的性能时,他曾特意写了八大段文字强调这只是特定场景的测试,不可推导出“Rust 永远比 C 快”的通用结论,但网络喷子依然第一时间指责他狂妄。在实际重写过程中,Cantrill 最初被原有的 C 语言思维牢牢束缚,试图用 C 的模式去生搬硬套 Rust。 在 C 语言中,多重所有权(Multiple Ownership)比比皆是。例如双向链表(Doubly Linked List),一个节点同时被前驱和后继节点所“拥有”,这在 C 中稀松平常,但在 Rust 中却寸步难行。社区甚至有专门的名篇讨论在 Rust 中实现双向链表的痛苦。Rust 强迫开发者重新思考问题本身,以一种所有权极其清晰明了的方式重新设计数据流与结构。只要理顺了所有权,Rust 便能回馈以极致的运行时性能。 在经历与 Borrow Checker 反复较劲、陷入绝望的“讨价还价”阶段后,Cantrill 放弃了在 Rust 里还原 C 语言复杂高性能模式的执念,转而采用最直白、最朴素甚至看似“笨拙”的方式先把逻辑跑通。他原以为这种妥协的写法在性能上绝不可能匹敌 C 语言,然而最终的系统测试结果却带来了意想不到的反转。 > “And what you want to do is with Rust and I think and and the whole Rust community I think has been very up-front about this bargain that you’re buying into with Rust is you want to rethink your problem and implement it in a way that makes your ownership clear.” —— Bryan Cantrill ### [43:25](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2605s) 性能意外反超:B 树与 Rust 的组合性优势 Bryan Cantrill 回顾了自己用 Rust 编写代码时一段不可思议的经历。当时为了绕开所有权模型带来的双向链表实现难题,他采用了一种极其“粗暴笨拙”的方法:用哈希表映射 UUID 和时间戳来模拟双向链表。每次想要获取下一个节点时,程序都必须拿 UUID 去哈希表里检索一次。写完之后他自己都觉得这种做法简直糟糕透顶,认定它的性能绝对会慢得惨不忍睹。然而实际基准测试的结果却让他大吃一惊:这段代码的运行速度竟然超越了他原本精心手写的 C 语言版本。面对这个反直觉的结果,Cantrill 决心刨根问底,搞清楚背后到底发生了什么。 深究之后他发现,问题出在数据结构的组合性(Composability)上。该应用程序绝大部分的运行时间都在有序数据结构中执行搜索操作。在原有的 C 语言实现中,他理所当然地采用了 AVL 树(AVL tree)。在操作系统内核与 C 开发中,AVL 树无处不在且极易嵌入:开发者可以在一个大的结构体里直接嵌入 AVL 节点指针,甚至通过嵌入多个节点指针让同一个对象同时挂在 15 棵不同的树上,而且底层库久经考验,几乎不会出错。然而,AVL 树虽然足够优秀,但在现代硬件架构下却并非最优解。相比之下,Rust 的标准集合库采用的是源自数据库领域的 B 树(B-tree)。 在当今的计算机体系结构中,随着内存层级结构(Memory hierarchy)的分化以及 CPU 缓存行(Cache line)特性的放大,具备高局部性(Cache locality)的 B 树在查找性能上全面碾压了指针跳转密集的 AVL 树。Rust 内部的 B 树实现堪称“硬核黑科技”(Rocket science),经过了社区长期的打磨与演进。Cantrill 展示了一张以运行时间为 Y 轴、矩形大小代表数据集规模的性能对比图,清晰印证了 B 树带来的巨大性能红利。 更深层次的问题在于:为什么在 C 语言中大家不写 B 树?Cantrill 指出,在 C 里实现一棵 B 树极其棘手(Gnarly),它的侵入性(Intrusive)太强,逻辑高度复杂,会把指针和内存管理搞得一团乱麻。即便网上有人开源了 C 语言的 B 树实现,面对那种晦涩的代码,任何人都会心惊胆战——一旦内部潜藏内存破坏(Memory corruption)Bug,在 C 语言中去排查将是无尽的梦魇。因此,写 C 时即便明知会损失一点性能,开发者也宁愿退守安全的 AVL 树。而 Rust 强大的类型系统和所有权抽象提供了坚固的组合性,使得开发者可以放心封装并直接调用高质量的 B 树,从而在无惧内存崩溃的前提下坐享极致性能。 ```mermaid flowchart TD accTitle: C 与 Rust 有序数据结构对比 accDescr: 对比 C 语言选择 AVL 树与 Rust 选择 B 树在缓存友好度、侵入性及安全性上的差异。 subgraph C_Impl["C 语言实现路线"] C1["选用 AVL 树"] --> C2["指针跳转频繁 / 缓存局部性差"] C1 --> C3["易于作为侵入式节点嵌入结构体"] C4["回避 B 树: 实现极端侵入、晦涩易出内存破坏"] end subgraph Rust_Impl["Rust 语言实现路线"] R1["标准库 B 树 (BTreeSet / BTreeMap)"] --> R2["匹配现代 Cache Line 局部性,吞吐极高"] R3["所有权与类型系统封装"] --> R4["零成本组合,无惧内存破坏 Bug"] end C2 -.->|"实际基准测试反超"| R2 ``` ### [45:56](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2756s) Rust 的核心杀手级特性与健康的社区文化 除了核心的所有权模型之外,Cantrill 还列举了一系列让他深感惊艳的 Rust 语言特性: - **代数数据类型(Algebraic Data Types)与错误处理**:Rust 中的枚举不仅是简单的整型常量,而是功能完备的代数类型,类似于 C 语言联合体(`union`)的一等公民进化版。它与 `Result` 类型相结合,强制开发者在编译期明确区分是正常返回还是错误状态,必须显式解包(Unwrap)或处理分支,从而构筑了无比优雅而健壮的错误处理管线。 - **卫生宏(Hygienic Macros)**:Cantrill 自嘲道,作为一个常年浸淫在粗糙 C 预处理器(CPP)中的程序员,他甚至直到两个月前写相关博客时才知道“Hygienic”这个单词该怎么拼。在 Rust 中,宏直接操作抽象语法树(AST),具备词法作用域隔离,成为极其强大且安全的一等语言原语。 - **双工外部函数接口(Full Duplex FFI)**:带垃圾回收(GC)的语言很难与操作系统底层代码共存,因为 GC 运行时很难与裸 C 代码在内存中无缝共享活对象。而 Rust 的 FFI 设计得极其周密,支持 Cantrill 所称的“全双工集成”——既能无缝将 C 引入 Rust,也能将 Rust 编译成库嵌入现有 C 架构中。 - **`unsafe` 关键字的精准逃生舱**:`unsafe` 并非让人随心所欲地胡来,而是针对裸指针解引用等极少数操作有节制地放宽编译检查。正是这一受控的逃生机制,才让开发者能够在底层安全地构建出高度可组合的高性能数据结构(如 Rust 自身的 B 树),并向上层暴露完全安全的接口。 谈及社区,Cantrill 注意到台下坐着的核心团队成员 Steve Klabnik,并高度赞扬了 Rust 社区的治理氛围。Cantrill 坦言,很多开源社区派系林立、争执不断,经常让他感到头痛欲裂;而 Rust 社区虽然身处喧嚣的互联网,偶有分歧与不当行为,但其共同的工程价值观犹如“船底的龙骨”(A keel in the water),使得绝大多数技术探讨都能保持在高度建设性的轨道上。 关于 C 语言中对 B 树的敬畏与 Rust 带来的解脱,Cantrill 精辟地总结道: > “I would still use an AVL tree in C even though I know I'm giving up some small amount of performance. But in Rust, I get to use a B-tree, which is really amazing.” > (在 C 语言里,哪怕我知道会牺牲少许性能,我依然会选择 AVL 树;但在 Rust 里,我却能直接用上 B 树,这简直太棒了。) ### [49:14](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2954s) 用 Rust 重写操作系统:运行期特性压倒开发期体验 话题终于切入核心议题:我们究竟能否、又该如何用 Rust 重写操作系统?Cantrill 首先抛出了操作系统架构史中一条颠扑不破的铁律:系统的运行时特性(Runtime characteristics)永远压倒其开发耗时(Development time)。 对于不写内核的普通用户和应用开发者而言,没有人会在意操作系统内核写起来是轻松还是痛苦,也没人在乎团队究竟花了多少年才开发出来。外界唯一在乎的只有两件事:它是否绝对正确(Correct),以及它运行起来是否足够飞快(Fast)。当年结构化高级语言之所以能够击败汇编语言成为系统编程的主流,正是因为它们最终证明了自己的执行效率能够匹敌汇编;一旦性能掉队,高级语言就绝无立足之地。时至今日,每一个现代操作系统依然在关键路径上保留着汇编代码。但 Cantrill 强调,Rust 的出现带来了过去三十多年来前所未有的全新契机——它终于是一门在运行时性能和控制力上足以胜任内核开发的全新候选语言。 在学术界和开源界,很早就有人展开了先锋探索。Cantrill 提到了 Alex Light 在其本科毕业论文中基于教学操作系统 Weenix 用 Rust 重新实现的 Redox(注:此处指 Alex Light 基于教学系统编写的早期 Rust 内核探索项目)。这篇本科论文详尽记录了初期内核开发踩过的深坑,其中最致命的一处架构阻碍在于:当时的 Rust 语言和标准库根本不支持内存分配失败的处理机制。 在早期的 Rust 运行时中,一旦内存分配失败,系统的默认行为就是直接 Panic 崩溃(Cantrill 调侃这种哲学如同“人生何其苦,不如大家一起自我了断”)。在用户态应用中,内存枯竭直接崩溃或许尚可接受,但作为掌管一切物理内存的操作系统内核,这是绝对无法容忍的死罪: - **内核具备自愈能力**:当物理内存耗尽时,内核不能自杀,而是必须主动执行回收机制(如冲刷页面缓存、换出匿名页,甚至极端情况下杀死特定进程来释放资源)。 - **内核对分配策略的精细控制**:Cantrill 以 SmartOS 为例指出,成熟的生产级内核极难彻底耗尽内存,因为内核很早就将内存分配分流为两种模式:一种是允许进入休眠等待内存释放的睡眠分配(Sleeping allocations,永不失败);另一种则是专用于中断上下文、绝不能休眠但允许返回分配失败的非睡眠分配(No-sleep allocations)。 Cantrill 在强调操作系统对性能和正确性的严苛要求时直言: > “Nobody cares how long it took you to develop it. And that it was easy to develop or hard to develop. It's like, no one cares. They care if it works and they care how fast it is.” > (没有人关心你花了多长时间来开发它,也没人关心它开发起来是容易还是困难。根本没人在意。大家关心的只有它能否正常工作,以及它运行起来有多快。) ### [52:10](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3130s) 内存分配失败与深入 Rust RFC 的设计探讨 内核级别的内存分配绝不能简单地在失败时崩溃,它必须具备处理分配失败的能力。Bryan Cantrill 指出,Rust 社区在 1.26 或 1.27 版本左右引入了全局分配器(Global Allocators),这一特性极大地改善了现状,但如何妥善处理内存分配失败(allocation failure)在 Rust 中仍然是一个尚未完全定论的活跃议题。Cantrill 建议开发者去深入查阅相关的 RFC 文档。他强调,任何进行实际 Rust 开发的人很快都会去翻阅 RFC,因为很多特性还非常崭新,即便利开发者正在使用现成的功能,也往往不可避免地会回头去看当初的设计讨论。 这些设计讨论不仅极具趣味性,甚至让人感到精神振奋。Cantrill 以原始字符串字面量(raw string literals)的设计讨论为例展开说明:Rust 采用 `r#"..."#` 的语法,即字符 `r` 加上若干个井号(octothorpe/pound sign/hashtag),中间是字符串字面量,最后以相同数量的井号闭合。他开玩笑地吐槽道:“我曾发誓绝不说 hashtag 这个词,但现在我的孩子们管什么都叫 hashtag,所以我只能妥协……它不是 hashtag,它是井号。”在阅读这个特性的 RFC 讨论时,大家探讨了各种不同系统对原始字面量的实现方式。Cantrill 坦言,缺乏原始字符串字面量一直让他在写 JavaScript 和 C 时感到非常痛苦,而这也是他深爱 Rust 的诸多原因之一。哪怕是一个极小的语法问题,背后的设计权衡也令人着迷。虽然全局分配器已经落地并解决了大部分问题,但针对分配失败的处理在 Rust 内部仍属于需要继续探索的未定领域。 ### [53:47](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3227s) 2015 年以来的从头构建系统及其现实局限 自 2015 年 Rust 1.0 发布以来,业界涌现出了一批用 Rust 编写的操作系统项目。Cantrill 列举了其中一些仍然能够编译成功的代表作:最知名的 Redox、Tifflin、用于物联网的 Tock、以及教学性质的 Intermezzos、Rust OS、Quilt OS、Rocks,还有 Philipp Oppermann 的《Writing an OS in Rust》(被 Cantrill 戏称为“Blog OS”)。Cantrill 现场向 Philipp Oppermann 隔空喊话:“Philipp Oppermann,请给这玩意儿起个名字吧。你想叫什么都行。你必须给它命名,这东西太重要了,值得拥有一个名字。我不想老管它叫‘Philipp Oppermann 的 Blog OS’。”这些项目中,有的专注于 unikernel(如仅允许运行 Rust 程序的 Quilt OS),有的专注于微型嵌入式设备,但它们都有一个共同属性:全部是 *de novo*(从头全新编写)的操作系统。 从零开始构建系统固然能够充分探索新思想,但也带来了两个极其严峻的现实挑战: 1. **丧失 Linux 二进制兼容性(Linux Binary Compatibility):** 放弃 Linux 兼容性意味着世上绝大多数现成软件都无法在上面运行。Cantrill 表示自己完全理解并尊重从零构建的追求,自己多年来也尝试过这种路径,但现实中让全球现有的软件生态直接跑起来至关重要。作为与 Linux 极其接近的类 Unix 系统开发者,Cantrill 团队在 SmartOS 上为了实现 Linux 二进制兼容性吃尽了苦头。这是一条极为漫长的长尾,充满了灰色地带;特别是像 `vfork` 和 `SIGCHLD` 这种语义混合在一起时,Cantrill 痛陈道:“`vfork` 语义极其糟糕,`SIGCHLD` 语义极其糟糕。`vfork` 加上 `SIGCHLD` 的语义,那简直是任何人都绝不该去招惹的、不可名状的语义超级垃圾场(semantic super fun site)。” 2. **陷入“第二系统综合征”(Second-System Syndrome):** 当工程师从零开工时,往往会忍不住想:“既然这趟列车都要重新出发了,不如把所有卡车都倒回来,让我们做一个全新的文件系统,换一套全新的设备驱动思考方式……”这种思维有时能激发创造力,但绝大多数时候会导致项目彻底陷入瘫痪。 ```mermaid flowchart TD accTitle: 从头重写操作系统的权衡 accDescr: 展示从零使用 Rust 重写操作系统所带来的探索优势与现实阻碍之间的冲突。 subgraph Advantages["从头构建 (De Novo) 的优势"] A1["摆脱历史技术债务"] A2["探索全新的设计理念与类型系统"] A3["极佳的教学与实验价值"] end subgraph Challenges["现实工程维度的致命瓶颈"] B1["丧失 Linux 二进制兼容性<br/>(无法运行既有世界级庞大软件栈)"] B2["第二系统综合征<br/>(盲目重构所有组件导致进度瘫痪)"] B3["复杂古怪的语义陷阱<br/>(例如 vfork + SIGCHLD)"] end Advantages -.->|"现实碰撞"| Challenges ``` ### [56:11](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3371s) 为什么单纯为了“安全”无法打动内核开发者 重写那些原本就在稳定运行的成熟代码,其核心优势到底在哪里?Cantrill 认为这一直是个巨大的疑问。对内核开发者而言,单纯搬出“内存安全(Safety)”的旗号并没有想象中那么有说服力。这并不是因为安全不重要,而是因为在成熟的内核中,代码本就已经被磨砺得相当安全:“因为一旦它不安全,系统就会直接炸掉,所有人都会抓狂;于是我们找出原因、修复它,并开发出一大堆优秀的工具链来彻底杜绝这些问题。”因此,要想说服内核开发者替换运行良好的现有代码,仅仅依靠安全性和出色的性能是远远不够的。 更棘手的是内核本身的架构特性:内核中充斥着多重归属(multiply owned)的数据结构。在内核里,几乎所有对象都同时挂在多条双向链表或查找树上。除了少数具有清晰分层结构的组件外,多数核心机制根本无法套进 Rust 那套严苛的所有权单亲关系中。以线程调度(thread dispatch)为例,其本身就存在大量固有的多所有权与并发复杂性。如果为了让 Rust 编译通过而处处引入 `unsafe` 块,那整个逻辑就陷入了自相矛盾:“每一次你在 Rust 操作系统中使用 `unsafe`,你都在一点点丧失 Rust 原本带来的优势。” 基于这一现实,Cantrill 直言,尽管重写整个内核具有历史情怀和表面上的吸引力,但对于人类在余生中仍将依赖的那些主力操作系统而言,这绝非正确的首选路径。Cantrill 坦承自己对全面重写的恐惧:“归根结底,我不想用 Rust 重写 ZFS。一想到要用 Rust 重写 ZFS,我就会在半夜惊出一身冷汗;你去看看 ZIO(ZFS I/O)管线,然后摸着良心告诉我你想用 Rust 重写它?免了吧,谢谢。Rust 编译器会彻底击垮我;它会赢,而我会输得一败涂地。” ### [57:53](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3473s) 务实的混合路线一:在现有内核中引入 Rust 组件 既然全盘推翻不可行,Cantrill 认为务实的方案是“混合路线(Hybrid Approaches)”。第一种混合路线是**在内核内部引入 Rust 组件(Rust In-Kernel Components)**。Rust 与 C 语言天然的互操作能力(FFI)是其最大优势之一。这种方案主张保留历经数十年检验的成熟 C 语言与汇编内核底层,但允许开发者使用 Rust 构建新的内核组件,例如驱动程序、文件系统或新的内核功能模块。 Justin 在先前的演讲中曾提出,系统开发者希望在内核空间中运行更多软件。而 Rust 正好提供了一种在内核中安全执行逻辑的有效途径。虽然既有的资深内核黑客不会轻易用 Rust 去重写内核核心,但在内核中构建一个安全的 Rust 沙盒却极具吸引力。这种增量式演进(Incremental Approach)遵循了 Unix 一贯小步快跑的传统;它允许将 Rust 用于全新的特性开发,而无需将现有的 C 语言复杂结构生硬地搬进 Rust 中承受编译器的折磨。目前 FreeBSD 社区已经有了这一构想的原型验证,Cantrill 表示 SmartOS 未来也绝对希望走这条道路。 ### [58:57](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3537s) 务实的混合路线二:用 Rust 重构操作系统用户态系统服务 第二种混合路线是**用 Rust 重构操作系统的用户态核心组件(Rust OS Components)**。Cantrill 提醒所有人:“操作系统绝不仅仅是内核。”一个完整的操作系统包含了海量的外围系统软件:基础实用工具(utilities)、系统守护进程(daemons)、服务管理器(Service Manager)、设备管理器(Device Manager)、故障管理引擎(Fault Management)以及系统级调试器。无论人们在情感上是否认同,`systemd` 毫无疑问就是现代 Linux 操作系统不可分割的一部分。 Cantrill 强调,如果今天重新编写 `systemd`,或者站在 Joyent 的立场上重新编写 Solaris/SmartOS 的 SMF(Service Management Facility),毫无疑问必须全盘选用 Rust。接着,Cantrill 道出了一句许多内核开发者不愿承认的大实话:“编写用户层(用户态系统级)软件,其实比编写内核代码要艰难得多。” 在内核中,一切都由开发者严格掌控。Cantrill 幽默地反问:“你知道在内核里我永远不用面对什么问题吗?那就是某个用户突然把配置文件给删掉了!我们在内核里从来不需要处理这种烂摊子。在内核里,配置本身就是内核;如果你把我删了,我根本就不会运行,那自然不是我的问题,因为你连机都开不了。”内核开发固然要面对硬件引导、CPU 架构漏洞(比如 Intel 隔三差五爆出的严重硬件漏洞)等诸多棘手难题,但它不需要处理复杂的分布式系统状态、脏配置文件以及不可预测的用户环境交互。正是在这些高度复杂、异常路径极多的系统级用户态软件中,Rust 强大的类型系统、代数数据类型和错误处理机制才能发挥出最极致的威力和价值。 - 操作系统涵盖的关键用户态组件: - 核心服务管理框架(如 Linux 的 `systemd`、Solaris/SmartOS 的 `SMF`) - 设备管理守护进程与故障恢复管理系统 - 系统诊断、底层追踪与核心调试工具链 - 内核软件 vs. 系统级用户态软件的开发挑战对比: - **内核空间:** 数据结构深度共享导致多所有权问题突出;但执行环境封闭自治,不需要面对人为删除配置文件等杂乱的用户态错误,无分布式系统的复杂一致性问题。 - **用户态系统服务:** 运行环境多变,异常边界极多(配置缺失、进程通信中断、外部环境不确定性),逻辑极其脆弱繁重,是 Rust 错误处理与所有权模型大展拳脚的理想场景。 ### [01:00:03](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3603s) 固件的深渊与 Rust 重写 OpenBMC 的契机 当系统软件逐渐演进为涵盖云控制平面(Cloud Control Plane)的整体架构时,操作层面上的大量传统故障在 Rust 中根本不会发生,这使得 Rust 成为云控制平面的极佳候选者。但 Bryan Cantrill 认为,还有一个更让人兴奋且亟待拯救的领域——那就是深藏于现代计算基石之下、技术上却几十年未曾真正进步的“固件(Firmware)”。他尖锐地指出,当今的固件本质上还在运行着被过度美化的 DOS,却承载着全人类的计算工作,形成了一个充斥着不可观测性(Unobservable)、又承载着关键任务的底层“下水道”。Cantrill 用他经典的警句直言当前的对抗现状:“As I'm fond of saying, like it is it is humanity versus firmware, and everyone needs to pick a side.”(正如我常说的那样,这是全人类与固件的战争,每个人都必须选边站队。) 面对固件的混乱现状,Rust 展现出了惊人的契合度。虽然嵌入式开发依然存在硬件挑战,但这些困难在 Cantrill 看来是完全可以克服的。编写固件既不需要处理操作系统内核中那些错综复杂、灾难般的多重所有权(Multiple Ownership),也没有严苛的二进制兼容性包袱,更不用从头去重新发明一套像 ZFS 这样的复杂文件系统。他尤其提到了服务器基板管理控制器开源项目 OpenBMC,并形象地描绘道:OpenBMC 宛如“双膝跪地、泪流满面地哀求别人用 Rust 重写它”。如果 BMC 必须向局域网甚至公网暴露网络端口,那么 Cantrill 绝不希望它是用 C++ 写的,而必须是 Rust。目前固件已成为所有人被迫运行的闭源专有软件孤岛,Cantrill 期盼看到一个由 Rust 驱动的开源固件新纪元。 ```mermaid flowchart TD accTitle: 从传统固件到 Rust 开源固件的转变 accDescr: 对比传统固件的隐患与 Rust 在开源固件中展现的结构性优势 subgraph Legacy["传统固件现状"] A["闭源专有软件底层"] --> B["类似于美化版 DOS"] B --> C["不可观测 / 暴露网络端口"] C --> D["内存与并发安全隐患重重"] end subgraph RustWay["Rust 与开源固件"] E["OpenBMC 等开源项目"] --> F["所有权清晰 / 无内核级所有权灾难"] F --> G["内存安全保障网络暴露面"] G --> H["为全人类构筑可靠基础设施"] end ``` ### [01:01:58](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3718s) 混合互操作之道与问答环节:如何与借用检查器“和解” 在总结演讲时,Cantrill 表达了对 Rust 的极度看好(Bullish)。他指出,这是系统软件领域多年未见的真正替代方案;尽管用它完整重写整个底层操作系统在工程实践上极度困难,但用于内核扩展以及构建软硬件交互系统则是绝佳选择。Rust 最具杀伤力的特性在于它能够与现有的原生 C 语言系统实现无缝的互操作与混合演进,为系统程序员打开了全新的视野。随后演讲进入现场提问环节,针对观众对借用检查器(Borrow Checker)痛苦折磨的调侃,Cantrill 幽默地澄清自己从未恨过它:“First of all, I've never felt I have only tried to bargain with the borrow checker. I have never Even when it is absolutely thrashing me, I know it is out of its own love for what it's trying to do.”(首先,我从来没有恨过它,我只是试图和借用检查器讨价还价。哪怕它把我虐得体无完肤,我也深知这完全出自它对自己使命的爱。) 为了帮助新手平稳度过学习曲线,Cantrill 严正告诫初学者千万不要一上来就去用 Rust 实现双向链表(Doubly Linked List),也不要去拿自己以前用各类语言反复刷过的数独解题器(Sudoku Solver)试水。因为当新手还不理解借用检查器的好处时,编译器每一次看似热心的建议(例如“试试加个 `mut`”、“改成引用吧”)都会演变成一场越改错误越多的滚雪球式挫败,让人欲哭无泪。 - **学习建议与阅读书目**:Cantrill 极力推崇市面上的两部核心著作,并建议并排参阅: - 由 Steve Klabnik、Carol Nichols 及社区撰写的官方指南《The Rust Programming Language》(社区常简称为 TRPL,或发音为 Turple/Triple)。 - O'Reilly 出版社的《Programming Rust》(系统编程领域公认的硬核指南)。 - **逐行敲代码的学习法**:Cantrill 分享了自己在阅读《Programming Rust》时的经历,他老老实实照着书中的完整示例逐行手敲输入代码。这种古朴的方法让他联想到了童年时代翻看《Enter》杂志封底、在 Commodore 64 上逐行录入 BASIC 代码的日子。这种练习能让开发者通过温和的拼写错误去理解类型与所有权,建立起对编译器的信任感。 - **关于学习 Rust 的心态**:对于年长、习惯了靠过往经验即兴“硬敲代码(Wing it)”的老程序员来说,必须放下身段退回学生时代,找个安静的角落买书精读、反复消化,因为 Rust 能够为这种深度的再学习提供巨大的回报。 ### [01:05:43](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3943s) 编译器成熟度与非词法作用域生命周期(NLL) 现场观众向 Cantrill 提问:在演讲伊始曾提到当年 Multics 操作系统因为缺少 PL/I 编译器而苦等三年,那么现在的 Rust 编译器与优化器是否已经达到了 GCC 和 Clang 对 C 语言那样成熟的工业级高度?Cantrill 坦言自己是一个“迟到者”(Latecomer),因而没有经历两年前早期采用者所饱受的折磨。就他目前的实际使用体验而言,Rust 已经非常优秀,生成的代码性能极高。比如增量编译(Incremental Compilation)在今年早些时候上线,在 Cantrill 接触时这项功能已经水到渠成,以至于他无法理解老用户此前对编译速度的抱怨;现在的 Rust 已经高度稳固,他会毫不犹豫地在生产环境中使用 Rust。 对于“是否应该再观望一年”的疑问,Cantrill 承认等待一年确实能够获得更完善的文档、更优秀的第三方包(Crates)以及更少的 RFC 纠葛。而更具决定性意义的飞跃是即将正式合入的非词法作用域生命周期(Non-Lexical Lifetimes, NLL)。Cantrill 现场调侃并预言道:“When you have non-lexical lifetimes, which is in nightly, and you can you can enable it uh optionally. December 6th is going to integrate December 6th. We're shipping it on December 6th. I don't care how broken it is. We are porting it on December 6th.”(非词法作用域生命周期目前已经在 nightly 版本中,并且可以通过选项开启。它将在 12 月 6 日合并进稳定版。不管到时候还有什么毛病,我们都会在 12 月 6 日发布并把代码迁移过去。)他解释道,目前的 Rust 仅支持严格的词法作用域生命周期(Lexical Lifetimes),开发者经常不得不对着编译器抓狂大喊“我保证这段数据我已经用完了,你跟我一起看眼代码啊”,但编译器依然按作用域大括号死板报错。一旦 NLL 普及,未来的开发者甚至都不会理解老一辈人到底在抱怨借用检查器什么。 ### [01:08:24](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=4104s) 借用检查器的未来与 Rust 的历史定位 Bryan Cantrill 在演讲的尾声部分谈到了 Rust 编译器的演进对开发者体验带来的深远改变,特别是围绕“借用检查器”(borrow checker)的改进。他向台下观众表示,随着编译器的优化与人体工程学的提升,未来的开发者甚至可能完全感受不到借用检查器曾经带来的那种痛苦与门槛;对于新来者而言,大家可能根本无法理解早期开发者为何要对借用检查器大惊小怪、吵得沸沸扬扬。这种体验上的巨大差异,将使开发者既能享受到高质量的软件构件(high quality artifacts),又无需承受过去那种陡峭的挫败感。 谈及 Rust 的长期生命力,Bryan Cantrill 给出了极高的历史评价,甚至将其比作古希腊语的形成。他坦言自己绝不是在催促大家盲目跟风,因为在他看来,Rust 将会长久地存在下去,工程师们有的是时间去慢慢学习和接纳。不过他也给那些持观望态度的开发者提了个醒:如果大家之前是因为觉得生态或语言变动太激进、局面太混乱而犹豫不决,那么现在时机已经成熟,大家已经无需再等下去了。 正如 Bryan Cantrill 在总结时所感叹的那样:“I mean, this is not like a hey, do this I mean, I honestly think Rust is going to be around forever. I really do. I think this is like you this is the formation of ancient Greek. So, I mean, there's no rush. You got thousands of years.” 随着这番话音落下,他向现场致谢并宣布演讲结束。 - 编译器的持续改进让借用检查器的阻力显著降低,开发者能够更自然地构建出高品质的软件系统。 - Bryan Cantrill 坚信 Rust 不会是昙花一现的潮流,它的诞生和稳固宛如“古希腊语的形成”一样具有开创性与长远性。 - 对过去因担心技术剧烈变动而处于观望阶段的团队而言,当前 Rust 的成熟度已足够稳定,完全没有必要再继续等待。 ## Notable quotes > [39:03](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2343s) Because even if you and I both know how to write memory safe C, it’s very hard for us to have an interface boundary where we can agree about who does what. — Bryan Cantrill > [49:14](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=2954s) Nobody cares how long it took you to develop it. And that it was easy to develop or hard to develop. It's like, no one cares. They care if it works and they care how fast it is. — Bryan Cantrill > [05:45](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=345s) And I can tell you as a technologist who has had the misfortune of being ahead of my time several times over, being ahead of your time is not commercially fruitful. — Bryan Cantrill > [01:00:03](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=3603s) As I'm fond of saying, like it is it is humanity versus firmware, and everyone needs to pick a side. — Bryan Cantrill > [02:21](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=141s) The operating system is the program, because it is a program, that abstracts the hardware to allow for the execution of other programs. — Bryan Cantrill > [01:08:24](https://www.youtube.com/watch?v=HgtRAbE1nBM&t=4104s) I honestly think Rust is going to be around forever. I really do. I think this is like you this is the formation of ancient Greek. So, I mean, there's no rush. You got thousands of years. — Bryan Cantrill ## Mentioned Rust, C, C++, Unix, Linux, Multics, Bryan Cantrill, Mike Olson, Jeff Hammerbacher, Mike Demmer, Burroughs B5000, MCP, CTSS, Fernando Corbató, Ken Thompson, Dennis Ritchie, Roger Faulkner, Justin Cormack, Steve Klabnik, Philipp Oppermann, Alex Light, Carol Nichols, Cloudera, Tera, Cray, Sun Microsystems, Bell Labs, Apple, IBM, Joyent, Google, DTrace, ZFS, OpenBMC, Redox, Tock, Tifflin, SmartOS, FreeBSD, Haiku, Copland, Spring, Taligent, JavaOS, Node.js, Go, systemd, SMF ## Video description QCon San Francisco, the international software conference, returns November 17-21, 2025. Join senior software practitioners from early adopter companies as they share real-world insights and actionable advice to help you adopt the right technologies and practices. Gain exposure to innovative approaches and fresh ideas in software development and engineering, designed to inspire and challenge you. Don’t miss this chance to deepen your knowledge, sharpen your skills, and stay ahead in the ever-evolving world of technology. Register now: https://bit.ly/3BYzfbe ---------------------------------------------------------------------------------------------------------------- Video with transcript included: https://bit.ly/2Nq7RW5 Bryan Cantrill explores Rust, explains why it has captured the imagination of so many systems software engineers, and outlines where it might best fit in the deep stack of operating system software. This presentation was recorded at QCon San Francisco 2018: https://bit.ly/2uYyHLb For more awesome presentations on innovator and early adopter topics check InfoQ’s selection of talks from conferences worldwide https://bit.ly/2tm9loz #Rust #OperatingSystems #SystemProgramming #InfoQ #QConSanFrancisco [Source](https://www.youtube.com/watch?v=HgtRAbE1nBM)

Source quotes

[39:03] Bryan Cantrill: Because even if you and I both know how to write memory safe C, it’s very hard for us to have an interface boundary where we can agree about who does what. [49:14] Bryan Cantrill: Nobody cares how long it took you to develop it. And that it was easy to develop or hard to develop. It's like, no one cares. They care if it works and they care how fast it is. [5:45] Bryan Cantrill: And I can tell you as a technologist who has had the misfortune of being ahead of my time several times over, being ahead of your time is not commercially fruitful. [60:03] Bryan Cantrill: As I'm fond of saying, like it is it is humanity versus firmware, and everyone needs to pick a side. [2:21] Bryan Cantrill: The operating system is the program, because it is a program, that abstracts the hardware to allow for the execution of other programs. [68:24] Bryan Cantrill: I honestly think Rust is going to be around forever. I really do. I think this is like you this is the formation of ancient Greek. So, I mean, there's no rush. You got thousands of years.

Source quotes

[37:31] Bryan Cantrill: Even if you and I both know how to write memory safe C, it's very hard for us to have an interface boundary where we can agree about who does what. [28:05] Bryan Cantrill: For system software, for the operating system, for that infrastructure, I think GC is a real problem. [57:53] Bryan Cantrill: It's harder to write user level software because more can go wrong. [60:03] Bryan Cantrill: I view OpenBMC as on its knees with tears streaming down its face begging for someone to please rewrite me in Rust. [68:24] Bryan Cantrill: I honestly think Rust is going to be around forever. Bryan Cantrill: The one thing I would advise if you have not implemented Rust, please, for the love of God, don't start with a doubly linked list.

Original transcript

[0:04] All right, so my question is is it time [0:07] to rewrite the operating system in rust? [0:09] Now, I will give you a little bit of a [0:10] spoiler alert. This is Betteridge's law [0:13] of headlines that any article that that [0:17] kind of asks a question, you can [0:19] reasonably say no to. So, um I [0:22] but I I you're not going to get a no out [0:25] of me on this one. You're going to get [0:26] um a bunch of qualified yeses, but just [0:28] to give you a spoiler, [0:30] um it is a question mark, not [0:31] necessarily an exclamation point for [0:32] reasons I'll get into. [0:34] Um but first and I actually I think it's [0:36] pretty great that I don't even think I'm [0:38] the first maybe not even the second [0:40] presentation to have a slide on what [0:42] even is the operating system. I mean, [0:44] we're we're in the OS track, operating [0:46] system's been around a long time, and [0:48] apparently we don't even know what they [0:50] are. Um [0:51] and it's actually harder to define than [0:54] than you might think. If you think this [0:55] is easy, by the way, just say like like [0:57] what is software? Give me a definition [0:59] of what software is. [1:01] And I can wrap you around the axle on [1:04] your definition of software. Cuz I get I [1:06] I I will pose you this follow-up [1:07] question, is Euclid's GCD algorithm [1:11] software? Was Euclid writing software [1:13] when he wrote the GCD algorithm? And if [1:16] your answer is like, well, no, of course [1:17] not because this predates hardware, [1:19] you're going to it's going to be a very [1:20] hard to come up with a self-consistent [1:22] answer to what software is. I actually [1:25] made the mistake um I know Mike Olson, [1:27] who's one of the the founders of [1:28] Cloudera, and I was up at Cloudera in [1:30] very early days. There were like six [1:31] people in the office. I don't even know [1:33] why he had me by. Um but I was having a [1:35] weird day, and I was like really [1:36] grappling with this like what is [1:38] software question. And they're like, [1:40] well, software's a thing that runs on [1:41] hardware. So I was like, okay, well, [1:41] what's hardware? You kind of take that [1:43] apart, and all everything starts [1:44] breaking down. Like you are T-minus 10 [1:47] minutes away from an existential crisis. [1:50] And you you you realize that you know [1:51] nothing, that the ancient Sumerians were [1:53] running software, um [1:56] that software is merely knowledge that [1:57] we run. It doesn't have to be Turing [1:59] incomplete. And at one point like Mike [2:01] can kind of see his team like, you know, [2:03] Jeff Hammerbacher, I think is beginning [2:05] to like stare at his hands and wonder if [2:07] anything means anything. And Mike takes [2:09] me aside, he's like, "What's going on? [2:11] Like are you are you on drugs? I mean, [2:12] what's like are you are you high right [2:14] now?" And what I'M LIKE, "NO, MIKE, I'm [2:16] just like, what is software?" He's like, [2:17] "All right, can you just like talk about [2:18] something else or whatever or get out of [2:19] here?" Um anyway. [2:21] So, you have the same problem with the [2:23] operating system. Like, what is the [2:24] operating system? Well, it's kind of [2:25] like any definition we would say with [2:26] the operating system is the thing that [2:27] abstracts hardware. But like, well, how [2:29] about virtual hardware? I mean, surely [2:31] abstracting virtual hardware, surely the [2:33] guest operating system that you run on [2:35] your AWS instance is surely an operating [2:38] system. That's not abstracting hardware, [2:40] not physical hardware, it's abstracting [2:41] virtual hardware. [2:42] Um so, it's really easy to to kind of [2:44] migrate around or or wander around and [2:46] not have a really concise concise [2:48] definition, but I think for our [2:49] purposes, let's say that an operating [2:51] system is the program, because it is a [2:54] program, that abstracts the hardware to [2:57] allow for the execution of other [2:58] programs. [2:59] Um so, I think that the the fact that [3:01] the operating system is itself just a [3:02] program, it's just kind of a super [3:04] program, um it's kind of an important [3:06] observation. My friend Mike Demmer, [3:08] who's here, um was uh went to work for a [3:10] company called Tera that you haven't [3:13] heard of. Tera actually bought Cray and [3:15] then took Cray's name, which is always [3:17] an interesting sign when you when the [3:19] acquiring company takes the acquired [3:22] company's name, you know it did not have [3:24] real good brand recognition. Um but Tera [3:27] was a super interesting company, doing [3:28] lots of interesting things, and Mike was [3:29] doing kernel development for them. And [3:31] had done that for maybe a year or two, [3:32] and he and I caught up, and he's like, [3:34] "You know what? I realized that an [3:35] operating system is really just a [3:37] program. It's just a complicated [3:38] program. It's an asynchronous program. [3:39] It's an interrupt-driven program." At [3:41] the time I was kind of offended by that [3:42] cuz I'm like, I'm not just a programmer, [3:44] I'm a kernel programmer. Like, don't rob [3:46] me of my dignity. But then you realize [3:47] like, actually, it is just a program. [3:48] It's a complicated program, but it is a [3:51] it's a program that runs other programs. [3:53] I do think that if your program doesn't [3:55] run other programs, it's a lot of great [3:57] things, that's valuable to to society [3:59] and humanity, but we probably shouldn't [4:00] call it an operating system. Um and I [4:02] think that the operating system also [4:03] should be abstracting hardware in some [4:05] way, shape, or form, but even that is I [4:09] is there there's shades of gray there. [4:11] Um and I think the operating system you [4:12] have to say defines the liveness of the [4:15] programs that it's running. If the [4:17] operating system dies, everything that [4:19] it's running dies. Can't live without [4:21] the operating system. So, it's it's [4:22] essential in that regard. Um and we it's [4:25] important not to conflate the actual the [4:27] operating system with the operating [4:28] system kernel. The operating system [4:30] kernel is the thing that runs in the [4:32] privileged context of the [4:33] microprocessor. That's going to be ring [4:35] zero if you're x86, although ring zero [4:38] becomes complicated because if again, if [4:39] you're on virtual hardware, you're in a [4:41] virtual ring zero. Um nested vert, you [4:43] could be in ring zeros that's several [4:45] rings away from the actual hardware. But [4:47] you are in, whether it's virtual [4:48] hardware or physical hardware, you are [4:50] seeing the entire machine or what you [4:52] believe to be the machine. [4:54] Virtualization has made this so [4:55] complicated. Like everything has to have [4:57] asterisks now. It was a lot easier when [4:58] we weren't virtualizing hardware. [5:01] But the um the operating system kernel [5:04] is only one part of the broader [5:07] operating system. The operating system [5:08] also includes libraries. It includes [5:11] commands. It includes demons. It [5:13] includes facilities. And yes, I and yes, [5:16] you know, Justin was in his earlier talk [5:18] was describing how we want to strip all [5:20] the stuff away. He blamed it all on Sun [5:22] somewhat strangely and then said that we [5:24] needed to to to strip it all away. Um [5:26] and that is true to a point, but there [5:28] there's still even in Justin's [5:30] stripped-down world, there are still a [5:31] bunch of components that aren't running [5:34] at that highest level of privilege on [5:36] the microprocessor or virtual [5:37] microprocessor, but we would still [5:39] consider the operating system. So, the [5:40] operating system is bigger than just the [5:42] kernel, and I think that that's that's [5:43] really important. [5:45] Now, I mean, if you've seen me talk [5:47] before, um you know there's no way we're [5:49] getting out of here without a history [5:50] lesson. Um and especially when we're [5:53] talking about operating systems, um [5:55] we've got to turn the wayback dial, uh [5:56] and we've got to talk about history. So, [5:58] um operating systems were actually uh [6:00] they used to be called executives, which [6:02] is a very kind of like strange [6:04] post-World War II hierarchical kind of [6:07] like uh I'm not sure, you know, it was [6:09] like a managerial structure. Um I guess [6:12] executives, I mean, there were there [6:13] were things that could have been worse, [6:14] but I guess we had executives for a [6:15] while, and they changed it to operating [6:16] systems. [6:17] Um and originally these things were all [6:21] written in assembly, of course. They [6:22] were all written in machine code. Um and [6:25] if there's, you know, one really [6:27] interesting architecture to look at, [6:28] it's the the Burroughs B 5000, uh 1961 [6:31] was a super interesting machine. And [6:34] that was the first um the first [6:36] operating system, which was called the [6:38] MCP, which is the master control [6:40] program. [6:41] Um yes, if you've seen Tron, it's a [6:43] master control program in Tron. If you [6:46] haven't seen Tron, do not see Tron. It [6:48] is a terrible terrible movie. Um go see [6:51] WarGames instead. So, you know, you've [6:52] got you're trying to catch up on the [6:54] movies you missed because they happened [6:55] 20 years before you were born, go watch [6:57] WarGames, not Tron, please. But um MCP [7:00] was the master control program, um [7:02] master control executive, just like all [7:04] about dominating programs, I guess. Um [7:08] but the thing that was really [7:08] interesting is this thing was written in [7:10] its own ALGOL derivative called ESPOL. [7:13] Um and if ESPOL, I just feel like ESPOL [7:15] I get Esperanto in my head immediately [7:16] when you say ESPOL. And ESPOL, as it [7:18] turns out, is um a not to be confused [7:21] with a university in Ecuador whose [7:24] abbreviation is ESPOL. So, um probably [7:26] much more famous ESPOL at this point. [7:28] But um ESPOL was this um ALGOL [7:31] derivative that they wrote the MCP in. [7:33] Um and this was a machine that was way [7:35] ahead of its time in many different [7:38] dimensions. And I can tell you as a [7:39] technologist who has had the misfortune [7:41] of being ahead of my time several times [7:42] over, being ahead of your time is not [7:44] commercially fruitful. Um you you know, [7:47] the the basically the biggest upside of [7:48] being ahead of your time is you [7:49] apparently need no introduction in the [7:50] OS track. But other than that, it's [7:53] it's it's all So, Burroughs is a machine [7:55] is is was a [7:57] wildly inventive company, incredible [8:00] machine, but you haven't heard of it [8:01] because they went out of business. Um [8:03] so, but the MCP was really interesting, [8:05] and um then in uh 1964, um [8:11] build a follow-on for uh CTSS, which is [8:13] compatible time-sharing system. Um and [8:16] they wanted to do everything right. Now, [8:19] this is something we call second system [8:22] syndrome. Um and second system syndrome, [8:25] every time I think second system [8:26] syndrome is a bit invented and maybe [8:27] it's a relic, maybe it's kind of a [8:28] mythical man-month kind of holdover from [8:30] the '60s, I get hit by another second [8:32] system. So, I think it is a it is a a [8:35] truth. And second system syndrome is, [8:36] hey, as long as we're going to rewrite [8:38] this, we should also rewrite that and [8:41] revisit this and redo it. We should [8:42] finally do this the right way, and let's [8:44] do that the right way, and let's finally [8:46] change the database to be something and [8:47] next thing you know, you have this [8:48] massive system that ships later and [8:51] later and later and later and finally [8:52] never ships at all. Um and that's the [8:55] second system syndrome. [8:57] Um [8:58] Project MAC had maybe the worst case of [9:00] second system syndrome with Multics. And [9:03] Multics and then this is also just a [9:05] misfortune of history in terms of when [9:06] they came along. They knew they wanted [9:07] to do it in a higher-level language, but [9:09] I guess they just didn't like the [9:11] higher-level languages that were out at [9:13] the time. Now, on the one hand, [9:14] according to Multics, like, well, the [9:15] only thing to pick the only things to [9:17] pick from were really Fortran and COBOL. [9:20] That's actually not the case. So, um [9:22] someone you probably haven't heard of [9:23] named Jean Sammet was an early computer [9:25] scientist, and she wrote an incredible [9:27] book called Programming Languages, um [9:30] which is a a taxonomy of programming [9:32] languages circa 1963 when that book was [9:34] written. And let me tell you, that book [9:36] is more than two pages long, okay? Like [9:37] there were THERE WERE A LOT OF [9:39] PROGRAMMING LANGUAGES, MAYBE TOO MANY [9:40] programming languages. It's like, you [9:42] know, like a Mesozoic era of programming [9:44] languages, too much CO2 in the in the [9:46] programming language atmosphere or [9:47] something, you know, 80-ft tall ferns or [9:49] whatever. But the um there were a lot of [9:51] programming languages to pick from, but [9:52] whatever, they didn't like them all. So, [9:54] fine. So, we're going to do it in this [9:56] programming language that hasn't yet [9:58] been implemented, but has this really [9:59] amazing specification called PL/1. [10:03] So, they did this in in in PL/1. That is [10:05] a Roman numeral one, by the way. If you [10:08] want to put PL/1 in your own slides, do [10:10] not call that a that that's not an [10:11] Arabic one, that is a Roman one. Very [10:13] important point. Um and so they they [10:16] they adopted PL/1 and then they wrote [10:18] the full specification for Multics with [10:21] not not only not a line of code, not a [10:24] compiler. They didn't have a compiler. [10:26] Uh in fact, they had they they decided I [10:29] don't know why do you do this? They [10:30] like, "Let's outsource the compiler." [10:33] It's like, "Okay, that makes sense. [10:34] Let's outsource the the first PL/1 [10:36] compiler ever to be written. We're going [10:38] to outsource this to this like group in [10:40] Los Angeles that no one had ever heard [10:41] of." Who, I mean, shows you how some [10:43] things never change. They're like, [10:44] "Sure, we can implement a PL/1 [10:45] compiler." And then, you know, a year [10:47] later they kind of fly out and be like, [10:49] "Where's the PL/1 compiler?" They're [10:50] like, "We don't know what we're doing. [10:51] We cannot [10:53] We cannot implement a PL/1 compiler." [10:54] So, all right, well, we have no PL/1 [10:55] compiler. So, [10:57] but we have an amazing operating system [10:59] that has been very detailed very [11:01] specified, but we can't actually write [11:03] any of it because we don't have a [11:04] compiler. [11:05] So, this is like ultimate waterfall, [11:07] like [11:08] nightmare waterfall. [11:10] And so, they didn't have a a PL/1 [11:11] compiler until 1966. [11:13] And I mean, that is that is a major [11:15] problem. Now, what I think is is really [11:16] funny about Multics and Multics is one [11:18] of these things that's like a super [11:20] divisive history in that depending on [11:22] who you talk to, Multics is either this [11:24] amazing success that no one gives [11:26] adequate credit to or this dismal [11:28] failure. And I'm I'm sure the truth is [11:30] in the middle. [11:31] But the [11:33] even and and Corbato wrote a Corbato, I [11:35] guess, wrote a an interesting piece, [11:38] Multics the first 7 years in 1971 after [11:41] Multics is already kind of failed-ish. A [11:43] bunch of companies had pulled out of [11:45] Multics. [11:46] And they viewed PL/1 as being this great [11:49] aspect of Multics. And there's no real [11:52] mention of the fact that well, you know, [11:53] you didn't have a compiler for 3 years. [11:56] Like I'm I'm not sure that it's it's an [11:58] unabashed success because [12:00] to me and again, it's hard because this [12:02] is all before I was born. [12:04] It's it's hard to know for certain, but [12:06] to me looking back on it, the fact that [12:08] the compiler was not available for so [12:09] long had to have exacerbated the decline [12:12] of the coalition. They actually called [12:14] them the triumvirate. The triumvirate [12:16] that ran Multics. And in particular, [12:18] Bell Labs pulled out of the Multics [12:21] project. So, Bell Labs is like, "All [12:23] right, we're done. [12:25] This is crazy. Like this is not going [12:26] anywhere. Like the compiler hasn't [12:28] showed up. It doesn't work." [12:29] They actually done because PL/1 wasn't [12:31] there, they developed a different [12:33] dialect of PL/1 called EPL. This is one [12:36] of the things that you read like what [12:37] the EPL people say and they're like, [12:39] "Boy, did we save the day? We came up [12:40] with this dialect of the PL/1 called [12:41] EPL." Then you read Corbato, he's like, [12:43] "People came up with the dialect of [12:44] PL/1, but we never used it and it was [12:45] completely insignificant." So, it's [12:46] like, "All right, I've got no idea what [12:48] happened." It's like watching a Rashomon [12:49] for operating systems. [12:51] Anyway, Bell Labs pulls out of Multics [12:54] in 1969 and there was a researcher that [12:56] was formerly on the Multics effort [12:57] called Ken Thompson, you've heard of him [12:59] or something. [13:00] He implemented a one implement a new [13:02] operating system [13:04] and now didn't have Multics, so [13:06] implemented this thing on the PDP-7. [13:09] It was later ported to the PDP-11/20 [13:12] and it was named Unix as a word play if [13:14] kind of a somewhat peculiar one [13:17] on on the complexity of Multics. [13:19] And you every time I kind of reread the [13:21] Unix history, I get kind of something [13:23] different out of it. I mean, there there [13:25] is it's unquestionable that Unix a lot [13:29] came together at Unix in terms of the [13:31] the late '60s, especially the early [13:33] '70s. They were in the right place at [13:35] the right time. One of the things that I [13:36] see, especially reading it side by side [13:38] with the Multics history, is it was [13:40] really top-down software versus [13:42] bottom-up software. [13:43] And Multics was very top-down command [13:45] and control, write the specification, [13:47] maybe we'll get a compiler, maybe we [13:48] won't kind of software. Unix was like, [13:51] "Ah, let's just get something working [13:52] and then I don't know, get some mouse [13:53] working and then we'll we'll write a man [13:55] page for whatever we already got [13:56] working." And it was it was very organic [13:59] and I think that the world was ready for [14:01] that kind of software model. And you [14:04] begin to realize kind of, you know, [14:05] looking at it in the larger lens of what [14:06] was going on socially and everything [14:07] else. [14:09] The as my my mother is fond of reminding [14:11] me that the because I didn't I wasn't [14:13] alive in 1968, I've got no idea what a [14:16] society on the brink actually looks [14:17] like. Every time I complain about the [14:18] present, she's like, "Oh, this has [14:19] NOTHING COMPARED TO 1968." I'M LIKE, [14:21] "ALL RIGHT, ALL RIGHT, MOM, FINE. You [14:22] know, Mom, you're kind of confirming the [14:24] stereotype of baby boomers." But let's [14:25] move on. [14:26] Um but in particular, [14:29] Unix was implemented entirely in [14:31] assembly. And I think it's it's a bit of [14:34] a myth that that Unix was implemented in [14:36] C. It wasn't. It was implemented in [14:37] assembly for the PDP-7. It was ported to [14:40] assembly on the PDP-11. [14:41] And yes, there was B. [14:44] There was an interpreted language that [14:45] was that B that was around. It's a BCPL [14:47] derivative. [14:49] B had a There's a very kind of like [14:52] strange kind of connection to Multics in [14:55] that that B and EPL were both [14:59] implemented in this thing called TMG. [15:02] TMG, I know that sounds like a teen web [15:05] series that my kids would watch. [15:08] I feel like I would like my 14-year-old [15:09] would watch the latest episode of TMG. [15:11] But no, TMG was the transmogrifier [15:14] that was a transpiler. And B was [15:17] actually interpreted. [15:19] And the actually TMG is another kind of [15:21] sorry, footnote on a footnote was [15:22] implemented by Robert Morris, Bob [15:25] Morris, the father of Robert Morris Jr. [15:28] who was the Morris worm. Even though I [15:30] think Paul Graham begins to like his eye [15:31] twitches whenever anyone says the Morris [15:33] worm because he and Robert Morris [15:34] founded a company together. He doesn't [15:35] like anyone to talk about the Morris [15:36] worm, but it's fine. I thought it was [15:38] Oh, whatever. [15:39] Um [15:40] Tandy's not Tandy's. But so, B actually [15:42] was only used in these like super [15:44] auxiliary capacities in in Unix. [15:47] And the it was used in the assembler. It [15:49] was used in a very version of DC. DC, a [15:52] command that you only may know from [15:54] mistyping CD. You ever done this where [15:57] it's like you want to like CD to your [15:58] home directory or whatever, you just [16:00] type CD enter, you get DC enter and [16:01] you're like, "Wait, what just happened?" [16:03] And you're like a captive prompt and [16:05] you're like [16:06] control C and it's like eats the control [16:08] C. You're like, "What is going on?" [16:09] You're like, "Control D." Like quit. [16:11] You're like, "I don't know how to quit." [16:12] Or whatever it says. It actually doesn't [16:13] say that. It doesn't actually say [16:14] anything. DC is a [16:16] it reflects its B heritage, suffice it [16:18] to say. [16:19] But so, they realized that B was not [16:21] going to do it. It didn't perform. They [16:22] needed a much higher performing language [16:24] than B. In particular, they [16:27] Dennis Ritchie and Ken Thompson saw the [16:30] power of implementing the operating [16:31] system in a higher level language, [16:33] but they knew that B in particular had [16:35] no byte addressability. So, the lack of [16:37] byte addressability was going to be a [16:38] big problem. [16:39] So, they came up with a new language [16:40] that they called C. [16:42] And so, C followed on for B. Kind of it [16:44] made sense. But again, contrary to the [16:46] myth, these are they they they are not [16:49] twins. They're not born at the same [16:50] time. Unix is born Unix begets C in a in [16:54] a very important, I guess, biblical way. [16:57] They they're siblings. They're not [16:58] they're not twins. And Unix is very much [16:59] the the older sibling. And so, C is is [17:03] rightfully called portable assembly. I I [17:05] I like the uh People call portable C [17:08] portable assembly as a slur, but I I [17:10] view it as praise. I love I mean, like, [17:13] "What? It's assembly and it's portable? [17:14] That sounds great." [17:16] I There's a lot that that is great about [17:20] C. And C also was way ahead of its time. [17:22] And if you look at those other [17:23] programming languages from that era and [17:27] then you look at C, they are [17:30] it's like stepping into modernity [17:32] suddenly. [17:34] First of all, the caps lock key [17:35] disappears. It feels like every language [17:37] prior to C was like screamed. It's like [17:39] is the Was the caps lock key broken? [17:42] And you know, I I I like to to collect [17:46] old programming manuals. [17:48] And I bought a kind of a weird lot from [17:52] someone in the UK and the box shows up [17:54] and it's like a bunch of crap. And then [17:55] there's this manual for this language [17:57] called language H. [17:59] And language H is a COBOL derivative [18:02] that you've not heard of for a reason. [18:03] This is an NCR [18:05] Elliott initiated language that had [18:07] Sterling as a keyword. [18:09] Every language H program begins with [18:13] chapter one. [18:16] And you're like, "Oh, okay. All right, [18:17] this is cool. Like it's a book. LIKE I [18:19] GET IT. LIKE WHATEVER. IT'S LIKE SUPER [18:21] NEW. Computing is super new and a book [18:22] is the is the best analog." And then you [18:24] get chapter one. It'll be like, you [18:26] know, Sterling and you [18:28] talk about tapes and all these kind of [18:29] calculations. And then every program [18:32] There's no chapter two, by the way. It's [18:33] only chapter one. Every program ends [18:36] with the keyword obey. [18:40] AND YOU'RE LIKE, "WHAT? [18:41] LIKE HOW ABOUT EPILOGUE OR END? LIKE [18:43] THAT DOESN'T make any sense. It's like [18:45] now all of a sudden these are like [18:46] orders that you're whatever. It doesn't [18:47] make any sense. AND SO, THAT'S what we [18:49] WENT FROM LIKE OBEY AND CHAPTER one to [18:52] modernity kind of in a very short period [18:54] of time. And [18:55] the C revolution was was real. [18:58] And it was organic. I mean, I think one [19:00] of the things that's interesting again, [19:01] if you look at this Multics as this [19:03] top-down approach, this kind of command [19:04] and control approach and Unix as this [19:07] more bottom-up approach, one of the [19:09] differences you see is that C kind of [19:11] grew things as it needed them. The first [19:13] implementation of C didn't have [19:14] structures. They added Now, admittedly, [19:17] structures were added in 1973. This came [19:19] in early. [19:20] But they didn't actually implement the [19:21] Unix kernel in C until they had [19:23] structures. They implemented structures [19:24] because they needed them for the Unix [19:26] kernel. [19:27] They implemented bit fields because they [19:29] needed fields for So, they they [19:30] implemented things as they needed them, [19:32] which had strengths and weaknesses. [19:35] One of the weaknesses from my [19:36] perspective, [19:38] there is no logical XOR in C. This is [19:40] one of the things that pisses me off [19:41] about C. I know Okay, I know this is [19:42] this is an arcane thing to get kind of [19:44] spun up about, but there's no logical [19:46] XOR. You've got you've got [19:47] you've got you've got bitwise XOR, but [19:50] and you've got bitwise and bitwise OR [19:52] and you've got logical and logical or. [19:54] So wait, where's logical XOR? And there [19:56] are actually times when you want logical [19:57] XOR in C, where you want to assert one [20:00] of these conditions is true or the other [20:01] condition is true, but not both of them [20:03] and at least one of them. Like that's [20:04] logical XOR. And you'll know you're in [20:06] my code because that code has that [20:08] assertion, yes, done with bitwise XOR. I [20:10] I know how to do it, okay? With an extra [20:12] line of code and then a block comment [20:14] like cursing the ghost of Dennis Ritchie [20:17] um [20:18] for not having logical XOR. Even more [20:19] galling actually was the I worked with a [20:21] guy named Roger Faulkner who worked with [20:22] Dennis Ritchie, and I was complaining [20:24] about this. He's like, "No, no, there's [20:25] no logical XOR because logical XOR can't [20:26] short circuit." And I'm like, "Roger, I [20:28] know that logical XOR can't short [20:29] circuit, but like did did is there a [20:31] some sort of biblical commandment that [20:33] logical operations must be able to short [20:35] circuit?" He's like, "No, no, I'm just [20:36] telling you that that's the reason." I'm [20:37] like, "That's that's not the reason." He [20:39] said, "Well, I'm going to ask I'll ask [20:40] Dennis." I'M LIKE, "GO ASK DENNIS. Do [20:43] it." And of course Dennis immediately [20:44] replies like, "No, no, we don't have [20:45] logical XOR because it can't short [20:46] circuit and you should just use [20:47] bitwise." I'm like, [20:49] "Anyway." [20:50] So I was wrong. But so there was this [20:51] kind of organic growth and the organic [20:53] growth was um was im- important. Uh in [20:55] particular, C grew very important [20:58] facilities like macro processing. Macro [21:00] processing and Rust has actually caused [21:02] me to reflect back on my own use of [21:04] macros because it's one of the things I [21:06] absolutely love about Rust. Um the the [21:08] macro facilities in C are really really [21:12] really important. Yes, I know you've [21:14] been told never use the preprocessor and [21:16] yeah, I get it. But like that's the same [21:18] thing like we tell our kids about like [21:20] you know, that they shouldn't you know, [21:20] drink and they shouldn't do drugs and [21:22] they shouldn't they shouldn't have sex. [21:24] Those are all the fun STUFF LIKE FUN [21:25] STUFF THAT LIKE ACTUALLY THEY SHOULD DO [21:26] WHEN they get older and they can do so [21:28] safely. Um and the um like I want [21:32] grandkids. Like I mean at some point [21:33] like figure it out, but then not now. Um [21:35] the and and the the preprocessor is the [21:38] same way. Um the the preprocessor is [21:41] really essential as essential for [21:42] Detroit, was essential for CFS, was [21:44] essential for a lot of the things we've [21:45] done because but the preprocessor really [21:48] leaves a lot to be desired. Um there [21:50] were standardization efforts, but they [21:51] came late, they were contentious and so [21:53] on. Um so it through the '80s, C just [21:57] dominated. Um and uh it was an excellent [22:00] fit for operating systems. Essentially [22:02] every OS, with some exceptions, I know [22:03] that the Mac was in Pascal, fine. So [22:06] there's going to be one of those in [22:06] every class. Um and there were there [22:08] were some research operating systems [22:09] that don't imply this is the totality of [22:10] operating systems, but the vast majority [22:13] of production systems were either in C [22:14] or they were still in assembly because [22:15] they were DOS. Um but um [22:18] many of them were were in C. Uh in the [22:21] 1990s, a great darkness came across the [22:23] land. Um it it was really a terrible [22:26] time to be alive, I'm convinced. Um so [22:28] the and now I can speak from personal [22:30] experience. I really it really felt very [22:32] because object-oriented programming um [22:35] with felt like there there was this kind [22:37] of uh this [22:39] this industrial complex object-oriented [22:40] industrial complex that that insisted [22:42] that everything be object-oriented. Um [22:44] and the C-based systems were thought to [22:46] be relics. Everything was going to be [22:47] every any C-based system was the walking [22:50] dead and this is all going to be [22:51] replaced by these beautiful C++ systems [22:54] or Java-based systems that are going to [22:56] replace all of this. So you can keep [22:57] working on this legacy garbage if you [22:58] want, but all of us are going to go over [23:01] here. All all of the the flowery people [23:03] are going to go over here. The [23:03] architects are going to architect this [23:05] great beauty. And all of those systems [23:08] failed. Um and they failed, I think, for [23:10] a bunch of reasons, but one is they were [23:12] all framed as second systems. Um and [23:16] they a lot of them suffered from the [23:17] second system syndrome. So [23:19] uh Apple's Copland. I mean, I you know, [23:21] kids go up today and they think of Apple [23:22] as a trillion-dollar company. I still [23:25] remember when Gil Amelio was running [23:26] Apple and every other Silicon Valley [23:28] company was eyeing them for their real [23:30] estate um because they couldn't ship an [23:33] operating system. They had this second [23:34] system called Copland. It was going [23:36] nowhere. Sun had Spring that was written [23:37] in all in C++. [23:39] Um we I was an undergraduate at the [23:41] time. We got this like this hot release [23:43] of Spring. You thought we would get we [23:44] were getting like some sort of something [23:46] that was like illegal. I mean, it was [23:48] like, "Okay, I'm going to give this to [23:49] you on a CD, but this is amazing." And [23:51] you run it, it's like, "Is this a hard [23:52] drive stress test?" Because all this [23:54] thing is doing is rattling the spindle. [23:57] And as soon as I get there, I go, "How [23:57] much DRAM do you have?" And I and at the [23:59] time, this is a this is a souped-up [24:01] machine that had 32 megabytes of DRAM. [24:04] And they're like, "Well, oh, no, no, no, [24:05] that's not nearly enough DRAM. You need [24:07] to have 128 megabytes of DRAM." Which is [24:09] like saying today, "Oh, you only have [24:10] 512 gigs of DRAM. Yeah, you're going to [24:12] need like 2 terabytes of DRAM." Um [24:16] And you're like, "Who has that?" Like, [24:17] "Well, all of the Spring developers HAVE [24:19] THAT." LIKE, "THAT'S A BAD IDEA." LIKE, [24:21] I DON'T I DON'T I DON'T NEED TO TELL YOU [24:22] that that's a very bad idea. To to give [24:24] you don't want to give your OS [24:25] developers so much memory because [24:27] they'll use it all and then it can't be [24:28] used on a system that's got less memory. [24:29] Anyway. So there was Sun Spring. Uh [24:31] there was Taligent. Have you ever heard [24:33] of Taligent? Do you know Yeah. So you're [24:34] So Taligent was Taligent was going to [24:37] rule us all. So Taligent was going to It [24:39] was IBM and Apple. It was the [24:42] the best and the brightest were going to [24:45] to go up to the ivory tower and hand us [24:47] down the operating system that we would [24:48] run. Taligent, as it turns out, I didn't [24:50] actually didn't learn that thankfully at [24:51] the time. I might have I might have [24:52] really gone down the drain. Taligent, as [24:54] it turns out, is a portmanteau of [24:56] talented and intelligent. [24:58] IT'S LIKE, "WOO [24:59] BOY." IT'S LIKE, "HOW ABOUT HOW [25:01] ABOUT THAT?" LIKE, THAT'S [25:03] UH [25:04] CALL YOUR COMPANY THAT. AND [25:06] AND AS IT turns out, they like nailed [25:09] the worst of IBM meeting the worst of [25:12] Apple in an effort that was a total [25:15] disaster. Um and it And again, and they [25:18] were going to do this this kind of this [25:19] operating system and they just it And it [25:21] wasn't necessarily their fault. It was [25:22] again, it was like because they were [25:23] taking this from-scratch approach, it's [25:26] just that it attracts all sorts of [25:28] abstraction garbage like some sort of [25:30] terrible abstraction fly strip. So um [25:33] and then the Java-based operating [25:34] systems, I mean, I was at Sun, so I you [25:35] know, I'm obligated to remind us all [25:37] that Java OS actually went nowhere. Um [25:40] this is where we're going to do Java [25:41] everywhere. We're going to do Java based [25:42] microprocessors. We'll do a Java OS. Um [25:45] Java had no unsigned type. Hard to talk [25:47] to hardware when you have no unsigned [25:48] type, as it turns out. So that didn't [25:50] that didn't didn't work very well. Um [25:52] but then uh Linux arrived. Um and the [25:55] nice thing about Linux, especially x86, [25:57] is that Unix had a huge resurgence. [26:00] Um and but with that, C-based systems [26:01] became really deeply entrenched. And [26:03] there there were a bunch of C++ efforts, [26:05] they all kind of withered. Uh and there [26:06] were I mean, there are exceptions to [26:08] this. Um you know, Haiku is an exception [26:09] to this. [26:11] Um but you know, when Haiku is is listed [26:13] as your example of a serious attempt, I [26:15] mean, I love Haiku, first of all, but [26:16] Haiku is was designed as like the [26:17] hobbyist operating system to emulate be. [26:19] It's great, I love it, but [26:22] it it's the counterexample. I mean [26:23] again, I'm sure there are others, but um [26:25] basically C++ kernels withered, [26:26] everything was in C. [26:28] Um and at the same time, um we were [26:30] there was this kind of era that was very [26:32] kind of hegemonical [26:33] um around Java and C++, but especially [26:36] Java Java Java Java. Um and then [26:38] appeared other languages, first Ruby and [26:39] then Python. Um JavaScript JavaScript [26:42] has If you look at the popularity of [26:43] JavaScript over time, there's kind of [26:44] this dip where JavaScript was used to [26:46] like, you know, for the blank tags or [26:47] whatever. And then in 2006, I, like a [26:50] bunch of people, were realized like, [26:52] "Wait a minute, there's a language on [26:53] the browser." And okay, I've got to use [26:55] it because like I need to build [26:57] something that's that's going to be this [26:58] kind of web app. And I again, like I [27:00] think a lot of people waited in the [27:01] JavaScript and realized like, "This is a [27:02] real language, actually. This is like [27:04] this actually is Lisp in disguise in the [27:07] browser, but with no parentheses." And I [27:09] I kind of hate the parentheses, so I'm [27:10] I'm fine with that. Um and so JavaScript [27:13] was super interesting. Um and there were [27:14] lots of things that were interesting [27:15] about that. But all those languages [27:17] really focused on the developer in [27:18] development and the speed of [27:20] development, the pace of development. I [27:21] cannot bring myself to put developer [27:23] velocity on a slide because it makes me [27:25] want to throw up. But that is what we [27:26] were trying to do. That is what that [27:27] that is what we're trying to do. We're [27:28] trying to make developers faster, but [27:30] there's there wasn't I don't think if [27:32] there was a Ruby OS effort um or a [27:36] Python OS effort. It's like, "Okay, then [27:38] maybe there was." It's like Steve [27:39] Steve's like, "Oh my god, we promised to [27:41] never to speak of the Ruby OS effort." [27:43] It was college. AND [27:44] EXACTLY. A LOT of mistakes were made. I [27:47] know, I I was experimenting. I was it [27:49] was age-appropriate. I know, that [27:52] that makes total sense. Um but there [27:54] would be a lot of problems making a Ruby [27:56] OS or a Python OS. And it wouldn't be it [27:57] would be something you would do because [27:59] you were in college, which is fine. Um [28:01] but [28:02] there were no real serious efforts. Um [28:05] So we hit kind of I guess what we're [28:07] going to end up calling the teens, which [28:08] makes no sense to me, but um we hit the [28:09] 20 the 2010s and system programmers were [28:13] looking for something different. Um they [28:15] they want they they we want the [28:17] performance of C, but they we had used [28:21] these other languages and and you know, [28:23] it's actually it's kind of nice to use a [28:25] map or a hash or it's kind of nice to be [28:28] able to to use the you know, to use [28:31] split or regular expressions or have [28:32] you. I mean, things that that are [28:33] powerful constructs, but I want them in [28:36] the in in in in the system that I'm [28:38] building. Um and there were kind of some [28:40] interesting things that came out of [28:41] that. Um one of them my my joint. So one [28:43] of the things that was an interesting [28:45] artifact around that is that there there [28:47] was a JavaScript war in the browser, [28:50] which led to some very powerful [28:51] JavaScript runtimes, V8 in particular, [28:54] that we could actually run on the [28:55] server. And actually we could run [28:57] reasonable servers writing in JavaScript [29:00] on the server side, Node.js. Um and that [29:03] was really interesting. Now, as it turns [29:05] out, that left a lot to be desired um [29:07] because it's still JavaScript. I mean, I [29:09] love JavaScript. I I um but I JavaScript [29:11] is the failed state of programming [29:13] languages. Um [29:14] the in in that JavaScript is [29:17] wants anything to go. Um wants to allow [29:20] so much that it can become difficult and [29:22] things like TypeScript kind of came into [29:23] that gap and so on. But like JavaScript [29:25] wasn't necessarily it for we used it at [29:27] a lot of Joint, but there were lots of [29:28] challenges in the broader world for for [29:30] system software. Um the Bell Labs [29:32] refugees um at Google [29:34] uh developed I'm I'm not sure if they [29:35] like to be called that or not. I I mean [29:37] that as praise, not not damnation, but [29:39] um [29:40] developed Go, of course. Many people use [29:41] Go. Um Go definitely solved some of [29:43] these problems, um but there are some [29:46] idiosyncrasies. Um [29:48] tried to put that in the best possible [29:50] way I could. There are some [29:51] idiosyncrasies. [29:52] Um, if you didn't see it, Aiden had a [29:54] great talk um earlier on G visor and [29:57] some of the the the the the I would say [30:00] both the promise and the peril of of go [30:03] for a very creative use case. One of the [30:05] problems with go from an operating [30:07] system perspective is that go and [30:08] JavaScript and others are GC. They're [30:10] garbage collected. And [30:12] they are very I mean, speaking for V8 [30:15] because I know V8 better than go, it's [30:17] very sophisticated GC. I mean, it's very [30:19] sophisticated garbage collection, but [30:20] it's still garbage collection. And [30:23] garbage collection, even when it's [30:24] sophisticated, it's not the garbage [30:26] collector. People like to blame the [30:27] garbage collector. It's like, "The [30:28] garbage collector is ruining my [30:30] process." It's like, "No, the garbage is [30:31] ruining your process." LIKE, "YOU'RE [30:33] CREATING THE GARBAGE. IT'S NOT THE GC [30:34] hates you." It's like, "STOP CREATING [30:36] CRAP AND THE GC won't have to do this." [30:38] Or worse, stop creating these huge I [30:40] mean, when the GC is really exhausting [30:43] itself, it's because you have a memory [30:45] leak. It's like, "But I can't have a [30:47] memory leak. It's GC." It's like, "Ah, [30:48] this is exactly how we got here." But [30:50] you because what you have is a big [30:52] object graph that's very connected and [30:53] in JavaScript it's very easy to [30:55] accidentally have a closure that has a [30:57] reference into this big object graph and [30:59] now none of this can go away because [31:02] it's not garbage. It's not a garbage [31:04] collector, it's a heap scanner. Um, AND [31:08] IT'S NOT FINDING ANY of the it's [31:09] scanning your heap very well, thank you [31:10] very much. Um [31:12] And this is tough. This is just tough [31:15] and it makes it like GC I I honor GC. I [31:19] I think it's great for a lot of [31:21] software, but there's a class of system [31:24] software for which GC just doesn't work [31:27] and it doesn't work because you don't [31:29] have predictable performance, because [31:31] the latencies can be significant and [31:33] because I think because it doesn't force [31:35] the programmer to really think about [31:37] this stuff. And I think thinking is [31:40] important [31:41] for [31:42] Wait. [31:45] That sounds pejorative the way I'm [31:47] phrasing it, but I don't mean it in a [31:47] pejorative way. I think there is a class [31:49] of software for which I I don't mind [31:53] requiring deeper thought up front. And [31:56] then there's another class of software [31:57] for like, "Hey, let's get something [31:58] working." And that's fine. So, it's this [32:01] is not like the entire world should be [32:03] GCed or none of the world should be [32:04] GCed. I actually do think there are [32:06] right tools for the job. But for system [32:07] software, for the operating system, for [32:09] that infrastructure, I think GC is a [32:12] real problem. Um, and at least for me [32:14] like I I [32:16] am at the point where it's like I don't [32:17] want to do performance critical software [32:19] in in GC. [32:20] And then so I was actually at a bit of a [32:22] quandary because like the node thing [32:25] like wasn't working out. I was kind of [32:26] getting out of a bad relationship on [32:27] that one and then it's like what what's [32:30] next? And [32:32] like C++ I had done way too much C++. I [32:35] talked about in college. I had done way [32:36] too much C++. So, Steve had done Ruby [32:38] OS. I had written 100,000 lines of C++. [32:40] They were both uh age appropriate [32:42] mistakes I think we would say in [32:43] retrospect. Um, I didn't want to go back [32:45] to C++. AND EVERYONE'S LIKE, "NO, NO, [32:46] BUT C++ is like totally changed now." [32:48] I'm like, "That's great for C++ and I [32:50] want C++ to have a happy and productive [32:51] life, but C++ dragged all my into [32:54] the street and lit it on fire. So, like [32:55] I am not going back to C++ ever and [32:58] that's not a value judgment. That's just [33:00] like that's self-preservation. But I'm [33:02] not going back to C++. [33:03] Um, so forget C++. It's not going to be [33:05] C++ even though again I honor the the [33:08] the the changes they made to it. [33:10] Uh, and it kind of like looking at this, [33:12] I was getting increasingly excited about [33:14] Rust. Rust was really interesting and I [33:17] think how many people are aware of Rust? [33:20] Okay, a lot of hands. [33:22] How many people have kind of pined after [33:25] Rust but haven't written any? [33:28] Okay, how many of you have written Rust? [33:30] That's a good number actually. I yeah, I [33:32] think it's a good number, right? Yeah, [33:33] it's a good number. But okay, but a lot [33:34] of you were in the category that I was [33:36] in not that long ago, which is kind of [33:38] like I was seriously Rust curious. [33:41] I I you know, I was like reading all the [33:44] blog entries and kind of like, you know, [33:46] looking at like, "Wow, like that looks [33:47] interesting and this looks interesting." [33:49] And um [33:51] like, "But what do they do about white [33:52] space?" You know, cuz I got I got I got [33:53] I got to find a reason to hate it, which [33:54] is wrong, right? You don't want to do [33:56] that. Like get over your white space. [33:57] This is what I've I've learned a tough [33:58] lesson. Um, but like I I I was really [34:01] interested about. I was intrigued by [34:03] Rust. Um, and in particular and the the [34:06] the values of Rust really represented [34:08] the values that I have for my own [34:10] software. So, values in the the in terms [34:12] of values of Rust they said very up [34:14] front. You go to any Rust website, it's [34:15] going to say that it is a system system [34:16] software language that runs fast and is [34:19] designed to be safe. Um, and that was [34:22] really interesting because those are the [34:23] things that I want to do. And I want to [34:26] build permanent software. Um, I want to [34:28] build software that that outlives me. [34:31] I've already written software that I [34:32] know absolutely will outlive me. And not [34:34] because I plan to walk in the traffic [34:35] tonight, but because I I um [34:38] the there is software that I've written [34:39] that's simply going to be uneconomical [34:41] to to rewrite because it works. And the [34:44] thing that is beautiful and amazing [34:45] about software is like Euclid's GCD [34:48] algorithm 2,000 years ago, when software [34:51] works, we've always focused on how [34:52] software is always broken and the world [34:54] is terrible, which is fine. That's true. [34:55] But when software works, it works [34:59] forever for the problem that it's [35:00] solving. It is mathematical machine. I [35:02] mean, we are really lucky because I you [35:04] know, most of us are software developers [35:05] and we are lucky to be alive now. We [35:08] live in a golden age of this stuff. The [35:11] our descendants will look back at this [35:12] time as this is like amazing time when [35:15] all the software was being written. And [35:18] now that said, some of the software is [35:19] not going to last forever. Some of the [35:21] software we are going to have to to [35:23] rewrite with software that can survive [35:26] in perpetuity. And what I see from Rust [35:29] is that aspiration for if in perpetuity. [35:31] And the I know Rust has like eight [35:33] different origin stories [35:35] all from Graydon. I mean, Graydon came [35:37] up with the name and has got like eight [35:38] different reasons why he called it Rust, [35:39] but the one that I like is that I like [35:41] kind of that industrial sense of [35:43] building things that that survive such [35:45] that the chemical reaction where the [35:47] oxygen is like your biggest problem. [35:49] Um, that's pretty neat. Um, so Rust is [35:51] really interesting. The thing that is [35:53] super interesting about Rust and to the [35:54] best of my knowledge unique is the [35:56] notion of ownership. So, Rust is able to [35:59] statically figure out who owns what and [36:03] when you, you being like the thread of [36:04] control, own the memory that you're [36:07] you're you're dealing with and when you [36:08] don't. And when you don't, when you're [36:10] done when it realizes like, "Okay, [36:12] you're done with this, I will take care [36:14] of freeing this for you." There is no [36:16] free in Rust per se. Um, it it memory is [36:19] allocated for you and then it is freed [36:22] when you are done with it. Now, that is [36:23] a the compiler makes this huge bargain [36:26] with you. It's like the the compiler [36:27] says to you, "I will do this for you. I, [36:29] the Rust compiler, will do this I will [36:31] give you this incredible superpower [36:33] where you don't need to think about [36:34] this, [36:35] but you're going to need to help me out. [36:37] And you're going to need to help me out [36:39] when I don't know where this memory is [36:41] going or who owns it. So, I'm going to [36:43] allow you to lend this memory out. I'm [36:46] going to allow you a caller to borrow [36:47] it, right? But I need to know when it [36:50] comes back. And sometimes I get [36:53] confused. I, the compiler, get confused. [36:54] And sometimes it uses lexical lifetimes [36:56] a lot to figure out like what owns what [36:58] when. And sometimes I will give you an [37:00] incredibly cryptic error message because [37:02] I am confused about what about who owns [37:04] what when. [37:05] And you are going to grind on this to [37:08] get this working even though it feels [37:09] painful and it feels like I'm giving you [37:11] mixed messages and telling you to like [37:12] fetch rocks [37:14] because I'm going to do this incredible [37:16] thing for you where I'm going to give [37:18] you the power of a garbage collected [37:21] language with the performance of manual [37:25] memory management. And that's amazing [37:28] and it's really important and it's [37:30] important for different people for [37:31] different reasons. So, there is a a [37:33] demographic probably a big demographic. [37:35] There are a lot of folks for whom it's [37:36] like the great thing about that is [37:37] memory safety. That I can't overwrite [37:40] anyone else's memory and that is really [37:43] important. I I'm not going to I'm not [37:45] going to have, you know, B0s that have [37:47] got the wrong size on them, mem copies [37:50] that have the wrong size on them. I'm [37:51] not going to One of the things I love [37:53] about Rust among other things is Rust [37:55] there are lots of kinds of data [37:56] corruption that Rust prevents. One thing [37:59] is when you initialize a structure, you [38:00] have to initialize all fields. And on [38:02] the one hand you're like, "Hey, Rust, [38:03] get out of my grill. Like why do I have [38:04] to initialize all of these?" And then [38:05] Rust should be like, "What's your [38:07] problem? Like why don't you want to [38:08] initialize this? Like you're the one [38:09] with the problem. LIKE YOU INITIALIZE [38:11] THIS. GIVE ME A VALUE. Like if it's me [38:12] zero, put a zero there." Fortunately [38:15] like the Rust compiler doesn't like get [38:16] that belligerent with you. Um, but [38:19] uh in FACT ACTUALLY WHAT'S REALLY FUNNY [38:20] is like the Rust compiler has like such [38:22] a fearsome reputation for somewhat [38:25] legitimate reasons just because the the [38:26] when when the borrow checker is angry [38:28] with you, um it it can feel like I I [38:30] don't know what to do. [38:32] So, the as a result, the Rust compiler [38:34] totally overcompensates and is like the [38:36] friendliest compiler I've ever [38:38] encountered on cuz Rust is like, "Oh, I [38:40] don't think you meant this. I think you [38:41] might have meant this thing." And like [38:43] here's exactly what I like look I got [38:44] all these ASCII colors where I'm showing [38:46] you exactly where it is. And you're [38:48] like, "WHY ARE YOU BEING so helpful? [38:50] This is great." It's like, "Just [38:51] remember this later, okay? Just like [38:53] remember my helpfulness at your darkest [38:56] hour because your darkest hour is [38:57] probably coming. Um [38:59] So, I but I actually do I love I love [39:01] love love the the the um the Rust [39:03] compiler. And the reason this is [39:04] important is for me memory safety isn't [39:08] as big of an issue because and I this is [39:10] going to sound I don't want to sound [39:11] arrogant, but this is like [39:13] I can write correct C. I And now I've [39:15] gotten it the hard way. It's not [39:17] something that I would recommend for [39:18] everybody, but like I can write correct [39:20] C. Um, and I've written a lot of correct [39:23] C. I can write C that frees memory [39:25] properly, that that that doesn't [39:27] overwrite and that basically doesn't [39:29] suffer from memory corruption. Um, the [39:32] uh arithmetic overflows are a lot harder [39:34] actually. Rust actually helps you that. [39:35] That one I'm not going to make the claim [39:36] because I've been those are super hard. [39:38] But the on memory safety I can I can do [39:42] that. [39:43] But Rust is still valuable for me [39:46] because what it what it I can do that [39:48] because I'm controlling heaven and earth [39:50] in my software. It makes it very hard to [39:54] compose software. [39:56] Because even if you and I both know how [39:58] to write memory safe C, it's very hard [40:01] for us to have an interface boundary [40:02] where we can agree about who does what. [40:05] So, you know how to free memory properly [40:06] and I know how to free memory properly. [40:08] Oh, but I didn't realize that in that [40:10] condition I was supposed to free the [40:11] handle that you'd given me, not you. Or [40:14] I there it's just it's a [40:15] miscommunication. And it's really easy [40:18] to do it. C is very very hard to compose [40:20] properly. [40:21] Um and one of the things that we do a [40:23] lot actually to compose C is we use [40:25] something called intrusive data [40:25] structures where you embed nodes of a [40:28] data structure in your larger structure, [40:30] which is great for C. It's terrible for [40:32] Rust. We'll talk about that in a bit. [40:33] Um [40:34] but Rust [40:36] allows you to actually compose, truly [40:39] compose. [40:40] Because you don't need to worry about [40:42] the borrow checker has assured that you [40:45] that the contracts for who owns what are [40:48] implicitly abided by. And that it's it's [40:50] amazing. [40:52] Um it really is amazing. Um because it [40:54] it you can build these really [40:55] sophisticated primitives. And in my [40:58] experience, and again, you know, [40:59] everyone's mileage may vary and I'm not [41:00] trying to uh you know, I had this blog [41:03] entry comparing the performance of Rust [41:05] and performance of C for my application. [41:07] And I had like these like eight [41:08] paragraphs trying to explain how this is [41:11] not trying to draw broader conclusions [41:13] about the performance of C versus the [41:15] performance of C. But of course it's [41:16] like it's the internet. The first [41:17] comment is like, I can't believe this [41:18] jerk thinks that Rust is always faster [41:20] than C. It's like, that's not what I'm [41:21] saying. For the software that I wrote [41:24] the the I had a body of software that [41:26] was in C. It was a C and end node. I [41:29] rewrote it in Rust. [41:31] Um and when I rewrote it in Rust, my [41:33] expectations were Well, I should say [41:34] when I was doing that, [41:36] uh I was almost inhibited by the fact [41:38] that I was effectively reporting my C [41:41] because I tried to do some of the things [41:43] the same way I had done them in C. And [41:45] in particular, in C we have multiple [41:47] ownership all the time. [41:48] So, when you a doubly linked list is [41:50] actually two linked lists, right? And a [41:53] given object is actually owned by two [41:55] things. The thing [41:57] your your next thing and your previous [41:58] thing. They both think they own you. [42:01] Rust does not want to do that. [42:02] I mean, you can do it. And there's a [42:03] great blog entry about a series of blog [42:06] entries on so you really want to write a [42:08] doubly linked list in Rust. You really [42:10] don't want to do it. Um and what you [42:12] want to do is with Rust and I think and [42:14] and the whole Rust community I think has [42:15] been very up-front about this bargain [42:17] that you're buying into with Rust is you [42:18] want to rethink your problem and [42:21] implement it in a way that makes your [42:23] ownership clear. [42:25] So, if you convey your ownership [42:27] clearly, Rust will give you very [42:29] high-performing code as a result. It's [42:31] like, okay, I can buy that. So, I had [42:33] done so many things. I've got 5 minutes [42:35] until we're done. [42:37] Woo boy, that went fast. Um [42:39] all right. We're going to pick up the [42:41] pace. [42:42] Um [42:43] oh, that that that's for questions. I'm [42:44] not taking questions. [42:46] That's fine. All right. Woo. False [42:48] alarm. I've got a so [42:50] False sense of time. Um so, the um in [42:53] but in my experience, the so I was going [42:56] to implement this thing in Rust. I'd [42:57] made all these compromises cuz I just [42:59] got to the point where I got so beaten [43:00] down. I mean, beaten down that's [43:02] too mild an expression. But I I got I [43:04] was bargaining. I was in the bargaining [43:05] phase. I'm like, I will do anything to [43:06] get this working. Like I need to I get [43:07] this working cuz I was trying to make it [43:08] high-performing and do what it done in [43:10] C. I'm like, I need to stop doing that. [43:12] I just need to get this damn thing [43:14] working. So, I got it working and I'm [43:16] like, I don't think it's going to have [43:17] anywhere near the performance of C [43:19] because I the way I did it was so dumb. [43:21] I mean, it's like I just did it like the [43:23] knuckleheaded way. [43:24] Um I implemented in particular I [43:25] implemented a doubly linked list [43:27] effectively by having hash tables of [43:29] UUIDs where I'm like constantly like, [43:31] you you want to get your next one? [43:32] You're like going to look up this UUID [43:34] effectively, this timestamp to get the [43:35] next thing. And I'm like, this is [43:36] terrible. Um this is going to be this is [43:38] going to perform abysmally. And it [43:39] outperformed my C. [43:42] It's like, outperformed? Like, what the [43:43] hell? And so I'm like, all right, I got [43:45] to take that apart. What actually [43:46] happened? What actually happened is and [43:48] it gets right to the composability [43:50] of Rust. What actually happened is this [43:52] particular code spends all of its time [43:55] searching in a data structure in a [43:57] sorted data structure. Which in my C [43:59] implementation was an AVL tree for a lot [44:01] of good reasons. We use AVL trees all [44:03] over the place because they're super [44:05] easy to use. You can embed an AVL node [44:06] in your broader C structure. You can [44:08] have a given C structure that's on 15 [44:10] different AVL trees at the same time by [44:12] embedding different AVL nodes. We've got [44:14] a library that gets it right. And you [44:16] you know, and so on and so forth. So, [44:17] it's there's a lot of good reasons to do [44:18] it. But an AVL tree is actually not it's [44:21] good, but it's not the best data [44:22] structure to use. And in particular, [44:25] Rust uses B-trees. [44:27] And B-trees, yes, from databases, [44:30] Rust implements data structures it just [44:32] a it's binary tree structure is a [44:34] B-tree. As it turns out, in in this era, [44:38] the memory hierarchy being what it is, [44:40] cache line sizes being what they are, [44:43] a B-tree is actually a better [44:44] implementation. And the implementation [44:46] for B-trees in Rust is [44:48] total rocket science. And there's some [44:49] great blog entries describing the [44:53] kind of the iteration on that. But it's [44:55] amazing stuff. So, a lot of this is a [44:58] lot of the performance delta here. And [44:59] this is the this is runtime on the on [45:02] the Y axis. Um [45:04] and the the rectangles kind of varies [45:07] the data set size effectively for this [45:09] particular application. Um a lot of the [45:11] win on the one hand was B-trees. And [45:14] it'd be easy to be like, wow, that's [45:15] just like that's B-trees, not AVL trees. [45:18] It's like, yeah, but the reason I could [45:20] use a B-tree and not an AVL tree is [45:23] because of that composability of Rust. [45:25] I like a B-tree in C [45:28] is gnarly. [45:29] Um and cuz a B-tree like is just I mean, [45:32] talk about intrusive. It's very [45:33] intrusive. It's very kind of up in [45:35] everything's underwear. And the and [45:37] someone of course in the internet is [45:38] like, well, you should go and use this [45:39] B-tree implementation. You look at the [45:40] B-tree implementation like, this is [45:41] rocky. And if this thing has if there's [45:44] a memory corruption issue in here, I do [45:45] not want to have to go find it. So, it's [45:47] the it I would still use an AVL tree in [45:50] C even though I know I'm giving up some [45:52] small amount of performance. But in [45:53] Rust, I get to use a B-tree, which is [45:54] really amazing. Now, beyond ownership, [45:56] there's tons of features in Rust that I [45:58] love. Um algebraic types [46:01] allow for amazing error handling. Um so, [46:04] algebraic types say, you know, this is a [46:06] thing that can be a this or a that or [46:09] this other thing. It can be That's only [46:10] one of these things. It's basically like [46:11] a union, but a C union, but with really [46:14] first-class support. And the great thing [46:16] about Rust is if you return a result [46:19] that it that that is one of these [46:20] things, you need to figure out which one [46:23] of these things it is. Is this an error? [46:24] Is that it Is this an actual thing that [46:25] I can unwrap? Um and it's beautiful. It [46:28] really results in in terrific error [46:29] handling. We can go into more details um [46:32] not now, but you can find that online. [46:33] There's a terrific chapter um in in the [46:35] book um on on error handling. Um [46:38] hygienic macros. Um [46:40] I so didn't have hygienic macros in C [46:44] that I I kid you not, I did not know how [46:46] to spell hygienic until I wrote a blog [46:49] entry on this like 2 months ago. Like I [46:51] literally did not know how to spell [46:52] hygienic as a C programmer. It's like [46:54] that extra I. What's that doing in [46:55] there? Um but the um hygienic macros are [46:58] just a macros that can actually get in [46:59] the abstract syntax tree are a [47:01] first-class primitive in Rust and it's [47:03] amazing. Um you can do crazy powerful [47:06] stuff um in macros. Um crazy powerful [47:10] safe stuff in macros. Uh the the foreign [47:13] function interface, which is so [47:14] difficult with GC languages, right? With [47:16] GC a point I should have made earlier [47:17] when we talked about GC languages, part [47:18] of the reason they are an ill fit for an [47:19] operating system is because it's very [47:21] hard to have C code that lives beside [47:24] GC'd code and can actually share objects [47:26] across them. Um the foreign function [47:28] interface in Rust is very well thought [47:31] out, very full, and supports I I kind of [47:33] invented this term. I'm sure there's a [47:34] better one out there. Full duplex [47:36] integration in that you can take C and [47:38] drop it into Rust or Rust and drop it [47:41] into C. And that is really interesting. [47:43] And there's a lot of I'd say community [47:46] interest in that. And one of the also [47:47] the neat things about Rust is that [47:49] there's an unsafe keyword that you can [47:50] use for unsafe operation. It's not [47:52] arbitrary operation. You can't just [47:54] like, you know, have the unsafe keyword [47:56] and then just like go to town. Um but [47:58] you [47:59] So, there are certain things that get [48:00] relaxed [48:02] um when you have this unsafe keyword, [48:03] which allows you to overrule the safety [48:05] guarantees just ever so slightly where [48:08] you need to, though with obvious peril. [48:10] Um but in particular, that allows you to [48:11] build composable data structures that [48:12] can be used by other things. So, [48:14] um and there's a ton of other great [48:16] features. Um but also and I um with um [48:19] with um [48:21] introduction of Stephen in the room, uh [48:23] it's a terrific community. It's a [48:24] terrific community if you're not in the [48:25] room. Uh I didn't even imply that it's [48:27] only terrific community if you're in the [48:28] room. But um [48:30] um the the Rust community is actually [48:31] really amazing. So, if you are in [48:34] I and I got to the point where like open [48:35] source communities just kind of give me [48:36] a throbbing headache um because it can [48:38] be really difficult to accommodate, you [48:41] know, such diversion opinions and [48:42] everything else. And I don't necessarily [48:45] know that it's any less, you know, it's [48:47] that it's any less difficult in Rust, [48:48] but I think that the the values of Rust [48:51] tend to drive conversations. That there [48:53] there is it feels like there's a keel in [48:56] the water. I mean, it's still the [48:57] internet. So, of course like it's still [48:59] like there's still bad behavior and [49:00] everything else. But the But [49:02] But it just it it feels so much more [49:05] constructive because the values for Rust [49:08] are so clear. Um so, I really um there's [49:12] a lot of value to it. So, okay. [49:14] Operating systems in Rust. Finally. The [49:16] the main event. So, I would say first of [49:18] all, if the history of OS development [49:20] teaches us anything, [49:22] it's that the runtime characteristics of [49:25] an operating system trump its [49:27] development time. [49:29] Nobody cares how long it took you to [49:32] develop it. [49:33] And that it was easy to develop or hard [49:35] to develop. It's like, no one cares. [49:37] Um they care if it works and they care [49:39] how fast it is. [49:40] That's what they care about. [49:42] And that when I say they, I mean you. [49:43] That's what you care about. You who do [49:45] not develop OS kernels. And that's what [49:46] you should care about. It's like that's [49:47] what we want. We want it to be fast and [49:49] we want it to be correct. And if it was [49:51] easy for them to implement, great. If it [49:53] was hard for them to implement, well, [49:54] okay, is it fast and correct? Yes, it's [49:56] fast and correct. Like, well, okay, [49:57] that's someone else's problem. Um and [49:59] that's fine. Um but we have to [50:01] understand that those are the loadstars. [50:03] So, we can't actually throw those things [50:05] out um if we're going to reimplement an [50:06] OS in Rust. Uh and we have to remember [50:08] that structured languages replaced [50:10] assembly because they performed as well. [50:11] They hadn't performed as well, they [50:12] would not have replaced them. Um and [50:14] every operating system has assembly [50:16] somewhere in it. Um but with but with [50:18] Rust and I do think that that that it's [50:20] important to realize that Rust does [50:22] represent a new opportunity um that we [50:24] have not had in [50:26] 30 years, more, which is a new [50:28] programming language that actually is a [50:30] candidate for this stuff. So, [50:32] um a first attempt the first one that [50:34] I'm aware of is Alex Light's Redox. Um [50:36] this was a teaching operating system [50:38] called Wenix [50:39] um that was reimplemented in Rust as an [50:42] undergrad thesis. The undergrad thesis [50:43] itself is a great read [50:45] um where Alex goes into a bunch of the [50:47] challenges that he ran into as he [50:49] implemented this. The biggest challenge [50:51] was that Rust then did not allow for any [50:54] allocation failure. So, if all [50:57] allocations effectively block or work or [51:01] death. Um like we all like the why live? [51:04] Let's just all blow our blow ourselves [51:06] up. Which is fine. I mean like I that's [51:08] actually not necessarily a bad attitude, [51:10] especially in New Zealand. But in the [51:12] operating system, we can't do that. In [51:13] the operating system, we need to be able [51:16] to know uh if this allocation is not [51:18] going to work and if we are out of [51:20] memory, I need to go do some work to [51:22] free up memory. It's like, well, how are [51:24] you can you like how can an application [51:26] possibly free up memory? It's like, [51:27] well, this application is the operating [51:29] system. So, we've got ways of freeing up [51:31] memory. LIKE WE I I WILL THROW out dead [51:33] processes until we've got memory freed [51:35] up. Um which we don't want to do. I mean [51:37] this is a definite point of divergence [51:39] between Linux and other kernels. It's [51:40] like we actually don't want to throw out [51:42] dead processes, but there are lots of [51:44] things that we can go do to free up [51:46] memory in the system because we control [51:48] all the memory in the system. Um so, [51:50] it's very hard to run a modern operating [51:52] system with SmartOS. SmartOS is very [51:54] hard to run SmartOS out of memory. Um we [51:56] but we have to be able to still [51:58] potentially fail allocations in some [52:00] isolated cases. So, one of the things we [52:02] did in the kernel a long time ago is we [52:03] introduced the notion of sleeping [52:04] allocations. Most allocations are [52:06] sleeping allocations that can't fail. [52:08] Some allocations are no sleeping [52:09] allocations, which means they don't have [52:10] a that that they can accommodate [52:11] failure. Um and we need the ability to [52:14] implement something like that. Um this [52:16] is an area that is um was made much [52:18] better with the addition of global [52:20] allocators, which was added uh it was [52:22] like 1.27, something like that. 1.26, [52:24] something like that. Um [52:26] uh but the dealing with memory [52:27] allocation failure is still very much an [52:29] active issue and settled [52:31] area for Rust. Uh I would check out um [52:33] the I'll give you the RFC number that [52:34] you can check out there. These RFCs, if [52:37] you do real Rust development, you're [52:38] going to end up in the RFCs pretty [52:39] quickly as you go to understand because [52:41] things are new enough that you're going [52:42] to be even if you're using things that [52:44] are there, you're going to inevitably [52:46] come back to the design discussions on I [52:48] and those design discussions can be [52:49] really interesting and actually [52:50] uplifting. And one of the things that [52:51] the the way Rust does uh raw literals, [52:54] for example, which is with R the the [52:56] character R, um a pound sign or an [52:59] octothorpe or a hashtag, depending on [53:00] what generation you are. Um which I [53:03] swore I'd never say hashtag, but now [53:04] like my kids always say hashtag for [53:05] everything. So, it's like it's not a [53:06] hashtag, it's pound sign, whatever. The [53:08] R and then hashtag, whatever. Uh some [53:11] number of hashtags and then your literal [53:13] and then the same number of hashtags as [53:15] as open literal. But just looking at [53:18] that discussion, a really interesting [53:19] discussion because everyone's talking [53:21] about how all these different systems do [53:23] it. Um and actually the absence of raw [53:26] string literals is a real problem for me [53:27] in JavaScript um and in C, too, as well. [53:30] Um and it's actually it's a [53:32] many many things that I love in Rust. [53:34] But that design discussion is a really [53:35] interesting and it's a super small [53:35] issue. These RFCs will get you into [53:37] those design discussions. This [53:38] particular area is an area that's still [53:40] unsettled with respect to allocation [53:41] failure, but is settled with respect to [53:44] global allocators, which actually allow [53:45] you to do a lot of it. [53:47] Um since 2015, there've been a there's [53:49] been this kind of explosion of little [53:51] operating systems in Rust. Um if I [53:53] forgot yours, I'm really sorry. Uh I [53:55] tried to get them all. Uh I tried to get [53:57] the ones that still compile. So, you got [53:59] Redox, which is kind of the big one, [54:00] then Tifflin and Talk and Intermezzos [54:01] and Rust OS and Quilt OS and Rocks and [54:04] then Philip Oberman's Blog OS. Philip [54:06] Oberman, please give this thing a name. [54:08] Call it whatever you want. You got to [54:09] name something. It's like important [54:11] enough to name it. I don't want to have [54:12] to call it Philip Oberman's Blog OS. So, [54:15] just give it a different name. It's a [54:16] great thing you're doing. Like, please [54:18] give it a different name. It's Blog OS, [54:19] I guess it's what it is. So, um some of [54:21] them are teaching systems. Um [54:23] Intermezzos, Blog OS. Some are [54:24] unikernels like Quilt OS that all that [54:26] only are going to allow for Rust [54:28] programs. They [54:30] kind of limit in that regard. Some are [54:31] targeted IoT like Talk. Um the the [54:34] attribute that all these systems have in [54:36] common is they are all denovo. They are [54:37] all written from scratch. [54:39] Which is great on the one hand because [54:42] it's Rust allows us to explore these new [54:45] ideas and build these operating systems. [54:47] And all these efforts should be [54:48] encouraged and especially the teaching [54:49] ones, all terrific. It's great. [54:53] Two buts on that. One is it means that [54:54] you're forsaking Linux binary [54:56] compatibility. Um and [54:58] that just means that a lot of the [54:59] world's software is not going to work. [55:00] And which is fine. I mean, I honor that. [55:02] I've experimented with that approach for [55:04] many years. I [55:05] For me, it doesn't work, but you know, [55:06] it's I I like having the world's [55:08] software work. And I say that as a Unix [55:10] operating system that's very very very [55:11] very very close to Linux. I'm serious. [55:13] But like no, no. So, so we've done all [55:14] sorts of things to allow for Linux [55:17] binary compatibility. Justin talked [55:18] about some of those in his talk. That [55:19] Linux binary compatibility is a really [55:21] long tail. It's really brutal. It's very [55:22] hard to do. You end up in a lot of [55:24] really ugly gray areas. Um and the um [55:28] again, I'd refer to Aiden's talk um for [55:31] some of the gray areas that they ran [55:32] into and we've run into a bunch. Um so, [55:35] that you get all those problems, which [55:36] you don't want to have to go resolve. [55:37] You do not want to have to deal with [55:38] vfork and sigchild semantics, trust me. [55:40] Uh like vfork semantics is terrible. [55:42] Sigchild semantics is terrible. Vfork [55:44] plus sigchild semantics, like [55:45] unspeakable semantic super fun site that [55:48] no one should have to deal with. Um and [55:50] they also means that they are all [55:51] fighting second system syndrome. They [55:53] are all fighting like, well, as long as [55:56] like the train is leaving, let's get [55:58] let's back up the truck and let's do a [55:59] new file system and let's do a new way [56:01] of thinking about devices and let's and [56:03] that can be really productive and [56:04] interesting, but that be paralyzing. So, [56:08] that that's that's a challenge. Um the [56:11] it's also a challenge like what is the [56:13] advantage of rewriting things that are [56:15] actually working code in Rust? And I you [56:17] know, I don't know. Um because the [56:19] safety argument just doesn't carry as [56:21] much weight for kernel developers, not [56:22] because the safety argument isn't really [56:24] really important. It's just because [56:26] like it's safe. Because when it's not [56:27] safe, like it blows up and everyone gets [56:29] really upset. We figure out why and we [56:30] fix it and we develop a lot of great [56:32] tooling to not have these problems. So, [56:34] it it it it has to have more of of a [56:36] grab than just the safety. The [56:38] performance is great. There are things [56:39] about it that are great. But this is [56:40] working code that we want to replace. [56:42] Like, why do we want to replace this [56:42] working code? Especially with respect to [56:44] the kernel, which has multiply owned [56:47] data structures everywhere. Everything [56:50] is on many lists. I mean, there are [56:51] things that are where you've got these [56:53] nice kind of like hierarchies of of [56:55] objects, but most things don't actually [56:57] have that. And yes, some of those are [57:00] semantic sewers and should be [57:01] reimplemented. Some of them there are [57:02] some endemic problems. And you look at [57:04] like thread dispatch and so on, there [57:05] are some endemic problems. And then [57:06] they're like, okay, well, we're just [57:07] going to work around this with unsafe. [57:08] Like, okay, but like every time you use [57:10] an unsafe in your Rust operating system, [57:12] it's like you're losing a little bit of [57:13] the advantage of Rust. But you know, [57:15] whatever. Um so, I think an OS kernel, [57:18] despite, you know, the historic appeal [57:20] um and the superficial fit, I don't know [57:23] that that's worth it. I don't know that [57:24] that's the good that's the right first [57:26] approach. Um I mean, again, these other [57:28] operating systems should be encouraged. [57:29] I think it's great. But I think for the [57:32] OSes that we're going to use for the [57:33] rest of our lives, I'm not sure that [57:35] that's a good that right first approach. [57:36] Maybe we'll get there eventually. I [57:37] don't want to rewrite ZFS in Rust. [57:39] That's what it boils down to. Like, I [57:40] wake up in a cold sweat when I think [57:41] about rewriting ZFS in Rust and go to [57:43] like go look at the ZIO pipeline and [57:45] tell me you want to rewrite that thing [57:46] in Rust. It's just like, no thank you. I [57:48] mean, the the Rust compiler The Rust [57:50] compiler would slay me. Like I it would [57:51] win and I and I would lose. [57:53] But let's look at hybrid approaches. So, [57:55] cuz I think a hybrid approach is really [57:56] the right approach here. Um one is to [57:59] have Rust in kernel components. So, I [58:01] love the fact that Rust can interoperate [58:02] with C. So, one hybrid approach is that [58:05] you retain your existing C assembly [58:08] based kernel, the way we we have had for [58:09] many years, and then you allow for Rust [58:13] based things to be developed. Rust based [58:15] drivers, Rust based file systems, Rust [58:17] based in kernel software. One of the [58:18] points that Justin made in in his talk [58:20] was that we want to have more in kernel [58:22] software. Well, this is a safe way of [58:24] doing it. Now, maybe the safety argument [58:27] is a big win because now it's like, hey, [58:29] more people can safely develop the [58:30] kernel code. Yes, fine. Existing kernel [58:32] developers are not going to write the [58:33] kernel, but having a safe Rust sandbox [58:36] inside the kernel, that might be [58:37] interesting. Um it would allow for an [58:39] incremental approach, which I think like [58:40] let's take that Unix BC approach. Um and [58:45] it allows Rust to be used for new [58:46] development where you're not trying to [58:47] kind of port existing C structure and [58:49] being burdened by the C implementation. [58:51] There's a prototype of this in FreeBSD. [58:52] Others are possible. I think we [58:53] definitely want to do this in SmartOS at [58:55] some point. Um [58:57] hybrid approach number two. Uh Rust OS [58:59] components. An operating system is not [59:00] just the kernel. There's a lot of other [59:01] stuff out there. There is utilities, [59:04] demons, service manager, device manager, [59:06] default management, the debuggers, etc. [59:08] Yes, systemd is a part of your operating [59:10] system, whether you acknowledge it or [59:11] not. Systemd is you know, you've got [59:13] these things that are part of your [59:14] operating system. These things are great [59:16] candidates for like if you were writing [59:18] systemd now, no question that it's in [59:21] Rust. If you're speaking for us, if we [59:22] were writing SMF now, no question in my [59:25] mind that that's in Rust. That is like [59:26] and in part because by the way, like let [59:29] me tell you something that most kernel [59:30] developers won't tell you. It's harder [59:33] to write user level software. [59:35] It's harder to write user level software [59:37] because more can go wrong. Like, do you [59:39] know a problem that I never have in the [59:40] kernel? A configuration file that [59:42] someone deleted. [59:43] I I That's just not the way we roll in [59:45] the kernel. [59:46] LIKE, WE DON'T HAVE THAT. LIKE, THE [59:47] CONFIGURATION file is the kernel. Like, [59:48] if you deleted me, I'm not here. IT'S [59:50] NOT MY PROBLEM. LIKE, you can't boot [59:51] anything. [59:52] I mean, I got lots of other problems. I [59:53] got like I got to boot this thing and [59:55] then I got like Intel introducing [59:56] vulnerabilities left and right. So, I [59:57] GOT LOTS OF OTHER PROBLEMS. I GOT PLENTY [59:58] OF PROBLEMS. [59:59] But I the I don't have the problems I [60:02] don't have distributed systems problems. [60:03] I don't have these kind of operational [60:05] failures. There are a lot of failures [60:06] that I don't have, and that's the stuff [60:07] that Rust is really good at. [60:09] So, that might be a really good [60:11] candidate for Rust. Um so, I think a lot [60:13] of those are are uh and especially as [60:15] the OS expands to beyond just an [60:17] operating system and consists of entire [60:18] cloud control plane. And then the the [60:21] third approach, um [60:22] one that I'm excited about at the [60:24] moment, um [60:26] is you know, there's there there is [60:27] another operating system out there, an [60:29] operating system that hasn't advanced [60:30] technologically as much um or at all, [60:34] that's basically still running glorified [60:36] DOS, [60:37] um but it's running all of humanity on [60:39] top of it. Um and that is the this [60:42] absolute sewer of unobservable system [60:45] critical, mission critical software [60:46] called firmware. [60:47] And firmware has lots of problems. Um as [60:51] I've been as I'm fond of saying, like it [60:53] is it is humanity versus firmware, and [60:55] everyone needs to pick a side. [60:58] Um [60:58] and [60:59] I think that that you look at some of [61:01] the challenges of the firmware, it's [61:02] like, "Oh, man, that is a that's a good [61:04] fit for Rust." Yes, some of the embedded [61:06] challenges that we have, but those seem [61:09] much more surmountable. I don't think [61:10] you're going to have like the the [61:10] multiple ownership disasters that you [61:12] have in a kernel. You don't have the [61:14] same expectations around binary [61:16] compatibility. You don't have to go [61:17] reinvent ZFS. I mean, it's like it's a [61:18] smaller kind of a thing. And so, when I [61:20] look at like OpenBMC, I view OpenBMC as [61:23] on its knees with tears streaming down [61:26] its face begging for someone to please [61:28] rewrite me in Rust. Because if you're if [61:30] we're going to have the [61:31] the [61:32] a the BMC that's going to hang out a [61:34] socket over the internet, God forbid, or [61:37] even over the network, I want that thing [61:39] to be in Rust. I don't want that to be [61:40] in C++. Um and there's a lot of stuff [61:44] like that. So, I think that is where I [61:46] want to see a world right now. Firmware [61:47] is the proprietary software that we're [61:49] all forced to run. I want to see a world [61:51] we get to open firmware, and I want to [61:52] see a lot of that written in Rust [61:53] because it's a much, much, much better [61:55] foundation for humanity. We'll do [61:56] humanity a huge favor. [61:58] So, looking forward. I mean, I'm very [62:00] bullish on Rust. I'm very excited about [62:01] Rust. It's something we really haven't [62:04] seen in a long time. It's a true [62:05] alternative. Um we can use it deep in [62:08] the stack. I think it is a I think we [62:10] can use it in firmware. I think we can [62:11] use it in operating systems. I just [62:12] think that that practically it's going [62:14] to be really, really hard, but I think [62:16] we can definitely use it in the way we [62:17] extend the kernel. And the beauty of [62:19] Rust is that it allows these hybrid [62:21] approaches. That the beauty of Rust is [62:23] its ability to to interoperate, [62:25] cooperate with native systems, with [62:28] C-based systems allow for vistas to open [62:31] up. Um and for us in system software, it [62:35] is a very, very exciting time. With [62:37] that, thank you very much. [62:46] I have 2 minutes for questions. [62:49] Any questions? Yes. [62:55] Um the the No, no. I don't hate I First [62:57] of all, I've never felt I have only [62:59] tried to bargain with the borrow [63:00] checker. I have never Even when it is [63:03] absolutely thrashing me, I know it is [63:05] out of its own love for what it's trying [63:07] to do. And And I think actually with [63:09] with Rust, it's important to get over [63:10] that hump. Like, the one thing I would [63:12] advise if you have not implemented Rust, [63:15] please, for the love of God, don't start [63:17] with a doubly linked list. In fact, I [63:19] wouldn't start like it's very often that [63:22] you like I've got like, "Oh, I've got [63:23] like the Sudoku thing." Or I've got the, [63:25] you know, the the this solver or that [63:27] thing that I've re-implemented in every [63:28] language. I'm going to go implement it [63:29] in Rust to try it on this side. Don't do [63:31] that. It's a bad idea. It's a just [63:34] because if you haven't seen what the [63:37] borrow checker can do for you, [63:39] the first time it begins to to uh [63:43] to fight with you, for lack of a better [63:44] word. The first time it tells you In [63:45] particular, it will tell you like, "Hey, [63:47] um I think you want to make that mute." [63:49] You're like, "Okay, I'll make that [63:49] mute." And then it's like, "Oh, you want [63:50] to make that a reference." Like, "Okay, [63:51] I'll make that a reference." Like, "No, [63:52] you don't want that to be a reference at [63:52] all." LIKE, "NO, NO, NO. THAT DOESN'T [63:54] WORK AT ALL." IT'S LIKE, "I JUST did [63:55] what you told me to do." And now I've [63:56] GOT LIKE SO MANY MORE ERRORS. But and [63:58] that's when the the tears begin to well [64:00] up. Um don't do that. And one of the [64:02] things that I did that I would really [64:03] recommend um is So, there are the the [64:05] two books on Rust are terrific. There [64:07] there's I don't know if you guys call it [64:09] triple. I call it triple or turple. [64:11] Turple, all right. The the Rust [64:12] programming language. Um so, um Steve [64:15] Klabnik and Carol Nichols and the entire [64:16] community wrote the that, which is [64:17] great, and you want to have that [64:19] reference. Um and then the other one is [64:21] uh Is it what? Systems programming in [64:22] Rust? Programming Programming Rust. Um [64:25] and the O'Reilly book. And I think that [64:27] I like having both of those books side [64:28] by side. One of the things that I really [64:30] like about Programming Rust is there is [64:31] a fully worked example, [64:33] and I just like typed that in when I [64:36] first got into Rust. Like, I hadn't done [64:37] that literally since Enter magazine from [64:39] back in the day. Like, I don't know if [64:40] you're as old as I am, but like you're [64:42] like opening up the back of Enter [64:43] magazine like typing in your basic on [64:45] the C64 or whatever. Um [64:47] it was really valuable to do that [64:49] because you'd have typos, and then you'd [64:51] fix them, but they'd all be very gentle [64:52] typos. And you know that you've got [64:54] something that's correct. And by the [64:56] time even I gotten through that simple [64:57] exercise, I'm like, "Okay, I see the [64:58] value." So, when the borrow checker [65:00] began to torment me, I'm like, "I know [65:02] this is out of love and not hate." [65:04] Um because it can feel confusing. Other [65:06] systems don't have ownership. [65:08] And the this is a novel thing about the [65:10] system. I think the community does a [65:11] great job doing this, but just to [65:12] emphasize this, with Rust, you need to [65:15] learn it. And if you are of the age [65:17] where you're learning lots of stuff, [65:19] that's easy. [65:20] And if you're of my age, where it's like [65:23] I've been able to kind of like wing it [65:25] by just like pounding stuff out for a [65:26] while, it's like I need to go back in [65:28] time and actually buy the books, sit [65:31] down, read the book in a quiet place, [65:33] really understand it, read it again, [65:35] read another source. [65:37] Um and but Rust rewards that. It rewards [65:40] that education a lot. [65:42] Long answer to the question. [65:43] Yes. [65:45] At the beginning you said that kernel [65:47] illness for Multics is where PL/I [65:50] compiler did not exist for [65:52] 3 years. [65:53] Rust [65:55] Do we now have [65:57] a Rust compiler and optimizer compiler [66:00] where this [66:02] at at the same level [66:03] where GCC and clang [66:06] now do for C So, [66:08] I'm a latecomer. So, I For me, yes. [66:12] Because I haven't lived in this kind of [66:14] tortured past, where I think there are [66:16] people that used Rust 2 years ago were [66:18] like, "Oh my God, never again." And [66:20] because they, you know, we had whatever, [66:22] because they they was early. For me, [66:24] it's been great. And it performs well. [66:26] It generates I think the incremental [66:28] comp When was incremental support [66:32] Early this year. So, like that's not [66:33] that long ago. Like, to me that's always [66:34] been there. So, it's like, "Now, like [66:36] why is everyone complaining about the [66:37] compile times of Rust? Like, this is [66:38] great." It's like, "Because like just [66:40] before you showed up, it was a serious [66:42] problem." That's why it was So, like [66:43] things like that, like Rust is really [66:46] solidifying, I think, quickly. To me, I [66:48] would not hesitate to use it for [66:49] production software. I mean, it's cuz [66:50] I'm ignorant of the very recent past. [66:52] But uh you know, whatever. Um I would [66:54] not I think it's ready. [66:56] Yes. Is there a benefit to waiting a [66:58] little longer, say another year? Sure. I [67:00] mean, I I think that there is a benefit [67:02] and I'll tell you a a couple of the [67:03] benefits will be that [67:06] this is a I think Rust is a big deal for [67:08] system software, and I think waiting for [67:11] another year is going to give you that [67:12] many more resources to learn from. It's [67:14] going to be the documentation is going [67:15] to be that much better. The crates are [67:17] going to be that much better. You're [67:18] going to be that much less likely to get [67:20] into the RFCs. I mean, you're going to [67:22] have non-lexical lifetimes in a year, [67:24] right? Okay, December 6th. December 6th, [67:27] perhaps of 2025, but but [67:31] you're going to And if you use Rust [67:33] after non-lexical lifetimes integrates, [67:36] you're not even going to realize what [67:38] the fuss was about borrowing checking. [67:39] You're like, "Why is everyone like [67:40] complaining about the borrow checker? [67:41] I've never had a problem." It's like, [67:42] "Because you Right now um Rust really [67:45] only uses lexical lifetimes, and there [67:47] are times where like, 'No, seriously, [67:49] Rust, I swear I'm done with this. Like, [67:51] look Like, LOOK AT THE CODE with me, [67:53] Rust compiler. Like, we're done with [67:54] this.'" And it's like, "Well, no, no, [67:55] no. I only use lexical lifetimes." When [67:56] you have non-lexical lifetimes, which is [67:57] in nightly, and you can you can enable [67:59] it uh optionally. December 6th is going [68:01] to integrate December 6th. [68:03] WE'RE SHIPPING IT ON DECEMBER 6TH. [68:05] I DON'T CARE HOW BROKEN IT IS. WE ARE [68:07] PORTING it on December 6th. [68:19] Right. [68:21] That's going to make a big difference. [68:22] The And And again, that's going to make [68:24] a difference that you literally won't [68:26] even be aware of. You will just not [68:28] understand what the fuss was about about [68:30] the borrow checker. [68:31] I think. I mean, I don't know if you [68:32] guys agree with that, but I think I [68:33] think I think [68:34] I mean, it's going to make a big, big, [68:35] big difference. [68:38] Right. So, and and like but it gives you [68:41] high quality the high quality artifacts [68:42] and everything else. So, yeah, I think [68:43] there is there's I mean, this is not [68:45] like a hey, do this I mean, I honestly [68:46] think Rust is going to be around [68:47] forever. I really do. I think this is [68:48] like you this is the formation of [68:51] ancient Greek. So, I mean, there's no [68:52] rush. You got thousands of years. I [68:54] mean, [68:55] there's no like, you know, take your [68:56] time. Um [68:58] and there and it we it is going to get [68:59] better, but I also think if you've been [69:01] waiting because things are are too [69:03] crazy, you you don't need to wait that [69:05] much longer or don't need to wait at [69:06] all. [69:09] All right. [69:10] With that, [69:12] I think we're done. Thank you very much, [69:13] Justin.