Oxide Takes On Enterprise Server Bloat With Rust and Open Firmware

Stanford Online

Traditional server makers leave enterprises running PC-era hardware with opaque firmware, forcing buyers back to physical racks that integrate open software down to the silicon.

AI-assisted summary of Stanford Seminar - The Soul of a New Machine: Rethinking the Computer — original source. Check the original for full context.

Source published: 2020-03-04

Key ideas

Enterprise data center hardware has largely stagnated as rebranded commodity PCs, whereas hyperscalers have moved to custom, rack-scale designs using DC busbars and centralized power shelves. (7:44)
Proprietary BMC firmware and legacy BIOS implementations introduce critical security vulnerabilities and telemetry black holes, making a clean-sheet open firmware model essential. (23:25)
Oxide Computer Company was launched to deliver an integrated, rack-scale computer targeting on-premises enterprise workloads with modern open-source firmware, Rust, and RISC-V. (61:45)
Cloud economics frequently compel scaling SaaS companies to repatriate compute workloads to on-premises infrastructure as public cloud hosting expenses directly hit cost of goods sold (COGS). (74:00)

Detailed notes

## Background Published by Stanford Online on YouTube on March 4, 2020, this Stanford Seminar features Bryan Cantrill, co-founder of Oxide Computer Company, delivering a talk recorded on February 26, 2020. The session outlines how mainstream server hardware has stagnated in a bygone era of PC architecture while hyperscalers have moved ahead, presenting Oxide's vision for a clean-sheet, rack-scale server machine built using open firmware, RISC-V, and Rust.

## TL;DR Bryan Cantrill traces the evolution of server computing to argue that modern enterprise servers remain glorified, fragile desktop PCs saddled with legacy baggage, opaque proprietary firmware, and insecure baseboard management controllers. While hyperscalers engineered custom rack-scale architectures to solve these problems, mainstream enterprises face steep operational hurdles and economic cloud lock-in without access to comparable hardware. Cantrill presents Oxide Computer Company's strategy to build a unified rack-scale machine that places the computer's boundary at the virtual machine provisioning API. He demonstrates how modern technological shifts—specifically Rust's memory safety, open-source firmware, RISC-V soft cores, and open EDA tools—make an auditable, vertically co-designed computer achievable.

## Atmosphere & style Delivered with Cantrill’s signature rapid-fire cadence, biting wit, and unbridled technical passion, the talk feels like an exhilarating manifesto delivered from the low-level systems engineering trenches. The mood balances laugh-out-loud irreverence—taking aim at legacy BIOS vendors, DOS baggage, and vendor spy-chip distractions—with a profound reverence for computer history and engineering lore. Watching it feels less like a dry academic seminar and more like a passionate call to arms for systems programmers ready to tear down decades of accumulated architectural compromise.

## Bulletin - [07:44](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=464s) Enterprise data center hardware has largely stagnated as rebranded commodity PCs, whereas hyperscalers have moved to custom, rack-scale designs using DC busbars and centralized power shelves. - [23:25](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1405s) Proprietary BMC firmware and legacy BIOS implementations introduce critical security vulnerabilities and telemetry black holes, making a clean-sheet open firmware model essential. - [01:01:45](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3705s) Oxide Computer Company was launched to deliver an integrated, rack-scale computer targeting on-premises enterprise workloads with modern open-source firmware, Rust, and RISC-V. - [01:14:00](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4440s) Cloud economics frequently compel scaling SaaS companies to repatriate compute workloads to on-premises infrastructure as public cloud hosting expenses directly hit cost of goods sold (COGS).

## Key takeaways - [07:44](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=464s) Modern commercial enterprise rack servers remain fundamentally commoditized personal computers burdened with legacy PC baggage like emulated interrupt controllers and VGA ports. - [13:22](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=802s) Hyperscalers successfully decoupled from PC architecture by engineering rack-level architectures like OCP Tioga Pass with DC busbars and blind-mate sleds. - [17:20](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1040s) Proprietary firmware obscures vital hardware telemetry, such as discarding correctable single-bit memory error warnings prior to uncorrectable fatal panics. - [23:25](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1405s) Baseboard Management Controllers (BMCs) present massive attack surfaces because they operate Linux and U-Boot, master the host PCIe bus, and expose internet-facing network interfaces. - [26:19](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1579s) Host boot firmware should adhere to a minimal role: initialize the hardware, hand execution cleanly to the operating system, and exit memory completely without persistent runtime traps. - [29:22](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1762s) True rack-scale computer design requires unified co-design across top-of-rack networking switches, compute blades, and shared power shelves rather than simple server stacking. - [32:15](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1935s) The functional abstraction boundary of the modern enterprise computer has shifted to the API endpoint used to provision a virtual machine. - [36:50](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=2210s) Speculative execution vulnerabilities such as Spectre and Meltdown forced the computing industry to prioritize hardware roots of trust and verifiable firmware attestation. - [40:42](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=2442s) The maturation of open electronic design automation (EDA) tooling, HDLs like Bluespec and Chisel, and the royalty-free RISC-V ISA enables rapid custom silicon and soft-core integration. - [55:28](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3328s) Rust provides the deterministic performance and low memory footprint of C while preventing memory safety and integer overflow vulnerabilities at compile time. - [58:06](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3486s) A naive Rust implementation outperformed hand-tuned C by 35% primarily due to standard library cache locality advantages with B-tree structures over traditional binary search trees. - [01:00:07](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3607s) Rust binaries can easily fit within 512 KB to 640 KB of embedded FPGA RAM, demonstrating the viability of writing modern firmware and embedded operating systems safely. - [01:12:04](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4324s) High-performance virtualized enterprise workloads still necessitate x86 microprocessors, whereas RISC-V shines in bounded embedded service processor roles. - [01:19:20](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4760s) High-speed motherboard trace propagation and DIMM training physics demonstrate why transparent, auditable boot firmware is necessary for operational reliability. - [01:23:07](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4987s) Enterprise engineering teams increasingly rely on open software layers and containers to decouple their software stacks from proprietary public cloud service lock-in.

## Detailed walkthrough ### [00:11](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=11s) Setting Expectations: Not a Book Club

Bryan Cantrill, CTO of Oxide Computer Company, opens by clearing up a misunderstanding circulating online. His friend Jon Masters (formerly of Red Hat, currently at Nuvia) had tweeted a link to the seminar citing only the pre-colon title: "Bryan's gonna talk on The Soul of a New Machine." This prompted a noticeable portion of the internet to assume Cantrill was hosting a book club on Tracy Kidder's classic 1981 book, *The Soul of a New Machine*.

Cantrill clarifies that while he holds Kidder's work in the highest regard, today's talk is not a book review. Kidder's book documents the intense engineering effort behind the Data General Eclipse MV/8000 (code-named "Eagle") in the late 1970s and early 1980s. Cantrill uses the title as homage, seeking to channel that era's passion, fervor, and engineering depth into modern server hardware:

> "I love it; it's an amazing book if you haven't read it. It is truly... the Iliad or the Odyssey; it is the literature of our domain, but I'm only using it as a title."

[01:36](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=96s) The Genesis of the Server: The IBM 709 and Mainframe Scale

Cantrill introduces the IBM 709 as the candidate for the very first real "server." Although earlier machines existed, the 709 was the first system possessing enough raw capacity to support time-shared computing. Demonstrated in the early 1960s with the Compatible Time-Sharing System (CTSS), it marked the birth of server-side computing—a paradigm where computing happens at a physical distance, out of sight, reach, and direct sensory contact of the operator.

The sheer scale of these vacuum-tube systems highlights the rapid physical transformation of computers. Displaying data from an early comprehensive survey of U.S. computer installations, Cantrill calls attention to a specific IBM 709 installation at Hughes Aircraft. In an era before transistors eliminated massive heat and power overheads, these systems demanded enormous facilities:

* **Power consumption**: The IBM 709 pulled 205 kW (compared to a modern data center rack, which draws roughly 10 kW to 25 kW). * **Physical weight**: Approximately 33,000 pounds. * **Footprint**: 1,900 square feet for the machine itself, plus an additional 1,000 square feet dedicated solely to air conditioning equipment.

[04:10](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=250s) The Minicomputer Revolution: The PDP-11/70

Moving into the 1970s, the minicomputer reshaped the computing landscape by decentralizing access. Cantrill points to Digital Equipment Corporation's (DEC) PDP-11/70 as the archetype of this generation. Unlike sprawling mainframes, the PDP-11/70 brought compute capability closer to local working groups and departments. Its cultural resonance endures: Cantrill notes that one of his co-founders keeps a working front-panel replica at home, powered by a Raspberry Pi driving the blinking lights.

The minicomputer wave dramatically shattered commercial forecasts. DEC originally anticipated building roughly 10,000 units of the PDP-11/70 family; instead, they shipped over 100,000 systems, cementing DEC's business and expanding the reach of Unix and multi-user operating systems across commercial and academic sites.

[05:51](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=351s) Big Iron in the Dot-Com Era: The Sun Enterprise 10000

Cantrill spent fourteen years at Sun Microsystems and introduces the Sun Enterprise 10000 (the "Starfire" or "E10K") as his personal generation's touchstone. Shipped in the late 1990s, the machine was an imposing monolithic server boasting up to 64 UltraSPARC CPUs and 64 GB of RAM, housed in a cabinet built to look imposing.

Cantrill warns younger engineers that the tech sector is not immune to economic downturns, reminiscing about the dot-com bubble when Starfire servers formed the backbone of the nascent commercial web. Companies often expanded by simply moving to larger single-node machines rather than refactoring their software for distributed resiliency. A notable casualty was eBay:

* eBay began on a simple dual-processor machine and scaled upward into a 64-processor Sun E10K. * Because the architecture failed to accommodate horizontal redundancy or scheduled maintenance windows, the single system became a single point of failure. * In 1997, eBay suffered a catastrophic outage lasting roughly 22 hours, exposing how brittle internet commerce was when anchored to individual monolithic machines:

> "It was eye-opening because of how important server-side computing had become that we were now actually running commerce on a single machine, running commerce with the server-side computers that we had actually not really designed with uptime in mind suddenly became very load-bearing with respect to this new economy."

[07:44](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=464s) The x86 Microprocessor Takeover and Architectural Compromise

By the early 2000s, the cost dynamics of microprocessors changed decisively. Established RISC processor architectures—including SPARC, IBM POWER, PA-RISC, and DEC Alpha—could no longer keep pace with the massive manufacturing and economic scale of Intel's commodity x86 chips. Customers found they could purchase commodity x86 hardware delivering comparable performance for a fraction of the cost, making large SMP servers uneconomical:

> "What people began to realize is like, actually I can get you this a 64-processor machine, but I can have a workload that runs maybe with a quarter of the number of CPUs... but I can do that for a 20th of the cost or 1/100th of the cost."

```mermaid flowchart TD accTitle: Evolution of Server Hardware Architectures accDescr: The shift from early vacuum-tube mainframes and minicomputers to massive RISC SMP boxes, followed by the takeover of PC-derived commodity x86 servers.

subgraph S1["Custom Large-Scale Platforms"] M1["IBM 709 (1961) Mainframe scale, CTSS, 205 kW"] M2["DEC PDP-11/70 (1970s) Minicomputer, local Unix compute"] M3["Sun Enterprise 10000 (1990s) 64-way SMP RISC (SPARC)"] M1 --> M2 --> M3 end

subgraph S2["Commodity x86 Disruption"] PC["IBM PC Architecture (1979) Designed for hobbyists (Boca Raton)"] X86["Microprocessor Economics Intel x86 volume undercuts RISC"] SRV["HP ProLiant DL380 Gen5 (2009) Rack-mounted PC: CD tray, VGA, 8259 PIC"] PC --> SRV X86 --> SRV end

M3 -.->|"Extinguished by cost/performance"| SRV ```

This market dynamic produced a strange technical regression. Cantrill highlights the HP ProLiant DL380 (specifically a Gen5 model from roughly 2009) as the ubiquitous successor to the enterprise server. Despite its rack-mounted form factor, the system fundamentally remained a personal computer adapted for a server rack.

Cantrill points out the oddities embedded in a 2009 enterprise machine: * A physical CD-ROM drive on the front panel, useless inside a remote data center. * A legacy analog VGA display port. * The continued emulation of legacy PC chips dating back to the original 1981 IBM PC (designed in Boca Raton around 1979), such as the Intel 8259 Programmable Interrupt Controller (PIC).

The dominant processor line won the commodity market on sheer economics, but in doing so, it dragged along three decades of legacy PC baggage into the core of enterprise server infrastructure. At the same time, companies facing massive hyperscale demand began recognizing that standard commercial rack servers no longer met their needs.

[11:11](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=671s) The Cheap First Steps of Hyperscale: Google’s Velcro Era

By 2009, Google’s infrastructure needs had expanded dramatically, leading the company to the realization that deploying ordinary personal computers in data centers made little sense. Their first attempt at designing custom hardware was an infamous, stripped-down system built to minimize costs at all costs. Storage disks and the power supply were held down with Velcro. Instead of relying on a centralized uninterruptible power supply (UPS), Google attached an individual 12-volt battery directly to each server chassis. The motherboard was a rock-bottom consumer board from Gigabyte—the kind of component Bryan Cantrill jokes you could sweep off the floor—reflecting an effort to shave every single penny off machines bought in massive volume.

```mermaid flowchart TD accTitle: Tradeoffs of Early Hyperscale Cost Cutting accDescr: Diagram illustrating Google's early cost-cutting measures versus the resulting operational failures up the stack.

subgraph CostCutting["Cost-Cutting Measures"] D1["Velcro-mounted Disks & PSU"] D2["12V Battery per Server (No UPS)"] D3["Refurbished / Non-ECC DRAM"] D4["Rock-bottom Consumer Motherboards"] end

subgraph HiddenToll["The Hidden Operational Toll"] F1["Silent Memory Corruption"] F2["Unpredictable Machine Panics"] F3["High Cascading Stack Failures"] end

CostCutting -->|"Pennies saved up front"| HiddenToll ```

That aggressive skimping came with severe drawbacks. In trying to build the cheapest hardware possible, Google relearned an age-old lesson in systems engineering: penny-pinching on component quality extracts an immense operational toll. Refurbished memory and non-ECC DRAM led to corruptions and erratic failures across the software stack that far outweighed the pennies saved up front. As Cantrill emphasizes:

> “Google learned a lesson that has been learned for time immemorial in computing that is that you always want to seek value, but you have to be careful about unwittingly really getting what you paid for.”

Even so, both Google and Facebook were onto something essential: they were actively stripping out consumer ephemera that had no place in server infrastructure, eliminating optical CD drives, legacy VGA ports, and bulky consumer accoutrements in favor of purpose-built machines.

---

[13:22](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=802s) Rack-Level Efficiency: Facebook’s Tioga Pass and OCP Innovations

By the mid-2010s (roughly 2016–2017), hyperscalers had evolved far beyond makeshift motherboards to true rack-level co-design. A prime example is Facebook’s Open Compute Project (OCP) compute sled, Tioga Pass. In this architecture, individual compute sleds eliminate dedicated onboard power supplies entirely. Instead, each sled slides directly into a DC bus bar at the back of the chassis, powered by a centralized power shelf in the middle of the rack that handles all conversion from AC. Because every stage of power conversion bleeds energy as waste heat, eliminating conversion hops is vital for driving down Power Usage Effectiveness (PUE) and ensuring that electrical power goes toward compute rather than heating the data center floor.

```mermaid flowchart LR accTitle: Hyperscale Rack Integration accDescr: Flowchart illustrating the modular components feeding into Facebook's Open Compute Project rack architecture.

subgraph Sled["Tioga Pass Compute Sled"] TP1["Dual/Multi Socket Sled"] TP2["Front-Accessible Network Cables"] TP3["Direct DC Blind-Mate Connector"] end

subgraph RackSystem["OCP Rack-Scale Infrastructure"] PS["Centralized 12V DC Power Shelf"] BB["Integrated Bus Bar"] SW["Wedge Network Switch"] ST["Bryce Canyon Storage Chassis"] end

TP3 -->|"Direct connection"| BB PS -->|"Powers"| BB SW --- RackSystem ST --- RackSystem ```

The physical layout was rethought from scratch to optimize operations and servicing: - **No onboard power supplies:** Power conversion is consolidated into a dedicated, efficient shelf in the middle of the rack, mating directly with compute sleds via an integrated DC bus bar. - **Front-accessible cabling:** Network cables and interconnects are placed at the front of the chassis, allowing technicians to service sleds without navigating hot aisles. - **Wider chassis geometry:** Rejecting the historical, arbitrary 19-inch rack form factor, the Open Rack standard fits over standard 24-inch floor tiles while accommodating a wider, taller envelope holding three sleds across instead of two. - **Cohesive open ecosystem:** Sleds integrate into an open portfolio featuring Facebook’s custom network switch (Wedge) and high-density storage enclosure (Bryce Canyon).

---

[15:34](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=934s) The Enterprise Stagnation: Personal Computers in Disguise

While hyperscalers built streamlined, rack-scale hardware, mainstream enterprise and server-side hardware barely progressed. Comparing a 2009 commercial server to a 2020 commercial server from vendors like HP shows that little fundamentally evolved: the CD drive was replaced by a DVD drive, and the VGA port was replaced by an HDMI port. These are incremental upgrades suited for a desktop computer running Microsoft Windows in an office, not an enterprise server engineered for a modern elastic data center. Enterprise servers remain rack-mounted personal computers.

Cantrill recounts his career path—spending 14 years at Sun Microsystems until Sun was “invaded” (acquired by Oracle), after which he departed to spend a decade at cloud provider Joyent (subsequently acquired by Samsung). Building out a large private cloud on standard commercial enterprise hardware proved profoundly frustrating. It was not merely an aesthetic grievance; running desktop architectures in enterprise racks introduces crippling operational failures up and down the stack. For organizations lacking the scale and “infrastructure privilege” of Google or Facebook, observing open-source hyperscale breakthroughs that remain inaccessible off-the-shelf only sharpens the frustration.

> “It sharpens the pain for those of us who don't have the infrastructure privilege of these folks to know that like actually there's great innovations, we just can't have them, and that's that's very frustrating.”

---

[17:20](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1040s) The Hardware-Software Divide and the Silenced DIMM Pandemic

Against the prevailing industry assumption that public cloud monopolies will swallow all computing, Cantrill defends a contrary position: ownership of physical hardware remains essential. Organizations choose to operate their own infrastructure for risk mitigation, control, and above all, compelling economic reasons—at scale, owning compute is far cheaper than renting it indefinitely from Amazon Web Services or other cloud giants.

However, companies attempting to build on-premises infrastructure face a persistent architectural standoff:

This divide causes real operational damage, exemplified by a severe DIMM failure pandemic Cantrill experienced in production. Nodes were continually hit by uncorrectable memory errors that instantly panicked systems without warning. In standard ECC memory, uncorrectable double-bit errors rarely strike out of nowhere; they are almost invariably preceded by an escalating pattern of correctable single-bit errors caused by design defects, heat, or poor DIMM calibration.

The investigation revealed that proprietary vendor firmware was silently intercepting and discarding all correctable error events instead of logging them upward. Because the operating system never received telemetry about the rising correctable error counts, the software could not proactively alert operators, isolate degrading nodes, or trigger memory retraining routines before an uncorrectable crash brought down the server:

> “The uncorrectable that would ultimately panic the box, and if you've been around memory for any length of time you know that's not really the way the world works, that you don't in ECC memory you don't see an uncorrectable generally out of clear blue sky. You generally see a rising number of correctable errors, and those correctable errors indicate you've got a problem here.”

[21:07](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1267s) The Clean Sheet Approach: Keeping the Microprocessor, Rebuilding Firmware

Bryan Cantrill recalls the exasperating struggle of trying to get PC-oriented hardware vendors to elevate hardware errors cleanly into operating system software, comparing the bureaucratic friction to “trying to get a bill through Congress.” When you approach a personal computer vendor asking for a custom firmware build tailored to hyperscale infrastructure, they react with utter bewilderment: “Why would anyone running a personal computer want their own firmware? It doesn't make any sense.” The fundamental disconnect is that cloud operators are not running personal computers; they are running a cloud. Because this disconnect has been felt acutely across the entire computing industry, Cantrill argues that incremental fixes are insufficient: it is time to take a clean sheet of paper and redesign the computer from scratch for the year 2020.

Starting from scratch, however, does not mean reinventing everything. Cantrill emphasizes that they have no interest in redesigning the core microprocessor itself. Modern microprocessors from Intel, AMD, or ARM are exceptionally sophisticated and fully fit for enterprise purposes. The true structural problems lie in the firmware ecosystem surrounding those processors. In traditional commercial servers, operators run machines with zero knowledge of the actual firmware binaries executing underneath them. Applying a standard vendor BIOS update means flashing an untrusted black box. Performing even rudimentary binary inspection on these vendor ROMs reveals terrifying anomalies, such as embedded web URLs in the lowest-level machine initialization code: “This is the brainstem of my data center. Why is there a URL in here?” When you attempt to ask vendors why their firmware contains URLs, you find that nobody can answer the question.

```mermaid flowchart LR accTitle: "Clean Sheet Redesign Scope" accDescr: "Separating components to keep versus components to overhaul in the 2020 clean-sheet redesign."

subgraph Keep["Keep As-Is"] CPU["Microprocessors (Intel / AMD / ARM)"] end

subgraph Overhaul["Complete Overhaul"] direction TB FW["Firmware & Attestation"] SP["Service Processor Architecture"] RS["Rack-Scale Integration"] end

Keep -.->|"Hosts Fit-for-Purpose Compute"| Overhaul ```

- Bryan Cantrill: “This disconnect was felt very acutely by us certainly, and it's been acutely felt by many, many, many people; this is not novel or not unique to us, so we feel you really need a fundamentally new approach.” - Bryan Cantrill: “Why is there a URL in here? You'll try to get an answer to that question; it's not that it's like an uncomfortable question, it's you literally can't find anyone who can answer it.”

---

[23:25](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1405s) The Supply Chain, American Megatrends, and the BMC Attack Surface

The opacity of server firmware creates deep security and provenance vulnerabilities. Even hardware vendors that profess a desire to open-source their firmware are often incapable of doing so because the firmware is not actually theirs. It is licensed from third parties whose legacy roots stretch back to the early PC era. Cantrill expresses genuine disbelief that American Megatrends Inc. (AMI) still runs the fundamental infrastructure of the internet: “How is AMI still running at the brainstem of humanity?” Because these third parties refuse to share their proprietary source code, customers cannot inspect, verify, or attest the software executing on their servers.

This vulnerability became glaringly apparent during the public reaction to the fall 2018 *Bloomberg* report alleging hardware supply chain infiltration at Supermicro. While Cantrill notes that the story as written was almost certainly factually incorrect—and no physical spy-chip hardware was ever produced—he points out the absurdity of suspecting an elaborate physical interdiction campaign when servers are already wide open at the software layer: “Why would anyone mount this very sophisticated attack frankly when you can just walk in through the firmware?” Most Supermicro Baseboard Management Controllers (BMCs) ship with factory default root credentials, remain directly exposed to the public internet, and suffer from severe vulnerabilities. Cantrill points to research by Rick Altherr presented at the Open Source Firmware Conference, where Altherr uncovered the "USB Anywhere" vulnerability and identified 77,000 publicly exposed BMCs that could have been completely compromised over the network.

The root of this systemic risk is the BMC itself—a secondary computer running inside the host server. Far from being a simple controller, a typical BMC is an Aspeed or Nuvoton System-on-Chip (SoC) running U-Boot, Linux, and opaque proprietary software stacks. Critically, the BMC acts as a master on the host server's PCIe bus while simultaneously maintaining direct external network interfaces. Leaving an unvetted, proprietary operating system wired directly between the wild internet and the machine's memory bus represents an untenable security architecture.

- Bryan Cantrill: “How is AMI still running at the brainstem of humanity? American Megatrends Incorporated, this is a company that was around when I was a kid—I'm old—it has, I mean again it's got a right to exist, but why does an AMI BIOS exist on industrial-grade machines? It makes no sense.” - Bryan Cantrill: “Why would anyone mount this very sophisticated attack frankly when you can just walk in through the firmware?”

---

[26:19](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1579s) Tearing Down the BMC and Killing the DOS Mentality in Host Firmware

To restore server integrity, Cantrill argues that the entire concept of the modern BMC must be dismantled and returned to a lean, purpose-built service processor backed by a true hardware root of trust that attests all firmware. A service processor should focus strictly on fundamental physical maintenance: power-cycling the host ("bouncing the box"), providing a serial console, and driving basic chassis thermal and fan controls. It has no business hosting virtual VGA display adapters, virtual USB stacks, or bloated internet-facing software layers.

```mermaid flowchart TD accTitle: "Service Processor vs Host Lifecycle Separation" accDescr: "Comparing the lean duties of an attested service processor with the ephemeral lifecycle of host firmware handing off to the OS."

subgraph SP["Attested Service Processor"] direction TB B1["Power Cycling ('Bounce the Box')"] B2["Serial Line Interface"] B3["Thermal & Fan Control"] end

subgraph Host["Clean Host Firmware Lifecycle"] direction TB H1["Hardware Initialization"] H2["Attestation & Handoff"] H3["Exit Memory Completely"] H1 --> H2 --> H3 end

SP -.->|"Out-of-Band Control"| Host H3 ==>|"Clean Slate Execution"| OS["Host Operating System"] ```

Host boot firmware requires an equally radical philosophical purge. In Cantrill’s view, host firmware exists for a single, transient purpose: initialize the hardware, hand the execution thread cleanly to the host operating system, and vanish from memory. Cantrill forcefully rejects the lingering "DOS mentality" of modern firmware architectures like System Management Mode (SMM) and ACPI runtime calls, where unseen vendor code remains resident indefinitely to execute arbitrary logic behind the operating system's back. Recalling his childhood frustrations of growing up using DOS without memory protection in the 1980s—wrongly assuming that computers were simply destined to crash and reboot at random until he discovered Unix in college—Cantrill insists that modern cloud servers should never rely on load-bearing, hidden firmware executing in stealth.

Compounding this problem are the secondary internal co-processors, such as the Intel Management Engine (ME) and the AMD Platform Security Processor (PSP). Intel's ME is an entirely independent microcontroller on the motherboard running an operating system based on MINIX with total dominion over system buses and hardware state, subject to proprietary vulnerabilities. While AMD's PSP at least lives directly on the processor die, it remains an opaque black box. Cantrill asserts that understanding every single instruction executing on the physical silicon is a completely reasonable and fundamental demand for modern enterprise computing.

- Bryan Cantrill: “Your job is to bring the host operating system its cup of coffee and shoot yourself in the head; that should be the role of host firmware.” - Bryan Cantrill: “This idea that firmware should execute and then somehow stay resident so it can be called into at arbitrary times to do arbitrary things is—this is DOS, that's that DOS mentality and approach, and DOS robbed me of my childhood.” - Bryan Cantrill: “Visibility into it is a reasonable thing: to be able to understand every instruction that's executing on your computer, that is a reasonable request.”

---

[29:22](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1762s) True Rack-Scale Co-Design

Beyond internal motherboard architecture, rethinking the computer requires treating the entire rack as the atomic unit of compute. Cantrill points out that simply stacking twenty standard 2U commodity servers into an enclosure and lashing them together with patch cables does not constitute a rack-scale design; it is merely “rack and stack.” A true rack-scale architecture demands tight, unified co-design between the top-of-rack (ToR) network switch, internal storage pools, and compute blades, making every layer deeply aware of the others.

This vision cannot be executed purely in hardware; it fundamentally requires unified system software. Cantrill points out that earlier industry initiatives—such as Intel’s Rack Scale Design (RSD)—consistently failed to deliver because hardware companies historically treated software as “someone else's job.” Without an integrated software operating substrate, rack-level hardware features cannot be coordinated. Furthermore, realizing this design requires shifting the boundaries of interoperability. While enterprise hardware buyers often treat universal switch interoperability as a sacred virtue, unconstrained interoperability acts as an enemy to tightly optimized systems engineering: “Interoperability is great in principle, but it is very corrosive when you want to make an actual tight integrated design.” Instead of forcing internal server trays to negotiate with arbitrary third-party ToR switches, the ToR switch must be pulled inside the system boundary, pushing external network interoperability out to the rack’s exterior boundary while maximizing chassis density and performance within.

- Bryan Cantrill: “By rack-scale design, I mean really thinking about the entire rack; taking a bunch of 2Us and slapping it into a rack is not a rack-scale design, that is just rack and stack.” - Bryan Cantrill: “Interoperability is great in principle, but it is very corrosive when you want to make an actual tight integrated design.”

[30:48](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1848s) The Power Reality of Mainstream Enterprises and Bus Bar Architecture

Hyperscalers operate in a world detached from standard enterprise constraints: they design data centers from scratch, locating facilities beside dams and running up to 25 kW per rack. Bryan Cantrill points out that while packing 25 kW into a rack sounds fantastic, it is completely impractical for mainstream enterprises. The power draw and cooling demands are simply too immense, leaving enterprise data centers constrained to a realistic power envelope of roughly 10 to 15 kW per rack.

Within that 10 to 15 kW budget, however, systems must achieve maximal compute density and adopt more efficient delivery mechanisms, such as bus bar-based power design. Cantrill expresses utter disbelief that major enterprises with massive computational footprints are completely locked out of this architecture:

> “Why can I not buy a machine if I'm, you know, General Electric, or I'm Barclays, if I am General Motors, I can't buy a bus bar-based machine, no matter how much compute I'm operating?”

He argues that it is ridiculous for mainstream infrastructure to ignore these efficiencies. High-power density does not mean low-power computing; rather, it demands relentlessly efficient operation from the hardware up.

- Hyperscaler data centers: Custom-built near major power sources (such as hydroelectric dams), drawing 25 kW per rack. - Mainstream enterprise envelope: Constrained to approximately 10 to 15 kW per rack due to practical power and cooling limits. - The bus bar gap: Despite running massive scale, large enterprises (GE, Barclays, GM) are barred from purchasing rack-level bus bar designs off the shelf.

---

[32:15](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1935s) Co-Designing Hardware and Software: The 2020 Computer Boundary

Cantrill attributes his perspective to his deep roots at Sun Microsystems, remarking that you simply cannot get the Sun out of him. He is convinced that true system quality comes only from the tight co-design of hardware and software. When both halves are designed together, engineers gain complete control over system elements, resulting in a significantly higher-performing, coherent artifact.

The definition of a computer has fundamentally shifted. In the 1970s and 1980s, the boundary of the computer lived at the multi-user shell. By 2020, that boundary has migrated all the way up the stack:

> “I think in terms of in 2020 the boundary for the computer is at the API endpoint to provision a virtual machine.”

Between that API endpoint and the bare metal lies an enormous software surface: operator and developer APIs, compute control planes, storage control planes, the top-of-rack (ToR) switch software, and the hypervisor. Cantrill laments that the modern industry forces end users to cobble all these pieces together themselves, calling the situation profoundly unfair and unjust.

```mermaid flowchart TD accTitle: The Modern Computer Stack Boundary accDescr: Diagram illustrating the modern boundary of a computer at the API provisioning layer down through the hardware substrate.

subgraph API_Layer ["Modern Computer Boundary"] API["Operator & Developer API Endpoints (VM Provisioning Boundary)"] end

subgraph Control_Planes ["Integrated Infrastructure Software"] CCP["Compute Control Plane"] SCP["Storage Control Plane"] TOR["Top-of-Rack (ToR) Switch Software"] HYP["Hypervisor"] end

subgraph Hardware_Substrate ["Co-Designed Physical Substrate"] HW["Rack-Scale Hardware (Bus Bar Power, Servers, Network)"] end

API --> CCP API --> SCP CCP --> HYP SCP --> HW TOR --> HW HYP --> HW ```

---

[33:49](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=2029s) Open Infrastructure and the Case for Attainability

While Cantrill advocates for delivering an integrated machine where the software is tightly welded to the hardware, he draws a sharp line against proprietary lock-in. The era of proprietary infrastructure software is dead, and enterprise servers cannot follow Apple's iPhone model. The iPhone offers a beloved user experience, but its complete inaccessibility makes it impossible to inspect, understand, or modify the underlying system. Instead, rack-scale system software must be built on the open-source tradition, fully open and rigorously attested.

Addressing the skepticism of those who treat server architecture stagnation as an intractable status quo, Cantrill admits that he was once among them: frustrated by the bifurcated world between hyperscalers and the enterprise, yet hoping things would magically improve. Upon deploying new hardware and realizing things had actually worsened due to deep structural dysfunction, he concluded that passivity was no longer viable:

> “We have to be able to do this, we have to solve this problem, because, one, this is not science, right? This is engineering.”

This problem is fully solvable because a purpose-built rack does not need to run every legacy operating system from MS-DOS 6.22 to modern obscure variants. Hyperscalers never run arbitrary bare-metal operating systems; they run their own hypervisors. By targeting specific hypervisors and a curated set of devices, engineers eliminate false generalities in both software and hardware, reducing a seemingly sprawling problem into a bounded, tractable engineering effort.

- Anti-pattern: The closed-box iPhone model that prevents user inspection and low-level system modifications. - Prerequisite: Fully open-source, fully attested infrastructure software stack. - Scoping advantage: Tightly tailoring bare-metal firmware and hypervisors eliminates decades of legacy operating system compatibility overhead.

---

[36:50](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=2210s) The Spectre/Meltdown Wake-Up Call and Hardware Roots of Trust

A major catalyst enabling this architectural shift was the catastrophic security revelations of 2018. Cantrill recalls discovering Spectre and Meltdown on January 1, 2018, reading a speculative Hacker News thread about suspicious Linux patches alongside the rest of the world. Despite running huge fleets of Intel processors, his team had received no embargo warning. As the details unfolded over subsequent days, the full horror of Meltdown set in.

The great irony of Meltdown and Spectre was that speculative execution was the exact architectural breakthrough that had originally enabled x86 to break through the memory wall and crush competitive microprocessors. Cantrill notes a conversation with Jon Masters regarding whether the SPARC architecture would suffer from similar vulnerabilities:

> “I was talking even John Masters, who was saying that like, ‘Hey, do you think SPARC would have any of these issues?’ Like, no, SPARC would not have any of these issues, because SPARC's problem was not speculative enough.”

These microarchitectural vulnerabilities acted as a beacon to security researchers eager for notoriety, unleashing intense scrutiny upon previously unexamined, ill-conceived firmware deep within the stack. This reckoning sparked an industry-wide scramble for hardware roots of trust and firmware attestation. Cantrill points to Microsoft's Project Cerberus and Google's Titan family as key milestones. In fact, Cantrill highlights the Google Chromebook as one of the best, most underappreciated computers built, precisely because it solved firmware attestation and hardware root of trust on client machines. Unlike the iPhone, a Chromebook permits users to flash custom firmware; however, its hardware root of trust ensures the system explicitly detects and acknowledges that it is running non-native firmware, refusing to blindly attest that its underlying software substrate can be trusted. This lineage has now progressed toward broader industry efforts such as Google's OpenTitan project.

[40:42](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=2442s) The Open EDA Movement, HDLs, and Bluespec

Bryan Cantrill draws an analogy between what is happening in modern hardware development and the container world, framing new open infrastructure initiatives as akin to a "Kubernetes versus Borg" moment. Joining forces with these broader collaborative efforts means a new startup is no longer stranded trying to reinvent every wheel from scratch. A parallel, yet underappreciated, revolution is taking place in electronic design automation (EDA). Pointing out the historical paradox of hardware development, Cantrill observes: “It was easier to get a hardware company funded two decades ago, but hardware has never been easier than it is today.”

```mermaid flowchart LR accTitle: Modern Open-Source Hardware Development Stack accDescr: Progression from higher abstraction design down through HDLs and synthesis to FPGA soft cores.

subgraph Design ["Design & HDL"] direction TB BS["Bluespec (High-level Abstraction)"] HDLs["Chisel / SpinalHDL / Verilog"] end

subgraph Toolchain ["Open Toolchain"] direction TB Yosys["Yosys Synthesis"] Verif["Open Verification Engines"] end

subgraph Target ["Silicon Target"] FPGA["FPGA Substrate"] RiscV["License-Free RISC-V Soft Core"] end

Design --> Toolchain Toolchain --> Target FPGA -.->|"instantiates"| RiscV ```

EDA software historically served as one of the last bastions of completely proprietary, locked-down software. Today, that fortress is giving way to open tools and expressive hardware description languages like Yosys, Chisel, and SpinalHDL, alongside open verification frameworks. Most notably, Bluespec—originally invented more than fifteen years prior at MIT—was released as open source only days before the seminar. Calling it merely an HDL does it a disservice, Cantrill argues, because it elevates the level of abstraction at which designers define FPGA behavior. That newfound accessibility radically lowers the barrier for engineers to roll their own custom FPGAs.

- Yosys provides modern open-source synthesis, pairing with newer hardware construction languages like Chisel and SpinalHDL. - Bluespec open-sourced its compiler and language roughly ten to fourteen days prior to this talk, providing a higher-level semantic model for FPGA logic design. - The barrier to designing, simulating, and deploying custom FPGA hardware has cratered compared to the proprietary toolchains that dominated two decades ago.

---

[42:24](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=2544s) The Rise of RISC-V: Clean Architecture and Free Soft Cores

Cantrill highlights the RISC-V instruction set architecture, spearheaded by David Patterson and his colleagues at UC Berkeley, as another foundational piece of the modern hardware puzzle. He enthusiastically recommends the *RISC-V Reader* (referring to the instruction set atlas), describing it as a delightful read because of the sheer intentionality behind the specification. Instead of piling on historical baggage, the designers systematically cataloged decades of processor architectural successes and failures, deliberately extracting the good ideas while discarding flawed architectural conventions.

Beyond the clean architecture, RISC-V’s decisive advantage is legal and economic: it is entirely open and royalty-free. Anyone can drop a license-free soft core onto an FPGA without paying IP fees or signing restrictive non-disclosure agreements:

> “And you can now have your own software running on that RISC-V core. Now is that software running with megabytes and megabytes of memory? No, it's running in a pretty constrained environment, but it allows you to actually have computational programmability at very, very, very low levels of the stack, i.e., in the FPGA.”

- RISC-V avoids the historical pitfalls of earlier instruction set architectures by conducting a deliberate post-mortem on decades of computer engineering. - Royalty-free implementations allow designers to spin up embedded soft cores in an FPGA fabric cheaply and quickly. - These soft cores afford developers full computational programmability in micro-environments deep inside server subsystems, even where memory is counted in kilobytes rather than megabytes.

---

[44:05](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=2645s) Open Firmware and the Open Compute Project (OCP)

For years, open firmware appeared to be a hopeless crusade, leaving servers condemned to proprietary legacy BIOS and opaque vendor blobs forever. However, Cantrill explains that hardware security debacles like Spectre and Meltdown shattered that inertia by laying bare the catastrophic danger of vulnerabilities lurking deep in the low-level silicon stack. Hyperscalers, unwilling to tolerate black-box firmware they could neither audit nor remediate, began demanding transparency from microprocessor vendors and directly funding open firmware efforts. Reflecting on this turning point, Cantrill says: “There was a time when I felt like open firmware was never going to happen—that is to say we would have a closed legacy BIOS forever. But with Spectre and Meltdown, and with the new focus on these vulnerabilities deep in the stack, and thanks to the hyperscalers who really demand this and are funding it accordingly... we are now seeing the arrival of open firmware.”

```mermaid flowchart TD accTitle: Drivers of the Open Firmware Transition accDescr: Security crises and hyperscaler leverage pushing the transition away from closed legacy BIOS to open firmware and OCP.

subgraph Catalysts ["Industry Catalysts"] SM["Spectre & Meltdown Vulnerabilities"] Hyper["Hyperscalers (Google, Meta/Facebook)"] end

subgraph Forum ["Collaboration Hubs"] OCP["Open Compute Project (OCP)"] OSFC["Open Source Firmware Conference"] end

subgraph Outcome ["Modern Open Stack"] FW["Auditable Open Firmware (Coreboot, LinuxBoot)"] Spec["Open Server Reference Hardware (e.g., Tioga Pass)"] end

Cantrill recalls attending the Open Source Firmware Conference (OSFC) hosted at Google and Facebook, describing it as the single best conference he had attended in the previous year. It evoked the nostalgic feeling of early tech gatherings from before the ubiquitous internet era, where engineers who assumed they were the only people on earth obsessing over an obscure discipline stepped into a room to discover hundreds of kindred spirits. Much of this momentum is driven through the Open Compute Project (OCP). Facebook established OCP around 2010–2011 to share power-efficiency innovations, like centralized DC busbar architectures and designs such as the Tioga Pass motherboard. While the notion of Facebook acting out of a heartfelt sense of “social responsibility” feels surreal in hindsight, the project quietly blossomed into an indispensable, transparent forum where competing hyperscalers share real-world data center hardware designs.

- Spectre and Meltdown forced hyperscalers to realize that black-box firmware represents an existential security risk. - The Open Source Firmware Conference (OSFC) has united a thriving developer community dedicated to dismantling legacy proprietary firmware blobs. - OCP originated from Facebook engineers seeking to standardize power distribution and server mechanics, growing into the primary venue where production hardware architectures are publicly disclosed.

---

[47:24](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=2844s) The System Software Shift: Databases, Consensus, and Languages

The system software landscape has transformed just as radically as hardware. Open source has evolved from an alternative paradigm into the baseline expectation for data center infrastructure. While venture-backed open-source companies now grapple with identity crises over commercialization and monetization, the operational ecosystem has matured enormously. Production-ready, resilient software runs open in data centers at massive scale, demonstrating stability that was unthinkable ten years prior.

| Era | Consensus Systems | Database Architectures | System Software Landscape | |:--- |:--- |:--- |:--- | | **c. 2010** | Apache ZooKeeper (finicky, complex, painful to operate) | Proprietary relational engines still dominant; nascent open-source options | Closed Unix/Windows kernels lingered; low-level firmware closed source | | **c. 2020** | Modern Raft-based systems, distributed key-value consensus engines | Open-source distributed databases run at planetary scale; proprietary engines declining | Open-source Linux/illumos kernels standard; emergence of Rust and open firmware |

A decade ago, distributed consensus was practically synonymous with Apache ZooKeeper. As Cantrill remarks, “ZooKeeper is not fun—even if you love ZooKeeper, you will acknowledge that it is not fun.” By 2020, battle-tested distributed databases and consensus mechanisms became ubiquitous off-the-shelf primitives. Operating systems are now almost universally open source, with proprietary databases similarly on the verge of obsolescence.

Yet the most staggering transformation was one Cantrill never would have anticipated: an architectural leap forward in systems programming languages. As an unapologetic, pragmatic systems hacker, he had historically treated programming languages merely as tools, looking back wryly on his own youthful fascination with features like operator overloading:

> “I think my past self would have tried to take a swing at my future self, because I'm a systems person... I view programming languages as part of getting a job done, very pragmatically. And yes, I went through, you know, a youthful phase of operator overloading and so on, but you grow out of that... because you'll realize that some of the neat things you can do in languages doesn't mean you should do them, certainly not in deployed systems. And especially debugging the code of others will help a great deal in this regard—if you're wondering how I can actually help accelerate this process, go debug the code of others.”

Having lived through the scars inflicted by MS-DOS in his youth and spent decades untangling complex C++, Cantrill sets the stage for why modern type safety and memory-safe systems languages are finally rewriting the rules of the trade.

[50:32](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3032s) The Fatal Flaws of Garbage Collection and the Rejection of C++

Reflecting on past language choices, Bryan Cantrill recalls having his young adulthood consumed by C++, leading to an intense love affair that ultimately ended in a vow never to return. While modern garbage-collected languages like Go and Java have their place, he insists that garbage collection has no business in low-level systems programming. Cantrill makes clear that he is not speaking pejoratively about garbage collection as a whole—recognizing its utility for the vast majority of software worldwide—but argues that the systems he builds require strict determinism. In systems programming, one cannot tolerate an unseen, unheard whistle being blown that halts the software so a runtime can scurry around doing uncontrolled work behind the scenes.

```mermaid flowchart TD accTitle: The Problem with Garbage Collection Under Load accDescr: Flowchart illustrating how garbage collection degrades under heavy workloads into an uncontrollable heap scanner.

subgraph Normal["Normal Conditions"] A["Small Heap Footprint"] --> B["Predictable Latency"] end

subgraph Surge["High Workload / Heavy Traffic"] C["Surging Workload"] --> D["Large, Fully Legitimate Heap"] D --> E["GC Becomes a Periodic Heap Scanner"] E --> F["Stop-the-World Latency Spikes"] end ```

The fundamental failure mode of garbage collection is that it works fine on sunny days, but degrades catastrophically on stormy ones. When a system is under heavy load, its legitimate, active heap swells with real work. If an engineer has built a meticulously manicured system where allocated memory is actually in active use rather than discarded garbage, the garbage collector degenerates into an unprompted, periodic heap scanner over which the engineer has zero control. Because of this unpredictability, Cantrill resolved years ago that garbage collection was simply incompatible with the systems he creates, resigning himself to the idea that he would write C for the rest of his career.

- Cantrill summarizes his systems philosophy: “I don't want garbage collection anywhere near the systems that I'm developing.” - Under heavy load, an efficient program with little actual garbage causes the collector to run periodic, expensive scans across a massive active heap. - Cantrill compares his acceptance of C to being a craftsperson who values knowing the exact physical reality beneath his code: “When you write C, you can feel the assembly underneath you.”

---

[52:47](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3167s) C, Undefined Behavior, and Vulnerabilities in DTrace

Although critics often disparage C as mere portable assembly, Cantrill views that low-level transparency as the exact quality a systems programmer demands to control the machine. However, the modern revolution in programming languages brought about by Rust has fundamentally changed his perspective. While C's lack of safety is notorious—likened to constantly playing with loaded weapons—the deeper systemic issue with C is composability. It is rarely the code written by an individual programmer that introduces disaster; rather, it is the inability to safely link against external libraries because C composes exceptionally poorly.

Even skilled engineers who believe they write correct C are working on shaky ground because the C standard is riddled with undefined behavior. Cantrill illustrates this vulnerability through personal experience with DTrace, a ubiquitous dynamic tracing system he co-authored around 2005. Over fifteen years of widespread deployment, DTrace suffered only a tiny handful of vulnerabilities, yet the two vulnerabilities Cantrill personally introduced were both caused by malicious integer overflow. In both cases, bounds checks that were completely correct for benign operating conditions failed when confronted with crafted malicious inputs, opening the door to arbitrary memory reads and privilege escalation. This painful lesson demonstrated that systems require more than just memory safety; they require integer safety as well.

- Cantrill emphasizes the historic importance of Rust: “I fervently believe that Rust is every bit as important revolution as C.” - In C, subtle integer arithmetic flaws bypass normal bounds checks: “The two vulnerabilities that I introduced were both due to effectively malicious integer overflow where I was doing a—I was comparing a bounds incorrectly.” - Unlike C, Rust's strict static model eliminates undefined behavior by enforcing rigorous memory ownership contracts and built-in protections against integer overflow.

---

[55:28](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3328s) Learning Rust, the Bargaining Phase, and the 35% Speedup

Rust shifts the cognitive burden of memory safety entirely to compile time through its strict ownership model. By tracking owners and lifetimes statically, the Rust compiler can guarantee memory safety and generate fast native binaries without requiring a runtime garbage collector. However, Cantrill warns that dabbling casually in Rust will punish newcomers: while an experienced C programmer can eventually develop an intuition for the compiler’s strictness by visualizing the underlying memory, the learning curve is notoriously steep and filled with frustration. Cantrill jokes that learning Rust forces a programmer through the five stages of grief until they find themselves bargaining with the compiler just to get code to build.

```mermaid flowchart LR accTitle: The C vs Rust Performance Surprise accDescr: Flowchart contrasting Cantrill's optimized C code with his naive Rust implementation utilizing standard B-trees.

subgraph C_Impl["Hand-Tuned C Implementation"] C1["Balanced Binary Tree"] --> C2["Manual Pointer Swapping"] C2 --> C3["Baseline Performance"] end

subgraph Rust_Impl["Naive Rust Implementation"] R1["Rust Standard BTreeMap"] --> R2["Memory Relocation & Cache Locality"] R2 --> R3["35% Faster Than Hand-Tuned C"] end ```

To test Rust on a small project, Cantrill initially tried writing a clever implementation, but soon realized he was not clever enough to fight the borrow checker. He abandoned cleverness and rewrote the software in the dumbest, most straightforward way possible just to achieve a clean compilation. Having previously authored an optimized version of the same program in C, he hoped the naive Rust code would land within 20% of his C performance. When he ran the benchmark, the Rust implementation was actually 35% faster than his hand-tuned C. The discovery triggered mixed emotions, feeling like a parent being soundly beaten at basketball by their own child: proud of the result, but deeply annoyed that his handcrafted C had been humbled.

- Cantrill notes the steep learning curve: “If you haven't waded into Rust, you definitely want to sit down and learn it; if you try to dabble with it, it will punish you because the—the model is different.” - Comparing the mental models of languages, Cantrill observes: “With C you could feel the underlying assembly; with Rust you can feel the underlying C.” - Upon measuring the benchmark results against his carefully written C implementation: “I ran it, and it was like 35% faster than my C.”

---

[58:06](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3486s) Why Rust Outperformed C: The Power of Safe B-Trees

Perplexed by why the Rust binary outperformed well-crafted C, Cantrill investigated the memory access patterns and contacted Rust core team member Steve Klabnik. Cantrill assumed that the Rust compiler must have achieved a groundbreaking breakthrough in solving the notorious memory disambiguation problem. To his surprise, Klabnik informed him that while LLVM and Rust could theoretically perform such exotic optimizations, none of them had actually been implemented yet. The speedup was not the result of compiler magic, but rather the intrinsic data structures mandated by Rust's design.

In Rust, the standard ordered associative collection is a B-tree (`BTreeMap`), rather than the balanced binary trees (such as red-black trees) ubiquitous in C. A B-tree offers dramatically better cache locality and algorithmic performance, but implementing one in C—especially as a reusable, general-purpose library—is extraordinarily difficult. Because balancing a B-tree involves physically copying and relocating elements in memory rather than merely swapping pointers, a C library would require an intricate, explicit ownership contract with its caller to avoid memory corruption, something C's type system cannot safely express. Because Rust encodes ownership and borrowing into the type system, safe and highly efficient B-tree implementations are straightforward to consume. Cantrill concludes that Rust forces developers into superior, high-performing architectural choices that would be nearly impossible to manage safely in C.

- Cantrill recalls asking Steve Klabnik about the performance disparity: “Steve, you know, it's amazing that the optimizations that you've pulled, and clearly that compiler knows so much about heaven and earth that it is able to truly solve the memory disambiguation problem.” - Klabnik's blunt response dismantled that theory: “No, it's—it's true that that's all possible; we haven't done any of that yet.” - Cantrill challenges developers to verify the difficulty of building robust, relocating data structures in unsafe languages: “There are not really good B-tree libraries in C—I mean, like take me up on this challenge, go look for a good B-tree library in C; it is really, really, really hard to implement.”

[01:00:07](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3607s) Rust’s Small Footprint and Embedded Systems

Bryan Cantrill discusses the extraordinary ability of the Rust programming language to compile down into tiny memory footprints without forfeiting expressiveness. On embedded soft-core RISC-V targets instantiated on an FPGA, available RAM might be limited to around 512 KB. Reflecting on a conversation with an embedded systems colleague, Cantrill jokes about the famous quote apocryphally attributed to Bill Gates that 640 KB ought to be enough for anybody:

> “640K is actually a lot of memory, it actually is, and you actually can do a lot with 640K, especially if you are careful about the way you spend it, and Rust allows you to slim things way, way down—borderline beyond what you can do with C.”

While C can yield extremely compact binaries, Rust achieves parity in minimal footprint while retaining rich, modern software abstractions. Cantrill highlights this as an exciting breakthrough for systems programming. In fact, the name of Oxide Computer Company is an explicit tip of the hat to Rust. He notes that the team has drawn deep inspiration from embedded systems like Tock OS—an open-source, embedded Rust operating system developed by Amit Levy, Phil Levis, and others at Stanford and partnering institutions.

[01:01:45](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3705s) Founding Oxide Computer Company and the Graveyard Slide Deck

Piecing together these advancements in open firmware, RISC-V, and Rust, Cantrill and his co-founders realized a commercial rack-scale server was finally buildable. They founded Oxide Computer Company and closed their seed funding round at the end of 2019. Raising seed capital for a hard-tech hardware venture was simultaneously frustrating and energizing, depending on whether investors grasped the thesis. By early 2020, Oxide had grown to a 15-person team, targeting a functional prototype by 2021 and production shipping systems by 2022.

To address the high failure rate among historical server startups, Cantrill prepared an extensive appendix for investor pitches, maintaining dedicated slides on virtually every failed computer company from the preceding 15 years:

> “When we were doing our raise, we I really wanted to learn from every single person's experience, and as a result I had an appendix slide on every single company you could possibly name that looked anything like a computer company in the last 15 years.”

This thoroughness sometimes backfired when pitching venture capitalists. Instantly jumping to an appendix slide for obscure startups—such as EJet or Elogex—served as a visceral reminder of dead investments, making some investors nervous that the team was overly fixated on industry failures. Alongside launching the company, Cantrill and co-founders Steve Tuck and Jess Frazelle recognized that practical wisdom at the hardware-software boundary was eroding, leading them to start *On the Metal*, a podcast exploring technical hardware and software lore across multiple engineering generations, including a recent interview with Star Simpson.

```mermaid flowchart LR accTitle: Oxide Computer Company Founding Milestones accDescr: Timeline of Oxide Computer Company from seed funding to production delivery.

A["Late 2019: Seed Round Closed"] --> B["Early 2020: 15-Person Team Ramped"] B --> C["2021 Target: Functional Prototype"] C --> D["2022 Target: Production Systems"] ```

[01:06:22](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3982s) Q&A: Ancestor Companies vs. Competitors

Opening the floor to questions, an audience member mentions historic hardware makers like Convex Computer and Multiflow from the mini-supercomputer era of the late 1980s and early 1990s. Cantrill recalls that Convex (later acquired by HP) reportedly had an outdoor cemetery on their Dallas campus where they physically engraved the tombstones of defeated competitors into concrete.

Cantrill observes that while keeping track of competition can be entertaining, studying dead predecessors is vastly more educational for an engineer:

> “It's more interesting, I think, to keep track of your ancestors than your competition, because I think they've got more to teach you than then you're—I mean it's easy to kind of disparage your competition, we never listen to that, but the it's the folks that seemed like they were the right place to the right time and and with the right technology and didn't succeed that aren't some ways more intriguing.”

[01:08:28](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4108s) The Target Market: The Pain of Cloud Costs and On-Prem Realities

When asked about Oxide's target demographic, Cantrill identifies organizations running compute on-premises. These customers fall into distinct buckets, frequently driven by strict regulatory constraints or staggering cloud expenditures that threaten company viability. For years, conventional industry dogma has insisted that running on-premises infrastructure is obsolete, treating those who remain as if they simply fail to comprehend modern cloud computing:

> “If you talk to these customers, they tend to be pretty animated because they're being told they don't exist, or to the degree they exist, it's like they exist because they're dummies who don't understand the cloud.”

Cantrill notes that cloud pricing structures are deceptively designed: they jump abruptly from practically zero—masked by investor subsidies or initial promotional credits—to staggering numbers that decimate a company's cost of goods sold (COGS). By the time leadership discovers they are being crippled by public cloud compute fees, migrating back on-premises proves excruciating. Off-the-shelf white-box and commercial servers are difficult to configure and operate without hyperscale-sized engineering teams, leaving frustrated operators with botched rack deployments and few viable alternatives.

- **Cloud Cost Trap**: Companies shift rapidly from "free" cloud credits to unsustainable monthly bills with very little warning or transitional pricing cushion. - **On-Prem Abandonment**: Enterprise IT vendors have treated on-premises customers as anachronisms, leaving them stranded with fragmented server hardware and poor software ergonomics.

[01:10:38](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4238s) The "Left for Dead" On-Prem Market and VC Reactions

While raising their financing round, Oxide arranged for prospective investors to conduct customer validation calls. Cantrill recalls being mindful of their design partners' time, only for customers to tell them enthusiastically: “Bring the VCs to me; I want to explain to them that I exist!” Venture capitalists frequently returned from these calls visibly shaken. When one investor reported back that the prospective customers seemed agitated, Cantrill explained that they were furious because the broader industry routinely tells them their market has vanished.

```mermaid flowchart LR accTitle: Evolution of the On-Premises Infrastructure Market accDescr: Diagram showing the two primary customer cohorts transitioning onto modern on-prem infrastructure. subgraph S1["Present Day"] A["Legacy Enterprises ('Left for dead' on-prem)"] end subgraph S2["Next Decade"] B["Cloud-Native Companies (Approaching S-1 filing / IPO)"] end subgraph S3["Destination"] C["Rack-Scale On-Premises Hardware Infrastructure"] end A --> C B -->|"Scale & margin economics (COGS)"| C ```

Cantrill emphasizes that the on-premises market is indisputably real. Today, that market consists of teams that have been “left for dead on-prem,” but over the coming decades, it will increasingly be populated by companies that were originally born in the cloud and are forced back on-prem by basic unit economics. As Cantrill put it to the investors: “Yeah, no, they're angry—I hope you didn't tell them they don't exist, because they get very, very upset if you tell them that they don't exist.”

[01:12:04](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4324s) Choosing an ISA: Intel, AMD, ARM, and RISC-V

Asked about processor architectures, Cantrill clarifies his stance on the major vendors. He harbors no animus toward Intel, remarking only that he wished Intel had evolved into a full computer company long ago rather than strictly a microprocessor company afraid of competing with its own channel. As for Microsoft, Cantrill jokes that he has avoided their software ever since “Bill Gates robbed me of my childhood,” making them an irrelevance to Oxide's platform.

Turning to microprocessors, Cantrill evaluates the options for high-performance server workloads: - **RISC-V**: Excellent for embedded microcontrollers, soft cores on FPGAs, or bounded hard cores where hardware behavior must be verified and reasoned about, but “RISC-V is nowhere near ready to run modern heavy-duty workloads.” It cannot yet compete at the cutting-edge process nodes (like 7nm) required for maximum density and throughput. - **ARM**: AWS's Graviton2 (based on the Neoverse N1 / Ares core) is impressive, but for Oxide, ARM represents a lateral move away from x86 without a compelling performance or power advantage. - **x86 (AMD / Intel)**: Cantrill has no desire to reignite historical instruction set architecture battles, observing: “I fought the ISA war before, and I really don't want to fight it again; I have long since accepted that it's going to be x86 from an ISA perspective, at least for these kind of virtualized workloads.”

[01:14:00](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4440s) Cloud Economics and the S-1 Margin Threshold

Cantrill rejects the idea that Oxide is an “anti-cloud” company. On the contrary, he argues that elastic compute is transformative and that any new startup should unequivocally begin its life in the cloud. However, organizations reach an architectural turning point once they scale.

The economic calculus flips when a venture-backed growth software-as-a-service (SaaS) business prepares to file its S-1 registration statement to go public. At that juncture, financial scrutiny shifts to gross margins, and cloud hosting costs shift from an operating line item into cost of goods sold (COGS). When hosting bills begin to visibly erode corporate margins at high scale, moving foundational workloads back to physical, customer-owned machines becomes a financial necessity.

- Anyone starting a project should begin on public cloud infrastructure. - The shift back to physical hardware is triggered when cloud spend turns into an unacceptable COGS drag. - “And when you start to view the cloud as COGS, now it's probably time to start thinking about being on your own physical machine at the scale we're talking about.”

[01:15:20](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4520s) Low-Level Rust and Embedded Systems

Addressing whether Rust can access raw hardware as intimately as C, Cantrill answers that Rust is actually far more coherent for systems programming. In traditional C operating system codebases, blending assembly with C forces the developer into compiler “best-of-luck land,” where the compiler can make unpredictable assumptions across boundaries. Rust provides explicit, structured primitives to bypass calling conventions entirely when required, such as writing naked functions without standard ABI register-saving preambles: “You have got the ability to indicate that a function should have no preamble, should have basically like, 'I want to be out of ABI land in this function.'”

```mermaid flowchart TD accTitle: Rust vs C Low-Level Safety Primitives accDescr: Comparison of compiler constraints between C and Rust for systems and embedded development. subgraph C_Lang["Traditional C Approach"] C1["Mixed C and Inline Assembly"] C2["Compiler 'best-of-luck land'"] C3["Lint: false positive storms or silent overflow"] C1 --> C2 --> C3 end subgraph Rust_Lang["Rust Systems Approach"] R1["Explicit Primitives (e.g., naked functions)"] R2["Strict Type Boundaries (no implicit sign mixing)"] R3["Compile-time overflow checks & guaranteed ABI opt-outs"] R1 --> R2 --> R3 end ```

Cantrill predicts that just as RISC-V will capture computing from the bottom up in embedded microcontrollers, Rust will conquer embedded systems software first. Embedded firmware has historically been brutal to develop and verify using C. He points the audience to the blog series *Learning Rust the Dangerous Way* by Cliff Biffle (an engineer at Oxide) to illustrate the sophisticated techniques possible when writing bare-metal Rust directly on hardware.

[01:17:49](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4669s) Integer Overflows: Why Lint Fails and Rust Succeeds

The discussion turns to integer overflow vulnerabilities in systems languages. In C, developers must meticulously craft bounds checks to avoid accidental unsigned wraparound, and traditional linters provide virtually no defense. Linters cannot aggressively flag potential unsigned integer wrap conditions because doing so generates an overwhelming flood of false positives that leads engineers to silence or ignore the tooling altogether.

Cantrill notes that while critics might dismiss integer overflow bugs as merely bad coding, the sheer difficulty of avoiding them across millions of lines of code proves that existing C tooling is fundamentally inadequate. Rust resolves this structural problem at the language level by refusing to allow implicit, transparent mixing of signed and unsigned types, forcing explicit conversions and providing predictable overflow handling. “The reality is it's really, really hard for lint—lint would generate too many false positives was the reason that lint is not warning about that condition, and people would never use them because it would never shut up, whereas Rust is much more constrained about the way you're able to use it.”

[01:19:20](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4760s) High-Speed Board Design and Signal Propagation

Asked about the source of applications returning to on-premise hardware, Cantrill references companies like Uber, who have publicly detailed migrating substantial footprints out of the public cloud and back onto on-premises infrastructure to run their proprietary distributed services.

The conversation then pivots to the physical engineering hurdles of high-speed motherboard design. Cantrill caveats that board layout is not his personal domain of expertise, but highlights the intense physics involved in routing high-frequency memory lines. Because electrical signals propagate on printed circuit boards at roughly one picosecond per inch, the trace lengths between processor sockets and memory channels must be matched with extreme precision. At the gigahertz operating speeds of DDR3, DDR4, and emerging DDR5 standards, even microscopic discrepancies in trace length introduce significant timing skews that corrupt data transmission.

[01:20:48](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4848s) Memory Controller Physics and the Mechanics of DIMM Training

An audience member asks about DIMM training and memory tuning, prompting Bryan Cantrill to delve into the physical realities of modern high-speed memory interfaces. Cantrill explains that because signal traces across a motherboard introduce varying propagation delays and react to subtle environmental conditions like temperature and voltage fluctuations, the memory controller cannot simply speak to memory blindly. Instead, the controller must perform DIMM training—a calibration process where it figures out the exact electrical timing offsets and parameters needed to reliably bridge the CPU to memory across extremely high-frequency physical buses.

Cantrill points out that this training process takes measurable time, calculating complex constants that hardware vendors traditionally guard behind proprietary, opaque blobs. While memory controllers maintain awareness of the identity of the DIMM slotted into each channel, reseating or moving a DIMM forces a full retrain of the bus. Although Cantrill readily admits that low-level silicon memory physics sits slightly beyond his personal specialty, he emphasizes why system designers avoid retraining whenever possible: “this is it you're out of my domain of expertise but it is long enough that it's not something you want to do when you if you do if you can avoid it yes”. Historically, system boot and firmware engineers have lacked deep visibility into this training phase, turning it into a notoriously difficult, black-box step during server initialization.

- DIMM training calculates dynamic timing constants based on physical trace length, board impedance, and ambient environmental conditions. - Reseating a memory module or moving it between slots disrupts previously known parameters and forces the controller to execute a retraining cycle. - The process relies heavily on vendor-proprietary algorithms that have historically offered minimal transparency or debug visibility to systems software.

---

[01:23:07](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4987s) Cloud De-coupling: Paring Down to Primitive Substrates

The conversation shifts to the broader software ecosystem when an audience question challenges whether a new on-premises hardware platform can succeed given that modern developers are hooked on high-level managed cloud services. Cantrill pushes back vigorously against the assumption that customers want total cloud vendor lock-in. While there was once an era when engineering organizations uncritically adopted every bespoke SaaS database or queuing system launched by public cloud providers, Cantrill argues that the pendulum has swung sharply in the opposite direction over the past half-decade.

Cantrill views the meteoric rise of Kubernetes and container platforms as an explicit revolt against vendor capture, allowing teams to reclaim autonomy over their operational runtime. Modern architects are deliberately avoiding sticky, proprietary offerings like Amazon Redshift, Amazon RDS, or SQS. Instead, they seek to strip their operational contract down to the absolute bare minimum: raw compute, block storage, and object storage. Cantrill states: “in my view people are really trying to pare down their dependencies to easy to EBS s3 and there are a logical go find another clouds with the idea being if you give me a say if you if you give me those services I can hydrate things up the stack but I don't want to actually build something out that's dependent upon RDS or depending upon you know sqs or what-have-you”. If a system can cleanly provide the equivalents of EC2, EBS, and S3, software developers can hydrate and run their own open-source stacks on top without becoming permanently shackled to a proprietary ecosystem.

```mermaid flowchart TD accTitle: Cloud Portability and Primitive Substrates accDescr: Diagram contrasting proprietary cloud services with a portable open substrate based on EC2, EBS, and S3 equivalents.

subgraph Proprietary["Proprietary Cloud Lock-In (Declining)"] A["Amazon Redshift / Specialized SaaS"] B["Amazon RDS / Proprietary Databases"] C["Amazon SQS / Managed Messaging"] end

subgraph Portable["Self-Hydrated Open Stack (Ascending)"] K8s["Kubernetes & Container Orchestration"] OpenApps["Open-Source Engines / Workloads"] end

subgraph Primitives["Core Commodity Primitives"] EC2["Compute (EC2 equivalent)"] EBS["Block Storage (EBS equivalent)"] S3["Object Storage (S3 equivalent)"] end

Primitives --> Portable Portable -.->|"Replaces"| Proprietary ```

---

[01:24:19](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=5059s) Toolchains, Modern Open Silicon, and Closing Remarks

Addressing an audience inquiry regarding development toolchains and cloud-hosted FPGAs, Cantrill acknowledges that services like Amazon Web Services' EC2 F1 instances allow developers to run hardware synthesis workloads remotely in the cloud. However, for Oxide's hands-on development style, they bypass hosted FPGA instances in favor of physical hardware development benches: “we don't use them because we're actually just doing an FPGA like the old-school way on what it would like a programmer and an FPGA it's sitting in front of you but those things do exist and they can be useful for certain applications it's my understanding”.

To conclude, Cantrill addresses whether Oxide's ambitious vision leaves them perilously dependent on external third parties delivering unproven technologies. He makes it clear that Oxide is riding a technological wave that is already mature and self-sustaining rather than speculative. Foundational elements like the Rust programming language have proven themselves to be battle-tested and production-ready for systems engineering. Hardware description ecosystems like Bluespec, Chisel, and SpinalHDL have already broken open, and RISC-V represents an open silicon reality that engineers can design and fabricate against immediately. Cantrill summarizes that while Oxide actively champions and contributes to these movements, the revolution is already unfolding across the industry on its own momentum: “we're not depending on anybody to deliver here in terms of these trends so the these are things that are already out there”. With that confident assessment of the hardware landscape, Cantrill thanks the Stanford audience, bringing the seminar to a close.

- Cloud-hosted FPGAs (like AWS EC2 F1 instances) serve specific remote validation needs, but direct local hardware programmers remain standard for rapid board bring-up. - Rust has matured beyond experimental status into a robust, production-grade language for writing systems and low-level firmware. - Open hardware and description languages—specifically RISC-V, Bluespec, Chisel, and SpinalHDL—provide an open ecosystem that permits new hardware companies to innovate without waiting on legacy proprietary toolchains.

## Notable quotes > [55:28](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3328s) With C you could feel the underlying assembly; with Rust you can feel the underlying C. — Bryan Cantrill

> [26:19](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1579s) Your job is to bring the host operating system its cup of coffee and shoot yourself in the head; that should be the role of host firmware. — Bryan Cantrill

> [40:42](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=2442s) It was easier to get a hardware company funded two decades ago, but hardware has never been easier than it is today. — Bryan Cantrill

> [29:22](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1762s) Interoperability is great in principle, but it is very corrosive when you want to make an actual tight integrated design. — Bryan Cantrill

> [52:47](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3167s) I fervently believe that Rust is every bit as important revolution as C. — Bryan Cantrill

> [29:22](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1762s) By rack-scale design, I mean really thinking about the entire rack; taking a bunch of 2Us and slapping it into a rack is not a rack-scale design, that is just rack and stack. — Bryan Cantrill

> [50:32](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3032s) I don't want garbage collection anywhere near the systems that I'm developing. — Bryan Cantrill

## Mentioned Bryan Cantrill, Jon Masters, Tracy Kidder, David Patterson, Steve Klabnik, Steve Tuck, Jess Frazelle, Star Simpson, Amit Levy, Phil Levis, Bill Gates, Cliff Biffle, Rick Altherr, The Soul of a New Machine, Data General Eclipse MV/8000, IBM 709, Compatible Time-Sharing System (CTSS), Hughes Aircraft, Digital Equipment Corporation (DEC), PDP-11/70, Raspberry Pi, Sun Microsystems, Sun Enterprise 10000, UltraSPARC, eBay, Intel, SPARC, IBM POWER, PA-RISC, DEC Alpha, HP ProLiant DL380 Gen5, IBM PC, Intel 8259 PIC, Google, Gigabyte, Facebook, Open Compute Project (OCP), Tioga Pass, Wedge, Bryce Canyon, Oracle, Joyent, Samsung, Amazon Web Services (AWS), American Megatrends Inc. (AMI), Bloomberg, Supermicro, Aspeed, Nuvoton, Intel Management Engine (ME), AMD Platform Security Processor (PSP), MINIX, General Electric, Barclays, General Motors, Project Cerberus, Google Titan, OpenTitan, Chromebook, Bluespec, Yosys, Chisel, SpinalHDL, RISC-V, Open Source Firmware Conference (OSFC), Coreboot, LinuxBoot, Apache ZooKeeper, DTrace, Tock OS, On the Metal, Convex Computer, Multiflow, AMD, ARM, AWS Graviton2, Neoverse N1, Uber, Kubernetes, Amazon EC2, Amazon EBS, Amazon S3, Amazon Redshift, Amazon RDS, Amazon SQS, EC2 F1, Rust, Oxide Computer Company

## Video description Bryan Cantrill Oxide Computer Company

February 26, 2020 While our software systems have become increasingly elastic, the physical substrate available to run that software (that is, the computer!) has remained stuck in a bygone era of PC architecture. Hyperscale infrastructure providers have long since figured this out, building machines that are fit to purpose -- but those advances have been denied to the mass market. In this talk, we will talk about our vision for a new, rack-scale, server-side machine -- and how we anticipate advances like open firmware, RISC-V, and Rust will play a central role in realizing that vision.

View the full playlist: https://www.youtube.com/playlist?list=PLoROMvodv4rMWw6rRoeSpkiseTHzWj6vu

► Check out our entire catalog of courses and programs: https://online.stanford.edu/explore

[Source](https://www.youtube.com/watch?v=vvZA9n3e5pc)

中文简报

Oxide借助Rust和开源固件应对企业级服务器臃肿问题传统服务器制造商让企业继续运行PC时代的硬件并采用不透明固件，迫使买家转向将开源软件深度集成至芯片级的物理机架系统。 Bryan Cantrill回顾了服务器计算的演进历程，认为现代企业级服务器仍旧只是徒有其表的脆弱台式机，背负着历史包袱、不透明的专有固件以及不安全的基板管理控制器（BMC）。虽然超大规模数据中心运营者设计了定制机架级架构来解决这些问题，但缺乏同类硬件的主流企业却面临严峻的运维障碍与云锁定经济风险。Cantrill展示了Oxide Computer Company的战略：构建一台将计算机边界置于虚拟机调配API的统一机架级机器。他说明了Rust的内存安全性、开源固件、RISC-V软核和开源EDA工具等现代技术变革，如何让一款可审计且纵向协同设计的计算机成为可能。企业数据中心硬件在很大程度上停滞不前，沦为换标的通用PC；而超大规模数据中心运营者则已转向利用直流母线与集中式电源架的定制机架级设计。 (7:44) 专有BMC固件和传统BIOS实现引入了严重的安全漏洞与遥测盲区，使得从零开始构建的开源固件模型变得不可或缺。 (23:25) Oxide Computer Company的成立旨在依托现代开源固件、Rust和RISC-V，推出面向本地企业工作负载的一体化机架级计算机。 (61:45) 随着公有云托管费用直接推高主营业务成本（COGS），云经济效益往往迫使扩张中的SaaS企业将计算工作负载遣回本地基础设施。 (74:00)

Reading summary

Bryan Cantrill, CTO of Oxide Computer Company, argues that enterprise hardware has stagnated by repackaging consumer PC architectures into servers, leaving mainstream operators stranded between brittle off-the-shelf machines and proprietary cloud monopolies. He contends that recent breakthroughs in open firmware, the RISC-V instruction set, and the Rust programming language make it feasible to build fully integrated, rack-scale on-premises systems. The Stagnation of Enterprise Hardware (7:44) Cantrill traces server evolution from massive early platforms like the 205 kW IBM 709 and the 64-processor Sun Enterprise 10000 to the commodity x86 takeover of the early 2000s. While commodity x86 chips crushed proprietary RISC economics, traditional server manufacturers such as HP and Dell trapped server design in personal computer paradigms. Between 2009 and 2020, commercial enterprise servers evolved marginally—swapping CD trays for DVD drives and VGA for HDMI—functioning essentially as rack-mounted desktop PCs running opaque, licensed firmware. Hyperscale Advantages and Enterprise Realities (11:11) Hyperscalers solved efficiency challenges by co-designing hardware at rack scale, moving from early makeshift efforts—such as Google's 2009 Velcro-secured motherboards with attached 12V batteries—to Facebook's Open Compute Project (OCP) Tioga Pass sleds, which rely on central DC bus bars to cut power conversion losses. However, hyperscaler racks consume up to 25 kW near dedicated power sources like dams, whereas mainstream corporate data centers operate within a realistic 10 to 15 kW power envelope. Cantrill criticizes vendors for denying mainstream enterprises access to bus bar-based architectures and cohesive rack-scale integration. Firmware Insecurity and Clean-Sheet Co-Design (21:07) Cantrill asserts that traditional servers run on an insecure foundation of proprietary firmware licensed from legacy vendors like American Megatrends Inc. (AMI), leaving Baseboard Management Controllers (BMCs) bloated and heavily exposed. Rather than reinventing microprocessors—where he chooses proven AMD and Intel x86 silicon for virtualized server workloads over still-maturing RISC-V compute nodes or lateral ARM options—Cantrill advocates replacing legacy BIOS and BMCs with open, attested service processors. In his view, modern computer boundaries have shifted up to the virtual machine provisioning API, necessitating open-source co-design across top-of-rack switches, storage, and hypervisors. Open Toolchains and Rust Systems Programming (40:42) Recent technological shifts make building integrated server systems viable without proprietary dependencies. Electronic design automation is opening up through tools like Yosys and high-level HDLs like Bluespec, while RISC-V provides royalty-free soft cores ideal for embedded FPGA microcontrollers. At the software layer, Cantrill argues that garbage-collected languages lack determinism under load, whereas C suffers from undefined behavior and integer overflows that linters fail to catch. Rust eliminates these vulnerabilities at compile time, allows compact footprints down to embedded FPGA targets with 512 KB of RAM, and delivers performance gains by enabling safe implementations of cache-friendly data structures like B-trees. Economic Drivers for Returning On-Premises (68:28) Cantrill rejects claims that on-premises computing is obsolete, distinguishing between early-stage development and high-scale operation. While new ventures should start on public clouds, high-volume workloads eventually transform cloud fees into a major drag on Cost of Goods Sold (COGS), especially as companies approach an IPO. Rather than remaining locked into high-level proprietary managed services, organizations are leveraging containerization and Kubernetes to decouple workloads back down to basic compute, block, and object storage primitives that run more economically on owned hardware. Cantrill maintains that open firmware, RISC-V microcontrollers, and Rust are mature, self-sustaining movements that eliminate historical startup risks in hardware engineering. However, success depends on whether enterprises facing mounting public cloud costs can effectively procure, deploy, and manage true rack-scale appliances within conventional data center constraints.

精炼阅读摘要

Oxide Computer Company 联合创始人兼首席技术官 Bryan Cantrill 指出，主流企业级服务器长期停滞于对个人电脑架构的简单拼凑与黑盒固件封装，未能适应现代云基础设施的要求；唯有基于 Rust、RISC-V 以及开源固件对整机柜软硬件展开协同重构，才能为高昂公有云成本下重返本地部署的企业提供自主可控的计算平台。企业硬件停滞与黑盒固件危机 (15:34) Cantrill 指出，与超大规模云厂商（如 Facebook 的 OCP Tioga Pass 架构，通过集中式 DC 母线供电消减转换损耗）相比，主流服务器厂商数十年间仅将机架式 PC 包装为企业级硬件，缺乏深层架构演进。这种硬件与云原生软件的割裂引发了严重的系统性故障，例如生产环境中无法及时上报的静默双比特内存损坏导致机器毫无预警地崩溃。雪上加霜的是底层固件的高度封闭性，服务器厂商普遍依赖 AMI 等第三方授权供应商的专有二进制代码，导致硬件行为完全不可见且无法审计。2018 年曝光的 Spectre 与 Meltdown 漏洞更是敲响了警钟：既然管理控制器（BMC）充斥着未经鉴权的默认配置和攻击面，攻击者根本无需物理拦截芯片即可借由固件漏洞直接穿透系统防御。整机柜协同重构与硬件信任根 (26:19) 为从根本上解决固件黑盒与接口断层，Cantrill 倡导在保留成熟现代 x86 处理器（如 AMD 或 Intel）计算性能的同时，清空传统 BIOS 和臃肿的 BMC 设计。在新型设计中，BMC 被精简为仅保留断电重启、串口通信及温控风扇基础功能的专用服务处理器，并辅以硬件信任根实现启动链的严格证明；主机固件的生命周期被严格限制为完成硬件初始化和环境证明后即退出内存，将控制权干净移交给上层操作系统。此外，Oxide 认为现代计算机的设计边界已上升至创建虚拟机的 API 端点，必须将机架视为原子计算单元，在 10 至 15 kW 的主流数据中心电力预算内，通过母线排直接供电，并将机顶交换机（ToR）、存储池与计算刀片进行软硬件深度协同设计，避免传统无约束互操作性带来的性能损耗。开源工具链、RISC-V 与 Rust 确定性基石 (40:42) Cantrill 强调，这种从白纸开始的硬件重构之所以在 2020 年成为现实，得益于成熟的开源生态演进。EDA 领域不再完全受制于闭源垄断工具，Bluespec、Chisel 以及 Yosys 等开源工具链使得 FPGA 逻辑综合与验证更加顺畅；RISC-V 指令集架构作为无需专利费的开放标准，能够以极低成本将可编程的嵌入式软核部署到 FPGA 中承担辅助计算。在系统软件层面，Cantrill 明确指出低延迟系统编程无法容忍垃圾回收（GC）机制在高负载下引发的停顿与堆扫描退化，而 C 语言的标准中又充斥着未定义行为，其历史维护的 DTrace 工具即曾因有符号/无符号整数溢出漏洞导致提权。Rust 凭借在编译期强制执行的静态所有权检查、内置的 B 树高效内存局部性以及显式的硬件抽象接口，为系统提供了免受内存溢出与 ABI 污染威胁的底层确定性支撑。云成本陷阱驱动的企业本地部署回归 (68:28) Cantrill 反驳了公有云将完全吞噬所有计算形态的行业论调。他并不反对初创企业在早期阶段利用云服务快速起步，但强调随着软件即服务（SaaS）业务规模扩大至提交 S-1 招股书冲刺上市时，云端账单将从单纯的运营支出转变为直接侵蚀毛利率的主营业务成本（COGS）。面对从初期补贴优惠到后期天价账单的陡峭跃升，如 Uber 等业务庞大的公司已公开证实正在将大量分布式服务从公有云迁回本地基础设施。Cantrill 表示，由于缺乏如超大规模云厂商那样的专属工程能力，主流企业长期以来被迫在简陋的白牌机与黑盒品牌服务器之间艰难抉择；而现代开发者借助 Kubernetes 剥离了对特定专有云服务的依赖，仅需计算、块存储与对象存储等原语，这为交付开箱即用、软硬一体的开源机架级服务器创造了庞大的真实市场需求。 Cantrill 承认在专用主板上布设 gigahertz 级高速差分信号线、处理微小物理延迟引发的内存时序训练等领域依然充满极高的物理挑战，但他坚信硬件创新的基础组件均已迈过实验阶段进入生产可用期。Oxide 选择避开 ISA 处理器之争，押注开源软硬件生态以填补公有云垄断与传统落后 OEM 之间的技术断层，其实际成效仍有待产品正式量产交付后的市场与运维检验。

Detailed notes — zh

## Background 本讲座由 Stanford Online 频道于 2020 年 3 月 4 日在 YouTube 上发布，由 Oxide Computer Company 联合创始人兼 CTO Bryan Cantrill 于 2020 年 2 月 26 日在斯坦福研讨会上发表演讲。视频简介指出，现代软件系统虽然极具弹性，但底层硬件物理基质仍停留在陈旧的 PC 架构时代，而超大规模基础设施巨头早已掌握整机定制技术却未向大众普及；演讲展示了 Oxide 打造新一代机架级服务器的愿景，以及开源固件、RISC-V 和 Rust 在其中发挥的核心作用。 ## TL;DR Bryan Cantrill 梳理了从真空管巨兽 IBM 709、DEC 小型机到现代机架式服务器的计算演进史，尖锐指出当前大众商用服务器本质上只是“套着机架外壳的 PC”，被闭源黑盒固件、不可控的 BMC 和虚假的通用性所拖累。他提出数据中心不能再依赖过度膨胀的公有云特权，而需要为普通企业打造软硬件协同设计的全新整机柜级计算机。Cantrill 结合自身在 Sun、DTrace 等数十年的系统工程血泪经验，阐述了 Oxide 如何利用生产级 Rust、RISC-V 开放架构、Bluespec 等现代 HDL 以及开源固件与硬件信任根，从底层重新发明没有历史包袱的现代服务器。 ## Atmosphere & style 整场讲座充盈着极具感染力的极客狂热与深厚的系统工程浪漫主义，Cantrill 语速极快、引经据典，将深奥的底层架构剖析转化为充满幽默感、自嘲与辛辣讽刺的戏剧性叙事。现场气氛在技术历史的宏大叙事与一线研发的真实挫败之间自由穿梭，从吐槽 C 语言未定义行为和失控 BMC 的啼笑皆非，到看到 Rust 压倒性性能时的“又惊又怒”，再到为被公有云账单逼入绝境的本地运维客户击节鸣不平，听众能真切感受到一位老牌系统黑客对打破行业旧体制的纯粹热情与执着信念。 ## Bulletin - [08:51](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=531s) 大众市场服务器架构长期停滞在 PC 时代的遗留产物，传统机架设备仍保留光驱和 VGA/HDMI 等冗余设计，无法满足现代数据中心的密集与高效需求。 - [13:22](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=802s) 超大规模云服务商早已转向定制机架级架构与集中式母线排供电，但这些经过验证的能效与维护创新长期未能普及至普通企业级硬件市场。 - [21:07](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1267s) Oxide Computer 提出软硬件协同设计的整机柜服务器新范式，依托 Rust、RISC-V 与开源 EDA 工具链重构失控的专有固件与 BMC 架构。 - [01:08:28](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4108s) 面对高昂的公有云账单与厂商锁定，越来越多成长型企业寻求回迁本地部署，但缺乏统一交付、开箱即用的现代化私有云硬件基座。 ## Key takeaways - [01:36](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=96s) IBM 709 标志着服务端计算的萌芽，通过分时系统开启了用户无需直面物理设备即可远距运行业务负载的计算模式。 - [05:51](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=351s) 早期互联网依赖如 Sun E10k 等大型纵向单机系统承载业务，但 eBay 等公司的宕机惨剧暴露了缺乏容灾与高可用设计的单点故障风险。 - [07:44](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=464s) x86 微处理器凭借压倒性的规模经济效应摧毁了 SPARC、POWER 等 RISC 架构，但同时也把初代 IBM PC 的历史兼容包袱全面拖入了数据中心。 - [11:11](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=671s) Google 初代自建服务器因过度压缩成本使用非 ECC 内存与翻新零件，导致上层软件排障成本激增，印证了盲目追求廉价硬件的反噬。 - [13:22](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=802s) Facebook 等厂商通过 OCP 推动机架级集中供电（DC Bus Bar）、前置布线及宽机架形态，大幅降低了电源转换损耗并提升了运维效率。 - [17:20](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1040s) 传统服务器专有黑盒固件（如 AMI 代码）常年缺乏可审计性，甚至静默吞噬可纠错内存错误，剥夺了上层监控对严重故障的预警能力。 - [24:10](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1450s) 基板管理控制器（BMC）作为系统内运行未知闭源软件且拥有直接总线访问权限的“第二台计算机”，已经沦为现代数据中心最脆弱的攻击面。 - [27:04](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1624s) 宿主固件应遵循“引导内核后立即退出并释放控制权”的极简原则，彻底终结源于 DOS 时代的常驻内存与未受控系统管理模式（SMM）。 - [29:22](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1762s) 真正的机架规模设计必须打破组件间零散的通用互操作性，将计算、存储与 ToR 交换机深度绑定协同研发，以整机柜对外暴露标准化接口。 - [30:48](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1848s) 现代计算机的软件边界已演变为置备虚拟机的 API 端点，硬件基础设施必须与包含 Hypervisor、网络及控制平面的开源软件栈进行协同设计。 - [36:54](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=2214s) Spectre 与 Meltdown 漏洞暴露了推测执行的底层隐患，彻底打破了硬件无懈可击的幻象，并强力推动了业界对开源固件与硬件信任根（如 OpenTitan）的落地。 - [40:42](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=2442s) Bluespec、Yosys 等开源 EDA 工具的成熟以及 RISC-V 开放架构的普及，使小型团队也能在 FPGA 层面低成本获得灵活的底层芯片级可编程能力。 - [50:32](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3032s) 垃圾回收机制在业务高负载与大内存场景下会引发不可控的暂停与堆扫描，因而无法满足底层系统工程对绝对确定性的严苛要求。 - [52:47](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3167s) C 语言充斥着未定义行为与隐蔽的整数溢出缺陷，且由于缺乏编译器强约束的所有权模型，极难安全地表达高效数据结构（如 B 树）在跨库调用时的内存契约。 - [55:28](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3328s) Rust 依靠编译期所有权模型兼顾了极小内存占用与高阶抽象，不仅在生产实践中超越了手写 C 代码的性能，也杜绝了静默的内存与整数安全隐患。 - [01:08:28](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4108s) 随着初创企业走向公开市场，公有云账单从最初近乎免费的试用额度演变为吞噬企业毛利的重度 COGS 成本，迫使其考虑基础设施的回迁与本地掌控。 - [01:12:04](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4324s) 在计算主核心架构的选择上，x86 依旧是高密度虚拟化工作负载的最成熟现实解，而 RISC-V 现阶段的最佳阵地是极度要求可控性的嵌入式与板级管理控制器。 - [01:23:07](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4987s) 行业普遍拥抱 Kubernetes 的深层诉求在于确立计算、块存储与对象存储等最基础的基础设施原语，防止企业被公有云厂商的高层专有托管服务深度绑定。 ## Detailed walkthrough ### [00:11](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=11s) 讲座开场与《新机器的灵魂》 Bryan Cantrill 开门见山地介绍了自己的身份——Oxide Computer Company 的联合创始人兼 CTO。他在开场时开玩笑说自己有点担忧，因为好友 Jon Masters（前 Red Hat、现 NUVIA 工程师）在 Twitter 上推介这场讲座时，只截取了冒号前面的前半部分标题：“Bryan 将就《新机器的灵魂》（The Soul of a New Machine）发表演讲”。结果互联网上有相当多的人误以为这会是一场关于特雷西·基德尔（Tracy Kidder）同名普利策获奖纪实名作的读书分享会。Bryan 明确澄清今天不是来开读书会的，但他坦言自己对那部作品推崇备至，称其为计算机领域的荷马史诗。基德尔在书中生动描绘了 20 世纪 70 年代末至 80 年代初数据通用公司（Data General）代号为“Eagle”（即 Eclipse MV/8000）的 32 位超级微型机的研发历程。Bryan 解释说，自己之所以借用这个书名，并非对原作不敬，而是希望借用基德尔笔下那种对计算机系统工程纯粹的热情、深厚的专注与探索精神。面对人们常常问他是否过度浪漫化了 70、80 甚至 90 年代的技术岁月，Bryan 笑着承认可能确实有一点点，但他依然认为，那是工程技术最具开拓性的黄金时代之一。 Bryan Cantrill 评价道：“It is the Iliad or the Odyssey; it is the literature of our domain, but I'm only using it as a title.”（它是我们这个领域的《伊利亚特》或《奥德赛》，是我们行业的经典文学，但我今天只是借用它作为标题。） ### [01:36](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=96s) 服务端计算的诞生与巨兽 IBM 709 在 Bryan 看来，真正意义上的“第一台服务端计算机”可以追溯到 IBM 709。作为真空管计算机时代的最后一批庞然大物，IBM 709 拥有足够强大的算力来承载早期分时操作系统（Compatible Time-Sharing System, CTSS）的演示。Bryan 指出，这标志着“服务端计算”（server-side computing）的雏形初现：人们不再需要直接坐在计算机面前触摸、观察甚至嗅探硬件，计算机可以在远距离之外独立运行用户的业务负载。 Bryan 展示了休斯（Hughes）公司当年部署 IBM 709 的普查统计数据，以此对比后晶体管时代的微型化奇迹。真空管不仅能耗极高，而且物理体积大得惊人。一台 IBM 709 的功耗高达 205 千瓦（作为对比，现代一个数据中心标准机架的功耗通常仅在 10 到 25 千瓦之间），重量达 33,000 磅（约 15 吨），设备本身占地 1,900 平方英尺，甚至连专门为其降温的配套空调系统也需要占据整整 1,000 平方英尺的空间。虽然从现代算力角度看它微不足道，但这头“吞云吐雾的巨兽”却真正开启了多用户共享计算资源的时代。关于服务端计算的本质，Bryan Cantrill 总结道：“By server-side computing, I mean a computer that you don't necessarily hold in your hands, you don't have in front of you, you don't necessarily see or hear or smell; it is running your workload at some kind of a distance.”（我所说的服务端计算，是指你不需要捧在手里、不需要放在眼前、甚至完全看不到、听不见或闻不到的计算机；它在与你相隔一定距离的地方运行着你的工作负载。） - **IBM 709 规格参数（1961 年普查数据）**： - 功耗：205 kW（现代单机架约为 10 - 25 kW） - 重量：33,000 磅（约 15 吨） - 机器占地面积：1,900 平方英尺 - 配套空调占地面积：1,000 平方英尺 - 历史地位：运行 CTSS（兼容分时系统）展示的首台机器，奠定现代分时服务端计算基础 ### [04:10](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=250s) 小型机革命与 DEC PDP-11/70 顺着计算历史快进到 20 世纪 70 年代，微型计算机（Mini-computer）革命彻底颠覆了 IBM 大型机（Mainframe）一手遮天的格局。Bryan 略过了经典的 IBM System/360，重点展示了对整整一代计算机工程师如同“母乳”般亲切的机器——DEC PDP-11/70。这类机器成功大幅度缩减了设备尺寸，使得组织和科研机构能够在本地直接运行自己的计算任务，而不必事事依赖昂贵庞大的集中式大型机。现场出现了一段有趣的互动：Bryan 提到 Oxide 的一位联合创始人（Jessie Frazelle）在家里摆放了一台 PDP-11/70 的微缩复刻模型，里面塞入了一块树莓派（Raspberry Pi）来控制面板上的指示灯不断闪烁。台下观众随即探讨起这类系统当年运行的具体操作系统是 TOPS-10、TOPS-20 还是 Unix。Bryan 确认 PDP-11/70 承载了早期 Unix 操作系统的繁荣。从商业角度看，DEC 最初预期只生产 10,000 台，但最终销量突破了 10 万台以上，彻底奠定了 DEC 作为计算巨头的产业地位。 Bryan Cantrill 对该机器的市场表现评价道：“They expected to make 10,000 of these, and they made 100,000-plus; really made the company.”（他们原本预计只生产 10,000 台，但最终造了超过 100,000 台；这真正造就了 DEC 这家公司。） ### [05:51](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=351s) 互联网泡沫的支柱：Sun Enterprise 10000 时间来到 20 世纪 90 年代末，Bryan 提到了自己工作了长达 14 年的母校——Sun Microsystems，并展示了被称为“Starfire”的巨型服务器 Sun Enterprise 10000（Sun E10k）。当时的宣传照甚至特意采用仰视视角拍摄，力求让机柜显得极具压迫感。这台配备 64 颗 SPARC 处理器、64 GB 内存的庞然大物，在 1999 年前后撑起了早期互联网相当大一部分核心业务。作为亲历过互联网泡沫破裂（Dot-com bust）的从业者，Bryan 幽默地告诫台下的年轻一代：千万不要误以为科技行业永远不会经历经济衰退。大型横向单机系统（scale-up）在为新兴互联网提供充沛算力的同时，也埋下了可怕的技术债务。Bryan 举了 eBay 极具代表性的崩溃惨案：eBay 最早运行在双处理器机器上，随着业务暴增，他们一路升级，最终将核心业务无缝迁移到了一台 64 处理器巨兽 Sun E10k 上。然而，eBay 在整个架构演进过程中从未重新审视过系统可用性，也从未设计过可以让机器停机维护的容灾方案。Bryan 形容他们等于在不知不觉中签下了死刑判决书。1997 年，eBay 核心服务器宕机整整 22 个小时，造成了灾难性后果，但也让全行业深刻意识到：商业活动已经不可逆转地被绑定在那些尚未做好高可用性准备的服务端硬件上。 Bryan Cantrill 直言：“eBay started off on a two-processor machine, ended up with one of these 64-processor giants, and because they didn't revisit their architecture along the way, hadn't bothered to make where they could take the machine down for any kind of maintenance at all, so of course they were operating under a death warrant that was signed that they didn't quite realize.”（eBay 最早从一台双处理器机器起步，最终换上了这台 64 颗处理器的庞然大物；但由于一路上从未重新审视架构，甚至懒得做到能让机器停机进行哪怕任何一种日常维护，他们实际上不知不觉间背负了一纸随时会生效的死亡判决书。） ```mermaid flowchart TD accTitle: eBay 架构演进与单点故障陷阱 accDescr: 展示了从低配双路服务器到 64 处理器 Sun E10k 的单体纵向升级，因缺乏冗余设计导致最终灾难性宕机。 subgraph Growth["系统规模演进"] A["初期架构: 双路处理器机器"] -->|"业务急速扩张 （缺乏冗余与容灾设计）"| B["终极架构: 64路 Sun E10k 单机"] end subgraph Catastrophe["技术债务爆发"] B -->|"无法停机维护 单一单点故障（SPOF）"| C["1997 年核心宕机长达 22 小时"] C --> D["催生对高可用/横向扩展架构的反思"] end ``` ### [07:44](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=464s) 微处理器大战：x86 的成本碾压与 RISC 的黄昏进入 21 世纪初期，服务器体系架构迎来了历史性拐点。以 Sun 的 SPARC 为代表，包括 IBM POWER、MIPS 以及 DEC Alpha 在内的诸多精简指令集（RISC）微处理器，在商业竞争中全面溃败给英特尔的 x86。Bryan 分析指出，造成这一局面的并不是纯粹的技术架构优劣，而是残酷的规模经济学（Economics）。企业用户很快意识到，虽然 64 颗处理器的 Sun 大机性能强劲，但一台基于廉价 x86 架构的小机器即使只有其四分之一或十分之一的 CPU 算力，其硬件采购成本却仅为大机的二十分之一甚至百分之一。面对如此悬殊的性价比差距，单体巨型服务器被彻底判了死刑，微处理器革命将原本由精巧大机占据的数据中心市场席卷殆尽。 Bryan Cantrill 对 RISC 的消亡与 x86 的胜出评价道：“SPARC and every other RISC microprocessor effectively lost the plot to Intel x86, and what happened was that the microprocessor in here and in Power machines and in MIPS RISC-based machines and in Alpha machines could no longer compete with the economics of the x86.”（SPARC 以及其他所有 RISC 微处理器实际上都输给了英特尔 x86；无论是这类机器、POWER 机器、基于 MIPS 的机器还是 Alpha 机器，其中的微处理器都再也无法在经济性上与 x86 抗衡。） ### [08:51](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=531s) 2009 年的倒退：披着服务器外衣的个人电脑微处理器革命虽然摧毁了昂贵的纵向单体大机，却给数据中心带来了始料未及的架构后遗症。Bryan 展示了 2009 年的行业主流服务器——惠普 HP ProLiant DL380（第 5 代，Gen 5）。在他看来，这台机器代表了服务器设计的一种倒退与悲哀。尽管这是一台安装在专业机柜里的标准机架式服务器，但面板上却赫然装着一个 CD 光盘托盘和一个 VGA 显示端口。Bryan 讽刺地质问：“难道我们要在数据中心里放 CD 听音乐吗？为什么服务器面板上会有 CD 托架和显示器接口？” 原因在于，这台所谓的现代服务器在体系结构上本质上只是一台强行塞入机箱的“个人电脑”（Personal Computer）。1979 年 IBM 在佛罗里达州博卡拉顿（Boca Raton）研发初代 IBM PC 时，原本以为这只是面向业余爱好者的边缘产品、预计只卖 10,000 台，结果却售出了数以千万计。x86 击败其他微处理器后，也将 IBM PC 体系架构的所有历史包袱一股脑拖进了 21 世纪的数据中心。直到 2009 年，服务器内部依然充斥着大量早已停产的古老 PC 芯片的逻辑影子——例如必须在硬件或固件中继续软件仿真的 Intel 8259 可编程中断控制器。与此同时，面对急剧膨胀的互联网业务规模，超大规模数据中心运营者们终于开始意识到，传统的 PC 延伸架构已经走向了死胡同。 Bryan Cantrill 尖锐地批评这种架构妥协：“And the answer is that this is actually, despite it being rack-mounted, this is a personal computer; this is architecturally a personal computer.”（答案在于，尽管它是机架式的，但它实际上是一台个人电脑；在体系架构层面上，它彻头彻尾就是一台个人电脑。） - **HP DL380 代表的 PC 架构在机架服务器中的历史包袱**： - 前面板 CD 光驱托盘（数据中心内无物理交互需求） - 前面板集成 VGA 显示接口（沿袭桌面图形输出设计） - 架构沿用 1979 年 IBM PC 团队在博卡拉顿做出的短期决策 - 强制向下兼容并软硬件仿真过时元器件（如 Intel 8259 中断控制器） ### [11:11](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=671s) 超大规模计算的觉醒：Google 的“魔术贴”与廉价零件代价大约在 2009 年，随着业务规模的急剧扩张，以 Google 为代表的科技巨头对市面上的现成服务器感到极其失望。他们清醒地意识到，数据中心不需要在机架上塞满个人电脑（PC），而需要一种真正针对数据中心场景量身定制的专用机器。Bryan Cantrill 展示了 Google 当年极为出名（甚至可以说是声名狼藉）的第一代自研服务器设计。在这台早期的自建机器上，Google 把降本做到了近乎荒谬的地步：硬盘和电源是用魔术贴（Velcro）粘在机架托盘上的；为了省去昂贵的不间断电源（UPS），他们在每台服务器旁边直接绑了一个 12V 的汽车级电池作为备用电源；主板选用的是低廉的技嘉（Gigabyte）主板，Bryan 打趣说这种主板“就好像是从工厂地板上随手扫起来的一样”。这种省去每一个便士、全力追求最廉价硬件的策略看似合乎逻辑——毕竟一次要采购海量机器。然而，Google 很快领会了一个在计算领域颠扑不破的古老教训。 Bryan Cantrill 直言，追求价值无可厚非，但过度压榨成本必然会遭遇“一分钱一分货”的报应：“You always want to seek value, but you have to be careful about unwittingly really getting what you paid for.” 当他们为了省钱而使用翻新的 DRAM，甚至采用缺少纠错能力的非 ECC 内存时，整个软件栈上方随之遭遇的系统性崩溃和排障代价，远远超过了硬件层面省下的那几枚硬币。不过，Google 和后来的 Facebook 等公司确实找准了正确的方向：他们大刀阔斧地剔除了数据中心根本不需要的个人电脑冗余附件（ephemera and accoutrements），例如光驱（CD Drive）、VGA 视频输出接口等，开启了服务器向专用化演进的历程。 - **Google 初代自研服务器特征**：魔术贴固定磁盘与电源、每台机器外挂 12V 电池替代集中式 UPS、极度压缩成本的消费级主板。 - **惨痛教训**：翻新元器件与非 ECC 内存导致上层软件栈频现难以追溯的硬件错误，总体维护成本远超初期节省的硬件采购款。 --- ### [13:22](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=802s) Facebook 与开放计算：从独立电源到机架级架构随着时间推移，超大规模基础设施设计进入了成熟期。Bryan Cantrill 以 Facebook 在 2016 至 2017 年间推出的 Tioga Pass 计算节点（Compute Sled）为例，展示了现代化超大规模硬件的形态。与早期每个节点自备电源适配器的做法完全不同，现代计算节点上的电源不见了，取而代之的是直接插入机柜背部的直流母线排（DC Bus Bar）。机架中间设立了专门的电源机架（Power Shelf），负责集中进行交流到直流（AC-DC）的电压转换。Bryan Cantrill 指出，在任何具有一定规模的设施中，这种集中供电都合理得多，因为行业很早就意识到“电源转换极其低效，转换次数越多，能量损失越大”。在数据中心全力降低电源使用效率（PUE, Power Usage Effectiveness）的背景下，必须尽可能减少不必要的交直流反复变换，杜绝能源白白变成机房废热。除了母线排供电，这一时期的机架级创新还包括： 1. **前置线缆设计**：所有网络接口与布线全部改在服务器正面，大幅提升运维可维护性与可访问性。 2. **超越 19 英寸机架标准**：放弃了源自电话时代的陈旧 19 英寸机柜标准，采用可在 24 英寸地板砖上摆放的更宽、更高的定制机架，单个机箱宽度内可横向并排容纳 3 个计算节点（而非传统的 2 个）。 3. **机架级协同生态**：配合自家研发的 Wedge 网络交换机和 Bryce Canyon 存储机箱，形成了软硬件深度协同的完整机架级产品，并且全部通过开放计算项目（Open Compute Project, OCP）开源。 ```mermaid flowchart TD accTitle: 传统机架与现代超大规模机架供电架构对比 accDescr: 对比传统机箱内部各自配备交流电源，与现代机架集中通过直流母线排供电的能效差异 subgraph Traditional["传统机架设计 (19 英寸)"] AC["数据中心 AC 交流电输入"] S1["服务器节点 1 (内置独立 PSU)"] S2["服务器节点 2 (内置独立 PSU)"] AC --> S1 AC --> S2 end subgraph Hyperscale["现代超大规模机架 (宽机架)"] AC_IN["数据中心 AC 交流电输入"] PS["集中式电源机架 (Power Shelf)"] BB["机架直流母线排 (DC Bus Bar)"] N1["计算节点 1 (Tioga Pass Sled)"] N2["计算节点 2 (Tioga Pass Sled)"] N3["计算节点 3 (Tioga Pass Sled)"] AC_IN --> PS PS --> BB BB --> N1 BB --> N2 BB --> N3 end ``` --- ### [15:34](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=934s) 大众市场的停滞：“机架式 PC”与基础设施特权正当超大规模厂商大步流星走向现代化架构时，面向大众商用市场的服务器形态却几乎冻结在原地。Bryan Cantrill 调侃地对比了 2009 年与 2020 年的商用主流服务器（以惠普 HP 为代表）：“坦白讲，不是故意针对 HP，但抱歉了 HP，我们从 2009 年走到了 2020 年，最显著的‘进步’无非是从 CD 光驱变成了 DVD 光驱，接口从 VGA 升级成了 HDMI。” 这些所谓的改进本质上是为单台桌面 PC 乃至 Windows 个人工作站设计的，而不是为一个严肃的数据中心环境打造的。商用市场卖给大众的依旧是“装在机架上的个人电脑”（rack-mounted personal computer）。 Bryan Cantrill 表达了深切的挫败感：虽然 Facebook 和 Google 慷慨地公布了所有前沿硬件设计，但对于没有这些巨头规模与资源的普通企业而言，这种落差反而放大了痛苦：“It sharpens the pain for those of us who don't have the infrastructure privilege of these folks to know that like actually there's great innovations we just can't have them.” 他提到自己在 Sun Microsystems 经历了 14 年，在 Sun 被甲骨文收购（他戏称为“被入侵”）后离开，随后在一家云计算公司（后被三星收购）工作了 10 年，负责搭建庞大的私有云。在这个过程中，他们不得不被迫采购这些商用机架 PC，其带来的后果绝不仅是审美上的鄙视，更是直接波及从硬件到软件全栈的严重可靠性缺陷。 - **商用服务器现实**：2020 年出厂的商业服务器依然标配 DVD 驱动器和 HDMI 接口，本质仍未摆脱个人电脑架构。 - **基础设施特权鸿沟**：先进的机架级设计仅供拥有“基础设施特权”的超大规模厂商自建，大众市场被阻隔在外。 --- ### [17:20](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1040s) 软硬件割裂的僵局与“吃掉错误”的固件隐患面对公共云大潮，Bryan Cantrill 提出了一个鲜明的反共识观点：“I actually don't believe that Jeff Bezos is going to own and operate every computer on the planet.” 他认为，出于经济成本核算、数据安全和风险管理等必然考量，许多企业在算力达到一定规模后，拥有并运行自己的物理硬件远比向云服务商长期租用更合算。然而，企业自建私有云的最大阻碍在于软硬件严重割裂的行业僵局： - **硬件制造商的局限**：服务器制造商被困在硬件这一层，其目标是制造出一台能向上兼容运行 Windows 的标准 PC，工程师即便清楚这并非最佳数据中心架构，但他们既不拥有也不控制上层软件栈。 - **软件开发者的傲慢与脱节**：掌控上层软件栈的组织往往既不理解硬件底座，也没有动力去适配和管理硬件底层。这种长达十余年的对峙导致任何想要自建私有云的企业，只能把市面上的商用 PC 服务器与并不适配大规模云运营的开源软件拼凑在一起，自行承担设计、集成、运营乃至最折磨人的全天候支持责任。 Bryan Cantrill 分享了他们在实际生产中遭遇的深切痛点：系统曾遭遇过大面积的内存（DIMM）故障危机。在 ECC 内存的设计原理中，罕见且致命的不可纠错错误（Uncorrectable Errors）通常不会突然凭空出现；在此之前，系统应当会先监测到逐渐激增的单比特可纠错错误（Correctable Errors），这往往代表着内存硬件训练不足（DIMM training）或电气设计存在潜在缺陷。软件团队本可以通过监控这些早期征兆提前告警、转移负载或强制内存重新训练。然而在现实中，服务器底层的黑盒固件（Firmware）在硬件底层将所有的可纠错错误信息直接“吞食”并隐瞒了下来。上层操作系统和监控软件对此一无所知，直到突如其来的不可纠错故障直接导致内核崩溃（Kernel Panic）。由于无法穿透专有固件的黑盒盲区，工程师对即将到来的软硬件灾难完全丧失了预警与干预能力。 ### [21:07](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1267s) 从零设计一台新计算机：保留微处理器，重塑固件生态在传统的企业级服务器中，试图让系统固件正确向上层操作系统报告并处理硬件错误，其难度之大堪比“在国会推动通过一项法案”。Bryan Cantrill 形象地吐槽道，如果你去找一家个人电脑制造厂商请求定制一份固件版本，对方的反应往往是黑人问号脸：“为什么会有人想在一台个人电脑上跑自己的专属固件？这根本说不通啊！”而运维者的回答只能是：“我跑的根本不是个人电脑，我跑的是云计算平台！”这种个人电脑思维与现代云基础设施需求之间的巨大断层，不仅让 Oxide 团队深感刺痛，也是整个行业长期以来的共性难题。因此，在 2020 年，必须拿出一张白纸从零开始重新审视计算机设计。不过，从头设计并不意味着重新造轮子去搞通用微处理器——无论是 Intel、AMD 还是 ARM，它们制造的 CPU 都极其复杂且非常契合企业级用途，真正亟待彻底改造的，是位于底层且失控的固件。现代服务器固件的状态混乱得令人窒息。管理员在机器上刷入官方 BIOS 更新时，对其内部究竟运行了什么代码一无所知。Bryan Cantrill 指出，只要对这些固件二进制文件稍作粗略分析，结果就会让人彻底失眠：“你会看到像 URL 之类的东西；一个 URL 凭什么会出现在我的 BIOS 里？这可是我数据中心的脑干，为什么里面会嵌着一个 URL？”去向厂商质问时，你甚至找不到任何能给出答案的人。这倒未必是因为供应商被什么恶意组织渗透了，而是因为连他们自己都不知道里面有什么代码；即便有些厂商声称愿意开放源码，核心固件其实也不是他们自己写的。令人难以置信的是，像 AMI（American Megatrends Incorporated）这种早在 Bryan 童年时期就存在的古老公司，至今居然仍然掌控着整个人类数字基础设施的核心命脉：“AMI 怎么可能至今还在人类数字文明的脑干位置上运行？”这种由不可见的闭源第三方固件堆叠起来的黑盒，潜藏着极其巨大的安全风险。 - 微处理器策略：保留 Intel、AMD 或 ARM 等成熟的高性能商用 CPU，不重造微处理器。 - 固件现实问题：闭源 BIOS 二进制包臃肿不堪，甚至暗藏未知 URL，缺乏可审计性与测试验证机制。 - 供应链困境：服务器整机厂商大多直接采购 AMI 等第三方 BIOS 供应商的代码，自身也无法掌控或审计底层固件源码。 ```mermaid flowchart TD accTitle: 传统服务器与新型设计理念的固件及硬件层对比 accDescr: 对比传统黑盒固件体系（堆叠 AMI 闭源代码与失控 BMC）与全新设计的透明固件体系（微处理器保留，重构服务处理器并移除冗余）。 subgraph Legacy["传统服务器架构（PC 时代的遗产）"] L_CPU["商用 CPU (Intel/AMD)"] L_FW["第三方闭源 BIOS (如 AMI)\n（混杂未知 URL 与残留驱动）"] L_BMC["失控的 BMC 控制器\n（运行闭源系统、开放网络/PCIe）"] L_CPU --- L_FW L_FW --- L_BMC end subgraph Modern["Oxide 理想的全新设计架构"] M_CPU["商用 CPU (Intel/AMD/ARM)"] M_SP["极简服务处理器 (Service Processor)\n（仅负责电源管理、串口控制、风扇）"] M_RoT["硬件信任根 (Hardware Root of Trust)\n（全链路固件可度量、可验证）"] M_CPU --- M_SP M_SP --- M_RoT end ``` ### [24:10](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1450s) 计算机体内的“计算机”：失控的 BMC 与虚妄的供应链阴谋论谈及 2018 年秋季《彭博商业周刊》（Bloomberg）关于超微（Supermicro）主板遭遇微型芯片供应链入侵的轰动报道，Bryan Cantrill 认为该报道所描述的极其精密的硬件攻击在现实中大概率并未发生，因为根本没有攻击者会去大费周章植入物理芯片。理由非常简单：“frankly when you can just walk in through the firmware”（坦率地说，你完全可以直接从固件大摇大摆走进去）。现实中大部分 Supermicro 服务器的基板管理控制器（BMC）都保留着默认的 root 密码，并直接暴露在公网之上。例如安全研究员 Rick Altherr 在开源固件大会（OSFC）上公布名为“USB Anywhere”的严重漏洞时提到，他在公网上轻易扫描到了 77,000 台随时可以被直接接管的暴露出网的 BMC。 BMC 本质上就是寄生在主计算机内部的另一台全功能计算机。它不仅搭载了自己的处理器，还跑着 U-Boot、Linux 以及来自 ASPEED 或 Nuvoton 的完全闭源私有软件。更致命的是，这台运行着未知黑盒代码的小机器同时扮演着 PCIe 总线主控（Bus Master），直通服务器的底层总线，而且厂商还建议用户将其直接连接网络。Bryan 强调，必须彻底终结这种畸形膨胀的 BMC 生态，重拾传统意义上功能明确的“服务处理器”（Service Processor）。服务处理器的职责极为纯粹：控制电源重启机器、提供物理串口通信，以及控制风扇转速即可；它绝不需要提供虚拟 VGA、虚拟 USB，甚至视具体网络布线架构而定，根本不该挂载复杂的网络协议栈。 “frankly when you can just walk in through the firmware” —— Bryan Cantrill - 彭博社 Supermicro 报道的谬误：现实世界的物理芯片植入缺乏实证，而直接通过固件和暴露的接口进行攻击的成本远比硬件供应链篡改低得多。 - BMC 的高危现实：Rick Altherr 披露“USB Anywhere”漏洞期间，在互联网上发现了 7.7 万台存在隐患且暴露在外的 BMC。 - 架构缺陷：BMC 拥有 PCIe Bus Master 权限，可以直达主机总线并访问系统内存，内部充斥着 ASPEED 或 Nuvoton 的不可见闭源驱动。 - 服务处理器（Service Processor）职责重构：剔除虚拟 VGA、虚拟 USB 等冗余功能，仅保留上下电控制、串行控制台访问以及必要的风扇散热监控。 ### [27:04](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1624s) 宿主固件的使命与隐藏的微控制器系统对于主机固件（Host Firmware）的生命周期定位，Bryan Cantrill 给出了一个极为生动犀利的论断：“host firmware what your job is is to bring the host operating system its cup of coffee and shoot yourself in the head”（宿主固件，你的全部职责就是给宿主操作系统端上一杯咖啡，然后给自己脑袋来上一枪）。这种要求固件在引导操作系统之后仍长期驻留内存、随时以不可见方式劫持 CPU 任意执行系统管理模式（SMM）代码的做法，本质上是 DOS 时代的残余心态。Bryan 自嘲 DOS 彻底剥夺了他的童年：“我是在内存保护机制早已诞生的年代里长大的，但小时候用的 DOS 却根本没有内存保护，这让我当年一直天真地以为计算机本来就会莫名其妙地随机重启。”既然现代操作系统具备完善的保护机制，基础设施就绝不能再依赖在阴暗角落里乱跑的未受控常驻固件。除了 BMC 之外，主板上甚至还潜藏着更令人不安的深层微系统。在 Intel 平台上，这是 Intel Management Engine（ME）；在 AMD 平台上，则是 Platform Security Processor（PSP）。Intel ME 本身就是一颗运行在芯片组内部、独立运行 MINIX 操作系统并对整机软硬件拥有绝对控制权的独立计算机，并且历史上多次被曝出严重安全漏洞；AMD 的 PSP 虽然集成在芯片裸晶（die）上相对更令人踏实一些，但其固件依然是闭源黑盒。Bryan 直言，现代数据中心应该具备一项最基本、最合情合理的能力，那就是“能够理解在你的计算机上执行的每一条指令”。即便当前产业离这一目标还有很长的路要走，也必须从建立硬件信任根（Root of Trust）和推动固件完全透明化做起。 “host firmware what your job is is to bring the host operating system its cup of coffee and shoot yourself in the head” —— Bryan Cantrill - 宿主固件定位哲学：固件仅负责完成硬件初始化并将控制权交给操作系统内核，随后必须彻底退出执行并释放控制权，杜绝类 DOS 式的无保护驻留模式。 - Intel ME 与 AMD PSP：Intel ME 运行 MINIX 系统并具备系统全权控制力，AMD PSP 尽管集成于片内但同样属于完全闭源系统。 - 可观测性追求：运维者有权审查并明确机器中每一条正在执行的指令，而不是放任黑盒系统在操作系统底层静默运行。 ### [29:22](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1762s) 真正的整机柜设计：软硬件协同设计打破通用互操作性枷锁实现新型计算机的另一个支柱是推行真正的“整机柜规模设计”（Rack Scale Design）。Bryan Cantrill 批评道，业界很多人对机柜级设计的理解极其肤浅，以为去市面上买上一堆 2U 服务器随便塞进机架里堆叠起来（Rack and Stack）就算机柜设计了。真正的整机柜设计，必须将机架顶部交换机（Top-of-Rack Switch, ToR）、网络、存储与计算节点进行全方位的深度融合与协同设计（Co-design），使硬件各组件之间能够原生感知彼此的存在。然而，如果缺乏深度绑定的系统软件，真正的整机柜设计就根本不可能成立。以 Intel 为例，他们长年累月投入大量精力研发机架规模架构，但始终刻意回避自研软件层，认为软件应当由其他人负责，最终导致整套方案无法落地交付。此外，传统观念中推崇的“无底线通用互操作性”（Interoperability）在深度集成的系统设计中实际上起到了反作用：“interoperability is great in principle but it is very corrosive when you want to make an actual tight integrated design”（互操作性在原则上非常美好，但当你想做一个真正紧密集成的系统设计时，它具有极强的破坏性）。为了打造极致密集的服务器形态与高效系统，设计团队不能再去适配市场上任意型号的 ToR 交换机，而是必须将 ToR 交换机直接纳入整机柜一体化研发体系，并将系统与外部网络的互操作边界向外推移至整个机柜的对外接口处，从而实现超大规模云计算中心所独有的超高计算密度。 “interoperability is great in principle but it is very corrosive when you want to make an actual tight integrated design” —— Bryan Cantrill - 伪机柜与真机柜的区别：仅仅将现成的 2U 服务器堆入物理机架只是 Rack and Stack，真正的机架规模需要网络、存储、计算及 ToR 交换机的一体化协同设计。 - 软件与硬件的强绑定：硬件本身无法独立实现机柜自治，必须由统一的系统软件栈驱动整体调度。 - 互操作性边界重构：舍弃对内部组件与任意第三方交换机的零散兼容，将机架作为单一计算机单元对待，把标准化互联边界推到机架整体对外的网络层。 ### [30:48](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1848s) 企业级机架的物理现实与软硬件协同设计理念 Bryan Cantrill 指出，超大规模云服务商（hyperscalers）拥有极其独特的优势：他们的数据中心从零开始自主设计，甚至直接建在水坝等发电设施旁边。普通企业级市场无论自身算力规模有多大，都根本不可能完全仿照这种模式。在传统企业数据中心里，把一个机架做成 25 kW 的功耗是完全不切实际的，因为既没有足够的供电，也解决不了散热负荷。现实中企业机架的功耗预算普遍在 10 到 15 kW 之间。在这样的预算限制下，必须最大化计算密度，并采用母线（bus bar）供电等高效设计。令人困惑的是，像通用电气（GE）、巴克莱银行（Barclays）或通用汽车（GM）这样拥有庞大计算需求的企业，却无法在市场上买到一台采用母线配电的服务器，这在追求高效运行的今天显得荒唐至极。硬件之外，更深层次的变革在于软件，而这正是 Bryan Cantrill 植根于血脉中的 Sun Microsystems 工程师信念。他坚信一台真正的计算机必须经过硬件与软件的协同设计（co-design），只有同时掌控设计的两端，才能打造出性能更卓越、体验更连贯的产物。在 2020 年的今天，一台计算机的边界已经发生了根本转变：如果说上世纪七八十年代计算机的用户边界是多用户 Shell，那么现代计算机的边界则是“用于置备虚拟机的 API 端点”。在这套 API 与实际置备之间，存在着庞大的软件栈——虚拟机监控器（hypervisor）、机架顶交换机（top-of-rack switch）控制系统、计算控制平面、存储控制平面，以及面向运维人员与开发者的 API 接口。现状却是传统厂商把硬件扔给用户，让最终用户自己去东拼西凑这些核心软件，Cantrill 直言这既不合理，也不公平。与此同时，虽然现代计算机需要深度焊合软件与硬件，但这绝不意味着重回专有软件的封闭老路。Cantrill 强调专有基础设施软件的时代已经彻底终结，绝不能照搬苹果（Apple）在 iPhone 上的封闭模式——iPhone 固然拥有出色的用户体验，但其黑盒化和对系统修改的阻断在基础设施领域是不可接受的。现代基础设施软件必须彻底开源（fully open）并且完全支持可信度量与证明（completely attested）。 > “I just believe strongly that a computer should have hardware and software that have been co-designed, that when you design things together you yield a better artifact because you are actually able to control elements of the design and you're able to yield a higher performing artifact.” —— Bryan Cantrill - **主流企业机架功耗预算**：通常限制在 10–15 kW，无法支持超大规模机房常见的 25 kW 机架。 - **母线设计（Bus bar）缺失**：大型成熟企业在现存商用市场上买不到原生母线供电的机架级服务器。 - **现代计算机的功能边界**：从历史上的多用户终端 Shell，跃升至“创建与置备虚拟机的 API 端点”。 - **端到端机架软件要素**：包含顶架交换机、Hypervisor、计算与存储控制平面以及运维/开发者 API。 --- ### [34:56](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=2096s) 摆脱历史包袱：消除虚假通用性与紧密裁剪面对重构整机系统的宏大构想，业内许多人早已对现状妥协，认为这一切积重难返、不可能改变；Cantrill 坦言自己过去也曾深陷这种无奈之中，每一次部署新算力都暗自祈祷硬件环境能有所改善，结果却一次次面对失望。但从工程本质来看，打造这样一台机架级计算机并不是前沿未知的基础科学问题，而是一道明确的工程实践题，所有技术难点在原理上都是已知的，真正阻碍它的是整个行业固化的组织架构。要让这套系统在工程上行得通，关键在于进行“紧密裁剪”（tightly tailored），果断抛弃“虚假通用性”（false generalities）。传统商用 PC 架构必须背负沉重的兼容历史包袱，从 DOS 6.2 一路兼容到所有现代操作系统，还要适配市面上能够买到的每一种硬件外设与每一个 Hypervisor。而超大规模云服务商的做法截然相反：他们绝不会在裸金属上随意运行任何未知的操作系统，而是只在其上运行经过定制的自有 Hypervisor。当硬件与软件共同定制时，软件不需要支持天底下每一款设备，只需要完美支持团队精心挑选的那几款组件即可，这大幅压缩了工程复杂度和攻击面。 ```mermaid flowchart TD accTitle: PC架构与定制机架架构对比 accDescr: 对比传统PC架构无节制的软硬件通用性与定制机架系统的紧密裁剪设计。 subgraph Legacy["传统通用 PC 架构（负担沉重）"] L_SW["运行任意 OS / Hypervisor （从 DOS 6.2 到现代系统）"] L_FW["黑盒专有固件与海量选项"] L_HW["适配市面所有硬件外设与板卡"] L_SW --- L_FW --- L_HW end subgraph Tailored["现代紧密裁剪架构（协同设计）"] T_API["统一的 VM 置备 API 端点"] T_CTL["定制控制平面与专用 Hypervisor"] T_HW["经过精挑细选的定制硬件组合"] T_API --> T_CTL --> T_HW end ``` > “Running a computer developing a computer that can run an arbitrary operating system from DOS 6.2 to whatever to a modern operating system is a very constrained problem... By designing one with the other you can eliminate false generalities in both software and in the hardware.” —— Bryan Cantrill --- ### [36:54](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=2214s) 硬件信任根的觉醒：Spectre、Meltdown 与固件安全转折点实现新型架构的历史契机，源于整个硬件底层安全范式的巨震。Cantrill 回顾了 2018 年这个硬件层面的“可怕之年”（annus horribilis）：2018 年 1 月 1 日，当全行业工程师正准备庆祝新年时，他和大多数人一样是在 Hacker News 的讨论帖里第一次得知英特尔 CPU 的巨大漏洞——当时有人发帖猜测 Linux 内核中一些行迹可疑的代码补丁，怀疑底层处理器存在致命缺陷。尽管 Cantrill 所在团队是英特尔的大客户，但在官方保密协议（embargo）之外事先未获任何通知。随后的几天里，Spectre（幽灵）和 Meltdown（熔断）席卷了整个计算世界。这些漏洞极具戏剧讽刺性：它们全部利用了微处理器的“推测执行”（speculative execution）机制来非法刺探受保护的核心内存。而正是这种推测执行机制，让 x86 架构当年成功突破了“内存墙”（memory wall），并在性能上彻底击败了几乎所有 RISC 架构。Cantrill 提到他曾与著名架构师 Jon Masters 讨论过 SPARC 芯片是否存在这类漏洞，结论是 SPARC 根本没有这个问题，因为 SPARC 芯片“推测得不够聪明”（wasn't smart enough to have these issues）。 Spectre 和 Meltdown 的爆发犹如一记警钟，让底层硬件问题直接穿透了整个技术栈。安全研究员们突然意识到，常年无人问津、代码质量堪忧的固件层其实是一个遍布宝藏的“高价值攻击目标”（target-rich environment）。大量顶尖高手蜂拥而至，开始对极其拙劣但深藏底层的固件代码进行反编译和审计，各类高危漏洞随之喷涌而出。这一震荡迫使全行业开始高度重视硬件信任根（Hardware Root of Trust）与固件度量认证（firmware attestation）。在此背景下，行业中涌现出两大标杆： 1. **微软 Cerberus 架构**：开源的硬件信任根保护规范。 2. **Google Titan 与 Chromebook**：Cantrill 评价 Chromebook 是常被世人低估的优秀计算机，它在 PC 领域率先落实了硬件信任根与固件可信链条（Titan 芯片）。虽然 Chromebook 允许用户刷入自定义固件，但它能精准感知到底层处于非受信固件环境，从而拒绝生成对应的平台可信度量。Google 随后推出了行业级的开源项目 OpenTitan，进一步将这种安全模型推向公共生态。 > “The presence of these vulnerabilities deep, deep, deep, deep, deep in the stack highlighted all of these problems up the stack, and suddenly all these firmware issues that had been people knew were out there, suddenly you had set this beacon to security researchers that this is a target-rich environment.” —— Bryan Cantrill - **发现 Meltdown / Spectre 的时间与途径**：2018 年元旦，通过 Hacker News 社区对异常 Linux 内核补丁的推测，而非供应商的事先通报。 - **架构机制的讽刺性**：帮助 x86 跨越内存墙、击败 SPARC 等竞品的推测执行机制，最终成为了导致敏感数据跨边界泄露的安全命门。 - **安全研究员的“黑帮式成名”（gangster famous）**：漏洞曝光后，全球白帽与黑客蜂拥至底层固件栈，挖掘此前缺乏审查的劣质嵌入式代码。 - **Chromebook 的安全模型借鉴**：支持用户自主更换固件，但通过 Titan 信任根实现透明的信任状态自知与隔离认证。 ### [40:42](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=2442s) 开源 EDA 与硬件描述语言的复兴 Bryan Cantrill 指出，当前围绕开放生态的协作努力就像是“Kubernetes 对阵 Borg”一样，通过汇聚开源力量共同解决基础设施难题。而在硬件领域，一个长期被严重忽视的重大趋势正是开源 EDA（电子设计自动化，Electronic Design Automation）运动。Cantrill 指出了一个荒谬的行业现实：二十年前成立一家硬件公司远比现在容易拿到投资，然而在技术实现层面，做硬件却从未像今天这样轻松。长久以来，EDA 领域一直是专有软件垄断的坚固堡垒，但如今开源生态中涌现出了诸如 Yosys、Chisel、SpinalHDL 等一系列现代 HDL（硬件描述语言）和形式化验证工具。更令人振奋的新进展是 Bluespec。Cantrill 特别提到，Bluespec 最初于 15 年前由麻省理工学院（MIT）开发，长期以来一直作为专有商业软件存在，但就在不久前（大约两周前）刚刚宣布开源。Cantrill 认为，简单地将 Bluespec 称为一种 HDL 简直是贬低了它，因为它在抽象层级上实现了真正的跃升，极大地提升了工程师定义 FPGA 的表达能力。Oxide 自身也在深度参与并使用 Bluespec。这种开源工具链的成熟，使得团队自主设计 FPGA 变得越来越简单可行。 - “It was easier to get a hardware company funded two decades ago, but hardware has never been easier than it is today.”（Bryan Cantrill 指出硬件融资门槛变高与实际开发难度大幅降低之间的鲜明对比。） - 传统 EDA 工具链极其昂贵且闭源，而 Yosys、Chisel、SpinalHDL 等开源工具正在打破专有壁垒。 - Bluespec 的开源将硬件设计抽象提升到了全新高度，为构建定制化 FPGA 逻辑铺平了道路。 --- ### [42:24](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=2544s) RISC-V 带来的芯片层级可编程性硬件领域的另一大核心突破是 RISC-V。Cantrill 极力推荐大家去阅读由 David Patterson 等人编写的 RISC-V 规范（The RISC-V Reader / 规范手册），称其为一段“令人愉悦的阅读体验”。RISC-V 团队在指令集架构（ISA）的设计上极为深思熟虑，系统地审视并汲取了历史上所有 ISA 的成败得失，摒弃了那些花哨但有害的想法，只提炼出最核心、最成熟的软硬件接口设计。最关键的是，RISC-V 完全免费且开放，不存在任何知识产权授权陷阱。这使得工程师可以在 FPGA 中极其轻松地嵌入一个免授权费的软核（soft core）。虽然这个软核并非用来运行占用数兆字节内存的大型软件，而是在受限的资源环境中执行轻量逻辑，但它赋予了开发者在系统栈最底层（即 FPGA 层面）具备图灵完备的计算与编程能力。 - “They are so deliberate about learning from everything in the history of instruction set design.”（Bryan Cantrill 盛赞 RISC-V 团队对计算机体系结构历史教训的吸纳与反思。） - RISC-V 的免授权特性让开发团队能够零成本将软核集成至 FPGA 中。 - 底层可编程性彻底改变了硬件与控制固件之间的交互边界。 --- ### [44:05](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=2645s) 开放固件与 OCP（Open Compute Project）的推动力 Cantrill 回顾道，曾几何时他一度感到绝望，以为工业界永远无法摆脱闭源的陈旧传统 BIOS。然而，Spectre 和 Meltdown 等底层硬件安全漏洞的爆发彻底改变了这一切。超大规模云计算厂商（Hyperscalers）意识到安全隐患深植于系统栈底，开始强烈要求并资助微处理器厂商开放固件。Cantrill 兴奋地分享了他此前参加开源固件大会（OSFC）的经历——大会在 Google 和 Facebook 附近举办，这种氛围让他仿佛回到了互联网普及之前的黄金时代：当你走进会场，原本以为世界上只有自己这极小众的一拨人在关心冷门技术，结果却迎面遇上了 300 位志同道合的知己。这一切背后最强大的组织推动力正是 Facebook 发起的 Open Compute Project（OCP）。Cantrill 解释道，此前展示的 Tioga Pass 架构正是 OCP 的标准化成果。Facebook 最初启动 OCP 时，抱有一种非常真诚、甚至在外界看来略显罕见的“社会责任感”，希望将母线（Bus Bar）供电优化等数据中心效率创新开放给非竞争对手。虽然有人会调侃如今的 Facebook 与“社会责任”这个词反差巨大，但在 2010 至 2011 年由硬件工程师主导的启动初期，那份开放精神是极其真挚的。OCP 多年来一直在默默积聚能量，已成为洞察超大规模数据中心最前沿技术与透明协作的核心阵地。 - “With Spectre and Meltdown and with the new focus on these vulnerabilities deep in the stack, and thanks to the hyperscalers who really demand this and are funding it accordingly... we are now seeing the arrival of open firmware.”（Bryan Cantrill 阐述开放固件迎来黎明背后的核心驱动力。） - 硬件层面的模块革新固然令人振奋，但它们距离组装成一个高度整合的完整服务器系统仍有很长的路要走。 - 尽管存在疫情担忧（注：2020 年初），OCP 全球峰会依然是了解超大规模硬件协作最重要的前沿窗口。 ```mermaid flowchart TD accTitle: 现代服务器系统栈的开源技术演进 accDescr: 展示从底层硬件描述、固件到上层系统基础设施软件的全面开源化路径与驱动力 subgraph Hardware ["底层硬件与芯片设计"] EDA["开源 EDA 与 HDL (Yosys, Chisel, Bluespec)"] RISCV["RISC-V 开放架构 (FPGA 软核 / 底层可编程)"] end subgraph Firmware ["固件与机架标准"] OpenFW["开源固件 (Open Firmware) (受 Spectre/Meltdown 漏洞推动)"] OCP["Open Compute Project (OCP) (机架/供电规范与超大规模厂商协作)"] end subgraph Software ["系统级基础设施软件"] OSS_DB["生产级开源数据库与存储"] OSS_Consensus["现代共识系统 (替代 ZooKeeper 复杂架构)"] Lang["现代安全系统编程语言 (Rust 的崛起)"] end Hardware --> Firmware Firmware --> Software ``` --- ### [47:24](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=2844s) 系统软件格局变迁与语言意识形态的转变转向系统级软件层面，Cantrill 观察到开源已经成为基础设施软件的默认行业标准。尽管这也引发了商业变现的困境——大量开源创业公司正在经历如何把开源做成产品的“中年危机”，但它确切地为行业留下了丰富且经受住数据中心规模检验的生产级基础设施。Cantrill 对比了十年前后的软件生态：过去分布式共识几乎只能依赖复杂且令人头疼的 Apache ZooKeeper，“即使你深爱 ZooKeeper，你也不得不承认它一点都不好玩”；而如今业界不仅拥有丰富易用的开源共识算法实现，连操作系统和数据库也几乎完全被开源所统治，专有闭源数据库正走向终局。最让 Cantrill 意想不到的变革发生在编程语言领域。他调侃道，如果十年前坐时光机告诉过去的自己“未来开源数据库和共识系统会统治世界”，过去的自己会觉得理所当然；但如果告诉他“编程语言领域将迎来极其颠覆性的重塑”，过去的自己一定会忍不住动手揍未来的自己一顿。作为一个务实坚定的底层系统工程师，Cantrill 过去一向将编程语言纯粹视为完成任务的螺丝刀，并对语法层面的花活抱有高度怀疑。他笑称年轻人都经历过沉迷于“运算符重载”等花哨特性的幼稚期，但只要去多调试调试别人写的生产代码，现实自会彻底击碎这种幻想；而在经历过早年 DOS 剥夺童年的痛苦之后，C++ 也曾给他带来过类似的梦魇。 - “If I were to go back to you know go back a decade to to myself in a time machine and I were to tell my decade ago self... there will also be a very important development in programming languages, I think my past self would have decked my future self.”（Bryan Cantrill 用生动的自嘲表达了一个老派系统程序员最初对编程语言革新的抵触与偏见。） - 基础设施软件的开源化已经彻底消灭了专有操作系统的生存空间，专有数据库也正在被加速淘汰。 - 治愈“语言特性滥用综合征”的最佳良药就是被派去在生产环境中调试别人的代码。 ### [50:32](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3032s) 为什么系统编程必须拒绝垃圾回收 Bryan Cantrill 回忆起自己年轻时的编程经历。他曾经历过与 C++ 的热恋期，但随后发誓再也不碰 C++；对于 Go 语言，他觉得很有意思，Java 也算不错，但他坚决不让垃圾回收（Garbage Collection, GC）出现在自己编写的底层系统软件中。这并不是对 GC 的贬低——Cantrill 承认 GC 非常重要，甚至可能适用于世界上绝大多数软件，但在底层系统领域，系统对确定性（determinism）的要求极高。他无法接受在软件运行过程中，不知从何处吹响一声自己听不见也看不见的“哨音”，导致整个软件瞬间停摆，随后某个机制背着他在后台四处乱窜，做着他完全无法洞悉且无法控制的事情。垃圾回收机制最致命的缺陷，在于它往往不是在风平浪静的晴天出问题，而是在惊涛骇浪的暴风雨时刻爆发。当系统承受着极高负荷、业务量暴增时，堆内存会急剧扩张。更糟的是，如果你的程序经过了精心修剪与调优，每一块占用的内存都是真正处于使用状态的有效数据，根本没有真正的“垃圾”可收，此时所谓的垃圾收集器就会彻底沦为一个“定期的堆内存全盘扫描器”（periodic heap scanner）。系统高负载运行时，它便自作主张地频繁扫描整个庞大的堆，完全脱离开发者的掌控。基于此，Cantrill 早已下定决心远离 GC，并一度认命地认为自己余生都只能与 C 语言为伴。作为一名底层工匠，Cantrill 坦言自己其实非常喜爱 C 语言。许多人用“可移植的汇编语言”（portable assembly）来贬低 C，但他觉得这正是系统程序员梦寐以求的特质——写 C 语言时，你能真切地感受到代码底下跳动的汇编指令，从而完全掌控底层机器。 - 系统级编程对执行确定性有严苛要求，无法容忍 GC 引发的不可控暂停（Stop-the-World）。 - 在业务高峰、堆内存巨大的场景下，GC 的扫描开销会对高负载系统造成毁灭性打击。 - Cantrill 曾一度认定 C 语言是自己唯一能接受的系统级开发工具。 > “The problem with garbage collection is it becomes most problematic not on sunny days but on stormy ones so the when garbage collection is a problem it's because your actual legitimate heap has grown large because you are under a height you have a high degree of work you have a surging amount of work.” —— Bryan Cantrill --- ### [52:47](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3167s) C 语言的致命短板与 DTrace 中的整数溢出教训然而，一门正在彻底颠覆 Cantrill 乃至众多底层开发者认知的新编程语言横空出世，那就是 Rust。Cantrill 坚信 Rust 是一场与当年 C 语言诞生完全同等量级的重大革命。回顾历史，在 C 语言诞生之前，那些系统显得如同史前时代般古老，而 C 的出现让一切骤然现代化；但如今，C 语言的问题暴露无遗。人们常常大谈 C 语言的不安全性（内存不安全、如同随时摆弄上了膛的武器），但 Cantrill 强调，现实中真正致命的痛点是“软件的组合性”（composability）。在软件工程中，“问题往往不在你，而在别人”，C 语言的根本问题通常不是你自己写的那几行代码，而是你必须链接的第三方库。即使在一个所有人都深谙如何编写“正确 C 代码”的理想世界里，C 的组合能力依然奇差无比；更何况，经历了大量 Rust 实践之后，Cantrill 不得不痛苦地承认：所谓的“正确 C 代码”本身就带着无数星号与附注，因为 C 标准中充斥着海量的未定义行为（Undefined Behavior）。 Cantrill 以自己亲身经历的惨痛教训为例：他在 2000 年代初主导开发了动态追踪工具 DTrace，该软件后来几乎运行在无数的机器上，极少出问题。但在诞生至今的十五年间，DTrace 一共被曝出过两个安全漏洞（均由 Cantrill 本人写出的代码引起；后来发现的第三个漏洞似乎是合著者 Adam Leventhal 引入的）。这两个漏洞的本质全都是恶意整数溢出（malicious integer overflow）。当时他在做边界检查（bounds check），在普通业务逻辑下该检查完全正确，程序运行良好；但面对恶意构造的边界输入时，检查便被绕过，攻击者借此可以任意读取内存，再与其他漏洞串联，最终实现权限提升。这一经历给 Cantrill 敲响了警钟，而 Rust 不仅是内存安全（memory safe）的，更是整数安全（integer safe）的。 - C 语言代码极难安全组合，调用第三方库时极易暴露出隐蔽的内存与状态冲突。 - C 语言标准包含大量未定义行为，导致“完全正确的 C 代码”在实际工程中几乎不可能实现。 - DTrace 历史上 Cantrill 引入的两次严重漏洞，均源于 C 语言中极易写错的边界比较与恶意整数溢出。 - Rust 通过编译器机制同时保证了内存安全与整数安全性。 > “The idea of correct C itself has so many asterisks on it because there's so much undefined behavior in C.” —— Bryan Cantrill --- ### [55:28](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3328s) Rust 的所有权模型与惨遭挫败的性能对比 Rust 最强大的地方在于所有权系统（ownership system）带来的无缝组合能力。Rust 编译器在编译期静态跟踪内存的归属与生命周期，明确谁拥有什么、谁在何时可以访问什么。这种静态判定让编译器在编译期就能确保安全性，从而生成既不需要垃圾回收器、运行速度又极其飞快的原生机器码二进制。 Cantrill 坦言，初学 Rust 的过程充满挫折。如果只是蜻蜓点水地尝试，Rust 会狠狠地惩罚你，因为它的所有权模型与传统语言截然不同。对于 C 语言老手来说，Rust 会有一种诡异的直觉感——写 C 时能感知到底层的汇编，写 Rust 时则能清晰感知到底层的 C，能明白编译器为何报错；而如果不理解内存底层，面对 Rust 编译器的反复报错只会让人泪流满面。Cantrill 回忆自己最初用 Rust 重写一个小型系统组件时，经历了典型的“悲伤五阶段”，最后进入向编译器妥协的讨价还价阶段。他原本试图写出极其精巧的代码，但很快发现自己根本不够聪明，最后只能选择用最笨拙、最直白的方式把代码敲出来，仅仅为了能通过编译。编译通过后，Cantrill 心想：只要这套用最蠢办法写出的 Rust 代码性能能在原本精心调优的 C 代码的 20% 差距以内，自己就心满意足了。然而基准测试的结果让他大受震撼——Rust 版本居然比他亲手调优的 C 版本快了整整 35%。这个结果反而让 Cantrill 感到非常沮丧和生气。这就像一个当父亲的在篮球场上突然被自己年幼的孩子暴扣一样，“你内心固然骄傲，但同时也充满了愤怒”。Cantrill 对自己的 C 语言功底极具自信，他无法接受自己精心打磨的 C 语言杰作被一段随便凑合写成的 Rust 代码在性能上轻松击败。 ```mermaid flowchart TD accTitle: Rust 所有权与性能优势机制 accDescr: 展示 Rust 编译器如何通过编译期静态所有权分析消除 GC 并驱动更优数据结构，最终实现高性能。 subgraph CompileTime["编译阶段 (Compile-Time)"] direction TB A["Rust 所有权与生命周期规则"] --> B["静态推导与安全判定"] B --> C["明确内存使用契约"] end subgraph RuntimeResult["运行阶段 (Runtime)"] direction TB C --> D["零运行时 GC 开销"] C --> E["强制/便于使用更优数据结构 (如高效 B-Tree)"] D --> F["超越手写 C 语言的高性能表现"] E --> F end ``` - 初学者无法通过随意试探来掌握 Rust，必须系统学习其所有权哲学。 - Cantrill 用最简单直接的方式编译出的 Rust 程序，性能比原本高度优化的 C 语言实现高出 35%。 - 该性能反差并非来自于深奥的代码级技巧，而是源于底层内存布局与语言内在特性的差异。 > “I ran it and it was like 35% faster than my C and it's like, no! And I got to tell you, like actually it was like not good news, I felt like I took pride in that C... it's kind of like when you get beat by your kids and some, you know your kids beat you in basketball or whatever, it's like, look kid you're not supposed to let dad dunk on you, it's—I mean you're proud, but also like angry.” —— Bryan Cantrill --- ### [58:04](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3484s) 为什么 B 树在 C 中是梦魇，而在 Rust 中顺理成章为了查清 Rust 为何能彻底碾压 C 语言，Cantrill 深入剖析了二者的内存行为，发现 Rust 版本的内存操作模式要优越得多。起初，他去请教 Rust 团队的核心成员 Steve Klabnik，赞叹道：“你们做的编译器优化太不可思议了！编译器显然对天地万物都了如指掌，以至于真正攻克了指针别名与内存消歧（memory disambiguation）问题。”结果 Steve Klabnik 坦率地回答：“理论上那些优化确实可能做到，但我们其实一项都还没实现呢。” Cantrill 这才恍然大悟：真正厉害的不是编译器的黑魔法，而是 Rust 语言本身促使开发者做出的架构决策。在 Rust 标准库中，没有常规的平衡二叉树（balanced binary tree），默认提供且广泛使用的是 B 树（B-Tree）。从数据结构角度来看，B 树在现代硬件上的缓存局部性和性能显著优于平衡二叉树。然而，在 C 语言中实现一颗 B 树是极其困难且棘手的；若是想在 C 中将 B 树打包成一个通用的第三方库，更是难如登天。原因在于，平衡二叉树在再平衡时只需要调整几个指针的指向，而 B 树在节点重构和平衡时，必须频繁地在内存中移动和搬迁（relocate / transplant）实际数据。一旦需要搬迁数据，通用库就必须与调用者之间建立起极度严格、绝对不容有失的内存契约——明确规定谁在什么时刻负责持有、移动和访问这块数据。在没有编译器强约束的 C 语言中，由于其天生的内存不安全性，根本无法在库边界上清晰、安全地表达这种生命周期契约；而在 Rust 中，所有权系统天然且精确地跟踪了“谁在何时拥有什么”，使得实现高难度的高效数据结构变得轻而易举。由于 Rust 强迫并鼓励系统使用更高效、对缓存更友好的数据结构，最终产出的软件自然具备极高的运行效能。 | 特性 / 维度 | C 语言通用库实现 | Rust 语言实现 | |:--- |:--- |:--- | | **典型数据结构** | 平衡二叉树（只需改变指针指向） | B 树（B-Tree，涉及频繁的数据搬迁与重平衡） | | **内存重平衡开销** | 内存局部性差，容易产生指针悬挂与未定义行为 | 缓存局部性极高，更适配现代 CPU 架构 | | **所有权契约表达** | 难以在 API 层面强制约束调用者何时能安全读写 | 编译器在编译期静态强制保证生命周期与所有权合规 | | **实现难度** | 极端复杂且容易引入安全漏洞，极少有可靠开源库 | 表达自然，借由编译器类型系统得到完全保障 | - Steve Klabnik 证实，Rust 当时的高性能表现并非依赖激进的内存消歧后端优化。 - B 树相比二叉树具有更好的硬件局部性，但因涉及频繁的数据搬迁，在 C 语言中极难安全实现为通用库。 - Rust 凭借严格的所有权与生命周期契约，扫清了实现复杂高效数据结构的工程障碍。 > “The unsafety of C makes hard to express that contract with the library user hard to the point of being damn near impossible. With Rust, this is actually really easy because Rust knows who owns what when, and you are able—you are forced to use very efficient data structures.” —— Bryan Cantrill ### [01:00:07](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3607s) Rust 的极简内存足迹与软硬件底层的现代抽象谈及 Rust 语言在系统开发中的核心优势，Bryan Cantrill 强调其最令人惊叹的特性之一便是“缩减到极小体积（get small）”的能力。在基于 FPGA 的 RISC-V 核心上开发时，工程师往往只能支配极少的内存空间，例如区区 512 KB。Bryan 分享了他与公司一位深耕嵌入式 Rust 的工程师交流时的趣事：人们常开玩笑引用据称是比尔·盖茨所说的那句名言——“640 KB 内存对任何人来说都足够了”。大众通常将其视作技术史上的著名笑柄，认为 640 KB 在现代根本无法运行任何东西。但该工程师却认真纠正道，事实并非如此；只要严谨克制地规划和使用，640 KB 实际上是一笔庞大的内存空间，完全能够完成极为复杂的任务。 Rust 让开发者能够将软件体积压缩到极致，其极限表现甚至在某些边界上超越了传统的 C 语言。使用 C 语言固然可以编写非常小巧的程序，但 Rust 既能实现同等甚至更低的资源占用，同时还能保留极其丰富的高级抽象能力。对于构建实际系统级底层的工程师而言，这是程序设计语言演进中的重大突破。Oxide Computer Company 的命名本身正是向 Rust 致敬，因为团队看到了这门语言在系统工程中的无限潜力。此外，学术界和开源社区中涌现的 Rust 系统也极具启发性，例如由斯坦福大学 Phil Levis 团队主导开发的嵌入式操作系统 Tock OS。正是这些底层软件抽象与开源硬件生态的交融，让重新设计整套服务器架构的设想从理论变为了现实。 - **极小内存环境**：在 FPGA 上运行的 RISC-V 软核通常仅配备约 512 KB 的 RAM。 - **抽象与体积的统一**：Rust 既拥有与 C 语言比肩的极致底层控制力和极小二进制体积，又具备现代语言的安全性和高阶抽象。 - **Tock OS 案例**：由斯坦福大学 Phil Levis 等人主导开发的基于 Rust 的嵌入式安全操作系统，是 Oxide 密切关注的核心技术参考之一。 ### [01:01:45](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3705s) 创立 Oxide：硬科技种子轮融资的残酷与生机在软硬件技术要素齐备之后，Bryan Cantrill 与联合创始人正式创立了 Oxide Computer Company，并在 2019 年底完成了种子轮融资。Bryan 直言，如果台下的听众感兴趣，他非常乐意分享在当下的投资环境中为“硬科技（hard tech）”项目筹集种子资金的惨痛经历。他形容这段融资旅途甚至算不上“忽冷忽热”，而是交织着极度沮丧与高度亢奋的双重极端：当你遇到真正理解底层系统价值的投资人时，那种共鸣会带来无与伦比的精神振奋；但现实是，绝大多数投资人完全不在同一个频道上，与这些不对路的人周旋则令人感到万分沮丧。尽管过程曲折，团队依然成功筹得资金以放手一搏。Oxide 在 2019 年底完成交割，截至演讲时团队已扩充至 15 人，计划于 2021 年推出功能完整的全尺寸原型机，并于 2022 年向市场交付商用系统。Bryan 坦言，这扇大门才刚刚推开，团队正式起步仅有八周左右的时间，但他已经看到了极其清晰且充满希望的技术方向。 ### [01:03:05](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3785s) “死去的计算机公司”：从尽调盲区到 eGenera 的反面教材重新发明计算机是一项雄心勃勃且凶险异常的任务，历史上无数公司尝试过却最终折戟沉沙。在融资路演期间，为了汲取历史教训，Bryan 准备了一份详尽得堪称走极端的附录：里面包含了过去 15 年间几乎所有能叫得出名字的服务器与硬件初创公司，逐一复盘它们的商业模式与倒闭原因。每当投资人抛出一个质疑：“你们和当年那家做硬件的公司有什么区别？”Bryan 就能立刻翻到对应的附录幻灯片精准作答。然而，这一策略很快产生了适得其反的心理效应。Bryan 笑着反思道，在投资人面前像报丧一样罗列一家接一家死去的公司，容易给对方留下极具冲击力的负面心理暗示：“在投资人看来，这就像是你脑子里装的全都是死掉的公司，他们会疑惑：‘难道我们要投的就是下一家死掉的公司吗？’”他特别回忆了一个标志性尴尬时刻：当某位投资人主动提到了曾经的刀片服务器公司 eGenera 时，Bryan 兴奋地让助手立刻跳到第 102 页展示精心制作的剖析材料。但当投资人看着幻灯片惊呼“天哪，他们融了那么多钱！天哪，最后全归零了！”时，Bryan 猛然意识到，自己原本试图展示深厚调研，结果却硬生生把谈判现场变成了敲响警钟的“劝退会”。不过，团队最终克服了这些阻碍，成功锁定融资并投入工程研发。 ### [01:04:49](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3889s) 传承软硬件断层经验：《On the Metal》播客与行业历史在演讲的结尾部分，Bryan 谈及了成立 Oxide 的另一个深层初衷。他和联合创始人 Steve Tuck、Jessie Frazelle 等人整个职业生涯都在软硬件交界面打拼。他们深切感受到，整个计算行业在向高层软件转移的过程中，大量的底层硬件与系统经验（lore）正在不可逆转地流失。为了填补这种对话的空白，Oxide 开设了名为《On the Metal》的播客节目，邀请那些常年在裸金属与底层系统交界面奋斗的工程师进行深度长谈。 Bryan 提到，就在演讲当天他们刚刚录制完一期非常精彩的节目，嘉宾是著名硬件黑客与工程师 Star Simpson。播客覆盖了跨越各个年代、不同背景工程师的跌宕历程。在回答现场观众提问时，一位听众提到了 1980 年代著名的 Convex Computer 及其标志性的“死亡公司墓地”——Convex 曾将竞争对手的名字刻进混凝土墓碑里。Bryan 感慨这是他本周第二次听到关于 Convex 的讨论，并分享了自己对待计算机历史的独特视角：“It's more interesting I think to keep track of your ancestors than your competition because I think they've got more to teach you than your... than I mean it's easy to kind of disparage your competition, we never listen to that, but the it's the folks that seemed like they were the right place at the right time and and with the right technology and didn't succeed that are in some ways more intriguing.”（Bryan Cantrill 强调：相比于盯着竞争对手，研究你的技术先驱要有趣得多；嘲弄对手很容易，但那些在对的时间、对的地点手握对的技术却最终失败的公司，能教会我们更多东西。） ### [01:08:28](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4108s) 本地算力用户的绝地反击与公有云账单之痛在现场问答环节，有听众提问 Oxide 的目标客户究竟是谁。Bryan 明确回答，Oxide 的核心目标受众正是那些在本地机房（on-premises）运行计算工作负载的企业。他将这群客户细分为不同类型，但指出他们普遍承受着巨大的舆论偏见：在公有云狂热的叙事下，全行业都在向他们灌输“本地运行算力毫无正当理由，所有东西都必须上云”的论调。这让本地基础设施团队感到极度被孤立与愤怒，Bryan 生动地为他们鸣不平：“如果找这些客户聊，他们往往情绪激动，因为外界一直在把他们当成傻子，认为他们之所以还留在线下只是因为不懂云计算。但他们会吼道：‘不，我懂云计算，我不是傻子！我用过公有云，我知道它怎么运作，但我有严格的合规性审查，或者我面对的是一张足以把整家公司彻底拖垮的 AWS 账单！’” Bryan 犀利地揭示了公有云定价模型的隐秘陷阱：云服务的定价机制被设计得极其精妙，前期几乎等同于免费——创业公司拿着花不完的云信用额度（cloud credits），根本不在乎每月的账单细节；然而随着业务规模扩大，云账单往往会在一夜之间从“毫不在意”骤升为“致命重压”。当企业猛然意识到公有云的高昂算力成本（COGS）正在蚕食全部利润时，往往已经深陷架构锁死之中，回天乏术。更残酷的现实在于，很多被高昂云账单逼入绝境的企业尝试迁回本地，却发现传统服务器市场支离破碎、软件堆栈陈旧不堪；由于缺乏超大规模厂商的内部工程能力，他们自行在机房组装机架的尝试最终都沦为惨烈灾难。这群既没有屋顶遮风避雨、又被公有云暴雨不断冲刷的愤懑客户，正是 Oxide 坚定看好的庞大市场基石。 ```mermaid flowchart TD accTitle: 公有云成本陷阱与企业回迁本地困境 accDescr: 展示企业从享受公有云免费额度到遭遇账单危机，进而尝试回迁本地机房却因传统硬件堆栈混乱而失败的完整生命周期。 subgraph CloudTrap["公有云成本陷阱"] A["创业初期：云信用额度抵扣 感知成本几乎为零"] --> B["业务扩张期：算力规模激增 账单夜之间变成沉重负担"] B --> C["成熟期：公有云售货成本 (COGS) 严重吞噬企业利润与生存空间"] end subgraph RepatriationCrisis["本地回迁困境"] C --> D["决策回迁本地机房 (On-Premises)"] D --> E{"自建机架系统"} E -->|"缺乏超大规模整合能力"| F["传统服务器堆栈混乱 部署沦为运维灾难"] E -->|"行业舆论压迫与偏见"| G["被外界贴上'不懂云'的标签 客户群体极为愤懑与沮丧"] end subgraph Solution["Oxide 目标市场切入"] F --> H["提供整机柜交付的一体化云级本地算力"] G --> H end ``` ### [01:10:38](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4238s) 融资中的客户验证与本地部署市场的真实存在在向风险投资人（VC）进行种子轮融资的过程中，投资机构往往要求对潜在客户进行访谈和尽调。Bryan Cantrill 分享了当时令人啼笑皆非的经历：由于 Oxide 团队深知客户工作繁忙，原本非常小心谨慎地保护这些支持者，生怕浪费他们的宝贵时间；然而客户们却对市场上缺乏合格硬件的现状积怨已久，主动要求与投资人对话。他们对 Bryan 说：“把那些 VC 带到我面前来！我要向他们证明我的存在！” 投资人在与这些客户通完电话后，甚至感到震惊和不知所措（shell-shocked）。他们回来告诉 Bryan：“你是对的，你确实有一批情绪极为激动的客户。”Bryan 回复道：“是的，他们非常愤怒；我希望你们没有跟他们说‘本地部署市场不存在’之类的话，因为如果你敢这么对他们说，他们会彻底抓狂。”在 Bryan 看来，当下的本地基础设施客户群是那些“在本地部署环境中被巨头们放弃等死的人”；而在未来数十年内，推动该市场的将是那些“生于云端、但由于经济考量而迁移回本地”的成熟科技企业。 - “So we know that the market is out there we know it very much exists and we think it is today it's those folks that have been left for dead on Prem but tomorrow and in the coming decades we believe it'll be those folks that are born on the cloud and move on Prem primarily for economic reasons.” —— Bryan Cantrill --- ### [01:12:04](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4324s) 处理器芯片选型：为何不是 RISC-V 或 ARM？现场观众就芯片架构的选择提问，询问 Oxide 对 Intel、Microsoft、ARM 以及 RISC-V 的看法。Bryan 幽默地回应称，他与 Intel 之间并无怨仇，唯一的遗憾是 Intel 在很久以前放弃了转型成为纯正计算机公司的机会，为了不与自家下游渠道竞争而只甘当微处理器厂商；至于 Microsoft，Bryan 调侃道“自从比尔·盖茨夺走了我的童年之后我就再也没用过微软的产品”，因此微软根本不在考量之内。在架构层面，Bryan 肯定了 RISC-V 的价值，但明确指出 RISC-V 目前根本无法运行高负载的现代数据中心任务。RISC-V 的最佳阵地是嵌入式控制系统、软核或专用辅助硬核，用于高可解释性场景，但其无法在尖端制程（如 7nm）与主流芯片竞争密度和绝对性能。针对 ARM（如 AWS 的 Graviton 2），Bryan 认为在当前 AMD EPYC 强大的表现下，从 x86 切换到 ARM 只是横向移动（lateral move），在性能和功耗上并不具备颠覆性优势。作为亲历过历史体系结构竞争的工程师，Bryan 直言自己不想再打一次指令集架构之战（ISA war），在处理虚拟化重负载计算时，x86 依旧是唯一经过充分验证的现实选择。 ```mermaid flowchart TD accTitle: 核心架构与应用场景匹配图 accDescr: 展示了从低功耗嵌入式到高密度虚拟化计算的不同处理器架构匹配关系 subgraph EmbeddedLayer["嵌入式与板级管理层"] RV["RISC-V 架构"] RV -->|"极佳适配"| M1["软核 / 硬核逻辑"] RV -->|"极佳适配"| M2["板级辅助与微控制器"] end subgraph ServerComputeLayer["机架级主计算与虚拟化层"] X86["x86-64 架构 (AMD EPYC / Intel)"] ARM["ARM 架构 (如 AWS Graviton 2)"] X86 -->|"首选方案"| PROD["高密度虚拟化工作负载"] ARM -.->|"当前优势不明显 (横向切换)"| PROD end ``` - “I thought the ISA war before and I really don't want to fight it again I have long since accepted that it's gonna be x86 from a nice perspective at least for these kind of virtualized workloads so for Austin's x86 but everything else we basically put on the table.” —— Bryan Cantrill --- ### [01:14:00](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4440s) 什么时候应该从公有云迁回本地部署？当被问及企业应如何在公有云与本地部署之间权衡决策时，Bryan 强调 Oxide 绝非“反云派（anti-cloud）”，团队完全拥抱弹性计算的价值。任何初创企业或新业务的第一天，都毫无疑问应当在公有云上起步。但随着公司规模的扩张，基础设施的经济模型会发生根本性逆转。Bryan 指出，这一转折点通常发生在企业准备递交 S-1 招股说明书（IPO 阶段）的前夕。在此之前，企业作为依靠风险投资驱动增长的 SaaS 公司，对毛利率（margin）可能并不敏感；但一旦要成为接受公开市场检验的上市公司，公有云账单便开始被计入销售成本（COGS）。当云支出演变成侵蚀公司利润率的主因时，拥有属于自己的物理硬件就成为了必然的技术与财务选择。 - “And when you start to view the cloud as cogs now it's probably time to start thinking about being on your own physical machine at the scale we're talking about.” —— Bryan Cantrill - 初创起步阶段：坚决建议采用公有云以获得弹性与快速迭代能力。 - 本地迁移节点：企业筹备上市、开始严控毛利率并将云服务支出视作 COGS 成本时。 --- ### [01:15:20](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4520s) Rust 在底层硬件编程与裸机嵌入式中的优势现场听众提问：采用 Rust 语言是否真能让人触达并掌控底层硬件？Bryan 给出了坚定的肯定回答，并指出 Rust 在底层硬件交互的连贯性上甚至超越了 C 语言。在传统的 C 语言开发（如操作系统内核）中，开发者频繁混写 C 与汇编语言，此时编译器几乎处于听天由命（best of luck land）的盲区状态，优化和行为难以预测。相比之下，Rust 提供了完备的底层控制机制。例如，Rust 允许开发者显式标记“裸函数”（naked functions），取消 ABI 调用约定的函数序言（prologue/preamble）。这在栈指针尚未建立、运行上下文极端苛刻的底层引导代码中至关重要。Bryan 预测，RISC-V 和 Rust 将率先在嵌入式领域掀起彻底的技术革命。C 语言在嵌入式环境中极易出现无符号数算术溢出、越界等难以被静态分析工具捕获的隐患（lint 工具为避免过多误报往往无法严格报警）；而 Rust 的强类型系统和严格检查杜绝了符号隐式转换，使得在裸机上编写高可靠、高性能软件成为可能。Bryan 推荐参阅 Oxide 工程师 Cliff Biffle 所写的《Learning Rust the Dangerous Way》系列技术博文。 - “I think risk 5 will take over the earth first with the embedded software I think you can make a case for Russ to taking over embedded software almost first which is kind of crazy cuz it's like it's got a lot of energy and momentum and other parts of the stack but it is so far in a way better than c4 that embedded use case and embedded software has historically been really really hard to write and get right.” —— Bryan Cantrill - 裸机控制能力：Rust 提供了明确脱离常规 ABI 约束的机制（如裸函数声明），使得引导代码与紧密依赖寄存器的底层逻辑更加安全可控。 - 类型系统安全性：Rust 杜绝了 C 语言中 signed 与 unsigned 类型的隐式混用，杜绝了静默整数溢出风险。 --- ### [01:19:20](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4760s) 客户应用场景与高速硬件走线物理约束在谈到运行在这些机器上的应用生态时，Bryan 指出，Oxide 并不会预先规定用户的顶层业务软件。随着诸如 Uber 等大型科技公司公开宣布因成本与架构考量而重返本地自建数据中心，这些企业本身拥有庞大的自研基础设施与业务系统，他们迁移回本地后需要的是现代化的机架与计算载体，而非去重新采购传统的陈旧商业软件堆栈（如非运行大型 Oracle 单体数据库的旧模式）。随后话题转向物理硬件的设计挑战，特别是在设计现代化服务器主板时所面临的高速电气互连难题。Bryan 坦言自己并非这一领域的纯粹硬件专家，但指出了微观物理规律对硬件工程的极端严苛要求：在电路板上，信号传输延迟大约为“每皮秒一英寸（picosecond per inch）”。在 DDR3、DDR4 以及即将到来的 DDR5 超高速内存总线频率下，由于不同走线（trace）在物理距离和路径上的细微差异，信号到达时间的微小偏差都会导致时序崩溃。因此，在物理布线中必须对每根走线长度进行极高精度的匹配设计。 - “It is very complicated to make high speed interfaces I mean this is not my domain of expertise so let's just like let's just do the big kind of like a caution here but it is um because you can go it's a picot inch right you can go an inch to a picosecond and the different traces that you take will have to take different lengths of time and at the speeds at which ddr4 ddr3 to your and coming to your five I'll operate all those different trace lengths have totally different...” —— Bryan Cantrill ### [01:20:48](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4848s) 内存训练的复杂性与硬件开销在回答观众关于底层启动与硬件初始化的提问时，Bryan Cantrill 深入探讨了 DIMM 内存训练（DIMM training）这一环节的机理与难点。Cantrill 解释道，由于主板走线（trace）、DIMM 颗粒特性以及环境物理条件等差异，信号时序必须经过非常精确的微调。内存训练本质上就是内存控制器自主进行参数校准、以在 CPU 和内存模组之间建立高速可靠数据通道的过程。因为当前的高速总线接口极快且极其敏感，这一步骤绝非微不足道。 Cantrill 坦言，如果重新插拔（reseat）了一根 DIMM，系统就必须重新执行漫长的训练过程，以此重新计算并确定关键的时序常量，而这部分软硬件逻辑通常高度专有且不透明。虽然他自谦道“这已经超出了我的专业领域（domain of expertise）”，但他明确指出，内存训练耗时极长，完全不是开发者在能够避免时愿意频繁经历的操作。通常情况下，内存控制器会记录特定插槽中的 DIMM 信息；但如果关机状态下发生物理变更，或者把内存移到不同插槽，就会触发重新训练。他感叹道，在传统的系统架构中，软件工程师对这个层面的可视性一直极低，排查和控制起来也相当棘手。 - 内存训练核心任务：内存控制器根据走线延迟、温度与物理环境等变量，计算出最佳通信参数。 - 重新训练的代价：重插或更换内存插槽均会强制触发时序校准，耗时显著，拖慢系统冷启动时间。 - 系统黑盒属性：相关训练算法与固件常量大多被芯片原厂严密封装，外界极难观察内部执行细节。 --- ### [01:23:07](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4987s) 摆脱云厂商锁定：精简的基础设施核心与 Kubernetes 现场观众提出了关于公有云厂商丰富托管服务（如专有数据库与高级配套服务）的竞争问题。Cantrill 回顾指出，曾经有一段时间业界普遍相信用户会照单全收、欣然接受云厂商提供的任何现成服务。然而行业风向已经发生了重大逆转，开发者的警惕性正在大幅提高。在 Cantrill 看来，业界拥抱 Kubernetes 的根本动力之一，恰恰是工程师群体希望从公有云巨头手中夺回技术栈的控制权，从而彻底避免与 Redshift 之类专属云组件深度绑定。针对当下架构趋势的变化，Cantrill 鲜明地陈述了自己的判断：“在我看来，人们正切实致力于将底层依赖精简到 EC2、EBS 和 S3（这些在其他云平台也能找到对标物），其核心逻辑在于：只要你提供这几项基础服务，我就能在上层自行构建系统，但我绝不想构建出一个死死依赖 RDS 或 SQS 这类组件的系统。”这种“最小公约数”架构理念，让企业能保留随时在不同公有云或私有裸机基础设施之间迁移的自由度，正是企业级用户对 Oxide 所代表的开箱即用机架级系统抱有强烈兴趣的技术背景。 - 行业趋势反转：五年前流行的“全盘采用托管云服务”正在退潮，避免厂商锁定（vendor lock-in）成为共识。 - 核心基础设施原语：现代云原生架构仅需可靠的计算（EC2 类似物）、块存储（EBS 类似物）以及对象存储（S3 类似物）。 - 软件自治权：通过在基础资源之上运行 Kubernetes 及自建服务，用户能够将控制权掌握在自己手中，摆脱对 RDS、SQS、Redshift 等专有 SaaS/PaaS 接口的强依赖。 --- ### [01:24:19](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=5059s) 硬件开发模式与开源软硬件生态的成熟针对有关云端 FPGA 实例（例如 AWS EC2 F1 实例）的提问，Cantrill 表示虽然公有云上的 FPGA 租赁服务确实存在并在特定算力场景下具备价值，但 Oxide 在硬件研发中并未采用这种方式。他们选择以最纯粹的经典硬件工程模式进行开发——工程师面前直接摆着物理硬件编程器和 FPGA 开发板进行真机调试。 ```mermaid flowchart TD accTitle: Oxide 技术栈演进依赖关系 accDescr: 展示 Oxide 如何依托 Rust、RISC-V 及开源硬件工具链构建自研机架系统 subgraph ECOSYSTEM["已成熟的开源技术底座"] RUST["Rust 编程语言 （已达生产级稳定性）"] RISCV["RISC-V 开放指令集架构 （生态现成可用）"] HDL["现代硬件开发工具 （Bluespec、Chisel、SpinalHDL）"] end subgraph OXIDE["Oxide 系统实现"] direction TB FIRMWARE["自研开箱即用固件与微控制器控制"] RACK["机架级统一服务器架构"] end RUST --> FIRMWARE RISCV --> FIRMWARE HDL --> RACK FIRMWARE --> RACK ``` 在总结公司所押注的技术趋势时，Cantrill 强调 Oxide 的宏大愿景并没有空想任何空中楼阁，也不寄希望于第三方未来交付不可预测的奇迹：“对于这些技术趋势，我们并没有依赖任何人去帮我们实现，因为它们早就在那里了。”他逐一列举了当前的坚实支柱：Rust 语言已经完全达到生产级（production-grade）要求并广泛可用；Bluespec 已全面开源，配合 Chisel、SpinalHDL 等新型硬件描述工具为设计赋能；RISC-V 生态更是触手可及，任何人今天都能在其上构建真实硬件。Oxide 既是这些独立繁荣的开源硬件、开放固件与现代系统语言浪潮的积极推动者与协作者，也是这一时代技术成熟度的最大受益者。演讲在热烈的掌声与致谢中圆满结束。 - 开发方法：Oxide 坚持在本地使用实体烧录器与 FPGA 板卡进行底层验证，确保对硬件底层的绝对控制。 - 生产级技术储备：Rust 语言与 RISC-V 架构已被实践证明完全能支撑企业级硬核基础设施的开发。 - 软硬件协同：Bluespec 开源化与新型生成式 HDL 的兴起，为全栈重构数据中心计算机提供了完备的工程土壤。 ## Notable quotes > [40:42](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=2442s) It was easier to get a hardware company funded two decades ago, but hardware has never been easier than it is today. — Bryan Cantrill > [29:22](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1762s) interoperability is great in principle but it is very corrosive when you want to make an actual tight integrated design — Bryan Cantrill > [30:48](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=1848s) I just believe strongly that a computer should have hardware and software that have been co-designed, that when you design things together you yield a better artifact because you are actually able to control elements of the design and you're able to yield a higher performing artifact. — Bryan Cantrill > [52:47](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=3167s) The idea of correct C itself has so many asterisks on it because there's so much undefined behavior in C. — Bryan Cantrill > [01:14:00](https://www.youtube.com/watch?v=vvZA9n3e5pc&t=4440s) And when you start to view the cloud as cogs now it's probably time to start thinking about being on your own physical machine at the scale we're talking about. — Bryan Cantrill ## Mentioned Bryan Cantrill, Jon Masters, Tracy Kidder, 《新机器的灵魂》（The Soul of a New Machine）, Data General, IBM 709, CTSS, DEC PDP-11/70, Jessie Frazelle, Steve Tuck, Steve Klabnik, Cliff Biffle, Rick Altherr, Star Simpson, David Patterson, Phil Levis, Jeff Bezos, Sun Microsystems, Sun Enterprise 10000 (Starfire), eBay, HP ProLiant DL380, IBM PC, Google, Facebook, Open Compute Project (OCP), Tioga Pass, Wedge, Bryce Canyon, Oxide Computer Company, Supermicro, ASPEED, Nuvoton, Intel Management Engine (ME), AMD Platform Security Processor (PSP), Spectre, Meltdown, Microsoft Cerberus, Google Titan, OpenTitan, Bluespec, Yosys, Chisel, SpinalHDL, RISC-V, Rust, DTrace, Tock OS, eGenera, Convex Computer, 《On the Metal》, AWS, Kubernetes ## Video description Bryan Cantrill Oxide Computer Company February 26, 2020 While our software systems have become increasingly elastic, the physical substrate available to run that software (that is, the computer!) has remained stuck in a bygone era of PC architecture. Hyperscale infrastructure providers have long since figured this out, building machines that are fit to purpose -- but those advances have been denied to the mass market. In this talk, we will talk about our vision for a new, rack-scale, server-side machine -- and how we anticipate advances like open firmware, RISC-V, and Rust will play a central role in realizing that vision. View the full playlist: https://www.youtube.com/playlist?list=PLoROMvodv4rMWw6rRoeSpkiseTHzWj6vu ► Check out our entire catalog of courses and programs: https://online.stanford.edu/explore [Source](https://www.youtube.com/watch?v=vvZA9n3e5pc)

Source quotes

[40:42] Bryan Cantrill: It was easier to get a hardware company funded two decades ago, but hardware has never been easier than it is today. [29:22] Bryan Cantrill: interoperability is great in principle but it is very corrosive when you want to make an actual tight integrated design [30:48] Bryan Cantrill: I just believe strongly that a computer should have hardware and software that have been co-designed, that when you design things together you yield a better artifact because you are actually able to control elements of the design and you're able to yield a higher performing artifact. [52:47] Bryan Cantrill: The idea of correct C itself has so many asterisks on it because there's so much undefined behavior in C. [74:00] Bryan Cantrill: And when you start to view the cloud as cogs now it's probably time to start thinking about being on your own physical machine at the scale we're talking about.

Source quotes

[55:28] Bryan Cantrill: With C you could feel the underlying assembly; with Rust you can feel the underlying C. [26:19] Bryan Cantrill: Your job is to bring the host operating system its cup of coffee and shoot yourself in the head; that should be the role of host firmware. [40:42] Bryan Cantrill: It was easier to get a hardware company funded two decades ago, but hardware has never been easier than it is today. [29:22] Bryan Cantrill: Interoperability is great in principle, but it is very corrosive when you want to make an actual tight integrated design. [52:47] Bryan Cantrill: I fervently believe that Rust is every bit as important revolution as C. [29:22] Bryan Cantrill: By rack-scale design, I mean really thinking about the entire rack; taking a bunch of 2Us and slapping it into a rack is not a rack-scale design, that is just rack and stack. [50:32] Bryan Cantrill: I don't want garbage collection anywhere near the systems that I'm developing.

Original transcript

[0:11] hey everyone I'm Brian Cantrell I'm the [0:13] CTO of the oxide computer company [0:15] I'm a little worried because John [0:18] masters who is a friend of mine from [0:21] wave Red Hat now at new via I tweeted [0:24] out a reference to this talk and he he [0:27] only included the title kind of up until [0:30] the : [0:30] Brian's gonna talk on the soul of a new [0:32] machine and I would say a decent amount [0:35] of the internet believed that I am [0:37] leading a book club discussion of Tracy [0:40] Kidder's superlative the soul of a new [0:43] machine I'm not so if I'm happy to talk [0:47] about the soul of a new machine I love [0:48] it it's an amazing book if you haven't [0:49] read it [0:50] it is truly it is it is the the Iliad or [0:55] the Odyssey it is the literature of our [0:57] domain but I'm only using it as a title [1:00] so no disrespect to cater um but what I [1:04] really want to speak of is really kind [1:05] of tap into some of that that same [1:07] fervor and passion and depth that that [1:09] kid are described in for those of you [1:11] don't know Sullivan a machine describes [1:13] the development of the the data general [1:15] eclipse couldn't eagle in the late 70s [1:17] and early 80's and it was it was kind of [1:20] it was a time when any of people asked [1:23] me if I romanticized you know the 70s or [1:25] the 80s or the 90s um and maybe a little [1:29] bit you know maybe maybe maybe a tiny [1:31] bit um I don't think I overly [1:33] romanticized them um but this is I would [1:36] call that the first real server because [1:40] this is the first machine that is really [1:43] powerful enough and um you know the 709 [1:47] is is an amazing machine it is it is the [1:51] first machine that is is really powerful [1:55] enough to actually have a time shared [1:59] workload on it and this is the this is [2:02] the last of the of the vacuum to vacuum [2:06] tube machines obviously machine this is [2:09] look at the control planner from the 7 [2:10] or 9 this is not the actual 709 itself [2:12] um [2:13] the this the actual 709 itself is quite [2:16] a bit larger than this I'm quite a bit [2:18] less beautiful this is actually from the [2:20] there was a terrific series of surveys [2:23] done in the the kind of the dawn of [2:26] computing of effectively all computers [2:27] in the US [2:28] and this is probably the this is the [2:30] last year I could have been reasonably [2:32] done but this is the the 709 that was at [2:36] Hughes there were a bunch of 709 I just [2:37] I call this one out because the the the [2:41] dimensions are something else for a [2:45] computer and this is one of the things [2:47] that that it's easy to forget and kind [2:49] of the post transistorized age vacuum [2:51] tubes have a lot of drawers there's not [2:52] the transistor is a really good idea [2:53] vacuum tubes are pretty power intensive [2:58] 205 kW for the IBM 709 which is so - as [3:04] a reminder like a rack draws somewhere [3:06] between if between 10 and 25 kW for a [3:10] rack today that is 205 kilowatts in 1961 [3:15] for a machine that is 33,000 pounds it [3:19] is 1,900 square feet I love the fact [3:22] that the machine is 2,000 square feet [3:25] and the air conditioner is another [3:26] thousand square feet so this is a [3:29] fire-breathing monster although it [3:31] wasn't really a monster I mean it was [3:33] really tepid from a sizing perspective [3:34] but it was the first machine that was [3:36] big enough and the reason it's important [3:37] is because it is the first server and it [3:41] in that it is the first machine that is [3:42] large enough to actually run a time [3:45] shared workload and indeed the [3:47] compatible time sharing system CDSs was [3:49] originally done on demonstrated on the [3:51] IBM 77 or 9 in the early sixties so the [3:54] server-side computing is born what I [3:55] mean by server-side computing I mean a [3:57] computer that you don't necessarily hold [4:00] in your hands you don't have in front of [4:02] you you don't necessarily see or hear or [4:04] smell it is running your workload at [4:07] some kind of a distance um you know fast [4:10] forward through third through the 70s um [4:12] you know if you are of a certain vintage [4:16] this is certainly Mother's Milk to you [4:18] this is the the pdp-11 70 this is the if [4:23] the the 709 was the the kind of the dawn [4:25] of the mainframe kind of kind of I've [4:27] cut out the 360 I would happily talk [4:28] about historical machines the entire [4:29] time I cut out the 360 obviously very [4:31] important machine but the pdp-11 70 is [4:34] certainly an interesting machine in many [4:35] different dimensions this is now a [4:38] scaled-down machine and one where you [4:40] can run workloads [4:42] much more locally than you could with [4:44] with a mainframe a very very important [4:47] machine actually when my co-founders has [4:49] gone a replica of this machine in her in [4:51] her house which is pretty great running [4:52] a Raspberry Pi blinking the lights which [4:54] is pretty neat but uh it and with the [4:57] kind of the mini computer revolution in [5:00] in the 70s [5:01] these machines were getting smaller yes [5:05] that is a good question that's a that's [5:07] a good one to register I actually don't [5:09] know I do know from having messed around [5:11] with the JavaScript emulator of the [5:14] pdp-11 70 that there are a lot to choose [5:16] from so uh the the you should be right [5:20] you should be running tops-10 tops-20 I [5:21] think ran on the PDP 1170 [5:24] no it didn't all right Brennan does it [5:27] see the sign nine [5:29] she certainly UNIX ran on the PDP 1170 I [5:32] don't think she's running a faux blown [5:33] UNIX girl anyway um important system ran [5:36] a bunch of different operating systems [5:37] obviously on the and a seminal machine [5:42] machine that definitely [5:44] outperformed from a deck perspective for [5:46] sure they expected to make ten thousand [5:48] of these and they made one hundred [5:49] thousand-plus really made the company um [5:51] then we get to so that you know this may [5:54] be Mother's Milk for many of you in the [5:55] room many people online this is Mother's [5:59] Milk for me so I worked for some [6:03] fourteen years this is the the son a 10k [6:06] I love the fact that this is this photo [6:08] is even taken from a vantage point to [6:10] make this machine look as imposing as [6:13] possible on this is a machine with 64 [6:16] CPUs [6:17] 64 gigabytes of memory and this is a [6:19] machine that essentially 1999 um you [6:23] know I I feel like as a survivor of the [6:26] dot-com bust I feel it's my [6:28] responsibility to younger generations to [6:30] explain that you actually are not in a [6:32] recession proof industry this box in [6:35] particular ran a good chunk of the [6:38] internet a lot of a lot of websites ran [6:39] on this sometimes infamously eBay [6:42] started off on a two processor machine [6:45] ended up with one of these 64 processor [6:47] giants and because they didn't revisit [6:50] their architecture along the way hadn't [6:53] bother to make or they could take the [6:55] Machine down excuse me um for any kind [6:58] of maintenance at all so of course they [7:01] were they were operating under a death [7:03] warrant that was signed that they didn't [7:04] quite realize and eBay very famously [7:07] went down in 1997 running one of these [7:10] machines in a disaster of absolutely [7:14] biblical proportions down for 22 hours [7:16] but it was it was eye-opening in a lot [7:19] of ways it was eye-opening because of [7:21] how important server-side computing had [7:24] become that we were now actually running [7:26] commerce on a single machine running [7:29] commerce with the server-side computers [7:32] that we had actually not really designed [7:36] with uptime in mind suddenly became very [7:39] load-bearing with respect to this new [7:42] economy and it was very very eye-opening [7:44] but so what what happened from 1999 and [7:48] from the kind of the you had a couple of [7:50] very keep your company's excuse me keep [7:53] your economies making very large [7:54] machines and what what happened is a [7:57] microprocessor and this was the spark [7:58] microprocessor which you know sadly the [8:02] the children today it's not necessarily [8:03] heard of because spark and every other [8:06] risk microprocessor effectively lost the [8:09] plot to to Intel x86 and what happened [8:12] was that the the the microprocessor in [8:15] here and in power machines and in PNP [8:18] risk based machines and in alpha [8:20] machines could no longer compete with [8:22] the economics of the x86 and what people [8:26] began to realize is like actually I can [8:28] get you this a 64 processor machine but [8:30] I can have a workload that runs maybe [8:32] maybe with a quarter of the number of [8:34] CPUs it's maybe a maybe a tenth but I [8:37] can do that for a twentieth of the cost [8:39] or one one-hundredth of the cost um and [8:42] that effectively um that revolution the [8:45] microprocessor revolution extinguished [8:48] these large servers and we ended up with [8:51] this um we went from from this in 1999 [8:55] as the state of the art in server-side [8:57] computing I think I'm gonna need some [8:59] water by the way um to my iPod is much [9:02] drier down here that it is an in San [9:03] Francisco um the two [9:06] um to this in in 2009 and uh this is sad [9:11] in a lot of ways um so um and I don't [9:15] mean to pick on it this is an important [9:16] machine this is the HP dl380 I think [9:19] this is a Gen 5 dl380 the dl380 became a [9:24] ubiquitous machine again x86 at its core [9:27] but the reason this is sad is because [9:29] this is a this is a server thank you all [9:32] right this is a server but you notice [9:35] there's I there's a CD tray in front [9:37] it's like what is it what does a CD tray [9:39] I mean we were going to be playing music [9:40] in the datacenter why do we have a why [9:43] do we have a CD tray in here and what is [9:46] the display port that's a VGA port [9:48] sitting over there uh what is that thing [9:51] for and the answer is that this is [9:54] actually despite it being rack-mounted [9:56] this is a personal computer [9:59] this is architectural a personal [10:01] computer architectural II this is the [10:04] machine talked about you talk to the [10:05] pdp-11 70s as outperforming expectations [10:07] the IBM PC really outperformed [10:09] expectations right and the IBM PC was [10:12] really designed to be you know IBM's [10:14] entree into what seemed to be a very [10:16] hobbyist market they thought they're [10:18] gonna sell 10,000 of them obviously sell [10:20] millions and millions of them and the [10:22] architectural decisions that were made [10:23] in Boca Raton in 1979 are still being [10:27] seen in this machine in 2009 so you've [10:29] got a whole bunch of parts that are no [10:32] longer being made in 2009 you've got the [10:34] the the IBM the the intel 8259 are all [10:39] these these little parts that are now [10:41] being emulated effectively in this [10:43] server in 2009 and the it's it's just a [10:48] and the only reason we have that is [10:49] because it was the microprocessor that [10:51] one is because the x86 conquered every [10:53] other microprocessor it dragged along [10:56] this kind of regretable architecture [10:57] with it and now at the same time um [11:00] there were a bunch of folks whose [11:02] computing needs were really growing very [11:05] very substantial and they were beginning [11:09] to as they were looking down these [11:11] machines they were unimpressed and these [11:13] are folks like Google when Todd talked [11:15] about Google in the intro by 2009 [11:17] Google's got very substantial [11:20] it needs and it's dawning on Google that [11:22] they actually don't want to run personal [11:24] computers in their data center they want [11:26] to run a more purpose fit machine and [11:29] this is kind of their first attempt at [11:32] this this is kind of an infamous attempt [11:34] at their first machine in that you can [11:36] see if they've got velcro so there are [11:38] those disks and the power supply are all [11:41] secured with velcro the you can see that [11:44] it goes a 12-volt battery there in in [11:47] lieu of a UPS they actually just had a [11:50] 12 volt battery attached to every server [11:53] this gigabyte motherboard is is the kind [11:58] of thing you'd sweep off a floor I mean [11:59] no offense a gigabyte but they've gone [12:01] with absolutely they have saved they've [12:05] shaved everything down to the pennies [12:07] here to try to have the cheapest [12:09] possible machine on which seems like a [12:13] reasonable thing to do like we're buying [12:14] a gazillion of these we should make [12:16] these as cheap as possible and Google [12:19] learned a lesson that has been learned [12:20] for time immemorial in computing that is [12:24] that um you are but you always want to [12:26] seek value but you have to be careful [12:28] about unwittingly really getting what [12:30] you paid for and when you have [12:33] components that are that are DRAM [12:36] components that have been refurbed for [12:37] example that have that have errors in [12:41] them or that are non-ecc I mean on each [12:44] CD Ram the the kinds of failures that [12:47] you have up the stack aren't worth the [12:50] pennies that you saved but they were [12:53] onto something they Google and Facebook [12:55] and others were onto something in that [12:58] they were getting rid of a bunch of [13:01] things that they didn't need so you know [13:04] wait aren't what you don't see on this [13:06] is you don't see the CD drive you don't [13:08] see the VGA port you don't see some of [13:11] these kind of these personal computer [13:14] ephemera and again accoutrements are [13:16] being stripped down to have a really a [13:19] purpose-built machine um and as time [13:22] went on so this is the hyper scale [13:24] computing circa 2009 this is kind of [13:27] hyper scale computing a mistake circa [13:29] 2020 this is more like 2016 2017 [13:32] so this is [13:33] a Facebook's Tioga Pass design um so [13:36] this is a compute sled this is a compute [13:39] sled that goes right into a bus bar so [13:41] there's no you know here we had a power [13:45] supply power supply is gone here thank [13:48] you very very much the power supply is [13:52] gone here there's no power supply this [13:55] is plugging straight into a bus bar a DC [13:57] bus bar there's a power shelf in the [13:59] middle of the rack that's doing all the [14:00] conversion and when you are at any kind [14:04] of modest scale this is a much more [14:06] sensible design it makes much more sense [14:08] to have a DC bus bar and everybody was [14:10] kind of coming to the same conclusion [14:12] however the hyper scale which we're all [14:13] kind of coming to the same conclusion [14:14] that people come to for all I add for a [14:16] long time which is that power conversion [14:17] is really inefficient and the more times [14:19] you do power conversion the more you [14:21] lose and as you begin to optimize around [14:23] PUA [14:23] and driving that PUA down it is try to [14:26] be as efficient as possible to have all [14:28] of your power go to your computing and [14:30] none of it going to heating the room or [14:33] as little as possible the heating the [14:35] room um you are naturally go to these [14:37] designs so there are couple [14:39] so there's bus there's a bus bar design [14:41] that's interesting the the networking [14:43] cables are actually in the front for for [14:45] accessibility this slides into a wider [14:48] chassis so this has got three across [14:51] instead of two across it is not the [14:54] 19-inch rack the 19-inch rack is a very [14:57] historical kind of a number um and this [15:00] is going into a rack that is a that it [15:02] fits on a floor tile fits in a 24-inch [15:03] floor tile but is a wider rack it's a [15:06] taller rack I'm it's got that that power [15:08] shelf in it and it's designed to be a [15:10] this is part of a rack that has true [15:12] rack level design face book design their [15:14] own switch wedge that goes with this [15:16] they've they designed their own storage [15:18] chassis Bryce Canyon that goes with us [15:20] and all made other designs all made [15:22] available in the open so this is this is [15:25] the evolution of computing it's kind of [15:26] good to see you like we're back on track [15:27] so computing is back on track we've got [15:30] these modern designs okay we can all [15:31] breathe easy well not so fast because [15:34] this is in hyper scale computing and [15:36] this is great and this is again this box [15:38] is a really nice box [15:39] um meanwhile this is a server-side [15:41] computing circa 2009 and here the grand [15:45] reveal server-side computing [15:47] circa 2020 again not to pick on HP but [15:50] sorry HP so we have gone from let's say [15:53] we've gone from a CD drive to a DVD [15:56] drive so literally you can I mean you [15:57] when you buy these machines today they [15:58] have a DVD drive um they've got we've [16:02] gone from VGA to HDMI I mean these are [16:05] not improvements these are improvements [16:07] for a personal computer or for a machine [16:09] that is designed to run Windows not a [16:11] machine that should be designed to run [16:14] in a datacenter this is still a [16:16] rack-mounted personal computer and it is [16:19] that much more galling that the world [16:23] that is operating at any kind of scale [16:24] has to actually look at this [16:26] almost I mean it's I'm so glad that [16:29] Facebook and Google and so on have been [16:31] so upfront about their designs butBut it [16:33] sharpens the pain for those of us who [16:36] don't have the infrastructure privilege [16:37] of these folks to know that like [16:39] actually there's great innovations we [16:41] just can't have them and that's that's [16:43] very frustrating it's very frustrating [16:44] to me personally so so my personal [16:47] history I was at Sun Microsystems for 14 [16:49] years and then Sun was invaded sadly I [16:52] left after the invasion [16:54] um the I went to a cloud computing [16:57] company and it was there for for ten [16:59] years I was bought by Samsung and we [17:01] were building out a very large private [17:03] cloud effectively and we were doing it [17:05] on more or less these kinds of machines [17:08] and it's not just these were [17:11] aesthetically displeasing it's that [17:13] there were real consequences up and down [17:15] the stack for running what is [17:17] effectively a personal computer so this [17:20] brings us to kind of the problem that we [17:21] have today there are and I'm in this is [17:24] the the our very contrarian thinking I I [17:27] actually don't believe that Jeff Bezos [17:29] is going to own and operate every [17:30] computer on the planet they're a [17:31] renegade belief I believe that there are [17:34] reasons to own and operate your own [17:35] computer [17:36] they are often economic reasons I'm [17:39] there a lot of reasons there can be risk [17:40] management reasons but they're they [17:42] often well down to economics on that one [17:45] you are running a certain amount of [17:47] compute it makes more sense to own it [17:49] rather than rent it and we've seen this [17:53] from a bunch of different companies at a [17:55] bunch of different scales [17:55] I'm not least at my former employer so [17:58] what we know that these economic reasons [18:00] exist but we [18:01] got the this bifurcated world and the [18:03] bifurcated world is actually even even [18:05] worse than it looks like because part of [18:08] the reason in that when we look at that [18:09] that HP commercial server we look at [18:11] this thing part of what it is trying to [18:13] do is present [18:16] - honestly Windows a personal computer [18:19] upon which to run so the part of its not [18:23] like the engineer's these companies [18:25] believe that this is the right [18:27] architecture it's just that they are [18:28] confined to their layer of the stack [18:32] which is the hardware the machine they [18:34] don't own the actual software substrate [18:36] that sits on top of it and they they [18:39] don't really have the ability to control [18:41] that software substrate that's that's [18:43] another company usually not just another [18:45] organization and meanwhile the people [18:47] that do control that software substrate [18:49] don't have necessarily any understanding [18:51] or desire to control the hardware [18:53] substrate so you end up with this this [18:56] divide the standoff that has endured now [18:58] for a decade plus where you you have [19:01] these two effectively technology pools [19:05] are entirely isolated from one another [19:07] and they operate at cross-purposes I [19:10] mean if you are tasked with right now if [19:13] you are tasked given a budget and told [19:15] to run an elastic compute infrastructure [19:18] platform on your own premises you are [19:21] stuck gobbling things together you are [19:23] stuck and and the the state of the art [19:25] is really no better than it was 10 years [19:27] ago um you are stuck with personal [19:29] computer companies you are stuck the [19:31] open source software that's out there is [19:33] not really designed to run clouds at any [19:35] kind of scale so you are gonna be [19:38] designing it you're gonna be integrating [19:40] it you're gonna be operating it and most [19:42] importantly you are going to be [19:43] supporting it and it's extremely [19:46] frustrating you can have arbitrary [19:48] sophistication and still be very [19:50] frustrated by this it's only that was [19:51] the case for us um we had had many many [19:54] many many problems um one of which was a [19:57] pandemic of dim failures and we were [20:00] seeing pent up pandemic of uncorrectable [20:02] dim failures and the thing that was very [20:03] frustrating to us is that we would see [20:06] no correctable x' before seeing that [20:08] uncorrectable the uncorrectable that [20:10] would ultimately panic the box and if [20:12] you've been around memory for any length [20:14] of time [20:14] you know that's not really the way the [20:16] world works that you don't in ECC memory [20:18] you don't see an uncorrectable generally [20:20] out of clear blue sky you generally see [20:23] a rising number of correctable errors [20:25] and those correctable errors indicate [20:26] you've got a problem here we're losing [20:28] data here and this can be due to design [20:31] defects this can be due to poor dim [20:33] training this can be due to other [20:34] there's all sorts of things that can [20:36] cause this kind of loss of data but [20:39] those correctable indicate that that's [20:40] happening well the problem is that the [20:42] firmware was eating all those [20:43] correctable --zz so we in in the [20:46] software that we controlled we had no [20:48] ability to forecast this was coming and [20:50] by the way if we knew known it was [20:52] coming we could have taken action on [20:53] this and if we could have known that hey [20:55] this machine by the way is has seen a [20:57] spike in correctable errors there are [20:59] things that we can go do to force that [21:01] force those dents to retrain or what [21:02] have you but without that kind of [21:04] insight and trying to get firmware that [21:07] would actually properly elevate these [21:10] errors into system software was like [21:12] trying to get a bill through Congress it [21:14] would just felt like you know you go to [21:16] again you go to a personal computer [21:17] company and ask for a custom firmware [21:19] build and they're like why would anyone [21:21] running a personal computer want their [21:23] own firmware doesn't make any sense like [21:24] well I'm not running a personal computer [21:25] I'm running a cloud but if that [21:28] disconnect was felt very acutely by us [21:31] certainly and it's been acutely felt by [21:33] many many many people this is not this [21:35] is not novel or not unique to us so we [21:38] feel you really need a fundamentally new [21:41] approach and so we think you need to [21:44] take a pretty clean sheet of paper here [21:47] and so if you're gonna go design a [21:49] computer in 2020 from scratch ish what [21:55] would you want um first let's not [21:58] actually redesign the microprocessor so [22:02] the the microprocessor be it Intel or [22:05] AMD or even arm the these are very [22:09] sophisticated devices and they [22:11] themselves are not are are fit for [22:14] enterprise purpose so Intel and AMD and [22:17] arm and so on are are fit for the [22:19] purpose we want one of those folks that [22:21] said there are a bunch of other things [22:23] we want to go revisit in particular one [22:25] of the things we want is real firmware [22:27] out of state [22:28] so when you run you know when you run [22:31] one of these guys you've got no [22:34] knowledge of the firmware that's [22:36] actually running on that machine and if [22:39] you have an issue where you're required [22:41] to take a firmware upgrade our firmware [22:43] update you get a BIOS upgrade or what [22:45] have you [22:45] you've got no idea what's in that um [22:48] what you do have if you do a kind of a [22:50] modicum of analysis on the binary you [22:53] will begin to lose sleep at night very [22:54] quickly because you will see things like [22:56] URLs like what is a URL doing in my bias [23:00] this is the brainstem of my data center [23:04] why is there a URL in here I'll try to [23:06] get an answer that question I am NOT [23:08] that it's like an uncomfortable question [23:10] it's you literally can't find anyone who [23:12] can answer it it's something they don't [23:13] want to answer it necessarily [23:14] hello bat - what is it URL doing and [23:18] presumably not because your vendor has [23:20] been infiltrated by kim jeong-hoon [23:22] presumably but you don't actually know [23:25] that and you've got no way of knowing [23:28] that you've got no way of knowing that [23:29] what you're running it is a tested [23:31] you've got third parties in there I mean [23:33] even when you have companies that wish [23:35] to actually provide the source code [23:37] they're from where they're from where [23:38] isn't even theirs this is one of these [23:39] things that's just stunning to me and I [23:41] mean no offense to them but how is am I [23:45] still running at the brainstem of [23:47] humanity the american megatrends [23:49] incorporated this is a company that was [23:51] around when I was a kid I'm old it has I [23:55] mean again it's got a right to exist but [23:57] why does an am i bias exist on [24:00] industrial grade machines it makes no [24:02] sense so you got these third parties [24:04] that that aren't gonna make available [24:06] their source code you can't actually see [24:07] into it um and you can't actually know [24:09] what you're running and this becomes [24:10] really really really relevant if you saw [24:12] the the Bloomberg story in 2018 the fall [24:16] of 2018 about the super micro [24:18] infiltration or a supply chain [24:20] infiltration in Super Micro um the and [24:23] it would describe a very sophisticated [24:24] attack that the reporter stood by and [24:26] there you can speculate about what [24:29] actually happened there almost certainly [24:31] the story as written was not true even [24:34] though I think the reporter believes is [24:35] true and that that super micro supply [24:39] chain was not actually infiltrated [24:41] and there was not a device we don't [24:42] believe I no one has produced that put [24:44] it that way but you may have had the [24:45] same reaction that I certainly had upon [24:47] reading that story which is out of [24:48] curiosity why would anyone mount this [24:52] very sophisticated attack frankly when [24:55] you can just walk in through the [24:56] firmware most Super Micro bmc's have the [25:00] default root password on them many of [25:03] them are on the internet they are many [25:05] of them are vulnerable to known [25:07] vulnerabilities I mean if you really [25:08] want to get terrified there's a [25:09] vulnerability called USB anywhere um [25:11] that found by Rick althought who [25:13] described it at the open source firmware [25:14] conference last year and as part of this [25:17] Rick was looking for BMC s that were on [25:19] the Internet he found 77,000 BMC's that [25:21] were on the internet that he that could [25:23] have been owned and rick is an extremely [25:25] responsible person so he made sure to [25:27] get it all communicated and so on but [25:29] that is actually terrifying and the BMC [25:31] by the way is the the base board [25:33] management controller this is the [25:35] computer within the computer well this [25:37] is one of the computers in the commune [25:38] there are several computers in the [25:39] computers it turns out and this is one [25:41] of the computers in the computer this is [25:43] the one that is the most troubling and [25:45] in some regards because it's a full [25:47] functioning machine in there it's [25:49] running you boot running Linux and [25:52] running proprietary software you can't [25:54] see from either a speed Anupa our new [25:56] baton and that thing is on the brainstem [25:58] that is a bus master that is sitting in [26:00] your computer um and by the way you hang [26:03] it out on the on the network because [26:05] that's what your vendor tells you to do [26:07] so this BMC is talking to both your PCI [26:10] bus is a PCI bus master and it's talking [26:12] to the network and you've got no idea [26:14] what's in that software that's [26:16] disturbing and troubling um we need to [26:19] get the we've got to revisit all this we [26:21] need an actual proper hardware root of [26:23] trust that can actually attest firmware [26:25] and this this kind of BMC outgrowth um [26:28] needs to be completely rethought and [26:31] slashed and we need to go back to a [26:34] service processor what this thing needs [26:36] to do is it needs to be able to bounce [26:38] the box it needs to be able to give you [26:40] a serial line and that's about it there [26:43] may be like you encountered on one hand [26:45] the other things to you yes you need to [26:47] have some fan controlling things in [26:48] there what it does not need to do is [26:50] provide virtual VGA [26:52] or virtual USB it doesn't even [26:54] necessarily need a networking stock kind [26:56] of depending on how you plumb it up but [26:57] we need to get back to a service [26:59] processor and away from that this is [27:01] kind of horrific ly complicated BMC um [27:04] we also need host firmware whose mission [27:08] in life should be to hand the CPU off to [27:13] the host operating system you host from [27:15] where what your job is is is to bring [27:18] the host operating system its cup of [27:20] coffee and shoot yourself in the head [27:22] that should be the role of host Ramon [27:25] it's a little graphic I apologize the [27:26] the that is the role of host firmware [27:30] this idea that firmware should execute [27:34] and then somehow stay resident so it can [27:37] be called into at arbitrary times to do [27:40] arbitrary things is it's this is Doss [27:42] that's that's the DOS mentality and [27:45] approach and boss robbed me of my [27:47] childhood okay I've got a chip much [27:48] older about thoughts because I grew up [27:50] without memory protection at a time [27:52] after memory protection existed and I [27:54] just thought computers just kind of [27:55] reset randomly um they don't and [27:58] shouldn't um and as soon as I found out [28:00] about memory protection when I arrived [28:01] in college I'm like wait a minute why [28:03] didn't I have Unix growing up uh I grew [28:05] up in the 80s UNIX was around and of [28:07] course so complicated were answers to [28:08] that question but we should not have a [28:11] Doss mentality we should not be relying [28:13] on this kind of unseen firmware that is [28:16] load-bearing and and needs to do works [28:19] there's a lot to be done on the firmware [28:20] side and we talked about the the the BMC [28:24] is the computers in the computer there [28:26] are actually several computers in the [28:27] computer there's one that's even in some [28:29] ways more disturbing some ways slightly [28:31] less because it's um called the Intel [28:33] management engine or the platform [28:35] security processor on AMD the PSP the [28:38] management engine is an entire operating [28:39] system that exists on on the another [28:42] computer that exists effectively on the [28:44] motherboard and that thing also has [28:47] total control over heaven and earth it [28:48] runs MINIX and yes there have been [28:51] vulnerabilities in it so on a management [28:53] engine is still completely proprietary [28:55] the PSP is AMD's equivalent of this it [28:58] actually at least lives on die makes me [29:00] ill slightly less worried about it it's [29:02] opposed to the PC age but we need to [29:06] visibility into it is a reasonable thing [29:10] to be able to understand every [29:12] instruction that's executing on your [29:14] computer that is a reasonable request we [29:16] are not there yet Rex you're a long way [29:18] from that but but we think that that's [29:19] that's needed for for a new computer um [29:22] we also need a true rack scale design so [29:25] there have been a couple in the industry [29:27] that have attended by Rack scale design [29:28] I mean really thinking about the entire [29:30] rack taking a bunch of to use and [29:32] slapping it into rack is not on a rack [29:34] scale design that that is that's just [29:35] rack and stacked but this is where we [29:37] are actually integrating the top of rack [29:39] switch with to integrate our networking [29:42] with storage with compute and actually [29:44] building these things together so they [29:46] are aware of one another we are actually [29:48] Co designing them you cannot do this [29:50] without software and a bunch of [29:54] companies they had one I do want to pick [29:56] on them you know an Intel is that had a [29:57] lot of rack scale design work for a long [30:00] time but they never wanted to have a [30:02] software component because software or [30:03] someone else's job the problem is that [30:05] if you don't have that software [30:06] component you cannot deliver a true rack [30:09] scale design so we need to take a real [30:13] rack scale approach what we're designing [30:15] these things together you can't just and [30:17] this is where you know interoperability [30:19] is great in principle but it is very [30:23] corrosive when you want to make an [30:24] actual tight integrated design and we [30:27] need to not be interoperable with [30:28] arbitrary topper rack switches you need [30:30] to pull that top rack switch in and then [30:32] push that boundary of interoperability [30:34] out to the network and clearly you need [30:36] to be able to talk to other switches and [30:37] so on in the network but we want racks [30:41] cosine um we also want to have a dense [30:43] form factor so we actually deserve the [30:47] kind of density that we see out of the [30:48] hyper scalars this is a real challenge [30:50] because the the hyper scalars really do [30:53] design their data centers basically from [30:56] scratch and they do a bunch of things at [30:59] the DC level that are very hard for [31:02] someone else to emulate so if you and [31:05] you can be a have a lot of compute but [31:08] it's very hard to get to kind of their [31:10] level of scale where you're locating [31:11] DC's next to dams and so on so this is a [31:15] challenge because we're in the [31:17] enterprise in the mainstream enterprise [31:18] you're not going to drop [31:20] 25 kW into Iraq I mean I love to it'd be [31:22] great but that's not practical because [31:25] that that it's just it's it's too much [31:28] draw too much too cool [31:30] so you defecate it run the plants are [31:32] running an IBM seven or nine or probably [31:33] been trashed but we need that said [31:37] within the kW budget we have for a rack [31:39] which on the order of 10 to 15 kW we [31:42] need to get maximal density and need to [31:44] look at things like a bus power base [31:45] design why why can I not buy a machine [31:48] if I'm you know General Electric or I'm [31:52] Manoa Beth I'm Barclays if I am General [31:55] Motors I can't buy a bus bar based [31:57] machine no matter how much compute I'm [31:59] operating that seems to be a little bit [32:00] ridiculous because it seems to me that [32:02] we should be having an eye towards [32:04] really efficient operation which is [32:05] unnecessary low-power operation by the [32:06] way this is a high power operation but [32:09] but really targeting efficiency so [32:11] there's a lot to be done from from the [32:13] hardware and firmware perspective and [32:15] then there's a lot to be done from the [32:17] software perspective and then this is [32:19] really you know I think in terms of the [32:20] the the beliefs that we and I have this [32:23] is probably the one that is is that the [32:25] the deepest in our marrow and this is [32:28] where you just you're just not gonna get [32:30] the Sun Microsystems out of me [32:31] I just believe strongly that a computer [32:35] should have hardware and software that [32:37] have been Co designed that when you [32:40] design things together you yield a [32:43] better artifact because you are actually [32:46] able to control elements of the design [32:48] and you're able to yield a higher [32:50] performing artifact but in terms of what [32:52] does that software mean what what you [32:55] know where do you kind of stop the scope [32:57] of that I think in terms of in 2020 the [33:00] boundary for the computer is at the API [33:03] endpoint to provision a virtual machine [33:05] you have to view the the you know if the [33:07] if the the boundary in the 70s and 80s [33:10] was kind of that that multi-user shell [33:13] um it was kind of the the boundary then [33:17] the boundary now is much higher up in [33:20] the stack where I can hit an API [33:21] endpoint provision and provision of a.m. [33:23] there's a whole lot of software but [33:25] between those API endpoints and the [33:27] actual provision is a hypervisor there [33:29] are there's a toprock switch there there [33:31] is a [33:32] computer control playing there's a [33:33] storage control plane and then you're [33:36] gonna have API endpoints for both [33:37] operator developer so a lot of software [33:39] to be done in the modern computer but I [33:41] think this is what this is what's [33:42] required because right now that software [33:44] is being cobbled together by the end [33:45] user and that that seems unfair and [33:49] unjust to me now that said okay that's a [33:51] lot of software so we gotta like well [33:53] the software onto the machine so well [33:56] yes and no because certainly the era of [33:59] proprietary infrastructure software is [34:01] well and truly over so we you don't want [34:04] to you don't want to do bluntly what [34:06] Apple does I love the iPhone but the the [34:09] inaccessibility and - I love you iPhone [34:11] from a user experience perspective but [34:13] the inaccessibility from the perspective [34:16] actually understanding the system and [34:17] being able to modify it actually is a [34:18] problem um so the software really needs [34:22] to be fully open and completely attested [34:25] so you want to build on on the this [34:27] tradition of open source software [34:28] fortunately a lot of what needs to be [34:30] done here [34:31] is software that much of if you if you [34:34] look at you begin to envision that rack [34:37] scale machine a lot of what needs to be [34:40] developed is software yes there is [34:41] hardware but a lot of the hardware and [34:43] the hardware innovations are actually [34:45] out there fortunately Facebook [34:47] interaction will talk about mid and [34:49] there the next slide in terms of how [34:50] things are changing but things are [34:51] changing such that this is actually an [34:52] easier problem so um is this attainable [34:56] at all can we do this and they're been [34:59] plenty in the industry by the way who [35:01] have accepted the status quo and have [35:06] just that you know what this is just not [35:08] gonna change hopefully this will get [35:10] better but it's not gonna change and [35:13] honestly um I founded myself among that [35:16] number I was very very frustrated by the [35:18] this kind of bifurcated state but I was [35:21] just kind of hoping it would get better [35:22] um and when I kind of went back and [35:25] deployed a bunch of new computing like [35:27] please let have gotten of course it's [35:28] not gotten better because it's so it's [35:30] structurally so problematic so um we we [35:34] have to be able to do this we have to [35:36] solve this problem because one this is [35:39] not science right this is engineering we [35:41] actually know how to solve all these [35:42] problems it's just that we've got this [35:44] kind of this this organizational stand [35:46] that is preventing them from being [35:47] solved but there are reasons to think [35:49] that this is attainable first of all [35:51] this can be tightly tailored so you [35:52] don't need to run importantly running a [35:55] computer developing a computer that can [35:57] run an arbitrary operating system from [36:00] das 6.2 to whatever - to a modern [36:06] operating system is a very constrained [36:09] problem and if you're gonna run every [36:12] operating system Under the Sun or gonna [36:13] run every hypervisor and support that [36:15] that becomes difficult when you can [36:17] tightly tailor things the opportunities [36:20] open up by the way with the hyper [36:21] scalars do the hyper scalars do not run [36:23] arbitrary operating systems on their [36:25] bare metal they run their hypervisors on [36:26] their bare metal so we can tightly [36:28] tailor things and by designing one with [36:31] the other you can eliminate false [36:34] generalities in both software and in the [36:36] hardware so you don't need to say in the [36:38] software you don't need to support every [36:39] device Under the Sun you can support the [36:41] devices that you've actually selected so [36:43] that you can make it a smaller [36:45] engineering problem by being by being [36:48] careful but importantly there are also [36:50] some larger trends that actually make [36:52] this easier than it has been [36:54] historically because historically I [36:56] think this is this would be actually [36:57] very difficult um there's a couple of [36:59] big trends a first is on the on the [37:02] hardware side lots of folks folks have [37:05] recognized the need for a harbor root of [37:08] trust and this is something that um you [37:10] know I think we have and I for this and [37:13] I think actually that the final point on [37:15] this slide I think we actually have [37:17] Mianus horriblest of 2018 to thank [37:20] inspector and meltdown so Spector in [37:23] meltdown were the vulnerabilities in [37:26] Intel's CPUs I discovered those on [37:31] January 1st 2018 with most of the world [37:34] in a hacker news thread where someone [37:37] was speculating about some kind of fishy [37:38] patches they saw the linux and wondering [37:41] if there had actually if there was a [37:44] vulnerability here and of course we had [37:46] not been a part of any embargo I hadn't [37:48] heard anything where a huge Intel [37:49] customer at the time I'm thinking like I [37:52] hope this is not the case but over the [37:54] next couple of days of course we learned [37:56] that we were vulnerable to not just [37:58] Spectre but [37:59] but meltdown and meltdown so for those [38:02] who are person we've heard a spectrum [38:04] meltdown meltdown in particular it is [38:07] really really bad all of these things [38:09] are effectively ironically using the the [38:13] speculative execution of the [38:14] microprocessor to get access to [38:16] information you're not supposed to have [38:17] and it's ironic because it was that [38:20] speculative execution that actually [38:22] allowed x86 to get past the memory wall [38:24] and best effectively every other [38:27] microprocessor out there in fact I was [38:29] talking even John masters who was saying [38:31] that like hey do you think SPARC would [38:33] have any of these issues like no spark [38:35] would not have any of these issues [38:36] because sparks problem was not [38:38] speculative enough I this spark wasn't [38:40] smart enough to have these issues [38:41] bluntly I mean over smart it's just [38:43] pejorative but um the spark didn't [38:46] speculate enough so the the presence of [38:48] these vulnerabilities deep deep deep [38:50] deep deep in the stack highlighted all [38:52] of these problems up the stack and [38:55] suddenly all these firmware issues that [38:57] had been people knew were out there [38:59] suddenly you had said this this beacon [39:02] to security researchers that this is a [39:05] target-rich environment you can become [39:08] famous by like gangster famous which is [39:11] what all the kind of security researcher [39:12] as far as I can tell want to become by [39:14] finding these vulnerabilities and some [39:18] very very clever people pored over some [39:21] very ill-conceived a written software [39:24] that was not designed to have that kind [39:26] of examination we found vulnerabilities [39:30] everywhere and people are now very [39:33] concerned about ha the hardware root of [39:34] trust and from ran a station this is [39:36] great so we've seen it Microsoft [39:39] Cerberus is an effort that they're there [39:40] they are opening up but more recently in [39:43] Google's open Titan so Titan is in fact [39:47] in many ways the the best computer that [39:50] people don't I think properly [39:51] acknowledged is the Chromebook the [39:53] Chromebook is actually a really really [39:55] impressive machine in a lot of ways the [39:57] Chromebook actually solves a lot of [39:58] these problems albeit for the personal [40:00] computer so it was firmware that was [40:02] attested did have a harbor trust alarm [40:04] that's Titan um and and allows you to [40:07] actually you can load your own firmware [40:09] on it in that regard it's in contrast to [40:11] the iPhone but when you load your own [40:12] firmware [40:13] on it then the Chromebook itself knows I [40:15] am running foreign firmware and it knows [40:18] that to not attach to something [40:20] it knows that it can't trust the [40:22] software substrate that it's on um so [40:24] the they did this with Titan and and the [40:27] microprocessors that Titan is kind of a [40:28] family of a six Google open Titan is [40:32] Google's effort to actually generate an [40:34] industry-wide effort now open Titan like [40:37] many things that Google does Google did [40:40] not open up Titan they are generating a [40:42] new effort that is in the open so this [40:44] is kind of like a kubernetes versus Borg [40:45] if you will but that's great that's [40:47] great effort certainly law that um and [40:50] tacking into those efforts collaborating [40:53] with them and so on makes it easier [40:55] we're not by ourselves trying to solve [40:57] this problem so that that's a really [40:58] interesting development another big [41:00] development that I don't think it's [41:01] nearly enough attention is the open EDA [41:04] movement um so the the grand irony is uh [41:08] you know people um it was easier to get [41:12] a hardware company funded two decades [41:15] ago but hardware has never been easier [41:18] than it is today it is the the [41:21] developing and you know a historically [41:23] the EDA space has been one of the last [41:25] bastions of proprietary software and now [41:28] you're seeing all sorts of interesting [41:30] things out there yo sis chisel spinal HD [41:33] all these are HDLs yes verification and [41:36] then something I mean this is a very [41:37] emerging area something called blue [41:39] speck which was actually developed 15 [41:41] plus years ago at MIT um was [41:44] historically a proprietary I would call [41:47] a VHDL it to call an HDL does it a [41:50] disservice because blue speck is [41:52] actually taking things up level in terms [41:54] of the abstraction in which you you [41:56] define the the FPGA um blue speck is [42:00] really interesting this is just open [42:01] source like ten days ago two weeks ago [42:03] something like that very recently um [42:05] really really interesting development um [42:07] and we are very engaged I would say in [42:13] blue spec so a lot of interesting things [42:15] on the FPGA side making it easier and [42:17] easier and easier to do one's own FPGA [42:20] which of course allows you to do all [42:22] sorts of interesting things another [42:24] terrific development is that [42:26] element of risk 5 so risk 5 is a new [42:32] instructor set architecture by Dave [42:34] Paterson & Co um if you haven't I really [42:37] recommend reading the the risk 5 Atlas [42:39] EE they call it where they go through [42:42] the instruction set it is a delightful [42:44] read because they are so deliberate [42:47] about learning from everything in the [42:50] history of instruction set design so [42:52] trying to do really trying to learn what [42:55] was done right and what was not done [42:57] right and in the history of instruction [43:00] sets that are a lot of like there are a [43:02] lot of ideas some good some less so but [43:07] we should be able to benefit from that [43:08] and risk 5 is is really interesting [43:11] because not only has it learned from all [43:14] of those things and tried to not [43:15] recreate you know and I for every I saw [43:18] they have really done a very good job of [43:20] looking at what they did right and what [43:22] they did wrong and trying to develop [43:23] best-of-breed but at the same time this [43:26] is all freely available there's no [43:27] licensing involved in this whatsoever [43:29] so this allows you to have now these [43:31] soft course so you can have a license [43:34] free soft core as part of an FPGA really [43:38] easily this is not hard at all and you [43:41] can now have your own software running [43:44] on that risk 5 core now is that software [43:48] running with megabytes of megabytes of [43:50] memory no it's running in a pretty [43:51] constrained environment but it allows [43:53] you to actually have computational [43:56] program ability at very very very low [43:59] levels of the stack I in the FPGA so [44:01] that's that's really neat um in a really [44:03] important development [44:05] um and then another thing that's [44:06] happening and again I and personally my [44:08] read is this is a consequence of [44:11] spectrum meltdown is that open firmware [44:13] is finally arriving so there was a time [44:17] when I felt like open from when I was [44:21] never going to happen [44:22] um that is to say we would have a closed [44:25] legacy bias forever but with spectrum [44:29] meltdown and with the new focus on these [44:31] vulnerabilities deep in the stack and [44:33] thanks to the hyper scalars who really [44:35] demand this and are funding it [44:37] accordingly and [44:39] are working with n microprocessor [44:40] vendors to demand open firmware and at [44:42] the same time funding efforts themselves [44:43] we are now seeing the arrival of open [44:45] firmware in fact the the best conference [44:48] I attended I think in the last year is [44:49] the open source firmware conference [44:50] which was just over here at Google and [44:53] Facebook which was terrific and reminded [44:55] me of a conference back in the day where [44:57] you would you know back before the [44:59] internet you as I tell the children when [45:02] you would go to these conferences [45:03] thinking that you were the only person [45:05] in the world interested in what was a [45:07] very kind of narrow slice of humanity [45:08] and you'd walk in and there'd be like [45:10] 300 kindred spirits and it felt very [45:13] empowering and that's what open forum [45:15] where it feels like it's a really [45:17] terrific community and a lot of fun and [45:19] all that is being encouraged by this [45:20] thing the Open Compute Project so OCP is [45:23] interesting developed by Facebook that [45:25] Tioga Pass is an OTP design that I [45:26] showed you the originally spearheaded by [45:30] Facebook with the actual very explicit [45:33] idea of passing these innovations on to [45:38] other folks that were not that are not [45:40] competitors with Facebook so Facebook [45:41] viewed it as honestly a social [45:44] responsibility which feels a little [45:46] strange because Facebook is that your [45:48] facebook I mean it's like becoming that [45:49] subvert of democracy felt social [45:51] responsibility like okay when did that [45:52] disappear um but the but talking to the [45:55] folks that led that effort in inside of [45:57] Facebook 2010-2011 very earnestly [46:00] heartfelt effort to pass on especially [46:03] some ladies efficiency gains around [46:05] things like the bus card base designs [46:06] and so on so that has been um that oh [46:10] the Open Compute Project has been [46:13] growing over the last couple of years [46:15] and you know I you might be if you kind [46:18] of tuned into this in 2010 or 11 or 12 [46:20] or 13 and then you thought well yeah [46:22] that kind of faded um it actually has [46:25] been kind of quietly building steam and [46:28] there's a lot of interesting stuff going [46:30] on it's that is where the hyper scalars [46:33] are really collaborating and if you want [46:35] to know like what's happening inside the [46:37] hyper scalars the Open Compute Project [46:38] is the place to check it out with a [46:40] remarkably candid and transparent about [46:42] the kinds of things to building which is [46:44] great actually OCP summit it assuming [46:46] that the coronavirus doesn't cancel it [46:47] which is like a little worried but the [46:50] OSP summit is coming up [46:52] next week here in San Jose and that's a [46:54] conference that I would really recommend [46:56] attending if this is something that is [46:58] of interest to you because there's [47:00] there's a lot that's going on with OCPD [47:01] that's interesting so a bunch of [47:03] hardware developments that make this [47:05] easier um these are components the these [47:08] are a long long long way from an [47:11] assembled system but they are things [47:13] that didn't exist even five years ago so [47:16] even five years ago this would be much [47:17] harder ten years ago much much harder so [47:19] there is reason for optimism from the [47:22] harbor perspective and then we get to [47:24] the system software perspective on so [47:28] even in system software things have [47:31] changed a lot um and they've changed at [47:34] a couple different port parts of the [47:35] stack first of all because open source [47:38] has now become the expectation for [47:41] infrastructure software um to the point [47:43] that it's kind of created its own [47:45] problems there's now a whole ecosystem [47:47] of startups that are open source [47:49] companies trying to figure out how to [47:51] actually sell a product to somebody and [47:53] they're kind of going through you know [47:55] their own kind of midlife crisis as they [47:56] try to figure out how one actually [47:58] monetizes open sources own problem but [48:01] an artifact of that is that we've got a [48:03] lot of software that is open source and [48:06] they're me it's very easy to kind of [48:08] complain about things that don't work [48:09] but there's actually a lot of stuff it's [48:11] an open source that does work that [48:12] actually is production grade that [48:14] actually is running at data center scale [48:17] and we see that I mean I would look at [48:20] like just the databases if you look at [48:22] what open source databases now versus [48:26] open source databases a decade ago or [48:28] open source consensus systems now versus [48:32] a decade ago [48:33] there is absolutely no comparison a [48:35] decade ago you were running zookeeper [48:36] for consensus and that was it and [48:39] zookeeper is not fun um even if you love [48:44] zookeeper you will acknowledge that it [48:45] is not fun in fact a zookeeper is very [48:47] complicated software to get working [48:49] there are now lots and lots of consensus [48:51] based systems that are out there that [48:52] are open source data for databases there [48:53] are lots and lots of open source [48:54] database systems that that one can go [48:56] build a system on and and that I even in [48:59] the last couple of years though those [49:00] have been developed so that that's all [49:02] that much more stuff kind of in the in [49:04] the tool [49:05] the open operating systems are clearly [49:07] all open source I mean there's they're [49:08] effectively no proprietary operating [49:10] systems left and virtually no [49:12] proprietary databases left hopefully we [49:14] will get vanquished the last of the [49:16] proprietary databases in my lifetime [49:18] um but the so I think the biggest [49:21] surprise and the thing that if you were [49:23] to kind of if I were to go back to you [49:25] know go back a decade to to myself in a [49:28] time machine and I were to tell my [49:30] decade ago self there'll be there'll be [49:32] open source databases okay that makes [49:33] sense and there'll be there will be open [49:36] source consensus based systems that make [49:38] sense [49:38] there will also be a very important [49:40] development in programming languages I [49:42] think my past self would have decked my [49:45] future self I think my past self would [49:46] have tried to take a swing at my future [49:47] self because I'm a systems person I do I [49:51] view programming language have viewed [49:54] programming languages as as part of [49:57] getting a job done very pragmatically [49:59] and yes I went through you know a [50:01] youthful phase of operator overloading [50:03] and so on but you you grow out of that [50:05] like you you and if you're you know [50:07] youth among you who are in that phase [50:09] don't worry it'll pass you'll grow out [50:10] of it um because you'll realize that [50:13] some of the neat things you should you [50:14] can do in languages doesn't mean you [50:16] should do them certainly not in deployed [50:17] systems and especially debugging the [50:19] code of others will help a great deal in [50:22] this regard if you're wondering how I [50:23] can actually help accelerate this [50:26] process go to bug the code of others um [50:28] and you know I talked about das robbing [50:30] me of my childhood I think I say C++ [50:32] robbed me of my young adulthood for much [50:35] the same reasons and I had my love [50:38] affair oc+ boss and then vowed to never [50:40] go back to it so for me and things like [50:43] go are interesting and Java is fine but [50:47] um I don't want garbage collection [50:49] anywhere near the systems that I'm [50:51] developing uh and it's that's not a [50:53] pejorative I mean garbage collection is [50:54] important and I it's important for for [50:57] maybe even most of the world's software [50:59] but for the software that that I write I [51:02] need too much determinism to allow for a [51:06] whistle to be blown that I can't see or [51:08] hear my software to be stopped and for [51:11] something to go scurrying around doing [51:13] something that I can't see with that I [51:15] have no control over [51:16] that's what garbage collection [51:17] effectively is and [51:18] garbage collection the the real problem [51:20] with garbage collection and I don't want [51:21] to I don't want to bag too much of [51:23] garbage collection but the problem is [51:24] garbage collection is it becomes most [51:26] problematic not on sunny days but on [51:29] stormy ones so the when garbage [51:32] collection is a problem it's because [51:34] your actual legitimate heap has grown [51:37] large because you are under a height you [51:40] have a high degree of work you have a [51:41] surging amount of work well garbage [51:43] collection now is especially if I mean [51:46] god forbid you have no garbage god [51:48] forbid that the heap that you have is [51:50] actually been as part of a very [51:52] carefully manicured program whereby when [51:55] you when you are using memory you are [51:57] actually using it because then your [51:58] garbage collector is actually a heap [51:59] scanner and it is a periodic heap [52:02] scanner over which you have no control [52:03] um that's a huge problem and I just long [52:06] since decided that it's like garbage [52:08] collection like free to be you and me [52:08] it's just not for me and it's not gonna [52:11] be with systems that I develop and [52:12] indeed I had resigned myself that like [52:14] it's gonna be basically see for the rest [52:17] of my life and that's fine I you know I [52:18] I I'm a craftsperson [52:20] and um there are there are things about [52:23] seed that I really love I love the fact [52:25] that you are in particular when you [52:27] write see you can feel the assembly [52:30] underneath you see is you know the set [52:32] of see pejoratively that it is portable [52:35] a portable assembly but to me that was [52:37] never preservative that that to me is [52:39] Assistance Program it's like no that's [52:40] what I want I want portable assembly [52:42] effectively because I want to be able to [52:44] control the machine so it was a little [52:47] bit surprising to me that there has been [52:49] a revolution in programming languages [52:52] that is really very much changing the [52:54] way I and a lot of people think and [52:55] that's rust so if you if you haven't [52:58] played with rust or haven't developed in [53:00] it you you need to because um I [53:03] fervently believe that rust is every bit [53:07] as important revolution as see I mean if [53:10] you look at see and you look at the [53:13] systems that came before it [53:14] they they look antediluvian they they [53:18] look prehistoric and then see comes [53:20] along and suddenly it feels modern [53:23] but it ended if see is portable assembly [53:26] the problem with see is that it's and [53:29] it's not even that [53:30] you because I think people make a big [53:33] deal by the fact that C is unsafe which [53:35] is definitely true and C is definitely [53:37] playing with loaded weapons all the time [53:39] that's definitely true the real problem [53:41] with that it's as always as with life [53:43] it's like it's not you it's always the [53:46] other person that's the problem and [53:48] we've see it's not the code that you [53:51] write it's the library that you want to [53:53] go linked to see composes very very [53:56] poorly even in a world where everyone [54:00] understands how to write correct C which [54:02] by the way I have very reluctantly now [54:05] realize after having done a lot of rust [54:07] or decent matter I said enough rust [54:08] anyway um the idea of correct C itself [54:12] has so many asterisks on it because [54:14] there's so much undefined behavior in C [54:16] I you know this was hit home for me when [54:20] I I had a body of software that I wrote [54:25] in with 2000 said that became pretty [54:27] ubiquitous or a lot of machines called [54:29] DTrace allows you to dynamically with [54:31] the system there have been there been [54:33] very few problems with the choice but [54:36] there have been two vulnerabilities in [54:38] DTrace over the in the decade and a half [54:41] since I wrote it um the or two [54:44] vulnerabilities from me person has been [54:45] a third but I think that was Adam one of [54:47] my the people I wrote it with um but the [54:50] two vulnerabilities that I introduced [54:51] were both do to effectively malicious [54:55] integer overflow where I was doing a I [54:58] was comparing a bounds incorrectly which [55:01] is very easy to do and see you're you [55:04] are you are doing a bounds check that is [55:06] correct from the perspective of like [55:09] this the software works but it won't [55:11] work with malicious input and with [55:13] malicious input it can be used to read [55:15] arbitrary memory and with our end and [55:16] you begin to string that together with [55:18] other things and suddenly you have a an [55:20] escalation and you're able that's quite [55:23] privileged so that was a real wake up to [55:25] me rust is not just memory safe its [55:27] integer safe [55:28] so there are rust is very there's a [55:33] bunch of things interesting about rust [55:34] one is that it gives you the the power [55:38] of that composability because of the [55:40] contracts that puts around memory so [55:42] russ has the system of ownership [55:44] and rust the compiler keeps track of who [55:48] owns what and what that actually means [55:50] is that it can statically make [55:52] determinations about whether things can [55:54] be done or not and what what that means [55:56] is that when it's made that [55:58] determination statically it knows that [56:00] this is safe it can now generate a [56:02] binary that doesn't need garbage [56:04] collection that is actually very very [56:06] fast and so the thing that they you know [56:09] when the kind of the really the light [56:11] went on wouldn't even start to rust is [56:13] playing around with it for a small [56:15] system and I as one does if you if you [56:20] haven't waited into rust you definitely [56:22] want to sit down and learn it if you try [56:24] to dabble with it it will punish you [56:27] because the the model is different but [56:30] if you're a C programmer like I am there [56:33] is a level which it feels intuitive you [56:35] would see you could feel the annoying [56:37] assembly where the rust you can feel the [56:38] underlying see you understand why the [56:41] compiler is complaining when if you [56:43] don't understand memory you'll be very [56:45] disoriented by rust because it will be [56:47] complaining and the tears will just be [56:49] streaming down your cheeks these are [56:50] often streaming down your cheeks no [56:51] matter what because there are times when [56:54] rust can be very up in your grill about [56:58] things that can be very legitimate [56:59] issues that are very frustrating rust [57:00] kind of shifts that cognitive load to [57:02] when you're actually developing the [57:03] software um but the thing that was [57:05] interesting to me is I finally kind of I [57:06] went through all the phases of greed [57:08] with rust was finally bargaining with it [57:09] and I just wanted to get this thing to [57:11] compile and I didn't care how I did it [57:12] and I'm like I just need to do this I I [57:14] was trying to do it a clever way I [57:16] decided that I was nowhere near clever [57:18] enough to do something clever and rust [57:19] and I actually needed to do it the [57:21] dumbest way possible just to get the [57:23] same to compile which I did and like [57:25] okay fine now it works now I'm hoping [57:27] that this thing is within 20% of my safe [57:30] I'd written this thing and see and we [57:31] can rerun in rust on and I ran it and it [57:34] was like 35% faster than my safe it's [57:38] like no I and I got to tell you like [57:39] actually it was like not good news I [57:42] felt like I took pride in that see like [57:44] that was well-written see okay so like [57:46] it was not so ignorant you're not [57:47] supposed to be performed that well it's [57:49] kind of like when you get beat by your [57:50] kids and some you know your kids beat [57:52] you in basketball or whatever it's like [57:54] look kid you're not supposed to let dad [57:56] dunk on you it's I mean [57:58] you're proud but also like angry and so [58:01] it was great that the rust is [58:02] outperforming the sea but I was also a [58:04] little bit upset I took it apart why I [58:06] was outperforming it and when looking at [58:08] the memory behavior was much much better [58:09] and I was talking to Steve cloud Nick [58:12] who's in in rust and I'm like Steve you [58:14] know it's amazing that the optimizations [58:15] that you've pulled and clearly that [58:17] compiler knows so much about heaven and [58:19] earth that it is able to truly solve the [58:22] memory disambiguation problem because [58:24] this and seems like no it's it's true [58:26] that that's all possible we haven't done [58:28] any of that yet I'm like oh then wait a [58:30] minute what so actually the RUS compiler [58:33] was not being exceedingly clever what [58:36] was happening is that it's rust itself [58:37] that's clever because I was using the [58:40] only data structure there is no balanced [58:42] binary tree data structure to use and [58:44] rust you always use a b-tree because [58:46] that's the data structure you use well [58:48] beech trees it turns out is actually a [58:50] better data structure than a balanced [58:52] binary tree a beech tree is also gnarly [58:55] to implement and a B tree in C is really [58:59] gnarly to implement and a B tree is to [59:01] implement a b-tree as a library is [59:03] extraordinarily gnarly because a B tree [59:07] is going to be relocating things [59:09] frequently as it rebalances the tree you [59:11] actually you're gonna transplant things [59:13] unlike with the balanced binary tree [59:15] where you're just gonna move pointers [59:16] around you're not gonna do some ooh [59:17] pointers run you're actually gonna [59:18] transit things and the second you do [59:21] that you need to have a real contract [59:22] with whomever is using that library [59:24] about who's gonna do what when so the [59:25] point was like you right there are not [59:27] really good B tree libraries and say I [59:28] mean like take me up on this challenge [59:30] go look for a good B tree library and [59:32] say it is really really really hard to [59:34] implement because the the unsafety of C [59:38] makes hard to express that contract with [59:42] the library user hard to the point of [59:44] being damn near impossible with rust [59:46] this is actually really easy because [59:48] Russ knows who owns what when and you [59:51] are able you are forced to use very [59:53] efficient data structures and by being [59:55] foresees efficient data structures you [59:56] know these artifacts that are [59:57] high-performing okay so it's [59:58] high-performing but okay where can you [60:01] actually use rust the thing is super [60:03] interesting about rust ways to me and [60:04] this is a new in the last year and a [60:07] half is rusts ability to get small [60:11] so you can actually you know on these [60:13] risk five cores on an FPGA you know you [60:17] might have [60:17] we've got 512 K of memory um you know [60:21] and I was actually with another engineer [60:23] of ours who's done a lot of embedded [60:24] rust work and um you know I was joking [60:28] about the famous thing that Bill Gates [60:29] actually never said but is subscribe to [60:31] him of 640k ought to be enough everybody [60:33] um and we kind of joke about that [60:35] because obviously turn 40k is not enough [60:37] for anything it's like well actually no [60:38] no that's not true [60:39] 640 K is actually a lot of memory it [60:42] actually is and you actually can do a [60:44] lot with 640k [60:45] especially if you are careful about the [60:49] way you spend it and rust allows you to [60:52] slim things way way down be a borderline [60:56] beyond what you can do with C with C you [60:59] can get things very slim um with with [61:01] rust you can get things really as slim [61:03] it's really remarkable but you still [61:06] have these very rich abstractions so [61:09] really interesting development and I [61:12] engine development in programming [61:13] languages for those of us that are [61:15] actually building systems and um it's [61:18] the name of the company indeed is is a [61:20] tip of a hat to rust because we we see a [61:22] lot of promise and rust and there are [61:25] some really interesting rust a systems [61:26] like talk talk is a rough space operates [61:28] as actually we're trying to alt here at [61:29] Stanford by Phil Elvis and Levy and Co [61:32] that are are really interesting that [61:34] that we're really fascinated by so you [61:36] kind of add all this up and there's a [61:38] possibility we think that that this is [61:41] something that actually can be built so [61:45] this is what we've done we have in we've [61:49] started the oxide computer company we [61:51] raised seed capital at the end of 2019 [61:54] if you if you want my happy to give my [61:57] war stories on what it's like to raise a [62:01] hardtack seed these days which is a it's [62:08] it's not even a hot and cold experience [62:10] it is both very very frustrating and [62:12] very energizing because when you find [62:14] the right folks [62:15] it can be very energizing but you find a [62:17] lot of them not right folks that's [62:18] that's not less energizing but we were [62:20] able to raise enough money where we give [62:23] ourselves a shot [62:24] that financed enclosed the end of 2019 [62:26] and we're ramping the team now I think [62:28] we've got 15 folks now going for a fully [62:31] functional prototype in 2021 and systems [62:34] themselves in 2022 [62:36] um and we're I mean obligatory if you're [62:40] if there's something it interests you [62:42] we'll definitely check us out we're [62:44] definitely out there and we've got a [62:46] long way to go I mean I want to [62:47] accurately represent where we are we [62:48] really have just gotten started on this [62:50] path and we are it is really only in the [62:54] last honestly couple of weeks that the [62:56] last you know eight weeks or so but even [62:59] then we have already seen that there is [63:03] a lot of promising direction to be had [63:05] so this is a very ambitious problem it's [63:08] very hard there are lots of companies [63:10] that have tried this and have not [63:13] succeeded I'm happy to go through those [63:16] at length if you're curious because I [63:17] feel like I've collected all of those [63:19] the maybe even to a fault actually [63:23] uh-huh when we were doing our raise we I [63:27] really wanted to learn from every single [63:29] person's experience and as a result I [63:32] had an appendix slide on every single [63:35] company you could possibly name that [63:37] looked anything like a computer company [63:39] in the last 15 years and the end the [63:42] slides were we talked about you know the [63:44] company and what happened and so on and [63:45] and then we would be able to go straight [63:48] to those appendix slides so someone [63:49] named a company could immediately go to [63:50] the side and point out and it I think it [63:54] may have gotten a couple because we were [63:57] then kind of viscerally reminding people [63:58] of all of these like dead companies it's [64:00] probably like not a good idea to parade [64:01] a bunch of dead companies in front of an [64:03] investor who was like what all you think [64:05] about is like dead companies it's like [64:06] well no it's important to learn from all [64:07] these it's like this is a good idea to [64:08] invest in or not so we may have overshot [64:11] the mark on that it was a moment where [64:14] an investor mentioned each enter ups [64:16] great because so first person to mention [64:18] degenerate I had an e generous slide [64:20] that it's been a long time preparing so [64:21] I'm like okay uh just go to slide you [64:24] know 102 she pulls up the agenor slide [64:26] oh wow [64:29] God they raised a lot of money oh wow oh [64:32] god that was a zero no no no they're [64:35] still alive in the oven and I'm like [64:36] wait a minute why are we talking [64:37] shouldn't [64:37] talking about companies that have [64:38] written alright so anyway would probably [64:40] overshot the mark on that but [64:42] nonetheless we prevailed we raised and [64:45] and we're going so very exciting the [64:49] other thing the one thing I would kind [64:50] of close on one of the things that we [64:53] wanted to do and then I'll open it up [64:55] for questions and happy to kind of go in [64:56] arbitrary directions for folks but one [64:59] of the things that that we wanted to do [65:01] as part of starting a company I felt we [65:05] feel my co-founders steve talk and [65:07] jasper sell [65:08] we've spent our careers at the hardware [65:10] software interface and i feel that too [65:13] that some of that lore is being lost if [65:17] you will so we wanted to start what we [65:20] thought was a kind of a missing [65:22] conversation and it started a podcast on [65:25] the metal where we have had [65:29] conversations with folks at the hardware [65:30] software interface and actually just had [65:31] a recording today that i'm very excited [65:34] to get out there it was really a [65:35] terrific recording with star simpson if [65:38] you know her really amazing but we have [65:40] had a bunch of really interesting [65:42] conversations and with stories from from [65:48] across the spectrum from across the ages [65:50] to to just recent a cross kind of all [65:53] ages and and and dispositions and so on [65:56] but definitely check that one out so [65:58] check out on the metals is that podcast [66:00] and with that i will gladly take any [66:04] questions yes [66:22] yep [66:23] by HP yeah the this is the second time [66:29] the convex has come up a conversation [66:30] this week actually so they drew the [66:48] daddy computer companies into the [66:50] concrete oh my goodness so those you are [66:53] online or chemists are you here to the [66:55] convex computer company based in Dallas [66:59] I think and they had apparently a [67:05] cemetery for AI well as the competition [67:09] is one thing uh yeah I mean you can [67:10] definitely the competition thing on it [67:12] it's it's more interesting I think to [67:14] keep track of your ancestors than your [67:17] competition because I think they've got [67:19] more to teach you then then you're then [67:22] I mean it's easy to kind of disparage [67:24] your competition we never listen to that [67:25] but the it's the folks that seemed like [67:29] they were the right place to the right [67:30] time and and with the right technology [67:32] and didn't succeed that aren't some ways [67:33] more intriguing [67:45] well let's get the videotape in 1990 [67:54] yeah right we went to MIT a couple [68:03] object is too soon probably yeah right [68:04] right yeah yeah I mean that's a very [68:07] different era - right I mean that you're [68:09] looking at a very very different era [68:11] where you are I mean some of those are [68:13] microprocessor based companies some of [68:14] them aren't your discrete logic you mean [68:17] you got a whole bunch of other things in [68:19] the mix yeah multi I do not have a slide [68:22] on multi flow or convex so yeah that's [68:28] great question the the target market is [68:30] those folks that are running compute [68:32] on-premises so they are and they kind of [68:36] come in to a couple different flavors [68:37] they are the folks that are running on [68:40] premises for good reasons which they are [68:42] there are actually good reasons to run [68:44] on premises and these folks have been [68:46] told that they're basically there are no [68:48] good reasons and everything should be in [68:50] the cloud and if you and you might be [68:52] one of these but if you talk to these [68:54] customers they tend to be pretty [68:56] animated because they're being told they [68:58] don't exist or to the degree they exist [69:00] it's like they exist because they're [69:02] dummies who don't understand the cloud [69:03] it's like no I understand a cloud I'm [69:05] not a dummy [69:06] I've been on the cloud I understand all [69:07] that I've got a regulatory issue or I [69:10] have got an AWS bill that will capsize [69:13] the company I've got a the cogs of cloud [69:16] and if you talked and there are lots of [69:18] different people in lots of different [69:19] domains and the thing that we see and we [69:22] lived this so we we know that this is [69:24] coming is that the clouds expensive I [69:27] mean the cloud is it's very artfully [69:30] priced in that it goes from free [69:33] effectively like I don't care I've got [69:35] cloud credits or like I don't even know [69:37] what the cloud bill is I mean I just [69:39] don't care - oh my god this is crushing [69:41] me seemingly overnight like that there's [69:44] seemingly no time we're like wow okay [69:47] I'm kind of groaning but all right it's [69:49] just like instantly you go from and and [69:52] by the time you really figure out we are [69:54] being debilitated by alas [69:56] compute prices it's very hard to do [69:59] something about it or can't be hard to [70:00] do something no there and there been a [70:01] lot of companies that have been [70:03] vacillating in in part because it is so [70:06] hard right now to go on Prem there are [70:09] companies that like God we're like we're [70:11] being killed but we want to go on Prem [70:14] but we don't have the expertise or this [70:16] is just too much of a mess or we've [70:17] tried we or what we see increasingly as [70:19] we stood up some racks and it was a [70:21] disaster [70:21] um and for all the reasons that we [70:24] understand so they we know that that [70:26] market exists and we in part because [70:29] like not only are they exist but like [70:31] they're ornery [70:32] I mean they've been like you know not [70:35] only are they don't have a roof on their [70:36] head but it's been like raining recently [70:38] like a cold rain and then if they're [70:39] pissed um and actually was anyone here [70:42] talking this because I'm agile is part [70:45] of raising Aranda you know investors [70:46] want to like talk to prospective [70:47] customers and we got to the point where [70:49] we had customers who and we were very [70:53] worried about being very mindful of [70:55] those folks and God blessed them and you [70:58] know who you are out there who are who [71:02] are so excited about this they've been [71:03] willing to go through this journey with [71:04] us and in particular have been willing [71:07] to talk to investors um but we were like [71:09] oh gosh you know I hope or being like [71:10] respectful your time you know know like [71:11] I like bring the VCS to me like I want [71:15] to explain to them that I exist and so [71:18] we have these Cesar been talking our [71:19] customers they almost we're like [71:20] shell-shocked they come back with a [71:22] comes back okay well you're you're right [71:24] you you got some customers that are spun [71:25] up it's like ya know they're angry like [71:27] I hope you didn't um they don't exist [71:28] because they get very very upset if you [71:30] tell them that they don't exist so we [71:32] know that the market is out there we [71:33] know it very much exists and we think it [71:36] is today it's those folks that have been [71:38] left for dead on Prem but tomorrow and [71:42] in the coming decades we believe it'll [71:43] be those folks that are born on the [71:45] cloud and move on Prem primarily for [71:47] economic reasons yes [72:04] yeah so in Intel actually I Intel is [72:07] actually fine I actually don't have a [72:08] problem with until I the Intel Intel's [72:10] only I the only thing I wish Intel had [72:14] decided to become a computer company a [72:16] long time ago instead of respeaking [72:17] microprocessor company but I didn't want [72:18] to compete with its own channel ah [72:19] Microsoft I don't use any Microsoft [72:21] products and haven't fir ever since go [72:23] gates robbed me of my childhood I'd not [72:24] use Microsoft products so Microsoft is [72:26] kind of a non-factor here on but for in [72:28] terms of microprocessors is the right [72:29] good question during the question is [72:30] what about arm what about risk 5 ah risk [72:33] 5 is is is great risk 5 is nowhere near [72:37] ready to run modern heavy duty workloads [72:41] what risk 5 is perfect for is these kind [72:43] of embedded use cases it's great and [72:45] where you need that kind of that soft [72:47] core even a hard core where you want to [72:49] really reason about it but you don't [72:51] need an actually high-performing core at [72:54] the at the boundary of process so if you [72:57] need a 7 enemy I don't think you can get [72:59] a 7 a to me I don't know what makes a 7 [73:00] enemy to risk 5 core but if you um if [73:03] you need to be if you want maximal [73:05] density maximal performance right now it [73:08] needs to be right now it's AMD or Intel [73:10] or or arm and like the graviton 2 is [73:13] really interesting AWS on the ares core [73:15] arm is ion or arm i arm feels very [73:20] lateral from x86 um especially AMD right [73:24] now is I mean I don't know that I buy a [73:26] performance argument and I don't [73:28] normally buy a power organ I only know [73:30] what problem I would be solving with arm [73:31] but that's not I mean no nothing against [73:33] arm and personally and maybe I'm just a [73:36] victim of the last worse I thought the [73:39] ISO wore before and I really don't want [73:41] to fight it again I have long since [73:43] accepted that it's gonna be x86 from a [73:44] nice uh perspective at least for these [73:46] kind of virtualized workloads so for [73:48] Austin's x86 but everything else we [73:51] basically put on the table yes [74:00] yeah so the questions how you how do you [74:02] evaluate whether you want to be a kind [74:04] of one or the other on Prem or on the [74:06] cloud um you you'll you know um if [74:09] you're in this economic use case you [74:11] definitely know and you should be [74:13] everyone to be on the cloud clouds great [74:14] in fact we talked some people rocked oh [74:15] I love the fact that you're like an anti [74:17] cloud play like no no no no not anti [74:19] cloud not anti cloud at all very Pro [74:21] cloud Elastic Compute makes a lot of [74:23] sense for most things anyone starting [74:26] anything should always start on the [74:27] cloud that said you should start on the [74:30] cloud knowing that depending on what [74:33] you're doing you may get to a scale [74:35] where you do need to evaluate going on [74:37] Prem and generally you know generally [74:41] it's when you're getting ready to file [74:42] your s-1 and you are going from a you [74:46] know you are a growth funded SAS company [74:49] and now you actually want to become a [74:51] public company and now actually you care [74:54] about margin and when you start to view [74:56] the cloud as cogs now it's probably time [74:59] to start thinking about being on your [75:01] own physical machine at the scale we're [75:03] talking about there are other reasons to [75:04] be on your own device for kind of you [75:05] know personal more personal reasons but [75:07] at the scale we're talking about it's [75:10] it's a skin problem that's documented [75:13] yeah yeah yes [75:20] yes yeah so questions like it does Russ [75:22] get you the cost of the hardware [75:23] absolutely yeah [75:25] actually in a way that is way more [75:28] coherent than with C because with C [75:32] actually it is really tricky to mix C an [75:36] assembly in in like you've got a C [75:38] program like an operating system has a [75:40] bunch of components that are an assembly [75:41] and intermixing that with C um you're [75:46] kind of in like the compilers in like [75:48] best of luck land and it kind of like [75:50] can do arbitrary things it's very hard [75:53] to kind of intermix those things and [75:54] with the profs to I mean your posture [75:56] also still a little bit best of luck [75:58] land but it is easier to actually [76:00] commingle I have found assembly with [76:03] with the higher-level language in Russ [76:05] ironically but Russ has got all sorts of [76:08] support that actually makes it easier so [76:10] you've got ways of so I mean that's you [76:12] know kind of a silly example but one of [76:14] the things that's important is um you [76:17] have got the ability to indicate that a [76:19] function should have no preamble should [76:22] have basically like I want to be out of [76:24] ABI land in this function it's like why [76:26] would you want to do that it's like well [76:27] trust me they're not lightly right it's [76:31] not something you do it's something you [76:32] do because you you have to do it for a [76:35] so for a body of software that may be [76:37] very constrained about how it operates [76:39] to the point that yeah I can't even save [76:41] a stack pointer I can't even do whatever [76:43] my ABI preamble is I can't safely do [76:46] that in this context and there's a way [76:48] to indicate that in rust like this is [76:50] this should be a effectively a naked [76:51] function and so there are lots of things [76:53] like that and with the rise of of rust [76:56] in embedded software where I think I [76:58] think rust just I think risk 5 will take [77:01] over the earth first with the embedded [77:02] software I think you can make a case for [77:05] Russ to taking over embedded software [77:07] almost first which is kind of crazy cuz [77:09] it's like it's got a lot of energy and [77:11] momentum and other parts of the stack [77:12] but it is so far in a way better than c4 [77:16] that embedded use case and embedded [77:18] software has historically been really [77:20] really hard to write and get right um [77:22] that I think those it's it's an actor I [77:25] really recommend there's a series of [77:27] blog entries learning rust the dangerous [77:29] way [77:30] by gonna move Biffle who actually now [77:33] works at oxide but cliff has a really [77:37] great writing on using rust in these [77:40] embedded contexts and you can see the [77:42] tricks that he pulls and it would the [77:44] tricks that he's able the poll to make [77:46] high-performing rust and rust that's on [77:48] the metal it's really interesting stuff [77:49] yes the C integer overflow oh is it that [77:58] if you you just need to make sure that [77:59] you're writing your bounds check in a [78:01] way that it doesn't actually overflow so [78:03] you need to eat you should be very [78:05] careful when you're doing unsigned [78:06] arithmetic and see in terms of the way [78:08] you actually structure the expression [78:09] and lint won't help you out at all and [78:12] the words in rust you will actually be [78:15] told like you can't do this like this [78:16] overflows that's right that's right um [78:23] sure at some level a bad coding I guess [78:27] but it's like the Minette is really [78:32] really really hard to get right that the [78:33] existing tooling does not help you get [78:35] right like again to me like lint where [78:37] are you if lint does not warn you about [78:39] it I mean certain degrees like okay like [78:42] this is my fault [78:43] but like hey went like this is the [78:45] reason we run lint too and the reality [78:48] is it's really really hard for lint to [78:50] Prato lint would be we would generate [78:52] too many false positives was the reason [78:54] that Linda's warning about that [78:55] conditioner and people would ever use [78:57] them because we've never shut up [78:59] whereas rust is much more constrained [79:01] about about the way you're able to use [79:04] rust is not going to allow you Trent to [79:06] transparently mix signed and unsigned [79:08] types and so on yes [79:20] yeah none of the application no no but [79:24] but where's what come from I know in [79:29] terms of the apps that with that well I [79:30] mean those come from whomever you mean [79:33] there are lots of applications in the [79:34] universe obviously but if you know if [79:35] you've got like a just a you know it's [79:38] an example of a customer that is [79:40] prominently going back on premise uber [79:43] they've talked about this publicly where [79:45] over is is now running on on premises [79:48] and so they've got a bunch of software [79:51] that they've written it that's right yep [79:56] that's right [79:57] yes yeah not running Oracle yeah um so [80:13] yeah so this is where so um it is very [80:19] complicated to make high speed [80:22] interfaces I mean this is not my domain [80:23] of expertise so let's just like let's [80:24] just do the big kind of like a caution [80:26] here but it is um because you can go [80:31] it's a picot inch right you can go an [80:32] inch to a picosecond [80:33] and the different traces that you take [80:36] will have to take different lengths of [80:38] time and at the speeds at which ddr4 [80:42] ddr3 to your and coming to your five [80:44] I'll operate all those different trace [80:46] lengths have totally different [80:48] manifestations in terms of so you have [80:50] to have effectively different timing [80:52] depending on your trace and dim train [80:55] and then environmental conditions about [80:56] other things and dim training is [80:58] effectively the memory controller [81:01] figuring out how to tune itself to get [81:05] from a CPU to memory whatever it is it [81:16] these interfaces are very fast and it's [81:20] so dim training right well in a vacuum [81:27] right so they in cooperative things a [81:29] lot slower but um [81:30] they don't said the DIMM training is [81:33] about figuring out and about and when [81:36] you receipt Adam for example it will [81:38] retrain and that takes time it takes [81:40] time to actually figure out what those [81:43] kind of constants are and that's very [81:44] proprietary kind of stuff I longer than [81:50] that and is yeah this is it you're out [81:52] of my domain of expertise but it is long [81:55] enough that it's not something you want [81:56] to do when you if you do if you can [81:57] avoid it yes so we I mean it will know [82:09] the the the actual the memory controller [82:12] will know what dim is in that slot so it [82:14] knows something about that dim and if [82:15] you change dim if you were receded well [82:19] yeah if you to recede it when when the [82:20] power was off I guess it would net [82:21] wouldn't necessarily know but there are [82:23] certainly ways to get the the dim to [82:26] retrain if you were to if you were to [82:27] change the slot for example that then [82:29] when we trained but this on scenes of an [82:31] area and system moved not an historic we [82:32] had a lot of visibility yes what's not [83:07] easy but yes yeah that's true in the [83:18] questions like what about all these [83:19] other services and there are a bunch of [83:21] services and there was a time when [83:23] people believed they were gonna consume [83:25] any service that any cloud provider put [83:27] in front of them [83:33] right but if you right so they're in I [83:37] mean you can view part of the energy of [83:39] kubernetes is about taking that control [83:41] back from the cloud providers so people [83:43] just don't want to have to have a [83:46] dependency on redshift or what-have-you [83:48] right maybe they actually are so you [83:51] know yes that's true to a degree that's [83:57] actually less true than it was five [83:58] years ago am I in my view in my view [84:00] people are really trying to pare down [84:02] their dependencies to easy to EBS s3 and [84:06] there are a logical go find another [84:07] clouds with the idea being if you give [84:09] me a say if you if you give me those [84:11] services I can hydrate things up the [84:13] stack but I don't want to actually build [84:14] something out that's dependent upon RDS [84:16] or depending upon you know sqs or [84:19] what-have-you [84:19] yeah yeah there are like even Suns got [84:28] their f1 thing right that is you can so [84:31] that they those do exist [84:32] i I we don't use them because we're [84:36] actually just doing an FPGA like the [84:39] old-school way on what it would like a [84:42] programmer and an FPGA it's sitting in [84:43] front of you but those things do exist [84:45] in there they can be useful for certain [84:46] applications it's my understanding yeah [85:01] uh we're not depending on anybody to [85:03] deliver here in terms of these trends so [85:05] the these are things that are already [85:07] out there so like we're definitely I [85:09] mean it's it's easy [85:10] yeah like rust is definitely [85:13] production-grade and and and is is [85:15] usable a lot of the software's is is [85:17] either proximities nor something we will [85:18] extend you know things like blue speck [85:21] are open source and chiselin spinal dl [85:23] are out there the risk five is [85:26] definitely out there you can build on [85:27] risk live today so the the the and then [85:30] meet a certain other these trends that [85:32] we are encouraging obviously we want to [85:33] see and want to collaborate without this [85:36] but I would say there's these are things [85:38] that are all happening kind of [85:39] independent of what we're doing but [85:41] things that we are certainly a part of [85:42] and artisan absolutely yeah yeah all [85:53] right thank you very much