Hetzner Runs Consumer AMD Ryzen Chips in Custom German Data Center

der8auer EN

In-house hardware customization and natural convection cooling allow the German hosting provider to run consumer-grade desktop processors in 24/7 enterprise environments while minimizing energy and electronic waste.

AI-assisted summary of Over 200,000 Servers in One Place! Visiting Hetzner in Falkenstein (Germany) — original source. Check the original for full context.

Source published: 2021-10-13

Key ideas

Hetzner operates around 250,000 active machines at its Falkenstein campus, utilizing 100% green energy and relying almost entirely on direct free cooling and natural building convection rather than energy-intensive active air conditioning. (1:20)
To minimize hardware waste and maximize efficiency, the facility deploys non-standard, custom-built sheet metal server chassis and proprietary racks rather than off-the-shelf 19-inch standards. (6:16)
Production server lines heavily integrate consumer desktop CPUs like AMD Ryzen on custom motherboards (such as the ASUS Pro WS565-ACE) alongside enterprise AMD EPYC platforms. (9:58)
Every server relies on single-fan directed cooling, supported by rigorous in-house validation including 40 °C continuous fan endurance testing and dynamic PSU load testing using an overclocked, water-cooled Threadripper rig. (16:04)

Detailed notes

## Background Published on YouTube by der8auer EN on October 13, 2021, this video documents Roman Hartung's comprehensive tour of Hetzner's massive data center campus in Falkenstein, Germany. The video's description provides supplementary links detailing Hetzner's operations alongside gear lists and channel affiliate promotions.

## TL;DR Roman Hartung tours Hetzner's Falkenstein facility to explore how the host operates approximately 250,000 servers using direct free convection cooling, 100% green energy, and custom in-house hardware engineering. The visit showcases historical data halls housing 14,000 legacy desktop midi-towers preserved to minimize e-waste, modern high-density aisles deploying custom-built AMD EPYC and consumer AMD Ryzen blades, and an automated drive diagnostic section. Hartung also investigates the site's assembly line and hardware validation labs, reviewing single-fan directional cooling shrouds, custom ASUS motherboards, lead-acid backup battery banks, and a liquid-cooled Threadripper test bench used for PSU stress testing.

## Atmosphere & style The presentation combines the technical enthusiasm of high-end PC enthusiast culture with industrial engineering scale, carried by Hartung's accessible, conversational delivery and self-deprecating humor regarding his reputation for breaking hardware. Navigating catwalks that Hartung likens to an action movie set and stepping between icy cold aisles and intense walls of exhaust heat, the video conveys an authentic, transparent look inside a tightly secured enterprise environment. The tone remains informative and appreciative throughout, highlighting the ingenuity of practical, minimalist hardware designs over sterile corporate conventions.

## Bulletin - [01:20](https://www.youtube.com/watch?v=5eo8nz_niiM&t=80s) Hetzner operates around 250,000 active machines at its Falkenstein campus, utilizing 100% green energy and relying almost entirely on direct free cooling and natural building convection rather than energy-intensive active air conditioning. - [06:16](https://www.youtube.com/watch?v=5eo8nz_niiM&t=376s) To minimize hardware waste and maximize efficiency, the facility deploys non-standard, custom-built sheet metal server chassis and proprietary racks rather than off-the-shelf 19-inch standards. - [09:58](https://www.youtube.com/watch?v=5eo8nz_niiM&t=598s) Production server lines heavily integrate consumer desktop CPUs like AMD Ryzen on custom motherboards (such as the ASUS Pro WS565-ACE) alongside enterprise AMD EPYC platforms. - [16:04](https://www.youtube.com/watch?v=5eo8nz_niiM&t=964s) Every server relies on single-fan directed cooling, supported by rigorous in-house validation including 40 °C continuous fan endurance testing and dynamic PSU load testing using an overclocked, water-cooled Threadripper rig.

## Key takeaways - [00:00](https://www.youtube.com/watch?v=5eo8nz_niiM&t=0s) Hetzner's Falkenstein data center campus operates on a massive scale with roughly 250,000 active machines and ongoing construction of a ninth facility. - [01:20](https://www.youtube.com/watch?v=5eo8nz_niiM&t=80s) The provider powers operations using 100% green energy across its locations, including dedicated wind power generation in Finland. - [02:11](https://www.youtube.com/watch?v=5eo8nz_niiM&t=131s) Building 1 preserves legacy operations with approximately 14,000 midi-tower desktop units running older hardware like Nehalem and Ivy Bridge processors to curb electronic waste. - [04:19](https://www.youtube.com/watch?v=5eo8nz_niiM&t=259s) Direct free cooling combined with angled roof architecture utilizes natural thermal convection, limiting active chiller air conditioning to only a few extreme summer days per year. - [06:16](https://www.youtube.com/watch?v=5eo8nz_niiM&t=376s) Modern server halls pack high density, housing approximately 1,400 servers per individual row with sealed acrylic cold aisle containment. - [06:16](https://www.youtube.com/watch?v=5eo8nz_niiM&t=376s) Hetzner departs from standard 19-inch racks, manufacturing its own chassis, racks, and cooling ducts completely in-house out of laser-cut sheet metal. - [08:31](https://www.youtube.com/watch?v=5eo8nz_niiM&t=511s) Decommissioned mechanical hard drives undergo physical destruction and specialized recycling to guarantee residual data security. - [09:58](https://www.youtube.com/watch?v=5eo8nz_niiM&t=598s) Custom ASUS Pro WS565-ACE motherboards running the AMD Pro 565 chipset intentionally restrict overclocking to ensure multi-year silicon reliability and thermal stability. - [09:58](https://www.youtube.com/watch?v=5eo8nz_niiM&t=598s) Proprietary chassis design channels intake airflow across memory, VRMs, and the CPU heatsink using just a single high-static-pressure rear fan. - [12:08](https://www.youtube.com/watch?v=5eo8nz_niiM&t=728s) An automated testing workstation monitors returning hard drives using multi-state diagnostic LED sequences to distinguish functional drives from defective ones. - [13:32](https://www.youtube.com/watch?v=5eo8nz_niiM&t=812s) Facility backup power relies on 120-cell 240 V DC lead-acid battery banks providing roughly 15 minutes of bridge power before emergency diesel generators spin up. - [16:04](https://www.youtube.com/watch?v=5eo8nz_niiM&t=964s) Chassis cooling fans undergo continuous 24/7 accelerated endurance testing in a specialized 40 °C chamber to detect bearing and motor failure before deployment. - [16:04](https://www.youtube.com/watch?v=5eo8nz_niiM&t=964s) Hardware lab validation includes thermal imaging of VRMs, 45 °C environmental chamber testing, and specialized boards with 90-degree rotated CPU sockets to optimize horizontal airflow. - [19:28](https://www.youtube.com/watch?v=5eo8nz_niiM&t=1168s) Engineers independently verify fan throughput specs and stress-test power supplies using an overclocked liquid-cooled AMD Threadripper rig capable of sustained 600W to 700W loads.

## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=5eo8nz_niiM&t=0s) Arrival at Falkenstein and the Scale of the Facility

Roman Hartung (der8auer) opens the tour standing outside in Falkenstein, an area in eastern Germany that he describes as feeling like “the middle of nowhere.” For gamers from Germany who grew up in what he fondly calls the “good old TeamSpeak days,” Falkenstein was a familiar name frequently seen in game server region lists. The trip brings him to Hetzner, a major German data center provider and channel partner. The goal of the visit is to examine the inner workings of an industrial-scale server facility: how servers are manufactured, how massive energy demands are managed amid modern environmental and climate concerns, and how Hetzner deploys consumer-grade hardware like AMD Ryzen desktop processors in long-term enterprise 24/7 workloads.

Surveying the vast complex from the outside, Hartung points out rapid facility expansions, noting that construction is underway on an enormous ninth building on the campus. Power delivery and thermal control at this scale present staggering logistical hurdles when handling hundreds of thousands of active nodes. Despite these demands, Hetzner has built a reputation for strictly tracking resource efficiency and maximizing green energy utilization across its infrastructure.

- **Location:** Falkenstein (Vogtland, Saxony, Germany). - **Facility expansion:** Ongoing construction of the site's ninth large data center building. - **Hardware focus:** Investigating the longevity of consumer CPUs (AMD Ryzen) deployed in enterprise server environments.

---

[01:20](https://www.youtube.com/watch?v=5eo8nz_niiM&t=80s) Perimeter Security and Green Energy Infrastructure

Approaching the secure boundary of the complex, Hartung jokes that the massive reinforced access doors and perimeter topped with barbed wire give the entrance “a slight touch of entering a prison.” Beyond the primary data center park in Falkenstein, Hetzner operates a second German facility in Nuremberg, as well as an international campus in Helsinki, Finland.

Hartner underscores Hetzner's commitment to ecological sustainability, noting that the provider powers its operations with 100% green energy. In the Finnish location, operations are sustained entirely through local wind generation. As he walks up to Building 1—the earliest structure established on the Falkenstein grounds around 2008—he prepares to step onto the floor housing a fraction of the park's roughly 250,000 active machines.

> “And what I find really good about Hetzner is that they're paying attention to use 100% green energy, and especially in Finland, they managed to only use wind energy.” — Roman Hartung

---

[02:11](https://www.youtube.com/watch?v=5eo8nz_niiM&t=131s) Building 1: The Legacy Era of 14,000 Midi-Towers

Entering Building 1 reveals an unexpected sight: long corridors filled entirely with standard consumer midi-tower PC cases stacked on open racks. Hartung explains that Hetzner adheres to a strict hardware retention policy aimed at minimizing electronic waste. Instead of scrapping operational units on arbitrary refresh cycles, systems remain active and rented out at lower tiers as long as customer demand persists and the parts function reliably.

A glance at the inventory labels on the rear I/O shields confirms the consumer pedigree of these legacy machines. One system runs an Intel Core i7-3770 with four 8 GB memory modules on a standard ASUS consumer motherboard; the machine directly adjacent houses a first-generation Nehalem Core i7-920. Hartung finds humor in the internal culture: a Hetzner technician apologized in advance for what they considered “messy” cable management, prompting Roman to remark that it looked meticulously organized compared to the chaotic cabling in his own workshop at home.

```mermaid flowchart TD accTitle: Building 1 Natural Convection Cooling Loop accDescr: Cold outdoor and return air mix at 20 degrees Celsius under the raised floor, flow through server faces, and exhaust out the angled roof.

subgraph Intake["Air Intake & Mixing"] A["Outside Air (~15-16°C)"] --> C["Air Mixing Chamber (~20°C)"] B["Internal Recirculated Warm Air"] --> C end

subgraph ServerFloor["Data Center Floor"] C --> D["Subfloor Channels (~1m Below)"] D --> E["Cold Aisle Floor Grates"] E --> F["Midi-Tower Front Intakes"] F --> G["Server Rear Exhaust (~30°C)"] end

subgraph Exhaust["Natural Draft Exhaust"] G --> H["Angled High Ceiling Roof"] H --> I["Roof Vents (Convection Ejection)"] end ```

> “Their policy is to make sure that they're using energy and resources as well as possible, and in this regard, if the hardware is still working, why would you just throw it away, right?” — Roman Hartung

- **Building 1 launch:** Established circa 2008. - **Form factor:** Approximately 14,000 midi-tower desktop cases. - **Hardware identified:** Intel Core i7-3770 (Ivy Bridge, 4x 8 GB RAM, ASUS consumer motherboard) and Intel Core i7-920 (Nehalem). - **Rack layout:** Opposing rows face one another inward toward a dedicated cold aisle supplied from raised floor channels roughly one meter below.

---

[04:19](https://www.youtube.com/watch?v=5eo8nz_niiM&t=259s) Direct Free Cooling and Natural Convection

Viewing the 14,000 midi towers from an elevated walkway above the rack canopy exposes Hetzner's cooling architecture. Dispelling the common assumption that massive server farms rely constantly on power-hungry active air conditioning, Hartung explains that Hetzner utilizes direct free cooling for almost the entire year. Active AC chillers are reserved exclusively for the rare summer days when ambient outdoor temperatures exceed 30°C.

To maintain the intake aisle at a steady target of 20°C, the facility blends cool outdoor air (around 15°C to 16°C on the day of filming) with warm ambient air recirculated from inside the building. The physical geometry of the building acts as a passive pump: the facility utilizes a steeply angled roof to harness natural thermal convection. Hot air discharged from the backs of the systems rises naturally toward the ceiling peaks, reaching roughly 30°C where Hartung is standing, before venting directly through roof dampers. Standing in the upper crawl spaces peering through the structural framework, he jokes that the high-security catwalks make him feel like Tom Cruise navigating a set in *Mission: Impossible*.

> “That's maybe a misconception that I had: I always thought that data centers with this size, they always have to rely on active AC cooling, but that's not the case; they only use the AC on few days per year.” — Roman Hartung

---

[06:16](https://www.youtube.com/watch?v=5eo8nz_niiM&t=376s) The 2019 Facility: Custom Racks and High-Density Cloud Compute

Moving into a newer data hall completed around 2019 reveals a generational transformation in design. The midi-towers are completely gone, replaced by dense, purpose-built server rows. Entering the hot-aisle exhaust corridor, Hartung is immediately hit by an intense wall of heat discharging from the backs of the machines.

Ascending to the viewing level, the hall initially appears sparser than the crowded legacy room, but Hartung points out that each modern row packs roughly 1,400 servers. Clear acrylic panels seal off the top of the rows to isolate cold intake air supplied through sub-floor floor grates. The servers do not rely solely on building drafts; they use integrated push, pull, or push-pull internal fan shrouds. A significant portion of this floor hosts AMD EPYC enterprise processors with TDP ratings reaching 280 W per socket. An area marked by hundreds of pulsating green LEDs denotes the cloud server deployment—a zone characterized by exceptionally high power density, requiring massive dedicated Uninterruptible Power Supply (UPS) backup infrastructure. Crucially, none of these machines adhere to conventional 19-inch rack standards; Hetzner designs and fabricates its server chassis, racks, and cooling ducts completely in-house to maximize space utilization and operational flexibility.

> “Everything which you can see down there is also not the normal 19-inch rack standard, because they are building their own servers, they're building their own racks, their own cases, their own cooling concept; everything is completely built in-house and developed in-house.” — Roman Hartung

- **Modern row density:** Approximately 1,400 rackmount servers per individual row. - **Enterprise compute:** AMD EPYC platforms dissipating up to 280 W TDP per CPU. - **Aisle containment:** Top-ducted acrylic cold aisle isolation with underfloor intake mesh. - **Infrastructure:** In-house proprietary rack and chassis sheet-metal design rather than commercial off-the-shelf 19-inch standards.

---

[08:31](https://www.youtube.com/watch?v=5eo8nz_niiM&t=511s) Drive Disposal and the Server Assembly Line

Maintaining over a quarter-million servers generates a continuous stream of failed hardware. Hartung stops by a storage area containing pallets of decommissioned mechanical hard drives. Because of strict data privacy requirements, these failed drives cannot merely be discarded or wiped via software; they are collected for mechanical destruction and specialized industrial recycling to guarantee that residual customer data is unrecoverable.

Hartung then transitions into Hetzner's server production and staging facility, where new systems are assembled. Metal shelves and rolling carts are lined with component trays filled with desktop CPUs: Intel Core i9-9900K and Core i7-8700 processors alongside trays of 12-core AMD Ryzen 9 3900 processors destined for Hetzner's AX61 server line. Surrounding the workstation benches are stacks of dense copper server heatsinks, thermal paste, and custom injection-molded or folded plastic air ducts. These ducts, such as models labeled for PX62 line systems running ASUS WS C246 workstations, channel air directly over socket components via high-static-pressure rear pull-fans.

> “I would like to touch things, but I tend to break things, that's why I'm not going to do it.” — Roman Hartung

- **Hardware recycling:** Physical destruction of failed mechanical hard drives before material reclamation. - **CPUs in production:** Trays of Intel Core i7-8700, Core i9-9900K, and AMD Ryzen 9 3900 (used in the Hetzner AX61 server line). - **Cooling methodology:** Passive copper server heatsinks enclosed in dedicated custom fan shrouds (e.g., PX62 shroud for ASUS WS C246 motherboards) operating primarily in a negative-pressure pull configuration.

[09:58](https://www.youtube.com/watch?v=5eo8nz_niiM&t=598s) Custom Motherboards, Minimalist Server Builds, and Single-Fan Cooling

Hetzner deploys custom mainboards built specifically for them by ASUS. Looking at the exact SKU, der8auer spots the Pro WS565-ACE, noting that he had never previously heard of this board or the underlying Pro 565 chipset. The chipset effectively blocks overclocking altogether. As an extreme overclocker, der8auer appreciates the logic behind this restriction in an enterprise environment: Hetzner designs its infrastructure around strict heat dissipation tolerances and cooling profiles. Running a 16-core AMD Ryzen 9 5950X with altered voltages or power levels would dramatically degrade the lifespan of the CPU silicon and VRMs. Hetzner’s priority is ensuring hardware runs reliably over multi-year lifecycles.

Moving to the assembly bench, the server build process mirrors the minimalist hardware philosophy: technicians install only what is strictly necessary. The chassis itself consists of just a few laser-cut, bent sheets of sheet metal manufactured 100% in Germany. The power supply unit is also custom-made with shortened, tailored cable lengths to reduce wasted materials and eliminate unnecessary cable clutter. Aside from these stripped-down elements, assembling the unit resembles putting together a standard consumer desktop PC.

```mermaid flowchart LR accTitle: Server Airflow Architecture accDescr: Directed airflow across server components driven by a single rear exhaust fan.

subgraph Intake ["Chassis Front"] AirIn["Cool Ambient Air Intake"] end

subgraph CoolingPath ["Directed Cooling Channel"] RAM["Memory Sticks"] VRM["VRM Heatsinks"] CPU["CPU Heatsink"] end

subgraph Exhaust ["Rear / Right Side"] Fan["Single High-Static Exhaust Fan"] AirOut["Hot Air Exhaust"] end

AirIn --> RAM AirIn --> VRM AirIn --> CPU RAM --> Fan VRM --> Fan CPU --> Fan Fan --> AirOut ```

The completed server assembly is engineered purely around thermal performance. Unlike a standard consumer tower, the server features a dedicated airflow channel running across the board with an exhaust fan positioned on the rear right. A single exhaust fan draws air through the central CPU heatsink while simultaneously dragging airflow across the side-mounted memory sticks and voltage regulator modules. This directional ducting ensures every heat-critical component stays within thermal limits without requiring multiple fans.

- **Motherboard SKU:** ASUS Pro WS565-ACE, powered by the enterprise AMD Pro 565 chipset. - **CPU Platform:** Configured here with an AMD Ryzen 9 5950X (16 cores, 32 threads). - **Chassis Construction:** 100% made in Germany from basic sheet metal plates. - **PSU Customization:** Shortened, application-specific cable harnesses to simplify management and minimize waste.

---

[12:08](https://www.youtube.com/watch?v=5eo8nz_niiM&t=728s) The Massive Hard Drive Testing Area

Handling tens of thousands of mechanical drives requires an infrastructure to triage customer tickets and hardware anomalies. Hetzner maintains a dedicated HDD testing area to evaluate whether hard drives pulled from service are genuinely defective or still fully functional after a client reports errors. der8auer inspects long test racks containing an immense sea of operational drives undergoing automated diagnostic sweeps.

Above each test bay, status LEDs numbered 1 through 7 communicate drive status via blinking sequences. A slow blink indicates an active diagnostic test is underway; a solid, permanent LED confirms the disk has completed and passed validation; and a rapid, frantic blinking pattern flags a confirmed damaged drive destined for RMA or disposal. Despite the massive scale, der8auer observes that Hetzner’s engineers are constantly attempting to refine the setup. Although the workstation appears meticulously organized to visitors, the staff consider it still somewhat messy, prompting them to 3D-print custom drive brackets that integrate status LEDs directly into the holders for cleaner cable and bay management.

---

[13:32](https://www.youtube.com/watch?v=5eo8nz_niiM&t=812s) The Cold Aisle, Server Racks, and Backup Power Infrastructure

Entering the raised-floor cold aisle viewed earlier from the mezzanine catwalk, der8auer finds himself bordered by roughly 700 servers on the left and 700 on the right. Chilled air is forced up through metallic floor grates, keeping the cold aisle at an ambient temperature of approximately 20 °C. The air passes directly through the server chassis and vents outward into the facility at roughly 28 °C. der8auer reflects on the trust required to film there:

> “And at that point I also want to highlight that's actually a huge honor and I just want to thank Hetzner that we can be here, that they have the trust to put me in here because I'm well known for breaking things and I could just rip apart I don't know things here and damage parts which I obviously don't want to do so yeah, absolutely amazing that they were so open and show everything to us and gave us the opportunity to show everything also to you guys out there.”

Examining the live rack rows reveals several generations of hardware: - **AX101:** The newly assembled workstation-class server built around desktop-grade Ryzen hardware, fitted with side-mounted dual SSDs. - **AX161:** A dedicated enterprise server powered by a single-socket AMD EPYC 7502 processor. - **AX41-NVMe:** The popular entry tier featuring an AMD Ryzen 5 3600 and fast NVMe storage, representing the recent progression from Hetzner’s historic consumer midi-tower deployments toward modular rack blades.

To safeguard against sudden utility grid blackouts, Hetzner relies on dedicated battery rooms filled with individual 2-volt lead-acid cells chained in series. Chaining 120 cells yields 240 V DC, which can be inverted to 230 V AC to match the servers' power supplies. This battery bank holds enough capacity to bridge an entire data center sector for roughly 15 minutes. However, the batteries rarely need to run that long: facility-grade emergency diesel generators spin up within seconds of a detected power interruption, guaranteeing uninterrupted uptime.

---

[16:04](https://www.youtube.com/watch?v=5eo8nz_niiM&t=964s) In-House Fan Endurance and the Hardware Validation Lab

Because each server's thermal survival relies on a single chassis fan, Hetzner subjects its cooling hardware to in-house endurance testing. In a specialized room maintained at a continuous 40 °C—well above typical operating conditions inside the servers—racks of fans run 24/7. Monitoring premature bearing wear and catastrophic motor breakdowns at elevated temperatures allows Hetzner to identify defective batches, replace aging fans proactively before they seize in production, and inform future procurement contracts. Massive industrial ventilation fans beneath the data center sub-floors are responsible for forcing the chilled air volume up through the grates.

der8auer next tours Hetzner's hardware test laboratory, where engineering teams validate new CPU families, motherboard revisions, and memory topologies before wide deployment, as well as test bespoke orders when clients request non-standard configurations in 100-node quantities. Testing stations in the lab include: - Dedicated disk test benches isolating hard drive and SSD write limits. - Thermal validation stations where an infrared thermal imaging camera tracks VRM component temperatures under synthetic stress loads like Prime95. - An ASRock Rack server board utilizing the same locked B550 / Pro 565 platform, but engineered with a 90-degree rotated CPU socket. Rotating the AM4 socket aligns the heatsink fins directly with the horizontal intake-exhaust path, allowing the single fan to cool the heatsink, memory slots, and both banks of VRMs without airflow dead zones. - An environmental thermal chamber equipped with integrated internal heating elements capable of soaking entire operational chassis at 45 °C ambient. While servers throttle under such extreme conditions, the lab tests confirm they will not hard crash if data center air conditioning systems ever suffer a partial failure.

[19:28](https://www.youtube.com/watch?v=5eo8nz_niiM&t=1168s) Component Reliability Testing and PSU Stress Testing with Threadripper

Inside Hetzner's reliability testing area, engineers rigorously validate incoming hardware components to confirm they perform according to manufacturer specifications rather than blindly trusting spec sheets. A prime example is fan testing: the team measures the actual airflow to ensure the advertised air volume matches real-world performance, double-checking whether a fan is truly pushing 50 cubic meters per hour or falling short at something like 25 cubic meters per hour.

```mermaid flowchart TD accTitle: Hetzner Hardware Testing Process accDescr: Workflow showing component validation for fans and power supplies in Hetzner's testing lab.

subgraph LAB["Reliability Testing Lab"] FANS["Fan Airflow Verification (Validates m³/h against spec)"] PSU["PSU Stress Testing"] end

subgraph RIG["Custom Load Rig"] TR["Overclocked AMD Threadripper (Water-Cooled)"] LOAD["Sustained 600W–700W Load"] end

PSU --> TR TR --> LOAD ```

Although Hetzner does not deploy water-cooled machines into production server racks, der8auer discovers an unusual liquid-cooled build sitting on a workbench in the lab. The bespoke rig serves a very specific purpose: torture-testing power supply units. der8auer explains the reasoning with a grin: “Because I mean what else would you use than an AMD Threadripper overclocked that can consume a ton of power and in case you want to test if your PSU can handle 600 or 700 watt load then you can just use your AMD Threadripper system.” With that extreme load-testing setup highlighted, der8auer concludes the tour of the Falkenstein facility, wrapping up the multi-part series exploring Hetzner's data center infrastructure.

- **Airflow verification:** Testing stations measure real-world fan throughput to prevent deploying underperforming cooling fans into production chassis. - **PSU validation rig:** An overclocked, liquid-cooled AMD Threadripper system serves as an adjustable, high-draw dynamic load capable of pulling 600 to 700 watts to ensure power supply stability under heavy stress.

## Mentioned Roman Hartung, der8auer, Hetzner, Falkenstein, Saxony, Nuremberg, Helsinki, Finland, AMD, AMD Ryzen, AMD Ryzen 5 3600, AMD Ryzen 9 3900, AMD Ryzen 9 5950X, AMD EPYC, AMD EPYC 7502, AMD Threadripper, Intel, Intel Core i7-920, Intel Core i7-3770, Intel Core i7-8700, Intel Core i9-9900K, ASUS, ASUS WS C246, ASUS Pro WS565-ACE, ASRock Rack, Prime95, TeamSpeak

## Video description More Info about Hetzner: https://derbauer.hetzner.com/en/image-211013/

--------------------------------------------------------- Support me on Patreon: https://www.patreon.com/der8auer

--------------------------------------------------------- Save 10% on your iFixit purchase: DER8AUER10 https://eustore.ifixit.com/der8auer Pro Tech Toolkit: http://bit.ly/2JOFD8f

--------------------------------------------------------- Find my products at Caseking: Delid Die Mate 2: https://bit.ly/2Rhv4y7 Delid Die Mate X: https://bit.ly/2EYLwwG Skylake-X Direct Die Frame: https://bit.ly/2GW6yyC 9th Gen OC Frame: https://bit.ly/2UVSubi Debug-LED: https://bit.ly/2QVUEt0 Pretested CPUs: https://bit.ly/2Aqhf6y

---------------------------------------------------------

My Equipment: USB- Microscope*: https://amzn.to/2Vi4dky My Camera*: https://amzn.to/2BN4h2O (*Affiliate Links: If you buy something, I will get a part of the profit)

---------------------------------------------------------

Music / Credits: -

Outro: Dylan Sitts feat. HDBeenDope - For The Record (Dylan Sitts Remix)

--------------------------------------------------------- Paid content in this video: - /

Samples: - /

[Source](https://www.youtube.com/watch?v=5eo8nz_niiM)

中文简报

Hetzner在德国定制数据中心运行消费级AMD Ryzen芯片自主硬件定制与自然对流冷却使这家德国主机服务商能够在7×24小时企业级环境中运行消费级桌面处理器，同时最大限度地减少能耗与电子垃圾。 Roman Hartung参观了Hetzner的法尔肯施泰因数据中心，深入探究该主机商如何利用直接自然对流冷却、100%绿色能源以及自主硬件工程来运行约25万台服务器。此次参观展示了容纳1.4万台老旧桌面中塔机箱以减少电子垃圾的历史数据机房、部署定制AMD EPYC及消费级AMD Ryzen刀片服务器的现代化高密度机道，以及自动化硬盘诊断区。Hartung还探访了该站点的装配线与硬件验证实验室，检视了单风扇导流罩、定制华硕主板、铅酸备用电池组，以及用于电源压力测试的液冷Threadripper测试平台。 Hetzner在其法尔肯施泰因园区运营着约25万台处于活动状态的机器，采用100%绿色能源，几乎完全依赖直接自然冷却和建筑自然对流，而非高耗能的主动空调系统。 (1:20) 为尽量减少硬件浪费并提高能效，该设施采用了非标准的定制钣金服务器机箱和专有机架，而非现成的标准19英寸机架。 (6:16) 生产环境服务器产品线在部署企业级AMD EPYC平台的同时，大量结合了运行在定制主板（如华硕Pro WS565-ACE）上的消费级桌面CPU（如AMD Ryzen）。 (9:58) 每台服务器均依赖单风扇定向冷却，并辅以严格的内部验证支持，包括40°C持续风扇耐力测试，以及使用超频水冷Threadripper平台进行的动态电源负载测试。 (16:04)

Reading summary

During a tour of Hetzner's Falkenstein data center campus, Roman Hartung (der8auer) explores how the German provider achieves high-density, low-cost enterprise hosting by blending consumer-grade PC hardware, custom in-house chassis design, and passive cooling principles. Direct Free Cooling and Natural Convection (4:19) Instead of relying constantly on power-hungry active air conditioning, Hetzner's Falkenstein facility utilizes direct free cooling for almost the entire year. Active AC chillers are reserved strictly for rare summer days when outdoor temperatures exceed 30°C. To maintain intake aisles at a target temperature of 20°C, the facility mixes cool outdoor air with recirculated warm internal air. Hardware Retention and Consumer CPU Deployments (2:11) Hetzner adheres to a hardware retention policy aimed at curbing electronic waste, continuing to rent operational legacy systems at lower price tiers as long as customer demand persists. In older sections such as Building 1, established around 2008, roughly 14,000 standard consumer midi-towers house hardware like Intel Core i7-3770 and Core i7-920 processors on off-the-shelf ASUS desktop boards. Across modern lines, consumer-tier desktop CPUs remain central, including trays of Intel Core i9-9900K, AMD Ryzen 9 3900 (AX61 line), Ryzen 5 3600 (AX41-NVMe), and Ryzen 9 5950X (AX101 line) deployed alongside enterprise AMD EPYC platforms. Proprietary Chassis and Minimalist Single-Fan Cooling (6:16) Departing from traditional 19-inch rack standards, Hetzner designs and fabricates its own server chassis, custom racks, and plastic cooling ducts in-house using basic sheet metal produced in Germany. The modern density reaches approximately 1,400 blade servers per individual row. These systems utilize stripped-down builds powered by custom PSUs with shortened cables and bespoke motherboards—such as the ASUS Pro WS565-ACE running a locked Pro 565 chipset that prevents overclocking to protect component longevity. Each server relies on a single high-static exhaust fan and directional shroud to simultaneously pull cool air across the CPU heatsink, memory modules, and VRM circuitry. Rigorous In-House Testing and Failure Management (12:08) Because of strict data privacy requirements, failed mechanical hard drives are physically destroyed and recycled rather than merely wiped. Returned or reported drives are triaged in a high-volume testing station where diagnostic LED patterns flag healthy versus defective units. To safeguard single-fan cooling reliability, Hetzner subjects production chassis fans to 24/7 endurance runs inside a continuous 40°C chamber while measuring actual airflow throughput against manufacturer specification sheets. The validation lab also evaluates memory topologies and VRM thermal limits using rotated AM4 socket boards, thermal imaging, 45°C ambient stress chambers, and a custom liquid-cooled AMD Threadripper rig capable of applying dynamic 600W to 700W loads to stress-test power supply units. Power Continuity and Sustainability Infrastructure (1:20) Hetzner powers its infrastructure with 100% green energy, utilizing local wind power at its Helsinki campus alongside its German operations in Falkenstein and Nuremberg. To protect roughly 250,000 active machines against grid blackouts, the facility maintains dedicated battery rooms housing 2-volt lead-acid cells chained in series to produce 240 V DC. This battery infrastructure can sustain a sector's power load for about 15 minutes, bridging the few seconds required for emergency diesel generators to spin up and assume the continuous electrical load. The Falkenstein tour demonstrates that enterprise-scale operations can diverge substantially from industry-standard off-the-shelf rack hardware, relying instead on custom sheet metal fabrication, consumer desktop silicon, and climate-assisted direct cooling to lower costs and maintain hardware across prolonged operational lifecycles.

精炼阅读摘要

硬件博主 Roman Hartung（der8auer）实地探访了德国数据中心提供商 Hetzner 位于福肯施泰因的园区，展示了该机构如何通过全流程自研定制硬件、直接自然冷却以及严苛的内部老化测试，使消费级组件能够在数十万台服务器的工业级全天候运行中保持高能效与长寿命。自研极简架构与消费级硬件的企业化部署 (9:58) Hetzner 摒弃了传统的标准 19 英寸机架方案，转而在德国本土完全自主设计与制造金属机箱、定制机架以及特定的散热风道。在组装线上，技术人员采用华硕定制的 Pro WS565-ACE 主板（搭载锁死超频功能的 AMD Pro 565 芯片组），搭配 AMD Ryzen 9 5950X 等消费级桌面处理器；主板锁定电压与功率不仅降低了能耗，也避免了供电模组和芯片因超频而加速老化。定制电源配备了剪短的专用线束以减少线缆杂乱并节省物料。为了在保证散热的同时简化系统复杂度，每个机箱仅依赖单颗高风压排气风扇，配合定制导风罩，同时为 CPU 被动铜质散热片、内存条以及 VRM 供电区域抽风散热。直接自然冷却与绿色能源基础设施 (4:19) 在能耗控制方面，Hetzner 园区整体采用 100% 绿色能源供电，其芬兰赫尔辛基站点更完全依靠当地风电。福肯施泰因园区的建筑设计结合了物理对流特性，长年依赖直接自然冷却而非高功耗的主动空调：室外 15℃ 至 16℃ 的冷空气与室内回流暖气在地板下混合至 20℃ 后送入封闭冷通道，穿过服务器排出约 28℃ 至 30℃ 的热空气，最终顺着倾斜的高屋顶自然上升并通过通风阀排出。只有在极少数室外气温超过 30℃ 的夏季日子里，数据中心才会短暂启动主动制冷设备。此外，机房配备由 120 节 2V 铅酸电池串联而成的 240V 直流备用电池组，可支撑大约 15 分钟的电力负荷，足以无缝等待应急柴油发电机在数秒内启动并联。硬件全生命周期管理与报废流程 (2:11) Hetzner 推行以最小化电子垃圾为目标的硬件延寿策略。在 2008 年建成的 1 号建筑内，依然运行着约 14,000 台标准的立式消费级 PC 机箱，其中包含服役多年的第一代 Nehalem 架构酷睿 i7-920 以及 Ivy Bridge 架构酷睿 i7-3770 机器，只要硬件可靠且有客户承租，系统就会降档继续提供服务。面对数十万台设备日常产生的大量报废机械硬盘，Hetzner 建立了专门的测试返修区域，利用带状态指示灯的机架对报错硬盘进行多轮自动化诊断，判定其是良品还是损坏。出于严格的数据隐私法规要求，最终确认损坏退役的机械硬盘不会仅通过软件格式化处理，而是统一集中进行物理机械粉碎销毁后再进行工业回收。高标准内部可靠性与热负荷压力验证 (16:04) 为规避供应商纸面参数的不确定性，Hetzner 设立了专门的硬件测试实验室。由于每台服务器的散热全部依赖单颗排气风扇，团队在 40℃ 的高温测试间对风扇进行 24 小时连续运转测试，以监测轴承磨损和电机故障率，并在实验室使用风量测试台核验风扇的实际排气体积是否达到标称规格。在新型号上线前，工程师会借助红外热成像仪监测 Prime95 满载下主板供电模块的温度，并利用内置加热元件的环境测试舱在 45℃ 极端工况下检验整机是否会发生宕机。此外，实验室还搭建了一套罕见的超频水冷 AMD Threadripper 平台，专门作为能够瞬间抽取 600 至 700 瓦功耗的高动态负载设备，用于极限拷机并验证定制电源的长期稳定性。 Hartung 的实地探访表明，数据中心在控制运营成本与碳足迹时，未必要全盘依赖昂贵的商业通用机架及高功耗制冷方案；但这种高度定制化模式的成功，依赖于企业自身强大的自研工程能力、严格的元器件耐用度检测以及对消费级硬件边界的精确控制。

Detailed notes — zh

## Background 本内容由知名硬件博主 der8auer EN 于 2021 年 10 月 13 日发布在 YouTube 平台。视频简介提供了 Hetzner 相关信息的专题链接，以及作者在 Patreon、iFixit 和 Caseking 上的相关硬件产品及赞助支持渠道。 ## TL;DR 硬件极客 der8auer 深入探访了德国知名服务商 Hetzner 位于福尔肯施泰因（Falkenstein）的大型数据中心园区，该园区容纳了超过 20 万台服务器。视频全方位展示了从 2008 年沿用至今的 14,000 台塔式 PC 机房、2019 年落成的现代化高密机房，到自主组装车间与可靠性测试实验室。Hetzner 展现了其大量采用定制消费级硬件（如 AMD Ryzen 系列处理器）的极简工业哲学，并通过精妙的自然通风混风架构、旋转 90 度 CPU 插座的特规主板以及严苛的 40℃/45℃ 极端高温老化测试，实现了兼顾极致能效与高可靠性的运营闭环。 ## Atmosphere & style 整部视频洋溢着资深硬件发烧友走进工业级殿堂时的纯粹兴奋与探索欲。der8auer 以其标志性的幽默、接地气且专业度极高的解说风格，带着观众在热浪滚滚的排气通道与整齐划一的服务器机架间穿梭，不时穿插对走线细节和“拆解破坏”体质的自我调侃。镜头语言硬核而真实，生动传递出工业极简主义的秩序美感，让人仿佛身临其境地感受到海量算力与澎湃风道交织带来的震撼科技体验。 ## Bulletin - [00:00](https://www.youtube.com/watch?v=5eo8nz_niiM&t=0s) Hetzner 位于德国福尔肯施泰因的数据中心园区体量庞大，运营超 25 万台服务器，且全设施采用 100% 绿色电力，依靠直接混风与自然对流设计在绝大多数时间无需开启高能耗空调制冷。 - [02:11](https://www.youtube.com/watch?v=5eo8nz_niiM&t=131s) 园区 1 号楼至今仍稳定运行约 14,000 台采用旧款桌面处理器（如 i7-3770、初代 i7-920）的标准中塔 PC，体现了其物尽其用、拒绝过早报废硬件的资源利用哲学。 - [06:16](https://www.youtube.com/watch?v=5eo8nz_niiM&t=376s) Hetzner 全面弃用标准 19 英寸机柜，在现代高密机房采用 100% 内部定制的钣金机箱、特规主板（如将 CPU 插座旋转 90 度以优化风道）与定制电源，大幅降低物料与散热成本。 - [15:05](https://www.youtube.com/watch?v=5eo8nz_niiM&t=905s) 针对消费级硬件在商用严苛环境的耐用性，机房设有 40℃ 风扇全天候老化测试、45℃ 高温断空调压力箱以及利用超频 AMD Threadripper 水冷平台对电源进行 600W–700W 极限功耗压测。 ## Key takeaways - [00:00](https://www.youtube.com/watch?v=5eo8nz_niiM&t=0s) Hetzner 德国园区目前拥有 8 栋大楼并正在兴建第 9 栋，全园区依托 100% 绿色电力运行，赫尔辛基节点更是实现全风力发电。 - [02:11](https://www.youtube.com/watch?v=5eo8nz_niiM&t=131s) 1 号机房建于 2008 年，至今仍运行着 14,000 台中塔式消费级 PC，展现了其不盲目报废可用硬件的务实运维哲学。 - [02:11](https://www.youtube.com/watch?v=5eo8nz_niiM&t=131s) 早期机房利用 15–16 °C 室外冷风与室内回风混合成 20 °C 进风，配合斜顶自然对流排热，常年无需开压缩机空调制冷。 - [06:16](https://www.youtube.com/watch?v=5eo8nz_niiM&t=376s) 2019 年建成的现代化高密机房彻底摒弃传统 19 英寸标准机柜，单排通道即可容纳多达 1,400 台服务器。 - [06:16](https://www.youtube.com/watch?v=5eo8nz_niiM&t=376s) 为确保数据安全，全园区损毁报废的机械硬盘均集中在现场进行物理销毁处理，防止数据外泄。 - [08:53](https://www.youtube.com/watch?v=5eo8nz_niiM&t=533s) 组装车间大量采用桌面级 CPU（如 i9-9900K、i7-8700、Ryzen 9 3900），并配备定制的无风扇高密度散热鳍片与抽风导风罩。 - [09:58](https://www.youtube.com/watch?v=5eo8nz_niiM&t=598s) 定制的华硕 Pro WS565-ACE 主板从底层硬件锁死超频功能，以确保在既定散热设计下保障供电模块和处理器的长期运行寿命。 - [09:58](https://www.youtube.com/watch?v=5eo8nz_niiM&t=598s) 服务器机箱采用 100% 德国本土制造的极简钣金设计，全机仅靠单颗高压风扇抽风即可贯穿冷却内存、CPU 鳍片与 VRM 供电。 - [12:08](https://www.youtube.com/watch?v=5eo8nz_niiM&t=728s) 硬盘返厂测试区自制带 LED 指示灯的 3D 打印测试架，通过闪烁频率直观判定硬盘是处于测试中、通过还是需要报废。 - [12:08](https://www.youtube.com/watch?v=5eo8nz_niiM&t=728s) 现代化冷通道送风温度维持在宜人的 20℃，热通道排风温度则攀升至 28℃，形成清晰温差隔断。 - [15:05](https://www.youtube.com/watch?v=5eo8nz_niiM&t=905s) 机房配备 120 节 2V 电池串联输出 240V 直流电的储能机房，提供 15 分钟缓冲供电，无缝衔接柴油应急发电机启动。 - [15:05](https://www.youtube.com/watch?v=5eo8nz_niiM&t=905s) 设立 40℃ 恒温实验室对关键排气风扇进行 24/7 不间断寿命老化考核，防范供应商标称性能虚标或轴承提前损坏。 - [16:46](https://www.youtube.com/watch?v=5eo8nz_niiM&t=1006s) 研发实验室采用永擎定制的 CPU 插座旋转 90 度主板，使鳍片风向与机箱进出风道完全平行，实现热交换效率最大化。 - [16:46](https://www.youtube.com/watch?v=5eo8nz_niiM&t=1006s) 在 45℃ 高温测试箱中模拟机房空调彻底瘫痪的极端工况，验证系统是否满足“高热降频但不死机崩溃”的可靠性底线。 - [19:28](https://www.youtube.com/watch?v=5eo8nz_niiM&t=1168s) 可靠性测试区利用超频的水冷 AMD Threadripper 平台搭建大功耗负载源，严密压测定制电源在 600W 至 700W 连续负载下的稳定性。 ## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=5eo8nz_niiM&t=0s) 造访福尔肯施泰因：Hetzner 数据中心全貌与能耗管理 der8auer 来到了位于德国东部、看似身处“荒郊野外”的小镇福尔肯施泰因（Falkenstein）。对于德国的老玩家来说，这个地名并不陌生，早在当年玩游戏挑选服务器或使用 TeamSpeak 语音联机的年代，很多服务器节点就显示设在此地。本次探访的合作方是知名数据中心服务商 Hetzner。在当下面临气候变化的大背景下，数据中心的能源管理格外受到关注，而这也正是 der8auer 极度好奇的部分。此外，Hetzner 还在服务器中采用了大量消费级硬件（例如 AMD Ryzen 处理器），这让人十分期待去观察：当消费级 CPU 长年累月运行在非原本设计的严苛服务器环境中时，究竟会表现如何。从园区外围望去，整个基地的规模极其庞大。der8auer 现场数了数，发现已有八栋建筑，而眼前正在动工兴建的是第九栋。如此庞大的体量，运行着数十万台服务器，必然伴随着海量的能耗与散热挑战。然而 Hetzner 一向以注重细节和极致能效著称，致力于最大限度地使用绿色能源。走近园区大门时，沉重的大铁门搭配顶部的铁丝网，让 der8auer 开玩笑说“有一种走进监狱的感觉”。除了福尔肯施泰因之外，Hetzner 在德国南部的纽伦堡（Nuremberg）和芬兰的赫尔辛基（Helsinki）也建有数据中心；他们坚持使用 100% 绿色电力，其中芬兰的数据中心更是实现了完全由风力发电供能。 - 园区建筑规模：探访时已建有 8 栋大楼，正在建设第 9 栋大楼。 - 绿色能源策略：全设施采用 100% 绿色电力，芬兰赫尔辛基节点完全依托风力发电。 - 关键探索课题：超大规模数据中心的风道散热架构、UPS 及供电保障、消费级硬件在商用服务器中的长期耐用度。 --- ### [02:11](https://www.youtube.com/watch?v=5eo8nz_niiM&t=131s) 1号机房与老当益壮的 14,000 台塔式 PC der8auer 来到了整个园区的 1 号楼（Building 1）。这是 Hetzner 在 2008 年左右建立的第一座机房。一推开门，眼前呈现出完全不同于传统现代机房的震撼景象：机房里整整齐齐码放着多达 14,000 台标准的立式中塔（Midi Tower）台式机外壳。之所以保留如此庞大规模的传统塔式 PC，是因为 Hetzner 的运营哲学是：只要硬件还能稳定工作，就绝不轻易丢弃，必须最大限度地榨取资源和设备的价值。 ```mermaid flowchart TD accTitle: 机房1中塔服务器自然对流散热架构 accDescr: 室外冷空气与室内空气混合后送入底部风道，吹过服务器前脸，热空气经斜面屋顶排出。 subgraph AirMix["空气混合阶段"] OutsideAir["室外自然冷风 (~15-16 °C)"] --> Mixer["通风风道混风处理"] IndoorAir["室内回流暖风"] --> Mixer end subgraph ServerZone["服务器机架区域"] Mixer -->|"提供约 20 °C 进风"| FloorVent["地下 1 米通风通道 (地板网孔出风)"] FloorVent --> ColdAisle["中塔机箱前脸对向冷通道"] ColdAisle --> MidTowers["14,000 台中塔服务器 (前脸进风 / 后方排热)"] end subgraph Exhaust["热气排出阶段"] MidTowers --> WarmCeiling["顶部高温滞留区 (自然对流聚热 ~30 °C)"] WarmCeiling --> AngledRoof["斜顶建筑专用排风口 (自然对流排至室外)"] end ``` 在这些老机器的背面可以看到标注规格的标签。der8auer 随手扫过几台机器，赫然看到了 Intel 酷睿 i7-3770、四根 8GB 内存与华硕消费级主板的组合；再往旁边看一台，甚至还有更古老的初代酷睿 i7-920。Hetzner 的技术人员在一旁打趣提醒他“不要介意这里杂乱的线缆管理”，但 der8auer 看着规整的走线连连赞叹，直呼“这比我家里整洁多了”。这一区域的冷却设计同样十分精妙： - 机房下方约一米处铺设了通风管道，两排塔式机箱前脸相对放置，冷风从底部的风道垂直送出，直接供机箱前面板吸入。 - 机箱内部硬件将空气加热后排至通道外侧。 - 冷却过程依靠直接混风技术：机房并不常开高耗能的压缩机空调（AC），而是将当时约 15–16 °C 的室外冷空气与室内暖风进行混合，精确调配出约 20 °C 的进气温度。 - 考虑到德国当地气候并不极端，一年中气温远超 30 °C 的炎热天气屈指可数，绝大多数时间完全无需依赖主动式空调制冷。 - 1 号楼采用了坡度明显的斜顶（angled roof）设计，利用热空气自然上升的对流原理，将积聚在建筑顶部约 30 °C 的热风直接通过屋顶排风口排出。der8auer 站在二层俯瞰时能明显感受到巨大的温差，打趣说这高耸严密的机房风道让人联想到《碟中谍》，仿佛汤姆·克鲁斯才会去攀爬。 --- ### [06:16](https://www.youtube.com/watch?v=5eo8nz_niiM&t=376s) 2019现代化高密机房：自研机架与全栈定制告别了 2008 年的塔式机房后，der8auer 步入了一栋约于 2019 年落成的现代化数据中心大楼。这里的温控环境截然不同，刚踏入排热通道区域，滚滚热浪便扑面而来。这一区域彻底告别了传统塔式机箱，转而采用专门定制的高密度刀片与机架设计。从上层俯瞰，新机房在视觉上似乎显得空旷一些，但这完全是错觉。机房里的单排通道就容纳了多达 1,400 台服务器。机架顶部采用亚克力板进行密封隔断，地板铺满金属网格以输送冷风。冷风从机架间底部垂直送出，穿透服务器核心元件后再排向外围通道。为了压制诸如 AMD EPYC（霄龙）等 TDP 高达 280W 的高功耗处理器，服务器机柜采用了风扇的 Pull（抽风）、Push（送风）或 Push-Pull 混合风道配置。机房下方绿光闪烁的区域属于 Hetzner 的云计算（Cloud）核心区，具备极高的功率密度。这种高密度硬件对电力冗余和不间断电源（UPS）的保障能力提出了严苛要求，也是设备排列留有特定间隔的原因。特别值得一提的是，机房内所见的一切均打破了常规标准的束缚： - 这里彻底摈弃了传统标准的 19 英寸机柜。 - 无论是机柜架构、服务器机箱、风道散热风道还是内部组件布局，全部由 Hetzner 自主研发并在内部自制，赋予了整体架构极高的灵活性与成本控制能力。 - 面对全园区超过 250,000 台服务器的庞大基数，每天都会产生自然损毁的机械硬盘（HDD）。所有报废硬盘均集中收集，并在现场进行专业物理销毁和回收，彻底杜绝数据泄露的隐患。 --- ### [08:53](https://www.youtube.com/watch?v=5eo8nz_niiM&t=533s) 服务器生产组装车间与核心配件承载 25 万台服务器的运转，背后必然有强大的本土制造能力支撑。der8auer 随后走进了 Hetzner 的服务器生产车间。工作台上整齐码放的托盘装 CPU 极为震撼： - 托盘中摆满了诸如 Intel 酷睿 i9-9900K、非 K 版的 i7-8700，以及 AMD Ryzen 9 3900 等桌面级处理器。 - 其中 i7-8700 便是为 Hetzner 旗下 AX61 这一系列服务器所配备的硬件。 - 面对堆成小山般的顶级 CPU，der8auer 忍不住开玩笑说自己虽然很想伸手摸摸看，但他平时有“碰啥啥坏”的体质，只得作罢。在装配工作区，随处可见大批定制的无风扇纯铜/铝鳍片高密度服务器散热器与工业导热硅脂。结合配套的特定风道罩（例如适配华硕 WS C246 主板的 PX62 定制导风罩），整套系统的冷却主要依靠机架后端大尺寸风扇强力抽风（Pull 架构），将气流精确导流过 CPU 散热片，展示出 Hetzner 高度垂直整合的工程研发水准。谈及淘汰与能效标准，der8auer 高度赞扬了 Hetzner 拒绝盲目报废可用设备的务实理念：“Their policy is to make sure that they're using energy and resources as well as possible and in this regard if the hardware is still working why would you just throw it away right?”（他们的原则是尽可能合理地利用能源与资源，从这个角度来说，如果硬件还能工作，你为什么要把它们扔掉呢？）随后，der8auer 便动身前往具体组装工位，准备近距离观摩这些专用服务器的实际生产流程。 ### [09:58](https://www.youtube.com/watch?v=5eo8nz_niiM&t=598s) 深度定制的主板与极简主义服务器装配 der8auer 仔细观察了 Hetzner 机房内使用的定制硬件。令人惊讶的是，Hetzner 拥有专门找华硕（ASUS）深度定制的主板，具体型号标识为 Pro WS565-ACE。作为一个资深的硬件超频玩家，der8auer 坦言自己此前从未听说过这款型号，也完全没听过主板上搭载的“Pro 565”芯片组。该芯片组在硬件底层直接禁用了超频功能，正如 der8auer 所指出的：“It's not like it's banning overclocking, but it just doesn't allow it, which is essentially the same, but it also makes sense that in the server you're not allowed to overclock the CPU.”在服务器环境中禁止超频完全合情合理，因为整套系统是基于特定的散热设计与散热功耗（TDP）构建的。以机型搭载的 AMD Ryzen 9 5950X 为例，如果客户随意修改 CPU 的功耗与供电电压，将严重影响供电模块（VRM）乃至处理器本身的硬件寿命，而 Hetzner 必须确保设备能在极长的生命周期内稳定可靠地运行。 ```mermaid flowchart LR accTitle: 单风扇风道散热示意图 accDescr: 展示冷空气如何依次经过内存、CPU散热片与VRM供电并由单风扇抽风排出的气流路径 subgraph INTAKE["冷区进风"] A["机箱进风口"] end subgraph COOLING_PATH["机箱内部散热风道"] B["内存条 (RAM)"] C["CPU 散热片"] D["主板 VRM 供电模块"] end subgraph EXHAUST["抽风排出"] E["顶部导流罩"] F["单颗高风压排气风扇"] end A --> B B --> C C --> D D --> E E --> F ``` 走到装配区，der8auer 发现整机组装流程与所选配件一样遵循极致的极简主义（Minimalistic）哲学——只保留绝对必需的组件。这里的机箱仅由几片钣金折弯组合而成，并且 100% 德国本土制造（Made in Germany）。电源同样为深度定制件，采用了大幅缩短的定制线材，既不浪费多余资源，又极大简化了装配中的理线难度。组装好的服务器整机结构几乎完全以散热为核心导向：其风道设计与常规桌面 PC 截然不同，排气口设在右后方，上方加装了专用的导流风道罩。整个系统仅依靠单颗强力排风风扇，就能在抽风的同时顺带抽走经过内存条、CPU 散热鳍片以及 VRM 供电区域的热量，用极低的物料成本达成了极高的散热效率。 - **定制主板型号**：ASUS Pro WS565-ACE，搭载特规 Pro 565 芯片组，底层锁死超频以延长使用寿命。 - **处理器配置**：展示机型装配了 16 核 32 线程的 AMD Ryzen 9 5950X。 - **机箱与电源**：100% 德国制造的超轻量化钣金外壳；搭配缩短线材的定制电源，节省内部空间。 - **极简散热方案**：整机仅使用 1 颗专用排气风扇，结合定制风道导流罩通盘冷却内存、CPU 和 VRM 供电。 --- ### [12:08](https://www.youtube.com/watch?v=5eo8nz_niiM&t=728s) 规模庞大的硬盘返厂测试区与冷通道机房走出装配区后，der8auer 来到了占地巨大的硬盘（HDD）测试检测区。此前在仓库见到的海量硬盘，都需要在此处进行严格的健康状态甄别。当租用服务器的客户报告硬盘可能存在异常时，Hetzner 必须通过测试平台验证盘体是否真的损坏。测试台架上排列着数不清的硬盘，顶部配有编号从 1 到 7 的 LED 状态指示灯：慢速闪烁代表硬盘正处于读写测试中；常亮则表明通过检测；如果出现急促的快速闪烁，则意味着该硬盘存在硬件损坏，必须立即做报废或更换处理。让 der8auer 感触颇深的是 Hetzner 对整洁度的极致苛求。尽管这片测试区在 der8auer 看来已经极其井井有条，但 Hetzner 的工程师仍觉得这里“还不够整洁”。为了进一步优化管理，他们开始利用 3D 打印技术自制专属硬盘测试支架，将 LED 状态指示灯直接内嵌集成在打印支架上。随后摄制组回到了此前俯瞰过的高密度机房区域。这里是典型的“冷通道”（Cold Aisle）设计，通道两侧各部署了约 700 台服务器（单通道共计约 1400 台）。冷空气直接从地板下的网格孔中垂直向上送入，穿透服务器后将热量排出。通道内部温度保持在非常宜人的 20℃ 左右，而一旦走到外部热通道，环境温度立刻攀升至 28℃。站在高密度的服务器列间，der8auer 难掩兴奋与感激之情，他开玩笑说自己在硬件圈是出了名喜欢“拆东西、搞破坏”的，非常感谢 Hetzner 能给予他如此大的信任，允许他近距离拍摄机房核心腹地并将这些罕见的工业细节公之于众。 - **硬盘检测逻辑**：LED 慢闪代表正在测试，常亮代表测试通过，急促快闪代表已损坏需报废更换。 - **精益求精的 3D 打印**：内部工程师自行 3D 打印内嵌 LED 的专用硬盘测试座，持续消除工位杂乱感。 - **冷通道实测温度**：地板网格送风的冷通道内约 20℃，外围排热区域温度约 28℃。 - **现场展示机型列表**： - **AX101**：搭载 Ryzen 9 5950X 与侧置双 SSD。 - **AX161**：搭载单路 AMD EPYC 7502 处理器。 - **AX41 NVMe**：视频赞助常客，搭载 AMD Ryzen 5 3600。相较早期采用的传统塔式机箱，这些机架产品代表了其硬件密度的最新迭代。 --- ### [15:05](https://www.youtube.com/watch?v=5eo8nz_niiM&t=905s) 不断电保障：电池备用房与严苛的 40℃ 风扇老化测试现代数据中心最害怕的事故之一便是电网断电。为了确保服务器永不宕机，Hetzner 配备了专门的大型电池蓄能机房。整个房间排列着大量的储能单体电池，每个电芯电压为 2V。通过将 120 块电池串联，可以输出 240V 的直流高压，随后通过逆变器转换为交流电并入机房配电母线。单个蓄电池机房所储备的电能，足以支撑一整片数据中心区域在满载下平稳运行 15 分钟。但在实际生产环境中，这 15 分钟的电池缓冲几乎永远用不满，因为外部电网一旦中断，机房配备的重型柴油应急发电机组就会在数秒钟内全自动点火并网供电，电池仅用来无缝填补这数秒的空档。 ```mermaid flowchart TD accTitle: 数据中心应急供电切换流程 accDescr: 展示主电网断电后由蓄电池机房与柴油发电机协同保障供电的逻辑 A["外部高压市电正常供电"] -->|突发电网断电故障| B["2V 单体串联蓄电池组瞬间接管"] B --> C["输出 240V 直流并逆变，提供最长 15 分钟缓冲"] B -.->|数秒之内自动响应| D["重型柴油发电机点火启动"] D -->|发电机达到额定工况| E["接管数据中心主供电回路 (服务器持续在线)"] ``` 由于整台定制服务器的散热完全寄托在尾部那唯一一颗排气风扇上，风扇本身的耐久度便成了关乎服务器生死存亡的命脉。为此，Hetzner 设立了专门的风扇可靠性内部实验室。实验室内常年维持在约 40℃ 的环境温度——这略高于服务器机箱内的日常运行温度，为风扇制造了更严酷的热应力环境。测试风扇在此处接受 24 小时全天候的不间断寿命老化考核。一旦发现某个批次或型号的风扇出现故障苗头，团队就能在机房批量暴雷前提前预防性更换，同时这些耐久数据也会直接指导公司未来的风扇采购招投标决策。离开风扇测试区时，der8auer 还展示了一颗用于机房地板下强制送风的巨型工业离心轴流风扇，其体积极其庞大，全速旋转时的噪音与气流十分惊人。 - **蓄电池参数**：单体电池 2V，120 节串联获得 240V 直流电，满载后备支撑时间为 15 分钟。 - **后备电源冗余**：柴油发电机组在数秒内即可自启动接管，电池主要用于平滑切换过程中的瞬时断电。 - **40℃ 烤机测试**：风扇测试区环境温度常年保持 40℃，24/7 满负荷运转，用于提前排查轴承与电机隐患。 --- ### [16:46](https://www.youtube.com/watch?v=5eo8nz_niiM&t=1006s) 研发测试实验室：旋转 90 度 CPU 插座与极端热应力验证随后，der8auer 探访了 Hetzner 的研发与验证实验室（Test Lab）。每当市场上出现全新一代 CPU，或者有大客户提出采购 100 台非标定制配置的特殊需求时，工程师都必须在实验室中对主板、内存及整机散热进行严苛的全面验证。实验室内部井然有序地划分了多个功能测试台：左侧专用于 HDD 和 SSD 的兼容性与稳定性跑盘；右侧则有一套正在执行 Prime95 满载烤机热应力测试的平台。在烤机平台上方架设着专业红外热成像仪，正实时监控主板 VRM 供电区域的高温发热点，严防供电元器件过热损坏。在测试架上，der8auer 发现了一块极其罕见的特规主板。该主板乍看之下采用传统的绿色服务器 PCB，基于 AMD B550 架构与 Pro 565 芯片组，但最精妙的设计在于其 CPU 插座被旋转了 90 度。这块由永擎（ASRock Rack）定制的主板打破了 der8auer 对“旋转插座只有 EVGA 会做”的刻板印象： “The special feature is that you can see it's a rotated socket, so EVGA is not the only mainboard company which is able to rotate the socket... and the reason why the socket is rotated, it's just a pure cooling aspect.” 正是因为将插座顺时针旋转了 90 度，散热鳍片的纵向走向与机箱前进后出的风道完全平行。冷空气从右侧进入机箱后，能够毫无阻碍地掠过整颗散热鳍片的全长，同时顺流吹拂主板两侧的 VRM 供电散热片与侧向插放的内存条，在极简风道下达成了顶级的热交换效率。实验室的另一大核心设备是专用的高温环境测试箱（Thermal Testing Chamber）。箱体内部内置了强力电加热元件，能够将环境温度人工拉升至 45℃。尽管数据中心机房内绝不可能达到如此高温，但 Hetzner 需要模拟机房空调系统（AC）彻底瘫痪的最极端工况。测试的核心诉求是：在 45℃ 极热环境下，服务器的 CPU 虽允许触发过热降频（Thermal Throttling），但整机绝对不能死机或崩溃。此外，测试箱旁还展示了专门用于评估风扇性能的 3D 打印测试治具，处处体现出工业定制与自主工程研发的深厚底蕴。 - **特殊硬件方案**：ASRock Rack 打造的 AMD B550 / Pro 565 特规主板，CPU 插座旋转 90 度以完美匹配单向抽风风道。 - **红外热成像监控**：利用热成像仪在 Prime95 高压测试下实时追踪主板供电 MOS 管温度。 - **45℃ 极热耐受测试**：在高温环境箱中模拟空调宕机故障，确保服务器在极端热负荷下“降频而不宕机”。 ### [19:28](https://www.youtube.com/watch?v=5eo8nz_niiM&t=1168s) 硬件可靠性测试区与电源极限压力测试在赫兹纳（Hetzner）位于福肯斯坦（Falkenstein）的数据中心内部，团队设有一个专门的可靠性测试区域（Reliability testing area）。这里会对各类硬件组件进行严格的实测与质检。例如，工作人员会亲自校验风扇的实际性能，验证其标称的风量规格是否属实。der8auer 解释道，运维团队必须反复核查风扇的风量表现，“因为他们必须反复核验，看看风量到底是标称的每小时 50 立方米，还是实际上可能只有 25 立方米”。这种严格把关确保了机房散热系统在长年高负荷运转下不会出现因为虚标而导致的风道散热不足。尽管 Hetzner 旗下投入运营的服务器通常不采用水冷散热系统，但 der8auer 在测试区意外发现了一台特殊的水冷平台。这台机器专门用于严苛测试电源供应器（PSU）。测试平台采用了一颗经过超频的 AMD Threadripper（线程撕裂者）处理器，能够在高负荷下榨取海量功耗。der8auer 幽默地指出，如果需要测试一款电源能否稳定承受 600 瓦甚至 700 瓦的极限负载，直接拉出一台超频的 AMD Threadripper 水冷平台来进行压测是再合适不过的选择。在视频的尾声，der8auer 向观众作了总结，宣布 Hetzner 探访系列视频圆满结束，并感谢观众的收看与支持。 - **风扇风量验证**：测试区会对风扇风量等关键参数进行严密复测，防止供应商虚标导致散热隐患。 - **PSU 压力测试平台**：利用超频的水冷 AMD Threadripper 系统作为极高负载源，专门检验待上架电源在 600W 至 700W 连续大功耗下的稳定性与耐用度。 ## Notable quotes > [09:58](https://www.youtube.com/watch?v=5eo8nz_niiM&t=598s) It's not like it's banning overclocking, but it just doesn't allow it, which is essentially the same, but it also makes sense that in the server you're not allowed to overclock the CPU. — der8auer > [08:53](https://www.youtube.com/watch?v=5eo8nz_niiM&t=533s) Their policy is to make sure that they're using energy and resources as well as possible and in this regard if the hardware is still working why would you just throw it away right? — der8auer ## Mentioned Hetzner, der8auer, AMD, Intel, ASUS, ASRock Rack, EVGA, TeamSpeak, AMD Ryzen, AMD Ryzen 9 5950X, AMD Ryzen 9 3900, AMD Ryzen 5 3600, AMD EPYC, AMD EPYC 7502, AMD Threadripper, Intel Core i7-3770, Intel Core i7-920, Intel Core i9-9900K, Intel Core i7-8700, ASUS Pro WS565-ACE, ASUS WS C246, Pro 565, AMD B550, Prime95 ## Video description More Info about Hetzner: https://derbauer.hetzner.com/en/image-211013/ --------------------------------------------------------- Support me on Patreon: https://www.patreon.com/der8auer --------------------------------------------------------- Save 10% on your iFixit purchase: DER8AUER10 https://eustore.ifixit.com/der8auer Pro Tech Toolkit: http://bit.ly/2JOFD8f --------------------------------------------------------- Find my products at Caseking: Delid Die Mate 2: https://bit.ly/2Rhv4y7 Delid Die Mate X: https://bit.ly/2EYLwwG Skylake-X Direct Die Frame: https://bit.ly/2GW6yyC 9th Gen OC Frame: https://bit.ly/2UVSubi Debug-LED: https://bit.ly/2QVUEt0 Pretested CPUs: https://bit.ly/2Aqhf6y --------------------------------------------------------- My Equipment: USB- Microscope*: https://amzn.to/2Vi4dky My Camera*: https://amzn.to/2BN4h2O (*Affiliate Links: If you buy something, I will get a part of the profit) --------------------------------------------------------- Music / Credits: - Outro: Dylan Sitts feat. HDBeenDope - For The Record (Dylan Sitts Remix) --------------------------------------------------------- Paid content in this video: - / Samples: - / [Source](https://www.youtube.com/watch?v=5eo8nz_niiM)

Source quotes

[9:58] der8auer: It's not like it's banning overclocking, but it just doesn't allow it, which is essentially the same, but it also makes sense that in the server you're not allowed to overclock the CPU. [8:53] der8auer: Their policy is to make sure that they're using energy and resources as well as possible and in this regard if the hardware is still working why would you just throw it away right?

Original transcript

[0:00] [Music] [0:05] hi and welcome back to a new video you [0:06] can see we are in the middle of nowhere [0:09] it's called falkenstein and i'm not sure [0:11] if you're familiar with this but at [0:12] least for me being from germany and like [0:15] the good old teamspeak days and when you [0:18] were still picking servers for your [0:20] games sometimes some game servers were [0:22] showing the location called falkenstein [0:25] and that is in the eastern part of [0:27] germany and we're here we are at our [0:29] corporation partner hetzner which is a [0:31] data center provider and i'm yeah very [0:34] curious to see [0:36] how everything here works out we will [0:38] get a tour today to see how they're [0:40] building servers how everything is set [0:42] up in regards of like energy management [0:44] which is also very important nowadays [0:46] considering the climate change and [0:48] everything they're also using some uh [0:51] consumer hardware like amd ryzen cpus in [0:54] servers which will also hopefully give [0:56] us some very interesting insights in [0:58] regard of what happens if you use amd [1:00] ryzen cpus for a very long time for a [1:03] use where they're maybe not intended to [1:05] so all of that could be interesting and [1:07] i also noticed that they're just [1:08] building a new entire building [1:10] from what i can count it's like one two [1:12] three four five six seven eight and this [1:15] is like the ninth building so it's [1:16] pretty massive all right let's just go [1:18] inside and check how things look what i [1:20] was actually most interested in is to [1:22] see how they manage cooling and also [1:24] power supply of such a huge facility [1:26] because obviously if they have several [1:29] thousand pcs or servers running it's a [1:31] very high energy and power supply [1:33] required which also takes up a lot of [1:35] resources but hetzner is known to pay [1:38] attention to those details and also try [1:40] to use as much screen energy as possible [1:44] has a slight touch of entering a prison [1:46] with this massive door and [1:48] like the barbed wire on top [1:51] apart from the data center in [1:52] falkenstein which we are visiting today [1:54] hudson has the second one in nuremberg [1:56] in the south of germany and also one [1:59] more in helsinki in finland and what i [2:01] find really good about hetzer is that [2:02] they're paying attention to use 100 [2:05] green energy and especially in finland [2:08] they managed to only use wind energy so [2:11] we are standing in front of the building [2:13] number one if you remember from the [2:15] intro that is the building like on the [2:17] total left side and that is the first [2:19] building they established around 2008 [2:22] and we will now enter the building and [2:24] check out probably one of the servers [2:26] out of 250 000 servers [2:30] that's quite a lot so as i just pointed [2:32] out this is the first data center which [2:34] was established around 2008 and [2:38] if you just look at this this is all [2:39] midi towers but that is mainly because [2:41] headset just wants to keep them running [2:43] as low as they're working as long as [2:45] they're working [2:46] their policy is to make sure that [2:48] they're using energy and resources [2:50] as well as possible and [2:52] in this regard if the hardware is still [2:54] working why would you just throw it away [2:57] right if you still have customers that [2:58] can use them they can just keep using [3:01] these systems and it's also quite [3:02] entertaining if you just look at the [3:04] systems they have labels on the back for [3:06] example if you just look at this label [3:08] it points out i7 377t [3:12] uh using four times eight gigabyte of [3:14] memory and like an asus motherboard [3:16] which is also just a consumer grade [3:18] and if we just move one pc to the right [3:20] then we can find an i7 and 920 which is [3:23] also quite old already so that is very [3:25] very interesting so the cooling concept [3:28] is also quite interesting [3:29] because it's actually underneath me uh [3:32] like one meter below me they have some [3:34] air ventilation channels and they go to [3:37] the site and those pcs are like yeah you [3:40] always have two rows and the fronts are [3:42] facing each other and you always have [3:44] the cold in a cold air intake uh through [3:47] the front so they have the ventilation [3:48] going through the bottom right here then [3:51] it will go up and they will have the [3:53] cold air intake uh from the front and [3:56] then one of the technicians from headset [3:58] just highlighted [3:59] that i should not be worried about the [4:01] messy cable management and i was just [4:03] looking at this i was like [4:05] this looks like perfectly organized i [4:07] mean they should they should just look [4:09] at my place at home it looks completely [4:11] messy compared to this but according to [4:13] their standards this is very messy and [4:14] not organized and i'm very curious to [4:16] see one of the new servers [4:19] we are now above those midi towers and [4:22] just look at them down there those are [4:24] 14 [4:25] 000 [4:27] midi tower pcs [4:28] obviously this is not 2021 standards [4:31] anymore we will get to a nowadays server [4:34] soon but it's just a comparison and also [4:36] for you to see the evolution they had at [4:39] hadzner when it just comes to the rack [4:41] design and everything but cooling wise [4:44] um now you can see if you check it from [4:46] above you can see the like cold air [4:49] intake which is this channel between the [4:51] midi towers in the center [4:53] and the cold air intake is about 20 [4:56] degrees celsius and that's also a mixed [4:58] air because they're mixing the cold air [5:00] from outside which is about currently 15 [5:03] to 16 degrees celsius they're mixing it [5:05] with the warm air that is already inside [5:09] the building so they're not even using [5:11] active ac because that's maybe a [5:13] misconcept conception that i had i [5:15] always thought that data centers with [5:17] this size they always have to rely on [5:19] active ac cooling but that's not the [5:21] case they only use the ac on few days [5:25] per year because [5:27] i mean it's germany germany isn't the [5:29] hottest climate there are only a few [5:31] days where you have much more than 30 [5:33] degrees celsius so the most time of the [5:34] year they can just rely on [5:37] the cold air from outside and then [5:39] they're mixing it with the ambient air [5:41] in here to always make sure that the [5:43] temperature in here is always 20 degrees [5:45] celsius well at least down there because [5:47] up here due to the natural convection it [5:50] is much warmer it's probably like 30 [5:51] degrees celsius up here and that's also [5:53] the reason why they have this angled [5:55] roof [5:56] because they rely on the convection for [5:58] the cooling or well for the air exhaust [6:01] right through the roof [6:02] the warm air can exit and i can totally [6:06] feel that and that's [6:07] i mean this kind of reminds me a bit of [6:09] mission impossible if i was some cruise [6:11] i would probably climb in there but yeah [6:13] security levels are very high here we [6:16] just entered a different part of the [6:18] area from hetzner and this building [6:19] right here was established in 2019 [6:22] roughly and just looking at this you can [6:24] see this is completely different no more [6:25] midi towers and [6:27] it is warm in here it is really really [6:29] warm that is because we are like at the [6:32] hot air outtake like hot air exhaust [6:34] area so if we just walk up here [6:37] we can feel that [6:39] the hot air is coming out here and yeah [6:42] i mean let's just go upstairs again and [6:44] to get a proper impression okay so we [6:45] just went upstairs again in the [6:47] different data center area which is a [6:49] lot newer and just looking down [6:51] it kind of feels like it's it's a bit [6:53] more empty than the other one [6:56] which could give you a wrong impression [6:59] one of those rows are 1400 servers [7:02] and what is quite cool in this case that [7:04] we can look through from the top [7:06] so the top is closed off with some [7:09] acrylic but if we take a look through [7:12] the two rows of the servers you can see [7:14] the mesh on the floor and that is the [7:16] cold air exhaust well that's the cold [7:18] air delivery for the servers it will go [7:20] up between the servers and then push [7:23] through and at the same time it's not [7:25] only getting pushed through from the [7:27] air conditioning or from the cooling [7:29] concept itself because the air [7:30] conditioning is not switched on as i [7:32] already explained but the servers are [7:34] also actively cool so they have [7:36] depending on the server configuration [7:37] they have pull configuration push [7:39] configuration push pull [7:41] because there are also a lot of epic [7:42] servers down there and [7:44] epic cpus depending on the cpu maybe 280 [7:47] watts tdp yeah they require cooling and [7:49] that is also why this kind of looks [7:51] empty because the area down there where [7:54] you have all those green blinking leds [7:56] that is a cloud area and the cloud area [7:58] has a very high power density which [8:00] means that it requires a lot of energy [8:03] at the same time they have to make sure [8:05] that they can cover everything with the [8:06] ups and yeah [8:09] that is the reason why it might look [8:10] empty but it's definitely not [8:12] everything which you can see down there [8:14] is also not the normal 19-inch rack [8:16] standard because they are building their [8:18] own servers they're building their own [8:20] racks their own cases their own cooling [8:22] concept everything is completely built [8:23] in-house and developed in-house which [8:25] means on first look it doesn't look like [8:27] a standard but it also gives them much [8:29] more flexibility [8:31] running 250 000 servers also requires a [8:34] lot of storage i mean that's what the [8:35] servers are for [8:37] and [8:37] sometimes some of those hdds fail [8:40] which you can see [8:42] they're all dead and they have to be [8:44] professionally recycled and obviously [8:47] also get destroyed because otherwise you [8:49] cannot make sure that the data would be [8:51] completely erased obviously 250 000 [8:54] servers have to be built somewhere and [8:56] that's why we just entered the [8:57] production area [8:58] just looking at this is already quite [9:00] impressive just trace of cpus for [9:02] example [9:04] 9900 case so case cues 8700 without k [9:08] those are for the ax 61 series that's [9:10] the service we also advertised on my [9:12] channel ryzen 9 3900 [9:15] one tray another tree underneath [9:17] i [9:18] i would like to touch things but i tend [9:20] to break things that's why i'm not going [9:21] to do it [9:22] and we have a lot of cooling equipment [9:25] the typical server heatsinks [9:27] thermal paste [9:28] yeah so that also shows the cooling [9:31] concept of hat center again because [9:32] they're usually relying on a pool [9:34] configuration for [9:36] the cooling itself just pulling out the [9:38] hot air that's why you also have those [9:40] like air channels [9:42] depending on the specific server like [9:43] for example px62 [9:45] it's a cooling channel [9:47] for asus [9:49] wsc [9:51] 246 okay [9:53] so hopefully we can now go to the [9:55] production area and actually see how [9:56] they're building some of those servers [9:58] as i just pointed out hetzner also i [10:00] mean they have their own main boards [10:02] those main boards are specifically built [10:04] from asus in this case for hatsner and [10:07] just looking at the skew it's a pro [10:08] ws565ace [10:10] never heard about this before i also [10:12] never heard about the pro 565 chipset [10:15] which is built on this mainboard and [10:17] this chipset [10:19] it it's not like it's spanning [10:20] overclocking but it just doesn't allow [10:22] it which is essentially the same but um [10:24] it also makes sense that in the server [10:26] you're not allowed to overclock the cpu [10:28] because it's [10:29] i mean they're building this for very [10:31] specific like heat dissipation and [10:33] cooling concept and if you're running in [10:36] this case it's a 5950x if you're [10:38] changing the power dissipation of the [10:39] cpu [10:40] run changing the voltage and everything [10:42] it will have a huge impact on lifetime [10:44] of everything of the vrms of your cpu [10:47] itself [10:48] and yeah they have to make sure that [10:49] everything runs reliable and runs on a [10:52] very long lifetime that's why they're [10:53] using this very specific chipset but [10:55] let's just go over and check out the [10:57] build process process itself the build [10:59] process of the server itself just as the [11:01] components is also very minimalistic you [11:03] only use what you need and that's why [11:05] the case itself for example is just a [11:07] few sheets of metal and by the way the [11:09] case itself is also 100 [11:11] made in germany and for the psu for [11:14] example it's also custom made to use for [11:15] shorter cables to only use the resources [11:18] they really need and also to make cable [11:19] management much easier apart from that [11:22] it's almost identical to any kind of [11:24] desktop build you know just using much [11:27] less components just using as much as [11:29] you really need so this would be our [11:31] server ready to go and as you can see [11:33] everything on this is purely cooling [11:35] focused and that's also why they have [11:36] this very specific layout with a [11:38] different cooling configuration or [11:40] cooling direction as what you would have [11:42] on a desktop pc because the exhaust is [11:44] on the rear on the right side with this [11:46] cooling channel on top and it will just [11:49] suck the air through the heatsink and [11:51] also at the same time make sure that [11:53] it's also cooling the memory sticks on [11:54] the side and also tries to cool the [11:58] vrm's at the same time from the same [11:59] direction so everything is very [12:01] efficient to cool the server with just a [12:04] single fan for exhaust and at the same [12:06] time cool every single relevant [12:07] component you will remember a huge pile [12:09] of hdds where we just set on and [12:12] somewhere you have to determine if those [12:14] hdds actually are damaged or if they're [12:16] still in a good working condition [12:18] because sometimes the customer might [12:20] report that they're seeing some kind of [12:21] issue within the hdd and hetzer has to [12:24] verify if that's true or not and that's [12:26] what's happening right here that's the [12:28] hdd testing area [12:30] just such an enormous amount of hdds and [12:34] you have those status leds like right on [12:36] top here number one to seven [12:39] they're blinking in a slow pattern which [12:41] means that they're in the testing [12:43] condition right now if we switch over [12:45] here so number five is permanently on [12:47] which means that it passed the test but [12:49] if we go down there you can see for [12:51] example this hdd [12:53] with this fast blinking pattern means [12:55] that this is actually damaged [12:57] and this one is actually also damaged so [12:59] those eventually will be replaced [13:01] even though hetzer pointed out that [13:03] they're working on this and in in their [13:06] mind this is still messy and to me this [13:08] is like like everything is perfectly [13:10] organized but they're still working on [13:12] making it even more perfect the [13:14] organization that's why they started [13:16] like 3d printing some hdd holders for [13:20] their testing equipment adding the leds [13:22] right [13:23] in the testing holder itself [13:26] and then have this like more organized [13:28] because in their mind this is not [13:29] organized but to me this looks like [13:31] perfect [13:32] this is the same area where we've been [13:34] before from above so this is the area [13:37] where we have about 700 servers on this [13:39] side 700 servers on that side so they [13:42] call this the cold aisle basically you [13:44] have the the cold air coming out from [13:47] the floor [13:48] through this mesh right here and then [13:51] the cold air goes all the way up [13:52] through the surface [13:54] back out [13:55] here the temperature is quite okay it's [13:57] about 20 degrees celsius if we go up [13:59] back outside it's about 28 degrees [14:01] celsius and at that point i also want to [14:03] highlight that's actually a huge honor [14:05] and i just want to thank hetzer that we [14:07] can be here [14:09] that they have the trust [14:11] to put me in here because i'm well known [14:13] for breaking things and i could just rip [14:16] apart i don't know things here and [14:18] damage parts which i obviously don't [14:20] want to do so yeah [14:22] absolutely amazing that they were so [14:24] open and show everything to us and gave [14:26] us the opportunity to show [14:28] everything also to you guys out there [14:30] right here it's again the ax 101 the [14:32] server which where we just uh showed and [14:35] watched the building process right here [14:37] with the two ssds on the side above we [14:40] have an ax 161 which is featuring an [14:42] epic 7502 [14:44] so single socket server epic above we [14:47] have an ax41 nvme that's the one which [14:49] we were advertising quite often in our [14:51] pre-roll videos ryzen 5 3600 and that is [14:55] compared to the midi towers which we saw [14:57] right at the beginning that is the most [14:59] recent [15:00] evolution of direct design and also like [15:02] also very recent hardware [15:05] in the event where you have a loss of [15:06] power then certainly you have to do [15:08] something that your servers are still [15:10] running so you have those battery rooms [15:14] it's just it's just a room full of [15:17] batteries [15:18] uh one of those cells has two volts so [15:20] you can just hook up a ton of them [15:22] um in a row [15:24] just 120 for example then you have 240 [15:27] volts which is still dc so you have to [15:30] convert it to ac i guess not sure how [15:31] exactly they're doing that um or you [15:34] just run it with 12 volt directly to the [15:36] server which is something you wouldn't [15:37] do because it's running on 230 volt yeah [15:41] it wouldn't make much sense anyway [15:43] so this is enough power to power [15:45] the area which you just saw one data [15:48] center that's the area we saw from above [15:50] for 15 minutes [15:52] uh yeah 15 minutes but that should [15:54] usually never happen because you always [15:55] have your emergency power generator and [15:58] which is running on diesel that should [16:00] run within seconds so this will never be [16:03] out of power usually since cooling is [16:05] the most relevant part of the server [16:06] itself and the entire cooling just [16:08] relies on a single fan they're also [16:10] doing in-house fan testing and the [16:12] in-house fan testing is in an area where [16:14] the ambient temperature is about 40 [16:15] degrees celsius which is slightly above [16:17] what you would have in the server itself [16:19] so it's tougher for the fans and those [16:22] fans are running 24 7 for long [16:24] durability testing so if one of those [16:26] fans or multiple will fail they know [16:28] there might be an issue with these fans [16:29] and they can replace it or could replace [16:31] it in the server before they are failing [16:33] and it will also have an impact on [16:35] future purchase decisions this is one of [16:37] the massive fans which are pushing the [16:39] air through the floor that is really [16:42] massive and especially if they're [16:43] rotating fast you can imagine it's also [16:45] quite loud we're taking a quick look [16:46] inside the test lab of hetzner because [16:49] obviously they have to verify different [16:51] kind of setups if new for example new [16:53] cpu generations are entering the market [16:55] they have to check cpu mainboard memory [16:58] configurations but also if there is like [16:59] an external request for example a [17:01] customer wants to have 100 servers with [17:04] a specific configuration they don't [17:06] usually do then they have to make sure [17:08] it's verified everything is working for [17:10] example the left they have a specific [17:12] test sub setup that's only testing hdds [17:14] or ssds and if we just continue to the [17:17] right [17:18] there are multiple different setups and [17:21] this one is just running right now with [17:22] some thermal testing load so they have a [17:24] thermal imaging camera right here and [17:26] this thermal imaging camera is tracking [17:29] the area down here of the vrms so it's [17:32] checking if the vrms are overheating [17:34] during let's say prime95 load for [17:36] example we also found this very [17:38] interesting unit uh the first look you [17:40] will just think that this is a typical [17:42] server because of the like greenish pcb [17:44] and everything but let's just go to the [17:46] right where we have the same type of [17:48] mainboard but it's just not populated [17:50] the base is b550 but it's again the pro [17:53] 56595 [17:55] chipset whatever [17:57] so it's the same chipset which is [17:58] basically blocking overclocking [18:00] but the special feature is that you can [18:02] see it's a rotated socket so evga is not [18:05] the only mainboard company which is able [18:07] to rotate the socket you can see this [18:08] one is from astro rack [18:10] and the reason why the socket is rotated [18:13] it's just a pure cooling aspect if we [18:15] move again to the left [18:17] so talking about the cooling concept [18:18] again the cold air will just enter the [18:21] server from the right side and because [18:23] the socket is rotated the cooling can [18:26] make use or utilize the full length of [18:28] the cooler at the same time cool the [18:30] brm's on the right side cool the vrm's [18:32] which are sitting underneath the tunnel [18:33] on the left side and also efficiently [18:35] cool the memory sticks so the only [18:37] reason why they had this specific [18:39] mainboard [18:41] made because of the rotated socket which [18:43] allows better cooling for their very [18:45] specific use the last also interesting [18:47] feature inside the test lab is this [18:49] thermal testing chamber where they can [18:50] increase the temperature inside the [18:52] chamber there's just some heating [18:53] elements sitting inside so they for [18:55] example can simulate an ambient [18:57] temperature of about 45 degrees celsius [18:59] which is something you would never have [19:01] in the data center but it could [19:03] theoretically happen that your ac is [19:05] maybe breaking down i don't know and in [19:07] that event you want to make sure that [19:09] your pc or your server doesn't crash it [19:12] might throttle because the cpu is [19:14] overheating but it shouldn't crash and [19:16] that's what they're testing right here [19:17] with their servers and then we also [19:18] found this interesting thing it's a 3d [19:21] 3d printed part [19:23] fan testing so they're also testing the [19:25] fans we already saw the fan like relia [19:28] reliability testing area they're also [19:30] checking the fans for example [19:32] is the advertised air volume on the fan [19:35] exactly what they're also delivering [19:36] because they also have to double check [19:38] if it's like 50 square meters per hour [19:40] or if it's maybe just 25 even though [19:43] hetzner usually doesn't do water cooled [19:45] systems or water cooled servers we still [19:47] found this piece right here and this is [19:49] for testing psus because i mean what [19:52] else would you use than an amd threaded [19:54] were overclocked that can consume a ton [19:56] of power and [19:58] in case you want to test if your psu can [19:59] handle 600 or 700 watt load then you can [20:02] just use your amd 30 per system all [20:04] right so that's the end of the video [20:06] series from visiting hetzner i hope you [20:08] enjoyed the video as much as we did and [20:10] thanks for tuning in see you next time [20:12] bye [20:28] you