Valve's Abrash Forecasts Consumer VR Hardware on PC by 2015

What VR Could, Should, and Almost Certainly Will Be within Two Years (Steam Dev Days 2014) — Steamworks Development

Steamworks Development

Consumer-priced virtual reality will arrive within two years because existing display and tracking technologies can cross the human perceptual threshold for presence without scientific breakthroughs.

AI-assisted summary of What VR Could, Should, and Almost Certainly Will Be within Two Years (Steam Dev Days 2014) — original source. Check the original for full context.

Source published:

Key ideas

  • Valve forecasts that compelling, consumer-priced VR hardware capable of true psychological presence can be delivered by around 2015 using modifications of existing commodity components rather than speculative breakthroughs. (0:00)
  • True presence requires meeting strict simultaneous hardware thresholds, including low-persistence displays (under 3 ms) at high refresh rates (~95 Hz), sub-20 ms motion-to-last-photon latency, and sub-millimeter 6DoF tracking. (16:06)
  • The PC ecosystem remains the mandatory home for early VR evolution due to the immense compute power needed for 95 Hz stereoscopic rendering, unconstrained developer iteration, and open collaboration. (5:23)
  • Software paradigms must be reinvented from scratch because traditional artificial FPS locomotion induces severe motion sickness, and conventional 3D art tricks like normal mapping break down under stereoscopic inspection. (23:35)

Detailed notes

## Background Published on YouTube by Steamworks Development on February 11, 2014, this video captures a presentation delivered by Valve's Michael Abrash at Steam Dev Days 2014. According to the video description, the talk covers what affordable consumer virtual reality hardware will be capable of achieving within two years, based on working prototypes built at Valve, and explains why developers should start building for the medium immediately.

## TL;DR Abrash outlines Valve's technical roadmap and thesis for bringing compelling, consumer-priced virtual reality to market by 2015. He defines the perceptual threshold of 'presence'—the subconscious conviction of being inside a virtual world—and details the strict hardware requirements needed to achieve it, including low-persistence displays, 95 Hz refresh rates, precise 6DoF tracking, and sub-20ms motion-to-photon latency. Highlighting Valve's open collaboration with Oculus, Abrash explains why high-performance PCs are the only viable home for early consumer VR and urges game developers to pioneer entirely new gameplay vocabularies rather than porting traditional 2D monitor titles.

## Atmosphere & style The presentation carries the measured, authoritative gravitas of a veteran systems architect delivering an industry prophecy rather than hype. Abrash speaks in a calm, analytical tone, blending rigorous perceptual neuroscience and display physics with personal anecdotes of working alongside John Carmack and standing frozen in visceral panic atop a simulated VR ledge. The room feels charged with quiet anticipation, as the talk methodically strips away science-fiction hand-waving to reveal an imminent, engineering-driven platform revolution.

## Bulletin - [00:00](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=0s) Valve forecasts that compelling, consumer-priced VR hardware capable of true psychological presence can be delivered by around 2015 using modifications of existing commodity components rather than speculative breakthroughs. - [16:06](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=966s) True presence requires meeting strict simultaneous hardware thresholds, including low-persistence displays (under 3 ms) at high refresh rates (~95 Hz), sub-20 ms motion-to-last-photon latency, and sub-millimeter 6DoF tracking. - [05:23](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=323s) The PC ecosystem remains the mandatory home for early VR evolution due to the immense compute power needed for 95 Hz stereoscopic rendering, unconstrained developer iteration, and open collaboration. - [23:35](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1415s) Software paradigms must be reinvented from scratch because traditional artificial FPS locomotion induces severe motion sickness, and conventional 3D art tricks like normal mapping break down under stereoscopic inspection.

## Key takeaways - [00:00](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=0s) Compelling consumer-priced VR hardware capable of transforming entertainment is technically feasible within two years. - [07:33](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=453s) Presence is a distinct psychological phenomenon where automated, low-level perceptual subsystems trick the brain into believing it is truly in another physical location. - [02:54](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=174s) The exact same engineering thresholds that trigger presence also eliminate the sensory cue conflicts responsible for simulator sickness. - [02:54](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=174s) Valve actively collaborated with Oculus by sharing tracking, optics, and low-persistence research, directly influencing the Rift 'Crystal Cove' prototype. - [09:20](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=560s) Low-level human vision constructs depth and scale outside conscious awareness, meaning immersion cannot be achieved purely through intellectual reasoning. - [13:20](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=800s) A wide field of view between 80 and 110 degrees is necessary to engage peripheral vision cues essential for balance and spatial orientation. - [13:20](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=800s) To prevent debilitating retinal smearing caused by the vestibulo-ocular reflex (VOR), pixel persistence must be reduced to 3 milliseconds or less. - [16:06](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=966s) Reducing pixel persistence causes severe flicker at 60 Hz, requiring display panels to run at elevated refresh rates such as 95 Hz. - [16:06](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=966s) Global display illumination is necessary to avoid the shearing, bending, and tilting artifacts inherent to rolling shutter displays during head motion. - [16:06](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=966s) HMD optical assemblies face heavy physical trade-offs, relying on one or two compact lenses per eye to magnify smartphone-derived display panels. - [18:41](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1121s) Precise optical calibration is critical; uncorrected lens distortion produces visual swimming during head movement, instantly inducing motion sickness. - [18:41](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1121s) Robust 6DoF tracking requires positional accuracy within 1 millimeter and rotational accuracy within 0.25 degrees across at least a 1.5-meter cubed volume. - [20:21](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1221s) Motion-to-last-photon latency must be kept strictly at or below 20 milliseconds (with 25 ms as the absolute ceiling) to maintain physical presence. - [20:21](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1221s) Presence is an emergent property that fails completely if any single technical requirement across tracking, latency, optics, or display falls short. - [21:25](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1285s) Significant unsolved hardware problems remain for future headsets, including retail-ready inside-out tracking, optical IPD adjustment, eye tracking, and wireless tether elimination. - [23:35](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1415s) Traditional flat-screen design rules fail in VR; developers must invent a new design language focused on slower movement, closer scale, and stereoscopically valid art assets. - [25:21](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1521s) PC is uniquely positioned to drive the VR platform revolution over consoles and mobile due to maximum raw FLOPS, open development, and rapid hardware iterations.

## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=0s) The 59-Word Summary of VR's Near Future

Introduced on stage at Steam Dev Days 2014, Michael Abrash of Valve opens by addressing the crowd's visible enthusiasm for virtual reality. Before plunging into the meat of his technical presentation, he decides to encapsulate his entire thesis into an exact 59-word statement, knowing someone in the audience would likely count the words:

> “Compelling consumer-priced VR hardware is coming, probably within two years. It's for real this time. We've built prototypes, and it's pretty incredible. Our technology should work for consumer products. VR will be best and will evolve most rapidly on the PC. Steam will support it well, and we think it's possible that it could transform the entire entertainment industry.”

Abrash stresses that even if attendees remember nothing else from his talk, taking away those 59 words alone makes their time well spent.

```mermaid flowchart LR accTitle: Valve VR Thesis accDescr: Flow showing consumer VR timeline leading to PC platform evolution and entertainment transformation. A["R&D & Prototypes (Valve)"] --> B["Consumer-Priced VR Hardware (~2 Years)"] B --> C["PC Platform Leadership (Steam Support)"] C --> D["Transformation of Entertainment Industry"] ```

[01:15](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=75s) From Snow Crash and John Carmack to Modern VR

Abrash reflects on the winding 20-year path that led him to virtual reality. In the mid-1990s, while working at Microsoft, he understood networking and was already writing rudimentary 3D rendering packages. He consumed science fiction and grasped the abstract premise of virtual worlds, yet the concept still felt like pure fantasy rather than something practical to build. That mindset shifted when he read Neal Stephenson’s novel *Snow Crash*. Thinking through the requirements of building the "Metaverse," Abrash estimated he knew how to implement around 80% of it using the technology of that era—an estimate he admits was overly optimistic, quipping that he is “still waiting for my first VR sword fight.”

Shortly afterward, Abrash met with John Carmack, who spoke uninterrupted for two hours about his grand vision: generating persistent, networked 3D servers that users could connect together and build upon freely, eventually accreting into cyberspace. Inspired by this vivid technical future, Abrash joined Carmack at id Software to build *Quake*, laying the groundwork for millions of people who now inhabit networked 3D spaces every day. Abrash credits Stephenson and Carmack for setting that path in motion, noting that while he does not consider himself a visionary, he hopes this talk will similarly point developers toward the next missing chapter of the Metaverse: virtual reality.

[02:54](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=174s) Achieving Presence and Valve's Collaboration with Oculus

While many developers viewed the original Oculus Rift DK1 as an interesting curiosity, Abrash insists VR will exert a major influence much faster than anticipated. Two years prior, Valve targeted VR as a transformative medium. Valve's R&D resulted in internal prototype head-mounted displays and bespoke room-scale demos that produce a profound psychological phenomenon known as "presence"—the unmistakable sensation of truly being inside another space, as if teleported.

Crucially, the exact same hardware capabilities required to induce presence also eliminate motion sickness. Abrash identifies Oculus as the leading candidate to deliver this level of consumer-grade hardware. While the DK1 lacked the hardware thresholds needed to induce presence, the newer Oculus Rift "Crystal Cove" prototype represents a major leap forward:

- Shipped over 50,000 Oculus Rift DK1 units to developers worldwide. - Built low-persistence prototype displays at Valve, demonstrating them to Oculus to convince them of low persistence's necessity in eliminating motion blur; this directly influenced the creation of the Crystal Cove prototype. - Shared Valve's internal tracking research, hardware architectures, optical calibration data, and lens designs directly with Oculus. - Crystal Cove integrates key presence enablers: positional translation tracking, lower latency, higher resolution panels, and low-persistence displays.

> “If Oculus executes well—and so far they seem to be—I think they could well deliver strong presence on the PC within the next two years.”

[05:23](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=323s) Why High-End PC Hardware Is Essential for Presence

Abrash argues that the advent of presence-capable VR will precipitate a sea change across entertainment, tipping the scales from passive, traditional media toward interactive computer technology. True presence cannot be achieved via streaming, films, or broadcast television because human perception demands extremely low motion-to-photon latency and massive rendering bandwidth. Those constraints mandate a head-mounted display wired directly to high-performance local hardware running heavy-duty real-time 3D rendering.

Because raw compute power is indispensable to satisfying these perceptual thresholds, the personal computer ecosystem—spanning Windows, Linux, and OS X—stands as the definitive home for VR:

> “A corollary is that the PC—Linux, Windows, and OS X—is going to be the best place for VR because that's where the most FLOPS live.”

Valve’s strategy to foster this ecosystem operates on three fronts: 1. **Steam Platform Integration:** Building native VR infrastructure into Steam (as detailed by Valve developer Joe Ludwig in another talk). 2. **Hardware R&D & Open Collaboration:** Continually advancing display, tracking, and optical research while collaborating openly with external partners like Oculus. 3. **Software Prototyping:** Experimenting internally to discover the native design language of VR and sharing findings with developers.

Abrash clarifies that while Valve has no current plans to manufacture or ship its own consumer VR hardware, that stance could change if necessary to ensure the PC platform thrives. He also extends an open invitation to any Steam partner visiting Seattle over the subsequent months to email him directly (`mabrash@valvesoftware.com`) to experience Valve's dedicated 30-minute room-scale VR demo firsthand.

[07:33](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=453s) The Neuroscience of Low-Level Perception

To explain why presence requires specific hardware specifications, Abrash delves into human perception. True immersion does not stem from conscious reasoning; rather, it requires convincing automated, low-level perceptual subsystems that function deep beneath conscious awareness.

To demonstrate how the subconscious brain manipulates perceived reality before conscious thought ever receives it, Abrash presents a classic visual illusion of an alleyway containing two spheres: - **Visual Cues Present:** Perspective convergence lines, surface textures, directional lighting, and cast shadows all signal significant spatial depth. - **The Task:** When participants are asked to adjust the nearer sphere until both circles project identical 2D sizes on the screen, they unfailingly resize the foreground circle so that it is physically larger than the background circle. - **The Reality:** In the slide demonstration, the sphere perceived as farther away looks significantly larger, yet the foreground circle is actually almost 20% larger in raw pixel dimensions.

Abrash points to functional MRI (fMRI) studies revealing that this calculation happens at the earliest visual processing layer of the brain (V1 in the visual cortex). Even before conscious awareness begins processing the image, the visual cortex allocates a larger surface area of brain tissue to the object perceived as distant, even when both objects project the exact same size onto the physical retina.

> “The visual system has already judged the sizes before the data reaches conscious awareness.”

Understanding these subconscious visual mechanisms is the foundational requirement for building hardware that can trigger genuine presence.

[09:20](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=560s) Low-Level Perception and the Cliff Demo

Michael Abrash presents an optical illusion to demonstrate how human visual processing constructs reality outside conscious awareness. Two visual targets show an "X" shape where one centerpiece appears gray and the other looks dark yellow. When the background is masked out without altering the centerpieces themselves, both patches are revealed to be the exact same shade. Because human vision evolved to maintain consistent perception across radically shifting illumination conditions, low-level visual processing relies heavily on context. Abrash emphasizes that this occurs entirely below conscious thought, just as the brain automatically warps the perceived size of shapes based on context. When virtual reality successfully stimulates these low-level sensory mechanisms, the user genuinely feels situated in a place; when it fails, they feel an indefinable disconnection.

```mermaid flowchart LR accTitle: Sensory Processing to Perception accDescr: Diagram showing sensory inputs passing through subconscious low-level processing before reaching conscious awareness.

subgraph SENSORY["Input Layer"] INP["Visual & Vestibular Cues"] end

subgraph SUBCONSCIOUS["Subconscious Processing"] LLP["Low-Level Context & VOR"] end

subgraph CONSCIOUS["Conscious Mind"] PERC["Sense of Presence & Reality"] end

INP --> LLP LLP --> PERC ```

To illustrate the visceral power of proper sensory stimulation, Abrash details a Valve VR demo room experience, issuing a humorous spoiler warning: “This is a bit of a spoiler for one of our demos so if you don't want to hear it cover your ears I'll wait a moment while you decide.” In the demo, the user stands on a diving-board-style stone ledge high above the floor of an enclosed box room textured with outdated web pages—including one announcing that Facebook is going public. Viewed on a conventional computer monitor, the flat or warped imagery appears completely unimpressive. Yet inside the head-mounted display, the physical response is uncontrollable:

> “Looking this at this on a screen even when it's not warped doesn't do anything for me but whenever I stand on that ledge in VR my knees lock up just like they did on top of the Empire State Building even though I know for certain that I'm in a demo room wearing a head mounted display looking at imagery of the inside of a badly textured box my body knows that I'm at the edge of a cliff.”

The subconscious reaction does not fade with time or repetition. Because the inputs satisfy the low-level perceptual system, the user's body remains convinced at an instinctive level that it stands over a lethal drop.

---

[11:25](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=685s) Defining Presence vs. Immersion

Abrash distinguishes between two terms often conflated in VR discussions: *immersion* and *presence*. Immersion simply means being surrounded by the imagery of a virtual world, whereas presence represents the profound sensation of actually existing inside that virtual space. Presence cannot be fully communicated in words; it can only be understood through firsthand experience.

Abrash describes the concrete physical manifestations that occur when presence takes hold: reaching out involuntarily to touch a rendered cube despite knowing the room is empty, ducking to avoid a hanging virtual pipe, experiencing visceral unease when a massive block suspends overhead, and feeling entirely unable to step off a virtual drop. Upon taking off the head-mounted display, users frequently suffer disorientation as their perceptual system readjusts to finding itself back in physical reality.

> “It's more than just looking at someplace interesting it's flipping the switch that makes you believe deep in your lizard brain that you are someplace interesting.”

Abrash explains that presence is among the most powerful non-real experiences imaginable precisely because it engages the exact same sensory channels as physical reality:

- **Visceral Engagement:** Presence operates on the "lizard brain" rather than intellectual belief, hooking deeper into the human perceptual system than any prior medium. - **Audience Variation:** Susceptibility to presence varies across individuals, but responses consistently strengthen as hardware systems refine and improve. - **The Core Thesis of Valve's VR Research:** Abrash summarizes their strategic direction: “Consequently we think that building Hardware that's capable of delivering a strong sense of presence is the key to VR success.”

---

[13:20](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=800s) Hardware Requirements: FOV, Resolution, and Low Persistence

Through iterative research and prototyping, Valve discovered that a strict baseline across multiple hardware specifications must be met simultaneously before presence can occur. Crucially, the exact same engineering factors required for presence also mitigate simulator sickness. Motion sickness typically stems from cue conflicts—such as tracking flaws that create a mismatch between visual input and the vestibular system in the inner ear.

Field of view (FOV) is the first critical requirement. Beyond creating visual immersion, a wide FOV supplies peripheral vision cues essential for motion sensing, human balance, and situational awareness. Presence begins to emerge around an 80-degree FOV and scales up substantially through at least 110 degrees, the maximum field Valve had tested.

Resolution presents a difficult hurdle because stretching an image across a wide FOV severely magnifies the display's individual pixels. In an HMD with a 110-degree FOV and a 1K × 1K per-eye resolution, the angular pixel density is merely one-seventh that of a standard big-screen television and one-tenth the resolving limit of the human eye—falling below the pixel density of the original *Quake* running at 320 × 200. Valve determined that a 1080p display represents the minimum viable resolution for presence, with 1440p and 2160p anticipated as massive visual upgrades once panels of appropriate physical dimensions become manufactured.

Pixel persistence—the duration each pixel remains illuminated per frame—is uniquely critical for head-mounted VR. On traditional stationary monitors, televisions, or mobile phones, full persistence goes largely unnoticed. In VR, however, the vestibulo-ocular reflex (VOR) keeps the human eye fixed on targets in the physical world while the head moves rapidly. If pixels stay lit throughout the entire refresh frame on a full-persistence display, the image drags across the moving retina:

> “The image on the left is with the head held still and the image on the right is a simulation of what happens with a leisurely 120° per second head turn rate on a 60 HZ full persistence display this results of in two degrees of smearing across the retina per frame which as you can see reduces detail considerably.”

To eliminate this debilitating retinal blur, Valve determined that pixel persistence must be cut to 3 milliseconds or less at 1K × 1K and 110 degrees FOV, with even shorter illumination windows required as pixel densities rise.

---

[16:06](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=966s) Refresh Rates, Global Display, and Optical Lens Design

Cutting pixel persistence to 3 milliseconds introduces a severe side effect: visible display flicker at conventional 60 Hz refresh rates. To eliminate flicker across a wide FOV, display refresh rates must be increased. Valve built an experimental low-persistence headset capable of running at 95 Hz, which successfully eliminated all perceptible flicker. Abrash notes that while an intermediate refresh rate might suffice, delivering consistent 95 Hz stereoscopic rendering will present an immense computational challenge for game engines as panel resolutions scale upward.

```mermaid flowchart TD accTitle: Trade-offs in VR Display Design accDescr: Flowchart illustrating the chain reaction from low persistence to high refresh rate demands and optical trade-offs.

A["Low Persistence (<3ms)"] -->|"Prevents retinal smear"| B["Flicker at 60 Hz"] B -->|"Requires higher refresh"| C["95 Hz Display Rate"] C -->|"Demands high performance"| D["Intense GPU Stereo Rendering Load"]

E["Cell Phone Panels"] -->|"Magnified by 1-2 lenses"| F["Severe Optical Aberrations"] F -->|"Corrected by trade-offs"| G["Subjective In-HMD Optimization"] ```

Display illumination timing also dictates image stability. Valve's prototype relies on *global display*, where every pixel in the panel lights up simultaneously. This eliminates the severe spatial distortions inherent to standard *rolling displays*, in which scan lines illuminate progressively over the frame, causing the virtual image to shear, compress, stretch, or tilt during head motion:

- **Correcting Rolling Displays:** While software reprojection could theoretically compensate for rolling shutter artifacts by updating the frame buffer mid-scan based on eye movement, that technique remains unproven. - **Eye-Tracking Bottlenecks:** Head movement cannot serve as a reliable proxy for eye motion across all edge cases, and low-latency head-mounted eye tracking remains an unsolved engineering hurdle. Consequently, global display remains the only proven method to preserve image geometry.

Turning to optics, Abrash demystifies the assumption that designing an HMD optical path is trivial. Practical consumer constraints require leveraging mass-manufactured smartphone panels magnified by lenses, as microdisplays and optical waveguides cannot provide a sufficiently wide FOV. Weight, form factor, and industrial design restrict the optics to only one or two lenses per eye.

Eliminating all optical aberrations and geometric distortions with perfection would require a massive, multi-element optical assembly. Abrash points out a diagram of an ideal optical stack where the front two lenses measure over a foot in diameter and the complete optical housing exceeds five pounds—an absurd impossibility for wearable headgear. With a hard limit of one or two compact lenses per eye, compensating for one aberration inevitably exacerbates another. Because focal lengths, eye relief, and optical distortions cannot be evaluated purely through automated mechanical metrics, every lens design requires subjective, time-consuming evaluation directly inside VR headsets across diverse users. Valve has produced lenses capable of sustaining presence, but the overall design space remains a delicate web of trade-offs.

[18:41](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1121s) Optical Calibration and Tracking Requirements for Presence

Optical calibration may initially sound like a secondary engineering detail, but Michael Abrash emphasizes that it is both hugely challenging and utterly critical because the human visual system is exquisitely sensitive to slight geometric and lens errors. A virtual scene might look immaculate when inspected statically, but the moment a user turns their head from side to side, miscalibrated optics cause straight lines and surfaces to warp and ripple horribly. This visual swimming instantly shatters the illusion of presence and triggers acute motion sickness almost immediately. Getting calibration dialed in is also a prerequisite for debugging the rest of the VR pipeline; Valve discovered that until their optical calibration was dead-on, they could not even detect minor head-tracking glitches because the optical instability masked them entirely. As Abrash notes, “The bottom line is that a highly accurate process for characterizing the lenses and correcting the rendered images is absolutely essential.”

Once visual clarity and calibration are achieved, the virtual image must be tracked and positioned so that the brain’s perceptual system accepts it as an unyielding physical reality. This necessitates rock-solid 6-degree-of-freedom (6DoF) head tracking that reports translation along the X, Y, and Z axes alongside angular orientation. Valve’s internal testing determined that sustaining presence requires tracking accuracy of within a single millimeter of translational position and a quarter of a degree in orientation, maintained continuously across a tracking volume of no less than 1.5 meters on each side. Delivering that precision is considerably harder than it sounds, and no existing consumer tracking solution at the time came close. Showing the room-scale tracking setup used in Valve's Dev Days demos, Abrash jokes that while their fiducial-marker rig works, “it isn't exactly consumer friendly” because it effectively requires “wallpapering your house.” However, Valve is developing alternative tracking architectures that dispense with wall markers, giving them high confidence that consumer-grade tracking is viable in the near term.

- Tracking precision targets for presence: ≤ 1 mm position error, ≤ 0.25° orientation error. - Minimum tracking volume required: at least 1.5 meters cubed. - Full 6DoF tracking (translation in X, Y, Z plus rotational orientation) is mandatory; rotational tracking alone fails to convince the human perceptual system.

[20:21](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1221s) Latency Budgets and Presence as an Emergent Phenomenon

Low latency is the next non-negotiable pillar: the virtual image must appear in the exact right place at the exact right moment. Valve measures latency strictly from the physical instant head motion occurs to the precise moment the final photon generated by that motion strikes the eye. Testing reveals that a motion-to-last-photon latency of 20 milliseconds, coupled with robust sensor prediction, reliably induces presence, while 25 milliseconds represents the absolute upper ceiling. Exceeding that threshold causes the virtual environment to feel detached rather than nailed down in space, causing the human perceptual system to reject the experience as reality.

Abrash underscores that presence is not an isolated attribute that can be achieved piecemeal through a single standout component; it is an emergent state that arises only when every technical element clears its required threshold simultaneously. If the lenses are improperly calibrated, scenes warp during head rotations; if tracking drifts or latency creeps above 20 to 25 milliseconds, the perceptual illusion collapses. As Abrash summarizes the challenge, “I want to emphasize that presence is not a property of any one of the elements I've discussed; it's a property that emerges when all the elements are good enough.”

```mermaid flowchart TD accTitle: Pipeline of Key Requirements for Inducing Presence accDescr: Interdependent hardware and tracking elements that together produce the emergent sense of VR presence. subgraph S1["Visual Presentation"] A["Optical Calibration<br/>Characterize lenses & correct rendering"] B["Low-Persistence Display<br/>Global illumination eliminating blur"] end subgraph S2["Spatial Tracking & Speed"] C["6DoF Tracking<br/>≤ 1 mm position, ≤ 0.25° orientation"] D["Low Latency<br/>≤ 20 ms motion-to-last-photon"] end subgraph S3["Emergent Outcome"] E["Perceptual Presence<br/>Brain accepts virtual world as real"] end A --> E B --> E C --> E D --> E ```

- Motion-to-photon latency limit: ≤ 20 ms (with 25 ms being the borderline limit). - Latency metric: measured from physical head motion onset to the emission of the last corresponding photon. - Emergent nature of presence: “Presence can't be induced if even one of the key elements is subpar.”

[21:25](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1285s) The 2015 Consumer Headset Specification and Hardware Roadblocks

Synthesizing Valve’s internal research and hardware prototypes, Abrash outlines what a viable consumer head-mounted display (HMD) could look like by 2015. Valve's working prototypes already achieve genuine presence using modified commodity parts, proving that cost-effective manufacturing at scale is within reach without waiting for speculative breakthroughs. While future headsets will eventually demand custom silicon and massive research investments, delivering an introductory consumer product requires only focused execution. Explaining this optimism, Abrash states, “In contrast, we believe everything on this slide is doable with relatively minor tweaks of existing technology—no breakthroughs or miracles are needed, just solid engineering.”

At the same time, Abrash is careful to distinguish between an internal laboratory prototype and a consumer-ready retail product. Turning an R&D demonstration into an affordable, manufacturable device entails significant commercial hurdles. In an ideal world, presence would benefit from vastly superior specs: displays could easily leverage up to 100 times as many pixels, wider fields of view, and optical assemblies that accommodate per-user lens positioning and interpupillary adjustments. Eye tracking would provide huge gains for rendering and presence but remains far from solved. Eliminating tether cables through wireless transmission represents another major ergonomic hurdle, while securing low-persistence, global-update, ultra-low-latency panels will require deep partnerships with display manufacturers.

- 2015 hardware forecast: an affordable consumer HMD capable of inducing true presence can be delivered within two years using modified commodity parts. - Display headroom: VR displays could realistically use up to 100× the pixel counts of current prototypes alongside wider fields of view. - Unsolved hardware engineering challenges: retail-ready 6DoF tracking, per-user optical and IPD calibration, robust eye tracking, tetherless wireless video transmission, and custom low-persistence panel manufacturing.

[23:35](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1415s) Unexplored Frontiers: Input, Locomotion, and a New Game Design Vocabulary

Beyond the visual headset, the non-visual facets of virtual reality represent even larger, more stubborn technical hurdles. Spatial 3D audio, haptic feedback, body tracking, and dedicated VR controllers are essential to long-term immersion, yet their interaction with core game mechanics is almost entirely uncharted territory. Abrash warns that figuring out these sensory domains will require years of collective experimentation across the game industry. In particular, software and game design must be reinvented from scratch because traditional 2D monitor design paradigms break down inside an immersive head-mounted display.

Locomotion exemplifies this design crisis: conventional first-person shooter (FPS) artificial movement instantly induces severe motion sickness in VR. Developers must likely shift toward slower character movement, confined scale, and intimate, close-up object manipulation, and then discover how to make those mechanics genuinely fun. Multiplayer VR promises to be far more compelling and socially profound than monitor-based gaming, evoking the groundbreaking early eras of deathmatch, player-created mods, and persistent dedicated servers. Visual aesthetics also require total re-evaluation. Normal mapping falls apart under stereoscopic inspection, high-frequency screen textures can look like cheap theatrical stage props, while surprisingly simple geometric scenes can evoke breathtaking realism. As Abrash points out, “In the end, a whole new VR gameplay vocabulary will need to be developed, just as was the case with FPSs.”

- Traditional FPS artificial locomotion causes severe motion sickness; VR demands new movement systems, likely focusing on slower speeds and close-up interactions. - Traditional art pipelines fail: normal maps look synthetic and flat in stereo 3D, and photorealistic monitor assets often look like cheesy stage sets. - Multiplayer VR carries the potential to become the most powerful online social medium yet devised. - Input and haptics remain harder and less explored than display hardware.

[25:21](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1521s) The Next Major Platform Shift and Why VR Belongs on PC

Abrash places the emergence of VR in the lineage of seminal industry platform shifts that enabled entirely new classes of software, comparing its trajectory to the arrivals of *Myst*, *Quake*, *Wii Sports*, and *Angry Birds*. Straightforward ports of existing flatscreen titles can offer interesting visual immersion, but true greatness demands experiences built natively from the ground up for virtual reality. Drawing a parallel to the dawn of real-time 3D gaming, Abrash notes the nostalgic cultural weight carried by id Software’s classic shooter on its recent twentieth milestone: “The first game that nails a VR experience is going to be remembered fondly for a long, long time.”

When that breakthrough software appears, Abrash argues it will almost certainly debut on the PC rather than on consoles or mobile. First, PC hardware iterates rapidly and openly, as demonstrated by the rapid evolution of the Oculus Rift, whereas game consoles remain locked into fixed hardware cycles for years. Second, the open PC ecosystem allows a far broader, unconstrained community of software and hardware developers to freely experiment. Third, VR demands staggering computing performance—rendering independent stereoscopic viewpoints at 95 Hz with ultra-low latency requires top-tier graphical power that consoles cannot match, while mobile chips remain strictly bounded by heat and battery limits. With Valve baking native VR support into Steam and actively sharing its R&D discoveries, Abrash concludes with an emphatic message to developers: “Today's key takeaway is simple, but it has powerful implications: a great VR system at a consumer price in 2015 is more than just possible, it's sitting there waiting to happen, and it will happen, if not in 2015, then soon after.”

- Historical parallels: VR represents a foundational platform shift comparable to CD-ROM (*Myst*), true 3D engines (*Quake*, *Doom*), motion controllers (*Wii Sports*), and touch interfaces (*Angry Birds*). - Performance baseline: VR requires dual-buffer stereo rendering running at 95 Hz minimum refresh rates. - PC advantages over consoles and mobile: - Rapid, continuous hardware revision cycles instead of static multi-year console lifespans. - Open experimentation for independent developers without platform gatekeepers. - Raw compute capacity uninhibited by mobile thermal envelopes or console power constraints. - Active ecosystem integration and driver investment through Steam and Valve R&D.

[27:35](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1655s) Building the Future of VR Together

Michael Abrash brings his presentation to a close by framing the advancement of virtual reality not as a solitary effort by Valve, but as an open, collaborative journey with the entire developer community. He emphasizes the necessity of an active dialogue, calling for “a two-way exchange so together we can make VR one of the great entertainment revolutions.” Acknowledging that the technical challenges, creative opportunities, and hardware breakthroughs discussed during the session only scratch the surface of what lies ahead, Abrash invites attendees to continue the discussion directly during the upcoming Q&A session as well as through his blog. With that, he thanks the audience and concludes his talk to enthusiastic applause.

- **Collaborative development**: Abrash stresses that realizing the promise of consumer VR requires open communication and shared problem-solving between hardware pioneers and software developers. - **Continuing the dialogue**: Attendees are invited to follow up with Abrash in the subsequent Q&A session and follow along with his ongoing writings and technical deep dives on his blog.

## Notable quotes > [25:21](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1521s) The first game that nails a VR experience is going to be remembered fondly for a long, long time. — Michael Abrash

> [20:21](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1221s) Presence can't be induced if even one of the key elements is subpar. — Michael Abrash

> [23:35](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1415s) In the end, a whole new VR gameplay vocabulary will need to be developed, just as was the case with FPSs. — Michael Abrash

> [02:54](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=174s) If Oculus executes well—and so far they seem to be—I think they could well deliver strong presence on the PC within the next two years. — Michael Abrash

## Mentioned Michael Abrash, Valve, Steam, Steam Dev Days, Neal Stephenson, Snow Crash, John Carmack, id Software, Quake, Doom, Oculus, Oculus Rift DK1, Crystal Cove, Joe Ludwig, Facebook, Empire State Building, Myst, Wii Sports, Angry Birds

## Video description We've figured out what affordable VR hardware will be capable of within a couple of years, and assembled a prototype that reveals that that level of VR hardware is capable of stunning VR experiences. That hardware is almost certainly going to appear in that timeframe, and it will be worth starting to develop for it now. This talk will discuss what that hardware is, and what it makes possible.

Speaker: Michael Abrash (Valve) Slides (PDF): http://media.steampowered.com/apps/steamdevdays/slides/vrshouldbe.pdf Slides (PowerPoint): http://media.steampowered.com/apps/steamdevdays/slides/vrshouldbe.ppt

[Source](https://www.youtube.com/watch?v=G-2dQoeqVVo)

中文简报

Valve的Abrash预测PC消费级VR硬件将于2015年面世 消费级价格的虚拟现实将在两年内到来,因为现有显示和追踪技术无需重大科学突破即可跨越人类对“临场感”的感知阈值。 Abrash概述了Valve的技术路线图与核心论点,旨在2015年前将极具吸引力且符合消费级定价的虚拟现实设备推向市场。他界定了“临场感”(即身处虚拟世界中的潜意识确信)的感知阈值,并详细列出了实现该效果所需的严苛硬件指标,包括低余晖显示屏、95 Hz刷新率、高精度6DoF追踪以及低于20毫秒的动作到光子延迟。Abrash强调了Valve与Oculus的开放合作,解释了为何高性能PC是早期消费级VR的唯一可行依托,并敦促游戏开发者探索全新的玩法语汇,而非单纯移植传统的2D屏幕游戏。 Valve预测,利用对现有通用组件的改进而非寄希望于突破性技术,到2015年左右即可推出能够带来真正心理临场感且价格亲民的VR硬件。 (0:00) 真正的临场感需要同时满足严格的硬件阈值,包括高刷新率(约95 Hz)下的低余晖显示(3毫秒以内)、低于20毫秒的动作到最后一光子延迟,以及亚毫米级的6DoF追踪。 (16:06) 由于95 Hz双目立体渲染所需的庞大算力、无拘束的开发者迭代以及开放合作环境,PC生态系统仍是早期VR演进不可或缺的阵地。 (5:23) 软件范式必须从零开始彻底重塑,因为传统FPS的人工移动方式会引发严重的晕动症,而法线贴图等常规3D美术技巧在立体视觉检验下也会失效。 (23:35)

Reading summary

Michael Abrash of Valve argues that compelling, consumer-priced virtual reality hardware capable of inducing genuine psychological presence is achievable within two years using modified commodity components, with the high-performance PC ecosystem serving as the essential foundation. The Subconscious Mechanics of Psychological Presence (7:33) Abrash distinguishes passive immersion—simply surrounding a user with virtual imagery—from presence, a profound physiological state where subconscious perceptual systems treat a simulated space as physical reality. Citing neuroscience studies and visual depth illusions, Abrash explains that low-level sensory mechanisms, such as visual cortex area V1 and the vestibulo-ocular reflex, evaluate spatial cues and context long before conscious awareness processes them. When virtual reality satisfies these automated sensory subsystems, the body reacts involuntarily. In Valve's room-scale cliff demo, for example, users stand on a virtual stone ledge high above a textured pit. Despite fully knowing they are in an office room wearing a prototype headset, users experience locked knees and genuine vertigo. Abrash notes that this subconscious reaction does not dissipate with repetition, demonstrating that presence bypasses intellectual skepticism to engage primitive perceptual pathways. Consequently, Valve identified triggering presence as the primary design benchmark for consumer hardware. Display and Optical Engineering Thresholds (13:20) Achieving presence and eliminating simulator sickness require overcoming strict display and optical constraints. Abrash highlights field of view as a primary requirement, noting that presence begins around an 80-degree field of view and strengthens through 110 degrees, which is critical for peripheral balance cues. However, expanding the field of view severely magnifies panel pixels; Valve identified 1080p as the bare minimum resolution for presence, while angular pixel density remains far below consumer televisions. Pixel persistence is equally vital. On conventional full-persistence displays, head rotation causes images to smear across the retina due to the vestibulo-ocular reflex, resulting in dramatic blur. Valve determined that persistence must drop to 3 milliseconds or less to maintain sharpness. Because low persistence produces severe display flicker at standard 60 Hz rates, refresh rates must scale up toward 95 Hz. Furthermore, panels must employ simultaneous global illumination rather than rolling displays to prevent geometric shearing. Wearable optical lenses must also be precisely calibrated to avoid dynamic warping, which instantly shatters presence and induces motion sickness. Spatial Tracking Precision and Latency Budgets (18:41) Abrash stresses that presence operates as an emergent phenomenon: it cannot be realized by optimizing a single component in isolation, and it collapses if any individual technical element is subpar. Tracking systems must deliver six degrees of freedom, capturing translation across the X, Y, and Z axes in addition to rotational orientation. Valve’s testing established that tracking requires position error under 1 millimeter and orientation error under 0.25 degrees across an operational volume of at least 1.5 meters cubed. Latency budgets present an equally uncompromising limit. Valve measures latency strictly from physical head movement to the emission of the final corresponding photon onto the retina. Experiments show that this motion-to-last-photon latency must not exceed 20 milliseconds (with 25 milliseconds as an absolute ceiling) paired with predictive sensor tracking. Exceeding these limits causes the virtual world to feel detached, prompting the subconscious perceptual system to reject the simulation. Uncharted Input, Locomotion, and Game Design (23:35) Developing presence-capable headsets solves only the visual half of virtual reality; interaction models and software design must be built from scratch. Abrash explains that conventional flat-screen gaming mechanics fail in head-mounted environments. Standard first-person artificial locomotion quickly causes severe motion sickness due to vestibular mismatch, forcing designers to investigate alternative paradigms such as slower character movement, confined scale, and intimate object manipulation. Visual rendering art pipelines also require complete re-evaluation. Techniques like normal mapping appear flat and artificial under stereoscopic view, and photorealistic high-frequency textures can look like cheap stage sets. Abrash warns that input devices, spatial 3D audio, body tracking, and haptic feedback represent even harder engineering problems than display optics, requiring years of broad industry experimentation to establish a mature VR design vocabulary. Why Open PC Platforms Must Lead Adoption (25:21) Abrash projects that an affordable consumer headset capable of delivering true presence can arrive by 2015 using slight modifications of existing commodity hardware rather than speculative scientific breakthroughs. He argues this commercial debut will center on personal computers—encompassing Windows, Linux, and OS X—rather than game consoles or mobile devices. First, rendering dual-eye stereoscopic viewpoints at 95 Hz with sub-20-millisecond latency demands massive computational power, which mobile chipsets cannot supply due to thermal and battery limits. Second, consoles remain locked into static, multi-year hardware generations, whereas PC hardware iterates continuously. Backed by Valve's integration of VR support directly into Steam and its collaborative sharing of tracking and optical R&D with external partners like Oculus, Abrash positions the PC platform as the vital testing ground for VR's initial platform shift. Abrash cautions that while an introductory consumer headset is viable within two years, a vast gap remains between first-generation products and ideal hardware. Long-term hurdles—such as solving robust eye tracking, eliminating tether cables through wireless transmission, engineering per-user optical and interpupillary adjustments, and securing customized display panel manufacturing—will require prolonged collaboration across the industry.

精炼阅读摘要

Valve的Michael Abrash在Steam Dev Days 2014上提出,具备低延迟、低余晖与精确定位追踪的消费级虚拟现实(VR)硬件预计将在两年内(约2015年)面世;该技术通过满足人类底层神经感知的严苛标准诱发“临场感”(Presence),并将依托PC生态的高算力推动交互娱乐产业的重大变革。 临场感与人类底层潜意识感知的神经机制 (7:33) Abrash明确区分了“沉浸感”(Immersion)与“临场感”(Presence):沉浸感仅代表用户被虚拟视听环境所包围,而临场感则是人类深层本能认知(即所谓“蜥蜴脑”)确信自己置身于另一物理空间的心理状态。这种体验并非由意识层面的理性思考决定,而是由大脑视觉皮层最早期的低级感知机制(如V1区)自动化处理驱动。Abrash通过暗巷透视球体以及同色方块在不同光影背景下的视错觉实验指出,人脑在信息送达意识之前,便已依据环境上下文自动扭曲并锁定了物体的真实尺寸与色调。 在Valve的VR悬崖实验中,即便受试者理智上清楚自己正佩戴头显站在普通演示室中,潜意识感知仍会强行触发身体反应,导致膝盖本能锁死、无法迈向虚拟悬崖。Abrash强调,临场感具有类似开关触发的非线性特征,能引发躲避虚拟物体或本能伸手触摸等生理动作;VR系统唯有全面满足人类深层感官通道的输入标准,才能让人产生不可抗拒的真实存在感。 硬件指标的严苛门槛与系统突现性 (13:20) Valve的实验表明,临场感并非依赖单一硬件参数的线性提升,而是一种各项技术指标必须同时达标才会诱发的“突现属性”(Emergent Property),且同一套硬件参数也是彻底消除晕动症的关键。在视野(FOV)上,80度是激活周边视觉运动感并诱发临场感的最低起点,110度则能大幅增强感知。分辨率方面,1080p是诱发临场感的底线,在110度广角下单眼1K×1K的角像素密度仅为人眼极限的十分之一,远低于传统显示屏的表现。 更严苛的制约来自显示余晖、刷新率与延迟。人眼前庭-动眼反射(VOR)会在头部转动时锁定视线,传统全余晖显示器在60Hz下以每秒120度慢速转头时,会在视网膜上产生2度的拖影涂抹;因此像素发光时间必须压至3毫秒以内(低余晖)。为了消除3毫秒低余晖带来的可见闪烁,刷新率必须推升至95Hz,并采用整屏同时发光的全局显示(Global Display)以规避滚动扫描造成的画面剪切畸变。此外,系统必须实现位置误差小于1毫米、角度误差小于0.25度的全自由度(6DoF)空间追踪,并将动作到最后一颗光子击中视网膜的延迟死守在20毫秒以内(25毫秒为绝对上限)。 消费级落地路径与短期工程瓶颈 (21:25) Abrash预测,消费级VR头显在2015年完全可以通过对现有商用元器件的工程改良来实现,无需等待遥不可及的基础科学突破。Valve已将自身在低余晖显示、光学校准、透镜设计与空间追踪的研究成果完全开放给Oculus,直接推动了Oculus Rift“Crystal Cove”原型机的研发。然而,实验室样机与量产消费设备之间仍横亘着显著的工程鸿沟。 在光学与显示领域,为了兼顾佩戴重量与工业设计,消费级头显只能依靠单眼1至2片透镜放大智能手机规格的显示屏,这必然引发复杂的几何畸变与色差。由于校正一种光学缺陷往往会恶化另一种缺陷,且无法仅靠自动化程序判定,光学系统必须经过漫长的人眼主观调优。此外,消费级大范围6DoF定位追踪、逐人匹配的瞳距与光学校准流程、用于降低渲染负载的高精度眼动追踪,以及摆脱线缆束缚的低延迟无线视频传输,在当时仍属于尚未完全解决的工程挑战。 软件交互重构与PC生态的算力优势 (23:35) Abrash认为,VR代表着与《Myst》、《Quake》或触控设备相当的平台级代际跃迁,仅靠将传统平铺屏幕游戏简单移植绝不可能取得成功。平面显示器积累的设计范式在VR中全面失效:传统第一人称射击游戏(FPS)的人工平移运动会直接引发严重眩晕,软件设计必须转向低移动速度、有限空间与近距离物体交互;立体视觉下,传统的法线贴图会被立刻看穿为虚假平面,高频贴图易显得像廉价舞台道具,促使美术渲染管线彻底重写。 在运行平台上,Abrash断言PC(包括Windows、Linux和OS X)必然是VR的最优归宿。首先,双眼独立立体渲染、95Hz超高刷新率以及20毫秒以内的超低延迟,带来了极高的计算吞吐负荷,移动芯片受制于功耗散热,而生命周期固定的游戏主机难以提供足够的FLOPS算力。其次,PC拥有快速迭代的硬件周期与无准入门槛的开放开发环境。Valve正将底层VR支持整合入Steam平台,致力于联合外部软硬件开发者共同确立VR时代的全新设计语汇。 Abrash总结指出,消费级高品质VR硬件在2015年面世已具备充分的技术可行性,但交互手柄、触觉反馈、空间音频以及原生游戏语汇等非视觉维度的成熟,仍需全行业经历数年的集体探索。尽管Valve在2014年表态暂无制造自主消费级硬件的计划(但保留视生态需要而介入的可能),其公开研发技术并建设Steam底层设施的核心目标,在于加速促成PC主导的下一代互动娱乐计算平台。

Detailed notes — zh

## Background 该视频由 Steamworks Development 官方频道于 2014 年 2 月 11 日在 YouTube 上发布,记录了 Valve 知名图形工程师 Michael Abrash 在 Steam Dev Days 上的演讲。视频简介指出,Valve 团队已经明确了未来两年内消费级 VR 硬件所能达到的性能水准,并通过高规格原型机验证了其带来的震撼体验,呼吁开发者现在就着手为此开发原生内容。 ## TL;DR Michael Abrash 在演讲中预言,具备强大临场感(Presence)且价格亲民的消费级 VR 硬件将在两年内(约 2015 年)落地,并将彻底颠覆整个娱乐产业。他从底层脑科学与人类视觉感知系统切入,详述了实现临场感并消除晕动症的关键硬件指标:包括双眼 1080p 以上分辨率、低于 3 毫秒的低余晖(Low Persistence)、95Hz 高刷新率、全局显示、精准的光学几何校准、低于 20 毫秒的端到端延迟以及毫米级的 6DoF 空间定位。Abrash 还阐明了 Valve 将自身研发无保留共享给 Oculus 等伙伴的生态策略,并强调凭借顶尖算力与快速迭代,PC 与 Steam 必将成为这场软硬件拓荒革命的绝对核心。 ## Atmosphere & style 整场演讲呈现出一种严谨硬核却又极具感染力的先驱布道风格。Abrash 既没有空洞吹捧未来,也没有故弄玄虚,而是以资深图形老兵的沉稳与坦诚,将晦涩的脑科学知觉实验、光学工程折中与严苛的硬件数据娓娓道来。伴随着从早期与 John Carmack 打造《雷神之锤》的回忆到站在虚拟悬崖边缘膝盖本能锁死的生动叙述,台下开发者能清晰感受到一个技术历史奇点即将来临的悸动与压迫感。 ## Bulletin - [00:00](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=0s) Valve 预测具备真正吸引力且消费级定价的 VR 硬件将在两年内(2015 年)落地,现有工业技术只需扎实工程优化即可达成,无需突破性科学奇迹。 - [13:20](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=800s) 实现 VR 核心体验“临场感”(Presence)必须同时满足一系列硬性指标:视场角需达 80° 至 110°、双眼分辨率最低 1080p、像素余晖时间低于 3 毫秒、刷新率推升至 95Hz、端到端延迟控制在 20 毫秒以内,并具备毫米级 6DoF 空间追踪。 - [04:11](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=251s) Valve 无意自行发售 VR 硬件,而是选择将低余晖与空间追踪等底层 R&D 成果全盘共享给 Oculus,直接推动了 Crystal Cove 原型机的诞生,并致力于将 Steam 打造为 PC VR 的生态中枢。 - [26:04](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1564s) PC 拥有无可比拟的浮点运算能力、散热设计与开放生态,足以支撑双目立体渲染与高刷新率需求,因此 VR 革命必将最先在 PC 平台爆发,而非主机或移动设备。 ## Key takeaways - [00:00](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=0s) Michael Abrash 以 59 个英文单词定调演讲核心:消费级高质量 VR 硬件将在两年内成真,并依托 PC 和 Steam 平台重塑整个娱乐产业。 - [01:15](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=75s) 回顾与 John Carmack 打造《雷神之锤》的历程,Abrash 指出《雪崩》所勾勒的元宇宙中,虚拟现实是尚未完全成真但时机已成熟的最后拼图。 - [02:53](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=173s) Valve 通过自研原型机锁定了 VR 的核心灵魂——“临场感”(Presence),一种让感知底层深信自己已瞬间移动至另一地点的强大感知体验。 - [02:53](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=173s) 解决临场感所需的硬件指标同时也是根治晕动症(Motion Sickness)的关键,两者在底层感知机制上高度统一。 - [04:11](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=251s) Valve 将低余晖(Low Persistence)等核心研究无保留共享给 Oculus,直接促成了 Oculus Crystal Cove 原型机的突破性演进。 - [06:05](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=365s) Valve 当期不计划推出自有品牌硬件,而是聚焦于底层技术研发、玩法原型摸索以及在 Steam 中深度整合原生 VR 支持。 - [07:22](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=442s) 人类视觉处理在初级皮层等下意识层面即完成了深度与尺寸运算,VR 必须精准刺激这些低级知觉处理器才能骗过大脑欺骗感知。 - [10:14](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=614s) 临场感与普通沉浸感(Immersion)存在本质差异:即使理智清楚置身测试室,峭壁边缘的虚拟场景依然会直接触发身体深层的恐慌本能与膝盖锁死。 - [13:20](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=800s) 视场角(FOV)必须在 80° 以上临场感才开始生效,达到 110° 体验大幅飞跃,同时双眼 1080p 是触发临场感的最低分辨率红线。 - [15:02](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=902s) 为避免眼球前庭反射(VOR)造成的画面剧烈拖影,像素余晖必须压缩至 3 毫秒以内,同时需将刷新率推升至 95Hz 以上以彻底消除可见闪烁。 - [16:40](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1000s) 相比逐行扫描导致的拉伸失真,全局显示(Global Display)是消除转头伪影的关键,而仅靠 1 至 2 片光学透镜消除像差是一场极难的工程妥协。 - [18:41](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1121s) 临场感对空间追踪要求苛刻:必须覆盖单边 1.5 米以上空间,达到 1 毫米位置精度和 0.25 度旋转精度,且运动到发光延迟需压至 20 毫秒以内。 - [18:41](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1121s) 临场感是一项整体涌现属性,任何单一参数拉跨都会导致体验彻底崩塌,所有硬件指标必须同时跨过及格线。 - [21:14](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1274s) 2015 年实现消费级临场感头显只需要扎实的工程落地与元件定制,无需不可控的科学突破,但输入交互、3D 音频与眼动追踪仍需长期攻坚。 - [24:02](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1442s) 传统 FPS 摇杆移动和法线贴图美术管线在 VR 下彻底失效,VR 急需建立从零定制的全新游戏交互机制与视觉语言,跨平台移植无法取得成功。 - [26:04](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1564s) PC 拥有快速迭代的硬件周期、无尽的峰值算力与自由的开发者生态,在性能上对主机和移动设备形成降维打击,是 VR 生态的首发主战场。 ## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=0s) 59 字精炼总结:VR 的变革即将在两年内发生 在 Steam Dev Days 的讲台上,Valve 的 Michael Abrash 一开场就对台下人头攒动的景象感到兴奋。他开门见山地表示,自己准备了一场干货满满、展望激动人心的未来的精彩演讲,但在展开所有细节之前,他选择先用精确的 59 个英文单词对全篇演讲的核心内容进行总结: “Compelling consumer priced VR hardware is coming, probably within two years. It's for real this time. We've built prototypes, and it's pretty incredible. Our technology should work for consumer products. VR will be best and will evolve most rapidly on the PC. Steam will support it well, and we think it's possible that it could transform the entire entertainment industry.”(具有吸引力且消费级价格的 VR 硬件即将到来,可能就在两年之内。这一次是玩真的。我们已经打造了原型机,效果惊艳。我们的技术应当适用于消费级产品。VR 在 PC 上表现最好且演进最快。Steam 会为其提供良好支持,我们认为它有可能彻底改变整个娱乐产业。) Abrash 强调,哪怕参会者今天听完只记住了这 59 个字,这场分享的时间也是花得极其值得的。 --- ### [01:15](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=75s) 个人之路:从《雪崩》到与 John Carmack 打造《雷神之锤》 Abrash 回顾了自己进入虚拟现实领域的漫长曲折路径。他感慨道,若不是 20 年前有幸遇到几位远见卓识的先驱,他可能永远不会迈出这第一步。当时他在微软工作,精通计算机网络,也写过简单的 3D 渲染软件包并见识过工作站上的 3D 图形渲染,加上平时阅读大量科幻小说,对“虚拟世界”的概念早有了解。但在当时,无论 3D 还是网络技术,都让他觉得虚拟世界更像是虚构文学,而非可以认真付诸实践的事物。 转折点出现在他阅读了尼尔·斯蒂芬森(Neal Stephenson)的科幻巨作《雪崩》(Snow Crash)。当 Abrash 试图评估构建“元宇宙”(Metaverse)所需的技术时,他乐观地认为自己大概掌握了用现有技术实现其中 80% 内容的思路(他自嘲当时太过乐观,因为至今他还在等待自己的第一次虚拟现实剑斗)。尽管如此,《雪崩》让他清醒地意识到,联机 3D 虚拟世界的时机已经成熟。随后他与 John Carmack 碰面,Carmack 连续不间断地讲了整整两个小时,详述如何通过在持久运行的服务器上构建 3D 虚拟世界、将服务器相互联结并允许任何人创建自己的世界,进而汇聚成网络空间(Cyberspace)。 Abrash 瞬间看到了未来的轮廓,下定决心要参与其中。他跳槽加入与 Carmack 并肩作战,诞生了划时代的《雷神之锤》(Quake)。如今,数以千万计的玩家每天沉浸在联网的 3D 世界中,而这一切都源自当年 Neal 与 John 为他指明了方向。虽然 Abrash 自谦并非有远见的人,但他希望今天也能为在场的开发者指明元宇宙中尚未完全成真的那一板块——虚拟现实。 --- ### [02:53](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=173s) 临场感、消除晕动症与 Valve 的技术探索 对于当时刚刚引发热潮的 Oculus Rift DK1,Abrash 推测台下大多数开发者可能仅将其视为一个有趣的“新奇玩意”(interesting curiosity)。但他郑重提醒大家必须密切关注,因为 VR 产生巨大冲击的速度将远超想象,而 Valve 正在全力加速这一天的到来。 大约两年前,Valve 将 VR 锁定为实现全新颠覆性体验的关键方向。经过长期的研发,他们打造出了先进的头戴式显示器(HMD)原型机以及一系列配套演示 Demo,这些成果共同带来了一种极其强烈的感受——“临场感”(Presence)。Abrash 解释道:“Presence is an incredibly powerful sensation, and it's unique to VR. There's no way to create it in any other medium.”(临场感是一种极其强大的感知体验,并且是 VR 所独有的。在任何其他媒介中都无法创造出这种感觉。)它会让体验者在虚拟现实中深信自己已经置身于另一个地方,产生如同瞬间移动(teleportation)般的奇幻感。 更关键的是,能够催生临场感的硬件基础,在抑制晕动症(motion sickness)方面同样极为有效。Valve 的原型机 Demo 证明,只要配备正确的硬件,不仅能创造出完全不让人产生眩晕恶心感的极致体验,而且这套技术完全能够在两年内落地为消费级产品。 ```mermaid flowchart TD accTitle: Valve VR R&D to Consumer Hardware accDescr: Flowchart showing how Valve research into low persistence and tracking feeds into partners to enable consumer presence VR. subgraph Valve_RD["Valve 研发成果"] PROTO["原型机研发 & Demo"] TECH["核心技术突破<br/>(低余晖 Low Persistence / 空间追踪)"] end subgraph Experience["感知体验落地"] PRESENCE["确立强烈的临场感 (Presence)"] NO_SICK["消除晕动症 (Anti-Motion Sickness)"] end subgraph Industry_Delivery["产业落地"] PARTNER["行业合作与反馈<br/>(如 Oculus Crystal Cove)"] CONSUMER["两年内推出消费级 VR 硬件"] end PROTO --> TECH TECH --> PRESENCE TECH --> NO_SICK PRESENCE --> PARTNER NO_SICK --> PARTNER PARTNER --> CONSUMER ``` --- ### [04:11](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=251s) 携手 Oculus:从 Crystal Cove 原型到 PC 算力主战场 谈及谁最有可能在两年内将这种级别的硬件推向消费者,Abrash 直言 Oculus 是最领头的候选者。Oculus 成功让 VR 重新成为全行业的热门话题,并且已经出货了超过 50,000 台 Rift DK1。尽管 DK1 还不具备为绝大多数人提供强临场感的能力,但 Oculus 刚刚公布的新原型机“Crystal Cove”迎来了跨越式升级:具备更高分辨率、更低延迟、低余晖(low persistence)以及位置平移追踪(translation)。 Abrash 透露了 Valve 在背后的巨大推动力:Valve 的目标是在 PC 上实现出色的 VR 生态,因此毫无保留地将自身 R&D 成果分享给了 Oculus。Valve 邀请 Oculus 团队体验原型机与演示,拆解硬件架构原理,并直接对 Oculus 的硬件设计提供反馈。特别是通过向 Oculus 展示 Valve 自己的低余晖原型机,彻底说服了对方采纳这一特性,从而消除了运动模糊,这直接促成了 Crystal Cove 的问世。双方还在追踪定位算法、显示屏、透镜以及系统校准等方面展开深度合作。Abrash 坚信,只要 Oculus 执行得当,他们极有可能在未来两年内于 PC 平台交付具备强临场感的产品。 Abrash 进而阐述了为什么这场革命必然首先在 PC 上爆发: - 临场感对带宽和低延迟有着近乎苛刻的要求,必须依靠头戴显示设备直连具备高负载 3D 渲染能力的本地计算硬件。 - 电视、电影广播或流媒体等缺乏强大本地计算能力的技术载体完全无法胜任。 - PC 平台(包括 Linux、Windows、OS X)拥有世界上最庞大的浮点运算能力(FLOPS),是孕育高品质 VR 的最佳土壤,甚至可能推动整个娱乐产业的重心从传统媒介彻底倾斜向计算机娱乐。 --- ### [06:05](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=365s) Valve 的战略定位与西雅图体验之约 Abrash 详细阐述了 Valve 在推进 VR 方面的三大具体战略举措,并明确回应了关于 Valve 是否会自己造硬件的传闻: - **Steam 平台支持**:如 Joe Ludwig 在另一场演讲中所介绍,Valve 正在将底层的 VR 支持直接原生整合进 Steam 系统中。 - **推动 PC 硬件进化**:持续开展内部前沿研发,并与 Oculus 等行业先驱展开全面技术合作。 - **探索软件玩法原型**:在内部制作大量软件与交互原型,摸索究竟什么在 VR 空间中才算真正“好玩”,并将积累的设计经验持续分享给开发社区。 关于硬件,Abrash 直言:“We have no current plans to ship VR hardware ourselves, but that could change in the future.”(我们目前没有自行发售 VR 硬件的计划,但这在未来可能会发生改变。)眼下 Valve 的核心诉求是探寻 VR 的乐趣所在,并做一切必要的工作去促成 PC 与 Steam 上的 VR 生态成熟。 针对现场开发者,Abrash 坦言建立认知的唯一方式是亲身体验。但遗憾的是,Valve 的 VR 体验 Demo 每次需要半小时,并且依赖于一间专门布置的房间(包含特殊的墙面追踪标记等环境要求),Dev Days 现场只搭了两间体验室,无法满足绝大多数参会者的体验需求。为此他公开做出了承诺:任何 Steam 合作伙伴在接下来的几个月内如果前往西雅图,均可直接向他的个人邮箱(`mabrash@valvesoftware.com`)发送邮件预约时间,亲自到 Valve 办公室体验这套颠覆认知的前沿原型。 --- ### [07:22](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=442s) 欺骗大脑:人类视觉底层的感知机制 在真正深入讲解“临场感”之前,Abrash 指出必须先建立关于人类认知系统的基础科学认知。Valve 在长期研究中发现,要实现无可替代的沉浸体验,核心在于攻克人类生理上的“低层级知觉系统”(low-level perceptual systems)。这些系统运作在人的意识知觉之下,潜意识地处理周围环境输入。 为了说明大脑低层感知如何在意识介入之前做出决断,Abrash 展示了一张经典的视觉错觉幻灯片(长廊球体透视错觉)。场景通过纵深阴影、环境光照、透视线收敛以及纹理梯度等线索,营造出极强的空间距离感: - 场景中有一前一后两个球体,测试任务是让受试者使用键盘方向键调节前方的球体大小,直到它在屏幕二维平面上看起来与远处的球体一样大。 - 实验中几乎所有人都会毫无例外地将前景的圆调得明显偏大。 - Abrash 指出,幻灯片里位于走廊深处的圆看起来明显更大,但实际测量却显示,前景的圆物理尺寸比背景的圆大了将近 20%。 Abrash 引用功能性磁共振成像(fMRI)的脑科学研究解释其深层原理:早在视觉信号从视网膜传递到意识层知觉之前,初级视觉皮层(V1)等极早期的视觉处理区域就已经将看起来更远的球体映射到了大脑中更大的神经元激活区域。换言之,在你的意识还没反应过来之前,你大脑底层的硬件处理器就已经依据立体与透视线索完成了尺寸测定。要想在 VR 中达成真正的现实欺骗,就必须从这种本能底层的处理机制入手。 ### [09:20](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=560s) 低级视觉线索与欺骗感知 人类对现实世界的感知很大程度上是由大脑未直接意识到的低级处理机制塑造的。Michael Abrash 展示了一个视错觉示例:画面中有两个字母“X”,其中心色块在肉眼看来一个像是灰色,另一个像是深黄色;然而一旦将背景遮蔽、完全不改动中心色块的实际颜色时,人们会惊讶地发现两者其实是完全相同的颜色。这是因为人类视觉系统为了在各种光照条件下维持色彩恒常性,在低级色彩处理过程中引入了巨大的上下文关联。 这种处理过程完全处于下意识层面,就像人们无法察觉大脑在自动篡改视觉中物体大小的判断一样。如果虚拟现实能够提供足以正确刺激这些低级处理器的输入信号,用户就会发自内心地感到自己“切实身处某个地方”;如果做不到这一点,用户就会在某种难以言喻的感知层面上与眼前所见的场景产生疏离感。 ```mermaid flowchart LR accTitle: Perception Mechanism of Reality and VR accDescr: Diagram showing how external sensory inputs trigger low-level perceptual processing and sub-conscious belief. subgraph Input["感知输入层"] A["外界视觉与运动信号"] end subgraph Subconscious["无意识处理层"] B["低级视觉处理器<br/>(色彩恒常性/尺寸校正)"] C["前庭与运动感知协调"] end subgraph Cognition["意识与身体反应"] D["蜥蜴脑(深层感知)<br/>产生'身临其境'信服感"] E["身体本能直觉反应<br/>(膝盖锁死/下意识躲闪)"] end A --> B A --> C B --> D C --> D D --> E ``` ### [10:14](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=614s) 峭壁边缘 Demo 与“临场感”(Presence)的本质 为了说明这一点在 VR 中有多么震撼,Michael Abrash 举了一个他们团队开发的 Demo。在 Demo 中,体验者站在一块像跳水板一样的石质悬空壁架边缘,俯瞰下方巨大的垂直落差。虽然房间四周的盒子贴图极其粗糙——甚至贴满了过时的网页截图(上面还写着“Facebook 即将上市”的旧闻),在普通显示器屏幕上看毫无感觉,但只要戴上头显站在 VR 中的悬崖边缘,生理反应便会彻底接管身体。 Michael Abrash 坦言,哪怕理智上百分之百清楚自己正站在 Valve 的测试室里、头上戴着头戴式显示器(HMD)、眼前面向的不过是一个贴图简陋的虚拟盒子,他的膝盖依然会瞬间锁死,反应就像当年站在帝国大厦楼顶边缘一模一样。更关键的是,这种生理恐慌感并不会随着时间和反复测试而消退。 在虚拟现实研究领域,这种在虚拟环境中产生“真真切切身处现实某地”的感觉被称为**临场感(Presence)**。Michael Abrash 强调,Presence 与一般的**沉浸感(Immersion)**有着本质区别:沉浸感仅仅意味着你感到自己被虚拟世界的图像所包围;而 Presence 则意味着你在感知层面上彻底确信自己就在那个虚拟世界之中。用言语去描述 Presence 总是显得苍白无力,它表现为:明知眼前空无一物却忍不住伸手想去触碰一个立方体;天花板垂下一根管子时头会本能地下意识躲闪;头顶悬挂巨石时感到莫名焦虑;面对悬崖边缘无论如何不敢迈出步子;摘下头显的瞬间,甚至会因为重返现实世界而产生瞬间的定向失真。 关于这种体验对大脑深层的震撼,Michael Abrash 总结道: > “It's more than just looking at someplace interesting it's flipping the switch that makes you believe deep in your lizard brain that you are someplace interesting.” 不同体质的人对 Presence 的感知阈值存在显著差异,但随着 Valve 硬件原型的持续迭代,绝大多数体验者的反馈都在不断增强。在 Valve 看来,伟大的 VR 体验必须建立在 Presence 之上,打造足以唤起强烈 Presence 的硬件,正是开启整个 VR 时代成功的钥匙。 ### [13:20](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=800s) 视场角(FOV)与分辨率的硬件门槛 引发强烈的 Presence 并不仅仅是某一项技术达标,而是需要一整套关键硬件参数同时跨过及格线。巧合的是,这些能够触发 Presence 的技术要素,恰好也是减轻甚至消除晕动症(Motion Sickness)的关键。晕动症通常源于眼睛看到的视觉运动与前庭系统(内耳平衡感官)感受到的身体加速度发生冲突,消除这些硬件感知瑕疵能直接改善这种感官撕裂。 在显示层面,首要的两个指标是视场角与分辨率: - **视场角(FOV)**:宽广的视场角不仅带来沉浸感,更为外周视觉提供了关乎运动判断、身体平衡和环境态势感知的核心线索。根据实验,Presence 在约 80° FOV 时开始起效;当扩展到测试过的最大 110° FOV 时,临场体验出现了极其显著的飞跃。 - **分辨率与像素密度**:VR 显示面临着极为苛刻的挑战,因为超大视场角相当于把屏幕像素用放大镜强行拉扯到整个视野中。一台具备 110° FOV、单眼分辨率为 1K×1K 的头显,其每度像素密度(PPD)仅有大屏电视的约七分之一、人眼视力极限的约十分之一,其像素密度甚至低于当年运行在 320×200 分辨率下的初代《雷神之锤》(Quake)。测试表明,双眼达到 1080p 是触发 Presence 的最低门槛,未来若能采用 1440p 乃至 2160p 规格的小尺寸面板,视觉质量将迎来质的飞跃。 ### [15:02](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=902s) 余晖时间(Low Persistence)与高刷新率 显示技术中另一个至关重要却常被传统屏幕忽视的概念是**余晖时间(Persistence)**,即每个像素在每一帧中保持点亮发光的时间长度。在电视、手机和电脑显示器上,全余晖(Full Persistence)并无大碍,但在 VR 中,它是导致画面严重拖影与模糊的头号元凶。 这背后的生理根源在于人类的前庭眼动反射(Vestibulo-Ocular Reflex, VOR)。当头部快速转动时,VOR 机制会驱动眼球以相同速度反向代偿运动,以保持视网膜上的图像稳定。如果头显采用传统的全余晖显示(如 60Hz 下整整发光 16.6ms),在头部以 120°/秒这种温和速率转动时,像素图像会在持续发光的过程中横向扫过视网膜,单帧内造成整整 2 度的视网膜拖影模糊(Smear)。实验证明,在 110° FOV、1K×1K 分辨率下,必须将余晖时间压缩至 **3 毫秒(ms)或更短**,才能维持足够的画面清晰度并保留 Presence。 然而,一旦将显示器切换到低余晖(短脉冲发光),在传统的 60Hz 刷新率下便会产生极其剧烈的全视野可察觉闪烁。为了彻底消除闪烁,Valve 定制打造了当时最快的低余晖头显原型,将刷新率直接推升至 **95Hz**。以 95Hz 的超高帧率稳定进行双眼立体渲染(Stereo Rendering),对当前的图形管线和计算算力提出了极其艰巨的挑战,随着未来面板分辨率的攀升,这一性能压力还将进一步加剧。 ### [16:40](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1000s) 全局显示与光学透镜设计困境 除了余晖与刷新率,像素的曝光方式同样决定了虚拟画面的真实度: - **全局显示(Global Display)与滚动显示(Rolling Display)**:目前绝大多数商业面板采用逐行扫描的“滚动显示”,这在用户眼球或头部转动时,会在视野中产生恼人的压缩、拉伸以及几何倾斜失真。Valve 原型机采用所有像素瞬时同步点亮的全局显示,彻底避免了这些伪影。虽然理论上可以通过根据眼睛运动实时校正帧缓冲区来补偿滚动显示的畸变,但在低延迟眼动追踪尚未攻克的背景下,全局显示是当时唯一经过验证切实可行的解决方案。 - **光学透镜的工程妥协**:VR 光学系统看似简单——由手机面板加上放大透镜组成,但其实是一场残酷的折中博弈。受限于头显的重量、视场角和工业设计,单眼最多只能容纳 1 到 2 片透镜。由于光学像差(Aberrations)与几何畸变的存在,若想达到完美的光学矫正,需要由巨大镜片组成的复杂透镜组(最前方的两片透镜直径超过一英尺,整套总重量将超过 5 磅,完全不可能戴在头上)。因此,在仅用 1 至 2 片镜片的严苛限制下,修复一种光学缺陷往往意味着恶化另一种缺陷。 关于光学透镜设计中无法依赖纯数学模拟的实际难点,Michael Abrash 指出: > “The only test that matters is actually using the lens in VR which is time consuming subjective and varies from person to person.” 在焦距、镜片尺寸、视距与大视场角的庞大参数空间中寻找最优解异常繁重,尽管 Valve 已经打磨出了能够支撑 Presence 的透镜组合,但光学工程的前沿依然充满了巨大的探索空间。 ### [18:41](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1121s) 光学校准、精准追踪与超低延迟:临场感的技术基石 光学系统校准看似是个次要细节,但事实证明它是极其关键且挑战巨大的难题。人类的视觉系统对哪怕极其微小的误差都异常敏感,尤其在涉及头部运动与直线几何时尤为明显。一幅静态下看起来无可挑剔的画面,一旦转动头部,就可能产生极其可怕的波纹扭曲;这种画面形变会瞬间打破临场感(Presence),并在眨眼间诱发严重的晕动症。更糟糕的是,如果光学几何校准没有达到绝对精准,研发人员根本无法定位其他系统瓶颈。例如在 Valve 团队彻底攻克校准问题之前,微小的追踪抖动甚至无法被察觉,因为图像本身的畸变晃动掩盖了追踪缺陷;唯有当校准足够精确后,追踪系统的问题才暴露出来。因此,对透镜特性的精确建模与对渲染图像的精准校正处理,是整个临场感链条中绝对不可或缺的一环。 即便具备了产生临场感所需的视觉质量,仅凭画面本身依然无法成功,图像必须以人类感知系统完全接纳的方式呈现出来。这要求系统具备坚如磐石的头部追踪能力,不仅能追踪俯仰、偏航与翻滚的旋转姿态,还必须精确报告 X、Y、Z 轴的三维平移位置。Valve 在实验中发现,要实现临场感,追踪系统的精度必须达到 1 毫米的位置精度和 0.25 度的角度精度,并且追踪范围至少需要覆盖单边 1.5 米以上的立方体空间。实现这种性能远比听起来要困难得多,市面上当时没有任何消费级设备能接近这一指标。Valve 在现场演示的原型系统虽然达成了目标,但其依赖墙壁张贴二维码标记的方案显然不适合普通家庭。团队目前正在研发几种不需要在家里贴满墙纸的新型追踪方案,尽管成败未定,但已足够让人相信消费级追踪系统在不久的将来切实可行。 除了位置准确,虚拟画面还必须在正确的时间出现在正确的位置,这对运动到发光的端到端延迟(Motion-to-Photon Latency)提出了极苛刻的要求——即从头部发生物理位移开始,到屏幕上因该动作而发射出最后一颗光子为止的时间差。Valve 验证发现,在配合优秀预测算法的前提下,20 毫秒的延迟表现良好,放宽到 25 毫秒或许也能勉强接受;一旦超出这个界限,虚拟世界就不再像被牢牢“钉”在原地,感知系统便不再相信眼前的事物是真实的。 Michael Abrash 强调,临场感绝不是上述某单一特性的产物,而是在所有技术指标同时达标后所涌现出的整体心理状态:“Presence is not a property of any one of the elements I've discussed; it's a property that emerges when all the elements are good enough.” 只要有一个环节拉跨,临场感就会瞬间瓦解。 - 光学几何校准:若未校准,静态完美的场景在转头时会产生剧烈扭曲并迅速致晕。 - 六自由度(6DoF)追踪要求:位置精度 1 毫米,旋转精度 0.25 度,追踪范围不小于 $1.5\text{m} \times 1.5\text{m} \times 1.5\text{m}$。 - 端到端延迟阈值:从头部运动到发光必须在 20 毫秒以内(极限放宽至 25 毫秒),配合前向动作预测。 - 临场感的不可分割性:“Presence can't be induced if even one of the key elements is subpar.” ```mermaid flowchart TD accTitle: 临场感达成的技术要素链路 accDescr: 展示光学系统、精准追踪和超低延迟共同作用,促成人类视觉感知系统产生临场感。 subgraph Hardware["硬件底层与传感器输入"] OPT["精确光学建模校准<br/>(消除转头波纹扭曲)"] TRK["6DoF 精密追踪<br/>(1mm 精度 / 0.25° 精度 / 1.5m 空间)"] LAT["运动到发光超低延迟<br/>(≤20ms 并配合预测)"] end subgraph Perception["人类感知与心理认知"] VIS["视觉系统接纳虚拟几何与动态"] STAB["虚拟空间被感知为绝对锚定"] end OPT --> VIS TRK --> STAB LAT --> STAB VIS & STAB --> PRESENCE["临场感 (Presence) 涌现"] ``` --- ### [21:14](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1274s) 2015 消费级硬件路线图与未解的技术版图 根据 Valve 原型机的研发数据与产业技术推演,硬件在未来几年完全有能力支撑消费级临场感。Michael Abrash 明确判断,基于 Valve 的原型系统与前瞻预测,工业界完全有能力在 2015 年制造出符合上述临场感规格的消费级头显设备。虽然远期 VR 仍有巨大提升空间,但那需要更长时间与更底层的硬件研发;相反,在 2015 年指标清单上的所有项目,都只需对现有工业技术进行相对微小的调整改良即可实现,无需任何科学突破或技术奇迹,只需要扎实的工程落地:“No breakthroughs or miracles are needed, just solid engineering.” Valve 打造的原型头显之所以能提供强烈的临场感,是因为它采用了经过改良的现成商用元件,为规模化、低成本制造指明了路径。然而,Abrash 坦言研发原型机(R&D prototype)与量产出货产品(shipping product)之间存在巨大鸿沟,仍有大量工程转移工作要做。除了制造成本与工规设计,整套系统的每一项基础参数仍有漫长的演进之路:比如分辨率还可以提升 100 倍,视场角(FOV)可以更广,延迟可以更低,光学设计尚远称不上完美,因人而异的瞳距与透镜对齐机制极具挑战,头部追踪尚未拿出能直接量产的消费级方案,而能大幅强化临场感的眼球追踪(Eye Tracking)技术更是远未成熟。此外,摆脱线缆束缚走向无线化、推动面板厂商联合定制具备低余晖(Low Persistence)、全局显示(Global Display)与超低延迟的专用显示面板,都是需要行业协同攻坚的硬骨头。 在视觉之外,非视觉层面的 VR 技术挑战更为深远。空间三维音频(3D Audio)、触觉反馈(Haptics)、全身动捕(Body Tracking)以及专属交互输入,对于临场感同样具有决定性影响,而这些问题甚至比打造头显本身更加庞大和棘手。Michael Abrash 强调:“In particular, VR input and its interaction with game design is at the core of the experience and almost completely unexplored.” 这一领域的研究需要整个游戏行业倾注多年的共同努力。 - 2015 消费级可行性:基于修改现有商用零部件,完全可在无奇迹突破的前提下量产实现临场感。 - 视觉极限演进空间:分辨率上限达当前 100 倍,光学镜片、个性化透镜对焦与眼动追踪仍需攻关。 - 显示定制需求:必须与面板供应商深度合作,实现低余晖、全局刷新的专用面板。 - 非视觉交互挑战:3D 音效、力反馈触觉、身体追踪和输入交互是比头显显示更难解决的课题。 --- ### [24:02](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1442s) 重构交互、玩法与视觉语言:软件层面的新拓荒时代 硬件铺平道路后,VR 革命的成败最终将由软件决定,而现阶段针对 VR 特性量身打造的杀手级体验还几乎是一片空白。传统的游戏规则无法直接搬进 VR 世界,以第一人称射击游戏(FPS)的移动机制为例,传统高速平移旋转的运动方式不仅极易引发严重眩晕,还会立刻摧毁临场感。经验表明,慢节奏的平移结合丰富的近身物理交互可能更适合 VR,游戏设计师必须围绕这一特性探索全新的趣味规则。就如同当年 3D 图形技术催生了第一代 FPS 交互语汇一样,VR 同样需要一套全新的游戏交互词典。 在多人联机领域,VR 将带来远比屏幕游戏深远且强烈的社会学震荡。与其他玩家共同置身于同一个具有物理真实感的虚拟空间中,有望创造出人类历史上最震撼的在线社交体验。这就像早期局域网死亡竞赛(Deathmatch)、MOD 模组社区以及常驻服务器团队对抗的演化历程一样,VR 联机体验将以百倍的烈度重演这种范式转移。与此对应,视觉美术资产的制作规范也面临彻底重构。在传统显示器上精致华丽的贴图场景,放进 VR 里可能会像廉价的舞台道具背景;相反,结构简单的几何场景却可能因为立体深度而呈现出不可思议的真实感。法线贴图(Normal Maps)在立体双目视差下会彻底穿帮失效,传统纹理材质也时灵时灵,美术团队必须为此摸索全新的视觉表现语言。 回顾计算机娱乐的历史,每一次平台跃迁都会催生开创性的软件体验——正如当年 CD-ROM 催生了《神秘岛》(Myst)、3D 算力催生了《雷神之锤》(Quake)、体感设备催生了《Wii Sports》以及移动触屏催生了《愤怒的小鸟》(Angry Birds)。VR 具备成为有史以来最大平台跃迁之一的潜力,然而简单的跨平台移植(Port)最多只能带来廉价的新鲜感,伟大的 VR 体验必须是彻底从零定制的软件。Michael Abrash 指出:“The first game that nails a VR experience is going to be remembered fondly for a long, long time.” - 移动机制与防眩晕:传统 FPS 摇杆移动极易致吐,VR 需转向慢速移动及近距离空间交互。 - 社交临场感:在逼真虚拟空间内实现多人共存,将带来超越传统多人游戏的社交颠覆。 - 美术资产颠覆:法线贴图失效,复杂精细的高光纹理易呈现“廉价舞台布景感”,需重建光影与几何语言。 - 平台跃迁的历史规律:类比《Myst》《Quake》《Wii Sports》《Angry Birds》,必须原生定制而非简单移植。 --- ### [26:04](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1564s) 为什么 PC 与 Steam 将领跑 VR 革命 首款真正奠定 VR 体验的里程碑式大作几乎注定会诞生在 PC 平台。Michael Abrash 梳理了 PC 领跑该领域的几大核心逻辑:首先,PC 平台的 VR 硬件迭代速度极快(正如 Oculus Rift 展现出的快速进化),而主机平台即使选择支持 VR,其硬件规格在数年周期内也将处于完全静止的状态。其次,开放的 PC 生态汇聚了远超主机平台的软硬件开发者,他们拥有完全自由的实验环境来摸索前沿技术。再者,VR 对算力的索求是无止境的——以双目立体渲染、95Hz 高刷新率的基准线来看,高端 PC 的性能已经大幅拉开与家用主机的差距,且差距在未来几年还会继续扩大;受限于功耗与发热物理定律,移动设备在算力上更是永远无法与 PC 比拟。最后,Valve 已经攻克了将临场感带入消费级 VR 的核心技术,并在 Steam 中加入了原生 VR 支持,Valve 将持续推动 PC 与 Steam 平台上的 VR 生态发展,使其在可预见的未来牢牢占据行业核心。 在演讲尾声,Abrash 留下了掷地有声的结论:在 2015 年以平民消费级价格打造出一套出色的 VR 硬件系统,不仅是完全可行的,而且它已经万事俱备、蓄势待发;即使不在 2015 年发生,也必然紧随其后成为现实。未来几年 PC 虚拟现实的发展浪潮,将是游戏史上最令人振奋的篇章。Valve 正在并将继续毫无保留地向全行业分享其研发所得,因为 VR 的未知领域无比广袤,单凭任何一家公司都无法独力探索完成,它需要所有开发者一同投身这场技术拓荒。 - 硬件迭代优势:PC 平台硬件快速进化,完全避开了主机长达数年的固定生命周期瓶颈。 - 算力硬性门槛:双目渲染在 95Hz 以上运行,高端 PC 在功耗、散热与峰值性能上碾压主机与移动端。 - Steam 平台驱动:Valve 将在底层技术与 Steam 平台双向持续投入,使其成为 VR 生态的孵化中枢。 - 时代定论:“A great VR system at a consumer price in 2015 is more than just possible; it's sitting there waiting to happen, and it will happen.” ### [27:35](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1655s) 携手开启娱乐革命:迈向双向交流与未来探讨 Michael Abrash 在演讲的尾声向在场的开发者们发出了真诚的合作邀请。他强调,推动虚拟现实的发展绝非单方面的输出,而应该是一个紧密互动的过程。他热切期盼能够与整个开发社区携手共进,攻克技术难关,共同将虚拟现实打造成人类历史上最伟大的娱乐革命之一。 Abrash 坦言,关于 VR 的未来和技术细节,还有极其丰富的内容值得深入探讨。他非常期待在接下来的问答互动(Q&A)环节以及自己的个人博客上,与大家继续就这些前沿话题展开对话与思辨。伴随着真挚的致谢,全场响起了热烈的掌声,本次主题演讲也在此圆满落幕。 正如 Michael Abrash 所总结的那样:“A two-way exchange so together we can make VR one of the great entertainment revolutions there's a lot more to talk about and I look forward to continuing the conversation with you at the Q&A and on my blog thank you.” - **合作愿景**:倡导 Valve 与开发者建立双向交流机制,共同推动 VR 娱乐形态的落地与普及。 - **后续交流渠道**:开发者可通过现场问答环节(Q&A)以及 Michael Abrash 的技术博客持续跟进深入探讨。 ## Notable quotes > [02:53](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=173s) Presence is an incredibly powerful sensation, and it's unique to VR. There's no way to create it in any other medium. — Michael Abrash > [18:41](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1121s) Presence is not a property of any one of the elements I've discussed; it's a property that emerges when all the elements are good enough. — Michael Abrash > [24:02](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1442s) The first game that nails a VR experience is going to be remembered fondly for a long, long time. — Michael Abrash > [26:04](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1564s) A great VR system at a consumer price in 2015 is more than just possible; it's sitting there waiting to happen, and it will happen. — Michael Abrash > [21:14](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1274s) In particular, VR input and its interaction with game design is at the core of the experience and almost completely unexplored. — Michael Abrash > [16:40](https://www.youtube.com/watch?v=G-2dQoeqVVo&t=1000s) The only test that matters is actually using the lens in VR which is time consuming subjective and varies from person to person. — Michael Abrash ## Mentioned Michael Abrash, John Carmack, Neal Stephenson, Joe Ludwig, Valve, Oculus, Steam, Steam Dev Days, Oculus Rift DK1, Crystal Cove, 雪崩 (Snow Crash), 雷神之锤 (Quake), 神秘岛 (Myst), Wii Sports, 愤怒的小鸟 (Angry Birds), Facebook, PC, Linux, Windows, OS X ## Video description We've figured out what affordable VR hardware will be capable of within a couple of years, and assembled a prototype that reveals that that level of VR hardware is capable of stunning VR experiences. That hardware is almost certainly going to appear in that timeframe, and it will be worth starting to develop for it now. This talk will discuss what that hardware is, and what it makes possible. Speaker: Michael Abrash (Valve) Slides (PDF): http://media.steampowered.com/apps/steamdevdays/slides/vrshouldbe.pdf Slides (PowerPoint): http://media.steampowered.com/apps/steamdevdays/slides/vrshouldbe.ppt [Source](https://www.youtube.com/watch?v=G-2dQoeqVVo)

Source quotes

[2:53] Michael Abrash: Presence is an incredibly powerful sensation, and it's unique to VR. There's no way to create it in any other medium. [18:41] Michael Abrash: Presence is not a property of any one of the elements I've discussed; it's a property that emerges when all the elements are good enough. [24:02] Michael Abrash: The first game that nails a VR experience is going to be remembered fondly for a long, long time. [26:04] Michael Abrash: A great VR system at a consumer price in 2015 is more than just possible; it's sitting there waiting to happen, and it will happen. [21:14] Michael Abrash: In particular, VR input and its interaction with game design is at the core of the experience and almost completely unexplored. [16:40] Michael Abrash: The only test that matters is actually using the lens in VR which is time consuming subjective and varies from person to person.

Source quotes

[25:21] Michael Abrash: The first game that nails a VR experience is going to be remembered fondly for a long, long time. [20:21] Michael Abrash: Presence can't be induced if even one of the key elements is subpar. [23:35] Michael Abrash: In the end, a whole new VR gameplay vocabulary will need to be developed, just as was the case with FPSs. [2:54] Michael Abrash: If Oculus executes well—and so far they seem to be—I think they could well deliver strong presence on the PC within the next two years.

Original transcript

[0:00] thank you for coming to what VR could [0:03] should and almost certainly will be [0:06] within two years please welcome Michael [0:09] abrash from [0:12] valve wow pretty exciting to see this [0:15] many people excited about [0:21] VR ouch um so I have a nice talk [0:26] prepared for you today with lots of meat [0:28] and a look at an exciting future and I'm [0:30] going to get to it in just a minute but [0:32] first I thought I'd sum the whole thing [0:33] up in just 59 words and I'm sure someone [0:36] out there is going to be counting so I'm [0:38] just going to read these compelling [0:40] consumer priced VR Hardware is coming [0:42] probably within two years it's for real [0:45] this time we've built prototypes and [0:47] it's pretty [0:48] incredible our technology should work [0:50] for Consumer products VR will be best [0:53] and will evolve most rapidly on the PC [0:55] steam will support it well and we think [0:58] it's possible that it could transform [0:59] the entire entertainment industry if you [1:02] come away today with nothing but those [1:04] 59 words your time will have been well [1:06] spent still there's a lot more to be [1:08] said on the subject so let's look at the [1:10] future of VR in more detail did this [1:12] actually work [1:15] yes my personal path to working on [1:17] virtual reality has been a long and [1:19] winding one and without the Good Fortune [1:21] to encounter a couple of Visionaries 20 [1:23] years ago I might never have taken the [1:25] first step on that path back then I [1:27] understood networking after all I worked [1:29] at my Microsoft and use the network [1:31] constantly and I had seen demos of 3D [1:33] rendering on workstations and had [1:35] written a simple 3D package myself so I [1:37] knew about realtime 3d2 and I'd read and [1:40] watched a ton of Science Fiction so I [1:42] was familiar with the concept of virtual [1:43] worlds fascinating as the idea was [1:46] though and even given my knowledge of 3D [1:48] and networking virtual world still felt [1:50] more like fiction than like something to [1:51] think about [1:52] seriously then I read Neil Stevenson's [1:55] snow crash and it all started to come [1:56] together when I thought about what it [1:58] would take to build the Met verse I [2:00] estimated that I had at least an idea of [2:02] how to implement maybe 80% of it with [2:04] existing technology I was too optimistic [2:07] I'm still waiting for my first VR sword [2:09] fight but still snow crash made me [2:12] realize the network 3D ver World virtual [2:14] worlds were ripe to happen later I got [2:17] together with John carac and he talked [2:18] for two hours straight about how he was [2:20] going to make it possible to create [2:21] virtual 3D worlds running on persistent [2:23] servers with the ability to link servers [2:26] together and for anyone to build their [2:27] own and how those Virtual Worlds would a [2:29] create to cyberspace and suddenly I saw [2:31] the shape of things to come and wanted [2:33] very much to be a part of it I jumped [2:35] into work with John Quake happened and [2:37] today tens of millions of people immerse [2:39] themselves in network 3D on a regular [2:42] basis and I wouldn't have been a part of [2:44] it without Neil and John pointing the [2:46] way to the [2:47] Future I'm not a Visionary but I hope to [2:50] do much the same for you today with a [2:52] part of the metaverse that hasn't yet [2:53] happened virtual reality I'm sure you're [2:55] all familiar with the Oculus Rift dk1 [2:58] and i' guess that most of you think [2:59] think it's just an interesting curiosity [3:01] at this point that's a reasonable take [3:04] right now but I think you should pay [3:05] close attention because VR is likely to [3:07] have a big impact much sooner than you [3:09] think and valves working hard to make [3:11] sure that that [3:12] happens two years ago we identified [3:15] virtual reality a promising way to [3:17] enable compelling new experiences our [3:19] research and development since then has [3:21] led to prototype headmount of displays [3:23] like this one and a set of demos that [3:25] together create a powerful sense of [3:27] what's known as [3:28] presence I'll talk much more about [3:30] presence later but briefly it's a sense [3:32] of being someplace else while in virtual [3:34] reality many people feel as if they've [3:36] been teleported presence is an [3:38] incredibly powerful sensation and it's [3:40] Unique to VR there's no way to create it [3:42] in any other medium most people find it [3:45] to be kind of magical and we think that [3:46] once people have experienced presence [3:48] they'll want it [3:50] badly the same Hardware that enables [3:53] presence is also highly effective at [3:54] reducing motion sickness our demos show [3:56] that given the right Hardware it's [3:58] entirely possible to create a astounding [4:00] VR experiences experiences that could [4:02] only happen in VR that don't make people [4:04] sick and we strongly believe that it's [4:06] feasible to use the same technology to [4:08] ship a consumer version within two [4:11] years the obvious question is who might [4:13] actually ship this class of VR Hardware [4:15] within two years Oculus is clearly the [4:18] leading candidate they've made VR a hot [4:20] topic again and they've already shipped [4:22] more than 50,000 Rift DK ones the dk1 is [4:25] not good enough to enable strong [4:27] presence for most people but Oculus is [4:29] new version of the Rift Crystal Cove [4:30] offers higher resolution lower latency [4:33] low persistence and translation all of [4:35] which are key elements of presence as [4:36] we'll see later so it's a big step in [4:38] the right [4:39] direction valve's goal is to enable [4:41] great VR for the PC so we've shared what [4:43] we've learned through our R&D with [4:45] Occulus we've showed them our prototypes [4:47] and demos we've explained how our [4:49] Hardware works and we've provided them [4:51] with feedback on their Hardware Designs [4:53] by showing them a prototype with low [4:55] persistence we convince them of its [4:57] importance and the lack of blur and [4:58] Crystal Cove is a direct result of that [5:01] we collaborated with them on tracking as [5:03] well and we're continuing to work with [5:05] them to improve tracking displays lenses [5:07] and calibration and we're excited about [5:09] where they're headed if Oculus executes [5:11] well and so far they seem to be I think [5:14] they could well deliver strong presence [5:16] on the PC within the next two years we [5:18] hope that happens because it would be a [5:20] huge boost for PC [5:23] VR once Hardware that ships presence [5:25] ship supports presence ships we think it [5:28] has a potential to cause a sea change in [5:30] the entertainment industry not only [5:32] could VR rapidly evolve into a major [5:34] platform but it could actually tip the [5:35] balance of the entire industry from [5:37] traditional media toward Computer [5:39] Entertainment you see for latency and [5:41] bandwidth reasons presence can only [5:43] happen in a head-mounted display [5:45] connected to a device capable of [5:47] heavyduty 3D rendering so there's no way [5:49] that TV movies streaming or anything [5:51] else that lacks lots of local compute [5:53] power is up to the task a Cory is that [5:56] the PC Linux windows and OSX is is going [5:59] to be the best place for VR because [6:01] that's where the most flops [6:03] live the bottom line is that we think VR [6:05] has tremendous potential as a gaming [6:07] platform and has an excellent shot at [6:09] taking off soon so we're working to [6:11] enable great VR for the Steam Community [6:14] First we're building VR support into [6:15] steam as you learn from Joe ludwick's [6:17] talk second we're driving VR hardware [6:20] for it on the PC by continuing to do R&D [6:22] and by collaborating with oculus finally [6:25] we're prototyping software to figure out [6:27] how to produce great VR experiences and [6:29] we'll share what we find with you as we [6:30] learn it we have no current plans to [6:33] ship VR Hardware ourselves but that [6:35] could change in the future right now [6:37] we're just figuring out what's fun about [6:39] VR and as we learn more we'll do [6:40] whatever is needed to enable VR on the [6:42] PC and [6:44] steam it should be pretty obvious by [6:46] this point why we think you should keep [6:48] an eye on VR but of course you should [6:49] develop your own opinions about its [6:51] potential the best way to do that is to [6:53] try our demos unfortunately the demos [6:56] currently take half an hour and require [6:58] a dedicated room and there are only two [7:00] such rooms at Dev days so we haven't [7:02] been able to give most of you demos here [7:04] what I can offer is this any steam [7:06] partner who's in Seattle in the next few [7:08] months and wants to see the demos can [7:10] email me at maash valv software.com and [7:12] we'll schedule a day in time once you've [7:15] tried the demos I think you'll agree [7:16] that the near-term future of VR is [7:18] brighter than almost anyone [7:20] imagines the key to that bright future [7:22] is presence so it's important that you [7:24] understand what it is why it matters and [7:27] what it takes to enable it before we can [7:29] discuss presence itself though we need [7:31] to lay some [7:33] groundwork at valve we spent a lot of [7:36] time investigating the factors that [7:37] affect how people experience virtual [7:39] reality most critically we've learned [7:42] that the key to making the experience [7:43] unique and compelling is convincing [7:45] perceptual systems that operate at a [7:47] lowlevel well below conscious awareness [7:49] that they're perceiving reality Since By [7:52] definition you're not aware of the [7:53] relevant systems it's hard to understand [7:55] what they are and why they matter to [7:58] illustrate I'll briefly describe an [8:01] example that doesn't involve VR but does [8:03] clearly demonstrate low-level processing [8:06] the example involves the scene you see [8:07] on this slide which contains a number of [8:09] cues that suggest a considerable [8:10] distance to the farther sphere including [8:13] lighting perspective texture convergence [8:16] and the relationship between the Spheres [8:17] and their [8:18] Shadows the user's task is to adjust the [8:21] size of the nearest sphere using the [8:22] arrow keys until the 2D circles formed [8:25] on the screen by the two spheres look [8:27] like they're the same size what [8:29] unfailingly happens is that the user [8:31] makes the front Circle larger than the [8:33] back Circle you can try it for yourself [8:35] at the URL shown in fact take a moment [8:38] now and look up the alley which Circle [8:40] seems [8:41] bigger to me the back circle is clearly [8:44] the bigger one in fact though the front [8:46] circle is nearly 20% larger this may not [8:50] work for you since seeing the illusion [8:51] at a distance is not ideal but if you [8:53] had a similar result that's not [8:55] surprising research using functional MRI [8:57] imaging has shown that that even in the [9:00] earliest layer of the visual cortex long [9:02] before conscious awareness comes into [9:04] play the sphere that appears farther [9:06] away projects to a larger area in the [9:08] brain even if it occupies the same area [9:10] on the retina the visual system has [9:12] already judged the sizes before the data [9:14] reaches conscious [9:16] awareness there are many similar [9:18] processors in your visual system all [9:20] contributing to your perception of the [9:22] world in ways that you're not directly [9:23] conscious of here's another one take a [9:27] look at the centerpiece of each X one's [9:29] gray and the other's dark yellow [9:32] right now let's mask out the background [9:34] without changing the colors of the [9:35] centerpieces and see what [9:38] happens so [9:41] again it turns out that due to the need [9:44] for consistent Vision across a wide [9:46] range of Illumination conditions there's [9:47] a huge contextual element involved in [9:49] low-level color processing consciously [9:52] you're not even aware of this just as [9:53] you're not aware of the processing that [9:55] alters the perceived sizes of the [9:56] circles in the previous example but [9:58] collect signals like these from low-l [10:01] processing form your sense of the world [10:03] if virtual reality provides inputs that [10:05] stimulate the low-level processors [10:07] properly you'll feel like you're [10:08] actually someplace if it doesn't you'll [10:10] feel disconnected from the scene in some [10:13] indefinable [10:14] way so far this hasn't been directly [10:17] relevant to VR so let me give another [10:19] example that's closer to home this is a [10:21] bit of a spoiler for one of our demos so [10:23] if you don't want to hear it cover your [10:24] ears I'll wait a moment while you [10:27] decide we have a demo where you're [10:30] standing on a Ledge looking down at a [10:31] substantial drop here's the scene the [10:35] stone texture is a diving board like [10:36] ledge far above the floor of a box room [10:38] that's textured with outdated web [10:42] pages yes Facebook is going public yes I [10:46] know it doesn't look like much of [10:47] anything here but that just illustrates [10:49] how different VR can be from staring at [10:51] a screen looking this at this on a [10:54] screen even when it's not warped doesn't [10:55] do anything for me but whenever I stand [10:58] on that ledge in VR my knees lock up [11:00] just like they did on top of the Empire [11:01] State Building even though I know for [11:04] certain that I'm in a demo room wearing [11:06] a head mounted display looking at [11:07] imagery of the inside of a badly [11:09] textured box my body knows that I'm at [11:12] the edge of a cliff what's more that [11:14] effect doesn't Fade with time or [11:16] repetition the inputs are convincing [11:18] enough that my body is certain at a [11:20] level below Consciousness that it's not [11:21] in the demo room it's someplace else [11:24] standing next to a [11:25] drop this feeling of being someplace [11:28] real when you're in VR is well known to [11:29] researchers and is referred to as [11:31] presence and it's presence that most [11:33] distinguishes VR from 3D on a screen [11:36] presence is distinct from immersion [11:38] which merely means that you feel [11:39] surrounded by the image of the virtual [11:41] world presence means that you feel like [11:43] you're in the virtual world trying to [11:45] describe presence is bound to come up [11:47] short you can only really understand it [11:49] by experiencing it but I'll give it a [11:51] shot presence is when even though you [11:53] know that you're in a demo room and [11:55] there's nothing really there you can't [11:56] help reaching out to try to touch a cube [12:00] when you automatically duck your head to [12:02] avoid a pipe that's dangling from a [12:03] ceiling when you feel uneasy because [12:05] there's a huge block hanging over your [12:07] head when you're unwilling to step off a [12:10] ledge it's taking off the head Mount of [12:12] display and being disoriented to find [12:14] yourself back in [12:15] reality it's more than just looking at [12:17] someplace interesting it's flipping the [12:19] switch that makes you believe deep in [12:21] your lizard brain that you are someplace [12:23] interesting presence is one of the most [12:25] powerful experiences you can have [12:27] outside reality precisely because [12:29] operates by engaging along many of the [12:31] same channels as reality for many people [12:34] presence is simply [12:36] magic different people experience VAR [12:38] degrees of presence in response to our [12:40] demos clearly there are significant [12:42] variations within the population [12:44] responses have strengthened overall as [12:46] we've improved the experience so we [12:47] expect presence to become steadily more [12:49] powerful as VR technology [12:52] emerges presence is hard to quantify but [12:54] our demos have shown that it is a very [12:56] real and compelling phenomenon one that [12:58] hooks far deeper into the perceptual [12:59] system than anything that's come before [13:01] and it's why we're so excited about VR [13:04] it's our belief that great VR will be [13:06] built on presence because it engages you [13:07] at a deeper more visceral level than any [13:10] other form of entertainment and can only [13:11] be experienced in VR consequently we [13:14] think that building Hardware that's [13:16] capable of delivering a strong sense of [13:17] presence is the key to VR [13:20] success so what does it take to cre a [13:22] sense of presence hard one experience [13:25] from a lot of R&D and prototyping has [13:26] taught us that all of the following [13:28] aspects have to be good enough before a [13:30] strong sense of presence [13:33] emerges I'll talk more about each of [13:35] these in more detail in a moment but [13:38] first I'd like to note that these [13:39] elements also reduce motion sickness [13:41] while the causes of motion sickness are [13:43] not well understood there are good [13:45] reasons to anticipate that many of the [13:47] same factors that affect presence should [13:49] affect motion sickness as well for [13:51] example flawed tracking causes a [13:53] mismatch between what your eyes see and [13:54] what your vestibular system reports and [13:56] those sorts of conflicts are thought to [13:58] be key to motion [13:59] sickness let's take a few minutes to [14:02] look at each of the elements and see why [14:03] they [14:05] matter a wide field of view is obviously [14:07] required so that you feel immersed but [14:09] also provides peripheral vision cues [14:11] that are critical for motion balance and [14:13] situational awareness presence starts to [14:15] work somewhere around an 80° field of [14:17] view and increases significantly at [14:20] least out to 110 degrees which is the [14:22] widest we've [14:23] tested resolution is a particular issue [14:26] with VR because the wide field of use [14:28] spreads out magnifies the pixels the per [14:31] degree pixel density of a 1K x1k 110° VR [14:34] display is roughly 17th out of a big [14:36] screen TV and about one tenen out of the [14:39] eye itself in fact it's actually lower [14:42] pixel density than the original Quake [14:44] running at 320 by 200 which as you can [14:46] see is pretty low [14:49] density we found that 1080p seems to be [14:52] enough for presence we expect at 1440p [14:54] or better yet 2160p would be huge steps [14:57] up but won't know until we can get [14:59] appropriately sized panels at those [15:02] resolutions persistence is the length of [15:04] time each pixel remains lit it's not [15:07] that important for TVs monitors or [15:09] mobile but it's uniquely important for [15:11] VR due to the much faster motion of the [15:13] eyes relative to the Head Mount of [15:14] display this is especially true due to [15:17] the low-level visual system called VR [15:20] which allows the eyes to remain steady [15:21] during rapid head motion the longer [15:23] pixels persist the farther the pixel [15:26] images smear across the retina when the [15:27] eye is moving and the blurry the scene [15:30] becomes here's an example of the sort of [15:32] smear that results from persistence the [15:34] image on the left is with the head held [15:36] still and the image on the right is a [15:37] simulation of what happens with a [15:39] leisurely 120° per second head turn rate [15:42] on a 60 HZ full persistence display this [15:45] results of in two degrees of smearing [15:47] across the retina per frame which as you [15:49] can see reduces detail [15:51] considerably we found that Persistence [15:53] of 3 milliseconds or less is required [15:55] for presence with a 1K by 1K 110 Dee [15:57] field of 100 10 degree field of view [16:00] head mounted display shorter [16:01] persistences will be required at higher [16:04] pixel [16:06] densities given a wide field of view [16:08] once persistence is lowered refresh rate [16:10] has to increase at 60 htz low [16:13] persistence images flicker badly in [16:15] order to address this we built the [16:16] fastest low persistence head mounted [16:18] display we could it runs at 95 Hertz and [16:21] successfully eliminates visible flicker [16:23] a somewhat lower refresh rate may well [16:25] be adequate but we haven't done the [16:27] experiments yet it's worth noting that [16:29] VR quality suffers noticeably when [16:31] rendering doesn't keep up with frame [16:32] rate and it's going to be a challenge to [16:35] maintain 95 HZ stereo rendering [16:37] especially as resolutions [16:40] climb our prototype uses Global display [16:43] where all pixels are illuminated [16:45] simultaneously this avoids the [16:46] compression stretching and tilting [16:48] problems that can occur with the more [16:49] standard rolling display where pixels [16:51] are illuminated in a scan sequence over [16:53] the course of a frame it may be that the [16:56] artifacts of rolling display can be [16:57] largely corrected by adjusting the frame [16:59] buffer during each frame to account for [17:02] eye motion but that's not yet proven [17:04] also while head motion can often be used [17:06] as a proxy for ey motion without eye [17:08] tracking there will always be failure [17:09] cases and low latency head-mounted eye [17:12] tracking is not a solved problem so [17:14] right now global display is the only [17:16] approach known to [17:19] work it seems like the selection of lens [17:21] or lenses and the design of the optical [17:23] path should be simple we have to use a [17:25] cell phone panel with magnifying lenses [17:28] because there's c l no reasonable way to [17:30] get a wide enough field of view with a [17:31] micro display or a wave guide [17:33] furthermore due to the demands of field [17:35] of view weight and Industrial design [17:37] there can only be one or maybe two [17:39] lenses per eye how hard could it be to [17:41] optimize such a simple system the [17:44] problem is that there's no way that just [17:45] one or two lenses can produce ideal VR [17:47] viewing there are many different types [17:49] of apparation and Distortion so [17:51] something like this would be required to [17:53] get everything exactly right for [17:55] reference the two big lenses on the left [17:57] are more than a foot in diameter [18:00] and the whole assembly would weigh more [18:01] than 5 pound not particularly practical [18:04] for a headmount of [18:05] display with a limitation of just one or [18:08] two lenses fixing one problem often [18:10] means making another one worse [18:11] especially with the requirements for [18:13] wide field of view this is further [18:15] Complicated by the many possible focal [18:17] lengths sizes and viewing distances so [18:20] lens design becomes a process of [18:22] searching for the best set of trade-offs [18:23] in a huge space made even more [18:25] complicated because there's no [18:26] mechanical way to evaluate the resulting [18:28] visual quality the only test that [18:31] matters is actually using the lens in VR [18:33] which is timec consuming subjective and [18:35] varies from person to person we've [18:37] developed lenses that work well enough [18:39] to allow presence but there's lots of [18:41] room for [18:42] improvement Optical calibration seems [18:44] like a minor point but has proven to be [18:46] critical and hugely challenging because [18:48] the human visual system is astonishingly [18:50] sensitive to slight errors especially [18:52] when motion and straight lines are [18:54] involved a scene that looks perfect when [18:56] viewed statically can Ripple horribly [18:57] when you swivel your head from side to [18:59] side this destroys presence and can [19:01] induce motion sickness almost [19:03] instantaneously it's also impossible to [19:06] figure out what else needs improving [19:07] until Optical calibration is dead on as [19:09] one example until we got calibration [19:11] right we didn't really notice small [19:13] tracking glitches because the image [19:14] wasn't any more stable than the tracking [19:16] once calibration was good enough the [19:18] glitches jumped out at us the bottom [19:20] line is that a highly accurate process [19:22] for characterizing the lenses and [19:23] correcting the r rendered images is [19:25] absolutely [19:27] essential at this point we have the [19:29] ingredients we need to produce the [19:30] visual quality required for presence [19:33] however that alone is not sufficient the [19:36] image has to be presented in such a way [19:37] that the perceptual system accepts it [19:39] and that requires Rock Solid head [19:41] tracking that reports translation [19:43] position in X Y and Z as well as [19:45] orientation we found that we can get [19:47] presence with a tracking accuracy of a [19:49] millimeter in position and a quarter [19:51] degree in orientation maintained over a [19:53] tracking volume of no less than a meter [19:55] and a half on a side achieving this is [19:58] much hard than it sounds there is no [20:00] currently available consumer system that [20:02] comes close here's what we're using for [20:04] our [20:05] demos it works but you can see that it [20:08] isn't exactly consumer friendly we have [20:10] a couple of promising tracking systems [20:12] in development that don't involve [20:13] wallpaper in your house and while they [20:15] may or may not pan out they're close [20:17] enough to make us confident that a [20:18] consumer grade tracking system is [20:19] feasible in the near [20:21] future the virtual image needs to be in [20:23] the right place at the right time which [20:25] means that latency has to be very low as [20:27] measured from the time of head motion [20:29] occurs to the time the last photon is [20:31] emitted due to that motion we found that [20:33] the the latency of 20 milliseconds [20:35] combined with good prediction works well [20:37] and it's possible that latency up to 25 [20:39] milliseconds may be adequate past that [20:42] the virtual world no longer seems nailed [20:44] in place and the human perceptual system [20:45] is no longer convinced that it's looking [20:47] at [20:49] reality I want to emphasize that [20:51] presence is not a property of any one of [20:53] the elements I've discussed it's a [20:55] property that emerges when all the [20:56] elements are good enough if the optic [20:58] aren't calibrated perfectly then a scene [21:00] will warp as you turn your head no [21:01] matter how good everything else is [21:03] likewise no amount of fidelity will [21:05] convince your visual system that a [21:07] virtual system is real if latency is too [21:09] high presence can't be induced if even [21:11] one of the key elements is [21:14] subpar okay due to time limitations this [21:17] has been a very fast overview of the [21:19] elements of presence if you'd like to [21:20] know more there's lots more detail on my [21:22] vve [21:24] blog so how does Hardware that could [21:27] plausibly appear in a consumer product [21:29] in the next couple of years measure up [21:31] with respect to the key elements of [21:32] presence as it turns out things look [21:34] surprisingly good it's my opinion based [21:37] on valve's R&D prototypes and [21:39] projections that a consumer headmount of [21:40] display could be built to the following [21:42] spec in [21:45] 2015 this head mounted display would [21:47] support a powerful sense of presence and [21:49] would have an excellent shot at [21:50] widespread adoption VR can certainly get [21:52] much better yet down the road but [21:54] that'll require time and major Hardware [21:56] R&D in contrast We Believe everything on [21:59] this slide is doable with relatively [22:00] minor tweaks of existing technology no [22:03] breakthroughs or Miracles are needed [22:04] just solid [22:06] engineering this specification is based [22:08] on Direct experience our prototype [22:10] headmount displays have all the [22:12] characteristics I've specified and [22:13] enable a strong sense of presence [22:15] they're also built from modified [22:16] commodity parts that point the way to [22:18] costeffective manufacturing at scale [22:21] that doesn't mean it'll be trivial to [22:22] build and ship consumer VR Hardware [22:25] we've created an R&D prototype not a [22:27] product we think that that the [22:28] technology we've used will be [22:29] transferable to Consumer head mounted [22:31] displays but lots of additional work [22:33] will be required to turn that into a [22:35] shipping product I'm confident that work [22:37] will happen soon now that we've [22:38] demonstrated what's possible and if that [22:40] happens it should be feasible to have [22:42] these head mounted displays on the [22:43] market within a couple of [22:45] years Beyond manufacturing issues [22:48] there's still a lot to be solved and [22:49] improved for one thing presents would [22:51] benefit from every one of the key [22:53] elements getting better than what's in [22:54] our prototypes we could literally use up [22:57] to 100 times as many pixels and a wider [22:59] field of view lower latency and all the [23:02] rest would also improve the experience [23:04] the Optics in particular are far from [23:06] optimal also getting per user lens [23:08] positioning right is a challenge as I [23:11] mentioned I think we're close on head [23:12] tracking but we don't have a shippable [23:14] solution yet and then there's eye [23:15] tracking which would greatly enhance [23:17] presence but is nowhere near solved [23:20] going to a wireless connection and [23:21] eliminating the tether would be a big [23:23] plus and while we believe that it's [23:25] possible to modify existing display [23:26] panels to support low persistance Global [23:29] display and low latency that remains to [23:31] be proven and will require the close [23:33] cooperation of a display [23:35] manufacturer then there's all the stuff [23:37] that needs to be figured out in [23:38] non-visual vrr areas 3D audio haptics [23:42] body tracking and input are going to be [23:44] huge positives for presence and they're [23:45] bigger and harder problems than head [23:47] mounted displays in particular VR input [23:50] and its interaction with game design is [23:51] at the core of the experience and almost [23:53] completely unexplored we're researching [23:56] those areas but it's going to take many [23:57] years and and the combined efforts of [23:59] the whole game industry to fully explore [24:02] them and then there's software once the [24:05] hardware is built it won't really matter [24:06] until the software provides great unique [24:08] VR experiences and those have yet to be [24:11] created in addition to the question of [24:13] how games will interact with input rules [24:15] about how players can move around a [24:16] virtual space without getting motion [24:18] sick or losing presence have yet to be [24:19] figured out we found that traditional [24:22] FPS movement is far from optimal and [24:24] tends to cause motion sickness so VR may [24:26] be best with slow movement and lots of [24:28] up interactions in which case we'll have [24:30] to learn how to create fun games around [24:32] that in the end a whole new VR gameplay [24:35] vocabulary will need to be developed [24:36] just as was the case with [24:38] fpss multiplayer VR is going to be very [24:41] different and more compelling than on a [24:42] screen sharing a virtual space that [24:44] feels real with other people has the [24:46] potential to be the most powerful online [24:48] social experience [24:50] yet but raises many questions remember [24:54] when deathmatch and mods and team games [24:56] on persistent servers evolved this is [24:58] going to be like that but much more so [25:01] then there's content for example no one [25:03] knows yet which art styles work well [25:04] with VR detailed scenes that look great [25:07] on a on a screen can look like cheesy [25:09] stage sets in VR and simple scenes can [25:11] seem startlingly real normal Maps don't [25:14] look good and textur sometimes do and [25:16] sometimes don't so we're going to have [25:18] to come up with a whole new visual [25:19] vocabulary for [25:21] VR2 this is where you come in platform [25:24] shifts enable software to create [25:26] breakthrough experiences think missed [25:28] quake wake Wii Sports Angry Birds VR has [25:31] the potential to be one of the biggest [25:32] platform shifts ever and you can be on [25:34] the Leading Edge of that that's all the [25:36] more true because we've learned that [25:37] great VR requires dedicated experiences [25:39] designed specifically for virtual [25:41] reality immersion can make ports to VR [25:43] interesting but great VR really requires [25:46] custom [25:47] software much as was the case with the [25:49] first 3D games pretty much everything [25:52] about the VR Gaming experience remains [25:53] to be invented I'm sure you saw some of [25:56] the adoring press on the 20th [25:57] anniversary of Doom the first game that [25:59] Nails a VR experience is going to be [26:01] remembered fondly for a long long [26:04] time when that first great VR game does [26:07] show up it's likely to be on the PC for [26:09] several reasons first VR Hardware is [26:11] going to evolve rapidly on the PC as you [26:13] can already see happening with the rift [26:15] while the consoles if they even support [26:17] VR will remain static for years second [26:20] they're going to be far more hardware [26:21] and software developers figuring out how [26:23] to do awesome VR on the PC than on the [26:25] consoles and they'll be much Freer to [26:27] experiment third VR needs as much [26:29] processing power as it can get remember [26:31] we're talking about stereo rendering at [26:33] 95 Hertz and high-end PCS are already [26:35] much more powerful than consoles with a [26:37] gap due to increase for years to come [26:40] and because of power and heat [26:41] constraints PCS will always be far more [26:43] powerful than mobile finally having [26:45] figured out how to bring presence to [26:47] Consumer [26:47] VR and having added VR support to steam [26:50] valve is going to continue to help Drive [26:52] VR in the PC and steam forward so that [26:54] all of us can deliver new compelling [26:56] experiences which means that the the PC [26:58] and steam are going to be at the heart [26:59] of VR for the foreseeable [27:03] future today's key takeaway is simple [27:06] but it has powerful implications a great [27:09] VR system at a consumer price in 2015 is [27:11] more than just possible it's sitting [27:13] there waiting to happen and it will [27:15] happen if not in 2015 then soon [27:18] after virtual reality on the PC over the [27:21] next few years may be as exciting as [27:23] anything that's ever happened in games [27:25] we're sharing what we've learned with [27:26] you and we'll continue to do so there's [27:28] a huge amount to be learned and figured [27:30] out about VR and we certainly can't [27:31] figure it all out by ourselves I hope [27:33] that as you dive into VR you'll make it [27:35] a two-way exchange so together we can [27:37] make VR one of the great entertainment [27:39] revolutions there's a lot more to talk [27:41] about and I look forward to continuing [27:43] the conversation with you at the Q&A and [27:45] on my blog thank you [27:53] [Applause]