E3 12: John Carmack's VR Presentation

Gamereactor

AI-assisted summary of E3 12: John Carmack's VR Presentation — original source. Check the original for full context.

Source published: 2012-07-27

Key ideas

John Carmack demonstrated Palmer Luckey's Oculus Rift prototype at E3 2012, highlighting its 90° horizontal and 110° vertical field of view alongside a $500 target price for an upcoming DIY hacker kit. (1:52)
To bypass expensive custom optics, Carmack implemented a programmable pixel shader within the Doom 3 engine that pre-warps renders with an inverse fisheye distortion, allowing cheap curved lenses to produce a straight image. (1:52)
The prototype leverages rocketry inertial tracking code from Armadillo Aerospace and custom high-poll-rate firmware from Hillcrest Labs, but remains limited to 3-DOF rotational tracking without native translational support. (1:52)
Palmer Luckey plans an imminent Kickstarter campaign targeting roughly 100 DIY kits, while industry representatives—including Sony R&D—have already examined the hardware. (10:08)

Detailed notes

## Background Published on YouTube by Gamereactor on July 27, 2012, this interview showcases id Software co-founder John Carmack demonstrating his work on an early head-mounted display prototype at E3 2012. According to the page description, the segment features a deep dive into the lag-free visor technology, a hands-on trial run using DOOM 3 BFG Edition, and a discussion of the medium's future industry implications.

## TL;DR John Carmack examines why previous generations of virtual reality failed due to claustrophobic fields of view and disorienting motion-to-photon latency, contrasting past efforts with Palmer Luckey's 90-degree-plus Oculus Rift prototype. Carmack explains his custom engineering fixes, which include porting inertial guidance math from Armadillo Aerospace rockets, commissioning high-frequency tracker firmware, and using reverse-fisheye pixel shaders to correct cheap optical lenses. A hands-on trial of DOOM 3 BFG Edition prompts the interviewer to instinctively duck incoming demon projectiles, underscoring both the power of sensory presence and the necessity of future six-degree-of-freedom positional tracking. Carmack outlines the road ahead, predicting that Luckey's upcoming DIY Kickstarter and grassroots hacker modding will catalyze mobile-driven hardware and major commercial adoption.

## Atmosphere & style The conversation carries an energetic blend of rapid-fire technical precision and historic wonder, driven by Carmack's trademark encyclopedic delivery and lucid breakdown of complex engineering hurdles. The tone shifts from academic dissection into immediate, visceral comedy during the live demo, as the hesitant interviewer straps into the duct-taped rig and shouts in surprise while dodging virtual fireballs. Watching the presentation feels like peering directly into a garage workshop at the inception of a medium, where raw ambition, pragmatic compromises, and visionary foresight collide.

## Bulletin - [01:52](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=112s) John Carmack demonstrated Palmer Luckey's Oculus Rift prototype at E3 2012, highlighting its 90° horizontal and 110° vertical field of view alongside a $500 target price for an upcoming DIY hacker kit. - [01:52](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=112s) To bypass expensive custom optics, Carmack implemented a programmable pixel shader within the Doom 3 engine that pre-warps renders with an inverse fisheye distortion, allowing cheap curved lenses to produce a straight image. - [01:52](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=112s) The prototype leverages rocketry inertial tracking code from Armadillo Aerospace and custom high-poll-rate firmware from Hillcrest Labs, but remains limited to 3-DOF rotational tracking without native translational support. - [10:08](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=608s) Palmer Luckey plans an imminent Kickstarter campaign targeting roughly 100 DIY kits, while industry representatives—including Sony R&D—have already examined the hardware.

## Key takeaways - [00:10](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=10s) Early consumer VR failed largely due to narrow 26° to 30° horizontal fields of view that felt like looking through cardboard tubes rather than inhabiting an alternate world. - [00:10](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=10s) Human perception is far less tolerant of motion-to-photon latency during head movement than during thumbstick or mouse inputs, immediately triggering nausea and breaking presence. - [01:52](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=112s) Carmack adapted precision inertial guidance code from Armadillo Aerospace and collaborated with Hillcrest Labs for custom tracker firmware to minimize tracking latency. - [01:52](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=112s) Carmack solved optical distortion in software using a reverse fisheye pixel shader pre-warp, eliminating the need for heavy, expensive multi-element physical lenses. - [04:21](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=261s) The prototype uses a split 1280x800 LCD panel (640x800 per eye), which exhibits screen-door effect and ghosting, though rapid mobile display advancements are expected to quickly resolve resolution constraints. - [04:21](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=261s) Lacking 6-DOF positional tracking, the prototype relies on a software neck-pivot model to simulate an ocular arc during head turns and reduce motion sickness. - [04:21](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=261s) Control mechanics decouple joypad aiming from head orientation, allowing players to turn with the thumbstick while independently aiming with natural head movement. - [07:07](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=427s) Physical reactions like ducking under virtual fireballs confirm the power of stereoscopic wide-FOV immersion, while proving why true positional tracking is the next vital engineering hurdle. - [07:07](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=427s) Magnetic tracking tests using the Razer Hydra provided millimeter-level relative precision, but room-scale angular drift rendered it unreliable for head tracking without continuous calibration. - [10:08](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=608s) Carmack argues wide-field VR delivers a psychological sense of co-presence and physical fear that conventional displays cannot achieve, regardless of GPU horsepower. - [10:08](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=608s) Palmer Luckey's upcoming Kickstarter DIY kits are intended to trigger open-community 'genetic experimentation' around optical focus, interpupillary distance, and ergonomics. - [11:12](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=672s) Mobile hardware already contains the core sensors and displays required for standalone VR headsets, pointing toward an eventual untethered future. - [12:23](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=743s) Major tech companies are actively monitoring the hardware, with Sony R&D inspecting the prototype and expressing intent to acquire development kits for research. - [12:23](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=743s) Carmack predicts that approximately twelve months of grassroots hacker iteration will precede full-scale commercialization and ecosystem development by major platform holders.

## Detailed walkthrough ### [00:10](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=10s) The Historical Pitfalls of VR: Field of View and Latency

John Carmack begins by asking the interviewer if he has ever tried a head-mounted display at a trade show. When the interviewer admits he has not, Carmack remarks that this is somewhat unfortunate, because the demonstration is far more startling to veterans who already know how disappointing virtual reality hardware has historically been. Since the initial VR craze of the early 1990s, the public has held a romanticized expectation of being transported directly into simulated realms, only to encounter severe disappointment. Early consumer gear suffered from miserable fields of view, often between 26 and 30 degrees, which Carmack likens to looking through cardboard toilet paper tubes rather than inhabiting an alternate reality. Even the top-tier consumer benchmark of early 2012—a Sony head-mounted display released that February—only reached a 45-degree horizontal field of view. To achieve genuine immersion, a display must completely block out peripheral glimpses of the floor and ceiling, blanketing the user's vision.

Latency is the other traditional failure point of virtual reality. Carmack explains that while input lag is already a crucial issue in conventional gaming—where players demand 60 frames per second and an instantaneous response between pressing a button and watching a weapon fire—VR drastically magnifies that sensitivity. The human sensory system is forgiving of input lag when manipulating an analog thumbstick, slightly less forgiving when aiming with a computer mouse, and completely intolerant when moving the physical head. The visual system instantly recognizes any mismatch between vestibular motion and display output, inducing nausea and breaking the illusion of presence.

“All of that is magnified in importance in VR, where you tolerate a lot when you're doing something with a thumbstick on there, you tolerate less when you're doing something with a mouse, and you tolerate even less when it's your head doing the moving—your brain knows what the world's supposed to look like,” Carmack explains.

- Early head-mounted displays were crippled by narrow 26° to 30° horizontal fields of view. - Sony's HMZ personal 3D viewer (released in February 2012) improved consumer FOV to 45°, which was still inadequate for full immersion. - The human brain tolerates latency on thumbstick inputs and mouse sweeps, but motion-to-photon latency during head movement immediately triggers sensory rejection.

---

[01:52](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=112s) Rocket Science, Doom 3, and Palmer Luckey’s Oculus Rift Prototype

To solve these persistent technical hurdles, Carmack pursued several parallel engineering paths, combining video game rendering with rocketry hardware. At his aerospace venture, Armadillo Aerospace, he scavenged precision inertial integration code originally written for guidance systems and ported it over to head-tracking sensors. He contacted sensor manufacturer Hillcrest Labs directly, convincing them to flash custom firmware that substantially boosted the hardware's internal update frequency. Carmack then leveraged the development of *DOOM 3: BFG Edition* at id Software to justify this research to the company's business side, using cutting-edge stereoscopic 3D and VR support to draw fresh attention to an eight-year-old title.

Although Carmack had five different prototype headsets under construction in his own office, the specific unit brought to E3 was built by Palmer Luckey under the name Oculus Rift. Luckey assembled the physical unit in his workshop, while Carmack contributed the motion sensors, custom mounting straps, and deep engine integration within *DOOM 3*. The prototype offers a dramatic leap in visual immersion: a 90-degree horizontal and 110-degree vertical field of view that seals off the real world. While Luckey designed the device as an unpolished, roughly $500 kit aimed strictly at hackers and tinkerers, Carmack devised a software trick to circumvent optical limitations. Standard optical lenses introduce extreme pincushion and fisheye distortions. Rather than relying on expensive, heavy multi-element glass lenses, Carmack inserted a programmable pixel shader into the rendering engine to pre-warp the frame with an inverse fisheye distortion, allowing inexpensive, lightweight lenses to bend the image back into a clean, rectilinear projection.

“I use a pixel shader to invert the warp of that, so I reverse fisheye the warp in the game and then the optics pull that back out and you're left with a straight and square view of the world,” Carmack notes.

```mermaid flowchart LR accTitle: Carmack Software Distortion Pipeline accDescr: Pipeline showing game engine rendering, inverse fisheye pixel shader pre-distortion, LCD output, and optical lens correction into the user eye. subgraph Engine ["Doom 3 Engine"] Render["3D Scene Render"] Shader["Pixel Shader: Reverse Fisheye Warp"] Render --> Shader end subgraph Headset ["Oculus Rift Hardware"] Panel["1280x800 Display Panel"] Optics["Inexpensive Curved Lenses"] Eye["User Eye: Straight & Square Image"] Panel --> Optics Optics --> Eye end Shader --> Panel ```

- Target price point for the experimental Palmer Luckey kit was roughly $500 for developers and hobbyists. - The prototype delivers a 90° horizontal and 110° vertical field of view, far exceeding prior commercial visors. - Carmack ported inertial navigation math from Armadillo Aerospace rockets and had Hillcrest Labs burn custom high-poll-rate tracker firmware.

---

[04:21](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=261s) Display Resolution, Neck Models, and Decoupled Controls

Carmack candidly outlines the technological compromises present in the prototype. The headset utilizes a single 1280x800 LCD panel split down the middle, allocating just 640x800 pixels per eye stretched across an immense visual angle. Individual pixels and the "screen-door effect" are plainly visible, though Carmack predicts rapid resolution jumps driven by mobile phone manufacturing, including upcoming 1080p panels and a compact Toshiba 2.5K 6-inch panel that compresses 30-inch monitor pixel densities into a wearable form factor. The prototype also suffers from LCD ghosting and slow pixel response times, in contrast to a 120 FPS OLED prototype headset Carmack had built but could not bring after having to return its driver boards.

A critical functional limitation is that the unit features three degrees of freedom (orientation tracking: roll, pitch, and yaw) but lacks positional tracking (x, y, and z translation). If a player sways their torso from side to side or ducks, the virtual camera remains static, an unnatural disconnect that reliably produces motion sickness. To soften this effect, Carmack implemented a kinematic neck-pivot model in the software so head rotations correctly offset the virtual eyes along an arc rather than spinning like a disembodied camera point. For controls, Carmack avoided the trap of forcing players to aim exclusively with their necks. The control scheme decouples orientation: the joypad's right thumbstick adjusts weapon pitch and rough yaw, while fine aiming is handled naturally with head orientation. Players instinctively lower their weapon when wandering through quiet corridors to clear their field of view and snap it back up when entering firefights.

“One of the things I had to change in the game is if you hack in head-mounted display support, normally you're stuck with aiming with your neck—you know, you're moving around like this—so you still need to be able to turn with the joypad so you don't have to turn circles and tie yourself up in the cables, but you can aim independently,” Carmack explains.

---

[07:07](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=427s) Stepping Into Hell: Hands-On Demo and the Positional Tracking Frontier

The demonstration transitions to the live trial as the Gamereactor interviewer prepares to put on the Oculus Rift alongside wireless headphones and an Xbox 360 controller. Carmack recommends standing up for maximum immersion, though he warns that some players experience equilibrium loss and fear toppling over. The interviewer jokes about wanting a bucket nearby in case of nausea, to which Carmack quips that nobody has vomited yet, urging him not to ruin the fragile prototype. Carmack promises to stand behind him and catch him if he stumbles. As the interviewer enters the demon-infested corridors of *DOOM 3: BFG Edition*, he is immediately overwhelmed by the scale, shouting in astonishment as imps attack him.

During combat, the interviewer instinctively leans and ducks his upper body to evade an oncoming fireball, only to realize the in-game perspective does not shift with his torso, causing him to take the projectile squarely in the face. Sitting back down to debrief, Carmack points out that this physical dodge reaction proves how persuasive the visual immersion is, while highlighting why true six-degree-of-freedom positional tracking is the next crucial milestone. Carmack recounts his laboratory tests with the Sixense Razer Hydra, a magnetic tracking controller system. While magnetic sensing provides sub-millimeter precision, its spatial accuracy drifts significantly across different parts of a room. While a 15-degree distortion goes unnoticed on a virtual sword or gun, even minor angular drift on head tracking breaks sensory alignment. When properly calibrated in test environments, however, positional tracking enabled breathtaking moments where users could drop to their hands and knees to inspect virtual floor pebbles up close.

“You don't care if your lightsaber is 15 degrees off, but you care a lot if your head is 15 degrees off, but still I had some demos set up with that where if you're looking the right direction and I calibrated it right, you could lean over and look at something—you could even get down on your knees and put your hand on the virtual floor and look down at pebbles on the ground, which was really amazing,” Carmack recalls.

- The interviewer instinctively attempted to physically duck under incoming fireballs due to the sheer convincing scale of the stereoscopic 3D world. - Carmack tested Sixense Razer Hydra magnetic trackers for translational motion, achieving millimeter-level precision. - Magnetic tracking demonstrated strong relative precision but lacked uniform absolute accuracy, exhibiting distortions across room volumes that disrupted head tracking calibration.

[10:08](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=608s) Stepping Back, the Upcoming Kickstarter, and the Power of Presence

When asked whether he has a clear cutoff point for his tinkering—given that an engineer could easily keep optimizing and modifying hardware forever—John Carmack explains that he is technically stepping away from the prototype for now. He has to return to his primary, long-term studio projects at id Software. Instead, the immediate future of the hardware rests in the hands of Palmer Luckey, who plans to launch a Kickstarter campaign later that month to assemble and distribute DIY kits. While Palmer originally anticipated an initial run of around 100 units, Carmack predicts that demand will drastically outstrip expectations.

Carmack argues that the device delivers a psychological shift that raw graphics horsepower cannot replicate. Increasing GPU power by a factor of 100 or achieving photorealistic movie rendering does not change the fact that the imagery is trapped on a flat screen. In contrast, putting on the wide-field headset puts the player directly inside the virtual environment. As Carmack describes the visceral impact of playing with the visor, “you get the sense that you are sharing this dark clammy world with those monsters in there and when they're coming around the corner it affects you in a way that looking at even your big screen TV does not do”.

- Palmer Luckey plans to launch a Kickstarter campaign later in the month to fund an initial run of hardware kits. - Palmer originally targeted roughly 100 units, though Carmack anticipates substantially higher consumer interest. - Flat-screen rendering—regardless of GPU fidelity—fails to produce the genuine feeling of physical co-presence that wide-FOV VR generates.

[11:12](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=672s) Mobile Hardware, Augmented Reality, and the Roots of Immersion

Addressing questions about making VR affordable and practical for everyday consumers, Carmack envisions an eventual shift toward standalone hardware driven by mobile phone components. Smartphones already incorporate nearly all of the necessary building blocks—displays, cameras, accelerometers, and gyroscopes—eliminating tethering cables entirely. While current mobile firmware is not calibrated for low-latency VR tracking and existing experiments (such as a Hasbro iPhone-based binocular toy) suffer from severe flaws along the pipeline, the concept proves that mobile-driven headsets are technically viable. Carmack also acknowledges the consumer reach of tech giants like Apple and Google, noting that Google's push into augmented reality could realistically put AR glasses in the hands of smartphone users within five years, even if their public promotional pitch currently overstates the actual state of the technology.

Despite the broader commercial potential of AR, Carmack emphasizes that his creative heart remains tied to fully immersive virtual reality. Creating games at id Software has always centered on pulling players into alternate spaces, and modern VR represents the realization of the ambitions developers harbored decades earlier. As Carmack explains, “my heart's still mostly with the immersive VR really that's all what we that's what we've always been doing with first-person shooters is trying to immerse you into a world and this takes it to the logical step and it is the vision that people had back when we were starting in the 90s here that you can actually make it happen now”.

- Untethered mobile architectures could package screens, gyroscopes, and processing into an all-in-one consumer headset. - Early phone-holding peripherals (like Hasbro's iPhone toy) showcase the general form factor, despite severe software and latency flaws. - Google Glass and mobile AR could become mainstream within five years, but Carmack cautions that a significant gulf remains between marketing concept videos and real-world performance.

[12:23](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=743s) Grassroots Tinkering, the Road to Productization, and Industry Interest

Carmack lays out a near-term roadmap centered on open community experimentation. In addition to contributing engine software and integrating support into *DOOM 3 BFG Edition* to ensure backers have a high-quality commercial game ready to play, he wants the hardware kits to circulate among tinkerers, makers, and workshop hobbyists. While many will build the standard design following instructions, others will iterate—modifying head straps, experimenting with alternative lens enclosures, and tuning optical focusing or interocular spacing. He characterizes this decentralized trial-and-error as essential “genetic experimentation” that closed corporate teams often miss. He points out that while Sony’s commercial head-mounted display is impressive in certain respects, it suffers from critical design flaws that open community iteration could have ironed out before release.

The response from major industry players is already underway; Carmack reveals that a representative from Sony inspected the prototype the day before and immediately expressed interest in securing several kits for their own R&D department. Looking forward, Carmack predicts that roughly a year of grassroots hobbyist experimentation will pave the way for a major platform holder or tech company to step in, build a full software ecosystem, and turn the technology into a polished, mainstream commercial product.

```mermaid flowchart TD accTitle: Path to VR Commercialization accDescr: Diagram tracing the development cycle from Palmer Luckey's Kickstarter kits through community modding to major corporate adoption.

subgraph Community ["Grassroots Distribution & Tuning"] A["Palmer Luckey's Kickstarter Kit"] --> B["Hobbyist Tinkering & Modding"] B --> C["Hardware Adjustments<br>(Interocular, Focus, Head Straps)"] D["Software Integration<br>(DOOM 3 BFG Edition)"] --> B end

subgraph RND ["Industry & R&D Validation"] B --> E["R&D Evaluation<br>(Sony & Tech Majors)"] end

subgraph Commercial ["Mass Market"] E --> F["Full Commercial Productization<br>& Ecosystem Support"] end ```

- Carmack will supply software code and official *DOOM 3 BFG Edition* VR support to provide a baseline commercial experience on the kit. - Tinkerer customization—focus adjustments, interocular distance tuning, and mounting designs—serves as necessary mechanical evolution. - A Sony representative evaluated the prototype on-site at E3 and confirmed their R&D department intends to acquire kits for internal research. - Carmack expects approximately twelve months of community experimentation before a major technology company steps in to formalize a complete consumer product ecosystem.

## Notable quotes > [07:07](https://www.youtube.com/watch?v=kw-DlWwlXHo&t=427s) You don't care if your lightsaber is 15 degrees off, but you care a lot if your head is 15 degrees off — John Carmack

## Mentioned John Carmack, Palmer Luckey, Oculus Rift, DOOM 3: BFG Edition, id Software, Armadillo Aerospace, Hillcrest Labs, Sony, Sony HMZ, Toshiba, Sixense Razer Hydra, Xbox 360, Apple, Google, Google Glass, Hasbro, Kickstarter, Gamereactor

## Video description We talk to the Doom creator about his newest piece of technology - a lag-free head-mounted display unit, trial it out with DOOM 3 BFG Edition, and discuss what implications the visor could have for the future of the industry.

[Source](https://www.youtube.com/watch?v=kw-DlWwlXHo)

中文简报

E3 2012：约翰·卡马克VR技术演讲约翰·卡马克深入分析了以往几代虚拟现实技术为何因狭窄的视野和令人眩晕的动光延迟而失败，并将其与帕尔默·拉齐拥有90度以上视场的Oculus Rift原型机进行了对比。卡马克阐述了自己的定制工程修复方案，包括移植来自犰狳宇航火箭的惯性制导数学算法、委托定制高频追踪器固件，以及利用反向鱼眼像素着色器来矫正廉价光学透镜。在《毁灭战士3 BFG版》的实际试玩中，采访者本能地躲避迎面而来的恶魔投掷物，这不仅凸显了感官临场感的震撼力量，也印证了未来实现六自由度定位追踪的必要性。卡马克还展望了未来的发展道路，预测拉齐即将推出的DIY众筹项目以及草根黑客的改装热潮，将推动移动硬件的发展并催生大规模的商业普及。约翰·卡马克在E3 2012上展示了帕尔默·拉齐的Oculus Rift原型机，重点介绍了其90°水平与110°垂直视场，以及即将推出的DIY黑客套件约500美元的目标价格。 (1:52) 为了避开昂贵的定制光学元件，卡马克在《毁灭战士3》引擎中实现了一种可编程像素着色器，该着色器通过反向鱼眼畸变对渲染画面进行预矫正，使廉价曲面透镜也能呈现平直的图像。 (1:52) 该原型机利用了来自犰狳宇航的火箭惯性追踪代码以及来自Hillcrest Labs的定制高轮询率固件，但仍局限于3自由度旋转追踪，缺乏原生平移支持。 (1:52) 帕尔默·拉齐计划近期发起Kickstarter众筹活动，目标约为100套DIY套件，而包括索尼研发部门在内的行业代表已经检视了该硬件。 (10:08)

Source quotes

[7:07] John Carmack: You don't care if your lightsaber is 15 degrees off, but you care a lot if your head is 15 degrees off

Original transcript

[0:00] [Music] [0:10] so have you ever tried a headmount a [0:11] display at a trade show stuck them on [0:14] it okay that that's a little bit [0:16] unfortunate because this demo is much [0:18] more impressive for people that have [0:19] tried the state of things before this I [0:22] you know those of us that that have been [0:23] around for a while from the early 90s on [0:25] there was the whole VR craze about how [0:27] it's going to throw you into these [0:29] Virtual Worlds and change everything so [0:30] people have this idea in their head of [0:32] what putting on a head mounted display [0:34] is like and it really hasn't been like [0:37] that Vision at all people wind up being [0:39] bitterly disappointed when you wind up [0:41] buying some little headmount display or [0:43] trying something at a trade show because [0:45] they they've had generally very small [0:47] fields of view I your typical like the [0:50] Sony display here was actually this came [0:52] out in February and it's great it's the [0:53] best consumer level device that that's [0:56] been available it's a big Advance this [0:58] has a 45° horizontal field of view [1:00] before that a lot of them had 26 to 30° [1:03] which is like toilet paper tubes you [1:05] know you you just are not immersed with [1:07] that but field of view is a big deal you [1:09] want to have it block out the world [1:10] around you you don't want to see like [1:12] you know everything down below or above [1:14] the screens on there you want to be [1:16] thrown into the world immersed so field [1:18] of view is a big deal the other thing [1:20] that's been really bad on VR demos is [1:22] the the latency from time for motion to [1:25] the time that the view actually changes [1:26] and I I harp about latency all the time [1:28] even in regular gaming about how you [1:30] want your time from button press to you [1:32] know gun firing on the screen to be as [1:34] short as possible you want your 60 [1:35] frames per second gaming and all of that [1:37] is magnified in importance in VR where [1:40] you tolerate a lot when you're doing [1:41] something with a thumb stick on there [1:43] you tolerate less when you're doing [1:45] something with a mouse and you tolerate [1:46] even less when it's your head doing the [1:48] moving your brain knows what the world's [1:49] supposed to look like so I'm pursuing [1:53] three or four different axes of [1:54] improvement on here I started building [1:56] some things myself on there I you know [1:58] I'd go over to my rocket shop at armadil [2:00] and and Cobble things together there I [2:02] and I wound up um like I wound up using [2:05] the inertial uh integration code for my [2:07] rocket at armadillo to replace what was [2:09] being done on the sensors here I got the [2:12] sensor company hrrest labs to actually [2:14] burn me a custom firmware that up the [2:16] update the uh the update rate from the [2:18] software on there which is great so I've [2:20] done a bunch of things through here but [2:22] then Doom 3 was this opportunity to turn [2:24] this stuff I was tinkering with for fun [2:27] and actually make it relevant to the [2:28] company's business on there so I could [2:30] do the 3D TV support I think I did a [2:32] good job on that I'm still not the [2:33] world's biggest booster of 3D TVs [2:36] especially for consoles where you have [2:37] to trade off some frame rate but then I [2:39] can use this VR experience as something [2:41] else that that is cutting edge that is [2:43] worth people coming and looking at an [2:45] 8-year-old game and you know and saying [2:46] some nice things about it so uh i' had [2:49] been building I actually have like five [2:52] head mounts in different stages of [2:54] construction at my office but the one [2:56] that I brought uh was actually built by [2:59] um guy named Palmer lucky it's called [3:01] the Oculus Rift and he built this in his [3:03] Workshop I added my sensors and a strap [3:06] to hold it on and I built software into [3:08] the game for it uh but what's amazing [3:10] about this is this is a 90° horizontal [3:13] field of view and it's 110° vertical it [3:16] cuts off Cuts you off completely from [3:18] the world it throws you into there and [3:20] this is intended to be available at [3:22] around $500 for a kit it's not a [3:24] consumer device but for the hacker Baker [3:26] crowd this is going to be amazingly cool [3:29] you know you still have to add a motion [3:30] tracker on here and then there's no soft [3:32] Doom 3 is going to be the only software [3:34] that really natively supports this [3:36] because the interesting thing about this [3:37] is the reason people didn't do this 20 [3:39] years ago on General VR stuff is you [3:43] have a couple cheap lenses in here over [3:44] a display it gives a very warped [3:47] fish-eyed view of the world and back [3:49] then you really couldn't do anything [3:50] about that but now I use a pixel Shader [3:52] to invert the warp of that so I reverse [3:55] fisheye the warp in the game and then [3:57] the Optics pull that back out and you're [3:59] left with a straight and square view of [4:01] the world so the things that you'll [4:03] notice about this it is very immersive [4:05] I'll hand you wireless headphones and a [4:08] wireless game controller on here you can [4:10] literally turn all the way around until [4:11] you trip on the cables with this [4:13] wouldn't recommend it though but yeah uh [4:15] but eventually this will all be Wireless [4:17] or it'll be self-contained like an iPad [4:19] on there and you'll be able to to do a [4:20] very different experience there so the [4:22] resolution is not very high it's a 1280x [4:24] 800 panel back here that has each eye [4:27] looking at half of it so each eye only [4:29] has 640x 800 resolution and it's [4:31] stretched over a huge area so you can [4:34] resolve pixels on it but that's there [4:36] are so many forces driving mobile [4:38] displays now we we'll have 1080p [4:40] displays you know within months and [4:42] toshiba's got a 2 and 1 12K display [4:44] which is amazing when you think about [4:46] it's a 30in monitor crunched down to 6 [4:48] in that will totally fit in this uh the [4:51] other aspect of it is this tracks [4:53] orientation on this uh it does not track [4:56] position so if you sway your body side [4:59] to side it does not pick that up and the [5:01] world won't respond and that's a good [5:02] way to make yourself feel a little sick [5:04] in this like look at the floor and sway [5:05] side to side and the world's not doing [5:07] what it's supposed to I was going to ask [5:08] is is there any buckets in the room here [5:10] just no one's lost it yet you know one [5:12] person started getting a little bit like [5:15] that like don't throw up on my head [5:16] Mount we've got more people to show this [5:18] to but in general this has people have [5:20] said that the the gameplay experience on [5:22] here is intuitive you should pick it up [5:24] pretty fast one of the things I had to [5:26] change in the game is if you hack in [5:28] head Mount display support normally [5:30] you're stuck with aiming with your neck [5:32] you know you're moving around like this [5:34] so you still need to be able to turn [5:35] with the joy pad so you don't have to [5:37] turn circles and tie yourself up in the [5:39] cables but you can aim independently so [5:42] the right thumb stick lets you pitch the [5:43] weapon up or down and what's interesting [5:45] is I find you kind of do the natural [5:47] behavior on there where when there's no [5:48] monsters around you pitch it down to get [5:50] it out of your view and you pitch it [5:51] back up when you're in combat and it's [5:53] an extra little interesting level there [5:55] and so you could you could sort of rough [5:57] it in with the joy pad and then you can [5:59] finese and keep your bead on them with [6:01] your head mount on there so um it is [6:04] possible there because the positioning [6:07] is a very important 3D Q in most games [6:09] are a disembodied eyeball you're just an [6:11] eyeball rotating in space uh to give a [6:14] little bit of motion to this I have a [6:15] head neck model so it pivots like this [6:18] so there's a small amount of motion with [6:20] that so if you're doing a certain motion [6:22] it's it feels correct but most motions [6:25] that you're moving around it's not going [6:26] to be exactly correct on there so it's [6:28] not a world locked in place this display [6:32] is a uh you know it's not a high-end [6:34] embedded LCD uh something like uh an [6:36] iPhone Retina Display switches a lot [6:38] faster and has denser pixels on here so [6:40] there's a little bit of ghosting it [6:42] takes a little bit of switching time [6:43] it's uh I had another display that I had [6:45] hoped to bring to show both that was 120 [6:47] frame per second OLED that was the [6:49] world's fastest updating head Mount but [6:51] uh I had to send back the boards for [6:53] that so those are the flaws of this [6:55] where it's uh the resolution's low it [6:58] doesn't track position it's a little bit [7:00] slow on the ghosting but it's still the [7:02] best VR demo that you'll see I like it [7:05] let's let's try [7:07] this [7:08] is oh my God is it best to sit or St [7:12] with this I think the experience is more [7:14] imersive standing but some people have [7:16] expressed some worry about falling over [7:18] I'll catch you if you fall back they won [7:20] okay I'm just here promise me that man [7:22] promise me that [7:30] oh my God [7:39] [Music] [7:55] [Music] [8:28] you're all right [8:30] Place [8:45] PC pretty awesome isn't it [8:49] yeah so you see the there there are [8:52] other sensors that we could use that [8:53] cont trct physical position there [8:55] because I could tell you must have eaten [8:56] a fireball in the face you're like that [8:58] didn't work and so there that's one of [9:00] the axes of improvement and in fact the [9:02] uh let's go a and sit back down all [9:04] right yeah yeah all right so one of the [9:06] axes of improvement from there is that [9:08] only had attitude sensing on there so [9:10] when you were trying to Bob and weave [9:11] around the Fireballs on that it wasn't [9:13] working on that so immersive that you [9:16] just feel the need to do that anyway but [9:18] we can do that that's the exciting thing [9:20] where there are a couple different [9:20] sensor technologies that can give us [9:22] what we want there and I've spent a [9:24] while working with the six sense razor [9:25] Hydra which is kind of a you know [9:27] lightsaber weote sort of thing on there [9:29] and I've taken some of those apart and [9:31] you tried to use those because they give [9:32] positional information there and they're [9:35] very precise and that they'll give [9:37] millimeter level accuracy on there but [9:39] they're not accurate where it's if it [9:41] works fine over here it's going to be [9:42] different over there and you don't care [9:44] if your lightsabers is 15 degrees off [9:46] but you care a lot if your head is 15° [9:48] off but still I had some demos set up [9:50] with that where if you're looking the [9:51] right direction and I calibrated it [9:53] right you could lean over and look at [9:55] something you could even get down on [9:57] your knees and put your hand on the [9:58] virtual floor and look down at Pebbles [10:01] on the ground which was really amazing [10:03] that is a whole another level I mean [10:04] this is cool but there are several [10:06] dimensions of extra coolness that can be [10:08] added on to this with the technology [10:10] that's available today just going to say [10:11] I mean how much more tinkering do you [10:13] want to do do you do you have a set goal [10:14] to go right at this point I'm going to [10:16] have to draw a ler cuz essentially you [10:18] could be infinitely just tinkering [10:19] Tinker tinkering yeah so technically I'm [10:21] off the project right now you know I'm [10:23] back on to the big projects that's going [10:24] to take a while longer on there but the [10:26] hope is that uh Palmer can get these [10:28] kits together he's going to run a [10:29] Kickstarter probably starting uh later [10:32] this month to try and get the first [10:33] batch together and initially he was [10:35] looking at probably 100 units I suspect [10:37] he's going to have a lot more demand [10:38] than that on this because this is one of [10:41] those fundamentally cool things it [10:43] doesn't matter if you give us a 100 [10:44] times more GPU power when we start [10:46] rendering movies they're still on a [10:47] screen and you get the sense that you [10:50] are sharing this dark clammy world with [10:52] those monsters in there and when they're [10:54] coming around the corner it affects you [10:55] in a way that looking at even your big [10:57] screen TV does not do [10:59] well now you're obviously saying that [11:01] it's cost prohibitive for for most [11:03] consumers it's high end but would you [11:05] like to see the idea of something like [11:06] that get to the point where it's it's [11:09] acceptable to buy for the common guy so [11:12] I think where this will wind up consumer [11:14] friendly is eventually you'll see this [11:15] driven by mobile phone style hardware so [11:18] there's no wires and it's self-contained [11:20] and they are mobile phones already have [11:22] most of the the gear needed for that [11:24] they've got the display they've got [11:26] cameras accelerometers Gyros the [11:28] firmware is not set up right for doing [11:29] all of this but there's actually a [11:31] Hasbro toy that you can stick your [11:33] iPhone in that gives kind of a view [11:34] Master binocular view it's it's done bad [11:37] in all sorts of ways along the chain but [11:39] that type of thing could in theory be [11:41] something that almost works there but I [11:43] wouldn't be surprised if somebody like [11:45] Apple or Google I mean Google's pushing [11:47] the augmented reality on there which I [11:50] do believe has more broad potential on [11:52] there that's the type of thing where you [11:54] could imagine everybody that has a [11:55] smartphone now has augmented reality [11:57] glasses in 5 years or something and that [11:59] could happen faster than we're expecting [12:01] on there but it is worth understanding [12:03] that the reality of what they've got is [12:04] not what their Vision pitch is there's [12:06] there's a broad Gulf there but my [12:08] heart's still mostly with the immersive [12:10] VR really that's all what we that's what [12:11] we've always been doing with [12:12] first-person shooters is trying to [12:14] immerse you into a world and this takes [12:16] it to the logical step and it is the [12:17] vision that people had back when we were [12:19] starting in the 90s here that you can [12:21] actually make it happen now so what I [12:23] think's going to happen is we'll get a [12:25] bunch of these kits out uh I hope to be [12:27] able to contribute some software as well [12:29] as the the Doom 3 BFG Edition stuff on [12:31] there so there's there's at least one [12:33] piece of consume commercial software [12:35] that will give a good experience with [12:37] this but it'll go out to the people that [12:38] make things in their workshop and we'll [12:40] get a 100 people doing different takes [12:42] on them most of them will build the [12:44] stock as it's as the instructions say [12:47] but there's going to be a good chunk of [12:48] the people that say no I want to adjust [12:49] my focusing I want to adjust the inter [12:51] ocular space I want to make a different [12:53] contain a different head mount on there [12:55] I want to attach it differently and [12:57] that's the type of genetic [12:58] experimentation that's going to be [13:00] really important for this because you [13:01] know like so Sony built this and it's a [13:03] good device in many ways but it's got a [13:05] lot of flaws and if they had had a 100 [13:08] people making different takes on it [13:10] before that they would have wound up [13:11] with a better project and we had [13:13] somebody from Sony in here yesterday [13:15] looking at this stuff and uh what was [13:17] their reaction yeah they're going to [13:18] want to pick up a couple kits for their [13:20] R&D group uh so I think that it'll be [13:22] great we'll have a year of [13:23] experimentation with this of people [13:25] doing different things and then it is [13:27] compelling enough that I think that one [13:29] of the majors will step in and want to [13:31] kind of productize it and do the whole [13:33] software ecosystem and have broad [13:35] support and all the things that's [13:37] necessary to make it something that [13:38] consumers would be happy with [13:42] [Music]