Niantic Deploys Visual Models to Replace GPS in Autonomous Robotics

Beyond the Screen: Engineering AI Models for the Real World With Niantic Spatial And The BBC — Ai4

Ai4

Autonomous robots and AI wearables cannot rely on degraded GPS signals or synthetic simulations, requiring real-world visual positioning systems to achieve centimeter-grade spatial navigation.

AI-assisted summary of Beyond the Screen: Engineering AI Models for the Real World With Niantic Spatial And The BBC — original source. Check the original for full context.

Source published:

Detailed notes

## Background Published on YouTube by the Ai4 channel on August 18, 2026, this session features a discussion between BBC senior technology journalist Thomas Germain and Niantic Spatial CTO Brian McClendon. According to the page description, McClendon explores the engineering hurdles of building spatial intelligence and Large Geospatial Models (LGMs), explaining how architectural principles behind Google Maps, Google Earth, and Pokémon GO are being adapted for autonomous robotics and AI wearables.

## TL;DR Brian McClendon traces the evolution of digital cartography from human-oriented navigation to ultra-precise 'machine maps' designed for autonomous agents and robotics. He explains how Niantic Spatial constructs resilient 3D world models by fusing pedestrian smartphone scans, low-cost 360-degree video, and aerial drone imagery to establish unambiguous physical ground truth. The discussion highlights Niantic's Visual Positioning System (VPS), which delivers sub-second, six-degree-of-freedom localization in GPS-denied environments like urban canyons, and demonstrates how neural depth estimation can reconstruct decades-old 2D archival video into millimeter-accurate 3D digital twins. McClendon concludes that grounding sim-to-real robotic policies in high-fidelity spatial data and immediate commercial utility is essential for the future of spatial computing.

## Atmosphere & style The session has a brisk, technical, and refreshingly pragmatic tone, moving past broad AI hype to focus on concrete algorithmic and operational realities. Thomas Germain keeps the dialogue moving at an engaging clip with wry humor—joking about stumbling humanoid robots and the unexpected intrigue of underwater barnacles—while McClendon speaks with the grounded authority of a mapping pioneer. Watching the exchange feels like an intimate, high-signal masterclass where decades of foundational geospatial engineering directly inform the next frontier of embodied AI.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=4s) Machine mapping requires centimeter-scale precision far beyond traditional human maps, as autonomous agents cannot rely on human interpretive judgment to navigate physical space. - [00:04](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=4s) Crowdsourced data from gaming communities like Ingress and Pokémon GO proved that real-world photography and user corrections are essential to exposing and fixing errors in commercial map data. - [02:46](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=166s) Affordable $500 360-degree cameras combined with aerial drone photogrammetry enable passive, multi-perspective ground-truth capture across varying lighting conditions. - [05:09](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=309s) Because satellite GPS signals fail in indoor settings and dense structures, Visual Positioning Systems (VPS) are necessary to determine precise 6-DoF pose, vertical floor level, and line of sight. - [06:50](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=410s) Due to the innovator's dilemma, multi-trillion-dollar tech giants often sideline micro-resolution 3D mapping and visual localization because emerging verticals cannot immediately generate tens of billions in revenue. - [08:13](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=493s) Computer vision models trained to infer metric depth directly from standard RGB feeds enable retrospective 3D reconstruction from decades-old video archives with sub-centimeter accuracy. - [10:43](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=643s) Closing the sim-to-real gap in embodied AI requires photorealistic digital twins of real spaces, allowing physical contact policies to be safely trained millions of times prior to physical deployment. - [13:02](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=782s) Visual positioning provides sub-second localization that remains operational in dense urban canyons, as well as in environments where GPS signals are actively jammed or spoofed. - [15:28](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=928s) Cross-view feature matching uses geospatial and structural semantic priors to accurately triangulate first-person ground viewpoints directly from single overhead satellite or drone images. - [16:52](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=1012s) Autonomous ground robotics requires dynamic micro-pathing awareness—such as detecting curb cuts and small obstacles—rather than the static 2D road topologies of earlier digital mapping eras. - [18:30](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=1110s) The emerging world model ecosystem will likely converge into modular serial pipelines, coupling geospatial topological engines with specialized physical motion and simulation models.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=4s) From Google Maps to Machine Maps

Thomas Germain introduces Brian McClendon, CTO of Niantic Spatial, joking that while everyone in the audience has likely touched McClendon's work through Google Maps or Pokémon GO, what he really wants to know is McClendon’s favorite Pokémon. With only twenty minutes on the clock, Germain dives into how McClendon's foundational work on Google Maps and Pokémon GO informs what Niantic Spatial is building today. McClendon reflects on the long lineage of cartography, noting that mapmaking has always chased higher accuracy—from early explorers sailing uncharted oceans to expand the borders of the Americas to the digital era.

When Google Maps launched in 2005, the team licensed what was considered the absolute best commercial data available from industry leaders Tele Atlas and Navteq. However, when Google launched Street View in 2007, user feedback exposed a glaring problem: real-world street photographs consistently contradicted the licensed map data. That revelation sparked an ongoing quest to build truly accurate representations of the world. Google initially tackled this with methodical fleets of Street View cars, but the real breakthrough came from crowdsourcing. Users flagged errors, uploaded photos, and sent real-time corrections. That user-driven model directly inspired Pokémon GO and Ingress, where millions of players submitted critical locations to form PokéStops and portals. McClendon notes that even today, Google Maps stays accurate primarily because users are constantly complaining, taking photos, and submitting updates. While that feedback loop solved maps for human navigation, McClendon emphasizes that the modern frontier is radically different: "the new challenge is really around machine maps," where autonomous agents and robots require a standard of precision far beyond traditional cartography.

```mermaid flowchart TD accTitle: Evolution from Human Maps to Machine Maps subgraph S1["Human Mapping (2005–Present)"] A["Licensed Road Data<br>(Tele Atlas / Navteq)"] --> B["Street View Ground Truth<br>(Discrepancies Discovered)"] B --> C["Crowdsourced Updates<br>(Google Users, Ingress, Pokémon GO)"] end subgraph S2["Machine Mapping (Niantic Spatial)"] C --> D["Centimeter-Scale Reconstructions"] D --> E["Visual Positioning Systems (VPS)<br>for Robotics & Wearable AI"] end ```

- In 2005, Google Maps relied entirely on licensed datasets from Tele Atlas and Navteq. - The launch of Street View in 2007 proved that traditional commercial map data was frequently incorrect when compared against photographic reality. - Players of Ingress and Pokémon GO generated a massive global dataset of physical landmarks and gameplay hotspots. - Machine maps require substantially higher spatial precision than human-oriented road maps because robots must navigate physical space without human interpretive judgment.

---

[02:46](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=166s) Building Ground Truth Across Aerial, Drone, and 360 Camera Feeds

To illustrate what a machine-ready world model looks like, Germain brings up footage of a small town that resembles drone cinematography but is actually an interactive 3D model generated by Niantic. McClendon compares it to drone-based modeling demonstrated by Spexy, explaining that the true power lies in uniting high-altitude perspectives with dense ground-level data. In machine learning, training dependable spatial AI models hinges on establishing unambiguous ground truth, which is best achieved by capturing multiple overlapping angles of the same physical space. Niantic draws on an immense library of pedestrian-level phone scans gathered through gaming communities, layering that ground perspective underneath aerial and drone passes.

Germain wonders how Niantic scales data collection beyond Pokémon GO players, noting that spatial models cannot simply scrape text from the open internet. McClendon highlights a quiet revolution in hardware: while smartphone cameras remain ubiquitous, high-quality 360-degree cameras have plummeted in cost to around $500. These cameras remove the friction of manual mapping. Instead of deliberately framing individual objects, a surveyor can simply mount a 360 camera onto a stick and walk down a street, around an outdoor plaza, or through rooms inside a home. The resulting spherical footage yields high-fidelity 3D reconstructions that approach ground truth on their own. By fusing inexpensive 360 walk-throughs with single photos, phone scans, and aerial drone captures across different times of day, Niantic trains world models that are exceptionally resilient to real-world visual shifts.

- Professional-grade 360-degree cameras priced around $500 allow passive, handheld walking captures of indoor and outdoor environments. - High-fidelity ground truth is generated by pairing ground-level smartphone scans with aerial drone photogrammetry. - Multi-view, multi-time-of-day data collection prevents visual models from failing under variable lighting and shifting environmental conditions.

---

[05:09](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=309s) Beyond GPS: Visual Positioning for Robots and AI Companions

Germain asks about edge cases where spatial models stumble, such as enclosed concrete parking garages or low-light scenes. McClendon steps back to pinpoint the core vulnerability of traditional navigation: GPS is fundamentally unreliable indoors and often blacks out completely inside concrete parking garages and tunnels. The only robust solution for navigating these GPS-denied environments is visual positioning. Autonomous vehicle companies like Waymo recognized this early on, methodically mapping every street, parking garage, and tunnel across San Francisco and Phoenix so their vehicles could maintain absolute spatial awareness even if GPS signals vanished entirely.

Niantic Spatial is extending that exact visual positioning architecture to the rest of the planet and beyond road vehicles. McClendon explains that sidewalk delivery robots, autonomous interior platforms, human users, and wearable AI assistants all require precise visual localization. Standard GPS cannot identify which floor of a multi-story building a user is standing on, nor can it determine which direction a person is looking. With a visual positioning system, an AI companion running on smart glasses can instantly pinpoint the user's six-degree-of-freedom pose and offer contextually accurate insights about the exact physical objects within their line of sight.

```mermaid flowchart LR accTitle: Visual Positioning vs GPS Localization subgraph GPS["Standard GPS Localization"] G1["Satellite Fix"] --> G2["Fails in Concrete/Indoors"] G2 --> G3["No Heading or Floor Level"] end subgraph VPS["Visual Positioning System (VPS)"] V1["RGB Camera Input"] --> V2["Match against 3D World Model"] V2 --> V3["Precise 6DoF Pose,<br>Floor Level, and Gaze Direction"] end ```

- Waymo maps every intended garage, tunnel, and street in operating zones (such as San Francisco and Phoenix) so cars can localize visually when GPS drops out. - GPS signals cannot penetrate dense concrete structures or differentiate between vertical floor levels inside multi-story buildings. - Visual positioning provides the precise location, elevation, and gaze direction necessary for AI companions to talk intelligently about what a user is seeing.

---

[06:50](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=410s) The Innovator's Dilemma and Next-Level Resolution

Germain questions why tech behemoths like Google, with their limitless GPU infrastructure and sprawling map repositories, have not already turned universal visual positioning into table stakes. McClendon frames the issue as a classic case of corporate focus and Clayton Christensen's innovator's dilemma. As McClendon points out, "if you're a billion-dollar company, you need a billion in revenue. They're now a multi-trillion-dollar company, and if you don't talk about 10 billion in revenue, it just doesn't move the needle for them."

This structural reality leaves mega-corporations struggling to justify high-effort, emerging technologies before their markets mature. While Google possesses immense archives of historical imagery, their mapping pipeline has largely stayed anchored to the baseline resolution of Street View vehicle panoramas and satellite passes, dabbling only slightly in indoor retail and commercial photography. McClendon notes that while a decent baseline model could theoretically be constructed from Google's legacy imagery, Niantic Spatial has aggressively focused on the downstream algorithmic challenges of micro-resolution 3D reconstruction and sub-meter localization that incumbent giants have overlooked.

- Large tech conglomerates require projected returns upwards of $10 billion to prioritize new ventures, sidelining early-stage spatial mapping initiatives. - Google’s data collection remains largely bounded by Street View drive-bys and aerial satellite views, with sparse coverage of indoor physical spaces. - Niantic Spatial focuses specifically on the algorithmic hurdles of high-resolution 3D reconstruction and precise visual localization.

---

[08:13](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=493s) Reconstructing 20-Year-Old Underwater Footage in 3D

Germain directs attention to a visual demo showing an industrial underwater oil installation. McClendon reveals that the clip on display is not a video recording, but rather a complete 3D reconstruction generated from standard underwater video footage captured twenty years earlier, around 2006. The original dive footage was never captured with 3D scanners or photogrammetry in mind, yet Niantic's spatial pipeline turned that low-fidelity historical recording into an accurate spatial model. McClendon highlights this capability as a superpower: instead of having to wait decades to monitor physical degradation, organizations can apply spatial models retrospectively to existing institutional archives to uncover historical trends.

Germain expresses amazement that decades-old monocular video could yield such precision without introducing AI hallucinations. McClendon explains that the system is tightly anchored to the source video: the reconstruction is so faithful that it even mapped a slow-moving fish hovering near the infrastructure. Looking closely at the pipes, viewers can inspect individual barnacles measuring less than half a centimeter across. By comparing a 2006 baseline model against scans taken in 2008 or 2010, operators can calculate the precise accumulation rate of marine growth and determine exactly when submerged infrastructure must be cleaned or replaced. McClendon reveals that the core technical engine enabling these reconstructions is a model trained specifically to infer and extract accurate depth maps directly from standard RGB video feeds.

- The demonstrated 3D model was reconstructed from 20-year-old underwater inspection video that was never intended for mapping purposes. - Reconstructed marine growth features barnacles resolved to under half a centimeter in size. - Retrospective 3D modeling enables structural comparisons across different inspection years (e.g., 2006 vs. 2008/2010) to assess structural integrity and maintenance schedules. - Niantic's proprietary computer vision model extracts accurate metric depth information directly from standard RGB video frames without requiring specialized depth sensors.

[10:43](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=643s) Grounding Sim-to-Real Robotics with Photorealistic Spatial Capture

Thomas Germain jokes that an AI conference diving deep into the specifics of underwater barnacle inspections on oil rigs is precisely the kind of unexpected insight attendees come for. Shifting from industrial inspection to autonomous systems, Germain highlights robotics footage where humanoid robots are still wobbling and tumbling over—noting wryly that seeing them fall down remains "reassuring for everyone." He asks Brian McClendon why high-fidelity spatial data is vital for training the next generation of embodied AI and physical robotics.

McClendon explains the fundamental friction between "sim-to-real" and "real-to-sim" workflows. While traditional robotics relies heavily on synthetic data generated by 3D rendering engines like Unreal Engine and Unity, synthetic data alone leaves robots fundamentally ill-equipped to interpret the physical world. By rapidly scanning a real-world environment—such as walking around a conference room to construct an accurate 360-degree photorealistic model—engineers provide robots with exact RGB ground truth. When a robot looks out of its cameras at that room, high geometric fidelity ensures that when its foot steps onto the carpet, "the carpet would be in the right place, the foot would hit in the right time, and the training data would now be realistic." Running simulated policies millions or billions of times inside photorealistic digital twins gives engineers the confidence to deploy robots in real spaces without fear of them toppling over onto bystanders.

- **Synthetic vs. Real Simulation**: Game engines generate massive volumes of synthetic imagery, but lack the edge cases, textural nuances, and physical precision of actual environments. - **Scale of Pre-Training**: Accurate spatial models allow physical policies to be simulated and refined "millions and billions of times" prior to physical deployment.

```mermaid flowchart TD accTitle: Real-to-Sim to Sim-to-Real Pipeline for Embodied AI subgraph RealWorld["Physical Environment"] A["Rapid 360° RGB Capture"] B["Real-World Deployment"] end subgraph DigitalTwin["Spatial Simulation Engine"] C["High-Fidelity Photorealistic Mesh"] D["Physical Contact Modeling\n(Footing, Surfaces, Obstacles)"] E["Massive Policy Training\n(Millions / Billions of Runs)"] end A --> C C --> D D --> E E --> B ```

---

[13:02](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=782s) Visual Positioning System (VPS) and Beating GPS Degradation

Germain brings up everyday consumer frustrations, such as stepping out of a station, opening a smartphone map, and awkwardly waving the device in a figure-eight to establish compass orientation. McClendon explains that Visual Positioning System (VPS) technology was built precisely to eliminate that friction. He describes the classic commuter dilemma: emerging from an unfamiliar subway exit onto a busy city street, completely unsure whether to turn left or right. Without VPS, pedestrians often walk an entire block just to read street signs and orient themselves to north; with VPS, pointing a camera at the street provides sub-second orientation and immediate routing.

Beyond smartphones, visual positioning solves a critical failure point for ground robotics operating in dense metropolitan environments. In urban canyons like downtown Los Angeles, towering skyscrapers bounce and demolish satellite signals, introducing GPS drift of a full city block or more. For autonomous sidewalk delivery vehicles, this error is catastrophic—causing missed turns, delayed deliveries, and cold food. McClendon highlights Niantic’s partnership with Coco Robotics, where VPS provides precise localization even when satellite connectivity is degraded, denied, or actively jammed and spoofed across 20- to 100-block radii.

- **Sub-Second Localization**: VPS determines precise user orientation and 6-DoF coordinates in under a second from a single camera frame. - **Urban Canyon Resilience**: Overcomes severe GPS multi-path errors where tall buildings degrade accuracy by entire city blocks. - **Counter-Electronic Warfare**: Operates reliably in contested environments where GPS signals are jammed or spoofed generic across wide geographic areas. - **Commercial Fleet Deployment**: Integrated into Coco Robotics delivery robots to ensure accurate sidewalk navigation and curb-level routing.

---

[15:28](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=928s) Cross-View Feature Matching: From Aerial Imagery to Ground-Level Triangulation

Examining a technical demonstration on screen, Germain asks what underlying data the system requires to pinpoint a user's exact standing position. McClendon directs the control booth to pause on a split-screen view: a top-down aerial photograph on the left (captured via drone or satellite) paired with a first-person ground-level smartphone view on the right showing a pedestrian looking at a monument.

McClendon breaks down the algorithmic feat taking place. The visual positioning model receives only a single overhead aerial perspective where a landmark appears as an indistinct, minuscule detail. Because the model has been trained on massive geospatial corpora, it possesses deep semantic priors—understanding that monuments typically sit on elevated stone pedestals and are surrounded by perimeter fencing. The model cross-references these semantic visual features between the top-down satellite view and the ground-level perspective, matching landmarks and triangulating coordinate rays to achieve centimeter-grade localization. McClendon notes that this "effectively better than human capability to find itself" is indispensable for both autonomous robotics and future AI wearables.

```mermaid flowchart LR accTitle: Cross-View Single-Image Localization Architecture subgraph Inputs["Multi-Perspective Inputs"] A["Overhead Satellite / Drone Image"] B["Ground-Level First-Person Camera"] end subgraph Matching["Geospatial Feature Matching"] C["Semantic Structural Priors\n(Pedestals, Fences, Monuments)"] D["Cross-View Triangulation Engine"] end subgraph Output["Spatial Localization"] E["Sub-Meter Ground Coordinates\n(< 1 Second)"] end A --> C B --> C C --> D D --> E ```

---

[16:52](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=1012s) Beyond Pokémon GO: High-Precision Dynamic Mapping for Autonomous Agents

Germain compares the scale of Niantic's consumer past—running Pokémon GO for a billion users—with its current mission of building a comprehensive world model for autonomous machines, asking if the engineering challenge is fundamentally the same. McClendon responds that the problem scales in an entirely different dimension. In the early days of Google Maps (circa 2005–2007) and early mobile gaming, simply having a semi-accurate static 2D representation of streets was considered a major triumph.

By contrast, an autonomous roadside delivery robot demands ultra-high-resolution, dynamic understanding of the immediate environment. A robot must evaluate whether an unexpected trash can blocks its path, calculate clearance down to the centimeter, decide whether to transition through a curb cut into a bike lane, and negotiate dynamic roadside hazards. Similarly, indoor navigation—which McClendon characterizes as historically "kind of terrible" in complex spaces like airports and convention centers—presents major pathfinding dilemmas. While robots excel at relative odometry in their immediate vicinity, global multi-room indoor routing (such as determining which specific door to enter across a massive facility) requires high-accuracy spatial intelligence, especially given the low operational speeds of physical robots.

- **Static Maps vs. Dynamic Reality**: Early mapping prioritized road topology, whereas robotics demands real-time spatial clearance and physical traversability metrics. - **Micro-Pathing Decisions**: Autonomous agents require real-time semantic awareness to switch between sidewalks, curb cuts, and bike lanes. - **Indoor Scale Challenges**: Moving beyond basic relative positioning to robust global navigation across complex, multi-room architectural facilities.

---

[18:30](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=1110s) The Future of World Models: Spatial Intelligence, Commercial Focus, and Industry Convergence

With one minute remaining, Germain asks about the fragmented landscape of spatial computing and AI world models, noting that different startups are tackling the problem from divergent angles—some focusing on video generation, others on physics simulations or geometric mapping. He asks whether these disparate architectures will ultimately merge into a unified world model or remain distinct solutions.

McClendon acknowledges that the industry is still in its experimental infancy and nobody knows the final architecture with certainty. He suggests that multiple models will likely be composed in a serial pipeline: combining Niantic’s geospatial positioning and topological intelligence with specialized physical motion models, such as those being developed by Odyssey. McClendon emphasizes that Niantic differentiates itself by remaining grounded in immediate commercial utility:

> "Unlike some models, I mean, we are looking at how can we solve customer problems early... because we would like to A, have revenue—that's very important—and B, also prove out these use cases with the real world."

Germain concludes the session by thanking McClendon for the engineering deep dive, with McClendon expressing enthusiasm for the early-stage transformation underway across the spatial computing landscape.

## Notable quotes > [00:04](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=4s) the new challenge is really around machine maps — Brian McClendon

> [06:50](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=410s) if you're a billion-dollar company, you need a billion in revenue. They're now a multi-trillion-dollar company, and if you don't talk about 10 billion in revenue, it just doesn't move the needle for them. — Brian McClendon

> [10:43](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=643s) reassuring for everyone — Thomas Germain

> [10:43](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=643s) the carpet would be in the right place, the foot would hit in the right time, and the training data would now be realistic. — Brian McClendon

> [10:43](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=643s) millions and billions of times — Brian McClendon

> [15:28](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=928s) effectively better than human capability to find itself — Brian McClendon

> [16:52](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=1012s) kind of terrible — Brian McClendon

> [18:30](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=1110s) Unlike some models, I mean, we are looking at how can we solve customer problems early... because we would like to A, have revenue—that's very important—and B, also prove out these use cases with the real world. — Brian McClendon

## Mentioned Brian McClendon, Thomas Germain, Niantic Spatial, The BBC, Ai4, Google Maps, Google Earth, Google, Street View, Pokémon GO, Ingress, Tele Atlas, Navteq, Spexy, Waymo, Clayton Christensen, Unreal Engine, Unity, Coco Robotics, Odyssey

## Video description Brian McClendon - CTO, Niantic Spatial Thomas Germain - Senior Technology Journalist, The BBC

In this session, Niantic CTO Brian McClendon skips the vision statements and dives straight into the challenges of building spatial intelligence and a Large Geospatial Model (LGM), drawing on his experience of building pioneering mapping technologies that have redefined how people and machines interact with the real world. He will discuss both the opportunities and the engineering hurdles as computer vision is applied to the chaotic, unconstrained variables of the physical world.

Context: Why it matters LLMs and world models built for simulation do not provide the level of understanding needed to interpret and navigate the real world. Solving the spatial intelligence problem is a prerequisite for everything from autonomous robotics to AI wearables, requiring engineers to build for algorithmic and scale computing challenges.

Key takeaways: What you will learn You will learn how the architectural principles that scaled Google Maps, Google Earth, and Pokémon GO are being rewired into modern AI world models.

Call to action: Why you should attend If you are an engineer tired of hype and want deep insights into the world’s most ambitious geospatial engine, this is the session for you.

See less

[Source](https://www.youtube.com/watch?v=wd-BFrxtFSM)

中文简报

Niantic部署视觉模型以替代自主机器人中的GPS 自主机器人和AI可穿戴设备无法依赖衰减的GPS信号或合成仿真，因而需要现实世界的视觉定位系统来实现厘米级空间导航。 Brian McClendon梳理了数字制图从面向人类的导航到专为自主智能体与机器人设计的超高精度“机器地图”的演变历程。他阐释了Niantic Spatial如何通过融合行人智能手机扫描、低成本全景视频以及无人机航拍图像，构建具有高韧性的3D世界模型，从而确立明确无误的物理地面实况。讨论重点介绍了Niantic的视觉定位系统（VPS），该系统可在城市峡谷等无GPS信号的环境中实现亚秒级、六自由度定位，并展示了神经深度估计如何将数十年前的2D档案视频重建为毫米级精度的3D数字孪生。McClendon总结称，将模拟到现实（sim-to-real）的机器人策略植根于高保真空间数据和直接的商业实用性中，对空间计算的未来至关重要。 机器制图需要远超传统人类地图的厘米级精度，因为自主智能体在物理空间导航时无法依赖人类的理解与判断能力。 (0:04) 来自《Ingress》和《Pokémon GO》等游戏社区的众包数据证明，现实摄影与用户修正对于暴露并修复商业地图数据中的错误至关重要。 (0:04) 售价500美元的实惠全景相机与无人机航拍摄影测量相结合，能够在多变的光照条件下被动采集多视角的地面实况数据。 (2:46) 由于卫星GPS信号在室内环境和密集建筑群中会失效，因此需要视觉定位系统（VPS）来测定精确的6自由度位姿、垂直楼层和视线。 (5:09) 受创新者窘境的影响，数万亿美元体量的科技巨头往往会搁置微分辨率3D制图与视觉定位，因为新兴垂直领域无法立即创造数百亿美元的收入。 (6:50)

Reading summary

Brian McClendon, CTO of Niantic Spatial, argues that embodied AI and autonomous robotics require a fundamental transition from human-oriented 2D cartography to high-precision, 3D "machine maps." Because traditional satellite navigation fails in complex, GPS-denied environments, spatial models powered by visual positioning systems are essential to ground physical robots and wearable AI in reality. The Shift from Human to Machine Maps (0:04) Google Maps initially relied on licensed road data from Tele Atlas and Navteq in 2005 before Street View imagery exposed frequent discrepancies with ground truth, spurring crowdsourced corrections from users and later through location-based games like Ingress and Pokémon GO. McClendon explains that while crowdsourcing solved navigation for human drivers, autonomous systems demand millimeter-to-centimeter spatial precision rather than simple road topology. Robots require unambiguous physical measurements because they lack human interpretive judgment when navigating physical spaces. While tech conglomerates often face Clayton Christensen's innovator's dilemma—requiring projected revenues above $10 billion to justify nascent technical initiatives—Niantic builds micro-resolution datasets by fusing crowdsourced smartphone imagery with aerial drone photogrammetry and inexpensive $500 360-degree cameras. Capturing multi-view, multi-time-of-day data establishes unambiguous ground truth, preventing vision models from failing across shifting environmental and lighting conditions. Visual Positioning in GPS-Denied Environments (5:09) Satellite GPS degrades significantly indoors, fails entirely inside concrete parking garages, and experiences multi-path drift of an entire city block in dense urban canyons like downtown Los Angeles. McClendon points to autonomous vehicle developers like Waymo, which pre-mapped operational zones in San Francisco and Phoenix so vehicles could localize visually without satellite fixes. Niantic's Visual Positioning System (VPS) broadens this architecture to sidewalks, interior spaces, and wearable hardware, determining a device's six-degree-of-freedom pose, floor level, and user gaze direction in under a second. This localization performance operates by matching standard camera feeds against pre-built 3D models using semantic visual priors, such as architectural pedestals and perimeter fences, allowing cross-view triangulation between single top-down satellite images and first-person ground views. McClendon highlights that commercial fleets like Coco Robotics deploy VPS on sidewalk delivery rovers to preserve curb-level routing even when satellite connectivity is jammed or spoofed across 20- to 100-block radii. Monocular Depth Extraction from Historical Video (8:13) Niantic's spatial pipeline employs computer vision models trained to extract accurate metric depth information directly from standard RGB video, removing the strict requirement for specialized LiDAR or depth hardware. McClendon demonstrated this capability using monocular underwater inspection video of an industrial oil platform filmed in 2006. Despite the source footage being twenty years old and never intended for mapping, the system generated a dense 3D reconstruction resolving individual barnacles measuring under half a centimeter across without introducing generative hallucinations. This retrospective spatial modeling allows industrial operators to audit aging visual archives against modern scans. By comparing 2006 baseline models against recordings captured in 2008 or 2010, organizations can mathematically track marine growth rates and physical degradation over time to schedule structural maintenance with greater certainty. Bridging the Robotic Sim-to-Real Gap (10:43) A major bottleneck in robotics is the discrepancy between synthetic simulation and physical deployment. McClendon notes that while game engines like Unity and Unreal generate vast volumes of synthetic data, they lack real-world textures, lighting shifts, and physical corner cases. By rapidly transforming physical walk-throughs into photorealistic 3D digital twins, developers can anchor simulation engines in exact physical dimensions. High geometric fidelity ensures physical contact modeling is realistic: when a simulated humanoid steps onto a carpeted surface, the contact point and timing match real-world physics. This enables control policies to be trained millions or billions of times in simulation prior to deployment, reducing hardware failures. Furthermore, high-precision machine maps supply the real-time spatial clearance and semantic awareness required for autonomous delivery units to negotiate curbs, bike lanes, dynamic obstacles, and multi-room indoor facilities. McClendon acknowledges that spatial computing and world models remain in an early, fragmented stage, with uncertainty over whether video generation, physics simulations, and geometric positioning will merge into a single model. He anticipates that spatial computing will progress by pipelining specialized engines—such as Niantic's geospatial localization coupled with third-party physical motion models—emphasizing that near-term progress will depend on solving concrete commercial use cases and achieving early revenue rather than waiting for unified architectures.

精炼阅读摘要

Niantic Spatial首席技术官Brian McClendon指出，制图学正从服务于人类导航的二维道路地图，转向面向具身智能与机器人的高精度“机器地图”；该公司正通过融合多源影像与视觉定位系统，为物理世界构建厘米级空间智能基础。 从人工纠错到机器高精度需求 (0:04) Google Maps在2005年上线时完全依赖Tele Atlas与Navteq的商业许可数据，但2007年推出的街景（Street View）真实照片暴露出传统商业地图的大量错误。随后，Google依靠街景车采集和大众众包纠错，而Niantic则借助《Ingress》与《Pokémon GO》数以百万计玩家上传的地标照片，建立了庞大的全球热点数据库。这种由用户持续投诉、拍照和修正构成的反馈闭环，基本解决了人类导航的可用性问题。 然而，McClendon指出，自动驾驶设备与机器人并不具备人类的常识与即兴判断能力，无法依赖粗粒度的拓扑数据运行。机器地图要求远超传统制图的精度，机器人必须获取精确到厘米级的物理环境重构，以实现无缝通行与避障。 多源融合与低成本真实数据捕获 (2:46) 空间AI模型的训练不能像语言模型那样直接抓取公开互联网文本，而是依赖多角度、无歧义的物理真实数据（ground truth）。Niantic将玩家社群采集的地面步行视角手机扫描，与高空卫星及无人机航拍数据进行分层融合。同时，售价降至约500美元的消费级360度全景相机极大降低了人工测绘门槛；作业人员只需手持全景相机步行穿过街道、室内外广场即可完成环境球形记录。 通过将低成本全景巡检、单张照片、手机扫描与不同时段的无人机航拍相融合，训练出的世界模型能够有效抵抗光照变化与复杂环境波动的干扰。在算法端，系统甚至能够直接从普通RGB单目视频序列中提取度量级深度信息。例如，Niantic曾利用2006年左右记录工业水下管道的普通低画质巡检视频，成功重构出毫米级精度的三维模型，不仅清晰识别出移动的游鱼，还能测量不到半厘米的附着藤壶，用于评估随时间推移的海洋生物腐蚀累积速度。 视觉定位系统破除GPS物理局限 (5:09) 传统卫星导航在水泥停车场、地下隧道和高楼林立的城市峡谷中极易失效；在多层建筑内，GPS无法识别楼层高度，也无法判断用户或智能设备的视线朝向。Waymo等自动驾驶公司早期即通过预先高精度测绘旧金山与凤凰城的街道和车库，利用视觉特征实现无GPS条件下的自主定位。Niantic将这一视觉定位系统（Visual Positioning System, VPS）架构拓展至道路车辆之外的更广泛场景，涵盖人行道配送机器人、室内轮式底盘以及智能眼镜等可穿戴AI设备。 VPS能在1秒内通过单帧RGB图像计算出六自由度（6DoF）位姿、海拔高度与视线方向。在洛杉矶市中心等摩天大楼密集区域，卫星信号多径反射会导致GPS产生整整一个街区的漂移，甚至遭遇20至100个街区范围的信号干扰与欺骗；Niantic与Coco Robotics合作，将其算法嵌入人行道配送车，确保车体在卫星失效时仍能精准完成路口下桥沿切换与避障。通过结合语义先验（如识别纪念碑基座与护栏形态），系统还支持将俯视卫星图与地面第一人称照片进行跨视角特征匹配，实现亚米级定位。 大企业创新困境与真实数字孪生闭环 (6:50) 针对科技巨头为何未统治微观视觉定位的疑问，McClendon援引克莱顿·克里斯滕森的“创新者的窘境”解释称：万亿市值规模的企业需要单项业务能带来百亿美元级营收的预期才会投入重注，而新兴空间测绘在早期阶段难以满足这种规模要求。这导致巨头往往停留在车载街景与卫星图的宏观分辨率，在室内及超高精度下行算法领域留出了市场空白。 在机器人训练上，McClendon强调了“真实到仿真再到真实”（real-to-sim to sim-to-real）的重要性。尽管Unreal Engine等游戏引擎能生成海量合成数据，但往往缺乏真实环境的物理接触细节与长尾瑕疵。通过对物理房间进行360度高保真三维重建，工程师能在数字孪生体内运行数百万至数十亿次策略模拟，确保机器人的落脚点与地毯完全吻合，避免在真实部署中因几何偏差跌倒或伤及路人。 McClendon指出，空间智能与AI世界模型领域仍处于早期实验阶段，未来技术路径可能不会收敛于单一的大一统架构，而是由Niantic的空间定位与Odyssey等物理运动模型进行串联拼接；Niantic现阶段优先专注于解决具体商业场景并验证现实应用用例。

Detailed notes — zh

## Background 该视频由 YouTube 频道 Ai4 于 2026 年 8 月 18 日发布。视频内容为 BBC 资深科技记者 Thomas Germain 对 Niantic Spatial 首席技术官 Brian McClendon 的炉边对谈，探讨从 Google 地图和《Pokémon GO》汲取的测绘架构经验，以及如何克服计算机视觉在物理世界中的工程挑战来构建大地理空间模型（LGM）与空间智能。 ## TL;DR 对谈中，Brian McClendon 回顾了从早期采购商业地图到通过街景与众包模式自主测绘的演变，并指出当前的核心瓶颈在于从面向人类的粗粒度地图转向面向机器人的高精度机器地图。Niantic Spatial 借助 500 美元级的消费级 360 度全景相机、无人机及历史单目影像，利用深度学习模型直接重构出亚厘米级的物理世界 3D 模型，用于工业资产检测与高保真机器人物理仿真。此外，McClendon 详细演示了视觉定位系统（VPS）如何在城市峡谷与无 GPS 极端环境下实现超越人类感知的单张图像特征对齐，并展望了高精度空间几何模型与动态运动模型在商业闭环中的协同落地路径。 ## Atmosphere & style 对谈风格务实、沉稳且兼具极客幽默，去除了浮夸的 AI 概念炒作，直击复杂的工程现实与物理落地细节。主持人 Germain 以轻松风趣的切入点调侃机器人的笨拙摔跤与水下藤壶的附着，而 McClendon 则以资深架构师的沉着气场条理清晰地拆解技术架构，时而暂停演示画面指出像素级的几何特征，流露出对高精度物理世界建模的极致严谨与工程自信。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=4s) 从面向人类屏幕的“人图”转向面向具身智能的“机器地图”，自主机器人和 AI 穿戴设备需要远超以往的毫米至厘米级绝对几何精度与三维语义理解。 - [00:04](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=4s) 早期 Google 地图依赖商业采购数据，因街景上线暴露偏差后转向自建与众包，而《Ingress》与《Pokémon GO》玩家标记的全球地标构成了庞大的地面空间数据库基石。 - [02:46](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=166s) 大地理空间模型（LGM）的数据采集门槛大幅降低，通过 500 美元级别的消费级 360° 全景相机结合无人机俯拍与手机扫描，即可低成本获取多视角地面真值构建高保真 3D 世界模型。 - [05:09](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=309s) 视觉定位系统（VPS）成为突破室内、地下停车场与复杂弱 GPS 极限的核心基底，能为配送机器人与 AI 伴侣提供精确到朝向与楼层的空间感知。 - [07:08](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=428s) 万亿美元科技巨头因“创新者的窘境”难以聚焦百亿美元营收门槛以下的早期底层工具，Niantic 借此在室内外高密度、超高分辨率空间重构上建立差异化壁垒。 - [08:13](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=493s) 端到端 RGB 单目深度估计模型可直接将 20 年前非测绘目的的历史低清单目录像重构成亚厘米级 3D 资产，且在极低幻觉下实现海洋工业设施等跨时序形变与附着物监测。 - [10:43](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=643s) 真实物理场景数字化（Real-to-Sim）解决了纯游戏引擎合成数据无法拟合现实细节的痛点，通过真实几何与纹理让机器人在虚拟环境中运行数十亿次高保真仿真，保障现实部署安全。 - [13:02](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=782s) 在摩天大楼林立的“城市峡谷”或遭遇大范围 GPS 拒止与电磁欺骗时，VPS 可在秒级内实现高精度绝对定位，避免末端配送机器人偏航与事故。 - [15:28](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=928s) 大地理空间模型具备物理常识与空间先验，能够将宏观高空航拍视角与地面微观街景进行跨尺度、跨视角的几何特征对齐，瞬时解算 6DoF 相机位姿。 - [16:52](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=1012s) 机器人空间规模化挑战截然不同于游戏或传统二维地图，其核心在于应对微观物理间隙判定、无障碍马路牙子绕行以及大型室内复杂建筑的长距路径规划。 - [18:30](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=1110s) 未来世界模型技术路线或将采用串联架构，由空间几何模型提供精准真实物理锚点，再由运动模型推演动态物理交互，并通过切实商业落地反向验证模型价值。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=4s) 从 Google 地图到机器地图：空间智能与众包测绘演进 BBC 资深科技记者 Thomas Germain 为对谈拉开序幕，介绍嘉宾为 Niantic Spatial 首席技术官 Brian McClendon。McClendon 参与打造了 Google 地图与《Pokémon GO》（精灵宝可梦 GO）等全球现象级产品。在被问及过往经验如何启发 Niantic 当前的空间计算项目时，McClendon 回顾了制图学的演进史：从大航海时代探险家绘制新大陆边界，到 2005 年 Google 地图推出初期依赖 Tele Atlas 和 Navteq 等供应商采购最优质的商业地图数据。 然而，2007 年 Google 街景（Street View）上线后，团队立即收到大量用户投诉——街景实景照片与底层地图数据存在严重偏差，暴露出传统采购地图数据的频繁错误。这促使 Google 展开自主测绘，早期通过街景车系统性地覆盖全球道路，随后借助全球海量用户的实时反馈、纠错与上传照片完成动态更新。这种用户驱动数据的模式随后延伸至 Niantic 的《Ingress》与《Pokémon GO》，全球玩家标记的 PokéStop（宝可梦驿站）与道馆构成了极为庞大的游戏化地理空间数据库。McClendon 指出，Google 地图在面向人类导航（Human Maps）方面已经做到极致，但当前的全新挑战在于“机器地图”（Machine Maps）——当教导机器人理解并穿梭于现实世界时，需要远超以往的高精度空间数据。 - **地图数据源演变**：从 2005 年采购 Tele Atlas 和 Navteq 商业数据，转向 2007 年街景车队与众包数据校准。 - **众包地理空间网络**：依托《Ingress》与《Pokémon GO》玩家对全球高价值游戏地标（PokéStops）的扫描与提交，形成高质量地面点位集合。 - **人图 vs. 机图**：面向人类的地图允许一定容错，但面向机器人与自主代理的“机器地图”需要毫米/厘米级的绝对几何精度与三维语义理解。 --- ### [02:46](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=166s) 多模态数据融合与低成本 3D 世界模型构建 现场展示了一段类似无人机俯拍小镇的高保真三维模型视频。McClendon 解释，这并非纯航拍视频，而是利用 AI 将无人机航拍影像与地面多源数据融合重构出的高质量 3D 世界模型。在训练空间 AI 大模型（Large Geospatial Model, LGM）时，最核心的要素是获取“地面真值”（Ground Truth），而获得真值的关键路径在于对同一物理场景进行多视角覆盖。 关于数据采集来源，McClendon 澄清并不单纯依赖游戏玩家历史扫描，而是结合了无人机采集以及消费级 360 度全景相机的普及。当前全景相机成本已降至约 500 美元，采集者无需复杂的测绘操作，只需将相机固定在自拍杆上并在室内、房间或室外街道随意走动挥动，即可捕获足以支撑高精度三维重建的数据流。将这些全景流与单张照片、手机扫描、无人机航拍以及不同时间段的光照变化相结合，使得训练出的空间模型具备极强的鲁棒性。 ```mermaid flowchart TD accTitle: Multi-Source Data Pipeline for Geospatial AI World Models subgraph DataCollection["多源空间数据采集"] A["无人机航拍影像\n(宏观俯重视角)"] B["消费级 360° 全景相机\n(500 美元便携设备)"] C["手机单张照片与扫描\n(游戏与众包历史数据)"] end subgraph Reconstruction["空间计算与多视角融合"] D["多视角几何对齐与地面真值计算"] E["RGB 单目深度估计与神经隐式重建"] end subgraph Applications["大地理空间模型 (LGM) 与应用"] F["高保真 3D 世界模型"] G["机器视觉定位系统 (VPS)"] H["具身智能机器人与空间 AI 伴侣"] end A --> D B --> D C --> D D --> E E --> F F --> G G --> H ``` - **全景硬件门槛大幅降低**：500 美元级别的 360 度相机即可替代繁琐的激光雷达扫描流程，支撑高质量 3D 重建。 - **多维度真实覆盖**：涵盖不同光照条件（不同时间段）、地面步行视角与空中俯重视角的重叠验证。 --- ### [05:09](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=309s) 视觉定位系统（VPS）突破室内与弱 GPS 极限 针对不同场景下的潜在失效风险（如地下停车场、低光照环境），McClendon 指出传统 GPS 在室内及由厚重水泥阻隔的停车场几乎完全失效。解决室内与复杂环境下自身定位的最优方案是“视觉定位系统”（Visual Positioning System, VPS）。以自动驾驶公司 Waymo 为例，Waymo 之所以能在旧金山和凤凰城实现全自动驾驶，是因为其车队提前扫描并建图了预定行驶的每个地下车库、隧道和街区；即使 GPS 信号完全中断，车辆也能将当前传感器捕获的视觉特征与预建地图实时匹配完成重定位。 Niantic 的目标是将这套能力推广至汽车道路以外的整个世界：涵盖在人行道上穿梭的配送机器人、在室内建筑移动的人类，以及搭载视觉传感器的 AI 伴侣。McClendon 强调，若仅依赖 GPS，AI 伴侣根本无法判断用户面朝哪个方向，也无法感知用户身处哪一楼层，更无法就用户眼前的具体物体展开精准交互。高精度的视觉定位系统将成为打通具身智能、机器人训练及空间增强现实的核心基石。 - **Waymo 的前验先验建图**：通过穷尽式扫描旧金山每处地下停车场与隧道，消除对 GPS 的单一依赖。 - **人行道与建筑内部空间拓展**：Niantic 将机器定位能力从车行道延伸至室内与步行道，为下一代穿戴式 AI 提供朝向与楼层感知。 --- ### [07:08](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=428s) 巨头创新困境与高分辨率空间数据的战略聚焦 当被问及为何 Google 等拥有海量算力（GPU）与数据的万亿美元级巨头没有垄断这一领域时，McClendon 剖析了科技巨头的“创新者的窘境”（Innovator's Dilemma）。他回顾道：“我过去常说，如果你是一家估值十亿美元的公司，你需要十亿美元的业务规模；而如今他们是数万亿美元市值的巨头，一项业务如果不能带来百亿美元级别的收入，根本无法引起内部的重视。” 大公司往往难以将精力聚焦于尚处于早期萌芽阶段的底层创新工具。尽管 Google 拥有海量的街景与卫星遥感数据，并在部分商业店铺进行过室内全景拍摄，但其数据颗粒度主要停留在“街景车 + 卫星图”层面，尚未深度推进到下一代超高分辨率的厘米级地面重构。Niantic 凭借在三维重建与实时定位领域的垂直深耕，在此类高分辨率空间服务上建立了差异化技术壁垒。 - **万亿巨头的营收门槛**：无法产生百亿美元级增量回报的前沿技术探索难以在大企业内部获得核心资源分配。 - **分辨率层级差异**：传统地图止步于宏观街景，而空间智能需要室内外无缝衔接、高密度几何特征的下一代空间模型。 --- ### [08:13](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=493s) 跨越 20 年的历史影像 3D 重构与工业资产监测 现场播放了一段海底石油输送设施的清晰 3D 渲染模型。McClendon 透露，这并非现代设备拍摄的新视频，而是基于 20 年前（2000 年代初）一段并非为测绘目的拍摄的普通单目低清晰度水下工业录像重构而成的 3D 数字资产。这项突破展示了 Niantic Spatial 的核心技术优势：能够利用极为有限、非结构化的历史影像数据提取高价值空间信息。 ``` 20 年前普通单目水下影像 ---> RGB 单目深度估计模型 ---> 厘米级 3D 几何重构 ---> 跨时序藤壶堆积监测与维护决策 ``` 面对记者关于“模型是否会产生幻觉（Hallucination）以及误差范围”的质询，McClendon 表示该方案在抗幻觉方面表现极为可靠。精度基本逼近原始视频的分辨率极限——模型甚至连背景中游动极慢的一条鱼都完成了三维重构，且金属管道表面附着的藤壶（Barnacles）细节清晰可见，颗粒尺寸小于半厘米。通过将 2006 年与 2008/2010 年的历史巡检录像分别进行 3D 重构，工程团队可以直接进行跨时间维度的结构对比，定量分析海洋生物的附着堆积速率，从而精准预测设备需要清洗或更换的周期。该技术底层基于团队专门训练的深度学习模型，能够直接从纯 RGB 二维图像中高保真地提取深度几何信息。 - **时空穿梭式资产管理**：无需等待未来 10 年收集新数据，可直接将过去 20 年积累的历史存档录像激活为精准的 3D 数据集。 - **亚厘米级微小特征复原**：实现水下小于 0.5 厘米级藤壶等微小附着物的结构量化与时间序列对比。 - **单目 RGB 深度提取**：摆脱昂贵硬件依赖，通过深度模型从单一颜色流中端到端解算空间几何与深度分布。 ### [10:43](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=643s) 从水下检测到机器人仿真：真实世界空间数据的价值 对话从水下设施的三维重建切入，主持人 Thomas Germain 幽默地调侃道：“谁能想到我们在 AI 峰会上居然讨论起了藤壶的附着质量，这就是大家能从本次会议获取的独到见解。”这种空间捕捉能力让潜水员能够下潜到多年无人涉足的海底石油钻井平台，精确厘清自身方位与前行路线。但空间智能的用武之地远不止于此，现场大屏幕随后切入了一段机器人跌跌撞撞的测试画面。Germain 笑着表示，机器人目前依然处于动不动就摔倒的阶段，“这对在座的人类来说倒是个让人安心的现状”。 Brian McClendon 借此深入剖析了机器人领域“仿真到真实（Sim-to-Real）”与“真实到仿真（Real-to-Sim）”的核心命题。当前许多机器人训练严重依赖由 Unity 或 Unreal 等游戏渲染引擎生成的合成图像（Synthetic Imagery）。虽然纯合成数据能支撑初步训练，但若仅凭这些假象，机器人根本无法理解物理世界的真实细节与精确尺度。相比之下，Niantic 的空间技术能够在极短时间内对现实物理场景进行 360 度精确数字化重建——例如快速扫描会议厅并建立毫米级精准的三维模型。 在具备精确空间与真实 RGB 图像输入的情况下，机器人在虚拟仿真中迈步踏上地毯的瞬间，地毯的位置与脚掌着地的时间节点能够与现实完全吻合。这种高度拟真的数字化训练环境，使开发者能在计算机内部对目标场地进行数百万乃至数十亿次的高保真模拟测试，从而确保机器人真正走出大门、进入人类身旁时，不会发生跌倒或撞伤人员的安全事故。 - **合成数据局限**：单纯依赖图形渲染引擎生成的合成图像，无法让机器人精准理解真实物理环境中的微小几何形变与复杂光影。 - **高精度空间输入**：结合超越纯深度图（Beyond Depth）的高保真 RGB 图像输入，机器人的“视觉系统”能在与真实环境完全一致的虚拟空间中运行数十亿次迭代训练。 ```mermaid flowchart TD accTitle: Real-to-Sim and Sim-to-Real Pipeline subgraph Capture["真实场景数字化 (Real-to-Sim)"] A["物理世界快速扫描<br/>(会议室 / 工业场景)"] --> B["高精度 360° 重建<br/>(精准几何 + 真实 RGB 纹理)"] end subgraph Simulation["物理引擎仿真训练"] B --> C["高保真虚拟环境<br/>(精确接触面与物理碰撞)"] C --> D["数十亿次策略迭代<br/>(步态控制 / 避障决策)"] end subgraph Deployment["现实落地验证 (Sim-to-Real)"] D --> E["实体机器人现实部署<br/>(稳定行走 / 安全交互)"] end ``` --- ### [13:02](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=782s) 视觉定位系统（VPS）与城市峡谷中的 GPS 困境 谈及消费级与工业级场景的痛点，Germain 提到了智能手机用户最普遍的烦恼：打开导航软件后，由于指南针和 GPS 漂移，用户不得不拿着手机在空中画“8”字摆动来校准朝向。McClendon 解释道，视觉定位系统（Visual Positioning System, VPS）正是 Niantic 多年前切入该领域的核心起点。他举了一个生动的日常例子：当人们走出地铁站出口来到街头时，除非对该出口极为熟悉，否则很难第一时间判断该向左还是向右转；通常半数以上的人必须一直走到十字路口，仰头查看两个路牌才能辨明哪边是北。而在 VPS 的赋能下，手机摄像头只需捕捉街景一瞥，系统便能在“不到一秒钟的时间内”给出厘米级的绝对位置与朝向指示。 这种视觉定位能力对城市末端配送机器人尤为生死攸关。以在洛杉矶闹市区运营的配送机器人公司 Coco Robotics 为例，高耸入云的摩天大楼形成了严重的“城市峡谷（Urban Canyons）”效应，多径反射导致 GPS 信号急剧恶化，定位偏差常常超过整整一个街区。对于配送机器人而言，如果连自己在哪个街区都无法判定，就会错过送餐点、走错单行道，导致“外卖变凉、事故频发”。Niantic 为其提供了即便在 GPS 严重失效时也能稳定导航的视觉定位基底。 更进一步，这项技术的应用空间已经从局部微观场景扩展到广域无 GPS 环境。当遭遇电磁干扰、GPS 欺骗（Spoofing）或卫星信号被完全屏蔽的极端环境时（例如大范围的区域性电子对抗或网络瘫痪），系统即便在方圆 20 到 100 个街区完全失去卫星信号的未知状态下，依然能够凭借纯视觉输入精准找回自身坐标。 - **城市峡谷误差**：摩天大楼遮挡与反射可导致消费级 GPS 产生一个街区以上的定位误差。 - **广域 GPS 拒止应对**：即便在数十至上百个街区的广阔区域内遭遇信号屏蔽或伪造干扰，VPS 仍能实现鲁棒的自主全局重定位。 --- ### [15:28](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=928s) 单张图像跨视角特征匹配与厘米级空间定位 在大屏幕展示的技术 Demo 中，McClendon 现场拆解了视觉定位算法底层的运算机制。该算法的核心任务是实现跨视角的单张图像特征对齐：输入端的一侧可能是一张由高空无人机拍摄的下视全景或卫星航拍图，另一侧则是地面行人手持手机或机器人机载传感器拍摄的前视街景。 McClendon 让控制台技术人员暂停视频画面，指着画面中极其微小的地标细节进行解释：在宏观俯视图中，雕像和纪念碑仅仅是画面角落一个微不足道的像素点。然而，Niantic 训练的大型地理空间模型（Large Geospatial Model, LGM）已经从海量数据中习得了现实世界的先验物理结构与语义常识——例如，模型理解底座、纪念碑与雕像的典型空间构型，并知晓其周围通常会环绕防护栏杆。 当来自地面的近景照片出现带有护栏的雕像时，模型能在一瞬间从高空航拍图的对应区域找到多处稳健的几何特征匹配点，并通过空间三角测量（Triangulation）即时解算出高精度的六自由度（6DoF）相机位姿。McClendon 直言，这种在极端尺度差异和视角形变下进行跨模态匹配的能力，在效率与精度上已经展现出“超越人类的感知能力”，这为近期的自主机器人导航与未来的具身通用人工智能奠定了关键的空间基石。 --- ### [16:52](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=1012s) 从《宝可梦 GO》到机器人世界模型：不同维度的扩展难题 针对 Germain 提出的问题——《Pokémon GO》曾支撑了十亿级用户的庞大规模，如今为机器人构建整个现实世界的模型，究竟是更严峻的工程挑战还是同一套扩展架构的换壳复用？McClendon 回应称，这是一种“完全不同维度的规模化（Scaling in a different dimension）”。 回顾 2005 至 2007 年打造 Google Maps 和 Google Earth 的早期阶段，能够制作出一张大致准确的数字化二维地图就已经是一场巨大的胜利。而在当今机器人自主导航的语境下，机器不仅需要知道路在哪里，更需要感知毫米级的动态变化： 1. **微观通行空间判定**：路边摆放的垃圾桶是否阻碍了通行？机器人轮廓与垃圾桶之间的物理间隙是否足够容纳其穿行？ 2. **动态路径决策**：前方人行道受阻时，是否需要行驶至无障碍马路牙子切口（Curb Cut）处绕道，借由相邻的自行车道继续前进？ 3. **室内长距离航道规划**：以往机场等室内场所的导航体验普遍极差，而移动速度缓慢的室内机器人虽然擅长在局部微观环境中维持相对定位，但要在庞大的会展中心从 A 点导航至 B 点，判定究竟该穿过哪扇门，直接决定了其作业能效与电池寿命。 现实世界充满了未被传统地图记录的混乱与动态变量，这使得面向具身机器人的物理世界建图挑战远比以往面向人类屏幕的地图复杂得多。 --- ### [18:30](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=1110s) 世界模型的融合路径与商业落地展望 在会议接近尾声时，Germain 询问了当前 AI 领域百家争鸣的技术路线分歧：面对构建世界模型（World Models）的终极目标，行业内不同公司各执一词、切入点迥异，未来究竟需要将所有不同方案相互融合，还是大家只是在以不同路径奔向同一个终点？ McClendon 坦言目前全行业“尚无定论”。不同厂商正在探索各具特色的技术能力，例如 Odyssey 等团队正全力专注于生成物理运动与动态视频表征（Motion Nature），而 Niantic Spatial 则深耕于高精度几何定位与地理空间表征（Positioning & Geographic Nature）。在实际落地中，这些技术很可能会以串联（Serial）架构的形式相互协同：先由空间模型提供严格的现实几何与定位锚点，再由运动模型推演物理交互。 McClendon 强调，与其他纯理论探索的研究模型不同，Niantic 的核心战略是“尽早解决客户的实际商业问题”。公司坚持追求切实的营收回报，并通过真实的商业落地闭环反向验证空间智能用例的有效性。尽管具身智能与大地理空间模型目前仍处于非常早期的发展阶段，但其在工业检测、末端配送与混合现实等全行业的落地大幕才刚刚拉开。 ## Notable quotes > [07:08](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=428s) 如果你是一家估值十亿美元的公司，你需要十亿美元的业务规模；而如今他们是数万亿美元市值的巨头，一项业务如果不能带来百亿美元级别的收入，根本无法引起内部的重视。 — Brian McClendon > [10:43](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=643s) 谁能想到我们在 AI 峰会上居然讨论起了藤壶的附着质量，这就是大家能从本次会议获取的独到见解。 — Thomas Germain > [10:43](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=643s) 机器人目前依然处于动不动就摔倒的阶段，这对在座的人类来说倒是个让人安心的现状。 — Thomas Germain > [13:02](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=782s) 不到一秒钟的时间内给出厘米级的绝对位置与朝向指示。 — Brian McClendon > [15:28](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=928s) 超越人类的感知能力 — Brian McClendon > [16:52](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=1012s) 完全不同维度的规模化（Scaling in a different dimension） — Brian McClendon > [18:30](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=1110s) 尚无定论 — Brian McClendon > [18:30](https://www.youtube.com/watch?v=wd-BFrxtFSM&t=1110s) 尽早解决客户的实际商业问题 — Brian McClendon ## Mentioned Niantic Spatial, Brian McClendon, Thomas Germain, BBC, Google 地图 (Google Maps), Google 地球 (Google Earth), Google 街景 (Street View), Pokémon GO, Ingress, Tele Atlas, Navteq, Waymo, Coco Robotics, Odyssey, Unity, Unreal Engine, 大地理空间模型 (Large Geospatial Model, LGM), 视觉定位系统 (Visual Positioning System, VPS) ## Video description Brian McClendon - CTO, Niantic Spatial Thomas Germain - Senior Technology Journalist, The BBC In this session, Niantic CTO Brian McClendon skips the vision statements and dives straight into the challenges of building spatial intelligence and a Large Geospatial Model (LGM), drawing on his experience of building pioneering mapping technologies that have redefined how people and machines interact with the real world. He will discuss both the opportunities and the engineering hurdles as computer vision is applied to the chaotic, unconstrained variables of the physical world. Context: Why it matters LLMs and world models built for simulation do not provide the level of understanding needed to interpret and navigate the real world. Solving the spatial intelligence problem is a prerequisite for everything from autonomous robotics to AI wearables, requiring engineers to build for algorithmic and scale computing challenges. Key takeaways: What you will learn You will learn how the architectural principles that scaled Google Maps, Google Earth, and Pokémon GO are being rewired into modern AI world models. Call to action: Why you should attend If you are an engineer tired of hype and want deep insights into the world’s most ambitious geospatial engine, this is the session for you. See less [Source](https://www.youtube.com/watch?v=wd-BFrxtFSM)

Source quotes

[7:08] Brian McClendon: 如果你是一家估值十亿美元的公司，你需要十亿美元的业务规模；而如今他们是数万亿美元市值的巨头，一项业务如果不能带来百亿美元级别的收入，根本无法引起内部的重视。 [10:43] Thomas Germain: 谁能想到我们在 AI 峰会上居然讨论起了藤壶的附着质量，这就是大家能从本次会议获取的独到见解。 [10:43] Thomas Germain: 机器人目前依然处于动不动就摔倒的阶段，这对在座的人类来说倒是个让人安心的现状。 [13:02] Brian McClendon: 不到一秒钟的时间内给出厘米级的绝对位置与朝向指示。 [15:28] Brian McClendon: 超越人类的感知能力 [16:52] Brian McClendon: 完全不同维度的规模化（Scaling in a different dimension） [18:30] Brian McClendon: 尚无定论 [18:30] Brian McClendon: 尽早解决客户的实际商业问题

Source quotes

[0:04] Brian McClendon: the new challenge is really around machine maps [6:50] Brian McClendon: if you're a billion-dollar company, you need a billion in revenue. They're now a multi-trillion-dollar company, and if you don't talk about 10 billion in revenue, it just doesn't move the needle for them. [10:43] Thomas Germain: reassuring for everyone [10:43] Brian McClendon: the carpet would be in the right place, the foot would hit in the right time, and the training data would now be realistic. [10:43] Brian McClendon: millions and billions of times [15:28] Brian McClendon: effectively better than human capability to find itself [16:52] Brian McClendon: kind of terrible [18:30] Brian McClendon: Unlike some models, I mean, we are looking at how can we solve customer problems early... because we would like to A, have revenue—that's very important—and B, also prove out these use cases with the real world.

Original transcript

[0:04] All right. Thanks everyone for joining  us. My name's Thomas Germain. I'm a [0:07] technology reporter with the BBC. I'm here  with Brian McLendon, CTO of Niantic Spatial. [0:13] Uh Brian has worked on products that every single  person at this conference has used maybe today. [0:20] Uh most famous perhaps for Pokémon Go. Uh what I  really want to know is what your favorite Pokémon [0:25] is, but we only have 20 minutes here. So, my first  question for you is you helped build Pokémon Go, [0:30] you helped build Google Maps. That seems in some  ways a similar but also fundamentally different [0:36] product from what you're working on now. Can you  walk us through some of what you learned from [0:41] those projects that helped inform what Niantic  is working on today? I mean, I think the uh [0:46] the the history of map making has always been  about trying to improve accuracy from, you know, [0:51] back when the ships went and tried to discover the  new world and, you know, add to the borders of of [0:56] uh of America before it was called America.  Uh all the way up to what we discovered when [1:02] we launched uh Google Maps in 2005. We would  license the absolute best data you could buy [1:08] from the best companies, Tele Atlas and and uh um  and uh Navteq uh were the names of the companies [1:14] at the time. And, you know, it was very it was  the best data. It's what everybody used when they [1:18] printed maps and all the stuff. We launched Street  View [clears throat] in 2007. And, the first thing [1:24] we discovered was that people complained that  the pictures that we had taken and published in [1:29] Street View did not match the map data. And,  it turns out that map data is wrong a lot. [1:35] And, so that led us on this quest to make better  maps of the world. And, uh we did that using uh [1:41] data collected from the Street View cars, which  is sort of the methodical cover the world. Um [1:47] but, we also at Google you were able to get  Google users to give us updates and feedback. [1:52] Uh that actually led to some of the ideas around  Pokémon Go where uh you know, we actually had [1:57] players giving us information about very important  gameplay locations around the world. And so the [2:05] the pokestops that you see in Pokémon Go and  Ingress which was one of the other games that [2:09] we built contributed this very large data set  of sort of game oriented locations. And so users [2:16] helped improve the data. And I think in Google  Maps today, you know, 10 years after I left, [2:21] you're seeing the impact of that which is that  the reason Google Maps is accurate now is not [2:26] necessarily from Street View. It's because the  users are continuously complaining, updating, [2:31] and taking pictures and giving Google those those  updated data. And so for human maps, I think you [2:36] know, Google Maps has done very well. But the new  the new challenge is really around machine maps. [2:42] Right, we have to get a lot more accurate when  we're teaching a robot how to navigate the world. [2:46] It might be useful to see some examples of what  this looks like in practice. We've got a video [2:51] here of a I think it's a small town. It looks like  this is drone footage, but this is your model, [2:59] right? Yeah, so this this is actually very  similar to what we just saw from Bill at Spexey. [3:06] So this is taking taking the drone imagery and  then building a you know, very high quality 3D [3:11] models. So if you're in the room for last talk,  same idea, but the application is around uh [3:19] you know, using that accurate data combined with  ground data. And this is where you get you know, [3:24] you think about AI models and training them,  what you want is ground truth. And and in one [3:29] way of getting ground truth is having multiple  views of the same thing. So we have you know, [3:33] enormous coverage of ground level phone scan data  from from our prior prior interaction with the [3:40] the games, but we now have this aerial imagery  that we can combine and this helps us build this [3:46] world model. I think this is really interesting.  We're combining multiple different sources of [3:51] data to put this all together here. I assume  it's not all coming from Pokémon GO players. [3:56] You can't just scrape all the text on the internet  for this stuff. Where does the data come from? [4:00] What are you using? So, the I mean, we we do get  a lot of historical you know, historical data from [4:05] Pokémon GO, but the the more recent data has come  from things like this drone collect data and also [4:12] from 360 cameras. So, one of the things that you  know, I think is a I would say a recent discovery [4:17] in the industry is that you know, phone cameras  are great. Everybody's got one in their pocket. [4:22] But, 360 cameras are now cheap relatively at $500  for a high-quality one and you don't have to think [4:30] about how to map the world. You basically  just put it on a stick and swing it around [4:34] within your house, in your room, in in an area,  or outside. You just walk down the street with it. [4:40] And that data is good enough quality that we can  build very high-quality 3D reconstructions of it. [4:45] And that approaches ground truth in and of  itself. And then when you combine that with [4:50] these observations of either single photos or  phone scans, you can build use that data to [4:56] train the model further. So, the more of this data  we have in more locations, more times of day, and [5:02] from you know, combining aerial and ground data  makes our model very robust. So, that that's kind [5:09] of interesting. One thing I'm curious about is can  you walk us through some failure points here? Like [5:13] I assume this isn't going to work in a parking  garage necessarily if you haven't gone into the [5:17] garage, right? You're there's all these different  applications. Is like low light a problem? Where [5:22] does this become more difficult? Well, to to step  one step back from that. So, most people use GPS, [5:27] but GPS doesn't actually work indoors very well  or in parking garages sometimes at all because [5:32] of the cement. And so, how you find yourself in  indoor locations, you know, can best be solved by [5:41] by visual positioning. And this is exactly  how self-driving cars have been working. [5:45] Waymo has mapped you know, all of San Francisco,  all of Phoenix. And when they say all of, they [5:50] mean they drove through every parking garage that  they ever intended to go into. And in every area, [5:56] every tunnel that they had to go through in San  Francisco, so that no matter where they were, [6:00] GPS could just disappear and they would be able to  connect the stuff they saw with this map that they [6:06] had built. And our goal is to build that map for  the rest of the world, for the rest of the users, [6:11] for robots driving down sidewalks instead of  down the street, but also for humans and AIs [6:17] who are walking through interior buildings. And  so, being able to build those models so you can [6:22] visually position yourself accurately then allows,  you know, one of the applications is, you know, if [6:28] I can localize myself looking here, my, you know,  AI that companion can then talk about exactly what [6:34] I see. If all they had was the GPS, they would not  know what direction you're facing, they would not [6:40] know what floor you were on, and they would not  actually give you a very good answer. And so, we [6:45] think that this visual positioning system is going  to have applications for AI, for robotic training, [6:50] and for humans. So, one thing I'm curious about  is, you know, we talked about other data sets that [6:58] are out there, right? Like Waymo has a map of San  Francisco, Google has a map of the entire world. [7:03] Google, for example, they've got a ton of data,  they've got the GPUs. Why isn't this table stakes [7:08] for other companies? Why isn't everyone  just doing this? I think that it's it's a [7:13] Well, one, it's a matter of focus, and two,  I think we have uh done some things with [7:18] both reconstruction localization that Google has  not done. Um and uh they, you know, if you're a [7:25] uh I used to say if you're a billion-dollar  company, you need a billion in revenue. [7:28] They're now a multi-trillion-dollar company, and  if you don't talk about 10 billion in revenue, [7:33] it just doesn't move the needle for them. And  so, I think, you know, big companies struggle [7:37] to build new innovative tools, and the innovator's  dilemma is sort of the main picture there. But the [7:44] you know, Google does have all this old data, but  they haven't done as much with like going to that [7:49] next level of resolution. You know, they've done  a little bit of indoor store photography indoor [7:55] uh uh uh commercial location photography. Um but  for the most part, they are at the Street View [8:02] plus aerial imagery level, and that data alone  I could build a pretty good model with that, [8:08] but I think we have a a better services. So,  let's look at some real-world applications here, [8:13] Johnny. And we've got some underwater footage  of a oil refinery, I think we can look at. Uh [8:20] Are Yes. Yeah. Are we above Yeah, here we are.  So, can you talk about what this is used for, [8:26] but also how you made this? I think that's kind  of impressive. So, so what what's interesting [8:30] here is that this this is not a video. This  is a reconstruction in 3D of a underwater [8:38] oil facility, oil transport that was the video  was taken 20 years ago. And it was not taken for [8:45] the purposes of mapping. And you know, one of our  one of our you know, superpowers is that I think [8:51] we are able to do reconstructions from very  limited data and produce, honestly, you know, [8:56] information and value out of sort of existing data  sets. And I think that's one of the applications [9:02] for this is that there's a lot of existing data  out there. And you know, we could have, you know, [9:07] Spexy map the next thing, but what about all those  historical photographs, all that historically [9:12] collected data? If you could do more with it,  you can actually manage, you know, to discover [9:18] trends over time. And you don't have to wait  10 years into the future to collect 10 years, [9:23] you can walk 10 years in the past. And in this  case, 20 years in the past to bring data, you [9:28] know, and add value to it. I mean, I think that's  really astonishing that you could take 20-year-old [9:33] footage not taken not taken with a 3D camera  and arrive at this what's the the actual mar [9:40] or error margin here? Like how accurate is it  and how do you know it's not hallucinating? Uh [9:45] we were I think we're we're pretty good on the  hallucination side. The accuracy is to a degree [9:50] at the limits of whatever the the video is. In  this case I think we are you know we actually [9:56] reconstructed a fish back there if you noticed it  because the fish was just not moving very fast. [10:01] Um but mostly you know I think the you know the  quality of the barnacles, you know these are [10:06] these are you know on the order of you know half  you know less than a centimeter across. You know [10:11] so I think it's accurate enough that you can you  know take the 2006 data and the 2008 or 10 data [10:17] when they did it again and discover how quickly do  barnacles accumulate and when are we going to have [10:22] to go down there and either replace or clean or  do whatever we do with this this equipment. And [10:29] you know one of our one of our technologies  that makes this possible is that we have a [10:36] you know we have trained a model that is able  to extract depth information from just RGB and [10:43] and that ability allows us to do these kind of  reconstructions where that same data without it it [10:48] would just be impossible. AI 4 who thought we'd be  here talking about barnacle quality. It's the kind [10:53] of insight you're getting at this conference. Um  as we're thinking about other things you can use [10:57] this for. So we can send guys down to look at an  oil rig that's underwater that no one has looked [11:05] at or no one has captured footage of in years and  years. They can figure out exactly where they are, [11:11] exactly where they need to go. But there's all  kinds of other applications for this technology. [11:16] We've got some great robot footage here. We're  still at the stage where they fall down which is [11:20] I'm sure reassuring for everyone. But why is this  kind of spatial data so important for training the [11:30] the generation of technology. Yeah so in in the  last two talks we've talked about sim to real and [11:35] real to sim. And so a lot of robotics is trained  on simulated input data. And it's very easy for [11:42] you know, rendering engines like Unity and Unreal  to create large amounts of synthetic imagery. And [11:47] you can actually do pretty good training a robot  on synthetic imagery. But at the end of the day, [11:52] if that's all you have, your robot doesn't  understand certain things and doesn't understand [11:57] the real world precisely. And so one of the  advantages of being able to capture a scene [12:02] very quickly, like the inside of this room, walk  around and build a 360 of it, it's accurate enough [12:09] that a robot looking out their eyes at the  RGB picture of this room, it would be accurate [12:14] enough that when they place their foot on this  carpet, the carpet would be in the right place, [12:18] the foot would hit in the right time, and the  training data would now be realistic. It would [12:22] be realistic training of your destination  site. And if you did that in the computer [12:27] enough training on just this room with that robot,  you could then feel a lot more safe about letting [12:33] that robot walk through that door, walk here, and  not be in danger of falling over on one of us. [12:39] Uh and you know, these cases that are shown on the  video are cases where the improved information, [12:45] you know, beyond depth in and having accurate  RGB picture data input to the robot eyeballs [12:51] allows them to run these simulations,  you know, millions and billions of times [12:56] um in a real world scenario instead of in a sort  of fake scenario. So, in terms of other things [13:02] this is useful for, we talked about robots, we  talked about like job site applications. There's [13:06] also some consumer applications here. I'm sure  we've all had the experience you pull up your [13:10] phone, you're trying to figure out which direction  you're facing with the GPS, and you kind of wave [13:15] it around to try and get it. Yep. This is also  useful in combination with other technologies. [13:20] Can you talk about how that works? Right. So,  visual positioning is is where where you know, [13:25] sort of where we started our our game several  years ago. And it has a lot of value, um, you [13:31] know, in in the phone. And this The example I I've  always used is you're coming up out of a subway um [13:38] subway station, you get up on the street, and  you're supposed to go somewhere. And unless [13:43] you're deeply familiar with that subway exit, you  don't know whether you need to go left or right. [13:48] But now, the moment we see a picture of the street  with the camera, we can tell you exactly where you [13:53] are, exactly where you're looking, and whether to  go left or right. And you know, that is uh that [13:57] is a situation where you have you know, half the  people walk all the way down at street corner, see [14:01] the two street signs, figure out the intersection,  and then try to figure out which way is north. [14:05] Um but with visual positioning, you get that  answer you know, in under a second. And uh so, can [14:12] we do that for the for a city? Yes. Can we do that  for robots? And robots struggle from the problem [14:18] that as a delivery robot rolls down uh you know,  downtown LA, um they have terrible GPS because [14:25] tall buildings kind of demolish the quality of  GPS. It's can be a full city block or more apart. [14:31] And if your robot doesn't know which city block  it's on, that's a major problem cuz it's going [14:35] to miss its delivery, it'll go the wrong way, you  know, the food's going to get cold, and just bad [14:40] things happen. And so, uh one of our customers  is Coco Robotics, and uh we can provide them [14:46] with greatly improved accuracy in those urban  canyon situations with GPS. But it goes farther. [14:54] Um we've now shown the ability to do  this not just in, you know, small areas, [15:00] but, you know, when you don't know where you are  in a 20 or 100 block radius. If your GPS has been [15:07] destroyed, demolished, denied, or spoofed  in, you know, in certain situations, [15:11] we now know that, you know, that Russians or  whoever can actually mess with GPS, you know, [15:15] generically across large areas. So, having this  ability to find yourself in spite of GPS accuracy [15:23] is I think very important and I think you know  we're get we're making huge strides in that area. [15:28] And can we talk about what we're looking at here?  I like this is when you understand the specifics [15:32] of what's going on. Like what information does  it need in order to tell you exactly where you're [15:37] standing? Right. So this this algorithm for  visual positioning is taking as input a single [15:44] image from in this case a drone, but it could  be a satellite. And it is finding the matching [15:50] uh features between the right side, which is what  the person with the phone is walking around and [15:55] looking at. If you pause right here, um Johnny,  there we go. Um right now the only input for the [16:04] data side is the picture on the left. And  yes, there is, you know, the statue is way [16:09] down there as a tiny little thing. The our model  has been trained on enough data that it knows A, [16:16] where pedestals and monuments and statues  typically are. It knows that there are typically [16:20] fences around them. And so when a picture of  a monument with fences around it appears here, [16:26] it's able to find that precisely in this feature  not just once, but enough times to triangulate and [16:32] give you very precise local positioning based on a  single photo match between the two. And you know, [16:38] building a model that you know has this sort of  effectively better than human capability to find [16:43] itself, I think is very valuable for for robots in  the near term and for AIs and you know, eventually [16:51] for you know, humans as well. So Pokemon Go had  a billion users, right? Uh now you're trying to [16:59] build a model of the entire world for robots. Is  that a more difficult challenge or is it the same [17:04] scaling issues for different customer? I think  it's it's scaling in a different dimension. I mean [17:08] the if all we if all we had to do was build a map  for either Google Maps or Pokemon Go, um you know, [17:16] back in 2005 or 2007, having a an semi-accurate  map at all was a huge win. You know, these days, a [17:24] robot going down the street wants to know, is the  trash can that's there, you know, in the way? Do I [17:30] have enough space to fit in between it to continue  down my path? Do I need to go around to the curb [17:34] cut and then go down the street because I have a  bike path that I can get on? All of these things [17:40] are dynamic and require much higher accuracy  than anything that has been collected before. [17:46] To to improve the navigation capabilities of that,  you know, road roadside robot. And I think the the [17:52] same is true for indoors. You know, the we've  had sort of indoor airport navigation for years [17:59] and it's been kind of terrible. But if we want  robots to successfully navigate indoors, you know, [18:05] they're really good at relative position in in  their immediate neighborhood. But then navigating [18:10] from point A to point B in the other parts of  this convention center gets very complicated [18:16] and should they go through this door or that  door is a is a very important understanding and [18:21] given how slow some robots move, you know,  it's a material decision on efficiency. So, [18:26] I think the the mapping challenges are much harder  in in this new world. So, we got about a minute [18:30] left here, but there are a bunch of different  companies in this corner of the industry that are [18:34] taking fundamentally different approaches,  solving different problems. As we move towards [18:40] world models, one thing I'm curious about is will  we need to combine all these different approaches [18:45] or are they all the same or different paths to the  same goal? I I I think the answer is we don't know [18:52] yet. And that's why, you know, I was on a panel  yesterday with some some of these folks and I [18:57] think each of these are, you know, interesting new  capabilities that different parts of our industry [19:03] are needing. But I think that the you know, at the  end of the day, you know, either combining in a [19:07] serial form where you you, you know, this is the  positioning and geographic nature and then there [19:12] is a motion nature to it like a uh you know I  guess the Odyssey Odyssey guys most interested in. [19:18] Um I think all of these are going to have  application very in the near term and you know [19:24] I I I would say unlike you know some some models  I mean we are looking at how can we solve customer [19:29] problems early um because uh you know we we would  like to A have revenue, that's very important, [19:35] um and and B also prove out these use  cases with uh with the real world. And so [19:41] you know we'd love to talk about applications  of this across across the industry and I think [19:46] that it's very early days. Well, there's  a lot more to talk about, but we are out [19:49] of time. Brian, thanks so much for joining  us. Absolutely. Thank you. Thanks everyone.