EliseAI Curbs LLM Use Over Failure Risks Across 60M Annual Calls

Mario Martone, Sophia Dew

Standard 90% academic benchmark accuracy guarantees catastrophic operational failure at enterprise scale, forcing essential industries to rely on proprietary deterministic models and continuous in-house evaluations.

AI-assisted summary of Why Real-World AI Is Harder Than the Benchmark | EliseAI’s Mario Martone — original source. Check the original for full context.

Source published: 2026-10-18

Key ideas

Mario Martone transitioned from 15 years as a theoretical physicist studying string theory to EliseAI, arguing that real-world applied AI presents even greater operational and intellectual complexity than fundamental physics. (0:00)
EliseAI processes roughly 5 million phone calls monthly (60 million annually) within a strict 1.5-second latency window, rendering academic 90% accuracy benchmarks completely unacceptable for high-stakes housing and healthcare workflows. (7:11)
Rather than relying solely on LLMs, EliseAI deploys a multi-model architecture combining fine-tuned language models, proprietary deep neural networks, and traditional machine learning for deterministic and tabular tasks like lease renewal predictions. (5:00)
The company is targeting affordable housing operations, aiming to introduce automation into an administratively complex, historically non-automated sector to drive down operational overhead for vulnerable residents. (20:00)

Detailed notes

## Background Published on October 18, 2026, by Mario Martone and Sophia Dew, this episode examines why deploying AI in essential industries like housing and healthcare demands a far stricter reliability standard than academic benchmarks require. According to the show description, EliseAI Head of Applied Research Mario Martone explains how the company manages roughly 5 million monthly phone calls, why large language models cannot solve every operational bottleneck, and how 15 years in theoretical physics helped shape the company's rigorous empirical research culture.

## TL;DR Former theoretical physicist Mario Martone explains how EliseAI deploys production AI into high-stakes sectors like housing and healthcare, which together represent roughly 40% of US GDP. He illustrates why academic benchmarks—such as a celebrated 90% accuracy score—break down at a scale of 60 million annual calls, requiring internal empirical evaluations that target 99.999% reliability. Rather than treating LLMs as a universal solution, the engineering team routes tasks across fine-tuned open-source models, proprietary deep neural networks, and traditional ML to satisfy tight 1.5-second voice latency budgets and ensure deterministic execution. Applying scientific first principles, Martone's team automates complex operational workflows ranging from self-guided apartment tours to reducing the administrative overhead of affordable housing.

## Atmosphere & style The discussion carries an intellectually rigorous yet refreshingly grounded tone, trading commercial AI hype for hard-won systems engineering pragmatism. Host Sophia Dew guides the exchange with structured curiosity, allowing Martone to thoughtfully contrast the deliberate, abstract elegance of string theory with the fast-moving, high-stakes realities of live operational software. The dialogue feels like an authentic technical post-mortem, marked by a deep reverence for the scientific method and an earnest awareness of the human consequences of algorithmic errors in daily life.

## Bulletin - `00:00` Mario Martone transitioned from 15 years as a theoretical physicist studying string theory to EliseAI, arguing that real-world applied AI presents even greater operational and intellectual complexity than fundamental physics. - `07:11` EliseAI processes roughly 5 million phone calls monthly (60 million annually) within a strict 1.5-second latency window, rendering academic 90% accuracy benchmarks completely unacceptable for high-stakes housing and healthcare workflows. - `05:00` Rather than relying solely on LLMs, EliseAI deploys a multi-model architecture combining fine-tuned language models, proprietary deep neural networks, and traditional machine learning for deterministic and tabular tasks like lease renewal predictions. - `20:00` The company is targeting affordable housing operations, aiming to introduce automation into an administratively complex, historically non-automated sector to drive down operational overhead for vulnerable residents.

## Key takeaways - `00:00` Foundational cognitive skills from theoretical physics, such as stamina for ill-defined problems and first-principles reasoning, transfer directly to navigating the daily shifts of applied AI. - `00:00` Deploying AI in essential sectors like housing and healthcare introduces real-world consequences that contrast sharply with the insulated environment of academic physics and frontier research labs. - `05:00` Enterprise conversational voice systems require executing tens to hundreds of distinct sub-operations inside an end-to-end latency budget of approximately 1.5 seconds. - `05:00` Large Language Models are ill-suited for tasks demanding strict determinism and fine-grained probabilistic calibration over tabular data, such as tenant lease renewal risk predictions. - `07:11` At a deployment scale of 60 million annual calls, a standard academic benchmark score of 90% accuracy translates into millions of unacceptable service failures. - `07:11` Achieving reliability beyond 99% toward 99.999% requires an exponential increase in engineering rigor and proprietary, continuous empirical evaluation pipelines. - `10:00` External model releases and viral leaderboard claims are systematically vetted against in-house evaluation harnesses rather than taken at face value. - `13:00` EliseAI conducted an intensive eight-month push into fine-tuning open-source models to build custom domain architectures that match or surpass hosted commercial frontier APIs. - `15:00` End-to-end leasing operations, from digital ad discovery and scheduling to physical property access codes and guided touring, can run autonomously without human agent intervention. - `15:00` Housing and healthcare comprise roughly 40% of US GDP, offering extensive headroom for automation before the company needs to expand into other vertical markets. - `20:00` Affordable housing represents a major portion of real estate with near-zero software automation, offering high-impact opportunities to reduce administrative overhead and tenant costs. - `21:09` Technical hiring across New York and San Francisco focuses on systems engineers and rare academic researchers driven to translate theoretical rigor into production impact.

## Detailed walkthrough ### `00:00` From String Theory to High-Stakes Applied AI

Mario Martone opens the conversation reflecting on the unexpected intellectual shift from theoretical physics to applied machine learning, admitting that the operational complexity he deals with now often exceeds what he encountered when studying fundamental physics. Before entering applied AI, Mario spent 15 years as a theoretical physicist and professor researching black holes, quantum field theory, and string theory. While string theory moves at a notoriously deliberate, multi-decade academic pace, the modern AI ecosystem shifts daily. Yet Mario explains that the foundational skill carries over directly: individuals who possess the stamina to think deeply about open, poorly defined questions and generate creative formulations have an immense advantage in a field that reinvents itself every morning.

```mermaid flowchart TD accTitle: Real-World AI Impact vs Frontier Lab Research accDescr: Diagram contrasting theoretical AI research in frontier labs with applied AI in essential industries like housing and healthcare. subgraph Research["Frontier Research Labs"] A["Focus on Big Abstract Problems"] --> B["Paper Benchmarks (90% Accuracy Target)"] B --> C["Decoupled from Immediate Real-World Risk"] end subgraph Applied["EliseAI Applied Engineering"] D["Essential Industries: Housing & Healthcare"] --> E["High-Stakes Decision Making"] E --> F["Zero-Tolerance Reliability (99.999% Target)"] end ```

The host notes that EliseAI co-founder Mina Song was recently on the show at the New York Stock Exchange discussing the necessity of diffusing intelligence directly into the real economy. Mario shares that when he first contemplated transitioning out of academia into industry, the standard path appeared to be joining a frontier research lab. However, frontier lab environments still mirror academic physics in that researchers remain largely insulated from immediate real-world consequences. EliseAI appealed to him precisely because conversational agents are deployed directly into housing and healthcare, making operational choices that actively alter people’s daily lives rather than merely outputting text on a screen.

As Mario Martone explains: “Like you're working on these big problems. But one thing I really wanted is to have an impact in the real world.” He emphasizes that when AI moves from a pure benchmark to essential human infrastructure, the stakes amplify: an autonomous agent helping someone schedule an apartment tour or handle a medical query can produce life-changing benefits when it functions properly, or catastrophic harm when it errs.

- Mario Martone spent 15 years in academia as a professor and theoretical physicist investigating string theory and black holes before transitioning to industry. - The pace of theoretical physics is exceptionally slow and deliberate, whereas the applied AI frontier shifts on a daily basis. - Frontier AI research labs often operate similarly to academic physics by focusing on abstract problems, whereas EliseAI focuses on shipping decision-making software directly into housing and healthcare.

---

`05:00` Choosing the Right Tool: Voice Latency, LLMs, and Specialized Neural Networks

Turning to EliseAI’s concrete architecture, the discussion dives into why broad foundation models cannot solve every engineering bottleneck in production. EliseAI entered the conversational voice market three years ago, backed by founders Mina Song and Tony Yip, well before low-latency speech-to-speech models existed. Mario points out that executing natural voice conversations in an enterprise setting requires coordinating tens to hundreds of distinct operations within a tight 1.5-second latency window. Beyond conversational dialogue, their platform manages complex predictive operations—such as calculating tenant lease renewal risk—where standard generative models prove ill-suited.

```mermaid flowchart LR accTitle: EliseAI Multi-Model Task Routing Architecture accDescr: Pipeline illustrating how tasks are routed to LLMs, custom deep neural networks, or traditional ML based on the need for determinism or conversation. subgraph Ingestion["Input Operations"] In["Tenant / Patient Interaction"] end subgraph Routing["Architectural Decision Layer"] In -->|"Dialogue & Reasoning"| LLM["Fine-Tuned LLMs"] In -->|"Tabular Data & Renewal Risks"| DNN["Proprietary Deep Neural Networks"] In -->|"Deterministic Rules & Classifiers"| ML["Traditional Machine Learning"] end subgraph Execution["Real-Time Execution (1.5s Voice Window)"] LLM --> Out["Operational Decisions & Voice Output"] DNN --> Out ML --> Out end ```

Rather than treating Large Language Models as a universal hammer, Mario’s applied research group explicitly differentiates between tasks requiring open reasoning and tasks requiring strict determinism or raw probabilistic calculations. For predicting lease renewals over vast datasets, EliseAI relies on deep, proprietary neural networks built entirely in-house alongside traditional statistical machine learning models. LLMs struggle severely with consistent deterministic behavior—failing to yield the exact same output for identical inputs across runs—and perform poorly when calibrating complex probabilities over tabular data.

As Mario Martone observes regarding the recent trend across the software industry: “And so here, it's like many people who have joined very recently, they just have one solution for everything, LLMs.” Because EliseAI must ship robust models rapidly, Mario’s role requires evaluating each engineering problem from a bird’s-eye perspective: determining what approach produces the highest real-world impact and what solution reaches near-optimal accuracy in the shortest delivery timeline.

- Voice AI requires executing tens to hundreds of distinct sub-operations inside an end-to-end latency budget of roughly 1.5 seconds. - EliseAI started developing its voice technology three years ago, deliberately pushing into production before standard off-the-shelf tooling matured. - Predicting lease renewals relies on proprietary in-house neural networks and traditional machine learning on structured data rather than LLMs. - LLMs inherently struggle with strict determinism and fine-grained probabilistic calibration.

---

`07:11` The 90% Fallacy: Why Academic Benchmarks Break at 60 Million Annual Calls

The conversation shifts to the vast disconnect between academic evaluation standards and enterprise reliability requirements. In academic papers and lab environments, an LLM scoring 90% accuracy on a benchmark is routinely celebrated as a state-of-the-art breakthrough. However, Mario points out that at EliseAI’s deployment scale—handling approximately 5 million phone calls every month, or roughly 60 million calls per year—a 10% failure rate equates to millions of broken conversations, missed medical follow-ups, or botches in the leasing process. In essential industries, an error rate of that magnitude is intolerable.

To bridge this gap, EliseAI blends top-tier academic talent—including former university professors—with elite systems developers to design proprietary, large-scale evaluation pipelines ("evals"). Rather than relying on generic public benchmarks, EliseAI constantly runs empirical evals against millions of real conversations to systematically measure how model updates alter edge-case behavior. As Mario stresses, achieving 99% accuracy is merely the first baseline; each additional "nine" (climbing toward 99.999% reliability) demands an exponential increase in engineering rigor because real human livelihoods are directly impacted by the system's decisions.

As Mario Martone explains: “Like when they're talking about reliability, they talk, okay, if the LLM can take it up to 90 percent. And then we say 90 percent? That means for like 60 million phone calls per month, per year, it means like an enormous amount of failures. That is totally unbearable.” By continuously running systematic evals over real interactions, the engineering team ensures that model updates answer directly to production data rather than artificial benchmark metrics.

- Academic research often treats a 90% benchmark success rate as an impressive milestone. - EliseAI processes roughly 5 million calls a month (approximately 60 million phone calls per year); a 10% failure rate would trigger millions of enterprise failures. - Every additional "nine" of reliability (from 99% up to 99.999%) presents an increasingly steep technical hurdle. - EliseAI builds proprietary evaluation systems to measure the real-world operational impact of model decisions across millions of conversations.

`10:00` Tackling Voice Latency and Running In-House Benchmarks

At a minor scale of five or six test conversations, most AI methods appear to function quite well. However, when deployed across millions of interactions, performance curves diverge radically. Sophia Dew highlights that this rigorous empirical testing mirrors applying the scientific method directly to product engineering. Mario Martone embraces the comparison, declaring: “I'm a strong believer that the scientific method is kind of one of the best ideas that the human accounts have ever generated.”

Voice interaction presents acute technical challenges because latency is not solely an issue of software; it hinges directly on hardware constraints and infrastructure architecture. EliseAI operates under immense throughput demands, managing approximately 5 million phone calls each month—totaling around 60 million calls per year. In real-time verbal communication, delays ruin the user experience, compelling the engineering team to diagnose bottlenecks methodically.

Martone explains that their advantage stems from having developed their core stack in-house. Because he largely engineered the system himself, the team possesses intimate architectural visibility into which processing stages create the most latency. They balance internal optimizations with an outward-looking evaluation of external third-party models, assessing whether newly released open-source weights, fine-tuning approaches, or specialized speech-to-text (STT) and text-to-speech (TTS) engines offer genuine step-function improvements over existing internal modules.

```mermaid flowchart TD accTitle: EliseAI Voice Pipeline Evaluation Loop accDescr: Flowchart showing how EliseAI monitors the market for speech models, routes them into internal benchmarks, and updates the core production voice stack.

subgraph Monitoring["External Discovery"] A["External Releases (STT / TTS / LLMs / Social Feeds)"] --> B["Team Flags Candidate (via Slack)"] end

subgraph Benchmarking["Internal Evaluation"] B --> C["Automated Testing Harness"] C --> D{"Evaluate vs. Internal Benchmarks"} end

subgraph Production["Production Architecture"] D -- "Outperforms Baseline" --> E["Full-Stack Integration"] D -- "Underperforms" --> F["Discard / Ignore Claims"] E --> G["Live Voice Calls (~5M Calls / Month)"] end ```

To prevent chasing unsubstantiated external claims, Martone’s team relies on a rapid-response testing harness. When team members flag bold claims—such as trending posts on X—they bypass external leaderboards and immediately pipe the model into their own evaluation suite. Martone points out: “Let's set it up. Let's run an experiment internally. So we can actually see a performance of every model on our own internal benchmarks versus the other ones that you cannot trust.” While a human remains in the loop during these triage procedures, the team is actively pushing toward full end-to-end evaluation automation.

- EliseAI handles ~5 million calls monthly, or approximately 60 million interactions annually. - In-house stack development gives the research group comprehensive transparency over granular latency sources across the inference pipeline. - Third-party benchmarks are treated with skepticism until verified against internal production test sets.

---

`13:00` Applying Physics Rigor to Fine-Tuning Open-Source Models

Martone draws a direct parallel between modern AI development and his previous career as a university professor in theoretical physics in London. Beyond foundational mathematics and programming skills, theoretical physics cultivates the rare cognitive discipline needed to formulate novel solutions to seemingly intractable problems. As Martone remarks: “Theoretical physics are the cutting edge of kind of humanity in terms of, like, reasoning and thinking. So coming up with new ideas, it's not easy.”

This mindset proved crucial over the preceding eight months as EliseAI made massive engineering investments in fine-tuning open-source language models. Instead of defaulting to hosted frontier APIs, Martone organized a dedicated research squad to determine precisely where fine-tuned models would yield high-leverage business returns. Having previously tackled complex theoretical physics questions gave the research group the confidence to build and customize architectures from scratch. That initiative delivered custom models that match or exceed generalized commercial solutions within EliseAI’s specific production workflows.

- Martone previously worked as a theoretical physics professor in London. - The company directed heavy research and infrastructure capital toward testing and fine-tuning open-source models over an eight-month push. - The team leverages scientific first-principles reasoning to independently architect domain-specific solutions without reliance on external validation.

---

`15:00` High-Stakes Automation in Essential Industries

Dew references a LinkedIn post by Martone examining the common tragedy of brilliant academic researchers producing minimal real-world impact, noting the contrast with EliseAI's mission to apply scientific rigor to societal cornerstones like housing and healthcare. Martone notes that the breakneck pace of foundational models requires anticipating operational horizons: “Six months from now, what problems will be solvable with LLMs that are not solvable now?” In an operational enterprise, these strategic bets carry major implications, as predicting technical capabilities six to twelve months in advance dictates whether an engineering team leads or lags.

Maintaining high-reliability autonomous operations is uniquely challenging for a company of a few hundred employees that processes millions of calls alongside hundreds of thousands of daily emails and SMS messages. Because automated workflows control critical interactions in housing and healthcare, failures have tangible, severe consequences for human lives. Martone stresses: “These impact, again, these are housing and healthcare. These are not like some random thing. These actually impact people's life. There's consequential changes that happen if we make a mistake.” To oversee this volume safely, a compact group of roughly ten researchers engineered an evaluation architecture capable of inspecting system-wide conversational health on an ongoing, daily basis.

The leasing cycle illustrates this autonomous architecture end-to-end: 1. An apartment seeker discovers a property listing on the web. 2. The AI guides their scheduling and issues digital entry credentials. 3. The prospective tenant arrives at the property, unlocks the door using the AI-provided access code, and completes a tour accompanied by conversational AI providing contextual explanations of the home. 4. The entire transaction—from inquiry to lease onboarding—can proceed without human agent intervention.

Resident maintenance requests follow a similarly automated workflow up to the physical labor threshold. While software cannot yet replace the mechanical intervention of a human contractor—such as fixing a plumbing leak—the AI system autonomously intakes the tenant’s work order, triages urgency, determines whether an immediate dispatch is required, and arranges the maintenance crew.

Addressing vertical versus horizontal expansion, Martone points out that EliseAI does not need to rush into unrelated industries. Together, the housing and healthcare sectors account for roughly 40% of the entire United States GDP, leaving immense headroom for vertical and horizontal penetration before exhausting their core markets.

- EliseAI operates with a team of a few hundred employees while scaling to millions of automated calls and hundreds of thousands of daily SMS and email interactions. - A core team of ~10 researchers monitors and evaluates conversational safety and reliability across production pipelines. - Housing and healthcare represent approximately 40% of US GDP, justifying deep focus on end-to-end automation within those sectors before targeting additional verticals.

`20:00` Tackling Affordable Housing and Automating Complex Workflows

EliseAI pairs deep domain understanding with business knowledge, leaning into customer feedback to determine the next major technological bets. Mario Martone explains that the team is heavily focused on cutting down the operational costs of affordable housing. This segment of real estate supports the most vulnerable populations with the fewest means, yet it represents a massive portion of the broader housing market. The day-to-day operations in affordable housing remain almost entirely unautomated, with existing technology being virtually non-existent despite the staggering administrative and regulatory complexity required to manage these properties.

Martone highlights that modernizing this sector could produce an immediate, direct social dividend. If software and applied AI can lower the overhead of managing affordable units, those efficiency gains translate into lower costs and better living experiences for tenants. As Mario Martone explains: “So one of the things, for example, that we are now facing very pretty heavily is to really cut down the cost of running affordable housing.” The difficulty of the domain is precisely what makes it compelling from an engineering and mission standpoint, as high friction and zero baseline automation offer a blank canvas for transformative technical intervention.

- Affordable housing operations currently feature near-zero software automation despite representing a major share of total real estate. - Reducing operational overhead directly improves affordability and service quality for vulnerable residents. - Product roadmaps at EliseAI are actively steered by direct feedback from real estate and healthcare operators.

---

`21:09` Hiring Engineering and Academic Research "Unicorns"

To execute against these goals, EliseAI is scaling its technical presence across two primary hubs: New York and San Francisco. Martone emphasizes that the company maintains an exceptionally high hiring bar and is actively seeking talent across two key disciplines. First, they need world-class software engineers because deploying reliable production AI requires an enormous foundation of core systems and infrastructure engineering. The San Francisco office in particular is expanding rapidly, though Martone notes they can barely hire fast enough to keep pace with demand.

Second, Martone looks for researchers who can bridge deep intellectual rigor with pragmatic problem-solving. Given his own academic roots, he looks for individuals with rigorous research training who possess an innate drive to see their theoretical ideas manifest in real production systems. He acknowledges that such candidates are exceedingly rare "unicorns" who must combine elite analytical depth with an urgent desire for tangible real-world impact. As Mario Martone puts it: “People who have exceptional academic background, but yet they are really interested in driving that into real world.”

- Primary technical hiring is concentrated in EliseAI’s New York headquarters and its fast-growing San Francisco hub. - Roles span core infrastructure and systems engineering as well as applied AI research. - Ideal research candidates possess deep academic backgrounds paired with an intense motivation to solve tangible real-world problems.

---

`22:42` Why Real-World AI Complexity Outweighs String Theory

Host Sophia Dew closes the conversation by reflecting on Martone's unconventional path from theoretical physics into production AI, noting how striking it is to see formal academic frameworks applied to messy real-world domains like housing and healthcare. Martone offers a candid parting thought on how working on applied production AI compares to fundamental physics. Despite spending fifteen years in academia studying string theory and high-energy physics, he finds that building reliable, production-ready AI systems deployed in high-stakes environments presents an even higher level of intellectual and practical complexity.

As Mario Martone observes: “Even coming from string theory, I would say that the complexity of the problem of working now is probably higher now than it was before.” Dew highlights this as a remarkably high bar, remarking that listeners are hearing it here first. She concludes the episode with a call to subscribe and share the podcast across platforms like YouTube, Apple Podcasts, Spotify, X, and Substack, followed by standard investment disclosures.

## Mentioned EliseAI, Mario Martone, Sophia Dew, Mina Song, Tony Yip, New York Stock Exchange, X, YouTube, Apple Podcasts, Spotify, Substack

## Video description What changes when an AI mistake doesn’t just hurt a benchmark score, but affects someone’s housing or healthcare? EliseAI Head of Applied Research Mario Martone argues that deploying AI in essential industries requires a very different standard of reliability. Mario joins MTS to explain how EliseAI handles roughly 5 million phone calls a month, why LLMs aren’t the right solution for every problem, and how the team combines language models, traditional machine learning, neural networks, and internally built evaluations to decide what actually works in production. Drawing on 15 years studying theoretical physics, Mario also explains how the scientific method shapes EliseAI’s research culture, from testing new models against its own benchmarks to fine-tuning open-source models and automating increasingly complex workflows across housing and healthcare. And he makes the case that applying AI to the real world can produce problems every bit as intellectually difficult as the ones he encountered studying string theory.

[Source](https://mgln.ai/e/1344/afp-848985-injected.calisto.simplecastaudio.com/3f86df7b-51c6-4101-88a2-550dba782de8/episodes/85a52844-6698-4658-9957-0133eabffba6/audio/128/default.mp3?aid=rss_feed&awCollectionId=3f86df7b-51c6-4101-88a2-550dba782de8&awEpisodeId=85a52844-6698-4658-9957-0133eabffba6&feed=JGE3yC0V)

中文简报

面对年均6000万次通话的故障风险，EliseAI限制大语言模型的使用学术基准中标准的90%准确率在企业级规模下必然导致灾难性业务瘫痪，迫使关键行业依赖专有确定性模型与持续的内部评估。前理论物理学家Mario Martone阐释了EliseAI如何将生产级AI部署到住房和医疗等高风险行业，这两大领域合计约占美国GDP的40%。他说明了为何备受推崇的90%学术基准准确率在每年6000万次通话的规模下会失效，从而需要以内测经验评估来追求99.999%的可靠性。该工程团队并未将大语言模型（LLM）视为万能方案，而是将任务分流至微调开源模型、专有深度神经网络以及传统机器学习，以满足1.5秒以内的严格语音延迟限制并确保执行的确定性。运用科学第一性原理，Martone的团队实现了从公寓自助看房到减轻保障性住房行政负担等一系列复杂业务流程的自动化。 Mario Martone在从事了15年弦理论研究的理论物理学生涯后转型加入EliseAI，他认为现实世界中的应用AI所面临的业务与认知复杂性，甚至超过了基础物理学。 (0:00) EliseAI每月处理约500万通电话（每年6000万通），并保持在1.5秒的严格延迟窗口内，这使得学术界90%的准确率基准在住房和医疗等高风险业务流程中变得完全不可接受。 (7:11) EliseAI并未完全依赖大语言模型，而是部署了多模型架构，将微调语言模型、专有深度神经网络和传统机器学习相结合，用于处理租约续签预测等确定性和表格型任务。 (5:00) 该公司正将目标对准保障性住房运营，力求为这一行政流程复杂且以往缺乏自动化的领域引入自动化技术，从而降低弱势居民相关的运营管理开销。 (20:00)

Reading summary

Former theoretical physicist Mario Martone argues that deploying applied AI in high-stakes industries like housing and healthcare demands far greater reliability and operational rigor than academic benchmarks require. Because mistakes carry immediate real-world consequences, EliseAI relies on specialized hybrid architectures and proprietary empirical evaluations rather than treating large language models as universal solutions. Academic Benchmarks Fail at Enterprise Scale (7:11) In academic research, an artificial intelligence model scoring 90% accuracy on a standard benchmark is routinely celebrated as a breakthrough. However, Martone explains that this metric falls apart under enterprise workloads. EliseAI processes roughly 5 million phone calls every month—about 60 million per year—alongside hundreds of thousands of daily emails and SMS messages. At that operational volume, a 10% failure rate produces millions of dropped medical queries, disrupted leasing appointments, and broken workflows. Architectural Diversity Beyond Universal LLMs (5:00) Martone critiques the industry tendency to apply LLMs indiscriminately to every problem. Conversational voice systems must execute tens to hundreds of distinct sub-operations inside an end-to-end latency budget of approximately 1.5 seconds. While fine-tuned LLMs excel at dialogue and open reasoning, they struggle with strict determinism and fine-grained probabilistic calibration. To manage non-conversational tasks such as predicting lease renewals over large tabular datasets, EliseAI utilizes proprietary deep neural networks and traditional statistical machine learning models. The company also avoids hosted frontier model APIs for core tasks, having spent eight months fine-tuning open-source models internally to match or exceed commercial alternatives within specific production pipelines. Empirical Testing and Latency Diagnosis (10:00) To prevent chasing unverified external claims from social media and public leaderboards, Martone’s applied research team—consisting of roughly ten researchers—operates an automated internal evaluation pipeline. Candidate models are immediately tested against real-world production interactions to measure their impact on edge cases and system reliability, with the engineering goal of pushing past 99% accuracy toward 99.999%. Because EliseAI engineered its voice stack in-house, the team maintains full architectural visibility over granular latency sources across inference pipelines, speech-to-text, and text-to-speech modules. Martone notes that while five or six test samples often make an experimental model look viable, empirical rigor is required to see how performance degrades across millions of real calls. High-Stakes Automation in Massive Sectors (15:00) Rather than expanding horizontally across diverse industries, EliseAI concentrates exclusively on housing and healthcare, sectors that collectively account for approximately 40% of United States GDP. Autonomous agents guide prospective tenants through end-to-end leasing cycles—from discovery and credential issuance to self-guided tours and lease execution—and triage urgent resident maintenance requests before humans are dispatched. A key engineering priority is reducing operational overhead in affordable housing, an area characterized by heavy regulatory complexity and almost zero existing software automation. Martone asserts that lowering administrative costs in this segment creates a direct social dividend by making housing more manageable for vulnerable populations. Martone concludes that navigating the poorly defined problems and zero-tolerance reliability demands of real-world infrastructure presents higher practical complexity than theoretical physics research. To support this scale, EliseAI is actively expanding infrastructure engineering and applied research teams in New York and San Francisco, though Martone acknowledges that finding 'unicorn' researchers who combine academic depth with a drive for immediate operational impact remains an ongoing challenge.

精炼阅读摘要

EliseAI 应用研究负责人 Mario Martone 认为，在住房与医疗等关键民生领域部署高可靠性 AI，其工程与运行复杂度甚至超过弦理论等理论物理研究；学术界普遍推崇的 90% 基准准确率在每年数千万通真实通话的产业规模下完全不可接受。学术基准缺陷与生产级可靠性鸿沟 (7:11) 在学术实验室中，大语言模型在公开基准上达到 90% 的准确率通常被视作顶尖突破，但 Martone 指出这一标准在工业级真实场景中存在致命漏洞。EliseAI 目前每月处理约 500 万通电话，全年处理量接近 6000 万通，外加每日数十万条短信与电子邮件。在此等吞吐量下，10% 的失误率意味着每年产生数百万次失败交互，直接导致医疗随访遗漏或租约签订差错，这在关键民生行业是完全无法承受的风险。面对这一现实瓶颈，EliseAI 放弃依赖通用的第三方公开排行榜，转而建立由前高校教授和底层系统工程师主导的专有内部评测体系（evals）。该系统持续对数百万真实对话样本进行实证测试，追踪模型更新对长尾边缘情况的影响。Martone 强调，从 99% 的准确率向 99.999% 的超高可靠性迈进，工程难度呈现指数级上升；唯有通过日级的高密度安全评估架构，才能在数百人规模的企业运作中保障重大民生决策的安全落地。反对大模型万能论与混合模型架构设计 (5:00) 针对当前软件行业倾向将大语言模型（LLM）当作“万能锤”解决所有工程问题的潮流，Martone 提出明确质疑。他指出，大语言模型在面对表格数据的高精度概率校准以及严格确定性输出（同一输入必须得到完全一致的结果）时表现较弱。为此，EliseAI 的应用研究团队将业务任务进行严格分流：开放式推理与多轮对话交由经过微调的大语言模型处理，而涉及海量结构化数据的租约续约风险预测等任务，则交由自研的深层专有神经网络和传统统计机器学习模型运算。此外，在企业级实时语音交互场景中，系统面临严苛的物理限制。整套语音交互必须在约 1.5 秒的端到端延迟窗口内协调几十至上百项离散子操作。EliseAI 团队早在三年前低延迟端到端语音模型尚不成熟时就自研了全套语音栈，这种自底向上的自主掌控使团队对每个推理环节的延迟瓶颈具备完整的白盒能见度。面对社交媒体或第三方厂商的宣传，团队均将其接入内部自动化测试框架严密验证，避免盲目跟随外部声称的技术指标。物理学第一性原理驱动开源模型定制 (13:00) 曾在伦敦高校从事理论物理与弦理论研究 15 年的 Martone 指出，处理黑洞和量子场论所培养出的核心认知素养——在定义不清的复杂前沿问题中深入思考并构建全新解题框架——能无缝迁移至日新月异的应用 AI 领域。前沿基础研究实验室往往与学术界类似，研究成果与现实风险隔离；而将物理学严谨性注入应用软件工程，则是攻克现实难题的关键驱动力。在过去八个月的攻坚中，EliseAI 并未完全依赖主流闭源的前沿商业 API，而是投入大量研发与算力资本专门针对开源基础模型进行高强度微调。依靠科学的第一性原理推理与架构定制，团队打造出的专用微调模型在特定垂直工作流中的表现达到甚至超越了通用商业闭源方案。Martone 强调，应用型 AI 研发必须具备预测未来技术演进的视野，提前 6 至 12 个月预判大模型能够攻克的边界，才能在技术浪潮中保持领先地位。深耕民生支柱产业与平价住房痛点 (15:00) 在商业战略拓展方面，Martone 明确表示 EliseAI 暂不急于横向扩张至无关领域。住房与医疗两大核心板块合计占据美国国内生产总值（GDP）的约 40%，具备广阔的纵深探索与落地空间。目前 EliseAI 已经实现了租房全流程的端到端自主自动化：从潜在租客在线看房咨询、自动生成电子门禁凭证，到租客持码实地由语音 AI 陪同看房与实时答疑，整个交易闭环均可在无人工中介介入的情况下完成；物业维修场景下，系统也能实现报修工单自动识别、紧急程度分级并自主调度工程团队。团队当前正重点攻坚平价住房（Affordable Housing）的运营降本问题。该领域服务于社会中最缺乏经济支持的弱势群体，具有极高的行政与法规合规复杂度，然而其日常运营几乎完全处于零软件自动化的原始状态。Martone 认为，通过应用 AI 消除繁复的行政管理冗余，所带来的效率收益将直接转化为弱势群体的居住成本下降和服务体验改善，具有极高的社会效益与技术改造价值。双中心引才策略与跨界研究型人才诉求 (21:09) 为了支撑业务体量的高速增长，EliseAI 正在纽约总部与快速扩建中的旧金山办公室加速扩充技术团队。Martone 指出，团队目前面临着极其迫切的人才需求，招聘方向聚焦于两类极其稀缺的核心角色。第一类是负责大规模底层系统与基础架构的高阶软件工程师，以支撑数千万量级并发通话与实时消息的高可用流转。第二类则是兼具顶尖学术背景与强烈现实落地意愿的应用研究“独角兽”人才。Martone 坦言，优秀的学术科研人员往往陷入成果无法落地的困境，而 EliseAI 期望招募具备严密逻辑推演能力、同时渴望通过自身算法直接改变真实世界现实运作的跨界学者。这种兼顾学术底蕴与实用主义的团队配置，是公司用小规模核心研究团队稳定管控海量高风险线上交互的关键所在。 Martone 总结认为，相较于纯粹抽象的物理学理论探讨，在住房与医疗等高风险真实世界场景中构建永不宕机、高精度的 AI 决策系统，面临着更高的现实综合复杂度。团队目前仍需在维持旧金山与纽约工程团队极高招聘门槛的同时，持续探索如何以低成本自动化方案深度改造平价住房等极度缺乏数字化的传统行业。

Detailed notes — zh

## Background 本期内容由 Mario Martone 与 Sophia Dew 于 2026 年 10 月 18 日在 RSS 及网络播客平台发布。页面简介显示，EliseAI 应用研究负责人 Mario Martone 探讨了当 AI 错误直接影响民众住房或医疗等关键民生时，部署生产级系统所面临的严苛可靠性标准，以及如何结合大模型、传统机器学习与自建评测系统处理每月数百万通电话。 ## TL;DR 拥有 15 年理论物理与弦理论研究背景的 Mario Martone 阐述了将科学严谨性迁移至工业落地的经验，指出在现实产业中部署 AI 的复杂性甚至超越了理论物理研究。针对每月高达 500 万通电话的超大流量，EliseAI 破除“大模型万能论”，将前沿大语言模型与自建深度神经网络、传统机器学习紧密协同，分别处理对话意图与确定性概率预测。为跨越 99.999% 的工程可靠性门槛，团队依托严密的自建评估体系（Evals）持续治理语音延迟与模型表现。目前团队深耕合计占据美国 GDP 40% 的住房与医疗两大垂直赛道，并重点攻坚保障性住房的端到端自动化降本增效。 ## Atmosphere & style 整场对话呈现出一种兼具学术严谨与工业务实的硬核对谈氛围。物理学家的抽象推演与真实业务场景下的残酷工程约束相互碰撞，主持人的精准提问逐步抽丝剥茧，引出嘉宾对于“前沿实验室象牙塔”与“现实世界高容错代价”的深刻反思。语调冷静、客观且充满思辨力量，让听众能切实感受到高难度理论科研与关乎数百万人民生利益的工程实践相融合时的智力张力。 ## Bulletin - `07:30` EliseAI 每月处理约 500 万通真实业务电话（年化 6000 万通），覆盖日均多达 100 万通电话及数十万条短信与邮件，要求系统具备超越学术评测的超高工程可靠性。 - `05:00` 针对续租风险预测等依赖海量结构化数据的场景，EliseAI 放弃高成本且缺乏确定性的大语言模型，采用自建深度神经网络与传统机器学习协同的混合技术架构。 - `10:00` 为突破语音实时交互 1.5 秒的严苛延迟约束，团队摒弃外部公开基准宣称，搭建端到端自研架构与私有评估闭环（Evals）系统化验证模型表现。 - `18:00` 公司业务深度聚焦住房与医疗两大占据美国约 40% GDP 的民生核心赛道，并正全力攻坚手工流程多、技术介入近乎空白的保障性住房运营降本场景。 ## Key takeaways - `00:00` 学术界 90% 的准确率在工业级规模下意味着灾难，在每年 6000 万通电话量级下，0.1% 的失误都会直接冲击普通人的住房与就医权益。 - `00:00` 从理论物理研究转向应用 AI，解决现实产业问题的复杂度与变量在综合维度上甚至超越了黑洞与弦理论的学术推演。 - `03:00` 相较于停留在象牙塔式闭环推演的前沿实验室，应用型 AI 的核心驱动力在于模型决策能否直接渗透进法律合同、排期调度等实体业务链条。 - `05:00` 破除盲目崇拜“大模型万能论”的误区，在确定性逻辑、大规模结构化回归任务中，传统 ML 与专用深度神经网络的效率远胜 LLM。 - `05:00` 语音实时交互不仅面临 1.5 秒内调度上百次计算检索的工程极限，还需要在数百万并发流量分化下依靠严密的端到端指标控制延迟。 - `07:30` 依托由理论学者与顶尖工程师混编的团队，自建的大规模内部评估体系（Evals）成为衡量每次算法迭代物理因果反馈的唯一准绳。 - `10:00` 面对开源模型与外部 STT/TTS 方案的宣称，团队建立私有基准实测闭环，持续将测试链条推进至自动化回归流水线。 - `13:00` 理论物理训练赋予团队直面极端前沿未知时独立设计突破性假说并自研微调模型的研究定力，而非盲目照搬行业通用做法。 - `15:00` 应用团队面临的最大智力挑战是前瞻预测：必须推演未来 6 到 12 个月内大语言模型将突破的瓶颈，提前进行业务痛点架构布局。 - `15:00` 住房租赁全链路从咨询、预约看房、动态发码到带看讲解已实现无人化闭环，除必须上门的实体物理维修外前期调度全由 AI 接管。 - `18:00` 团队战略选择深耕住房与医疗赛道而非盲目泛化，因二者合计占据美国约 40% 的 GDP 规模，具备极深的产业护城河与技术发挥空间。 - `20:00` 保障性住房长期缺乏自动化工具且高度依赖手工，借助 AI 端到端压缩其复杂运营成本具有重大的社会公益与商业价值。 - `21:26` 研发人才招聘重点锁定高水平工程专家与兼具顶尖学术功底及落地热情的“独角兽”研究员，以应对交织极高社会责任的复杂工程难题。 ## Detailed walkthrough ### `00:00` 从弦理论到真实世界：前沿物理学家转型应用 AI 在学术界，一篇论文如果能证明 AI 系统在基准测试中达到 90% 的准确率，往往就足以被称为重大突破；但在现实生产环境中，当系统每月需要处理数百万通关乎普通人住房和医疗等切身利益的电话时，那剩下的 10% 缺陷就会演变成一场巨大的灾难。作为 EliseAI 的应用研究负责人（Head of Applied Research），拥有双博士学位的物理学家 Mario Martone 指出，将 AI 从象牙塔式的实验室推向关键民生行业，对可靠性的要求有着天壤之别。Mario 曾在理论物理领域深耕 15 年，专注于研究黑洞与弦理论等宏观与微观宇宙的终极奥秘，而如今他将那种严密的科研思维全面带入了工业界。在与主持人谈到从大学教授转变为应用 AI 负责人的跨界时，Mario 坦言两者的节奏截然不同。在弦理论的研究中，由于探索的是自然界最底层的复杂定律，一个问题的推演往往极其缓慢且旷日持久；而在 AI 领域，技术的迭代一日千里。但 Mario 强调，两者在思维内核上是高度一致的——面对未知而快速演进的宏观环境，那些能够深入思考底层机理、具备强创造力并能不断提出新假设的人，才能拥有巨大的竞争优势。正如 Mario Martone 所总结的：“I did not expect that, but even after coming from string theory, I would say the complexity of the problem of working now is probably higher now than it was before.” - **学术背景**：Mario Martone 拥有双博士学位（“Dr. Doctor”），在理论物理界担任过大学教授，拥有 15 年探索宇宙起源、黑洞以及弦理论的研究经验。 - **业务领域**：EliseAI 专注于为住房（Housing）与医疗（Healthcare）等关乎基本民生的关键行业（Essential Industries）构建会话式 AI 系统，覆盖从租房咨询、预约看房、租约续签到入住全流程。 ```mermaid flowchart TD accTitle: 理论物理思维向真实世界 AI 的转化框架 accDescr: 展示 Mario Martone 从 15 年理论物理学术研究向 EliseAI 真实世界工业落地转型的思维演进与现实影响。 subgraph Theory["理论物理 (15年)"] T1["深入思考与底层探索"] T2["长周期、慢节奏的理论假设"] end subgraph Industry["前沿应用 AI (EliseAI)"] I1["应对日新月异的技术变化"] I2["直接影响现实决策 (住房/医疗)"] end T1 --> I1 T2 -->|"思维迁移与加速迭代"| I2 ``` --- ### `03:00` 走出前沿实验室：让模型决策真正渗透进现实行业主持人谈到，上周在纽约证券交易所与 EliseAI 联合创始人 Minna Song 交流时，他们就深入讨论过“让机器智能扩散至实体行业”的迫切性。仅仅在头部前沿实验室（Frontier Labs）里训练出算力更强、参数更大的基准模型，并不等同于这些智能真正被应用到了现实世界中。对此，Mario 深有感触。他透露自己在离开理论物理界时，最初也曾考虑过加入这些顶级前沿实验室，因为在那里的研究方式其实与理论物理极为相似——都是在一群学者构成的闭环体系中，针对宏观的学术大问题进行推演。但他最终选择加入 EliseAI，是因为他渴望自己的工作能对现实社会产生切实的物理冲击。在许多消费级场景中，大语言模型（LLM）生成的文本只是屏幕上浮现的闲聊或建议；但在 EliseAI 打造的产品中，AI 做出的每一次决策都在直接驱动并改变真实人群的生活——从能否顺利租到公寓、安排看诊，到紧急维修与续约。这种决策带来的后果是极其真实的，既可能带来极大的便利，也可能造成极其严重的负面影响。这种高风险、高影响力的落地场景，构成了驱动应用研究团队不断前行的核心动力。 - **前沿实验室与落地公司的分歧**：在顶级 AI 研究院做纯研究，模式更接近理论物理；而在应用型公司做 AI，核心在于将底层智能注入复杂的实际生产链条。 - **决策的实体影响力**：EliseAI 的系统所输出的不是玩具式的对话，而是直接触发法律合同、租约状态、房屋巡查及排期等现实动作。 --- ### `05:00` 破除“大模型万能论”：语音瓶颈与传统机器学习的协同在探讨 EliseAI 的技术栈时，Mario 强调了工程落地的极高复杂性。早在三年前，公司就在联合创始人 Minna 与 Tony 的远见下提前押注了语音技术，那时底层模型和基础设施甚至还不成熟。语音交互的难度在于极度严苛的延迟与吞吐要求：系统必须在短短 1.5 秒之内，高并发地协同调度数十次乃至上百次计算与检索操作，同时还要准确无误地理解说话人的即时行为与意图。针对当前很多新进入 AI 领域的人盲目将 LLM 视为“万能解药”的行业倾向，Mario 直言这是严重的误区。例如在预测租客“续租风险”（Renewal Risk）这类重度依赖海量历史结构化数据的场景下，大语言模型不仅算力成本高昂，而且效果极差。EliseAI 内部为此自主研发了复杂的自建深度神经网络架构，同时结合了大量传统机器学习模型，完全不依赖 LLM。Mario 解释道，大模型在表面上虽然能生成看似漂亮的答案，但在需要绝对确定性（Determinism）与严谨概率推理（Probabilities）的任务上面临固有缺陷，而负责应用研究的特权与挑战，正是站在全局视角挑选最合适、最高效的工具组合。正如 Mario Martone 所批评的：“And so here, it's like many people who have joined very recently, they just have one solution for everything, LLMs.” | 技术维度 | 前沿大语言模型 (LLMs) | 传统机器学习与自建深度神经网络 (ML / In-house DNN) | |:--- |:--- |:--- | | **优势领域** | 开放域自然语言对话、意图理解与语义映射 | 确定性逻辑、海量结构化数据回归、续租概率预测 | | **核心短板** | 缺乏严格确定性、输出存在概率幻觉、数学概率推演弱 | 缺乏开放语境理解能力、冷启动复杂 | | **典型应用** | 租客电话智能应答、上下文多轮协商 | 租约续签概率计算（Predicting Renewal Risks） | --- ### `07:30` 跨越“99.999%”的高山：自建真实评估体系应对巨量通话规模当系统承担着庞大的实际调用体量时，容错率被压缩到了极致。在学术论文中，研究人员往往认为模型准确率达到 90% 甚至 99% 就已经非常理想；但对于 EliseAI 而言，平台每年需要处理高达 6000 万通真实电话（每月约 500 万通），在这样的规模下，哪怕只有 1% 甚至 0.1% 的错误率，其所对应的失败绝对次数都代表着数以万计甚至数十万被搞砸的现实业务。Mario 戏称，工程上每多追求一个“9”（从 99% 攀升到 99.999%），其难度都像是在攀登一座全新的高山。为了攻克这一难题，EliseAI 构建了一支由顶尖大学教授、理论学者与顶级工程开发人员混编的研发团队，并全力构建了一套内部评估体系（Evals）。这一评估框架的核心目的，在于从每天数以百万计的真实对话与决策流中，系统化、规模化地衡量每一次微调或算法迭代在全链路中产生的实际连锁反应。只有依靠这种基于物理科学式的严格实验闭环，研发团队才能跳出论文上的理论指标，直面真实生产环境下的因果反馈。正如 Mario Martone 所阐述的：“Evance is kind of the answer to your questions. Like, can we, we have these millions of conversations every day. We make decisions. Can we understand at scale the impact of those decisions?” - **通话规模数据**：EliseAI 每月处理约 500 万通电话，全年处理量高达 6000 万通。 - **学术与工业的容错脱节**：学术界视为优异的 90% 成功率在 6000 万通话量级下意味着每年 600 万次失败，这在住房与医疗场景中是不可接受的灾难。 - **核心解决方案**：依托由顶级学者与顶尖工程师构成的团队，以自建的大规模评估系统（Evals）作为技术迭代的准绳，持续系统化地监控并优化模型决策的真实影响。 ### `10:00` 语音延迟治理与科学闭环评估系统在大规模生产环境中，AI 交互系统的表现会发生剧烈分化。五六通对话的小规模样本中，各类方法看似表现良好；然而一旦进入数百万级的流量，不同技术路径的差距便会成倍放大。针对 EliseAI 每月需处理约 500 万通电话（每年约 600 万通）的超大负载，主持人提问团队如何在硬件与软件交织的复杂约束下，攻克语音实时交互的关键瓶颈——延迟。 ```mermaid flowchart LR accTitle: EliseAI 语音延迟优化与模型评估闭环 accDescr: 展现从全栈瓶颈诊断到新模型接入与内部基准评测的闭环流程。 subgraph STACK["全栈诊断"] A["端到端自研架构"] --> B["定位最耗时瓶颈环节"] end subgraph MARKET["方案探索"] C["开源模型微调"] D["新型 STT / TTS 模型"] end subgraph EVAL["闭环评估体系"] E["Slack 捕捉前沿情报"] --> F["内部基准自动化评测"] F --> G["真实验证后全量部署"] end B --> EVAL C --> EVAL D --> EVAL ``` Mario Martone 指出，解决这一问题的核心在于从两个维度夹逼：一方面是对现有技术栈的透彻把控，另一方面是持续跟踪外部市场动向。由于底层架构大部分由 Mario Martone 亲手搭建，团队对整个技术链路拥有完全自主的把控力，能够清晰定位究竟是哪个环节拖慢了耗时。对于外部层出不穷的开源微调、新型语音转文字（STT）和文本转语音（TTS）模型，团队并不盲信外界宣传，而是建立了一套极度敏捷的内部实验闭环。当员工在 Slack 上分享社交平台 X 上的最新技术突破时，团队会直接接入系统，在私有基准上运行评测，以此验证模型的真实水平。 - 语音业务规模：每月处理约 500 万通电话，折合年处理量约 6000 万通。 - 技术闭环机制：针对外部供应商或开源社区的宣称，绕过公开基准，统一采用 EliseAI 内部基准进行实测。 - 自动化探索：目前评测链路仍包含部分人工介入，团队正致力于将其升级为全自动化的模型回归测试流水线。 > “在小规模下，比如五六次对话，看起来都挺好。但当规模扩大到数百万级别时，你就会发现不同方法之间出现了巨大的分化。” —— Mario Martone --- ### `13:00` 理论物理思维在模型微调与底层创新中的迁移谈及从伦敦知名学府的理论物理学教授转型至工业界的研究经历，Mario Martone 分享了高难度基础科学训练对 AI 研发文化的深刻影响。他认为，除了数学建模与编程能力等硬技能之外，理论物理学者最重要的资产在于“敢于直面极端前沿难题并独立提出突破性假说”的思考范式。在面临未经验证的复杂领域时，这种思维方式能赋予团队极强的研究自信。过去八个月中，EliseAI 重点发力开源大模型的微调探索。在面对模型适配边界与应用场景等未知数时，团队没有照搬行业通用做法，而是成立了专门的研究小组，依靠底层科研直觉设计训练策略。Mario Martone 强调，敢于独立开辟路径的研究定力，使他们最终在自研模型上取得了卓越成效，并顺利将成果推向生产环境。 > “理论物理处于人类逻辑推理与思考的最前沿。因此，提出全新的想法从来都不容易……如果这事很简单，那每个人都能做到了。” —— Mario Martone --- ### `15:00` 关键产业落地挑战：前瞻预测与日均百万级通话自动化针对在领英上探讨的“顶尖科研背景与极度匮乏的现实影响力”这一反差，Mario Martone 表达了将科学严谨性注入住房与医疗等基本民生领域的极高热情。他指出，团队当前面临的最大智力挑战之一是“前瞻预测”：由于大语言模型演进极其迅猛，团队必须时刻推演未来 6 个月甚至 1 年内，有哪些眼下不可行的难题将被 LLM 解决，从而提前在涉及数百万套公寓的业务痛点前进行战略布局。 ```mermaid flowchart TD accTitle: 住房全流程自动化与物理干预边界 accDescr: 展示从租房前看房到租后报修的自动化流程及其实体物理边界。 subgraph LEASE["租赁流程：全链路自动化"] L1["网站浏览与咨询"] --> L2["自主预约看房"] L2 --> L3["AI 发放门禁码并远程讲解"] L3 --> L4["签约与正式入住"] end subgraph MAINT["租后维修：数字与物理分流"] M1["住户报修工单"] --> M2{"AI 研判紧急程度"} M2 -->|"分类流转与排期"| M3["自动化工单系统"] M3 -->|"物理维修断点"| M4["实体技术人员上门维修"] end ``` 另一个重磅挑战是在团队规模仅有几百人的前提下，实现日均百万级语音通话、数十万条邮件与短信的无人值守闭环。由于住房与医疗业务关乎重大民生福祉，错误代价极高，绝不能容忍不可控幻觉。EliseAI 构建了一套极其严密的自动化评估机制，使得仅需极小规模的核心团队，即可每日全方位监控海量会话质量。在租房业务端，从线上咨询、预约看房、动态分发门禁码到 AI 实时导览并促成入住，已全链路无人化；而在维修业务端，除必须依赖实体人工上门的实际物理修理外，工单分类、紧急度判定与调度排期均已实现自动化。 - 运营吞吐量：每天自动化处理多达 100 万通电话，以及数十万条短信和电子邮件。 - 租赁全流程无人化：涵盖看房预约、门禁授权发放、AI 伴随式带看讲解与入住引导。 - 物理瓶颈界限：当前维修场景的完全闭环仍受限于机器人技术进展，需真人师傅上门作业，但前期意图研判与工单调度已完全由 AI 接管。 > “我们开发了完全自动化的闭环。这非常棒、非常了不起。但另一方面，你必须真正建立起评估体系……因为这些业务关乎住房与医疗，不是什么无关紧要的琐事，而是切切实实会影响人们的生活。” —— Mario Martone --- ### `18:00` 垂直深耕与横向扩张：聚焦占据美国 GDP 40% 的核心赛道面对 EliseAI 是继续沿纵向技术栈下潜还是向横向行业扩展的战略选择，Mario Martone 给出了清晰的判断。团队过去在住房垂直领域建立起了深厚的系统工程壁垒，打穿了除实体机器人上门之外的所有业务断点；而如今正在将这一套经过实战验证的智能化闭环能力，横向平移至医疗健康领域。针对未来是否跨足更多垂直行业的疑问，Mario Martone 直言团队目前专注于深耕这两大赛道。住房与医疗合计占据了美国约 40% 的 GDP 规模，其市场体量与服务纵深为技术创新提供了足够广阔的施展空间。EliseAI 打造高价值 AI 的核心底气，正是在于深耕核心垂直产业所沉淀的不可替代的技术积淀与行业护城河。 > “眼下我们手头的工作已经足够充实了。医疗和住房加起来占据了美国约 40% 的 GDP，因此我们还有很大的增长空间。” —— Mario Martone ### `20:00` 聚焦保障性住房：利用 AI 降低弱势群体的居住成本 Mario Martone 阐述了 EliseAI 团队的核心竞争壁垒：将极具深度的前沿技术理解与垂直行业的业务认知深度融合。正是由于具备这种跨界复合技能，团队能够敏锐地捕捉并押注客户最核心的需求痛点。当前 EliseAI 最重大的发力方向之一，就是大力攻坚保障性住房（Affordable Housing）的运营成本缩减。这一领域对于社会中最脆弱、经济手段最匮乏的弱势群体而言具有直接而深远的影响，但其背后的业务逻辑却极度复杂繁冗，且业内技术水平极度匮乏，目前几乎完全依赖手工流程、缺乏任何自动化工具。保障性住房在整个房地产行业中占据巨大比重，如果能借助 AI 显著压缩其运营成本，将能够立竿见影地改善普通人的生活。正如主持人 Sophia Dew 所评价的，最艰难的现实问题往往能吸引最顶尖的人才。面对这个技术几乎空白却体量庞大的市场，EliseAI 正在积极投入资源，通过端到端工程实践打通复杂业务流，让底层住户和租户直接享受到降本增效带来的实际收益。 - EliseAI 现阶段的核心技术与商业赌注聚焦于大幅降低保障性住房的运营开销。 - 该领域长期以来技术介入几乎为零，业务处理繁琐且完全没有实现自动化。 - 保障性住房占据整个房地产市场极大的体量，具备巨大的社会影响力与商业价值。 --- ### `21:26` 人才招聘标准与“学术独角兽”：从弦理论到现实世界 AI 在谈到团队扩张与招聘策略时，Mario Martone 指出，EliseAI 目前主要在旧金山和纽约两大中心扩充技术团队，且坚持极高的录用门槛。团队当前最迫切需要两类顶尖人才：一是能够扛起海量工程建设的高水平工程师，由于旧金山研发中心业务扩张极为迅猛，工程人才的缺口非常巨大；二是兼具深厚学术功底与工程实践热情的顶尖研究员。Mario 将这类研究员形容为“独角兽（Unicorns）”——他们既具备扎实的学术深度与严密的科学思维，内心又充满强烈渴望，希望将深度思考转化为能够切实改善普通人生活的现实影响力。访谈尾声，主持人回顾了 Mario Martone 从 15 年理论物理与弦理论（String Theory）研究转向现实世界 AI 应用的独特经历。Mario Martone 给出了一句出人意料却掷地有声的定论：“And I want to say to the people out there, I did not expect that. But even after coming from string theory, I would say that the complexity of the problem of working now is probably higher now than it was before.”（我想对大家说，我以前从没料到会这样。但即使在我研究过弦理论之后，我依然认为现在所面对的问题的复杂性，可能比以前更高。）在现实世界的产业落地场景中，AI 面临的变量、工程可靠性与社会责任交织在一起，其智力挑战绝不亚于甚至超越了最深奥的理论物理探索。 ```mermaid flowchart LR accTitle: EliseAI 核心人才画像与价值输出 accDescr: 展示 EliseAI 招募的顶尖工程人才与研究员独角兽如何共同解决高复杂度现实问题 subgraph Talent["人才构成与标准"] E["顶尖工程师 (纽约/旧金山中心急缺)"] R["研究员独角兽 (深厚学术背景 + 强烈落地激情)"] end subgraph Impact["现实工程与社会价值"] C["极高复杂度现实问题 (智力挑战超越弦理论)"] V["改善住房与医疗等基本民生 (直接赋能普通人生活)"] end E --> C R --> C C --> V ``` - EliseAI 在旧金山和纽约大力扩招，重点吸纳工程与前沿应用研究人才。 - 研究员招聘的核心标准：具备顶级学术严谨性，同时极度专注于“产生现实世界影响力”。 - Mario Martone 坦言，在民生关键行业部署生产级 AI 所面对的复杂性，已然超越了弦理论等前沿物理学课题。 ## Notable quotes > `10:00` 在小规模下，比如五六次对话，看起来都挺好。但当规模扩大到数百万级别时，你就会发现不同方法之间出现了巨大的分化。 — Mario Martone > `15:00` 你必须真正建立起评估体系……因为这些业务关乎住房与医疗，不是什么无关紧要的琐事，而是切切实实会影响人们的生活。 — Mario Martone ## Mentioned EliseAI, Mario Martone, Sophia Dew, Minna Song, Tony, 纽约证券交易所, Slack, X ## Video description What changes when an AI mistake doesn’t just hurt a benchmark score, but affects someone’s housing or healthcare? EliseAI Head of Applied Research Mario Martone argues that deploying AI in essential industries requires a very different standard of reliability. Mario joins MTS to explain how EliseAI handles roughly 5 million phone calls a month, why LLMs aren’t the right solution for every problem, and how the team combines language models, traditional machine learning, neural networks, and internally built evaluations to decide what actually works in production. Drawing on 15 years studying theoretical physics, Mario also explains how the scientific method shapes EliseAI’s research culture, from testing new models against its own benchmarks to fine-tuning open-source models and automating increasingly complex workflows across housing and healthcare. And he makes the case that applying AI to the real world can produce problems every bit as intellectually difficult as the ones he encountered studying string theory. [Source](https://mgln.ai/e/1344/afp-848985-injected.calisto.simplecastaudio.com/3f86df7b-51c6-4101-88a2-550dba782de8/episodes/85a52844-6698-4658-9957-0133eabffba6/audio/128/default.mp3?aid=rss_feed&awCollectionId=3f86df7b-51c6-4101-88a2-550dba782de8&awEpisodeId=85a52844-6698-4658-9957-0133eabffba6&feed=JGE3yC0V)

Source quotes

[10:00] Mario Martone: 在小规模下，比如五六次对话，看起来都挺好。但当规模扩大到数百万级别时，你就会发现不同方法之间出现了巨大的分化。 [15:00] Mario Martone: 你必须真正建立起评估体系……因为这些业务关乎住房与医疗，不是什么无关紧要的琐事，而是切切实实会影响人们的生活。

Original transcript

[0:00] I did not expect that, but even after coming from shrink theory, I would say the complexity of the problem of working now is probably higher now than it was before. An AI system that works 90% of the time might look impressive in a research paper, but when you're handling millions of conversations about people's housing and healthcare, that remaining 10% becomes an enormous problem. In this episode, Elise AI Head of Applied Research Mario Martone joins MTS to explain what it takes to move AI from the lab into essential industries. Mario spent 15 years in theoretical physics, studying problems like black holes and string theory. Now he applies that same research mindset to production AI, where his team has to decide when to use an LLM, when traditional machine learning is actually better, and how to evaluate systems operating across millions of real-world interactions. They get into Elise AI's voice stack, its work fine-tuning open-source models, and how the company has automated workflows that can take someone from looking for an apartment to touring it and eventually moving in. And for Mario, the intellectual challenge hasn't gotten easier. He argues that some of the problems his team is tackling in production AI may be even more complex than the ones he encountered in theoretical physics. Hello, everyone, and welcome back to MTS. by Dr. Doctor, because we have two PhDs, so again, a double doctor, Mario Martone. He is the head of applied research at Elise AI, a company building conversational AI for essential industries like housing and healthcare. And he has a background in theoretical physics, physicist. He now leads a team applying the latest machine learning advancements to real world problems. So Mario, welcome to MTS. Thank you. That's great. Yeah, welcome to the show. I did want to ask a little because you started off being a professor and theoretical physicist, and now you're actually applying all of that learning into applied AI. And I was just sharing, we were having a little conversation right before we went live about why it's so important to use these skills and apply them, especially to essential industries like housing and healthcare, probably arguably some of the most essential industries. And so I wanted to just start off learning a little bit about you, Thank you. take with you. Right. Well, I think I agree with you that it's a big gap. So just to be precise, I mostly worked for 15 years on trying to understand the universe, black holes, string theory. Things are very, very kind of different from what I do daily on Analyst AI. But I think there is something that actually is quite important, right? Like we are living in this moment where AI is moving so fast and it's really unclear, right, what the future will hold. And so there are open questions. And so the similarity that I can see, while obviously, you know, working in string theory, things take forever, it's much slower than the pace. And here it's like, boom, boom, boom, every day there's like a change. But the easy question of like, the people who are able to like really think deeply about the problems can be very creative, can constantly kind of come up with new ideas, have a huge edge because AI is like a new field every day. So that's the thing that really, I think, brought me here and allowed me to like the space, one of the big things, I mean, I'm a little bit biased because we just had Nina Song on last week at the New York Stock Exchange. We're talking all about essential industries and why it's so important to actually use this intelligence and get it diffused inside these industries. Just having stronger models on their own inside labs doesn't necessarily mean the impacts of that intelligence actually gets diffused into the real world, into actual industries. I was kind of interested in going to industry is to, for example, try to join one of these like Frontier Lab, right? But the research that you do at Frontier Lab, it's kind of obviously different from theoretical physics, but it's much more alike. Like you're working on these big problems. But one thing I really wanted is to have an impact in the real world. And I think Elise is like such a unique company in that respect because, you know, you can have these LLMs that are getting growing capabilities. decisions. So we actually, the decisions that the LLMs make for Elise, for our products, they're not just like some random things that appears on the screen, but like drive impacts on people's life. And that could be very positive or very negative. So that's the thing that really excites me. And I think it makes Elise kind of such a unique space to be in. Completely. I mean, it excites me too. I was talking, what's unique about your role, so you do applied research at Elise AI. I think right now, Elise AI's tech stack, it includes predicting renewal risks, And so because you're at this layer where you're the head of the applied research angle, I wanted you to just tell me a little bit more about this tech stock and what specifically is really hard about these enterprises. [5:00] engineering problems. Right. Many, many things are very hard. I would say that you touched upon some of the best examples. We were very early on on voice. We've been in it for three years. We definitely, and that's one of the things I love about Mina and Tony and their vision. We definitely started when it was way too early. So we are trying to make it work when the technology wasn't there. But voice is extremely difficult. You're talking about doing tens, not how to hundreds of operations in a span of like a second and a half. And you're also touching upon another thing, like we really need to make sure that we understand people's behavior, right, in a very precise way. So that's another thing that is kind of unique about the lease, is that when you touch upon predicting renewals, we have a mixture of cutting-edge AI in a kind of more LLM's frontier base, but we also use a lot of traditional machine learning, ton of data. And so for that, LLMs are not really the answer. And so we have really a full spectrum of tools that we're using, and we're using them full force. So for example, the predictions of renewals that we're using really elaborate kind of neural network that we built in the house, but they have nothing to do with LLMs. So for me, as a head of live research, it's kind of a privileged position because I don't have to be like kind of tunnel vision on one thing. I can really see the advancement of the entire field of AI across the board. this advancement you're seeing, especially because I think people try to throw LLMs at all these problems and not all of these problems are fully probabilistic. You need some deterministic angles. You need a mix of all these different things. And so you get to have this bird's eye view of just like what's happening at a macro level where the entire field of AI and research is going and then you get to apply it actually at the company. So tell me a little bit more about that. Right. And then, you know, not only that, but like one of the major things is to make producing probability for a person that wants to renew their lease. Should we throw an LLM? Should we throw a deep neural network? Should we throw very simple machine learning models? And I think that's the kind of decision that me and other people have to make early on. And so here, it's like many people who have joined very recently, they just have one solution for everything, LLMs. I understand the feeling because LLMs can generate When you kind of like dig a little bit deeper in the surface, like you see that some of the crap is like not really good. And there are some problems like, for example, having, as you mentioned, determinism is not there. Like trying to read the same behavior given their input, LLMs fail. And probabilities is also something that the LLMs have really hard... time to elaborate. So I need to kind of like look at a problem, try to understand two things. Like one, what's the one the most impactful? And then two, because we also have to ship at a very fast pace. Okay, what's the solutions that can really get us almost as close as possible to the optimal outcome, but in the fastest way possible. And then we have to double down on that. Can you give me an example about this actual method that you're using within your team? So you're able to identify what is wrong with kind of the way that maybe traditional products are trying to build these things, especially when there's high risk here. You can't get it wrong. Like even if it's 99% accurate, that's not good enough. That has to be 99.999. And they joke that every nine that you add is another hill to climb. It's kind of funny because I still read some papers from academia and there you see really the difference. Like when they're talking about reliability, they talk, okay, if the LLM can take it up to 90 percent. And then we say 90 percent? That means for like 60 million phone calls per month, per year, it means like an enormous amount of failures. That is totally unbearable. So you can really see the difference between people who are like thinking about this problem theoretically and people who actually pay the consequences of those mistakes, us. And also like the impact on people's life. And so what we did, which I think is quite exceptional, and to me at least is what really engages me and really keeps me knowing, is to build a mixture of expertise, right? So we have at least we have top academic talent, you know, including like myself, professors of top universities, as well as like top developers. and we're trying to use all these tools to build Evance, to understand. Evance is kind of the answer to your questions. Like, can we, we have these millions of conversations every day. We make decisions. Can we understand at scale the impact of those decisions? And then once you start doing that and you're doing systematically, kind of like the results speak back to you. So you really understand, you really see that difference. If you're just like trying [10:00] At small scale, like five or six conversations, all look quite good. But at scale, when you get to the millions, then you see that different methods diverge significantly. Yeah, no, it's pretty interesting. It's almost like you're applying the scientific method to product design and how you actually build the product. I 100% agree with you. And I think, you know, I'm a strong believer that the scientific method is kind of one of the best ideas that the human accounts have ever generated. you're able to manage voice latency. I know you guys were working on voice and communication way before even some of the Frontier Labs were working on that. But one of the big issues with it is this latency issue because it's not just a software problem. It's also hardware. You have all these considerations, especially you guys. I don't remember the exact number off the top of my head, but in terms of the customers that you have who are interacting, it's like hundreds of millions. We have about 5 million phone calls a month. So about 60 million a year. It's just pretty hard. Exactly. And for those, you can't have this weird latency between how voice is actually interacting with these consumers. And so how do you think about this problem? I'd love to get into your head, into your teams. It's like, how do you think about this problem at Elise AI? How do you break it down? How do you figure out what's the first thing you guys should work on to tackle it? Right. So, again, unfortunately, this is like, so there are two parameters here, right? There is one, you know, knowing exactly the current stack. Like, you know exactly what the current stack. I built it mostly largely myself, so I had a really unique vision. Another huge advantage that I have to say, we built it entirely on our own. So we have full understanding of every single part of it. So you have the full stack. You understand where the bottlenecks are today. What are the steps that takes the longest, right? And then you are the outward phasing kind of perspective where you have a lot of different providers, different changes, like fine-tuning open sources or newer speech-to-text, text-to-speech models. and you try to match them to with the current stock. And basically, the idea is, okay, if we obviously have something that is really bad on our current stock and there's new solutions that you can apply, then you go in full force. Otherwise, you're trying to constantly understanding and prioritizing, okay, given the solutions that the market has built, this piece of our stock can be improved. And I'm going to say that another thing that we did to speed up that process is that we built a way to experiment. So we have a very close loop where something happens, somebody on Slack messaged me, dude, like I saw this post on X. I don't really have to believe it. I don't even The thing we do is like, okay, let's set it up. Let's run an experiment internally. So we can actually see a performance of every model on our own internal benchmarks versus the other ones that you cannot trust. Wait, that's so cool. So it comes in on Slack and then automatically, is this all automated or do you come in? We do have some Ubuntu loop, but we're working on animating it actually fully. Wow, that's so cool. And so tell me a little bit about this research background that you have. You're, I mean, a professor at some of the top, in London, right? One of the top universities. what you took from that entire framework for how you do research, especially theoretical physics research. And then how does it actually apply, especially with your team? How do you actually use the same methods internally? Yeah, because we are able to... So, again, the two things is, one, obviously, there is a gain of having some sort of technical strength, mathematics, and just understanding math, and it helps. And programming, those things are actually useful. with a new idea. You know, like, theoretical physics are the cutting edge of kind of humanity in terms of, like, reasoning and thinking. So coming up with new ideas, it's not easy. Like, how do you, get an insight that really change things. Like if it was easy, everybody would have it. And so like there is a kind of a process. I think this is very useful to do it. And then we bring it back. Right. So for example, you know, we brought this to fine tune, right? There was like a big, one of the things that we invested on heavily in the last eight months or so is to try open source models. And so the thing was, okay, now we have the possibility that, you know, it? Where should we do it? What are the new things that we should really try? And because we have the background of being able to generate ideas independently, and you know that you've tackled one of the hardest questions that are out there, you're not scared of like tackling yourself. And so then we set a group of researchers and we started heavily tackling that. And we have been able to actually get incredible results with training our own models and delivering it to I'm curious as well, you posted on LinkedIn, this other post the other day, and it was talking about brilliant research backgrounds. [15:00] with minimal real-world impact. And this is also what we were talking about right before we went live. Exactly. And so I think what's really cool, especially as you mentioned, and I feel your energy right now, just getting so excited about applying all this scientific rigor, but to real-world and essential industries like housing and healthcare. And so what are some of the hardest problems you say your team at Elise AI right now, what are some of the hardest problems example, LLMs are moving fast, right? So now one of the very legitimate question is, six months from now, what problems will be solvable with LLMs that are not solvable now? And then push it to a year ago or a year from now and so forth. So like constantly prioritizing what are the things that we could solve with LLMs and we could automate. And again, we're not talking about math problem. We're talking about real world iterations. So like you have to be sure that, Here, there's something that is really hard that is a real huge friction for millions of apartments, and now we can actually even. So this is one of the things that we're constantly spending time on. And it's very, and depends, it's a high stake decisions because moving early and moving fast allows us to be in a much better position six months from now. And then the other thing that I think is extremely difficult is that, again, unless it's growing fast, but we're still 600, a few hundred people in the company. operate at a scale where we automate millions, one million phone cards a day, hundreds of thousands of emails and SMSs. So we really have developed... loops which are fully automated, completely automated. And now this is wonderful, it's great, it's glorious. But on the other hand, you need to really build some evaluations so that you know, like we have tons of conversations that go on without our inputs. And these impact, again, these are housing and healthcare. These are not like some random thing. These actually impact people's life. There's consequential changes that happen if we make a mistake. And so building 10 people, like they're able to like get an understanding every day how the conversations are going. That's a really, really hard task, which we have largely cracked, to be honest. Give me some specific examples there. So what's an example of something that you fully automated this entire loop from? Oh, basically, the entirety of voice calls is out of it. For example, the two main loops that we've automated, one is leasing people that are interested in apartments. They basically can do things. The entirety of starting from looking at a website and deciding that they are interested in apartments to moving in, it could be kind of fully, literally fully automated. For example, you can book a tour. Then the AI sends you the code. You can go to the house. Then you open the house with the code. You interact with the AI. The AI is explaining you the house. So you're doing the whole thing on your own, right? The other thing is resonant for resonant. calling and they have issues, there is still a gap that we've not been able to automate because we need robotics to fully automate that. Because some of the fixes require actual human labor. And so LLMs can't really do that. But we have automated a large chunk of that because you can now submit a work order to tell you, look, fix this thing. The AI is capable to understand whether or not it requires a fix immediately. A person And then the last piece Is a person Actual physical person Coming into your house Fixing it Maybe we'll get to like, automated the whole thing in a year too. Wow, that's pretty interesting. So I want to know a little bit about what's next for you guys. One of these big, this big conversation I was having with Mina is around this vertical versus horizontal scaling. I think what's unique about, Elise AI is vertically down the stack, especially when it came to housing, because that's where you guys started. You fully went through every single problem, except, I guess, the robotics physically going into your home. But you'll get there. And now you're applying it more horizontally. So now also going really deep down in health care. What's next? Are you thinking of another industry or going deeper on this vertical angle? Yeah, so we definitely... Right now, we have a handful. I mean, healthcare and housing is like 40% of the GDP in the U.S. So, you know, we have a route to grow a little bit. And especially, we do find that our best way to build AI that matters, right, is because, you know, one of the things that's unique about Elise is that we have these incredible things. And combined with, they've acquired, combined with like [20:00] deep domain understanding and business knowledge, right? So we really do have this unique mixture of skills. And so we want to continuously move that way. So we want to be driven by what is next big bet based on what our clients say. So one of the things, for example, that we are now facing very pretty heavily is to really cut down the cost of running affordable housing. This seems yet extremely impactful and relevant for especially the component of the population that has the least means, like the ones that is most vulnerable. And so we really want to be able to make them more affordable. It's extremely complicated, right? It's basically not automated at all. The amount of tech that is currently in the industry is nearly non-existent. And so we are really aggressively trying to, and it's a massive component of the entirety of real estate. So that would be massively impactful if we could bring down the cost of football housing significantly so that will impact consumers immediately and make their lives much better. I know, that's pretty interesting. Some of the hardest problems attract the top talent. One last thing I wanted to ask you before we go. I know your team is hiring, and I wanted to just better understand what you're looking for. You said earlier that you have a mixture of people with many different backgrounds, who have a mixture of skills. You come from the academia world. I just wanted to better understand, like, what are you looking for when it comes to how you grow your team? So the reasons why we are mostly hiring, especially on the technical side in San Francisco and New York, is because we want the best. I mean, we're pretty high standards. And the two main things we have, we want best engineers. There's a ton of engineering that needs to be done. And we are so short on that. And in particular, in San Francisco, our second hub is growing, you know, fantastically well. But we need more. We just can't grow hard enough. And then secondly, we love, in particular me, obviously, researchers. And the background there is people who... have a very strong research background, but they really have this fire in them to... wanting to use their ideas, their depth of thought in actually impact. So we want, these are kind of very difficult, kind of unicorns to find. People who have exceptional academic background, but yet they are really interested in driving that into real world. And so researchers with a clear focus on impact people's life and make them better. That's exciting. All right, well, thank you so much for joining us today, Mario. This is such a fascinating conversation. the least and getting to hear from your perspective also minutes perspective but just the different background and approach that you have coming from that theoretical physics background and then applying rich academia to some of the toughest problems we have now in AI. And I want to say to the people out there, I did not expect that. But even after coming from string theory, I would say that the complexity of the problem of working now is probably higher now than it was before. Even coming from string theory. It's pretty high bar. You hear it here first. Well, Mario, thanks for listening to this episode of the a16z podcast if you like this episode be sure to like comment subscribe leave us a rating or review and share it with your friends and family for more episodes go to youtube apple podcast and spotify follow us on x a16z and subscribe to our sub stack at a16z.substack.com thanks again for listening and i'll see you in the next episode this information is not a recommendation to buy, hold, or sell any investment or financial product. This podcast has been produced by a third party and may include paid promotional advertisements, other company references, and individuals unaffiliated with A16Z. Such advertisements, companies, and individuals are not endorsed by AH Capital Management LLC, A16Z, or any of its affiliates. Information is from sources deemed reliable on the date of publication, but A16Z does not guarantee its accuracy. Bye. you you you