Zocdoc's Rao Says AI Adaptive Systems Require Behavioral Controls

Adaptive Recommendations at Scale Using AI With Zocdoc (ex-Netflix, Walmart, Twitter) — Ai4

Ai4

Software engineering must transition from static code deployment to managing dynamically evolving system behavior through continuous feedback and multi-clock control infrastructure.

AI-assisted summary of Adaptive Recommendations at Scale Using AI With Zocdoc (ex-Netflix, Walmart, Twitter) — original source. Check the original for full context.

Source published: 2026-08-14

Detailed notes

## Background Published on YouTube by Ai4 on August 14, 2026, this talk features Mallika Rao, Senior Engineering Manager at Zocdoc (formerly of Netflix, Walmart, and Twitter), examining the evolution of recommendation systems from batch personalization into adaptive, real-time discovery engines. The presentation explores event-driven continuous intelligence, architecting for ambiguity, and the leadership practices needed to balance experimentation, model debt, and infrastructure stability in AI-native systems.

## TL;DR Mallika Rao argues that recommendation systems are the original learning software, offering a foundational blueprint for modern AI-native workloads and agentic systems. Because adaptive systems mutate post-deployment, dynamically evolving behavior—rather than static code—becomes the primary production artifact within a closed-loop control system that both learns and actively shapes user taste. Rao introduces 'The Three Clocks' framework (World, Learning, and Control) to maintain equilibrium between ingestion speed and safety, emphasizing that operational trust infrastructure must replace raw model accuracy as the core engineering bottleneck. She concludes that engineering has entered an era of Behavioral Software Engineering, which requires reimagining testing, rollbacks, and intent modeling as autonomous AI agents increasingly act as intermediary learners.

## Atmosphere & style The presentation carries a crisp, authoritative, and deeply reflective tone, seamlessly bridging high-level systems philosophy with battle-tested engineering realities from Netflix, Twitter, Walmart, and Zocdoc. Rao speaks with calm precision and structured clarity, using thought experiments and first-principles diagrams to deconstruct complex control-loop dynamics. It feels like an advanced masterclass on production AI maturity, replacing industry hype with pragmatic architectural rigor and insightful inquiry into the future of human-agent interaction.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=4s) Recommendation systems serve as the foundational blueprint for modern AI-native workloads, agents, and co-pilots because they were the first software systems forced to learn and adapt at hyper-scale. - [01:16](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=76s) In adaptive systems, dynamically evolving behavior—rather than static code—is the primary production artifact that engineers must version, test, and maintain. - [04:18](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=258s) Adaptive platforms operate as closed-loop control systems rather than linear pipelines, because the recommendations they serve directly shape user behavior and subsequent training data. - [04:18](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=258s) Every adaptive architecture must fulfill six technology-agnostic responsibilities: Sense, Represent, Adapt, Decide, Act, and Learn & Control. - [06:07](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=367s) Real-time inference capability is distinct from real-time learning; the true benchmark of an adaptive system is how quickly it can sense, understand, and adapt to environmental changes. - [06:07](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=367s) Accelerating learning speed introduces the danger of rapidly reinforcing incorrect, skewed, or toxic patterns that corrupt downstream data distributions. - [06:07](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=367s) Human domain expertise and operational knowledge should be integrated as first-class, high-signal data streams rather than treated as exception overrides. - [10:03](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=603s) The Three Clocks framework requires maintaining equilibrium between the World Clock (reality), the Learning Clock (model adaptation), and the Control Clock (safety intervention and rollbacks). - [12:45](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=765s) Production bottlenecks have shifted from model accuracy to operational trust, demanding specialized infrastructure like shadow canaries, preview environments, and instant behavioral rollbacks. - [14:02](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=842s) Recommenders act as control systems that can create self-fulfilling feedback loops, actively shaping user preferences rather than merely discovering pre-existing taste. - [16:33](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=993s) The discipline of software engineering is entering the Behavioral Software Engineering era, redefining traditional concepts of testing, versioning, deployment, and correctness for non-deterministic systems. - [19:42](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=1182s) The emergence of autonomous AI agents introduces an intermediary learner between users and platforms, complicating whether systems are optimizing for human taste or agent taste.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=4s) When Software Starts Learning: Recommenders as the Lens for AI Systems

Mallika Rao opens her talk at AI4 by framing a foundational question that has guided her career building search, personalization, recommendation, and AI platforms across Netflix, Twitter, Walmart, and Zocdoc: "What happens to software engineering when the software itself starts learning?" Recommendation engines were the very first software systems forced to confront this paradigm shift at hyper-scale. Rather than being a legacy niche, the architectural hard truths learned from building recommenders provide the direct blueprint for modern AI-native workloads, autonomous agents, co-pilots, and next-generation search engines.

```mermaid flowchart LR accTitle: Closed Loop Adaptive System subgraph Sensing["1. Sensing & Ingestion"] A["User Actions & Context"] --> B["Streaming Signals"] end subgraph Adaptation["2. Learning & Adaptation"] B --> C["Model Retraining & Embeddings"] C --> D["Behavior & Decision Engine"] end subgraph Actuation["3. Actuation & Impact"] D --> E["Recommendations / Actions"] E --> A end ```

[01:16](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=76s) The Thought Experiment: Behavior as the True Production Artifact

To illustrate why learning software breaks classical paradigms, Rao presents a thought experiment: imagine an engineering team designs the "perfect" recommendation system—the machine learning models are flawless, ranking algorithms are tuned, data pipelines are pristine, and personalization is exact. The fatal flaw is that the system was only perfect *yesterday*. Overnight, external reality shifted: new content was published, breaking trends surfaced, user preferences shifted, catalog inventories rotated, or healthcare providers opened new appointment slots on Zocdoc. The recommendations produced today are not mathematically incorrect; they are simply stale.

This realization fundamentally changes the definition of software engineering. Traditional software operates on a rigid contract: engineers write code, test it, deploy it, and expect its execution behavior to remain deterministic and static until the next deployment. Adaptive systems shatter this premise—their production behavior continuously mutates even when not a single line of deployed code has changed. In traditional engineering, code is the versioned, tested, and deployed artifact; in adaptive systems, code merely describes the underlying mechanism, while the dynamically evolving *behavior* is the actual production artifact that must be engineered with equal rigor.

[04:18](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=258s) From Static Pipelines to Closed Loops: The Six Core Responsibilities

The defining distinction between traditional software and adaptive systems lies in how they interact with the world. Traditional software observes input and responds. Adaptive software, by contrast, actively participates in shaping its environment. When a platform serves a recommendation, it directly dictates what a user consumes, when they consume it, and how long they engage. That resulting consumption pattern immediately becomes the new training data fed back into the system, influencing what will be served next. The architecture is no longer a linear, unidirectional data pipeline—it is an active closed-loop control system.

When architecting these closed-loop ecosystems, Rao highlights six core, technology-agnostic engineering responsibilities that every adaptive system must fulfill: - **Sense**: Capturing and ingesting fresh, real-time contextual signals from the environment. - **Represent**: Mapping raw signals, item spaces, and user states into meaningful latent representations and embeddings. - **Adapt**: Updating model parameters, index states, or ranking heuristics based on incoming data. - **Decide**: Evaluating trade-offs across candidates to select optimal outputs. - **Act**: Serving the selected decision to the user or downstream consumers. - **Learn & Control**: Monitoring outcomes, maintaining guardrails, and safely feeding new signals into subsequent learning cycles.

[06:07](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=367s) The Core Tension: Real-Time Learning vs. Learning the Wrong Things

Whenever an abrupt shift occurs in the world—such as a trending song on social media, a breaking news cycle, or a high-demand doctor listing open availability—the critical engineering metric is not how fast the system performs inference. A real-time inference system capable of generating predictions in milliseconds is fundamentally different from a real-time learning system. The vital question is: "How long until the system knows?" How quickly can the platform sense, adapt, understand, and decide upon an environmental change?

This introduces the central tension of adaptive systems engineering: platforms want to learn as rapidly as possible, but accelerating the rate of learning dramatically increases the risk of the model learning incorrect, skewed, or toxic patterns. Because every output shapes future training inputs, bad learning quickly corrupts downstream data distributions. Furthermore, Rao challenges the common assumption that adaptive systems exist to eliminate human judgment. Operations teams, merchandisers, and domain experts frequently recognize localized disruptions, seasonal shifts, or market-specific events long before statistical algorithms detect them. Rather than treating humans as messy exception overrides, high-performing architectures treat human input as a first-class, high-signal data stream.

[10:03](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=603s) The Three Clocks Framework: Balancing World, Learning, and Control

To evaluate and architect adaptive systems, Rao relies on a mental model she terms "The Three Clocks." Every autonomous and learning platform operates on three distinct, competing temporal frequencies:

Engineering excellence is not about blindly maximizing the speed of every clock; it is about maintaining continuous equilibrium between them. Architectural decisions ultimately boil down to a single question: *"Which clock are we optimizing at this point of time? And which are we ignoring?"*

[12:45](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=765s) The Infrastructure of Trust: Moving Past Model Accuracy

For decades, machine learning practitioners operated under the belief that model accuracy was the primary performance bottleneck. In production, however, the real bottleneck is operational trust. Trust cannot remain an abstract feeling or a product aspiration; it must be constructed directly into the infrastructure via specialized tooling and verification primitives: - **Preview & Dry-Run Environments**: Validating model updates against offline scenarios before exposing them to users. - **Shadow Traffic & Shadow Canaries**: Running candidate models parallel to production traffic to evaluate behavioral shifts under live distribution without user impact. - **Continuous Evaluation & Integration Harnesses**: Dynamically scoring safety, bias, and business guardrails in real time. - **Instant Behavioral Rollbacks**: Mechanisms capable of resetting model states or falling back to safe deterministic baselines the moment anomalous behavior is detected.

[14:02](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=842s) Recommenders as Control Systems: Shaping Taste and Self-Fulfilling Loops

Feedback loops are commonly treated as purely beneficial mechanisms in machine learning, but closed loops introduce profound feedback bias. If a system promotes an item simply because it is trending, more users see it, click it, and engage with it. The algorithm observes these clicks as positive reinforcement and amplifies the item even further. This raises a fundamental dilemma: "Did we actually discover a preference or did we actually create one?"

At this stage, recommender systems behave less like passive ranking algorithms and more like active control systems that reshape future data distributions. The relationship between users and learning algorithms is mutual rather than unidirectional: while the system learns human taste from past behavior, it simultaneously shapes human taste by governing what users discover, what they ignore, and what they seek next.

```mermaid flowchart TD accTitle: Taste Shaping Loop subgraph FeedbackBias["Self-Fulfilling Feedback Loop"] A["Item Recommended Due to Popularity"] --> B["Increased Exposure & Engagement"] B --> C["System Learns Strong Positive Signal"] C --> A end subgraph DistributionShift["Data Distribution Impact"] C --> D["Mutates Future Training Distribution"] D --> E["Shapes User Discovery & Taste Over Time"] end ```

[16:33](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=993s) Crossing the Boundary: From ML and Distributed Systems to Behavioral Software Engineering

Reflecting on her career path across Twitter, Walmart, and Zocdoc, Rao outlines three evolutionary eras of recommendation engineering: 1. **The Machine Learning Era**: Focused strictly on statistical modeling—building better ranking algorithms, relevance functions, vector embeddings, and feature stores. 2. **The Distributed Systems Era**: Focused on scaling data pipelines—managing state, minimizing serving latency, and maintaining freshness across distributed backends and mobile clients. 3. **The Behavioral Software Engineering Era**: Focused on safely deploying and managing systems whose behavior continuously evolves post-deployment.

This third era forces engineering teams to redefine fundamental software concepts: - **Testing**: How to unit-test and integration-test a system whose output is non-deterministic and designed to change over time. - **Deployments & Rollbacks**: What it means to deploy or roll back an evolving policy or learned behavioral artifact rather than a binary code release. - **Versioning**: How to version behavioral checkpoints alongside data schemas and code commits. - **System Correctness**: Defining what "correctness" means when a system is explicitly expected to mutate its logic based on environmental feedback.

[19:42](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=1182s) The Agentic Frontier: Explicit Intent, Implicit Intent, and Intermediary Learners

Rao traces the progression of how systems interpret human intent. Search queries provided an explicit window into curiosity, confusion, and direct user need. Recommendation systems introduced an implicit window, inferring underlying taste through indirect actions like clicks, skips, dwell times, and watch durations. Together, explicit and implicit signals provided a comprehensive view of human judgment.

The emergence of AI agents, however, adds a disruptive new layer to this architecture: an intermediate, autonomous learner acting on behalf of the user. When an agent executes searches, evaluates alternatives, and ingests dozens of papers or provider schedules without the user directly clicking or browsing, it creates an architectural puzzle: *"Whose taste are we learning? Is it learning my taste? Is it learning my agent's taste?"*

Rao concludes by emphasizing that the defining software engineering challenge of the next decade will be architecting scalable infrastructure that enables continuous learning across multiple intelligent actors while preserving human judgment, explainability, and safety when the behavior of software never stops changing.

## Notable quotes > [00:04](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=4s) What happens to software engineering when the software itself starts learning? — Mallika Rao

> [06:07](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=367s) How long until the system knows? — Mallika Rao

> [10:03](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=603s) Which clock are we optimizing at this point of time? And which are we ignoring? — Mallika Rao

> [14:02](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=842s) Did we actually discover a preference or did we actually create one? — Mallika Rao

> [19:42](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=1182s) Whose taste are we learning? Is it learning my taste? Is it learning my agent's taste? — Mallika Rao

## Mentioned Mallika Rao, Zocdoc, Netflix, Walmart, Twitter, AI4

## Video description Mallika Rao - Senior Engineering Manager, Zocdoc (ex-Netflix, Walmart, Twitter)

As recommendation systems evolve from batch-based personalization to adaptive, real-time discovery engines, infrastructure must transform too. This talk draws from large-scale experiences powering billions of recommendations and searches per day—spanning event-driven architectures, model-driven indexing, and feedback-loops built for continuous learning. We’ll explore how to evolve traditional recommender pipelines to support AI-native workloads—where training, inference, and evaluation all coexist in production. You’ll learn how leaders can architect not just systems, but trust and ambiguity—balancing experimentation and reliability while enabling teams to thrive amid shifting signals.

This is a story about how distributed systems meet adaptive intelligence—and what it takes to make recommendations feel instant, relevant, and alive.

In the era of AI-driven personalization, recommendation systems are no longer static ranking engines—they are adaptive ecosystems that continuously evolve through user feedback, contextual cues, and model retraining.

This talk unpacks how large-scale distributed systems can transform to support this new paradigm of real-time discovery. Drawing from experience designing large recommendation and catalog-slicing infrastructures across consumer platforms, we’ll explore:

Event-Driven Adaptation — How streaming signals and micro-batches bridge online and offline learning, turning user interactions into live feedback loops.

Infra for Continuous Intelligence — Designing infra that allows training, inference, and evaluation to coexist seamlessly, without fragmenting data pipelines or reliability.

Architecting for Ambiguity — Techniques for scaling before the problem is fully defined—building systems that embrace evolving data distributions and model behaviors.

Trust & Leadership in the Loop — Why leaders must treat “infra debt” and “model debt” with equal rigor, architecting psychological and technical safety so teams can innovate responsibly.

Dual-Path Execution — How maintaining separate product and infra strategies for AI adoption enables organizations to ship fast while investing deep.

By the end, attendees will walk away with a playbook for evolving legacy recommender systems into adaptive, AI-native discovery platforms—and insights on how to lead teams confidently through that transition.

Key Takeaways How to evolve recommendation infrastructure for real-time, adaptive intelligence.

How to architect for ambiguity—designing systems and teams that thrive amid uncertainty.

Why evaluation and feedback loops are as critical as training.

How to balance product innovation with infra stability through dual execution paths.

Leadership lessons for navigating the AI-driven evolution of large distributed systems.

[Source](https://www.youtube.com/watch?v=KMHuxIZGTlg)

中文简报

Zocdoc的Rao表示AI自适应系统需要行为控制软件工程必须通过持续反馈与多时钟控制基础设施，从静态代码部署转向管理动态演进的系统行为。 Mallika Rao指出，推荐系统是最早的学习型软件，为现代AI原生工作负载和智能体系统提供了基础蓝图。由于自适应系统在部署后会发生变化，在一个既进行学习又主动塑造用户偏好的闭环控制系统中，动态演进的行为（而非静态代码）成为了主要生产制品。Rao提出了“三个时钟”框架（世界、学习与控制），以在摄取速度与安全性之间保持平衡，并强调运营信任基础设施必须取代原始模型精度，成为核心的工程瓶颈。她总结认为，随着自主AI智能体日益充当中间学习者的角色，软件工程已进入“行为软件工程”时代，这需要对测试、回滚和意图建模进行重新构想。推荐系统是现代AI原生工作负载、智能体和Copilot的基础蓝图，因为它们是首批被迫在超大规模下进行学习和适应的软件系统。 (0:04) 在自适应系统中，动态演进的行为（而非静态代码）才是工程师必须进行版本控制、测试和维护的核心生产制品。 (1:16) 自适应平台是以闭环控制系统而非线性流水线的方式运行的，因为它们所提供的推荐内容会直接塑造用户行为及后续的训练数据。 (4:18) 任何自适应架构都必须履行六项与技术无关的职责：感知、表征、适应、决策、行动以及学习与控制。 (4:18) 实时推理能力与实时学习有所不同；自适应系统的真正衡量基准在于其感知、理解并适应环境变化的速度。 (6:07)

Reading summary

Mallika Rao argues that recommendation engines provide the foundational blueprint for modern AI-native systems, where dynamically evolving runtime behavior—rather than static code—becomes the primary production artifact that engineers must govern. Redefining Software Engineering Around Mutating Behavior (1:16) Traditional software engineering operates on a deterministic contract: engineers write, test, and deploy code, expecting its execution logic to remain static until the next deployment. Rao observes that adaptive systems fundamentally disrupt this model because external environments continuously shift. Even if an engineering team deploys a mathematically tuned recommendation model, fresh content, shifting consumer preferences, and newly opened appointment slots on platforms like Zocdoc render yesterday's system stale overnight without a single line of code changing. In learning systems, deployed code merely defines the underlying mechanism, while the mutating behavior itself is the production artifact requiring rigorous engineering. To manage this closed-loop interaction, Rao outlines six technology-agnostic engineering responsibilities: sensing real-time environmental signals, representing them into latent embeddings, adapting model parameters or heuristics, deciding candidate trade-offs, acting by serving decisions, and maintaining continuous control through monitoring and guardrails. Real-Time Learning Versus Amplified Feedback Bias (6:07) Adaptive platforms operate as closed-loop control systems rather than linear data pipelines. When an algorithm serves a recommendation, it directly dictates what a user consumes; that resulting engagement is immediately fed back as training data for subsequent model cycles. This architecture generates a fundamental engineering tension between learning speed and safety. While organizations want systems to adapt rapidly to real-world shifts, accelerating the learning rate dramatically heightens the risk of internalizing and propagating skewed, incorrect, or toxic patterns. Furthermore, closed loops induce significant feedback bias, raising the dilemma of whether a system is genuinely discovering user taste or actively manufacturing it through repeated exposure. To counter systemic blind spots, Rao argues against viewing automation as a tool to eliminate human judgment. High-signal domain experts, merchandisers, and operations teams often recognize market disruptions long before statistical algorithms detect them, making human input an essential first-class data stream in robust architectures. Balancing Three Clocks Through Trust Infrastructure (10:03) To evaluate system health, Rao introduces "The Three Clocks" framework, which models the balance across three distinct operational speeds: the World Clock (the rate of external change), the Learning Clock (the rate at which models ingest data and update behavior), and the Control Clock (the speed at which the platform detects errors and safely intervenes). If the Learning Clock falls behind the World Clock, recommendations become stale; if it outpaces the Control Clock, the system becomes erratic, biased, and potentially unsafe. Engineering success depends on maintaining equilibrium among all three rather than maximizing speed alone. Because operational trust—rather than raw model accuracy—is the primary bottleneck in production, platforms require concrete infrastructural verification tooling. Rao highlights several essential primitives: offline preview and dry-run environments, shadow traffic canaries to evaluate behavioral variance without impacting users, dynamic continuous integration harnesses for bias and business safety, and instant rollback capabilities that can revert model states to deterministic baselines when anomalies arise. Behavioral Software Engineering and the Agentic Frontier (16:33) Rao traces the discipline's evolution through three successive phases: an initial machine learning era focused on statistical ranking and feature stores; a distributed systems era centered on pipeline scalability, data freshness, and low-latency serving; and an emerging behavioral software engineering era. This third era forces teams to redefine classical software concepts, establishing new standards for non-deterministic testing, behavioral versioning, policy deployments, and system correctness. This paradigm is further complicated by autonomous AI agents acting as intermediaries between platforms and people. Whereas search engines capture explicit user queries and recommenders infer implicit taste from signals like clicks and dwell time, AI agents actively browse, filter, and schedule on behalf of users. Rao notes that this intermediary layer introduces a critical architectural puzzle: systems must now distinguish whether they are modeling the underlying preferences of the human user or the synthetic operational patterns of the agent. Rao concludes that the central challenge of the next decade will be architecting scalable infrastructure that supports continuous learning across multiple autonomous actors while preserving human judgment, explainability, and operational safety.

精炼阅读摘要

Mallika Rao 提出，随着软件具备持续自主学习的能力，推荐系统率先打破了传统工程范式：在线运行的代码仅是承载机制，其动态演进的“系统行为”才是核心生产制品。她认为，AI 时代工程设计的核心挑战在于超越纯粹的模型精度，通过闭环架构、“三时钟”平衡和信任基础设施来安全管控持续自适应系统的行为变异。行为作为生产制品与闭环架构 (1:16) Rao 指出，传统软件工程建立在确定性契约之上——代码经过测试和部署后，其执行行为在下一次发布前保持静态。然而自适应系统颠覆了该模式，即便线上代码未做任何改动，其生产行为也会随着外部世界的内容变动、趋势爆发或用户偏好迁移而持续演化。昨天调优至完美的模型，在今天可能因现实环境的改变而陷入陈旧。因此，工程团队交付的核心制品不再仅是静态代码，而是随时间动态演化的系统行为。在这种背景下，推荐架构从单向线性数据管道转变为双向控制闭环。当系统向用户展示推荐时，它直接干预了用户的注意力与消费选择；这些消费行为产生的数据又立刻作为新的训练信号反哺系统，直接决定下一轮输出。Rao 将自适应系统的技术职责提炼为六个核心环节：感知（Sense）、表征（Represent）、自适应调整（Adapt）、决策（Decide）、执行（Act）以及学习与控制（Learn & Control）。这意味着软件不再只是外部环境的被动观察者，而是重塑自身输入分布的主动参与者。三时钟框架与实时学习的核心张力 (6:07) Rao 区分了毫秒级推理与真正的实时学习能力：面对外部突发事件或需求变动，关键指标并非系统生成预测的计算速度，而是平台感知、理解并适应环境变化所需的延迟（即“系统需要多久才能知晓”）。然而，加速学习频率伴随着系统汲取偏斜、毒性或错误模式的高风险。一旦错误决策被作为正向信号反馈，将迅速污染下游数据分布。为此，Rao 反对将自适应系统简单视作“消除人类干预”的工具，主张将运营人员和业务专家的先验判断视为高权重的一级数据流，而非异常覆写。为了在动态响应与安全可控之间维持平衡，Rao 提出了“三时钟”工程心智模型。自适应系统同时运行在三个相互竞争的时间频率上：“世界时钟”代表外部现实演进的速度；“学习时钟”代表模型摄取数据与更新行为的速度；“控制时钟”则代表系统识别错误学习、进行安全干预与回滚的速度。若学习时钟落后于世界时钟，系统会陈旧失效；若学习时钟超过控制时钟，系统则极易陷入不可预测的失控状态。架构决策的核心在于动态权衡当前优先优化哪一时钟，而非盲目拉高单一速度。信任基础设施与塑造品味的反馈偏置 (12:45) 长期以来，机器学习界习惯将模型精度视为主要性能瓶颈。但 Rao 强调，在规模化生产环境中，真正的瓶颈在于操作层面的系统“信任”。信任无法仅凭抽象理念建立，必须落实在工程基础设施层面，包括离线校验的预览演练环境、利用线上流量评估行为变化的影子金丝雀测试、动态评估安全与业务护栏的持续集成评估套件，以及在异常发生时能瞬间退回确定性基线的行为回滚机制。基础设施之所以至关重要，是因为闭环系统具有深刻的“自证预言”反馈偏置。当系统因某项内容具备热度而予以推荐，曝光的增加会带来更多点击与互动，算法误将这些指标视作强烈的正向偏好并进一步放大曝光。Rao 指出，此时系统已不仅是发现用户品味的排序工具，而是转变成重塑未来数据分布与人类品味的主动控制系统。算法一方面从过往行为中学习偏好，另一方面也在通过决定曝光边界来重塑用户的探索与审美。行为软件工程与智能体代际挑战 (16:33) 回顾在 Twitter、Walmart 及 Zocdoc 的实践，Rao 将推荐工程划分为三个发展阶段：第一阶段是聚焦统计建模与向量表征的“机器学习时代”；第二阶段是聚焦低延迟、高并发与数据新鲜度的“分布式系统时代”；当前则进入了“行为软件工程时代”。该阶段迫使工程学重构基础定义：测试不再局限于确定性断言，而是评估非确定性演化；发布与回滚的对象从二进制代码变为学习策略与行为检查点；“系统正确性”也必须在逻辑允许自我突变的语境下重新确立。 AI 智能体（Agents）的兴起进一步加剧了系统架构的复杂度。过去，搜索查询代表了用户的显式意图，推荐行为捕捉了点击、停留等隐式意图，两者结合构成了完整的人类意图图谱。但当自主智能体作为中间代理代为执行搜索、比对并筛选方案时，系统面临着“究竟是在学习用户偏好还是在学习智能体行为”的根本挑战。在多智能体交互演化的未来，如何构建能持续学习同时保持可解释性与安全边界的底层架构，将是软件工程的核心议题。 Rao 总结认为，未来十年的软件工程核心命题，是在软件行为永不停止变化的现实中，构建出兼顾多智能体持续学习与人类安全可控的底层系统。然而，该演讲主要聚焦于高层架构原则与工程心智模型，对于如何量化区分“真实用户偏好”与“自证偏置”、以及当用户意图与智能体决策逻辑冲突时如何实施确定性判定，仍留有待业界深入探索的开放性挑战。

Detailed notes — zh

## Background 本内容由频道 Ai4 于 2026 年 8 月 14 日发布在 YouTube 平台上，演讲者为 Zocdoc 高级工程经理 Mallika Rao（曾任职于 Netflix、Walmart 及 Twitter）。页面简介指出，本次演讲基于百亿级推荐与搜索架构的实战经验，探讨推荐系统如何从批处理个性化转变为具备实时自适应能力的 AI 原生发现引擎，并分享应对系统与模型债务、构建架构信任及双轨执行的领导力实战指南。 ## TL;DR 演讲深入剖析了当软件具备自主学习能力时对传统软件工程范式带来的颠覆，指出自适应系统的核心生产制品已从静态代码演进为动态演化的“行为”。面对推荐系统闭环学习中“发现偏好还是制造偏好”的控制论张力，Rao 提出了“世界时钟、学习时钟与管控时钟”协同平衡的心智模型，强调盲目追求学习速度的风险。随着自主 Agent 逐步介入人机交互，现代分布式架构必须建立起包含影子流量验证、行为版本化与一键安全回滚在内的完备信任工程体系。 ## Atmosphere & style 整场演讲呈现出资深系统架构师特有的深邃、沉稳与哲思气质，叙事节奏从工程实践中的具体痛点自然升华至控制理论与认知科学的维度。演讲者以极具穿透力的思想实验开局，语言冷静克制却充满洞察力，既没有空泛的 AI 炒作，也没有堆砌晦涩代码，而是抽丝剥茧般将高维工程张力娓娓道来。观众在聆听过程中能强烈感受到在大规模工业界实战中淬炼出的技术严谨感，以及对未来人机交互范式演进的敏锐前瞻。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=4s) 当软件本身具备自主学习能力时，软件工程的生产制品从静态的“代码”转变为持续演进的“系统行为”，必须以工程化代码同等的严谨度来工程化系统的行为。 - [00:04](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=4s) 传统推荐系统面临“昨天是完美的”陈旧性挑战，现实世界环境、用户兴趣与外部信号的快速更迭，要求系统架构从静态排序管道演进为自适应学习系统。 - [04:18](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=258s) 推荐系统是一个参与并重塑世界的自反馈“闭环系统”，用户的交互行为会反哺下一轮推荐，架构设计必须围绕感知、表征、自适应、决策、执行、学习与管控等核心职责展开。 - [04:18](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=258s) 实时推理并不等同于实时学习，搜索、点击、停留时长等丰富信号构成了对复杂品味的高维表征，工程的核心难点在于如何在安全可控边界内负责任地快速学习。 - [04:18](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=258s) 人工干预（Human-in-the-loop）不再只是系统的兜底机制，业务与专业选品人员能更敏锐捕捉突发变化，应将其正式纳入系统的高价值输入信号流中。 - [10:03](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=603s) 自适应系统依赖“三个时钟”的协同平衡：世界时钟（现实变化速率）、学习时钟（系统适应速率）与管控时钟（异常检测与干预速率）。 - [10:03](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=603s) 架构优化的目标不是盲目加速单一时钟，若学习时钟大幅超越管控时钟会导致系统失控；建立信任需要影子流量、金丝雀演练与行为级一键回滚等工程基础设施。 - [14:02](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=842s) 推荐系统的曝光决策会主动篡改未来的数据分布，面临“究竟是发现了偏好还是制造了偏好”的控制论挑战，系统在学习品味的同时也在深层塑造品味。 - [14:02](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=842s) 推荐系统经历了从“算法排序模型”到“分布式低延迟系统”、再到“自适应软件工程”的演进，当前核心挑战是如何对持续漂移的系统行为进行版本化与正确性验证。 - [19:42](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=1182s) 在未来人类、推荐系统与自主 Agent 共同交互的生态中，如何为行为永不停歇演化的多 Agent 系统构建具备确定性安全边界的基础设施，成为下一代自适应工程的关键课题。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=4s) 软件自主学习对软件工程的颠覆 Mallika Rao 开场回顾了自己在搜索、个性化推荐以及 AI 系统领域的长期从业经历（涵盖 Twitter、Walmart 以及 Zocdoc 等平台）。她指出，贯穿其职业生涯的一个核心命题是：“当软件本身开始自主学习时，软件工程会发生什么改变？”（"what happens to software engineering when the software itself starts learning?"）推荐系统是最早在大规模生产环境中直面该问题的领域之一，而推荐系统发展过程中沉淀的工程经验，正在成为构建未来搜索、Copilot、Agent 以及各类自适应系统（Adaptive Systems）的关键参考。 Rao 提出了一个思想实验：假设团队构建了一个完美的推荐系统——模型精湛、排序精准、数据齐备、个性化体验毫无瑕疵。但它存在一个致命问题：“它只是在昨天是完美的。”真实世界瞬息万变：新内容不断涌现、用户兴趣随时转移、上下文环境更迭、新的爆款或社交热点产生，或是在医疗场景中，平台接入了新的医生排班窗口。此时推荐结果并没有逻辑错误，它只是“陈旧”（stale）了。这促使工程师转变心智模型：推荐系统不再只是静态的排序管道，而是具备持续学习能力的软件体系。 ```mermaid flowchart LR accTitle: 传统软件与自适应系统的生产交付范式对比 subgraph Traditional["传统软件系统"] T1["编写代码"] --> T2["测试与版本管理"] T2 --> T3["部署上线"] T3 --> T4["确定性行为 (静态制品)"] end subgraph Adaptive["自适应系统"] A1["代码 (实现执行机制)"] --> A2["部署与运行"] A2 --> A3["吸收外部动态信号"] A3 --> A4["持续学习与演进"] A4 --> A5["行为成为生产制品 (动态演进)"] end ``` - 传统软件契约：编写代码并部署上线，在下一次代码部署前，其运行行为被假定保持恒定。 - 自适应系统特性：即便没有工程师修改代码，系统行为也会随着学习新数据而自主演进。 - 生产制品演变：传统软件的核心生产交付物是代码（版本控制、测试、回滚、灰度）；自适应系统的代码仅解释运行机制，真正的生产交付物是“行为”（Behavior），必须以对待代码同等的严谨度来工程化“行为”。 --- ### [04:18](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=258s) 推荐系统的闭环本质与六大核心职责传统软件主要扮演“观察世界并作出响应”的单向角色，而自适应系统则深度“参与并重塑世界”。例如，推荐系统决定了用户观看什么视频、何时观看以及观看时长；而用户的观看行为又转化为新数据，反哺并塑造下一轮的推荐策略。这不再是线性的数据流处理管道（Pipeline），而是一个动态自反馈的“闭环系统”（Closed Loop）。 ```mermaid flowchart TD accTitle: 推荐系统闭环学习生命周期 subgraph Loop["闭环自反馈生命周期"] Sense["感知 (Sense) & 表征 (Represent)"] --> Adapt["自适应 (Adapt) & 决策 (Decide)"] Adapt --> Act["执行动作 (Act)"] Act --> World["影响外部环境 / 用户交互"] World --> Learn["学习反哺 (Learn)"] Learn --> Control["安全管控 (Control)"] Control --> Sense end ``` 面对闭环系统，工程架构不应仅从技术选型切入，而必须围绕六至七项核心系统职责展开设计： 1. **感知（Sense）**：如何捕获外部世界与用户行为的突发变化。 2. **表征（Represent）**：如何建模用户偏好、内容特征与复杂的上下文。 3. **自适应（Adapt）**：模型与特征如何根据新信号动态调整。 4. **决策（Decide）**：在候选集中评估最优收益与策略。 5. **执行（Act）**：将推荐结果实时下发并触达用户。 6. **学习（Learn）**：将交互转化为有效训练与反馈样本。 7. **管控（Control）**：对异常漂移与风险行为进行实时干预。 - 实时推理并不等同于实时学习：“一个实时推理系统并不必然是一个实时学习系统”（"A real-time inference system is not necessarily a real-time learning system"）。 - 人类信号的品味表征：搜索、点击、播放、跳过、停留时长（dwell times）等人类交互信号共同构成了对复杂“品味”（Taste）的高维表征。 - 核心工程张力：系统学习速度越快，学到错误信号或虚假相关性的风险就越高。工程挑战不在于盲目追求“学得更快”，而在于如何在安全可控的边界内实现“负责任地快速学习”。 - 人类在环（Human-in-the-loop）的新定位：业务运营与专业选品人员往往比算法更快察觉局部地域突发事件或细分人群异动。人工干预不再被视为系统的异常或后备兜底（Override），而被正式纳入高价值输入信号体系。 --- ### [10:03](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=603s) 三个时钟理论与架构信任体系为了在工程上度量并解耦自适应系统的速度与安全，Rao 提出了“三个时钟”（The Three Clocks）心智模型： ```mermaid flowchart LR accTitle: 自适应系统的三个时钟平衡模型 subgraph Clocks["三个时钟协同平衡"] WC["1. 世界时钟 (World Clock)<br/>现实事件与环境变化速率"] LC["2. 学习时钟 (Learning Clock)<br/>系统感知、理解并适应的速率"] CC["3. 管控时钟 (Control Clock)<br/>检测错误并安全干预的速率"] end WC <--> LC LC <--> CC ``` - **世界时钟（World Clock）**：外部客观世界、突发事件与用户真实意图的变动频率。 - **学习时钟（Learning Clock）**：系统感知环境变化、更新模型并将新特征注入推荐链路的速率。若学习时钟落后于世界时钟，系统推荐将陷入陈旧。 - **管控时钟（Control Clock）**：系统检测到模型学习偏差、异常数据或错误决策，并实施安全干预、解释与回滚的速率。架构优化的核心目标不是让每个时钟盲目加速，而是维持三者的动态对齐。若学习时钟大幅超越管控时钟，系统将变得不可预测甚至极其危险。架构决策的关键考量在于：“在当前阶段，我们正在优化哪一个时钟，又在权衡妥协哪一个时钟？” 在生产落地维度，算法模型的效果提升已不再是唯一瓶颈，核心瓶颈已转向“系统信任”（Trust）。建立信任需要完备的工程基础设施： - 影子流量（Shadow Traffic）与影子金丝雀（Shadow Canaries）验证机制。 - 离线/近线 Dry-run 演练与集成测试框架。 - 动态评估监控与行为级一键安全回滚能力。 --- ### [14:02](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=842s) 反馈循环的控制动力学与品味塑造推荐系统面临着闭环控制理论层面的固有挑战。当系统因某项内容具备流行度而进行推荐时，曝光增加必然带来更多点击与互动，系统继而认定该趋势并进一步加大曝光。这就引发了核心哲学与工程审视：“系统究竟是发现了用户的真实偏好，还是亲手制造了该偏好？”（"did we actually discover a preference or did we actually create one?"）因此，推荐系统不仅在被动响应现实世界，其输出结果正在主动篡改未来的数据分布（Future Data Distribution）。系统不仅在学习用户的“品味”，同时也在深层次塑造用户的“品味”。更复杂的是，随着自主 Agent 介入，Agent 开始代表人类执行检索与长文阅读，推荐系统所学习的画像究竟代表人类真实意图，还是代理 Agent 的决策偏差，成为下一代自适应系统必须直面的新变量。 | 发展阶段 | 核心认知定位 | 关键技术侧重点 | 面临的核心挑战 | |:--- |:--- |:--- |:--- | | **早期阶段** | 机器学习/算法问题 | 排序算法、相关性、向量表征（Embeddings）、数据存储 | 模型预估精度与离线指标提升 | | **成熟阶段** | 分布式系统问题 | 数据新鲜度（Freshness）、端到端低延迟、移动端状态管理、高可用 | 大规模分布式环境下实时特征与状态的一致性 | | **当前及未来** | 软件工程/自适应系统问题 | 行为版本化（Version Behavior）、行为可测性、演进系统的正确性定义 | 行为持续漂移条件下的安全管控、部署与回滚机制 | --- ### [19:42](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=1182s) 意图窗口演进与 Agent 时代的工程展望搜索查询通过显式指令（Explicit Window）直接展现了人类的困惑、好奇与明确需求；传统推荐系统则通过交互行为构建了理解人类潜意识意图的隐式窗口（Implicit Window）。当两相结合时，构成了洞察人类决策与品味的核心链路。随着具备自主能力的智能体（Agents）作为第三方决策者融入交互，人类不再亲自执行每一次搜索或浏览数十篇文献，系统的交互格局被彻底重构。未来的核心工程挑战在于：在包含人类、推荐系统与多个自主 Agent 协同交互的复杂生态中，如何构建出一整套即便运行行为永不停歇地演化、也依然可控、可靠且具备确定性安全边界的软件工程基础设施。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=4s) what happens to software engineering when the software itself starts learning? — Mallika Rao > [00:04](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=4s) 它只是在昨天是完美的。 — Mallika Rao > [00:04](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=4s) 自适应系统的代码仅解释运行机制，真正的生产交付物是“行为”（Behavior），必须以对待代码同等的严谨度来工程化“行为”。 — Mallika Rao > [04:18](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=258s) A real-time inference system is not necessarily a real-time learning system — Mallika Rao > [04:18](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=258s) 工程挑战不在于盲目追求“学得更快”，而在于如何在安全可控的边界内实现“负责任地快速学习”。 — Mallika Rao > [04:18](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=258s) 人工干预不再被视为系统的异常或后备兜底（Override），而被正式纳入高价值输入信号体系。 — Mallika Rao > [10:03](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=603s) 架构决策的关键考量在于：“在当前阶段，我们正在优化哪一个时钟，又在权衡妥协哪一个时钟？” — Mallika Rao > [14:02](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=842s) did we actually discover a preference or did we actually create one? — Mallika Rao > [14:02](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=842s) 推荐系统不仅在被动响应现实世界，其输出结果正在主动篡改未来的数据分布。 — Mallika Rao > [19:42](https://www.youtube.com/watch?v=KMHuxIZGTlg&t=1182s) 搜索查询通过显式指令直接展现了人类的困惑、好奇与明确需求；传统推荐系统则通过交互行为构建了理解人类潜意识意图的隐式窗口。 — Mallika Rao ## Mentioned Mallika Rao, Zocdoc, Netflix, Walmart, Twitter, Ai4 ## Video description Mallika Rao - Senior Engineering Manager, Zocdoc (ex-Netflix, Walmart, Twitter) As recommendation systems evolve from batch-based personalization to adaptive, real-time discovery engines, infrastructure must transform too. This talk draws from large-scale experiences powering billions of recommendations and searches per day—spanning event-driven architectures, model-driven indexing, and feedback-loops built for continuous learning. We’ll explore how to evolve traditional recommender pipelines to support AI-native workloads—where training, inference, and evaluation all coexist in production. You’ll learn how leaders can architect not just systems, but trust and ambiguity—balancing experimentation and reliability while enabling teams to thrive amid shifting signals. This is a story about how distributed systems meet adaptive intelligence—and what it takes to make recommendations feel instant, relevant, and alive. In the era of AI-driven personalization, recommendation systems are no longer static ranking engines—they are adaptive ecosystems that continuously evolve through user feedback, contextual cues, and model retraining. This talk unpacks how large-scale distributed systems can transform to support this new paradigm of real-time discovery. Drawing from experience designing large recommendation and catalog-slicing infrastructures across consumer platforms, we’ll explore: Event-Driven Adaptation — How streaming signals and micro-batches bridge online and offline learning, turning user interactions into live feedback loops. Infra for Continuous Intelligence — Designing infra that allows training, inference, and evaluation to coexist seamlessly, without fragmenting data pipelines or reliability. Architecting for Ambiguity — Techniques for scaling before the problem is fully defined—building systems that embrace evolving data distributions and model behaviors. Trust & Leadership in the Loop — Why leaders must treat “infra debt” and “model debt” with equal rigor, architecting psychological and technical safety so teams can innovate responsibly. Dual-Path Execution — How maintaining separate product and infra strategies for AI adoption enables organizations to ship fast while investing deep. By the end, attendees will walk away with a playbook for evolving legacy recommender systems into adaptive, AI-native discovery platforms—and insights on how to lead teams confidently through that transition. Key Takeaways How to evolve recommendation infrastructure for real-time, adaptive intelligence. How to architect for ambiguity—designing systems and teams that thrive amid uncertainty. Why evaluation and feedback loops are as critical as training. How to balance product innovation with infra stability through dual execution paths. Leadership lessons for navigating the AI-driven evolution of large distributed systems. [Source](https://www.youtube.com/watch?v=KMHuxIZGTlg)

Source quotes

[0:04] Mallika Rao: what happens to software engineering when the software itself starts learning? [0:04] Mallika Rao: 它只是在昨天是完美的。 [0:04] Mallika Rao: 自适应系统的代码仅解释运行机制，真正的生产交付物是“行为”（Behavior），必须以对待代码同等的严谨度来工程化“行为”。 [4:18] Mallika Rao: A real-time inference system is not necessarily a real-time learning system [4:18] Mallika Rao: 工程挑战不在于盲目追求“学得更快”，而在于如何在安全可控的边界内实现“负责任地快速学习”。 [4:18] Mallika Rao: 人工干预不再被视为系统的异常或后备兜底（Override），而被正式纳入高价值输入信号体系。 [10:03] Mallika Rao: 架构决策的关键考量在于：“在当前阶段，我们正在优化哪一个时钟，又在权衡妥协哪一个时钟？” [14:02] Mallika Rao: did we actually discover a preference or did we actually create one? [14:02] Mallika Rao: 推荐系统不仅在被动响应现实世界，其输出结果正在主动篡改未来的数据分布。 [19:42] Mallika Rao: 搜索查询通过显式指令直接展现了人类的困惑、好奇与明确需求；传统推荐系统则通过交互行为构建了理解人类潜意识意图的隐式窗口。

Source quotes

[0:04] Mallika Rao: What happens to software engineering when the software itself starts learning? [6:07] Mallika Rao: How long until the system knows? [10:03] Mallika Rao: Which clock are we optimizing at this point of time? And which are we ignoring? [14:02] Mallika Rao: Did we actually discover a preference or did we actually create one? [19:42] Mallika Rao: Whose taste are we learning? Is it learning my taste? Is it learning my agent's taste?

Original transcript

[0:04] Good morning everyone. It's wonderful to be here at AI 4. Um I'm Mallika Rao and over the [0:13] years I've had the opportunity to um build and work in the space of search, personalization, [0:21] recommendation, and AI systems. And one question that has stayed with me through all of that work [0:30] is what happens to software engineering when the software itself starts learning? [0:38] Recommendation systems were one of the first places, I think we had to confront that at scale. [0:45] And I think the lessons that we have learned there are becoming increasingly relevant today [0:51] as we build search, the future of search, co-pilots, agents, and other adaptive systems. [0:58] Um the focus is shifting, especially in the future. So, I would like to use recommendation [1:05] systems as our lens today to actually explore what happens when software itself starts learning. [1:16] Let's start with a thought experiment. Imagine you've built the best perfect recommendation [1:22] systems. Uh the models are excellent, the ranking's excellent, uh the data is there, [1:28] the personalization is perfect. Except there's one problem, right? It was perfect yesterday. [1:38] Since then, the world has changed. Um new content has appeared, the users' interests have changed, [1:44] the context has changed. Uh something started trending, new products became available, [1:51] or new providers in the health care market came available with new availability windows. [1:57] So, the recommendation necessarily isn't wrong, right? It's just stale. [2:04] And once I started thinking about recommendation systems that way, [2:09] I realized I wasn't building a ranking system anymore. What it was truly was building software [2:16] that is capable of learning continuously. And that's a slightly different mental model. [2:24] If you think about it, traditional software has one simple contract. Right? We write code, [2:30] we deploy it, and until we deploy it again, we expect its behavior to largely remain the same. [2:38] Adaptive systems change that assumption. Their behavior can evolve even when the deployed code [2:47] doesn't. Not because an engineer changed the code behind the scenes or changed the software, [2:54] the system, but because the system learns something new. And once behavior [3:01] can change independently of code, software engineering starts to look very different. [3:11] That realization or just that way of thinking actually changed the way I think about these [3:18] adaptive systems. In traditional systems, mostly code is your production artifact. [3:26] Right? We version it, we test it, we deploy it, we roll it back, figure out how to roll it forward, [3:32] all of that good stuff. But in adaptive systems, I feel the code explains the mechanism, [3:40] but behavior defines that production artifact. And behavior evolves as the world around it evolves. [3:49] Models adapt, the data changes, the users change. [3:54] And because of this, I have gradually started to think about this differently. [4:00] Now, behavior is our production artifact. And perhaps, we should engineer behavior [4:07] with the same rigor and um effectiveness that we historically applied to code. [4:18] There's one difference also between traditional software and adaptive systems that I think changes [4:27] pretty much a lot, uh if not everything. Traditional software mostly observes the world [4:34] and responds to it. Right? Adaptive systems do something so much more. They actually start [4:42] participating in it. A recommendation changes what I watch, when I watch, how much I watch. [4:51] And what I watch actually becomes the new data. And that data shapes the next recommendations. [4:59] The system is influencing the way the signals uh behave with that learning system and what gets [5:06] shown later. So, if you think about it, that's no longer a pipeline. That is actually a closed loop. [5:17] And once I started thinking about recommendation systems as a closed loop system, the architecture [5:23] starts to look very different. I found myself returning to the same six responsibilities, not [5:32] technologies. Right? These are responsibilities like, how does the system sense, represent, [5:39] adapt, decide, act, learn, control. So, like I said, these are not six technologies. These are [5:48] responsibilities that pretty much every adaptive system has to cater to. And because every action [5:57] feeds that next cycle of learning, every decision influences what the system becomes tomorrow. [6:07] Let's make this concrete. Imagine something in the world suddenly changes, right? Like a product [6:13] comes available, a breaking story, something goes trending, there's a popular song somewhere [6:22] trending heavily on social media. Or in health care, like I said, [6:27] a provider, a very popular provider having new availability exposed in the system. [6:34] And the engineering question, irrespective of these domains, always remains the same, which is [6:40] how long until the system knows? Not how quickly can it make the prediction, right? That's a little [6:48] different, but how quickly can it sense, adapt, learn, and understand and decide the change? [6:58] A real-time inference system is not necessarily a real-time learning system, [7:05] and that raises another question, which is how does the system know? How is it learning? [7:13] Historically, our richer signals have come directly from people, from humans, [7:19] which is search, clicks, watch, skips, dwell times, right? Direct metrics. [7:26] And together, they help the system understand and build a representation of something that is [7:33] very complex and very important, and something that we talk a lot about today, especially, [7:39] which is taste. And we'll come back to it a little bit later. [7:45] So, those responsibilities, the six to seven responsibilities that we saw, [7:51] create one unavoidable tension in these systems, right? Um we want systems that can learn faster. [8:00] But the faster a system learns, the easier it becomes for that system to also learn something [8:07] that is wrong. And remember, every decision changes tomorrow's data. So, the engineering [8:15] challenge is not simply learning faster, but can we learn faster in a responsible way? [8:23] And without learning the wrong things. I think that's one of the defining engineering [8:29] core tensions of adaptive systems. And a lot about how we architect [8:34] the trade-offs and how our systems evolve actually reside on that one question. [8:42] There's another lesson that I've changed my mind about. We often think these adaptive systems [8:48] replace human judgment. But in practice, some of these highest quality signals have actually come [8:55] from people. A domain expert often understands a change or the data that is associated with [9:02] that change uh much better or much faster than the algorithm does. Imagine there is a local [9:10] event happening or something that is specific to a particular location or a geography or a class of [9:18] uh people, a segment in your market. That is usually operations or the merchandising folks who [9:25] actually discover this uh sooner than the system. So, I have actually stopped thinking about humans [9:32] in the system as an override or something that is exception that feeds into the system. [9:39] Instead, I have started thinking about that as another very high quality signal. And in my [9:46] experience, I feel the the best adaptive systems don't remove humans from the loop, but they give [9:54] them a much more principled place to embed into that architecture and have a place inside it. [10:03] The second mental model I have found useful is something I call the three clocks. Um it's really [10:11] helped me wrap my head through these systems that I have built. Um I think every adaptive system is [10:17] running on three different clocks, right? One is the world clock. What the one that [10:24] represents how quickly the reality is changing, the events, whatever is happening in the world. [10:30] The learning clock is how quickly the system can observe change, um understand it, and adapt to it. [10:39] We once had a system where the model quality was great, the ranking systems were great. [10:45] Um so the issue definitely wasn't the ranking or the relevance layer. The issue was that it took [10:49] a day before those signals, that new behavior could actually become part of the next set of [10:55] recommendations. So that's an example of the world clock and the learning clock not being aligned. [11:03] And the third clock is really the control clock, which is how quickly can the change be detected, [11:11] and how quickly can the system learn that it has learned something wrong, [11:15] and safely intervene. That last part is also very important. [11:21] So the goal, if you look at the three clocks, is not necessarily to make every clock go faster, [11:28] right? The goal is more to keep them in a fine balance. It's to make them learn at a speed [11:35] uh that we can safely govern, safely intervene, have explainability. Because if the learning clock [11:43] falls behind the world clock, then your system is stale. We definitely don't want that. But [11:48] if the learning clock outruns the control clock, then your system just becomes unpredictable. And [11:55] in some domains, it becomes dangerous. And there are numerous examples and case studies that I can [12:02] um share with you all if you connect uh behind the scenes later. Happy to share those stories. So, [12:09] the engineering challenge isn't simply building that system that can keep learning faster, but [12:14] it's actually building these systems in a way that learning stays governable and safe. [snorts] And [12:21] that's why I think a large number of architecture decisions can be reduced to one question, [12:27] right? Which clock are we optimizing at this point of time? And which are we ignoring? And if the [12:33] goal is to keep them aligned um and optimize for that, then it's a continuous check of how do you [12:39] want to evolve these clocks? And what's important for your product at a given point of time? [12:45] And for a long time, I thought that our core challenge was building better models, [12:51] right? I don't think that anymore. I think we're past that stage. Today, I think the harder [12:56] question is how can we trust the behavior of our recommendation systems and build uh it in a way [13:04] that it works in production at scale? And that single question just changes the infrastructure [13:10] and the logistics that we need uh as a baseline. Um how do we preview? How do we do shadow traffic? [13:18] What is our canarying infrastructure? Do we have shadow canaries? Can we do dry runs? Do we have [13:23] integration infrastructures? Um evaluations, roll back, these kind of production artifacts [13:32] that can actually help us build that uh trust. Because for the longest time I thought accuracy [13:38] in recommendation systems is our main bottleneck. Right? But I learned that trust is actually that [13:44] bottleneck. And trust isn't just that feeling, right? It's actually the architecture itself, [13:50] which our engineers need to think about when they are putting together design docs and evaluating [13:56] trade-offs and bringing that scientific thinking before we implement these systems. [14:02] Let's come back to that feedback loop a little bit because I feel like feedback [14:05] in recommendation systems is generally associated with something inherently positive. [14:10] Right? More feedback, better learning. But every engineer in this room knows that [14:17] closed-loop systems have their own dynamics that is complex. Imagine we recommend something [14:26] because it's popular. More people see it, more people engage with it, more people click on it. [14:32] The system learns this trend and it starts recommend- recommending it even more. [14:38] So, here the interesting question is did we actually discover a preference or did we actually [14:46] create one? So, recommendation systems start to look less and less like ranking engines and they [14:53] start looking like control systems. And [snorts] every recommendation system needs to think about [15:02] not just how does it respond to the world, but how does it use that data that it will later observe. [15:09] And if you're thinking in terms of machine learning, [15:11] that's just changing that future data distribution. Right? [15:18] Which leads to something that I've been thinking about a lot uh lately. [15:22] Um we often say recommendation systems learn our taste, but over time, I feel they also help [15:31] um to shape it in a lot of ways. Um every recommendation system subtly influences [15:38] what we discover, what we ignore, what do we want to discover next um in the future. [15:44] So, perhaps the relationship is not one directional, it's so much more mutual. Um I am um [15:51] influenced by what it is recommending, my taste is getting shaped by what the system recommends. Um [15:59] it's also learning from my taste, so um in reality, it's so much more both ways. [16:05] And I think um we're about to make that feedback loop even more complicated because increasingly, [16:13] there is another learner inside this loop, right? Not just the human, [16:17] not just the recommendation system, but another intelligence system that is um taking uh [16:24] actions and making decisions on behalf of the human. Um and I'll come back to that in a moment. [16:33] One thing that I have found as pretty much a constant working in AI is how quickly everything [16:38] changes. Uh right, new models, new architectures, new data, new terminologies. And whenever that [16:45] happens, I find myself going back to the first principles. Because the technologies evolve much [16:52] more faster than the engineering responsibilities that make product um decisions happen. Um [17:00] and that realization really changed the way I think about recommendation systems. [17:06] Earlier in my career, I used to think about it primarily as a machine learning problem, [17:11] right? Across my uh work at both Twitter and Walmart, it felt much more like a [17:16] machine learning uh problem, which is basically better models, better ranking, better relevance, [17:22] better stores, better embeddings, better representations. And then I spent more [17:27] and more time building it in production. And then that time I realized it's so much more a [17:32] distributed systems problem because freshness mattered, latency mattered. Um state mattered, [17:39] uh reliability mattered because learning was still happening in this distributed systems on mobile. [17:47] But today I think we are crossing another boundary. [17:50] Right, the hardest problems are gradually becoming less about making better predictions. [17:56] And they're more about managing this continuously changing behavior safely. If you think about it, [18:04] that's a software engineering problem. Because we're no longer just deploying software. [18:10] We're deploying systems whose behavior is expected to change after the deployment. And um I realized [18:18] the job now is not to just build more reliable systems, but um building systems that can adapt [18:27] reliably and making that part of the architecture. And that forces us to rethink some of the most [18:34] fundamental ideas. What does testing mean in such a situation? What does deployment mean? How do we [18:41] roll back safely? What does it mean to version behavior? And perhaps the deepest question, [18:48] what does it mean for a system to be correct when behavior is supposed to and expected to be [18:54] evolving? And if you think that question is hard today, it's only going to get harder tomorrow. [19:01] So recommendation systems have been wrestling with these questions for a long time. What [19:06] started as a recommendation system increasingly looks more like a software engineering question. [19:12] And I think um recommendation systems are no longer that niche question application, [19:19] right? They feel like large scale adaptive systems and that's where the next generation of [19:27] co-pilots agentic software systems are really going and it also makes space for [19:35] something that is an intelligent player in this ecosystem and they may be agents. [19:42] One more shift that I think about a lot recently is um um Search queries have always given a very [19:52] clear window into human intent. It just captures that curiosity so vividly. Confusion, that need, [20:00] exploration in just a few clicks, right? I think recommendation systems gave us another window. [20:09] If search gave us that very explicit window into human intent recommendation systems gave us a very [20:16] implied a very implicit window into that human intent and I think together they really capture [20:26] something that is very core to understanding what is the next [20:32] set of decisions that we need to make to really understand that human judgment. [20:38] And now imagine an agent that relationship, right? I don't issue those searches. I'm not [20:44] reading those 50 papers. My agent is doing that on my behalf. So whose taste are we learning? [20:52] Is it learning my taste? Is it learning my agent's taste? That becomes a core question [20:59] in this journey. And I think that's a fascinating question. Something that we [21:04] haven't figured out the full answer to, but it's going to be one of the core questions that will [21:10] really influence what is the future of agentic search recommendation systems going forward. [21:21] So, as we get closer to me wrapping up here, I think the next decade will really teach us how [21:29] to build those systems that can learn without losing that human judgment even if there are [21:36] multiple intelligent players in the system. And I think that is one of the defining software [21:41] engineering challenges of our generation. So, if I have to really leave you with one question, [21:48] it's that. How do we build the next set of infrastructure software systems, [21:55] and what does great software engineering really look like when the behavior never stops changing? [22:01] In fact, in a ecosystem where we expect the behavior to continuously change and evolve, [22:09] what do our software engineering systems look like? Thank you so much. I have a few [22:15] more slides which cover the case studies and examples. Happy to hang out and continue the [22:22] conversation. Do come and find me behind the scenes as well. Thank you again. [applause]