Ai4
Enterprise artificial intelligence agents break in production because static manuals ignore actual work, requiring live screen telemetry to create a deterministic context map that cuts token costs and prevents operational failure.
AI-assisted summary of The Enterprise Context Graph of Work: A Foundation Model Is Only as Good as the Context Behind It — original source. Check the original for full context.
Source published:
Detailed notes
## Background Published on YouTube by the Ai4 channel on August 22, 2026, this session features Avinash Misra, CEO and Co-Founder of Skan AI, alongside Amit Patel, Partner at Plug and Play Ventures. The page description highlights that while frontier AI models are intelligent, they lack specific enterprise context, making a living, observed Enterprise Context Graph of Work essential to grounding AI agents in operational reality.
## TL;DR The discussion explores why context, rather than foundation model selection, is becoming the primary durable advantage for enterprise AI. Avinash Misra explains how traditional approaches relying on static SOPs and RAG lead to broken production agents, whereas capturing the kinetic 'telemetry of work' directly from employee screens provides an accurate operational map. By treating work as a language and combining deterministic business rules with probabilistic LLMs, organizations can drastically lower token costs and navigate 'process collapse' as workflows transition to autonomous agents. Ultimately, the session emphasizes that enterprises must build and own their unique competitive context rather than relying solely on rented model intelligence.
## Atmosphere & style The session feels like an engaging, grounded fireside chat between a seasoned venture capitalist and a pragmatic technical founder, tailored for a late-day conference audience. The tone balances lighthearted camaraderie with sharp executive insights, blending practical industry analogies—like intelligent drivers versus detailed maps—with deep technical paradigms. It feels insightful, practical, and candid, steering clear of typical AI hype to focus on the messy realities of live enterprise deployment.
## Key takeaways - [00:08](https://www.youtube.com/watch?v=2kjXV50-iiA&t=8s) Enterprise context graph of work acts as the high-resolution map, whereas foundation models serve as the drivers traversing that territory. - [00:08](https://www.youtube.com/watch?v=2kjXV50-iiA&t=8s) Enterprise workflows are strictly deterministic and governed by rules, policies, and contracts; probabilistic AI must be grounded in this operational reality rather than guessing from first principles. - [03:08](https://www.youtube.com/watch?v=2kjXV50-iiA&t=188s) Standard Operating Procedures (SOPs) and documentation are merely declarations of intent, whereas true operational reality lives in real-time kinetic work telemetry and execution history. - [03:08](https://www.youtube.com/watch?v=2kjXV50-iiA&t=188s) Grounding AI through traditional RAG on static documents fails in production because it lacks exposure to real-world exceptions, workarounds, and human decision-making nuances. - [07:08](https://www.youtube.com/watch?v=2kjXV50-iiA&t=428s) Human workflow data contains significant noise, but continuous screen featurization allows AI systems to filter distractions and distill structural execution context. - [09:41](https://www.youtube.com/watch?v=2kjXV50-iiA&t=581s) Sandbox environments create a false sense of security by ignoring real-world workflow variations, causing AI agents to break when transitioning to production. - [09:41](https://www.youtube.com/watch?v=2kjXV50-iiA&t=581s) Bounding probabilistic AI within deterministic operational rules drastically improves tokenomics by lowering API consumption, computational waste, and electrical energy. - [13:36](https://www.youtube.com/watch?v=2kjXV50-iiA&t=816s) Enterprises will rent external AI models and raw intelligence, but their durable competitive advantage lies in building and owning their proprietary competitive context. - [13:36](https://www.youtube.com/watch?v=2kjXV50-iiA&t=816s) Just as human employees require onboarding to master internal processes, external AI foundation models require rigorous contextual onboarding to perform effectively. - [16:51](https://www.youtube.com/watch?v=2kjXV50-iiA&t=1011s) Work can be modeled like language, where screen UI elements and user actions are tokens, and end-to-end operational cases form complete sentences. - [18:02](https://www.youtube.com/watch?v=2kjXV50-iiA&t=1082s) Legacy enterprise user interfaces were designed around human memory limitations, showing only a fraction of data at a time. - [18:02](https://www.youtube.com/watch?v=2kjXV50-iiA&t=1082s) Agentic AI enables process collapse, consolidating fragmented application steps and complex UIs into streamlined, direct workflows. - [20:28](https://www.youtube.com/watch?v=2kjXV50-iiA&t=1228s) The primary differentiator in enterprise AI deployment is not the raw intelligence of the model driver, but the quality and depth of the context map it uses.
## Detailed walkthrough Here is a detailed, faithful walkthrough of the provided transcript segment:
[00:08](https://www.youtube.com/watch?v=2kjXV50-iiA&t=8s) Introductions and the Difference Between Probabilistic and Deterministic AI
Amit Patel, Partner at Plug and Play Ventures, opens the session by welcoming the tired late-day crowd, joking that his lesson from previous conferences was to swap formal shoes for Toms and flats. He introduces Plug and Play Ventures as a global venture firm with the world’s largest corporate innovation network, collaborating with major enterprises like Nike, Walmart, and Mercedes to drive growth and AI initiatives by connecting them with startups. Over the past decade, Plug and Play has backed 38 unicorns. He then introduces Avinash Misra, CEO and Co-Founder of Skan AI, noting that Plug and Play was fortunate to invest in Skan early in its startup journey.
Avinash frames Skan's central philosophy: "A foundation model is only as good as the context around it." He draws an analogy: while the tech world is obsessed with finding the absolute smartest driver on the road, Skan is obsessed with building the most detailed, high-resolution map of the territory in which that driving happens.
Amit steers the conversation toward the core technical distinction every enterprise leader must grasp: the difference between probabilistic and deterministic AI, and why that boundary matters when building context. Avinash explains that enterprise workflows—whether approving a bank loan or paying out an insurance claim—are fundamentally deterministic because they are governed strictly by written policies, regulations, and contracts. Decisions are binary: approved or denied. In human life, no one calculates $2 + 2 = 4$ by deriving arithmetic from first principles every single time; we rely on observed memory. Similarly, probabilistic engines like Large Language Models (LLMs) should not try to reinvent core business rules or guess from scratch. Instead, probabilistic AI must be firmly grounded in the deterministic operational reality of corporate policy, contract constraints, and the organization's lived execution history to choose the correct, compliant path.
- **Plug and Play Ventures track record:** Over 10 years of investing with 38 unicorns in its global portfolio. - **Corporate partners mentioned:** Nike, Walmart, Mercedes. - **The Core Analogy:** Frontier models are the intelligent drivers; enterprise context graph is the high-resolution map.
---
[03:08](https://www.youtube.com/watch?v=2kjXV50-iiA&t=188s) Telemetry of Work: Capturing Lived History vs. Declared Intent
Amit points out that Avinash was pioneering this concept long before "context graph" became a tech industry buzzword—referring to a Foundation Capital post by Juho Sarvikas that popularized the term. When Amit and Avinash first met around 2019, Avinash termed this concept the "telemetry of work." Amit emphasizes that while many process discovery tools exist, Skan stood out across Plug and Play's vast ecosystem because it captures real-world variations and human decision-making nuances rather than relying on static documents or standard operating procedures (SOPs).
Avinash clarifies that company documentation, manuals, and SOPs are merely "declarations of intent"—they describe how work *should* be done in theory, not how it actually happens in practice. The true "lived history" and "kinetic work" executed by employees daily contain all the exceptions, workarounds, edge cases, and nuances that define operational reality. Grounding AI solely through Retrieval-Augmented Generation (RAG) pulled from stale documentation fails because it lacks exposure to this kinetic execution plane.
To build an authentic enterprise context graph, Skan’s technology quietly observes work directly on the screens of hundreds of employees simultaneously. It featurizes everything on screen—mapping human decisions in real time across systems. Avinash argues that in an era where frontier AI models commoditize intelligence and level the playing field, an enterprise’s sole remaining competitive advantage lies in its execution history and daily kinetic work telemetry.
``` [ Declaration of Intent ] • Standard Operating Procedures (SOPs) • Static Process Documentation • System Logs & Static Rulebooks │ ▼ [ Traditional RAG Pull ] ───► Agent Breaks in Production (Grounded in Fiction)
───────────────────────────────────────────────────────────────────
[ Kinetic Work Telemetry ] • Observed Real-Time Screen Action • Human Exception Handling & Decisions • Cross-System Variations & Nuances │ ▼ [ Living Context Graph / Skan Engine ] ──► Resilient Enterprise AI ```
---
[07:08](https://www.youtube.com/watch?v=2kjXV50-iiA&t=428s) Individual Decision-Making vs. Institutional Intent
Amit shares a recent conversation with a Fortune 50 Head of AI who stated that his top priority was creating a "digital twin" of his company's operating model. However, Amit raises a crucial challenge: institutional memory only captures intent at the macro level. To build truly autonomous AI agents, companies must capture individual-level decision-making—the tacit knowledge and nuances held in the minds of individual workers.
Avinash acknowledges that capturing ground-level human activity yields immense amounts of raw noise. On any given task involving $N$ steps, a human worker encounters dozens of side deviations, open browser tabs, and momentary distractions. Skan’s technology is explicitly designed to observe this noisy stream, filter out the distractions, and distil the underlying structural context. Once this context is mapped, the process—which previously required constant human intervention—can be safely collapsed into an automated AI agent driven by a high-context execution layer.
- **Data Granularity Gap:** Institutional-level data reflects corporate intent, whereas ground-level worker data reveals actual operational execution. - **Noise vs. Signal:** Human workflow data is inherently noisy ($N$ steps interspersed with deviations); the Skan model distills raw screen activity into structured execution pathways.
---
[09:41](https://www.youtube.com/watch?v=2kjXV50-iiA&t=581s) Moving Agents from Sandbox to Scalable Production
Amit and Avinash tackle why enterprise AI agents frequently perform brilliantly in pilot sandboxes yet break when deployed into live production. Amit cites a recent meeting with the Chief AI Officer of a top-five US bank, who noted that tools must scale across millions of workflow variations to survive in production.
Avinash explains that sandbox environments are a trap because they use unrealistically clean, stripped-down processes with minimal edge cases. This creates a false sense of security during early demonstrations, acting as "poison" when transitioning to production. Production readiness requires enriching AI models with deep execution pathways and historical operational realities observed over time. When an AI agent encounters a situation it has seen the organization handle historically, it operates with true situational awareness; when it encounters a brand-new anomaly, it cleanly handsoff execution to a human.
Amit stresses that enterprises must "design for production from day one." Relying on simplistic sandbox testing hides severe issues like token consumption inflation and unviable "tokenomics." Leading enterprises like Walmart succeed with AI precisely because they enforce rigorous diligence upfront and design every system architecture with enterprisewide scale in mind.
Avinash adds that balancing deterministic rules with probabilistic AI drastically reduces computational waste. Restricting an LLM to choose among validated deterministic outcomes consumes significantly fewer tokens and less electrical energy than asking an unconstrained model to generate solutions from scratch.
- **The Sandbox Fallacy:** Sandbox tests succeed because they ignore variations, making them poor predictors of real-world deployment. - **Efficiency & Tokenomics:** Grounding LLM responses within deterministic bounds lowers token consumption, reduces API costs, and conserves energy.
---
[13:36](https://www.youtube.com/watch?v=2kjXV50-iiA&t=816s) Renting the Model vs. Owning the Competitive Context
Addressing a trend popularized by tech leaders like Satya Nadella and Alex Karp—the mantra to "rent the model, but own the intelligence"—Amit asks how enterprises can build lasting differentiation when everyone utilizes the exact same commoditized foundation models.
Avinash offers a crucial refinement: enterprises will rent both the foundational models and the raw AI intelligence. What companies must aggressively build and own is their own **competitive context**. Drawing a historical parallel, Avinash notes that 20 years ago, enterprises already utilized massive intelligent models—they were 20-watt biological brains sitting inside human skulls. To make a new employee productive, a company had to onboard them, teach them internal processes, and ground them in business nuances. Scaling external AI models requires the exact same onboarding discipline.
Amit points out a common blind spot among enterprise leaders: many assume that hosting a model in a private cloud container prevents data leakage, yet they fail to realize that frontier AI labs capture decision-making insights, exception handling, and evaluation logic. Avinash asserts that enterprise context must be strictly owned by the enterprise itself, framing Skan's role as helping clients continuously capture, own, and wire that context directly into their operational execution layers.
In the final minutes, the session opens up for audience questions. An audience member asks how Skan captures the specific reasoning and constraints behind complex human decisions. Avinash explains two-part methodology: 1. Long-term exposure of the system to all execution flows happening across the enterprise over extended timeframes. 2. Real-time screen featurization. Skan breaks down every attribute present on a worker's screen—such as customer onboarding state, geographic address, income brackets, or specific insurance claim loss parameters—and correlates those visual features across thousands of historical examples to model the underlying logic behind human choices.
- **Historical Parallel:** Human brains operate on ~20 watts inside the cranium; AI models require the same contextual onboarding process that human employees undergo. - **Screen Featurization:** Captures granular screen attributes (e.g., claim loss type, customer state, income levels) to infer the contextual logic driving decision-making.
[16:51](https://www.youtube.com/watch?v=2kjXV50-iiA&t=1011s) Treating Work as Language and Building a Model of Execution
The speaker articulates the foundational thesis behind Skan AI's technology by drawing a direct comparison to how large language models function. When OpenAI built its frontier models, it took the totality of the world's text—words, sentences, paragraphs, and n-grams—and passed them through a transformer architecture. By training the model to predict the next token, a broad, emergent understanding of language developed. Skan AI applies this exact framework to business execution by treating work as a language. In this paradigm, every element captured on a user's screen—the user interface components, individual action steps, and back-and-forth interactions between personnel—acts as the individual words of a sentence. A complete case, such as an insurance claim moving from initial intake to final adjudication, represents a full sentence.
When a model is constrained to observe countless iterations of these execution "sentences" and asked to predict the next token, a deep understanding of the organization's unique operational language emerges. For instance, the system notices underlying patterns in geographic workflows, observing that tasks involving the state of California consistently follow one path, while those involving the state of Texas branch off into another. Over time and at scale, the model accurately infers alternative execution paths, establishing a living mental model of how work actually runs across various systems.
[18:02](https://www.youtube.com/watch?v=2kjXV50-iiA&t=1082s) Optimizing Human Context Windows and Process Collapse
Addressing an audience question about leveraging user feedback to refine context, the speaker emphasizes that incorporating user interaction data is a primary use case in enterprise deployments. Traditional enterprise architectures span complex environments, including mainframe systems, web applications, and desktop platforms. By observing how employees navigate these tools, Skan AI's model identifies opportunities to short-circuit cumbersome workflows. Rather than forcing workers to rely on memory and context to move data between fragmented applications, the platform proactively delivers the required data right at the beginning or end of a task.
The speaker notes that current user interfaces are inherently constrained by the limits of human memory and cognition. Software interfaces were designed around human frailty—built to present 15 items on a screen rather than 500 because a human can typically hold only three to five details in mind at once. As enterprise workflows shift from human execution to agentic execution, a phenomenon described as "process collapse" occurs. The underlying technology stack and procedural steps collapse directly into autonomous agent workflows. Mapping how humans traverse the current software landscape serves two critical purposes: it immediately optimizes human productivity and provides the exact structural roadmap required for AI agents to operate within those same environments.
``` TRADITIONAL HUMAN WORKFLOW AGENTIC "PROCESS COLLAPSE" +---------------------------------------+ +---------------------------+ | Human User (Limited Context Window) | | Skan AI Context Graph | | * Navigates Mainframes, Web, Desktop | | * Maps execution steps | | * Manually transfers data | | * Short-circuits steps | +---------------------------------------+ +---------------------------+ | | v v +---------------------------------------+ +---------------------------+ | UI Built for Human Memory Limits | | Autonomous Agent | | * Displays 15 items, not 500 | | * Direct, unified | | * High friction & context switching | | process execution | +---------------------------------------+ +---------------------------+ ```
[20:28](https://www.youtube.com/watch?v=2kjXV50-iiA&t=1228s) Capturing Decision Nuance and the Driver vs. Map Analogy
Reflecting on Skan AI's impact across enterprise networks, the moderator highlights the critical need to capture subtle organizational nuances that standard platforms overlook. Autonomous agents require more than simple scenario feeds; they must understand historical institutional context, such as knowing that a specific decision was rejected three years prior in an executive conference room for a very specific reason. Capturing these fine variations and nuances in decision-making is what sets a true enterprise context engine apart.
Concluding the session, the speaker offers a single core insight for CIOs, CXOs, and organizational AI leaders: the primary differentiator in AI performance is not the raw intelligence or "convergence of the driver," but rather the quality of the map that the driver possesses.
- **Key Concept:** Skan AI treats enterprise work as a language where screen elements and user actions are words, and full cases (intake to adjudication) are sentences. - **Human Limitation:** Traditional software UIs are restricted by human context windows, designed for viewing ~15 elements rather than hundreds at a time. - **Process Collapse:** The transition from human-driven application navigation to direct agentic execution. - **Core Takeaway:** "The intelligence convergence of the driver is not at play. What is at play is the quality of the map that the driver has."
## Notable quotes > [00:08](https://www.youtube.com/watch?v=2kjXV50-iiA&t=8s) A foundation model is only as good as the context around it. — Avinash Misra
> [03:08](https://www.youtube.com/watch?v=2kjXV50-iiA&t=188s) declarations of intent — Avinash Misra
> [13:36](https://www.youtube.com/watch?v=2kjXV50-iiA&t=816s) rent the model, but own the intelligence — Amit Patel
> [20:28](https://www.youtube.com/watch?v=2kjXV50-iiA&t=1228s) The intelligence convergence of the driver is not at play. What is at play is the quality of the map that the driver has. — Avinash Misra
## Mentioned Avinash Misra, Amit Patel, Skan AI, Plug and Play Ventures, Nike, Walmart, Mercedes, Juho Sarvikas, Foundation Capital, Satya Nadella, Alex Karp, OpenAI
## Video description Avinash Misra - CEO & Co-Founder, Skan AI Amit Patel - Partner, Plug and Play Ventures
Frontier models are brilliant but blind. They know everything except your company: how work actually runs, where it breaks, and the institutional knowledge that lives in no training set. This layer, enterprise context, is the most underestimated and strategically important part of the AI stack, and the hardest for any model to replicate.
Most teams try to fill it with stale documents and system logs, and agents grounded in fiction break in production.
Avinash Misra, CEO of Skan AI, is ready to explore a different path: grounding AI in a living, observed model of real work. By observing how work runs across people and systems, enterprises can build an Enterprise Context Graph of Work, a continuously updated source of ground truth that closes the gap between what agents are told and what is true. The session covers why context, not model choice, is becoming the durable advantage, and how enterprises can start mapping it today.
[Source](https://www.youtube.com/watch?v=2kjXV50-iiA)