Ai4
Unchecked autonomous agents risk project cancellations across 40% of enterprises unless platform teams implement centralized infrastructure control over model and tool consumption.
AI-assisted summary of Tokens, Tools, and Trust: How Agent Gateways Keep Your AI ROI from Going Rogue With Nutanix — original source. Check the original for full context.
Source published:
Detailed notes
## Background This presentation was delivered by Tuhina Goel, Director of AI Product Marketing at Nutanix, published on YouTube by the Ai4 channel on August 17, 2026. The page description indicates that the talk covers the role of token management and agent gateways in establishing cost controls, securing tool interactions, and scaling enterprise agentic AI applications safely.
## TL;DR As enterprises scale autonomous agentic AI applications, many face runaway token costs and security governance risks—a phenomenon illustrated by the 'Monday morning surprise' of exponential bill shock. Drawing on Nutanix's 15-year history in IT infrastructure, the presentation argues that rogue AI is an infrastructure control plane problem rather than a model intelligence issue. Nutanix proposes the Nutanix Agent Gateway alongside a 'Two-Tier Intelligence' model, where 20% of complex tasks rent frontier models while 80% run on predictable, fixed-cost private infrastructure. By providing centralized traffic routing, token rate limits, and tool-level filtering via MCP servers, enterprises can secure their AI front door and achieve sustainable AI ROI.
## Atmosphere & style The presentation is authoritative, polished, and strategically pragmatic, delivered with the clear confidence of an enterprise IT veteran. The narrative follows a sharp trajectory—opening with a relatable, cautionary scenario of weekend agent sprawl and financial shock before grounding the crisis in solid industry metrics. Through memorable analogies like 'putting a Ferrari engine into a lawnmower,' the session maintains an engaging, consultative tone that transforms complex governance challenges into manageable architectural solutions.
## Key takeaways - [00:04](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=4s) Nutanix brings 15 years of IT infrastructure management experience to AI, viewing AI adoption challenges as familiar infrastructure problems like shadow IT and lack of central control. - [01:30](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=90s) Enterprise AI success can cause financial panic when autonomous agents split tasks and default to calling expensive frontier models without proper guardrails. - [02:48](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=168s) Gartner predicts over 40% of agentic AI projects will be canceled by 2027 due to unmanaged costs, unclear business value, and inadequate risk controls. - [02:48](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=168s) Token consumption per task has evolved three orders of magnitude in three years—from thousands of tokens for basic chatbots to billions for long-running autonomous agents. - [05:00](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=300s) Rogue AI is a missing control plane problem rather than a model issue, requiring centralized control for what agents spend (tokens) and what they touch (tools). - [07:47](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=467s) The Nutanix Agent Gateway provides a centralized control layer, universal API translation, and traffic management for public, private, and tool-connected AI workloads. - [09:21](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=561s) Only ~20% of enterprise AI tasks require top-shelf frontier models, while 80% are repeatable tasks that can run on smaller, cheaper, or self-hosted models. - [09:21](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=561s) Effective token governance relies on three pillars: visibility across users and teams, token budgets and rate limits, and cost-to-value dynamic routing. - [12:00](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=720s) The Agent Gateway intercepts tool calls via Model Context Protocol (MCP) to enforce granular, function-level security policies and maintain detailed decision audit trails. - [14:45](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=885s) Relying on a single frontier model provider introduces severe vulnerabilities around cloud outages, vendor price increases, and regulatory shifts. - [17:33](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=1053s) Organizations should adopt a Two-Tier Intelligence model: renting frontier LLMs for complex reasoning (20%) and owning private inferencing for high-volume tasks (80%). - [17:33](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=1053s) Long-term winners in the agentic AI era will be defined by unit economics and infrastructure control planes rather than raw model performance.
## Detailed walkthrough Here is a detailed, faithful walkthrough of the transcript section.
---
[00:04](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=4s) Introduction and Nutanix's Heritage in IT Infrastructure
Tuhina Goel introduces herself as the Director of AI Product Marketing at Nutanix. She sets the stage by highlighting Nutanix's 15-year history of simplifying complex enterprise IT challenges—guiding customers through historical shifts from Hyper-Converged Infrastructure (HCI) to virtualization and Kubernetes. Looking at current AI infrastructure, she observes that it is not fundamentally different from past IT transitions; it exhibits the same familiar, chaotic patterns: fragmented DIY stacks, uncontrolled "shadow IT," and a severe lack of central governance and operational control.
Because Nutanix pioneered infrastructure governance and management, she emphasizes that the exact same foundational concepts and theories apply to AI infrastructure today. The presentation focuses on three tightly linked core pillars: 1. **Tokens:** Navigating and controlling the financial costs of model consumption. 2. **Tools:** Managing data access, integrations, and governance for actions taken by AI. 3. **Trust:** Bridging tokens and tools through infrastructural stability, security, and predictability.
- **Speaker:** Tuhina Goel, Director of AI Product Marketing at Nutanix. - **Nutanix Track Record:** 15 years spent streamlining complex enterprise IT shifts (HCI $\rightarrow$ Virtualization $\rightarrow$ Kubernetes). - **Core Problem:** AI adoption currently suffers from DIY stacks, shadow IT, and lack of central control. - **Thesis:** AI governance is fundamentally an infrastructure problem—a domain Nutanix specializes in.
---
[01:30](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=90s) The Monday Morning Surprise: When AI Success Causes Financial Panic
Tuhina illustrates a common scenario playing out across enterprise platform teams. A team builds a proof of concept (POC) or demo that performs brilliantly, prompting them to launch the agentic AI application into production. Over the weekend, the system functions exactly as designed: it takes a massive task, breaks it down into sub-tasks, spawns an autonomous agent, and that agent in turn spawns multiple sub-agents. Because no guardrails or routing policies were put in place, every single sub-agent defaults to calling the top-shelf, expensive frontier models ("because why not—nobody told them not to").
When leadership and platform teams return on Monday morning, no one is celebrating model capabilities anymore. Instead, finance is in the room asking hard questions about runaway costs, lack of control, and missing trust. The AI project did not technically fail—it succeeded brilliantly, but at the cost of consuming billions of tokens over a single weekend. The core issue wasn't the model's intelligence or capability, but rather a total absence of operational control.
- **The Agentic Multiplier Effect:** Autonomous agents split complex tasks into smaller sub-tasks $\rightarrow$ spawn multiple autonomous sub-agents $\rightarrow$ each sub-agent routinely queries top-shelf frontier models. - **The Core Breakdown:** AI applications perform as built, but burn through billions of tokens unchecked, leading to massive financial "bill shock."
---
[02:48](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=168s) The Scale of the Crisis and the Evolution of Token Consumption
This pattern is not an isolated fringe anecdote. Tuhina highlights a Gartner prediction stating that **over 40% of all agentic AI projects will be canceled by 2027**. This high failure rate is not due to bad models—as frontier models are rapidly improving—but rather stems from unmanaged costs, vague business value, and inadequate risk controls. Crucially, these aren't small lab experiments quietly dying; they are well-funded, load-bearing enterprise systems being actively shut down in production.
Tracing the three-year evolution of AI token consumption reveals an exponential growth in computational demands: - **2022 (Chatbots):** Simple question-in, answer-out interactions (e.g., early ChatGPT) consuming a few thousand tokens per task. - **Agents:** Emergence of reasoning loops and tool calls, jumping to tens of thousands of tokens per task. - **Autonomous Agents (Present):** Long-running, multi-step reasoning agents running for hours, days, or months. They maintain full conversation history, process files, retry failed queries, and generate millions or billions of tokens per task.
``` +-----------------------------------------------------------------------+ | AI TOKEN EVOLUTION (3-YEAR SHIFT) | | | | 2022: Chatbots --------------> Thousands of tokens / task | | (Single Q&A, short window) | | | | Agents ----------------------> Tens of Thousands of tokens / task | | (Reasoning loops, tool calls) | | | | Autonomous Agents -----------> Millions to Billions of tokens / task | | (Long-running loops, deep history, file references) | +-----------------------------------------------------------------------+ ```
Furthermore, tokens represent only half the problem. Modern agents don't just generate text; they act, trigger events, and invoke external systems. This expands enterprise exposure along two distinct axes: **what agents spend** (tokens) and **what agents touch** (tools).
- **Gartner Statistic:** Over 40% of agentic AI projects will be canceled by 2027 due to cost, governance, and risk controls. - **Magnitude Shift:** Token consumption per task has increased by **three orders of magnitude** in just three years.
---
[05:00](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=300s) Cost and Trust: Two Symptoms of a Missing Control Plane
Enterprise AI failure stories consistently stem from two interconnected issues: 1. **Cost (Token Chaos):** Organizations use a mix of OpenAI, Anthropic, and self-hosted models, yet cannot answer basic financial questions like, *"What did AI cost last month, and who spent it?"* Traditional enterprise financial models are built around predictable upfront software licensing or long-term consumption agreements. Modern AI operates completely differently—every model swap or vendor repricing fundamentally alters unit economics. 2. **Trust (Tool Risk):** Agents execute actions—reading databases, filing tickets, and interacting with GitHub repositories. Securing these actions is no longer a matter of simply drafting policy documents; it is an active engineering problem.
Enterprises often treat Cost and Trust as two separate problems owned by different teams. However, they share the same root cause: **Rogue AI is not a model problem—it is a missing control plane problem.**
``` +-----------------------------------------------------------------------+ | THE MISSING CONTROL PLANE PROBLEM | | | | Traditional Infrastructure Stack Current Agentic AI Stack | | +--------------------------------+ +------------------------+ | | | Network: Routers & Firewalls | | Autonomous Agents | | | +--------------------------------+ | (Tool-wielding & | | | | Cloud: IAM & Governance | | money-spending) | | | +--------------------------------+ +------------------------+ | | | Control Plane (Enterprise Grade)| | NO FRONT DOOR CONTROL | | | +--------------------------------+ +------------------------+ | +-----------------------------------------------------------------------+ ```
Historically, every major technology shift (such as networking or cloud adoption) required a centralized control plane (firewalls, routers, IAM) to become enterprise-ready. Deploying autonomous agents without a control plane at the front door is a fundamental architectural oversight. The key question enterprises must ask is: *"Who is governing the front door?"*
- **Financial Paradigm Shift:** Enterprises are unequipped for dynamic model pricing where costs fluctuate per request, model swap, or vendor price adjustment. - **The Core Analogy:** Running autonomous agents without an AI gateway is like running a network without routers/firewalls or a cloud without IAM.
---
[07:47](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=467s) Introducing the Nutanix Agent Gateway
To solve this missing architectural layer, Nutanix created the **Nutanix Agent Gateway**—a centralized control layer and traffic controller for all agentic AI transactions and workloads. Every incoming request, whether originated by a human or an autonomous agent, passes through this gateway.
``` +-----------------------------------------------------------------------+ | NUTANIX AGENT GATEWAY | | | | [ Humans / Autonomous Agents ] | | | | | v | | +-----------------------------------------------------------------+ | | | NUTANIX AGENT GATEWAY | | | | (Unified Endpoint, Traffic Control, API Translation Layer) | | | +-----------------------------------------------------------------+ | | | | | | | v v v | | [ Frontier Models ] [ Self-Hosted Models ] [ MCP Servers ] | | (OpenAI, Anthropic, (Private Infrastructure) (Tools & Private | | Google) Data) | +-----------------------------------------------------------------------+ ```
#### Key Capabilities of Nutanix Agent Gateway: - **Unified Endpoint:** Provides a single access point for developers and applications while routing requests behind the scenes to public frontier models (OpenAI, Anthropic, Google) or private, self-hosted models. - **Universal API Translation Layer:** Enables seamless provider swapping, automatic fallbacks when API credits run out, and gradual traffic shifting to smaller, cheaper models without rewriting application code. - **Centralized Visibility:** Creates a single enforcement point that eliminates "Shadow AI" without slowing down developer velocity.
- **Role:** Centralized control layer and traffic controller for all AI transactions. - **Vendor Neutrality:** Allows dynamic switching between hosted LLMs and self-hosted private models without application code modifications.
---
[09:21](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=561s) Cost-to-Value Routing and the Three Pillars of Token Control
Tuhina notes that organizations frequently misuse top-shelf frontier LLMs for mundane tasks, ranging from generating cat memes or cookie recipes to building complex GTM strategies or performing medical differential diagnoses. Because no guardrails exist, teams end up calling top-tier models for everything.
In reality, **only ~20% of complex tasks require top-shelf frontier models**. The remaining **80% consists of high-volume, repeatable work** that can run efficiently on smaller, less expensive models. Failing to route requests based on a "cost-to-value" basis leads to structural overpayment. Tuhina uses her favorite analogy: *"It's like putting a Ferrari engine into a lawnmower—we don't need it."*
``` +-----------------------------------------------------------------------+ | TASK & MODEL DISTRIBUTION | | | | +-----------------------------------------------------------------+ | | | 20% Complex Tasks --> Top-Shelf Frontier Models | | | | (OpenAI, Anthropic, Google) | | | +-----------------------------------------------------------------+ | | | 80% Repeatable Work --> Smaller, Cheaper, or Self-Hosted Models | | | | (High volume, fixed price) | | | +-----------------------------------------------------------------+ | +-----------------------------------------------------------------------+ ```
#### The Three Pillars of Token Governance: 1. **Visibility:** Granular telemetry tracking token usage per user, team, agent, and model. *"You cannot govern what you cannot attribute."* 2. **Control:** Defining strict token budgets and rate limits per department, team, or API key. Agents hit a "budget wall" instead of impacting the P&L with surprise bills. 3. **Optimization:** Dynamically routing traffic based on cost-to-value requirements, adaptively handling model price changes, outages, and evolving regulations.
- **The 80/20 Rule:** 80% of enterprise AI tasks can be offloaded to smaller/cheaper models; only 20% need frontier LLMs. - **Analogy:** Using frontier models for basic, high-volume tasks is like "putting a Ferrari engine into a lawnmower."
---
[12:00](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=720s) Securing Tools and Establishing Trust via MCP and Audit Trails
Because AI agents interact with systems via tools—frequently connected through Model Context Protocol (MCP) servers acting as "connective tissue"—the Nutanix Agent Gateway intercepts every tool call to enforce precise security policies.
``` +-----------------------------------------------------------------------+ | TOOL-LEVEL FILTERING VIA MCP | | | | [ Coding Agent ] ---> [ Nutanix Agent Gateway ] ---> [ MCP Server ] | | | | | | v v | | (Tool-Level Policy) [ GitHub Repo ] | | - ALLOW: Read/Check-in | | - BLOCK: Delete Repo | +-----------------------------------------------------------------------+ ```
#### Governance Features for Tools and Data: - **Tool-Level Filtering:** Operates at the granular tool function level rather than just the connection level. For instance, a coding agent can be granted permission to view GitHub issues and check in code, but explicitly blocked from deleting a repository. - **Comprehensive Audit Trails:** Records every tool call and agent interaction, exporting data to existing enterprise observability dashboards. - **Decision Integrity:** Beyond traditional role-based access control (RBAC), modern audit trails must document *why, how, and when* an agent made a specific decision.
By delivering policy-driven security, the gateway systematically eliminates Shadow AI across the enterprise while maintaining operational consistency.
- **Connective Tissue:** MCP (Model Context Protocol) servers link AI agents with enterprise tools and private data. - **Granular Control:** Filtering policies block destructive actions (e.g., deleting repos) while permitting routine functions (e.g., checking in code). - **Decision Integrity:** Audit trails track agent reasoning paths in addition to standard data access logs.
---
[14:45](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=885s) Strategic Questions: Renting vs. Owning Your AI Intelligence
Tuhina encourages organizations to reflect on three critical strategic questions when building their AI infrastructure:
1. **Are you paying per token for everything?** Paying frontier prices for basic work is unnecessary. Enterprise infrastructure should leverage fixed-price inferencing on self-hosted, private infrastructure already owned by the company. For many basic workflow tasks, simple "if/then" structures or smaller niche models work far better than public frontier models. 2. **How flexible is your AI strategy?** Vendor lock-in creates severe vulnerabilities. For example, a Cloudflare outage in November 2025 brought down ChatGPT globally for approximately five hours. Relying on a single vendor leaves organizations exposed to cloud outages, shifting government regulations, GPU supply constraints, and sudden vendor repricing. Vendor price hikes should trigger simple routing policy adjustments, not full architectural rewrites. 3. **Are you outsourcing your entire AI strategy?** If answers to these questions depend entirely on a single vendor's roadmap, the enterprise isn't just outsourcing intelligence—it is outsourcing its overarching AI strategy.
- **Real-World Incident:** A Cloudflare outage in November 2025 knocked out ChatGPT globally for roughly 5 hours, demonstrating the operational risk of single-vendor dependence. - **Routing vs. Rewriting:** Price changes from model providers should be handled by updating gateway routing rules, rather than re-architecting applications.
---
[17:33](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=1053s) Two-Tier Intelligence and the Broader Platform Paradigm
A standalone gateway is insufficient; it must be backed by an inferencing platform. The Nutanix Agent Gateway is integrated directly into the **Nutanix Enterprise AI Platform**, combining front-door traffic management with a predictable, fixed-price private inferencing engine.
Because both components belong to a unified platform sharing the same role-based access controls, API keys, and policies, switching between models is completely invisible to developers and platform engineers. Whether an application calls a public frontier model or falls back to a self-hosted open-source model, the user experience and security policies remain identical.
``` +-----------------------------------------------------------------------+ | TWO-TIER INTELLIGENCE | | | | Tier 1: Rent Frontier Models (20%) | | - Reserved for complex, non-repeatable reasoning tasks. | | - Paid on a variable, per-token basis. | | | | Tier 2: Own Private Inferencing (80%) | | - High-volume, repeatable, structured workloads. | | - Runs on existing enterprise infrastructure at a predictable, | | fixed cost. | +-----------------------------------------------------------------------+ ```
#### Two-Tier Intelligence Framework: - **Rent Tier (20%):** Pay per token to access frontier model capabilities for highly complex, non-standard tasks. - **Own Tier (80%):** Run high-volume, repeatable tasks on owned, private infrastructure at a predictable fixed cost.
Closing with a broader industry perspective, Tuhina notes that major platform shifts follow consistent economic patterns. The winners of the cloud era were not necessarily the companies with the best applications, but those with the best underlying economics and infrastructure controls. Agentic AI follows the exact same pattern: the long-term winners will be enterprises with superior unit economics around intelligence and the control planes required to enforce them.
She concludes by asking the audience: *"Who really governs your AI front door, and do you want to rent your AI strategy or own it?"*
- **Platform Integration:** Nutanix Enterprise AI Platform combines gateway control with private inferencing using unified keys and RBAC policies. - **Historical Parallel:** Just as in the cloud era, AI market leaders will be defined by unit economics and infrastructure control rather than raw model capabilities.
---
<ElicitationsGroup message="Explore further details about enterprise AI gateways and Nutanix solutions:"> <Elicitation label="Summarize Nutanix Enterprise AI Platform features" query="What are the key architecture details and features of the Nutanix Enterprise AI Platform mentioned in this talk?"/> <Elicitation label="Explain Model Context Protocol (MCP) in AI agents" query="How does Model Context Protocol (MCP) work for AI agent tool integration and enterprise security?"/> <Elicitation label="Compare rented vs. owned AI model infrastructure" query="What are the unit economics and cost comparisons between renting frontier models vs owning private self-hosted models?"/> </ElicitationsGroup>
[20:21](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=1221s) Conclusion and Call to Action
Tuhina Goel concludes the presentation by thanking the audience for joining the session and inviting attendees to continue the conversation in person. She encourages anyone interested in learning more about token management, agent gateways, or Nutanix's AI solutions to stop by the Nutanix booth to ask questions and discuss how to keep enterprise AI ROI on track.
- **Booth Location:** Attendees are invited to visit the Nutanix booth for further discussions and Q&A.
## Notable quotes > [01:30](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=90s) because why not—nobody told them not to — Tuhina Goel
> [05:00](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=300s) What did AI cost last month, and who spent it? — Tuhina Goel
> [05:00](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=300s) Rogue AI is not a model problem—it is a missing control plane problem. — Tuhina Goel
> [05:00](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=300s) Who is governing the front door? — Tuhina Goel
> [09:21](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=561s) only ~20% of complex tasks require top-shelf frontier models — Tuhina Goel
> [09:21](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=561s) 80% consists of high-volume, repeatable work — Tuhina Goel
> [09:21](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=561s) It's like putting a Ferrari engine into a lawnmower—we don't need it. — Tuhina Goel
> [09:21](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=561s) You cannot govern what you cannot attribute. — Tuhina Goel
> [14:45](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=885s) Are you paying per token for everything? — Tuhina Goel
> [14:45](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=885s) How flexible is your AI strategy? — Tuhina Goel
> [14:45](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=885s) Are you outsourcing your entire AI strategy? — Tuhina Goel
> [17:33](https://www.youtube.com/watch?v=uhTJt8IAiRM&t=1053s) Who really governs your AI front door, and do you want to rent your AI strategy or own it? — Tuhina Goel
## Mentioned Tuhina Goel, Nutanix, Nutanix Agent Gateway, Nutanix Enterprise AI Platform, Gartner, OpenAI, Anthropic, Google, ChatGPT, Cloudflare, GitHub, Model Context Protocol (MCP), Hyper-Converged Infrastructure (HCI), Kubernetes
## Video description Tuhina Goel - Director, AI Product Marketing, Nutanix
As organizations scale agentic AI applications, many struggle to move past the pilot phase due to unpredictable token costs and governance concerns. This session explores the critical role of token management in establishing precise cost controls and resource oversight, alongside the implementation of an agent gateway to secure and monitor AI agent, tool, and data interactions. Learn how these solutions enable the deployment of sophisticated workflows without compromising enterprise safety or trust. By unifying security, cost oversight, and scalability, organizations can shift from "AI-enabled" to "AI-efficient" so that every token spent translates into measurable business value and sustainable enterprise AI ROI.
[Source](https://www.youtube.com/watch?v=uhTJt8IAiRM)