MarcoPolo Data Gateway Cuts AI Agent Token Costs Up to 60%
Ai4
Purging raw enterprise data immediately after active agent sessions prevents security leaks while reducing token context window expenses by 30% to 60%.
## Background
Published on YouTube by Ai4 on August 15, 2026, this presentation features Aseem Chandra, Co-Founder of MarcoPolo.dev. According to the page description, the session addresses how MarcoPolo serves as a single data gateway for AI agents to securely access enterprise systems (like CRMs, ERPs, and databases), keeping credentials hidden, enforcing access controls, and providing an auditable record of all actions.
## TL;DR
The presentation highlights how MarcoPolo enables AI agents to transition from generating static reports to safely executing real-world actions across live systems like Salesforce and Jira. By enforcing tightly bounded execution scopes, MarcoPolo ensures sensitive raw data is accessed only during active sessions and purged immediately after, while retaining generated assets as corporate intellectual property. Additionally, removing raw datasets from prompt context windows reduces token costs by 30% to 60%. The session concludes by emphasizing granular identity-level security and inviting risk, legal, and IT leaders to collaborate on evolving AI data governance.
## Atmosphere & style
The presentation carries a pragmatic, technical, and collaborative tone, tailored for an audience of enterprise leaders navigating modern AI security challenges. The speaker presents a grounded perspective, combining clear architectural principles with a candid acknowledgment that AI governance is an evolving field. The delivery feels educational yet forward-thinking, inviting open partnership rather than pitching a one-size-fits-all solution.
## Key takeaways
- [00:00](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=0s) Dynamic AI agents actively query and modify live systems like CRMs, ERPs, and ticketing systems without human-in-the-loop oversight.
- [00:00](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=0s) MarcoPolo serves as a unified data gateway that sits between AI agents and enterprise data systems to enforce strict access governance.
- [00:00](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=0s) Governance via MarcoPolo keeps enterprise credentials and sensitive raw records completely out of model prompt contexts.
- [00:00](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=0s) All operations executed by AI agents are recorded into a complete, auditable log for risk, legal, and IT leadership transparency.
- [18:47](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=1127s) By bounding execution rights to narrow scopes, agents can safely issue CLI commands and execute multi-system actions across tools like Salesforce and Jira.
- [18:47](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=1127s) Raw enterprise data remains accessible strictly during active sessions and is immediately purged upon session termination.
- [18:47](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=1127s) Customized queries, reports, and dashboards produced by AI agents are retained long-term as valuable enterprise intellectual property.
- [18:47](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=1127s) Keeping raw datasets out of model context windows yields a 30% to 60% reduction in token costs.
- [20:00](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=1200s) Effective data gateway architectures rely on governed workspaces, abstracted tooling, compounding enterprise context, and granular identity governance.
- [20:00](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=1200s) Enterprise AI security requires identifying and auditing individual agent and user actions rather than relying on shared service accounts or generic OAuth tokens.
## Detailed walkthrough
### [18:47](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=1127s) Moving from Reporting to Actionable Agentic Workflows
With the proper governance infrastructure in place, enterprise AI agents can evolve from simply generating static reports to actively executing actions across live corporate systems. Addressing the core question posed earlier in the presentation, the presenter explains that agents can now be given full authorization to act because MarcoPolo provides a single, controlled observation surface. For instance, an agent can initiate a CLI command to Salesforce using all relevant context maintained from prior interactions, retrieve the appropriate records, and then pivot to query Jira using a different operation. Because its execution rights are bounded within a tightly defined scope, the agent can update a live system safely without risking broader system damage or unauthorized data modification.
Throughout this operational cycle, raw enterprise data remains securely within MarcoPolo and is made accessible to the AI model strictly during the active session. Once the session terminates, all raw data is immediately purged. However, the intellectual property generated by the AI—such as customized queries, reports, or dashboards—is retained long-term. Beyond regulatory compliance and security, this architectural design delivers massive cost savings; organizations utilizing MarcoPolo in this manner report a 30% to 60% reduction in token costs because raw datasets are kept out of the model's prompt context window.
- **Actionable Execution:** Agents execute CLI commands and update live enterprise systems like Salesforce and Jira within narrow, defined guardrails.
- **Data Lifecycle & IP:** Raw data is accessible during the active session and deleted immediately afterward, while generated artifacts (queries, dashboards) are retained as enterprise intellectual property.
- **Cost Savings:** Token usage costs drop by **30% to 60%** due to efficient context window management.
[20:00](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=1200s) Defining the Data Gateway Architecture and Final Remarks
To properly govern AI agents at scale, building a dedicated data gateway requires a governed workspace equipped with appropriately abstracted tools, context that compounds over time as enterprise property, and robust identity-level security. The presenter stresses that modern AI security must extend far beyond standard OAuth tokens or shared service accounts. Instead, governance frameworks must individually identify and audit every distinct user and agent action across all connected systems.
Concluding the presentation, the speaker invites attendees to visit booth **841** to meet with MarcoPolo's engineering team or scan the displayed QR code to schedule a follow-up discussion. Frame-of-mind transparency marks the closing thought: the MarcoPolo team does not claim to have every single challenge of this emerging paradigm completely solved, but recognizes that a fundamental shift in business operations is underway. They actively seek partnerships with enterprise risk, legal, and IT leaders to co-develop the future of AI data governance.
- **Data Gateway Pillars:** Governed workspaces, abstracted tooling, compounding enterprise context, and granular, individual-level identity governance beyond shared service accounts.
- **Booth & Contact Information:** Attendees are directed to **Booth 841** or the provided QR code to connect with engineers and schedule deeper technical sessions.
- **Collaborative Vision:** Acknowledge that enterprise AI workflows are actively evolving, emphasizing community partnership to build effective, long-term governance solutions.
## Video description
Aseem Chandra - Co-Founder, MarcoPolo.dev
AI agents no longer just produce insights for people to act on. Instead, they reach into CRMs, ERPs, ticketing systems, databases, and warehouses in real time and act on the data themselves. That collapses the traditional governance model: there's no human in the loop every time to catch a mistake, credentials and sensitive records can slip into prompts, and an agent pointed at live systems can retrieve data it should never see. In this session, you'll see how MarcoPolo, a data gateway for AI to access data, becomes the single checkpoint through which every agent reaches enterprise data, enforcing who can access what, keeping credentials away from the model, and creating a complete, auditable record of every action an agent takes. Join us for real examples of how risk, legal and IT leaders can give AI agents access to real operational data with oversight and transparency, turning governance from a barrier into an accelerator of AI adoption at scale.
MarcoPolo argues that enterprise AI agents can safely transition from static reporting to executing live system actions by routing interactions through a dedicated data gateway that enforces strict execution boundaries, ephemeral data handling, and granular identity controls.
Safe Action Execution and Data Lifecycle Control (18:47)
According to the presentation, establishing a controlled observation surface allows enterprise AI agents to move beyond static reporting to executing live operations, such as running CLI commands in Salesforce or querying Jira. Restricting agent execution rights to tightly bounded scopes prevents system damage and unauthorized data modification while maintaining context across tools.
Under this architecture, raw corporate data remains within MarcoPolo and is accessible to AI models only during an active session, after which it is immediately deleted. Generated intellectual property—including custom queries, reports, and dashboards—is retained long-term. The presenter notes that keeping raw datasets out of model context windows also yields significant cost efficiencies, with organizations reporting a 30% to 60% reduction in token expenses.
Identity-Level Governance and Gateway Architecture (20:00)
Governing agentic workflows at scale requires a data gateway structured around governed workspaces, abstracted tooling, compounding organizational context, and identity-level security. The presenter asserts that standard shared service accounts and basic OAuth tokens are insufficient for agentic environments. Instead, enterprise systems must be able to distinctly identify, track, and audit individual users and agent actions across all integrated applications.
MarcoPolo concedes that the technical and operational challenges of enterprise agent governance are not entirely solved. Acknowledging an ongoing shift in business operations, the team emphasizes the need to co-develop evolving governance standards in partnership with enterprise legal, risk, and IT leaders.
精炼阅读摘要
演讲者提出，通过部署提供集中受控界面的数据网关，企业可推动 AI 智能体安全地跨系统执行操作，在通过会话级数据隔离降低 Token 成本的同时，实现深入到个体与智能体级别的合规治理。
受控执行与会话级数据隔离 (18:47)
根据演讲者的阐述，依托 MarcoPolo 构建的受控观察界面，企业 AI 智能体能够超越静态报告生成阶段，转而在生产环境中执行具体操作。智能体可以在受限的权限范围内运行 CLI 命令，例如在 Salesforce 中提取记录并结合上下文转至 Jira 发起查询，从而在规避未授权修改或系统损坏风险的前提下完成跨系统协作。
在数据生命周期与成本控制方面，该架构确保原始业务数据仅在活跃会话内对模型可见，并在会话终止后立即被清除，而模型产出的定制查询与仪表盘等知识产权则被长期保留。演讲者指出，将原始数据集排除在模型 Prompt 上下文窗口之外的做法，已帮助采用该架构的机构降低了 30% 至 60% 的 Token 使用成本。
数据网关架构与个体级身份审计 (20:00)
演讲者强调，规模化治理 AI 智能体需要依托由受控工作区、抽象化工具、持续累积的企业上下文以及身份级安全组成的数据网关架构。针对安全合规机制，演讲者明确提出现代 AI 安全防护不能停留在共享服务账户或通用 OAuth 令牌层面，系统必须具备对每个独立用户和智能体操作进行单独识别与全面审计追踪的能力。
演讲者坦言团队并未解决这一新兴范式下的全部难题，当前的重点是通过与企业风险、法务及 IT 管理团队展开合作，在企业业务操作模式重塑的过程中共同构建长期的 AI 数据治理方案。
Detailed notes — zh
## Background
本内容由 Ai4 频道于 2026 年 8 月 15 日在 YouTube 平台发布，由 MarcoPolo.dev 联合创始人 Aseem Chandra 主讲。视频重点探讨了随着 AI Agent 从被动提供洞察演进为主动访问与回写企业核心系统，传统数据治理模式面临瓦解，MarcoPolo 如何作为统一数据网关提供鉴权、凭据隔离与全生命周期审计，以加速企业级 AI 的安全规模化落地。
## TL;DR
Aseem Chandra 深入剖析了企业直连 MCP 带来的数据泄露、治理失控与 Token 成本黑洞等严重安全隐患，并指出未来 18 个月内缺乏控制面的 AI 项目将迎来撤退潮。为此，MarcoPolo 提出了统一 AI 数据网关架构，在 VPC 私有环境中利用 DuckDB 沙箱执行数据关联与过滤，确保原始凭据与敏感明细对模型完全隐蔽。同时，系统通过抽象七大标准 CLI 动词和沉淀私有化上下文，不仅将 Token 消耗降低 30% 至 60%，更帮助企业将'向模型租用智能'转变为'自持业务记忆'，实现跨系统的安全回写与自动化闭环。
## Atmosphere & style
整场演讲呈现出典型硅谷技术峰会务实、严谨且极具前瞻性的专业氛围。主讲人 Aseem Chandra 语调笃定从容，善于用高校数据违规等生动现实案例揭示行业隐秘痛点，在冷静剖析技术漏洞的同时给出体系化的解法架构。演讲逻辑层层递进，既有对 CISO 和架构师痛点的深度共情，又有开箱见山的技术推演，让听众在感受到治理危机的紧迫感之余，清晰看到可落地的工程化路径。
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=4s) 企业级 AI 的核心瓶颈不在于模型智力或算力，而在于缺乏能够安全、受控连接核心系统的数据网关控制面。
- [00:04](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=4s) MCP 协议虽已迅速成为连接 AI 与企业数据的行业标准，但其本身在安全控制、权限隔离和合规审计方面存在严重缺位。
- [01:43](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=103s) 企业对 AI 的采纳速度远超风控建设，在遭受 AI 接口层攻击的企业中，高达 92% 事前完全没有部署访问控制机制。
- [01:43](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=103s) Gartner 预测未来 18 个月内将迎来企业 AI Agent 的强制撤回潮，主因是缺乏控制面而非模型能力不足。
- [03:05](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=185s) AI Agent 颠覆了传统软件静态预设的治理逻辑，其运行时即兴生成工作流的特性让事前静态权限审批彻底失效。
- [03:05](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=185s) AI Agent 正迈向具备系统回写能力的自主执行阶段，未经审计的生产系统写入权限将带来指数级上升的崩溃与合规风险。
- [06:46](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=406s) 直接通过 MCP 连接底层会导致跨系统多表关联在 LLM 上下文窗口中裸奔，引发严重数据外泄、审计黑洞和高昂 Token 浪费。
- [08:36](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=516s) MarcoPolo 数据网关通过构建独立工作空间与单一管控检查点，物理隔离鉴权凭据，使 LLM 永远无法直接接触真实密钥与 API Key。
- [11:29](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=689s) 采用 DuckDB 隔离架构，原始数据就地处理，LLM 仅编写脚本执行计算并获取脱敏聚合结果，彻底隔绝底层行级敏感数据。
- [13:30](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=810s) 通过统一 MCP 接入并抽象出七大标准化 CLI 动词，彻底消除了异构数据源的碎片化，大幅减轻大模型的推理负担与思维链开销。
- [15:15](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=915s) 企业应坚持“向厂商租用智能，私有化自持上下文”，将业务逻辑、数据关联与验证过的查询脚本沉淀为专有数字资产。
- [17:29](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=1049s) MarcoPolo 支持 VPC 私有部署与单次安全认证，不仅为 CISO 团队减负，更通过仅返回精简聚合结果实现 30% 至 60% 的 Token 成本节约。
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=4s) 重新定义企业数据边界：从 MCP 爆发到 AI 数据网关的诞生
MarcoPolo 联合创始人 Aseem Chandra 开门见山地指出了企业当前面临的核心技术挑战：在企业级 AI 时代，企业亟需一个介于 AI Agent 与底层系统之间的“数据网关”（Data Gateway）。MarcoPolo 团队大约在两年半前便启动了该平台的研发，其核心定位就是作为一道治理与安全闸口，死死守在智能体（包括 Claude、ChatGPT、Codex、Perplexity 以及企业内部自研的各类专有 Agent）与企业核心系统（ERP、供应链、CRM、临床病患数据、财务数据库等）的正中央。当企业将这些高敏感的核心资产向自主决策的 AI 全面敞开时，传统的数据治理模式瞬间失效，构建专门的数据网关不再是可选项，而是必然前提。
Aseem Chandra 提到，模型上下文协议（Model Context Protocol，即 MCP 标准）在业内确立并普及的时间并不算长，在场几乎所有技术人都已深度接触或使用过该标准。MCP 俨然已经成为连接 AI 与企业数据的行业标准协议层，各类 MCP Server 在企业内部全面开花。然而，这种繁荣的背后隐藏着巨大的治理缺失——原生的 MCP 协议仅解决了连接通信问题，但在权限隔离、合规审计以及数据流转管控方面留下了危险的空白。
- **核心定位**：MarcoPolo 数据网关横亘在 AI 智能体（公有大模型及内部自研 Agent）与企业核心数据系统（CRM、ERP、财务、临床数据等）之间，作为统一治理关口。
- **协议现状**：MCP 迅速成为企业内 AI 连接数据的通用标准协议，但其本身在安全控制、长期上下文管理和细粒度访问策略上存在严重缺位。
---
### [01:43](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=103s) 失控的采用与缺失的控制面：企业 AI 即将面临的撤退潮
企业内部对 AI 的采纳速度远超安全与风控团队的预期，这也导致了极其触目惊心的安全盲区。Aseem Chandra 披露了一项关键数据：在遭遇 AI 层攻击的企业中，高达 92% 的机构在此前从未在 AI 接入层部署过任何访问控制手段。业务一线员工和开发人员的拥抱速度已经远远甩开了公司整体的治理建设，他们迫不及待地将全套企业数据直接暴露给各种大语言模型，导致“后门大开”。
这种控制面（Control Plane）的严重滞后引发了权威机构的高度担忧。Gartner 的预测显示，尽管目前 AI 部署势如破竹，但由于缺乏健全的控制面，在未来 18 个月内，许多盲目铺开 AI Agent 的企业将被迫大规模撤回、关停其生产部署。Aseem Chandra 强调，企业踩刹车绝不是因为底层模型不够聪明或者无法胜任业务——“模型本身已经非常强大，算力和智力根本不是瓶颈所在”，真正的瓶颈在于企业如何能够安全、受控、合规地将它们部署到组织中；如果当下不补齐这块拼图，所有盲目推进的企业很快都要为此付出沉重的治理与安全代价。
- **攻击暴露率**：在遭受 AI 接口层攻击的企业中，92% 的组织此前完全没有设立任何前置访问控制机制。
- **Gartner 预测趋势**：未来 18 个月内将迎来一波企业级 AI Agent 的强制撤回潮，其诱因并非模型能力缺陷，而是安全控制平面的缺失。
---
### [03:05](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=185s) 治理范式颠覆：当“运行时自主工作流”撞上敏感数据回传与写操作
在传统软件工程中，数据治理的逻辑是静态且完全预设的。工程师从业务需求出发，逆向推导需要构建哪些界面、编写什么 SQL 查询、设计何种工作流与数据库 Schema，以及数据应该存储在何处。在任何系统上线前，IT 和安全团队拥有绝对完备的上下文信息，可以从容定义访问权限与合规策略。然而，当 AI Agent 介入时，这一模型被彻底颠覆——Agent 在接收到 Prompt 后，是在运行时动态生成工作流，现场决定调用哪些系统与工具，开发团队在事前根本无法预知它会做出什么动作、读取哪些库表，传统的静态权限策略在动态生成的运行时面前彻底瓦解。
与这种不确定性相伴而生的，是严重的影子 AI（Shadow AI）与合规违规乱象。Aseem Chandra 分享了他在美东一家私立大学调研时的亲身见闻：该校获得美国联邦学生资助法案第四章（Title IV）资质，受严格的学生隐私法规约束。然而该校首席执行官为了辅助处于困境中的学生，竟自掏腰包开通 AI 账号，将包含大量隐私的学生数据整包喂入公有大模型寻求辅导方案。现实中，无数员工正掏出信用卡支付 20 美元，将包含客户、患者、学生甚至核心供应链的未脱敏 CSV 文件直接导出并上传到第三方大模型平台。尽管许多企业自欺欺人地认为采购的企业版协议具有“零保留政策”（Zero-day retention policy），但随着大模型厂商引入如 Fable 等安全与滥用检测机制，厂商为了审计敏感提问往往会强制在后端保留长达 30 天的数据。企业的数据治理权自此彻底失控。
更具威胁的是正在到来的第三波 Agent 落地浪潮：第一波是开发者利用 AI 辅助编写代码；第二波是知识工作者构建动态分析看板；而第三波则是具备“系统回写能力”（Write-back）的自主 Agent。这些智能体不仅读取分析数据，更能自主做出决策，并将决策结果直接回写到企业核心生产系统（如 Salesforce、Workday、PostgreSQL、Snowflake 等）。如果任由 Agent 获得未经审计的生产系统写入权限，企业将承担巨大的系统崩溃与合规风险。
```mermaid
flowchart TD
accTitle: 传统软件开发与 AI Agent 治理范式对比
subgraph TRAD ["传统软件治理模式（静态可预测）"]
T1["明确业务需求"] --> T2["固定代码、Schema 与静态工作流"]
T2 --> T3["上线前预设完整权限与安全策略"]
T3 --> T4["受控访问生产系统"]
end
subgraph AGENT ["AI Agent 治理困境（动态不可控）"]
A1["自然语言 Prompt"] --> A2["运行时即兴生成工作流"]
A2 --> A3["动态调取多源数据并回写生产库"]
A3 -.-> A4["权限黑盒：事前无法预知行为，引发合规失控"]
end
```
- **治理反转**：传统开发拥有全量事前信息；AI Agent 则是运行时自主规划路径，打破了传统的静态安全审批逻辑。
- **真实违规案例**：高校管理者因善意初衷，将受法律保护的学生隐私档案输入第三方公有模型，构成严重的合规违规。
- **协议漏洞**：大模型厂商因内容安全评估政策，可能留存数据 30 天，企业自以为稳固的“零天留存”在监管层面形同虚设。
- **业务回写冲击**：Agent 正在从只读的洞察助手演化为直接修改 CRM、HR、ERP 和数据库记录的行动者，操作失控的代价呈指数级上升。
---
### [06:46](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=406s) 直连 MCP 的致命硬伤：上下文内 Join、盲盒合规与成本黑洞
为了说明直接将 AI 通过 MCP 连接企业底层系统的致命隐患，Aseem Chandra 举了一个极为常见的技术管理查询：“上季度有哪些交易是因为公司的工程技术问题而发生推迟（Slipped）的？”在传统的直接 MCP 架构下，AI 智能体为了回答这个问题，必须先通过 Salesforce 的 MCP Server 建立连接，在大模型内部判定调用哪款工具，将 CRM 中的商机与客户数据全盘捞回大模型的 Context Window 中；接着，它又通过 Jira 的 MCP Server 进行独立鉴权，将研发工单与缺陷数据同样整包拉入上下文。
这种模式的致命伤在于：**两套核心系统的多表关联（Join）操作完全暴露在大语言模型的 Context Window 中执行**。这不仅直接把核心销售数据和内部工程缺陷拱手送给第三方大模型服务商，更给企业的 IT 和风控团队造成了巨大的审计黑洞——安全团队根本无法知晓大模型究竟拉取了哪些敏感字段，一旦发生数据污染或越权，没有任何审计日志能够复盘事故细节。此外，这也带来了沉重的财务负担：多个 MCP Server 具备不同抽象层级的工具，而这些工具上下文无法在各层之间妥善留存，模型每次执行复杂检索都必须反复调用、重新推理并承担高昂的重试 Token 费用。
```mermaid
flowchart LR
accTitle: 常见直接 MCP 直连模式下的数据泄露与上下文开销示意图
subgraph CLIENT ["大语言模型运行时"]
LLM["LLM Context Window<br/>(在此执行跨库 Join 数据拼接)"]
end
subgraph TOOLS ["直接暴露的 MCP 节点"]
MCP1["Salesforce MCP Server"]
MCP2["Jira MCP Server"]
end
subgraph ENTERPRISE ["企业底层系统"]
SF["Salesforce (CRM 敏感商业数据)"]
JIRA["Jira (内部研发工单与技术缺陷)"]
end
LLM -- "1. 鉴权并全量拉取商机" --> MCP1 --> SF
SF -.-> MCP1 -. "返回原始数据".-> LLM
LLM -- "2. 再次鉴权并拉取 Bug 记录" --> MCP2 --> JIRA
JIRA -.-> MCP2 -. "返回原始数据".-> LLM
```
- **数据外泄风险**：跨系统的表连接与过滤均在 LLM 的上下文窗口中裸奔，敏感业务数据直接暴露在公有云模型端。
- **审计盲区**：IT 治理人员完全无法感知何时有哪些数据被推入了第三方模型，合规追踪全面瘫痪。
- **Token 浪费**：由于缺乏持久化的工具上下文管理，模型在跨 MCP Server 处理工具定义时需要反复重试与解析，产生无谓的高昂开销。
---
### [08:36](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=516s) 解构 MarcoPolo 数据网关：隔离凭据与企业级可审计执行层
面对直连 MCP 的种种乱象，许多团队的第一反应是“自己写脚本把 MCP 节点包一层”。Aseem Chandra 直言，如果不从整体架构层面重构思考，缝缝补补根本行不通。正确的解法必须包含两个不可或缺的核心构件：第一是一个**独立的工作空间（Workspace）**，让数据处理、过滤与表关联在安全的受控环境内完成，严禁将未加工的原始底表全量喂进 LLM 上下文；第二是一个**单一管控检查点（Single Checkpoint）**，让所有 Agent 的工具调用与系统连接都必须强制穿越该检查点，以实现全生命周期的可审计性与细粒度授权。这就是 MarcoPolo 所倡导的企业级 AI 数据网关。
在技术架构剖析中，MarcoPolo 数据网关精准坐落在大模型 Agent（左侧）与企业海量数据源（右侧）之间，对外屏蔽底层复杂性，对内构建了严密的分层架构。网关自底向上设计为四层：最底层是**连接层（Connection Layer）**，原生预置了超过 100 种企业级系统连接器，无缝覆盖各类对象存储、关系型数据库、数据仓库、数据湖以及主流 SaaS 平台；连接层之上则是至关重要的**凭证作用域层（Scoped Credentials Layer）**。MarcoPolo 在此彻底剥离并隔离了所有鉴权凭据——LLM 永远无法触碰真实的系统密码、API Key 或数据库 Token，凭据完全封存在企业专有的控制层中。当新的 Prompt 到来时，网关会参照类似 Linux/Unix 的细粒度权限模型，动态解析并校验该提示词对应的合法权限范围，从而在不泄露凭据的前提下安全代理访问下游系统。
```mermaid
flowchart LR
accTitle: MarcoPolo 数据网关分层体系架构
subgraph AGENTS ["AI 客户端 / 智能体"]
A["Claude / ChatGPT / 自研 Agent"]
end
subgraph GATEWAY ["MarcoPolo Data Gateway (统一检查点)"]
direction TB
G1["执行工作空间 (安全 Join / 过滤计算)"]
G2["作用域凭证层 (隔离密钥，LLM 永不可见)"]
G3["治理与审计日志层 (端到端可追溯)"]
G4["连接层 (预置 100+ 连接器)"]
G1 --> G2 --> G3 --> G4
end
subgraph DATA ["企业核心资产"]
D1["SaaS (Salesforce / Workday)"]
D2["数据库 (PostgreSQL / Snowflake)"]
D3["数据仓库 / 数据湖 / 存储"]
end
A <==>|"受控 Prompt 与脱敏结果"| GATEWAY
G4 <==>|"真实凭证代管通信"| DATA
```
- **安全计算空间**：通过专属 Workspace 接管计算，仅将脱敏与聚合后的必要结果返还给大模型，避免上下文窗口被原始敏感数据淹没。
- **凭据物理隔离**：LLM 无法接触任何系统账户密码或 Token，所有权限解析在企业受控网关层内部完成。
- **预置连接生态**：开箱即用支持 100+ 种常用存储、数据仓库、数据库与 SaaS 软件的标准接入。
- **类 Unix 权限控制**：在组织级与用户级构建细粒度的访问上下文与审计轨迹，使每一次 Agent 动作均具备法律级别的合规留痕。
### [11:29](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=689s) Schema 预注入与 DuckDB 隔离架构
大语言模型（LLM）要想与企业数据有效交互，核心瓶颈在于缺乏对 Schema（表结构）与业务上下文的深刻理解。Aseem Chandra 形象地类比：这与一名刚入职的新员工毫无二致——新人会反复在 Slack 里或向资深工程师发问：“这些数据去哪儿找？”、“这个字段和表到底是什么业务含义？”企业过去依赖老员工答疑解惑，如今若让 LLM 在生产环境中自行盲目探索，不仅成本高昂且极易出错。MarcoPolo 采取前置预注入（pre-seed）机制，在最前端按系统类型主动向模型提供定制引导：例如告知模型访问 Salesforce 需要编写 SOQL 查询及对应范例，而访问关系型数据库或 Snowflake 则需编写标准 SQL 查询。这种指导让模型能迅速锚定目标系统，省去海量低效试错。
```mermaid
flowchart TD
accTitle: MarcoPolo 数据隔离与治理架构
subgraph Client["前端与模型层"]
Agent["AI Agent / LLM"]
end
subgraph Gateway["MarcoPolo 网关层"]
Auth["访问控制与预置 Context / Schema"]
DuckDB["DuckDB 隔离计算沙箱"]
end
subgraph EnterpriseData["企业数据源"]
SFDC["Salesforce (SOQL)"]
DB["企业数据库 / 数据仓库 (SQL)"]
end
Agent -->|"1. 统一 CLI 指令"| Auth
Auth -->|"2. 路由查询"| SFDC
Auth -->|"2. 路由查询"| DB
SFDC -->|"3. 原始数据返回"| DuckDB
DB -->|"3. 原始数据返回"| DuckDB
Agent <-->|"4. Python/Shell 执行计算"| DuckDB
DuckDB -->|"5. 仅脱敏聚合结果"| Agent
```
系统更关键的突破在于彻底阻断 LLM 直接接触底层原始数据。来自各类企业系统的原始数据会被直接送入 MarcoPolo 托管的 DuckDB 实例中就地驻留。LLM 并不读取原始行级记录，而是编写 Python 脚本或 Shell 脚本，在 DuckDB 容器内部完成表关联、数据对齐与聚合运算；模型最终收到的仅仅是高度精简的汇总结果，用于生成 Trace 视图并向终端用户展示。这一架构不仅严格隔绝了敏感数据泄露，更实现了端到端的行为审计——模型写的每一行代码、执行的每一条 SQL、接触的每一个数据节点均被全量捕获，从而打破冷启动难题，让执行经验不断累积沉淀。
---
### [13:30](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=810s) 统一 MCP 接入与七大标准 CLI 动词
MarcoPolo 架构的核心理念在于构建单一管控检查点（Single Checkpoint）：无论企业内部员工使用的是哪一种 AI 工具或模型，底层均统一路由至 MarcoPolo 网关。集成流程极其精简，客户端只需从官网下载 Skills 配置文件，通过标准 MCP（Model Context Protocol）连接协议即可在几分钟内完成部署。在统一网关层，系统即刻接管全生命周期的安全治理：从用户身份校验（Identity）、细粒度角色（Role）、连接会话、数据集权限范围，到即时权限撤销与审计追踪。若 Agent 触发的 Prompt 试图跨越授权边界触碰未开放的数据集，请求将在网关层被当场拦截并拒绝。
为了解决多工具调用带来的 Token 膨胀与推理困境，MarcoPolo 推出了一套跨系统通用的标准化 CLI 工具集，高度抽象出七个核心动词：
- `list`（枚举可用资源）
- `query`（查询业务数据）
- `describe`（获取元数据与架构描述）
- `browse`（浏览目录与层级结构）
- `upload`（上传数据）
- `download`（下载数据）
- 以及针对不同场景的执行动作
这一抽象彻底解放了 LLM 的推理负担。模型不再需要花费昂贵的思维链推理（Reasoning）去苦恼“该调用哪一个 MCP Server”、“目标工具有何特定入参”，而是无论面对 Salesforce、HubSpot 还是自建 CRM，均以统一的 `query` 语法执行检索，从根本上消除了异构数据源在协议层与接口层上的碎片化问题。
---
### [15:15](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=915s) 打造企业自持上下文：租用智能，私有化记忆
业界近期的重要共识在于：企业除了要将评测集（Evals）牢牢掌控为自身的数字资产，底层沉淀的上下文层（Context Layer）更是未来不可替代的“企业大脑”。上下文精准记录了业务流程、核心业务指标定义、跨系统数据集关联，以及具体权限流动规则。然而现状令人担忧：绝大多数员工在每次开启对话时，都在不断重复手写 Prompt 描述这些背景；这些核心业务逻辑散落在各大 AI 提供商的外围 Harness 层中，完全归服务商所有，企业一旦计划切换底层大模型，就不得不面临数据导出、迁移甚至完全重构的巨大阻碍。
> “继续向大模型厂商租用智能，但必须牢牢拥有属于你自己的上下文。”
Aseem Chandra 尖锐指出，正如 20 年前大家意识到数据是战略资产而引发大数据革命一样，上下文如今正是企业必须自持的核心战略资产。MarcoPolo 通过三项机制保障资产沉淀：
- **消除冷启动**：预先注入模型运转所需的全部业务语义与系统信息，使系统在初次接入时即可全速运转。
- **查询自迭代复用**：系统持续记录并沉淀所有验证有效的历史查询脚本，后续任务直接基于前人成果继承优化，拒绝推倒重来。
- **分层工作区继承**：支持在数据连接级、项目级、用户级多维度动态创建隔离 Workspace，新项目或新入职人员可以直接继承过去的上下文资产，实现真正的“开箱即拥有全部经验”。
---
### [17:29](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=1049s) VPC 私有部署、CISO 减负与跨系统自动化闭环
面对企业客户最关心的安全合规与部署形态，MarcoPolo 采取完全驻留在客户私有云环境（VPC）的部署模式，企业对数据出境（Egress）与数据驻留（Data Residency）拥有 100% 的绝对物理控制权。针对现代企业安全团队（CISO）疲于应付的困境，Chandra 分享了一个真实案例：他们为了敲定一次安全审查，曾耗费整整六周时间，原因在于安全团队早已被内部员工随意引入的大量新 AI 工具、新型 MCP Server 和零散连接搞得焦头烂额。MarcoPolo 提供了“一次认证，全面覆盖”的解决方案：安全团队仅需针对网关本身进行单次审查，配置 20 到 40 个允许访问的企业后台，后续所有审计日志即刻全量对接推送至企业现有的 Okta、Azure AD、Splunk 或 Datadog 监控面板中。
这一闭环使 AI Agent 从单纯的“被动分析汇报”跃迁为“受控主动执行”：
| 架构维度 | 传统 Agent 接入模式 | MarcoPolo 网关治理模式 |
|:--- |:--- |:--- |
| **执行闭环能力** | 仅停留在只读报告层，无法落地写操作 | 跨 Salesforce、Jira 等系统安全更新，执行范围受严格白名单限定 |
| **原始数据流向** | 敏感明细直接塞入外部 LLM 上下文窗口 | 原始数据仅留存于本地会话沙箱中，会话终止后即刻销毁 |
| **生成物所有权** | 查询语句与代码逻辑随 Session 废弃丢失 | 生成的 SQL、脚本、看板资产均沉淀为企业自持 IP |
| **Token 消耗成本** | 随着全量数据丢入窗口，消耗失控膨胀 | **Token 支出直接削减 30% 至 60%**（仅返回精简聚合结果） |
| **身份与权限体系** | 依赖粗粒度的共享 Service Account 或通用 OAuth | 精准透传并追踪具体个人身份（Who is doing what） |
演讲尾声，Chandra 谦逊地总结：MarcoPolo 并不标榜自己已经彻底搞定了一切挑战，当前企业工作流正迎来一场底层范式大迁移，他们正与全球企业伙伴并肩探索新边界。他热情邀请与会者前往展台 841 与核心创始研发团队交流，共同定义 AI Agent 时代的数据治理新秩序。
## Notable quotes
> [01:43](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=103s) 模型本身已经非常强大，算力和智力根本不是瓶颈所在 — Aseem Chandra
> [06:46](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=406s) 两套核心系统的多表关联（Join）操作完全暴露在大语言模型的 Context Window 中执行 — Aseem Chandra
> [15:15](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=915s) 继续向大模型厂商租用智能，但必须牢牢拥有属于你自己的上下文。 — Aseem Chandra
> [15:15](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=915s) 开箱即拥有全部经验 — Aseem Chandra
> [17:29](https://www.youtube.com/watch?v=gk__MkWu0Ng&t=1049s) 一次认证，全面覆盖 — Aseem Chandra
## Mentioned
MarcoPolo, Aseem Chandra, Model Context Protocol (MCP), Claude, ChatGPT, Codex, Perplexity, Gartner, Fable, Salesforce, Workday, PostgreSQL, Snowflake, Jira, DuckDB, HubSpot, Okta, Azure AD, Splunk, Datadog
## Video description
Aseem Chandra - Co-Founder, MarcoPolo.dev
AI agents no longer just produce insights for people to act on. Instead, they reach into CRMs, ERPs, ticketing systems, databases, and warehouses in real time and act on the data themselves. That collapses the traditional governance model: there's no human in the loop every time to catch a mistake, credentials and sensitive records can slip into prompts, and an agent pointed at live systems can retrieve data it should never see. In this session, you'll see how MarcoPolo, a data gateway for AI to access data, becomes the single checkpoint through which every agent reaches enterprise data, enforcing who can access what, keeping credentials away from the model, and creating a complete, auditable record of every action an agent takes. Join us for real examples of how risk, legal and IT leaders can give AI agents access to real operational data with oversight and transparency, turning governance from a barrier into an accelerator of AI adoption at scale.
[Source](https://www.youtube.com/watch?v=gk__MkWu0Ng)
[0:04] My name is Aim. Uh I'm with Marco Polo. I'm
a co-founder with this company. Uh we started
[0:09] about two and a half uh years ago uh in terms of
building out what I'm going to show you today.
[0:15] And uh I'm going to talk to you a little bit about
what is a data gateway and why do you need one?
[0:21] Um and the key thing here to remember is we
sit between your AI agents and your enterprise
[0:28] systems. So when I use the word AI agent,
I mean quad or chat GPD codecs, perplexity.
[0:37] It also includes any agents you're building
yourself internally in your organization.
[0:42] Uh and when I talk about enterprise systems, I
mean anything that contains your enterprise data,
[0:48] ERP, supply chain, CRM, uh clinical
data, patient data, financial data, etc.
[0:56] Um, so that's the scope of what we're going
to talk about here and why data governance is
[1:01] incredibly important when you open up access to
AI agents to all of your enterprise systems. So,
[1:08] you know, the interesting thing here is the MCP
standard standard was introduced only about 18
[1:13] months ago. I bet you there's not one person in
this room that has not worked with it or doesn't
[1:18] know what it is. It is now the standard adoption.
uh I mean the the adoption is completely obviously
[1:25] uh taken hold inside the enterprise
uh with with these MCP servers and
[1:31] this is the protocol layer that connects your
AI to your enterprise data. Um but there's quite
[1:38] a few things that are missing with it. We'll
talk about that in a minute. The second thing
[1:43] I wanted to talk about is while AI adoption
is just off the charts inside the enterprise.
[1:49] The problem uh that we're seeing is that 92%
of the organizations that experience an attack
[1:56] uh at their AI layer had never actually put in any
kind of an access control in front of it. So think
[2:02] about that for a second because your users are
actually way ahead of where most companies are
[2:07] in terms of deploying AI and starting to uh
give it access to all of your enterprise data.
[2:13] Um and then the third point is really around this
projection that I came across uh from Gartner
[2:19] uh that really caught my attention because
while AI deployment is pretty widespread,
[2:25] the control plan hasn't quite fully uh been
implemented yet. So what that is going to lead to
[2:31] is in the next 18 months or so you're going to
see that a lot of companies that are deploying
[2:35] AI agents across their business are going to
have to pull back some of those deployments.
[2:40] And it's not because the agents will not be smart
enough or they can't do the work. The models are
[2:45] already very smart. That's not the constraint.
uh the constraint is how do you deploy them
[2:51] securely and safely inside your organization and
for that we're going to all have to pay some price
[2:56] uh as these things start to take uh
take hold in the organization okay
[3:05] so the question is why is this so hard right
we have been working with data for a long time
[3:11] um but when AI agents meet your enterprise
data it's a very different kind of workload.
[3:19] Historically, traditionally, we're all familiar
with building all kinds of software. We start by
[3:25] defining what the business user needs or the user
needs. We then work our way back to what screens,
[3:31] what queries, what workflows need to be built
out, what schema needs to be defined, where does
[3:35] the data sit, how does it govern, etc. So you have
full information about everything you need to know
[3:42] before making a decision on what level of
governance are you going to put on that data.
[3:48] With AI, that model completely flips. When
you deploy an AI agent inside your enterprise,
[3:54] you may or may not have a good understanding of
what it is about to go do because it's building
[3:58] workflows on the fly. It's accessing different
systems on the fly. And so you really don't
[4:03] have a lot of information ahead of time that is
defined that helps you determine what level of
[4:08] access to give to these agents. The second thing
that happens is these agents are moving through
[4:16] live data and pushing that into uh into LLM and
different model providers all over the place.
[4:24] Uh I met with a CEO of a university
on the east coast. I won't name them.
[4:31] Um, and I was just having lunch with their uh CEO
and you know I asked them how they manage student
[4:38] information and um they're a title four school and
so they essentially the C is a small school so the
[4:46] CEO is taking all of the student data and putting
it into the AI and asking it for information or
[4:52] guidance on how to navigate some of the students
who are struggling. It's a wonderful concept,
[4:57] but legally you're not uh supposed to put student
information into these third party systems.
[5:06] I can assure you, I have talked to so many
companies where there's a belief that everything
[5:10] is secure because we've tightened the hatch down.
Your users are pulling out their credit cards,
[5:15] putting in 20 bucks, downloading data in a CSV
file from wherever it is that currently sits,
[5:19] and they upload it into uh into these LLMs. So you
have customer data, patient data, student data,
[5:26] supply chain data all sitting in these LLMs. Their
enterprise contracts will say that there's a zero
[5:32] day retention policy. Uh that changed with Fable.
So because the LLMs now need to be able to control
[5:40] uh what response they give you based on the
sensitivity of the questions being asked, they
[5:44] are going to have to retain your data for 30 days.
So now from a data governance standpoint, you have
[5:49] no idea what data sits in third party systems
and you have no control over it any longer.
[5:56] Okay. And the last point I'm going to make
here is agents need to be able to write back
[6:00] to operational systems. So this is different from
you know the first wave was developers writing
[6:04] code using AI. The second wave we saw with co-work
being launched in January where knowledge workers
[6:09] start to build dashboards and going all over the
place. Now we're showing the next wave that we
[6:14] see coming is these agents that will now not only
look at your data but they'll make decisions on
[6:19] what to do next and then they will operationalize
those decision decisions in the systems that your
[6:23] teams currently own. So that might be Salesforce
or workday or a [ __ ] database or a Postgress
[6:28] database or snowflake whatever it is that
they're currently using. So the the question is
[6:34] are you going to let these agents act and how
are you going to manage what they're able to do
[6:39] in your systems in the same way that you worked
with uh your employees in the past. Okay.
[6:46] So this is a example very simple a common query
that you see is which deal slipped last quarter
[6:56] because of engineering issues in the company and
any tech company any IT organization that works
[7:03] uh with data knows this types of questions come
at them all the time and the first thing that
[7:10] if you're directly connecting your AI to an
MCP uh connection is the AI goes and looks for
[7:16] information on Salesforce connects to it. Uh
it will then resolve which tool to use to get
[7:22] information about what data to bring back. It's
bringing that data back into its context window.
[7:29] Then it goes and looks into Jira assuming
that's the system you're using for ticketing
[7:33] uh and and tracking. It's going to go authenticate
there separately, return another set of data
[7:38] and brings it back into its context window. Why?
Because it needs to be able to join across those
[7:41] and those joins are happening in the context
window of the LLM. So your data now is already
[7:46] exposed to an LLM provider where the policies
are changing over time as they're responding
[7:53] to some of the threats that they see um uh
from their standpoint. So there that's one
[7:58] problem. The second problem is from a IT and
a governance standpoint you have no visibility
[8:04] into what data was just pushed into a third party
system and you have no control over it. So how do
[8:09] you audit it? How do you know when something
went bad? What is it exactly that happened?
[8:14] By the way, there's a whole discussion around cost
of multiple MCP servers with different tools at
[8:19] different layers of abstraction that the LLM has
to resolve each time it accesses those servers
[8:24] because the context for what those tools
can do is not retain anywhere at the moment.
[8:30] So each time you're paying for retries and
and uh resolving what that context looks like.
[8:36] Okay. So the normal reaction that anybody
would have is okay let me go figure this out.
[8:42] Let me go start with what I know which is MCP
servers work. Uh they are the industry standard
[8:48] of the moment. Let me uh wire up more connections.
Let me try to put some controls around it
[8:54] and let me put some governance around
it and build those things myself.
[8:58] The problem is you unless you are taking a
a big step back and thinking this through
[9:04] from an architectural perspective, you're not
going to land at the right answer. The right
[9:08] answer requires you to have a workspace, a place
where the AI can actually do the work without
[9:14] bringing all that raw data into its own context.
That's the first part of the solution in terms of
[9:19] how you uh how to start thinking in terms of uh
what that solution looks like. The second part of
[9:24] this is what is that single checkpoint where every
tool call, every uh connection has to pass through
[9:32] so that you have full visibility and you're able
to audit back and see what actually happened
[9:37] when these AIs are accessing your systems. So
what you need what we're calling is a data gateway
[9:43] which allows you to manage access to preserve
context over a period of time at not just a
[9:50] user level but an organizational level. and then
have data governance at scale. So you can look
[9:55] back and audit and see what actually happened,
who's got access to what systems. So this um
[10:02] here is the architecture slide that explains a
little bit of what this looks like behind the
[10:07] scenes for those of you that uh like to dig under
the covers and see how the sauce I just made.
[10:13] Um so I'll start from the left uh your left and
you know you have all these different AI systems,
[10:21] agents, etc. running inside uh your
business, inside your companies.
[10:25] Users are spinning up their own ones
depending on what they're comfortable with.
[10:28] On the right hand side, you have all these
different data sources that we just talked about.
[10:33] The Marco Polo data gateway sits in the
middle. Every connection goes through
[10:37] the Marco Polo data gateway. Um and it has four
layers to it. It starts with a connection layer
[10:44] where we have pre-built connections to 100 plus
different systems. that includes storage systems,
[10:52] databases, data warehouses, data links, uh SAS
applications and tools. You'll find a list of
[10:58] these on our website. Uh then above that there is
a set of scope credentials. So we have completely
[11:04] isolated the credentials from what the the the
LLM sees. So the LLM is not able to actually
[11:09] access the credentials. The credentials live in a
layer that belongs to you and your organization.
[11:15] And so anytime there's a new prompt, the LLM
has to resolve the credentials for that prompt
[11:21] and then be able to access the
systems on the other side of it.
[11:25] If you're familiar with how things work at
the Unix level or at the Linux level with
[11:29] uh these types of controls, it's it's able to it's
able to ask for certain resources to be deployed,
[11:35] but it can't directly access them. It's a
very similar model that we're following here.
[11:40] The next layer is the one that I feel like a lot
of uh folks have not quite figured it out yet and
[11:45] hasn't hit their radars. Um but in order for an LM
to work with your data, it needs an understanding
[11:51] of the schema. It needs an understanding of the
context. And it's no different than a new employee
[11:55] joining your organization asking questions like
where do I find this data? What does this table
[11:59] mean? And what what do I you know? So all of those
questions which are being answered in Slack or by
[12:04] senior developers or engineers in the company, you
need to provide a lot of that information to LMS
[12:09] in the front end and instead of it discovering
it on its own, we actually preede it with all
[12:14] of the right information at different levels. So
we can tell it for example that for Salesforce
[12:18] access you need to write a soal query and this
is what it looks like and for a [ __ ] access
[12:23] you need to write a regular SQL query
and this is what that looks like.
[12:26] So that actually really narrows down how quickly
it can move into the right parts of your systems
[12:31] to get access to the data. And then the third
the fourth layer is the most important which is
[12:35] all the data that comes back from different
systems is not the raw data is not sent to your
[12:42] AI at all. It is held in this duct DB instance and
over there the LLM can write Python code or shell
[12:51] scripts to join data to build relationships to
aggregate it and then the final result of course
[12:57] has to be shipped back to the LLM so it can render
it in its trace file and your users can see it.
[13:02] So this isolates the LLM from looking at
raw data and creates a certain level of uh
[13:09] of data governance around it. every query that
it writes, every piece of code that it touches,
[13:14] all the data that it touches, we record
everything so you can go back and look at it.
[13:19] It also helps in building this context over
time so that the LM doesn't have to redo
[13:24] each time and start from scratch. So we
avoid that whole cold start problem. Okay,
[13:30] so um I'm going to go through each of these really
quickly here, but if you're curious to learn more,
[13:36] we're at booth 841. Stop by. my founding engineers
are here with me. Uh we'd love to talk to you.
[13:42] Okay, so this is the core point is from
an access standpoint, there is a central
[13:47] point. No matter which AI you're using, it
will route through the Marco Polo gateway.
[13:53] The connection is very simple to establish.
It's just literally an MCP connection that
[13:57] you establish by downloading of skills files from
our website and set it up. Takes a few minutes.
[14:02] Um but you can see here that we're enforcing
identity, the role, the connection, the data set,
[14:07] the revocation, the audit trail from that one
single connection. And if that prompt doesn't um
[14:15] have access to the data that it's trying to
reach, we reject it right at the at that level.
[14:21] The second innovation that we've done
here is introducing a standard CLI
[14:27] with seven verbs that work across all
the systems that we're connecting into.
[14:33] So now the LLM is not spending a lot of time or
tokens thinking through which MCP server to use,
[14:41] what tool to invoke, what does that tool do, etc.
And we've uh abstracted it to a common level of
[14:48] list, query, describe, browse, upload or
download data. So when you say connect to
[14:55] say Salesforce and [snorts] then query customer
data, it that query will work consistently whether
[15:01] it's HubSpot or your own CRM or some
other database. Okay. Um so there's one
[15:09] the core point here is there's one place to to
see all the information and one place to audit
[15:15] this context piece is really important and I
think this is going to be you know uh we've just
[15:20] I think my previous uh presenter here was coding
SATA we've seen some recent quotes from him
[15:25] talking about not just how eval are
incredibly important for you to own the IP
[15:30] but the context layer that you're building which
describes your business processes your key metrics
[15:35] the relationships between different data sets
uh how that data is being used, who can use what
[15:40] information etc. This becomes the new um brain
of the organization over a period of time as you
[15:48] accumulate it. Most of this is being described
in prompts each time a user starts a new session
[15:55] and each time that user is coming up with
their own prompts to describe all this
[15:59] context information for the LLM to operate
that sits in the harness layer of the AI,
[16:07] not something that you own. Uh if you decide
to switch from one to another uh AI, you now
[16:12] have to either import and export or rebuild uh
and think through how you're going to do that.
[16:17] So what we're advocating is you should continue
to rent the intelligence from these LLM providers
[16:23] but you should own your own context. Context is
just another form of data. Most of you all figured
[16:28] out that data is really really important 20 years
ago. Um and so we had the big data revolution.
[16:33] Now context is equally important as a strategic
asset that you need to continue to own. So what
[16:38] do we do here? one, we um initialize all of
this information of what the AI needs so that
[16:44] there's no cold starts when it starts
to use your system for the first time.
[16:49] Every query that you do is being recorded and it
improves each time it runs those queries and it
[16:55] reuses the queries that have already worked in the
past. So it doesn't start from scratch. It starts
[16:59] from what it knows from the last time uh that
it ran through your systems. And then lastly,
[17:04] these workspaces can be spun up at a connection
level, at a project level, at a user or level.
[17:11] And so these workspaces can now inherit the
context of these other previous iterations
[17:17] and not start from scratch. So that's
as if a new employee joined your company
[17:20] and already knows what has already been done
and doesn't have to start from scratch. Okay.
[17:29] The co most common question I get uh when I was at
the booth yesterday is probably about a hundred of
[17:33] you stopped by and uh where does this run uh from
a security and governance standpoint. So obviously
[17:38] we deploy this in your VPC uh in your cloud
environment by uh we can manage all of your data
[17:45] residency requirements all of the egress of the
data is in your control. And the one thing that
[17:51] if any of your CISOs or security people in the
room uh that you're going to appreciate is right
[17:56] now I tried to get one of our customers CISO
to uh join us for a call so we could review our
[18:03] our uh uh stance. Um and it took
six weeks to get the meeting
[18:07] with the internal champion constantly you know
supporting us and and reaching out on our behalf.
[18:12] And the reason is because they're incredibly
busy trying to figure out how to secure all
[18:16] these systems that their users are exploiting and
pushing pushing data into. And so they have to do
[18:21] that each time somebody deploys a new MCP server,
a new connection, new AI. What we're advocating
[18:26] is connect to us once, certify us, tell us
the 20, 30, 40 different systems that your
[18:31] employees are allowed to connect to. Certify it
once and you're done. And so your CISO should
[18:37] breathe a sigh of relief. All of the information,
all the data from our systems can be pushed to,
[18:43] you know, octa or Azure AD or Splunk or data
dog, whatever it is that you use internally,
[18:47] so that you still have one surface from where
you can observe everything that happened.
[18:52] So now that question we asked four or five
slides ago, if you ask the same question again,
[18:57] you can now with this infrastructure
in place tell the agent to act,
[19:01] not just report. And so now it will run a CLI
command to Salesforce with all of the context
[19:09] previous uh from prior interactions available to
it. It'll get the right information. It will then
[19:15] query Jira with the same command with a different
verb. Uh get the right information and then it
[19:20] can go and update a live system in a very narrow
scope that you've defined of what it's able to do.
[19:27] The raw data all stays in Marco Polo. It's
available during the session to the AI. Once
[19:32] the session is over, the raw data is removed.
The artifacts that the AI creates, that might
[19:38] be queries or dashboards or whatever else, those
are retained as your IP over a period of time.
[19:46] The side benefit of this customers that are
using us in this way are reporting 30 to 60%
[19:52] drop in token costs because they're not sending
all their raw data into the uh context window
[19:59] of the of the AI. So this is what we believe
it takes to build a data gateway is a governed
[20:04] workspace with the right tools that are abstracted
at the right level context that you own and that
[20:08] compounds over a period of time and then of course
security and data governance that's just beyond a
[20:14] ooth token or a uh shared uh service account but
it identifies individually who's doing what in
[20:21] those systems. Um, I'd encourage you to stop by
our booth, uh, 8:41. Uh, meet with our engineers,
[20:29] talk to us. Um, if you'd like to schedule
some time with us, that QR code up there will
[20:34] take you to a link from where you can schedule
some time with us. And after this conference,
[20:39] we're happy to talk to you and uh, and dig a
little bit deeper. And I'm going to leave you
[20:44] with this. We're we're not here claiming that
we have this completely figured out. We're here
[20:50] saying there's a new way in which business works.
We're trying to figure it out and we value your
[20:56] partnership to figure out how to make it happen.
So even if you'd just like to talk to us, schedule
[21:00] some time with us and we're happy to talk to
you. Thank you so much for your time. [applause]
