Presentation by Tray.ai: How Zuora Is Deploying Agentic AI Across Its Enterprise With TrayAi & Zuora

Ai4

AI-assisted summary of Presentation by Tray.ai: How Zuora Is Deploying Agentic AI Across Its Enterprise With TrayAi & Zuora — original source. Check the original for full context.

Source published: 2026-08-22

Detailed notes

## Background Published on YouTube by the Ai4 channel on August 22, 2026, this session features Josh Noble, Field CTO at Tray.ai, and Mark Gill, Head of Infrastructure and Services at Zuora. According to the page description, the talk covers practical strategies for moving enterprise AI beyond pilot stages into daily production use while maintaining control over governance, risk, cost, and Model Context Protocol (MCP) services.

## TL;DR Josh Noble and Mark Gill break down Zuora's journey across three phases of enterprise AI adoption, highlighting production breakthroughs like their multi-source Slack "Sales Agent" alongside the operational frictions of employee "vibe coding." To maintain control over rapid prototyping, Zuora implemented automated workstation configuration files (such as agents.md) and a dedicated "Vibe AI" Git deployment pipeline into AWS. While custom builds offer agility in modular domains like contract management, Gill cautions that generated micro-apps can rapidly overwhelm downstream human IT support teams. The presentation concludes with urgent warnings regarding software supply-chain risks from unvetted AI package downloads and the industry-wide need for discrete, auditable agent identities equipped with operational kill switches.

## Atmosphere & style The session carries a candid, conversational, and pragmatic tone, eschewing canned vendor slide decks in favor of an unvarnished retrospective from the technical trenches. Josh Noble maintains an inquisitive, upbeat rhythm, while Mark Gill provides dry, battle-tested realism about the illusions of rapid front-end prototyping and the hidden complexities of backend infrastructure. Watching the discussion feels like sitting in on an honest peer-to-peer debrief that balances genuine enthusiasm for agentic development with sober, experienced-based warnings about enterprise security and operational debt.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=4s) Tray.ai evolved from data integration connectors and deterministic orchestration workflows into autonomous agent tool actions and full-stack TypeScript UI generation with Tray Helix. - [03:12](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=192s) Zuora consolidated fragmented tools like MuleSoft and Workato onto Tray, navigating three distinct AI adoption phases: initial exploration, vendor-embedded AI, and desktop vibe coding. - [03:12](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=192s) Zuora's internal 'Sales Agent' automated executive customer briefings by connecting Slack to 15 disparate backend systems, reducing prep time from a week of manual effort to seconds. - [03:12](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=192s) Enterprise agents must respect user context and role-based permissions, dynamically tailoring responses depending on the specific access rights of the requester. - [07:23](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=443s) Vibe coding creates an illusion of near-completion by quickly generating polished UIs, though backend integration and infrastructure require the majority of production effort. - [07:23](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=443s) Zuora developed an internal 'Vibe AI' deployment bridge to route locally generated vibe-code into standardized Git repositories and governed AWS cloud environments. - [11:18](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=678s) The democratization of AI coding is replacing rote programming jobs with specialized roles like 'AI Solution Design Engineers' who structure prompts, context, and architectures. - [11:18](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=678s) Zuora programmatically deploys configuration markdown files (`agents.md`, `claude.md`) to employee machines to enforce modular coding standards and internal deployment targets. - [14:57](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=897s) AI coding enables companies to selectively build modular internal tools (like contract lifecycle management) while keeping core systems of record firmly on commercial platforms. - [14:57](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=897s) Accelerating application generation creates severe downstream support bottlenecks, as human IT and service management capacity does not automatically scale with AI-generated software. - [17:45](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=1065s) Unmonitored coding agents present major supply-chain security vulnerabilities by autonomously pulling unverified third-party packages and potential malware from the public web. - [17:45](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=1065s) Agents require distinct worker identities rather than generic service accounts, enabling precise auditing and an immediate kill switch to revoke data access if an agent goes rogue.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=4s) Introductions and the Evolution of Tray.ai

Josh Noble, Field CTO at Tray.ai, opens the session at Ai4 by joking that he has "the coolest job out there," spending his days in the skunkworks experimenting with customers to build their "squirreliest solutions to their hardest problems with bubble gum and duct tape." Joining him on stage is Mark Gill, Head of Infrastructure and Services at Zuora, who oversees infrastructure operations, enterprise security, service management, and IT service management (ITSM). Rather than running a static slide deck or canned software demo, Noble and Gill frame the session as an open, practical retrospective on how Zuora operationalizes AI, manages risk, and scales automation across the enterprise.

To set the stage, Noble details how Tray.ai's platform has evolved across several distinct technical generations. It originally launched as a pure integration connector framework focused on reading, writing, and synchronizing data between disparate enterprise systems. That foundation subsequently expanded into an orchestration engine for deterministic, multi-step business logic workflows. As generative AI matured, Tray enabled those deterministic workflows to be exposed directly as tools and actions for autonomous AI agents. Most recently, Tray announced Tray Helix, a development layer that lets builders use LLM coding models like Claude and Codex to generate full-stack TypeScript front ends and interactive user interfaces—such as custom supplier portals, contract review dashboards, or executive tools—that deploy securely within an enterprise-governed environment.

```mermaid flowchart LR accTitle: Evolution of Tray.ai Platform subgraph S1["1. Connectors"] A["Read / Write / Sync Data"] end subgraph S2["2. Orchestration"] B["Deterministic Workflows"] end subgraph S3["3. Agent Tools"] C["AI Agent Actions"] end subgraph S4["4. Full-Stack UI"] D["Tray Helix Apps (TypeScript)"] end A --> B --> C --> D ```

[03:12](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=192s) Zuora’s Three-Phase AI Journey and the "Sales Agent"

Zuora’s engagement with Tray began roughly four years prior as a consolidation initiative across an overly fragmented integration stack. At the time, Zuora was juggling instances of MuleSoft and Workato alongside custom AWS Lambda functions and point-to-point Apex code inside Salesforce. Zuora deprecated the competing platforms and standardized its entire integration layer onto Tray. When Zuora kicked off its dedicated AI initiatives three years ago, Gill observed their progress unfold across three distinct phases:

* **Phase 1 (Exploration & Poking in the Dark):** Zuora brought in external consultants to build an LLM-powered ticket deflection system for IT support. The project failed to deliver in production and was eventually decommissioned, but it provided crucial lessons regarding the necessity of data quality, business context, and guardrails. * **Phase 2 (Vendor AI & Context-Aware Agents):** Enterprise vendors began embedding native AI into ITSM tooling to achieve real ticket deflection. Concurrently, Zuora leveraged Tray’s new LLM workflow features to build internal agentic tools. * **Phase 3 (Desktop Vibe Coding):** Employees across business units began actively using AI coding environments at their desks, introducing new operational challenges surrounding identity, data access, and unvetted deployments.

The standout milestone of Phase 2 was an internal system dubbed "Sales Agent." Prior to its rollout, whenever an executive such as the CFO planned to visit a client, up to three staff members would spend an entire week manually querying disparate internal systems to compile a briefing document. Zuora used Tray to hook an LLM and a Slack interface into 15 distinct backend data sources. An executive could simply request a customer brief inside Slack, prompting the agent to determine relevant data sources, extract the records, synthesize the insights, and return a tailored brief in seconds.

Crucially, the Sales Agent highlighted the importance of contextual enterprise identity. If the Chief Revenue Officer and the Chief Financial Officer queried the exact same customer in Slack, the system generated different outputs because each request inherited the requester's specific enterprise permissions and data access rights.

```mermaid flowchart TD accTitle: Zuora Sales Agent Architecture User["Executive in Slack"] -->|"Asks: Customer Brief"| Bot["Slack AI Agent"] Bot -->|"Evaluates User Identity & RBAC"| Engine["Tray.ai Orchestration & LLM"] subgraph Sources["15 Connected Data Sources"] CRM["CRM Records"] Fin["Financial / Billing Data"] Usage["Product Usage Logs"] Support["Support History"] end Engine <-->|"Filtered Data Extraction"| Sources Engine -->|"Synthesized Briefing"| User ```

[07:23](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=443s) The Reality of Vibe Coding and Zuora's "Vibe AI" Pipeline

As Phase 3 took hold, widespread access to AI coding assistants radically altered internal software development. Gill notes that tools like Claude and Codex create an illusion of completion: "what kills me is the first thing it gives anybody trying to vibe code is this big beautiful UI. And they look at it and they go, 'I'm 80% done.' And you go, 'Actually, you're not. You're about 10% done. Because the hard work is what happens behind that UI.'" While vibe coding replaced abstract wireframes and functional specification documents with tangible, interactive prototypes, business teams immediately hit a wall when attempting production deployments.

Because non-technical users build using unstructured prompts and divergent coding styles, the resulting artifacts are rarely production-ready for direct deployment into enterprise infrastructure. Zuora prohibits broad developer access to AWS production environments, which initially caused a massive influx of manual deployment tickets. To solve this friction, Gill’s team developed an internal service named **Vibe AI**—a structured pipeline accessible directly from a user's local project that routes vibe-coded applications into Git repositories and runs standardized deployment pipelines into AWS.

Gill admits that enterprise deployment requirements are shifting rapidly. After attending Tray Helix sessions earlier in the week, he drafted a project specification, spent a few hours iterating with Claude connected headlessly to Tray, and had a working project deployed to the Helix cloud that evening without needing manual Git interventions or deployment tickets.

- Vibe coding gives business stakeholders an immediate medium to express requirements, replacing lengthy business analyst (BA) requirement loops. - Prototypes created by vibe coding frequently mask underlying backend complexities, data wiring, and infrastructure gaps. - Zuora's custom "Vibe AI" deployment bridge requires users to pass code through Git to reach governed AWS runtimes, mitigating shadow deployments.

[11:18](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=678s) Evolving Roles, Prompt Design, and Enterprise Governance

The widespread use of AI coding tools is driving a major shift in enterprise technical roles. Noble points out that early buzzwords like "prompt engineer" have largely fallen out of favor. Gill argues that while general demand for certain rote programming positions is declining, new specialized roles centered on structured technical architecture are emerging. When users vibe code for hours without foundational architectural boundaries, they inevitably create a "crazy bowl of spaghetti that you might not find your way out of." Gill suggests that future engineering teams will require dedicated specialists—such as an "AI Solution Design Engineer"—whose sole mandate is properly structuring data context, prompt architecture, and component design before feeding tasks into an LLM.

```mermaid flowchart TD accTitle: Standardized Enterprise Vibe Coding Governance subgraph ClientEndpoints["Managed Developer Workstations"] Config["Pushed Configs: agents.md / claude.md"] LLM["AI Coding Assistants (Claude / Codex)"] Config -.->|"Injects internal standards"| LLM end LLM -->|"Generates Modular Code"| LocalApp["Standardized Local App"] LocalApp -->|"Targets Approved Endpoint"| VibePipe["Vibe AI / Helix Cloud Pipeline"] ```

To enforce consistency across local development environments without mandating rigid training courses, Gill leverages his client engineering team, which manages all Zuora employee workstations. The team programmatically deploys configuration markdown files (such as `agents.md` and `claude.md`) directly to local machine environments:

- Injected markdown configuration files supply AI models with baseline enterprise coding standards before the user begins typing prompts. - Standardization prevents models from generating monolithic 30,000-line TypeScript files, nudging them instead toward clean, modular microservice patterns. - Local AI instances are explicitly configured to recognize the internal deployment pipeline as the target environment, making generated code deployable out of the box.

[14:57](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=897s) Build vs. Buy Decisions and the 10x Paradigm

The rise of conversational AI coding has sparked debates over whether organizations should build custom software rather than purchase commercial off-the-shelf SaaS. While Gill dismisses extreme notions such as vibe coding an enterprise CRM or ERP from scratch—noting that core financial platforms and systems of record remain firmly off-limits—he highlights targeted opportunities for custom builds.

For instance, Zuora recently reevaluated its need for a dedicated third-party Contract Lifecycle Management (CLM) platform. Rather than purchasing complex enterprise software, Zuora recognized that contract workflows are modular, compartmentalized, and well-suited for bespoke internal development tailored exactly to their operational needs.

Gill frames this operational strategy around Zuora's internal "10x concept," which challenges teams to rethink standard workflows to achieve 2x, 5x, or 10x leverage through AI. However, he cautions that rapid application generation creates downstream support bottlenecks. Accelerating the deployment of 20 additional custom micro-apps does not scale the capacity of the human service management and IT support teams responsible for debugging and maintaining them. Gill emphasizes that applying AI to support these generated applications hits diminishing returns, as human personnel are ultimately required when production issues arise.

[17:45](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=1065s) Lessons Learned: The Terrors of Vibe Coding and the Unsolved Problem of Agent Identity

With roughly two minutes left on the clock, Josh Noble prompts Mark Gill to share a parting lesson or a cautionary tale from the trenches: "Maybe like a horror story that you wish other folks... here's something to avoid as your organization" builds out its AI footprint. Gill laughs, admitting that several emerging patterns keep him awake at night. At the very top of his list is the rapid rise of unconstrained "vibe coding," where developers rely on coding agents that autonomously reach out to the public internet and download whatever package or dependency they hallucinate or deem relevant to build a feature. To an infrastructure and security leader, this practice is genuinely terrifying: the very first time an agent blindly pulls in a malicious, malware-infected package, enterprise integrity is immediately compromised. Gill notes that Zuora has already caught a few isolated incidents where employees experimented in this way, though thankfully their internal compartmentalization controls were robust enough to catch and contain the rogue packages before damage could be done.

```mermaid flowchart TD accTitle: Enterprise Agent Identity and Kill Switch Flow subgraph AgentIAM["Agent Identity & Access Management"] A["Agentic System / Worker"] --> B["Distinct Worker Identity (Not a generic service account)"] B --> C["Permissions & Scope (What is it allowed to do?)"] B --> D["Audit Trail (What has it done?)"] end subgraph ControlPlane["Governance & Security Gateway"] C --> E["MCP Gateway"] D --> E E --> F["Enterprise Systems & Data Access"] G["Security Operations / Kill Switch"] -.->|"Instant Revocation"| E end ```

Beyond the immediate threat of vibe coding, Gill highlights what he views as a deeper, fundamental architectural challenge that the entire industry has yet to solve: agent identity. Throughout Zuora's deployments, Gill has repeatedly emphasized that systems striving to act autonomously need clear identities, exactly like a human employee in the organizational chart. Organizations cannot simply sprinkle generic service accounts across every system. Instead, every agentic compartment needs a designated worker identity where security teams can clearly audit what the solution is authorized to do and track every specific action it has executed. Having this discrete identity is the only way to establish an effective enterprise kill switch: if an agent behaves erratically or acts maliciously, security teams must be able to instantly disable that identity and cut off its underlying data access. While emerging architectures like an Model Context Protocol (MCP) gateway point toward viable remedies, Gill candidly notes that industry-wide, "we have not cracked that nut yet."

- **The vibe-coding vulnerability**: Developers allowing AI agents to source and inject unvetted third-party packages from the web creates an acute supply-chain vector for malware. - **Service accounts vs. agent identities**: Treating agents as traditional system service accounts obscures accountability; agents require distinct worker-level identities and granular audit trails. - **The operational kill switch**: Robust identity controls are vital so that rogue or compromised agents can be shut down and severed from enterprise data instantly via the gateway layer.

---

[19:27](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=1167s) Wrapping Up and Connecting at the Tray.ai Booth

As the session draws to a close, Josh Noble thanks the audience for attending and expresses hope that the audience was able to take away a few practical, actionable nuggets from Zuora's enterprise rollout. With the formal presentation finished, both Noble and Mark Gill offer to linger in the hallway immediately outside the room to answer follow-up questions and continue the discussion with attendees who want to talk through specific implementation challenges.

Noble also invites the crowd down into the main conference exhibit hall to see live demonstrations of the Tray.ai platform. He provides walking directions, directing attendees straight to the right on the expo floor, just past the Dell booth. For teams looking to inspect production agent workflows up close or walk through custom, personalized enterprise use cases, Noble offers to dive into the technical weeds directly at the booth or schedule deep-dive strategy sessions for the following week.

## Notable quotes > [00:04](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=4s) the coolest job out there — Josh Noble

> [00:04](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=4s) squirreliest solutions to their hardest problems with bubble gum and duct tape — Josh Noble

> [07:23](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=443s) what kills me is the first thing it gives anybody trying to vibe code is this big beautiful UI. And they look at it and they go, 'I'm 80% done.' And you go, 'Actually, you're not. You're about 10% done. Because the hard work is what happens behind that UI.' — Mark Gill

> [11:18](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=678s) crazy bowl of spaghetti that you might not find your way out of — Mark Gill

> [17:45](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=1065s) Maybe like a horror story that you wish other folks... here's something to avoid as your organization — Josh Noble

> [17:45](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=1065s) we have not cracked that nut yet — Mark Gill

## Mentioned Josh Noble, Mark Gill, Tray.ai, Zuora, Ai4, Tray Helix, Claude, Codex, TypeScript, MuleSoft, Workato, AWS Lambda, Salesforce, Apex, Slack, Sales Agent, Vibe AI, AWS, Git, agents.md, claude.md, Dell, Model Context Protocol (MCP)

## Video description Josh Noble — Field CTO, Tray.ai Mark Gill — Head of Infrastructure and Services, Zuora

How do you get AI past the pilot and into daily use — without losing control of governance, risk, or cost? Join me, Mark Gill, Head of Infrastructure and Services at Zuora, on stage with Josh Noble, Field CTO, Tray.ai. I’ve delivered dozens of agents, integrations, and MCP services in production. I’ll share how my team is driving real AI adoption: what we govern, how we deploy, and where the results actually come from. I’ll take you through how to ensure governance and reduce risk as you scale, how to adopt MCP and ship agents that get used, and where to find the lowest-hanging fruit that drives business results. It’ll be a great conversation between Josh and me – join us for practical insights on taking AI into production.

[Source](https://www.youtube.com/watch?v=KSUQ4nh7H9Y)

中文简报

Tray.ai 演讲：Zuora 如何在企业内全面部署智能体 AI Josh Noble 与 Mark Gill 深入剖析了 Zuora 经历的企业 AI 落地三个阶段，重点介绍了其基于 Slack 连接多数据源的“销售智能体（Sales Agent）”等生产级突破，同时也指出了员工“氛围编码（vibe coding）”带来的运维摩擦。为在快速原型开发中保持把控，Zuora 实施了自动化工作站配置文件（如 agents.md）以及专门部署至 AWS 的“Vibe AI” Git 交付流水线。尽管定制构建在合同管理等模块化领域带来了敏捷性，但 Gill 警告称，自动生成的微型应用可能会迅速让下游的 IT 人工支持团队不堪重负。演讲最后对未经审查的 AI 代码包下载所带来的软件供应链风险提出了紧迫警示，并强调整个行业都需要配备操作级紧急熔断开关（kill switch）且独立、可审计的智能体身份体系。 Tray.ai 已从数据集成连接器和确定性编排工作流，演进为自主智能体工具操作，并通过 Tray Helix 实现了全栈 TypeScript UI 生成。 (0:04) Zuora 将 MuleSoft 和 Workato 等碎片化工具整合到 Tray，经历了三个不同的 AI 采纳阶段：初步探索、供应商嵌入式 AI 以及桌面端氛围编码。 (3:12) Zuora 的内部“销售智能体”通过将 Slack 连接至 15 个不同的后端系统，实现了高管客户简报的自动化，将原本需要一周的人工准备时间缩短至数秒。 (3:12) 企业级智能体必须遵循用户上下文与基于角色的权限控制，根据请求者的具体访问权限动态调整回复内容。 (3:12) 氛围编码通过快速生成精美 UI 营造出接近完工的假象，但实际投产的大部分工作量仍集中在后端集成和基础设施层面。 (7:23)

Reading summary

At the Ai4 conference, Zuora's Mark Gill and Tray.ai's Josh Noble demonstrated that while agentic AI and conversational coding accelerate internal enterprise workflows, production viability depends on rigorous integration layers, structured developer guardrails, and unresolved models of agent identity. Consolidating Integration Stacks for Contextual Agents (3:12) Zuora transitioned from a fragmented integration footprint—combining MuleSoft, Workato, AWS Lambda, and Salesforce Apex—to Tray.ai, progressing through three AI adoption phases: initial ticket deflection experiments, vendor-embedded AI alongside internal workflow tools, and desktop AI coding. A primary milestone was Zuora's internal 'Sales Agent,' which automated an executive briefing process that previously required up to three staff members working for a week across siloed databases. By routing an LLM through Slack across 15 enterprise data sources via Tray, the Sales Agent compiles synthesized customer briefs within seconds. Crucially, the architecture incorporates enterprise role-based permissions: queries for the same account yield different data outputs for the Chief Revenue Officer versus the Chief Financial Officer based on each user's authenticated data access rights. Governing Local Vibe Coding and Deployments (7:23) The widespread adoption of AI coding assistants like Claude and Codex introduced what Gill described as an 'illusion of completion,' where users mistake an automatically generated user interface for an 80% complete application, despite lacking necessary backend wiring, data connectors, and production infrastructure. Because non-technical staff generate unstructured code and lack production AWS access, Zuora introduced 'Vibe AI,' an internal pipeline that channels locally authored code into Git repositories and governed AWS deployment paths. To prevent unconstrained coding from generating monolithic 30,000-line TypeScript files, Gill’s workstation engineering team programmatically distributes configuration markdown files (such as agents.md and claude.md) to employee laptops. These files inject baseline enterprise standards and architectural patterns directly into the models before prompting begins, ensuring the generated applications are modular and preconfigured to target approved internal deployment pipelines. Selective Software Development and Maintenance Limits (14:57) While Gill ruled out using generative coding to replace foundational enterprise platforms like ERPs or CRMs, Zuora utilizes AI to bypass third-party software for modular, compartmentalized workflows. For example, rather than purchasing a commercial Contract Lifecycle Management (CLM) system, Zuora developed bespoke contract tools tailored to its internal operating needs under an operational framework aimed at achieving 2x to 10x leverage. However, Gill warned that rapid application creation introduces critical downstream support bottlenecks. Generating dozens of custom micro-applications outpaces the operational capacity of human IT service management teams tasked with troubleshooting and maintaining them when production issues arise, highlighting the diminishing returns of relying on AI to support newly generated code. Unvetted Dependencies and Unresolved Agent Identity (17:45) Unconstrained coding agents present acute enterprise security risks when they autonomously download hallucinated or unvetted third-party packages from the public internet, creating an immediate software supply-chain malware vector. While internal isolation controls contained early occurrences at Zuora, Gill identified an even more critical industry-wide architectural deficiency: the absence of standardized agent identity frameworks. Treating autonomous software agents as generic service accounts eliminates granular operational accountability. Gill maintained that autonomous agents require discrete worker identities—analogous to employee accounts—complete with explicitly audited permission scopes and instant kill switches routed through governance gateways, such as Model Context Protocol (MCP) gateways, to sever data access if an agent behaves erratically. Gill and Noble concluded that while agentic orchestration and conversational development drastically reduce prototyping friction, enterprise scale ultimately hinges on establishing disciplined backend guardrails, manageable long-term maintenance models, and standardized frameworks for autonomous agent identity.

精炼阅读摘要

Zuora与Tray.ai在Ai4会议上探讨了企业级智能体AI的落地路径与治理实践，指出AI应用落地的核心障碍不在于前端原型生成，而在于企业身份权限控制、标准化合规部署流水线及后端运维承载等系统性工程挑战。企业AI演化路径与权限感知智能体 (3:12) Zuora基础设施与服务负责人Mark Gill回顾了公司的三阶段AI演进过程：三年前Zuora曾聘请外部顾问构建IT工单拦截系统，但因缺乏数据质量、业务上下文与治理护栏而在生产环境中失效；随后进入供应商原生AI与内部智能体结合的探索期；目前则演化至员工自主使用AI编码工具的“氛围编程”（Vibe Coding）阶段。在第二阶段中，Zuora利用Tray.ai将大语言模型与Slack界面接入15个后端数据源，开发出内部“销售智能体”（Sales Agent）。该智能体改变了此前高管拜访客户前需多名员工耗费一周手动检索各系统生成简报的做法，高管只需在Slack输入请求，即可在数秒内获得提炼后的客户简报。Gill强调，企业级身份与基于角色的访问控制（RBAC）在其中起到决定性作用：当首席财务官与首席营收官查询同一客户时，智能体会严格继承请求者各自的企业权限，生成具有差异化数据范围的定制报告，从而杜绝敏感数据越权暴露。氛围编程的落差与标准部署流水线 (7:23) 随着员工广泛使用Claude和Codex等编码助手，企业内部软件开发模式发生了剧烈变化。然而Gill指出，这类工具带来了“完成度的假象”：AI迅速生成的直观界面常让业务人员误以为项目已完成80%，但由于缺乏后端逻辑和基础设施连接，真实进度实际上仅约10%。此外，非技术人员生成的松散代码难以直接部署到受控生产环境，导致IT部门收到了大量繁琐的手动部署工单。为解决交付摩擦并防止不受监管的影子部署，Zuora工程团队开发了名为“Vibe AI”的内部服务桥梁。该管道强制要求本地生成的应用代码接入Git仓库，并通过标准化的自动化流水线部署至AWS环境。与此同时，Tray.ai推出的Tray Helix开发层也支持利用大模型生成全栈TypeScript前端，并在托管云环境中直接部署，大幅压缩了从业务构想到实际上线的时间周期。端点配置下发与架构设计角色转型 (11:18) Tray.ai现场CTO Josh Noble指出，“提示词工程师”这一概念已逐步降温。Gill进一步认为，虽然基础性重复编程岗位的需求在下降，但专注于系统性架构设计的新角色正在涌现。如果缺乏严谨的前期架构设计，长时间的无序AI编程极易产生无法维护的混乱代码。因此，工程团队未来需要引入类似“AI解决方案设计工程师”的岗位，其核心职责是在向大模型输入任务前，先构建好数据上下文、提示词架构和模块化边界。为在全公司范围内推行统一标准且避免冗长的员工培训，Zuora的客户端工程团队直接向全体员工工作站集中下发`agents.md`和`claude.md`等配置文件。这些文件能够在用户输入提示词前自动向模型注入企业编码规范，引导模型生成清晰的微服务模块化架构而非臃肿的单体代码，并默认将内部合规部署流水线设为目标环境，使生成的代码能够开箱即用。自建采购权衡与运维支持承载瓶颈 (14:57) AI编码工具的普及引发了企业软件究竟应“自主构建”还是“外部采购”的讨论。Gill明确表示，ERP、CRM等核心财务系统与权威记录系统绝不能通过AI从零构建；但在合同生命周期管理（CLM）等模块化、业务高度定制且独立的领域，企业完全有能力依靠内部AI能力进行定制自建，从而替代复杂的第三方SaaS平台。Zuora内部推行了“10倍概念”（10x concept），鼓励团队通过AI寻找实现2倍、5倍乃至10倍效能杠杆的工作流。然而Gill提醒，快速生成应用将给下游支持团队带来严峻挑战。AI生成20个微型应用的速度虽然极快，但负责排查故障与日常维护的IT服务管理团队承载力并不能同步扩张。他强调，在应用维护和线上排错阶段，AI的增效能力存在明显的边际递减效应，生产故障发生时最终仍高度依赖人类专业人员介入排查。依赖供应链漏洞与智能体身份挑战 (17:45) 谈及AI落地过程中的安全隐患，Gill指出最令其担忧的是无约束的AI编码工具可能会自主从互联网下载虚构或未经审核的第三方依赖包，从而直接引入恶意软件并破坏企业安全防线。Zuora虽已通过内部隔离机制拦截了数起员工此类操作，但这仍是高危的供应链安全攻击向量。此外，Gill指出整个行业尚未解决的根本性架构难题是“智能体身份”（Agent Identity）。在企业系统中，自治智能体不能简单使用宽泛的通用服务账户，而必须拥有类似人类员工的独立工作者身份，并严格绑定授权范围与审计日志。只有建立精细的身份层，企业安全运营团队才能在智能体出现异常行为时，通过模型上下文协议（MCP）网关等控制平面一键触发“终止开关”（Kill Switch），彻底切断其数据访问权限。 Mark Gill认为，尽管模型上下文协议等新兴网关架构为安全防护指明了方向，但整个行业在建立完善的智能体身份体系和即时阻断机制上仍处于探索阶段；企业在享受AI快速原型生成红利的同时，必须将架构边界治理、第三方供应链安全审查和下游IT运维承载力置于同等优先级。

Detailed notes — zh

## Background 该视频由 Ai4 于 2026 年 8 月 22 日发布在 YouTube 平台。内容记录了 Tray.ai 现场 CTO Josh Noble 与 Zuora 基础设施和服务负责人 Mark Gill 的联合演讲，深入探讨 Zuora 如何借助 Tray.ai 将 Agentic AI 从试点推向日常生产环境，并平衡好治理、风险与成本。 ## TL;DR Zuora 分享了从早期工单分流试错、打造标杆 Sales Agent 到全民“氛围编码”（Vibe Coding）的三阶段 AI 演进历程，展示了如何通过 Tray Helix 解决本地开发到云端部署的交付断层。Mark 指出企业岗位正从提示词工程转向严谨的 AI 方案设计，并提出了警惕下游运维人力瓶颈与自研外购边界的理性思考。针对生产落地中的核心风险，演讲深度剖析了自主智能体随意拉取外部依赖带来的供应链安全威胁，并强调必须将智能体作为独立员工纳管身份权限，配备紧急熔断机制（Kill Switch）。 ## Atmosphere & style 分享整体呈现出务实、犀利且极具工程实战感的技术交流氛围，Josh 的风趣主持与 Mark 的硬核坦诚形成了自然的对话张力。Mark 毫不避讳地揭露“氛围编码”背后仅完成 10% 的工程真相与夜不能寐的依赖投毒风险，打破了对大模型的盲目狂热，整场演讲在冷静克制的反思与前沿探索的兴奋感中给听众带来极高的信息密度。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=4s) Tray.ai 完成技术闭环演进：从异构系统连接器和确定性工作流，升级为 AI Agent 工具库，并推出 Tray Helix 实现自然语言直接生成前端交互与安全部署。 - [03:06](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=186s) Zuora 总结 AI 落地三阶段：经历外部咨询工单分流的摸黑试错、Sales Agent 标杆应用的初见成效，最终走向全员 Vibe Coding 与工程治理阶段。 - [03:06](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=186s) 早期试错沉淀的核心认知：单纯依赖 LLM 进行工单分流极易失败，高质量的企业数据、精确的上下文边界（Context）与安全防护栏（Guardrails）才是落地的先决条件。 - [03:06](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=186s) 企业级身份（Identity）至关重要：Zuora 的 Sales Agent 连接了 15 个内部系统，相同的提示词因发起人（如 CFO 与 CRO）身份及数据权限不同而生成完全不同的业务简报。 - [07:06](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=426s) 警惕 Vibe Coding 的交付幻觉：AI 生成精美 UI 容易给人“已完成 80%”的错觉，但底层复杂的数据交互和业务逻辑开发意味着实际仅完成了约 10%。 - [07:06](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=426s) 缩短交付周期：借助 Tray Helix 与无头连接能力，开发者仅凭一份 Spec 规范文档就能在数小时内生成并一键部署云端预览，绕过冗长复杂的部署流水线工单。 - [11:18](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=678s) 组织技能重构：单纯的“提示词工程师”价值下降，业界更需要能够在编码前完成严谨系统架构与上下文定义的“AI 方案设计工程师”，以避免产生不可维护的“代码意面”。 - [11:18](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=678s) 通过标准化规则约束代码生成：在开发环境中集中注入配置文件，强制大模型按模块化或微服务结构拆分代码，防止生成数万行的超大单体文件并天然适配部署标准。 - [14:57](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=897s) 自研与外购（Build vs. Buy）的清晰界限：涉及财务底座的核心大型系统（如 ERP）坚持外购，而高度模块化且现有市场方案难以精准匹配的场景（如 CLM 合同生命周期管理）则是内部自研的理想切入点。 - [14:57](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=897s) 下游运维面临边际效应递减挑战：AI 虽然极大地加速了应用构建，但运维支持与排障能力难以同步成倍扩张，过度依赖 AI 运维会出现边际效益递减，仍需警惕运维团队的承载瓶颈。 - [17:45](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=1065s) 第三方依赖与供应链安全隐患：智能体自主编码时可能直接从公网下载未经审核的依赖包，存在引入恶意软件与供应链投毒的风险，必须依靠严格的环境隔离仓进行拦截防御。 - [18:40](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=1120s) 智能体身份治理与紧急熔断（Kill Switch）：智能体不能被当作普通服务账号对待，必须具备类似正式员工的独立纳管身份、权限审计与追踪能力，并支持在失控时瞬间吊销访问权限。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=4s) 两位讲者登场与 Tray.ai 的产品演进在 AI4 大会的第三天，Tray.ai 现场首席技术官（Field CTO）Josh Noble 与 Zuora 基础设施、运营与安全负责人 Mark Gill 共同登台。Josh 幽默地形容自己的工作是“全天下最酷的差事”——置身于前沿的“臭鼬工场”（skunk works）实验端，与客户一同用最灵活的手段（甚至像“口香糖与强力胶带”一样就地取材）攻关最棘手、最怪异的技术挑战。Mark 则负责 Zuora 内部包括基础设施、IT 服务管理（ITSM）及安全合规在内的全套技术底座。 Josh 梳理了 Tray.ai 历经的数次重大演进：最初它是一个连接器框架（Connector Framework），用于在异构系统间实现数据的读写与双向同步；随后演进为支持在这些系统间编排确定性的业务流程（Workflows）；随着大模型时代的到来，这些既有的工作流被直接封装为 AI Agent 可调用的工具库（Tools）；而在本周，Tray 正式推出了 Tray Helix，允许用户借助 Claude 或 Codex 等模型直接生成 TypeScript 前端应用，并在企业级安全合规治理下快速部署分发，实现了从后端数据流到前端交互界面的端到端闭环。 ```mermaid flowchart LR accTitle: Tray.ai 技术架构演进路径 subgraph S1["数据与流程底座"] A["连接器框架 系统数据读写"] --> B["确定性工作流 业务流程编排"] end subgraph S2["智能化与前端交付"] B --> C["AI Agent 工具 调用工作流执行"] C --> D["Tray Helix 自然语言生成前端"] end ``` --- ### [03:06](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=186s) Zuora 的 AI 落地三阶段与 Sales Agent 的破局 Zuora 与 Tray 的合作始于四年前的基础设施整合。当时 Zuora 的集成架构极其分散混乱，同时运行着 MuleSoft、Workato，以及大量 AWS Lambda 代码和 Salesforce 中的 Apex 脚本。Zuora 最终将集成层全量收敛至 Tray。回顾 Zuora 过去三年的 AI 探索，Mark 将其划分为三个明确阶段： * **阶段一（摸黑试错）：** 团队在黑暗中摸索，曾聘请外部咨询团队试图用 LLM 实现客服工单分流（Ticket Deflection）。尽管该项目最终宣告失败且已被废弃，但它让团队沉淀了核心认知——数据质量、上下文边界（Context）与防护栏（Guardrails）至关重要。 * **阶段二（初见成效）：** 厂商软件开始原生集成 AI 能力（如 ITSM 工单系统实现真正的人效分流），同时 Tray 开始支持在工作流中调用大语言模型。Zuora 在此阶段诞生了首个标杆应用——**Sales Agent**。 * **阶段三（全面提效）：** 进入全民“Vibe Coding”（氛围编码）时代，业务人员普遍在本地使用 AI 编码工具，但随之带来了身份认证与部署治理的新挑战。 Sales Agent 彻底改变了高管客户拜访的筹备模式。在过去，如果首席财务官（CFO）计划下周拜访客户 A，其团队往往需要抽调三名员工耗费整整一周，从散落各处的系统中手工搜集数据并汇总成一份简报文档。Zuora 借助 Tray 连接了 15 个不同的内部数据源并打通 Slack。当高管在 Slack 频道提出简报需求时，系统自动识别关联数据源、抽取数据并经由 LLM 提炼生成客户简报。这一实践让团队深刻意识到了**身份认证（Identity）**的重要性：当 CFO 和首席营收官（CRO）输入完全相同的指令时，生成的简报内容截然不同。这是因为请求绑定了发起人的企业身份，而不同身份在底层系统中的数据访问权限存在严格区隔。 --- ### [07:06](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=426s) Vibe Coding 时代的交付鸿沟与架构突围大模型工具的普惠让企业内人人都想成为开发者，但这也带来了新的工程痛点。Mark 犀利地指出，Claude 或 Codex 等工具为用户生成惊艳的 UI 界面后，业务人员往往会产生“我已经完成了 80%”的错觉，但实际上“你才完成了大约 10%”，因为系统底层复杂的数据交互和业务逻辑才是真正的大头。为了解决业务人员在本地写完代码却无法部署到 AWS 的困境（Zuora 严格禁止全员直连云基础设施），内部团队开发了名为 **Vibe AI** 的自动化部署流水线，引导用户将本地代码提交至 Git 并触发 AWS 部署流程。然而，这种模式仍需平台团队大量的支持与介入。 ```mermaid flowchart TD accTitle: Zuora 代码交付流程对比 subgraph VibeAI["传统/过渡方案：Vibe AI 流水线"] V1["本地 Vibe Coding"] --> V2["提交至 Git 仓库"] V2 --> V3["手动提单与审核"] V3 --> V4["部署至 AWS 环境"] end subgraph Helix["新型方案：Tray Helix 即时交付"] H1["编写 Spec 需求规范"] --> H2["Claude + Headless Tray"] H2 --> H3["一键部署至 Helix Cloud 预览"] end ``` 这种交付痛点在引入 Tray Helix 后迎来了转机。Mark 分享了他在大会期间的亲身体验：基于周二参会的构想，他仅编写了一份 Spec 规范文档，通过 Claude 结合 Tray 的 Headless 无头连接能力进行构建，短短两小时内就直接将应用部署到了 Helix 云端，甚至无需中途提交 Git 或走工单审批，便在开发测试环境中看到了近乎实时的运行效果。 --- ### [11:18](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=678s) 组织技能重构与代码规范标准化随着 AI 编码工具的普及，企业所需的岗位职能正在发生剧烈震荡。“提示词工程师”（Prompt Engineer）这类曾被热炒的头衔已逐渐降温，取而代之的是更深层次的系统设计需求。Mark 强调，如果缺乏系统化的架构设计和上下文定义，漫无目的地 Vibe Coding 四个小时，最终只会产出一团不可维护的“代码意面”（spaghetti code）。业界未来急需的是能够熟练驾驭工具、在编码前完成严谨架构设计的“AI 方案设计工程师”（Solution Design Engineer for AI）。在落地治理方面，Mark 的团队（同时负责全公司的客户端工程）通过集中下发统一的规则配置文件（如 `agents.mmd` 或 `claude.mmd` / Markdown 指南）实现标准化治理： - **防止单体代码膨胀：** 防止大模型在开箱状态下直接生成包含三万行代码的庞大 TypeScript 单体文件，强制要求其按微服务或模块化结构进行拆分。 - **预设部署目标：** 在用户打开本地开发环境时，环境自动加载标准架构规范，使 AI 生成的代码天然适配企业的部署流水线标准，大幅降低不可交付代码的比例。 --- ### [14:57](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=897s) 自研与外购边界（Build vs. Buy）及 10x 运维挑战在生成式 AI 降低开发门槛后，企业内部时常出现“为什么不直接 Vibe Code 一个 CRM”的论调。Mark 对此给出了清醒的界定：涉及企业财务底座的大型复杂系统（如 ERP）在相当长时间内仍需坚持外购；而那些流程相对独立、模块化程度高且现有市场方案难以精准匹配的场景，则是自研的黄金切入点。 - **合同生命周期管理（CLM）：** Zuora 当前正处于评估 CLM 的业务体量，因其具备高度模块化的工作流特征，团队开始尝试通过内部自研来完全定制所需功能。 - **“10x”效率理念：** Zuora 在内部推行 10x 思考模式，鼓励团队借助新技术重构业务逻辑，探寻 2 倍、5 倍乃至 10 倍的生产力提升机会。 - **运维支持的边际效应递减：** 尽管 AI 极大加快了应用的构建速度，但下游运维与 ITSM 服务团队的人力并没有成倍扩张。如果平台短时间内激增 20 款新应用，纯靠 AI 进行系统运维和答疑很快会遭遇边际效益递减，最终仍需要人类专家介入排障，企业必须警惕应用激增带来的运维支撑滞后。 ### [17:45](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=1065s) 智能体落地的“恐怖故事”：Vibe Coding 与第三方依赖风险在演讲尾声仅剩两分钟时，主持人 Josh Noble 询问 Mark Gill 是否有痛彻心扉的教训，或者在实际推进过程中遭遇过的“恐怖故事”（horror story），以便给正在进行 AI 探索的企业敲响警钟。Mark Gill 直言不讳地指出，最让他夜不能寐（"keep me awake"）的事情之一就是所谓的“氛围编码”（vibe coding）现象。在员工或开发者使用 AI 智能体自主编写代码时，智能体会直接连接互联网，凭自身判断拉取它认为与当前解决方案相关的任何第三方软件包。Mark Gill 强调这种行为极其可怕（"horrifying"），因为只要智能体拉取的开源包中潜藏了恶意软件或存在供应链投毒，整个企业系统就会立刻暴露在安全威胁之下。所幸的是，Zuora 依靠内部严密的空间隔离机制（compartmentalized），在发现少数员工出现类似尝试时及时将其拦截在了防线之内，避免了演变为真正的灾难。 - **依赖管控风险**：自主智能体往往会根据上下文意图直接从公网下载未经审核的依赖包，存在引入恶意代码和供应链投毒的极高安全隐患。 - **隔离防御策略**：通过构建环境与数据隔离仓，企业能够在恶意或未经审核的组件扩散至生产核心前实施拦截。 --- ### [18:40](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=1120s) 尚未攻克的身份治理难题：智能体不是普通的服务账号除了代码生成的安全隐患外，Mark Gill 指出当前企业级落地中最严峻的现实挑战在于“智能体身份认证与访问控制”（Identity and Access Management, IAM）。在复杂的企业架构中，每一个试图具备自主行动能力的系统或隔离模块，都需要拥有自身专属的合法身份，这种身份必须像组织中的真实员工一样被纳管，而不是简单粗暴地随意创建无名服务账号（"I don't want service accounts everywhere"）。企业必须清晰掌握该智能体究竟代表谁、被赋予了哪些具体权限、以及它究竟执行了哪些操作。拥有独立且透明的身份体系至关重要，因为一旦智能体失控或发生异常，管理员必须拥有切断一切的“自杀开关”（kill switch）——能够瞬间吊销其身份凭证并阻断其数据访问通道。尽管业内已经出现了 MCP 网关（MCP gateway）等新兴治理手段，但在庞大的企业级生态中真正打通智能体身份体系这颗“硬核桃”，依然是尚未彻底攻克的治理瓶颈。 ```mermaid flowchart TD accTitle: Agent Identity and Governance Architecture subgraph Execution["智能体执行与网关层"] A["Agentic 解决方案"] --> B["MCP Gateway / 治理网关"] end subgraph IAM["企业身份纳管层"] B --> C["独立数字身份 (Worker Identity)"] C --> D{"行为审计与权限校验"} D -->|授权通过| E["企业数据与业务系统"] end subgraph Control["应急响应与熔断"] F["Kill Switch (紧急熔断开关)"] -.->|瞬间吊销身份与访问| C end ``` - **实体化员工视角**：智能体不能被当作黑盒后台服务对待，必须具备类似于正式员工的权限范围、归属审查与生命周期管理。 - **治理与审计追踪**：明确智能体“被允许做什么”与“实际做了什么”，提供精确的可观测性与审计日志。 - **紧急熔断能力**：通过与 MCP 网关联动，一旦出现异常行为即可通过 Kill Switch 快速禁用该身份并切断数据流通。 --- ### [19:27](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=1167s) 现场总结与展位互动邀约 Josh Noble 对 Mark Gill 的坦诚分享表达了感谢，并向在场听众作结语致辞，希望大家能从 Zuora 的实战案例中汲取到极具价值的干货经验（"a few nuggets of takeaway"）。由于演讲时间已到，Josh 与 Mark 表示会在演讲结束后留在会场过道（hallway）稍作停留，欢迎有深入探讨意愿的参会者前来面对面交流。同时，Josh 邀请大家前往展厅中的 Tray.ai 展位——位置就在展厅右侧穿过戴尔（Dell）展位不远处。无论是想现场观摩 Tray.ai 与 Zuora 合作落地方案的实际演示，还是希望结合各家企业自身的业务场景进行个性化梳理与深研，团队都非常欢迎在现场或下周安排专项沟通。最后两位讲者在全场掌声中互相致谢，结束了本次演讲分享。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=4s) 全天下最酷的差事——置身于前沿的“臭鼬工场”（skunk works）实验端，与客户一同用最灵活的手段（甚至像“口香糖与强力胶带”一样就地取材）攻关最棘手、最怪异的技术挑战。 — Josh Noble > [03:06](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=186s) 数据质量、上下文边界（Context）与防护栏（Guardrails）至关重要。 — Mark Gill > [07:06](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=426s) 我已经完成了 80% — Mark Gill > [07:06](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=426s) 你才完成了大约 10% — Mark Gill > [11:18](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=678s) 如果缺乏系统化的架构设计和上下文定义，漫无目的地 Vibe Coding 四个小时，最终只会产出一团不可维护的“代码意面”（spaghetti code）。 — Mark Gill > [14:57](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=897s) 为什么不直接 Vibe Code 一个 CRM — Mark Gill > [17:45](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=1065s) keep me awake — Mark Gill > [17:45](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=1065s) horrifying — Mark Gill > [18:40](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=1120s) I don't want service accounts everywhere — Mark Gill > [19:27](https://www.youtube.com/watch?v=KSUQ4nh7H9Y&t=1167s) a few nuggets of takeaway — Josh Noble ## Mentioned Tray.ai, Zuora, Josh Noble, Mark Gill, Tray Helix, Claude, Codex, MuleSoft, Workato, AWS Lambda, Salesforce, Apex, Slack, Vibe AI, Git, AWS, MCP Gateway, Dell ## Video description Josh Noble — Field CTO, Tray.ai Mark Gill — Head of Infrastructure and Services, Zuora How do you get AI past the pilot and into daily use — without losing control of governance, risk, or cost? Join me, Mark Gill, Head of Infrastructure and Services at Zuora, on stage with Josh Noble, Field CTO, Tray.ai. I’ve delivered dozens of agents, integrations, and MCP services in production. I’ll share how my team is driving real AI adoption: what we govern, how we deploy, and where the results actually come from. I’ll take you through how to ensure governance and reduce risk as you scale, how to adopt MCP and ship agents that get used, and where to find the lowest-hanging fruit that drives business results. It’ll be a great conversation between Josh and me – join us for practical insights on taking AI into production. [Source](https://www.youtube.com/watch?v=KSUQ4nh7H9Y)

Source quotes

[0:04] Josh Noble: 全天下最酷的差事——置身于前沿的“臭鼬工场”（skunk works）实验端，与客户一同用最灵活的手段（甚至像“口香糖与强力胶带”一样就地取材）攻关最棘手、最怪异的技术挑战。 [3:06] Mark Gill: 数据质量、上下文边界（Context）与防护栏（Guardrails）至关重要。 [7:06] Mark Gill: 我已经完成了 80% [7:06] Mark Gill: 你才完成了大约 10% [11:18] Mark Gill: 如果缺乏系统化的架构设计和上下文定义，漫无目的地 Vibe Coding 四个小时，最终只会产出一团不可维护的“代码意面”（spaghetti code）。 [14:57] Mark Gill: 为什么不直接 Vibe Code 一个 CRM [17:45] Mark Gill: keep me awake [17:45] Mark Gill: horrifying [18:40] Mark Gill: I don't want service accounts everywhere [19:27] Josh Noble: a few nuggets of takeaway

Source quotes

[0:04] Josh Noble: the coolest job out there [0:04] Josh Noble: squirreliest solutions to their hardest problems with bubble gum and duct tape [7:23] Mark Gill: what kills me is the first thing it gives anybody trying to vibe code is this big beautiful UI. And they look at it and they go, 'I'm 80% done.' And you go, 'Actually, you're not. You're about 10% done. Because the hard work is what happens behind that UI.' [11:18] Mark Gill: crazy bowl of spaghetti that you might not find your way out of [17:45] Josh Noble: Maybe like a horror story that you wish other folks... here's something to avoid as your organization [17:45] Mark Gill: we have not cracked that nut yet

Original transcript

[0:04] appreciate everybody um joining us here. I guess uh for way of introductions, my name's Josh Noble [0:11] from from Trey. I'm Trey's field CTO and I think I probably have the coolest job out there. I I [0:16] get to work with our work with our our customers, kind of have a first seat into what we're doing [0:20] from a product development side of things and work with them to build their squirreliest solutions [0:25] to their hardest problems with bubble gum and duct tape a lot of the a lot of the time. So, get to be [0:29] in the skunk works experiment side of of things. So, um nice to chat with everybody today today. [0:34] Were you Mark? Hi everyone. Um thanks for taking 20 minutes out of your day. I hope uh day three of [0:40] AI4 is treating everybody well. Um my name is Mark Gil. I'm responsible for infrastructure operations [0:46] and security for a company called Zora. And uh I do extensive work with Trey and also take care of [0:53] everything infrastructure internally. Uh service management. if somebody wants to talk about ITSM, [0:57] I can do that and also the security aspects of all of this. So, we have a lot of AI going on [1:02] right now at Zora and we're going to talk about that today. So, thanks for being here. Well, [1:06] I I guess for to tee everybody up, we're just going to have a conversation. Not going to [1:10] demo demo stuff to you. If we want to do that, let's do something personalized. We'll set up [1:14] something after the fact. Let us know. Happy to chat in the hall with you directly about kind of [1:17] your use cases and what challenges you guys are running into directly. But want to get help you [1:22] guys get an idea of, you know, how Zora is using Trey. what are you guys seeing from a a day-to-day [1:27] perspective whether that's you know trade specific or just broadly across you know your your stack on [1:31] things so hopefully there's some kind of lessons learned from Mark who's you know been through the [1:35] journey and back um so for reference if anybody doesn't know what Trey is I think it's getting [1:41] harder and harder to go and explain exactly what we are at this point so maybe a little bit on the [1:45] evolution side of things so we started as really a connector framework so I want to be able to read [1:50] and write data to you know various different systems sync those those up um that evolved to [1:57] let's start to build business processes and you know in between those connections now that I have [2:02] those systems connected later on that starts to evolve into well I I've got these workflows I've [2:07] got these business processes that are mapped out that are these deterministic workflows [2:11] what if I can start to give that connection that access those workflows over to AI agents [2:16] to be able to use those workflows as AI tools to navigate through through their work. Um, and now, [2:23] uh, we we had a lot of customers that say, "Great, we've we've got all this automation, all this [2:26] backend stuff. I want to start to build apps. I want to have a front-end experience now because [2:30] I want customized dashboards or supplier portal or, you know, some sort of contract management [2:35] process. And now I want a UI for that." And so we've just this week uh launched Trey Helix, [2:40] which is the ability for you to use a a claude or a codeex to build uh app frontends for you, [2:48] TypeScript apps, and then easily be able to, you know, deploy those, share those [2:52] uh in a way that's secure, governed uh within your your organization. So um kind of endto-end [2:57] experience uh that you're able to give out to your folks to quickly iterate and rapidly uh you know, [3:02] build through their solutions. So I guess you know Mark, you've been on the journey for a while. [3:06] If you want to give everybody a bit of an idea of how are you using Trey? What are you what are [3:10] you doing with it with it today? Yeah. So our journey with Trey started gosh probably four [3:16] years ago when we were trying to standardize our um integration layer. And so Zora had an [3:21] instance of Mulesoft running. We had an instance of Wcado running and we were looking oh and we had [3:27] a bunch of lambda code and we had people running integrations out of Apex code and Salesforce and [3:32] it was very convoluted and messy. Uh so we went through uh the work to consolidate our integration [3:40] layer to Trey. Um and so we retired those other systems and we began running all integration [3:46] that we built and revised in Trey. And so our relationship with Trey was already there. And then [3:52] about three years ago, our AI journey started. And we were talking earlier, I think about our [3:58] journey in three phases, right? The first phase um was basically us in the dark poking at things, [4:05] trying to figure out what what things worked. And so we thought to ourselves, let's try to bring in [4:11] um a consultant team to help us use some LLMs to deflect tickets, right? Something very simple, [4:18] a very simple use case. It did not go well. But what we learned was a long list of things [4:24] about data matters, context matters, guard rails matter, um, and and the list goes on. And so we we [4:32] that solution that we built doesn't exist in our landscape anymore. But phase one taught us a bunch [4:37] of real core fundamentals. Phase two is where we actually started doing things that had hope, [4:43] right? And so phase two is where we saw vendors bringing AI solutions that we could leverage in [4:51] a way that actually made a difference. So service management, our ticketing system came to the table [4:57] with AI at the core and we actually were able to see true deflection using AI. But at the same [5:02] time, Trey also started enabling our ability to leverage LLMs and workflows. And so we our [5:09] first real success in our landscape was something called what we call sales agent. Um and actually [5:15] some of the yes Gore is here from the team. He remembers sales agent. Uh sales agent was our [5:20] first real use case that made a difference and got people excited. If you think of I'll I'll briefly [5:28] describe it. If you think of an executive at your company who is going to meet with a customer next [5:32] week. CFO says I'm going to meet with customer A. They go to their team and they say, "I need to [5:37] build a briefing for this customer." And three people on his team or her team spend a week [5:44] gathering data from all the different systems in our landscape and then have to somehow synthesize [5:48] that manually into some sort of document. And so we knew we had Trey, we knew we had connectors, [5:53] we knew we now had access to to language models. And so we took Trey and we connected it to 15 [5:59] different sources. We connected it up to Slack. And then when somebody when an executive went to [6:04] the Slack channel and said, "I need a customer brief for customer A," Slack looked at that [6:11] request, determined which of the connected systems had relevant data, grabbed that data, and then we [6:16] used an LLM to deliver a customer brief. It sounds super simple in the context of where we are today, [6:22] but it was the first real mechanism that we had where we actually saw something work and actually [6:26] make a small difference to to our senior leaders day. And so what did we learn about that? Well, [6:33] we learned the more complicated things like I need an identity for this stuff. I need a way [6:39] to like CFO would ask a question. CRO would ask the same question and their customer brief would [6:46] look different. Well, why did it look different? Because that request is using their identity in [6:51] our landscape and their identity has different access to data, right? And so then we started [6:56] processing. We're learning about data, identity, guardrails again. And so that was our first real [7:03] solution that Trey helped us work through. Now in phase three, we're in everybody's vibe coding. [7:10] Everybody's got access to the tools on their desktop. And now there's a new set of issues [7:14] and challenges that we have. Identity is still a problem. Data access is still a problem. And [7:19] these are the kinds of things that we're working through. So th those are our phases um that we've [7:23] gone through and Trey's been a part of that. Well, so on the on the vibe coding side of things, hey, [7:29] two years ago, you couldn't really use an agent to go build everything, build things for you. Now, [7:34] everybody wants to be a a developer um with varying degrees of of success. So, you know, [7:40] how has that changed how you folks around your organization are, you know, building or or you [7:45] know, who is the the point person? Are they still all coming to you? Are they trying to do things [7:49] off on their own in their own little silo? Yeah. So, we've given them the tools. I don't know if [7:54] anybody else has had this experience, but but Claude, Codeex, all of these vibe coding tools, [8:00] what kills me is the first thing it gives anybody trying to vibe code is this big beautiful UI. [8:05] And they look at it and they go, I'm 80% done. And you go, actually, you're not. You're about [8:10] 10% done. Because the hard work is what happens behind that UI, right? And so in our old world, [8:17] we were talking about wireframes. We were talking about documents trying to describe what people [8:23] wanted. The tools today give our business partners a better mechanism to show what they're thinking, [8:29] how they want things to work. But the ones that actually continue to build and continue to vibe [8:34] code, now they want it deployed, right? And deployment is hard because everybody viodes in [8:41] a different way. Their words are different. Their vibe code solution is different. And so we end up [8:45] with a pile of stuff. they want me to deploy this to AWS and you can't because it's not ready. It's [8:52] not formed that way. So, we've done a bunch of work um at Zora. We have something we're calling [8:56] Vibe AI. So, we've actually built a pipeline that's accessible through your vibe coded app [9:02] that you can actually request a deployment and it will ask you to pass your code into git and [9:08] then it will actually run a pipeline that deploys into AWS. And so this is something that we've lit [9:14] literally just enabled, but it still requires a lot of handholding. We're trying in in Zora, [9:21] you can build and run on your laptop, but nobody has access to AWS to deploy. So they still have to [9:26] come to us. But it's changed what what the people on our team have to do and what we have to work on [9:31] because now we have all these deployment requests that we didn't have 12 months ago. Well, from [9:35] like a spec side of things, that means folks can build out the prototype of what they're looking [9:40] for without the constant loop of having your BAS meet with them, put something together. No, that's [9:47] not quite what I want. Have to have to keep going through that iteration cycle. They can now say, [9:51] here's what I'm looking for. Now, I don't know all the wiring that h needs to happen behind the [9:55] scenes to make this function, but you know, this is it. Um, inspe it. Um, where was I going going [10:03] with that? Um well I I so I would say actually um if any of you guys have seen the the tray counter [10:11] or the the the tray table downstairs the um the the helix mechanisms that they've deployed you [10:18] announced this week. Yeah. Mhm. Um, I'm sad we did the work to do the pipeline because yesterday I [10:24] started building my first Helix project and and it with with Claude, keep me honest, headless [10:32] connection to Trey and being able to deploy into Helix, right? That's that's the right path. Yeah. [10:37] So yesterday, literally based on the sessions that I sat through on Tuesday, I took a bunch of ideas [10:43] and actually created a spec document and spent a couple hours yesterday with with Claude and I'm [10:49] literally already deployed into the Helix cloud last night and I'm able to actually see what I'm [10:55] building near real time. Like I I I I didn't have to stop at Git. I didn't have to ask for a ticket [11:01] to deploy something. I'm actually already deployed into a dev test state, right? And so the world is [11:08] changing so fast that 6 months ago we said we need a pipeline and now all of a sudden we have [11:13] solutions to solve for that pipeline need. So it's it's we're all on our toes. So anyway I that's [11:18] where my brain goes when we talk about that. So h how have the roles that you guys have needed [11:24] changed within the the organization? Because I I think the the the job titles that we needed [11:29] to hire for a year ago, year and a half ago, I I I don't know if anybody's even using the whole uh is [11:35] it the um prompt prompt engineer? I haven't heard that one in a while. Um we were uh where were we? [11:41] We were talking in a session about a year ago and somebody asked me you know what's your take [11:48] on what's happening with jobs right what's your take on job loss and at the time my answer was [11:54] um a specific a specific kind of job the demand is going down but there are other roles that we have [12:01] not yet defined that are actually going up. So we were talking today the the the the thinking and [12:09] the knowledge that you use to efficiently work for example with claude you can go and vibe code with [12:15] no context you can go and just start asking it to build stuff but if you don't take the time to [12:21] structure and build understand prompt understand context you if you vibe code for 4 hours you're [12:29] going to start creating a crazy bowl of spaghetti that you might not find your way out of if for [12:35] those of you that have gone through this, it's mind-numbing. It drives me crazy. Right? So, [12:39] there are roles like how do you actually proper design a solution prior to using these tools to [12:47] build? I I really believe that's going to be a relevant role. What do we call that? Is that [12:51] a solution design engineer for AI? I don't know. That's the closest thing I can think of. But there [12:56] are other roles now that are coming to the table. And if I use the the pipeline example, we built [13:04] this pipeline because we didn't have another tool. But if you go out there on the floor, [13:07] we're getting to where the pipeline isn't going to be something you have to build or manage anymore [13:12] either. That's going to become a logical extension of the solution options that we have. So yeah, [13:17] it's a lot lot of change. [snorts] What are you doing to help enable business folks to to to deal [13:24] with that? So, for example, I think with a lot of customers that I would think you'd be putting [13:28] folks through some sort of data engineering, vibe coding training, like some best practices sort of [13:35] things, but I most companies I don't think have really a wellestablished formal program to say, [13:39] "Hey, we'll let you go and do this, but here's some things you might want to know." For example, [13:44] clawed out the gate is going to want to build some monolithic, you know, 30,000line main TypeScript [13:50] file that is going to be very very difficult to, you know, turn back into microservices and things [13:55] downstream. But if you tell it, here's what I want to do, plan for that, it can go and build it the [14:00] proper structure right up right up front. M I've had some customers that instead of going through [14:05] like a training thing, they give their their users at least a markdown file that defines, [14:10] hey, here's how we like our solutions to be built internally. And Claude will then go and [14:14] at least do a lot of that for them without them having to be Yeah. Um part of my remitt is also [14:21] client engineering. So all of the endpoints for Zora are also owned by my team. And so we have [14:27] done multiple iterations of pushing agents.mmd cloud.mmd so we can actually standardize across [14:35] everybody who's choosing to vibe code. There's a fundamental standard structure that every time [14:39] they pull up their environment it reads and understands. So now we're to a place where [14:43] we understand the pipeline. So everybody's client in the company understands that the [14:48] target is going to be that pipeline which makes a huge difference in the output of what their [14:52] bi coding solution looks like. because without that context, we were getting all sorts of stuff [14:57] that's not deployable. H how are you guys going to define then, you know, build versus buy out there? [15:04] Because I I think everybody looks at this stuff and says, "Okay, well, great. I can use codeex, [15:08] whatever to build anything." I've heard people I've heard people say, "Why do I need a CRM? I [15:12] can just vibe code a CRM." Um I think there are systems like ERPs that are a long way from being [15:20] vibecoded, right? But there are other solutions like we were talking we were talking at Zora we [15:27] we don't we're we're just at the size where we would start having a conversation about CLM a [15:33] contract life cycle management solution. Um that's a compartmentalized set of workflows that exists [15:41] that might be an opportunity to develop yourself and do exactly what you want it to do. Right. [15:47] um we had been the last 3 years for us had been predominantly um buy we were we were fully on [15:54] the buy track and now we're starting to see these opportunities these pockets of opportunities that [15:59] are compartmentalized where let's try a build right um but the big complex stuff especially [16:06] the stuff that your financials rely on I think we're a ways away from doing it we are we're a [16:12] ways away from doing it in that space how how has what your what your team's able to build, [16:20] how has that changed kind of over the the course of the last couple years? We've we talked about [16:24] like a contract life cycle management use case where it keeps coming up year over year but it [16:29] keeps getting pushed off to the side because it's not the thing but are you able to just go and take [16:34] on a lot more of those projects at this point or how was your thinking? We have this we have this [16:38] conversation we have this this concept that we've been talking about at Zora for it might even have [16:43] been a year now. we have this 10x concept. How do we actually leverage this new technology we [16:48] have to find 10x opportunities, right? So, if you know you've got this level of productivity [16:54] or this level of thinking, how can you restructure or reframe or rethink to find 2x, 5x, 10x? And so, [17:02] it's allowing us to actually think about more things than we used to. There are parts [17:09] of the operational flow though that are still constrained. like if I go deploy 20 more apps, [17:15] I still only have uh a service management team of this many people now trying to deal with 20 [17:22] more apps, right? Like that's not a you you get to the point of diminishing returns if you try [17:29] to keep applying AI to supporting apps. You you eventually need a person, right? So there are [17:34] parts that allow us to build more. I don't think we have felt the pain of what happens when we [17:41] deploy more like we're we're we're just sneaking up on that and I don't think I can tell you what's [17:45] going to happen but I I anticipate constraints. So some areas better some areas still constrained. [17:50] Yeah. Are there we we've got two minutes left. Are is there any sort of Yeah. Just big lesson [17:56] learned. Maybe maybe like a horror story that you you you wish other folks like horror story. [18:02] We we ran into this weird thing. here's a here's something to avoid you know as your organization [18:06] to we are well so I have a couple things that keep me awake um one is when people vibe code [18:13] um those agents go online and pull whatever package they think is relevant for that particular [18:19] solution and that is horrifying to me because the first time a package that gets pulled has malware [18:25] in it it's a problem right that's a problem so I that one keeps me up at night we have discovered [18:30] a few people that have gotten a few things that have been compartment compartmentalized. So, we've [18:34] been able to stop. Um, so I'm not a fan of that. Not necessarily a horror story, but a significant [18:40] challenge is we still haven't solved, you heard me mention identity a couple time. We still haven't [18:44] solved for every one of these systems, every one of these compartments that's trying to be agentic [18:52] or be like an agent. It needs its own identity. It can't it has to be like a like a worker in the [18:59] population. And it can't be just another system because the way identity and access is designed, [19:04] it's not I don't want service accounts everywhere. I need to understand the identity of this solution [19:10] and what that identity is allowed to do and what that identity has done because then if [19:16] the something bad happens then I have a kill switch. Yeah, I can disable the identity. I [19:20] can disable data access. We didn't even talk about MCP gateway, right? So, there are solves, but we [19:27] have not cracked that nut yet. Yeah. Well, again, wrapping up here, appreciate everybody hopping in. [19:33] Hopefully, there was a few nuggets of takeaway, happy to uh hang out in the hall afterwards. Yeah, [19:38] we'll stick around for a minute. Yeah. Anybody's interested in in chatting. And if you want to see [19:41] anything that we're doing with with with Trey, come down to the booth. Like, if you come down [19:45] into the exhibit hall, we're off straight to the right. It's a little past the Dell booth over [19:49] there. if you want to see some of this stuff in in action or if you want to, you know, go through [19:53] your use case and do something a little more personalized, happy to set up something next week [19:56] to get in the weeds with you. But appreciate it, Mark. All right, man. Thanks. Thanks, everybody.