Make VP Says AI Pilots Stall From Organizational Friction, Not LLM Limits
Ai4
AI agent projects stall during production deployment because of misconfigured autonomy, vague operational ownership, and narrow legacy metrics rather than shortcomings in artificial intelligence technology.
## Background
This session was presented by Darin Patterson, VP of Market Strategy at Make, and published by the channel Ai4 on YouTube on August 20, 2026. According to the page description, the talk addresses why AI agent pilots stall between proof of concept and production, drawing on lessons from thousands of enterprise implementations to highlight three key gaps—control, governance, and measurement—alongside real-world case studies and a practical diagnostic framework.
## TL;DR
Darin Patterson explains that AI agent projects rarely fail because the AI model itself is inadequate, but rather because organizations encounter solvable operational friction when moving from controlled pilots to live production. He identifies three main gaps blocking deployment: misconfigured autonomy controls, rigid top-down governance that alienates business process owners, and narrow measurement frameworks focused solely on headcount reduction. Through case studies from Stellantis, Payhawk, and Dude Wipes, Patterson demonstrates how balancing deterministic workflows with AI, decentralizing agent creation to line-of-business teams, and measuring broader business velocity enable companies to reliably scale AI agents to production.
## Atmosphere & style
The session carries an insightful, authoritative, and engaging presentation style, blending executive-level strategy with actionable operational advice. Patterson connects effortlessly with his audience, maintaining an encouraging and pragmatic tone that debunks common tech myths without diminishing the real complexity of enterprise transformation. The talk feels highly grounded and personal, concluding with a warm emphasis on human connection and real-world collaboration beyond the technology itself.
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=4s) Darin Patterson (VP of Market Strategy at Make) highlights that the main challenge in enterprise AI is moving from working demos to production-grade deployments.
- [02:08](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=128s) AI pilots fail or stall primarily due to organizational friction, governance issues, and structural roadblocks—not because modern LLMs are inadequate.
- [02:08](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=128s) Transitioning from a forgiving proof-of-concept environment to production requires handling real-world edge cases proactively rather than relying on manual intervention.
- [04:50](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=290s) The 'Control Dial' framework balances deterministic rules, embedded workflow AI, and autonomous agents to minimize execution risk and avoid high token costs.
- [04:50](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=290s) Over-autonomy is a key pitfall; static, predictable process steps should be hardcoded as deterministic logic rather than left to an agent's discretion.
- [09:51](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=591s) Stellantis UK automated customer communications using Make, resolving over 18,000 inquiries automatically and converting their top complaint into a top NPS driver.
- [11:50](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=710s) Effective governance means assigning process ownership and change management authority to line-of-business managers who own the outcomes, rather than centralizing everything in IT.
- [13:58](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=838s) Payhawk decentralized agent creation by enabling its finance team to build an automated billing triage agent, cutting resolution times from two days to five minutes.
- [15:06](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=906s) Measuring AI success through short-term headcount reduction misjudges ROI; organizations should measure strategic velocity, process speed, and customer experience.
- [16:39](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=999s) Dude Wipes scaled revenue from $72M to $250M while maintaining a 40-person team by deploying multiple targeted agents for freight logistics and sample order management.
- [17:44](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=1064s) Across diverse industries, unblocking AI pilots relies on setting autonomy correctly, empowering line-of-business teams to co-build, and focusing on macro value metrics.
- [18:49](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=1129s) To diagnose stalled initiatives, teams should evaluate their control vs. flexibility balance, clear process ownership, and long-term value capture.
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=4s) Moving from AI Agent Demos to Production Reality
Darin Patterson, VP of Market Strategy at Make, opens by reflecting on his 20-year career helping organizations modernize the way they work. Lately, that transformation revolves around AI and agentic capabilities. He notes that while the tech community has collectively embraced what AI agents are and successfully built impressive demos or proofs of concept (POCs), the current challenge is reaching the next "holy grail moment": taking those working demos and deploying them reliably into production to capture true enterprise-level value.
<Image alt="Darin Patterson speaking on stage about AI agent pilots at Ai4 conference" caption="Darin Patterson on AI agent deployment" src="image_agent_tag_1"/>
Patterson grounds his perspective in his work at Make, a visual AI agent and automation orchestration platform. Make allows business users—those directly handling operational workflows—to build, automate, inspect, and audit complex business processes by connecting the tools and software applications they already rely on daily.
- **Speaker Experience:** 20 years guiding global enterprise work transformation and process automation.
- **Platform Capability:** Make offers visual, audited orchestration across existing application stacks for line-of-business users.
---
[02:08](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=128s) The Uncomfortable Truth: Why AI Projects Stall
Addressing a widely cited statistic from analysts like Gartner, Patterson acknowledges the "uncomfortable truth" that a vast majority of AI initiatives are being scrapped or stalled. However, he strongly refutes the common assumption that these failures occur because "the AI isn't good enough yet." Rather than hitting technical limitations of modern LLMs, projects stall due to organizational friction and operational roadblocks that are entirely solvable by human and structural adjustments.
> "They're not dying because the AI is not good enough or they're not stalled because the AI is not good enough. They're dying often or getting stalled because of production roadblocks that are usually solvable by organizational changes."
Transitioning from a pilot to production represents a massive operational leap:
* **Proof of Concept (POC) Environment:** Highly forgiving by design. Usually monitored by a single champion who catches errors manually, operating on a narrow set of predictable, known inputs.
* **Production Environment:** Unforgiving and automated at scale. Errors must be prevented proactively rather than caught after the fact, and system architecture must handle edge cases and unexpected real-world inputs gracefully.
---
[04:50](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=290s) The Control Dial: Balancing Autonomy and Determinism
Analyzing thousands of enterprise automation implementations, Make identified three critical gaps that determine whether an agent ships or stalls. The first is the **control dial**—finding the correct balance between absolute control and open-ended flexibility across the automation spectrum.
Patterson emphasizes that a complete business process is rarely 100% autonomous or 100% rigid. A common failure mode occurs when teams grant an agent total autonomy over predictable steps that do not require reasoning. For instance, if a prompt uses ALL-CAPS instructions like *"ALWAYS look up the customer's account history first,"* that step should actually be built as a deterministic, hardcoded rule rather than left to an agent's discretion.
- **Over-Autonomy Risks:** Forcing LLMs to reason through simple, static steps increases execution risk (the agent might skip the step under strange inputs) and dramatically spikes token costs.
- **The "Autonomy Problem":** What teams often diagnose as prompt engineering or evaluation failures are actually dynamic autonomy misconfigurations.
- **Architectural Solution:** Embed AI agents inside defined, structured workflows and equip them with constrained, process-specific tools rather than open-ended database write permissions.
---
[09:51](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=591s) Case Study: Stellantis Solves Customer Service Friction
To illustrate the control dial in action, Patterson highlights Connor, a service department team member at automotive giant Stellantis in the UK (managing brands like Fiat, Chrysler, and Jeep). Facing high customer dissatisfaction around service updates, Connor built an automated communication agent using Make without writing code.
Instead of turning the entire customer conversation over to an unconstrained AI, Connor mapped the business logic strictly. Deterministic triggers handle outbound status alerts, while an embedded AI agent interprets ambiguous inbound customer responses (e.g., analyzing whether a customer's *"okay"* means approval to wait extra days for backordered tires) and routes actionable updates to technicians.
- **Scale & Volume:** Over 47,000 customer messages analyzed and 18,000 customer inquiries automatically resolved.
- **Business Impact:** Shifted customer communication from the company's **#1 customer complaint** into its **#1 driver of positive Net Promoter Score (NPS)**.
---
[11:50](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=710s) Governance: Operational Ownership vs. IT Bureaucracy
The second major bottleneck holding pilots back is **governance**. Patterson clarifies that effective governance is not about restrictive top-down corporate bureaucracy, but rather establishing clear operational ownership, change management protocols, and decision-making authority once a system goes live.
Production deployment forces organizations to resolve four core questions:
1. **Process Ownership:** Who actually owns the end outcome? (Centralized IT teams build agents, but line-of-business managers own the underlying business processes.)
2. **Change Control:** Who has authority to adjust prompts, update tools, or publish workflow revisions?
3. **Human-in-the-Loop Checkpoints:** Should outputs be reviewed prior to customer delivery, or audited asynchronously post-execution?
4. **Accountability:** When an edge case occurs or external tools change, who takes responsibility for fixing the system?
> "If you're building agents only centrally through an IT team and something goes wrong... I suspect your IT team is not going to say, 'Yep, we're the ones accountable.' It's actually owned and best done by the business process owner themselves."
---
[13:58](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=838s) Case Study: Payhawk Empowers Finance Teams to Build
Financial tech platform Payhawk (referenced in the session as Perk) provides a blueprint for successful governance by empowering non-technical line-of-business teams to build and maintain their own agents. Gabriel, a Senior Engineering Manager, decentralized agent development by providing the finance operations team with the tools to build their own custom workflows in Make.
The finance team created an AI agent named "Ariel" to serve as the initial triage layer for inbound billing and invoice inquiries. Ariel analyzes customer intent and automatically routes or resolves incoming tickets, allowing the team to meet strict operational targets.
- **Efficiency Gain:** Resolved **30% of incoming finance tickets fully automatically**.
- **SLA Recovery:** Cut response times from a backlogged **2-day resolution window down to 5 minutes**, staying within internal SLAs.
---
[15:06](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=906s) Measurement: Playing the Long Game Beyond Headcount
The final silent killer of AI pilots is poor **measurement**. Organizations frequently rely on legacy software metrics—primarily short-term headcount reduction or basic time-savings—which fails to capture the true strategic return on investment (ROI) of agentic systems.
Patterson warns against treating AI simply as a tool to shrink staff, advocating instead for business cases built around organizational velocity and employee fulfillment.
---
[16:39](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=999s) Case Study: How Dude Wipes Scaled Revenue 40x
An example of strategic measurement and continuous automation comes from CPG brand Dude Wipes. Zach Means joined as employee #14 when the company was doing $6 million a month. By aggressively deploying AI agents to handle repetitive administrative logistics, the company scaled revenue dramatically while maintaining an extraordinarily lean workforce.
Instead of seeking one massive, silver-bullet AI project, Dude Wipes deployed multiple targeted agents across operations:
* **Logistics Agent:** Manages end-to-end freight and shipment planning from manufacturers to regional distribution warehouses.
* **Marketing & Samples Agent:** Scans incoming sales emails, identifies sample request details, auto-generates orders, and sends tracking notifications.
- **Revenue Growth:** Scaled from **$72M annually to $250M per year** (doing $6M/month originally).
- **Headcount Efficiency:** Expanded the team to just **40 total employees** while supporting massive retail distribution.
- **Time Returned:** Returned over **800 hours** of manual administrative work back to staff to focus on strategic brand growth.
---
<ElicitationsGroup message="To explore these framework concepts further:">
<Elicitation label="Analyze the Control Dial trade-offs between cost, latency, and reliability" query="Break down the Control Dial framework showing trade-offs between cost, latency, and reliability across different AI autonomy levels."/>
<Elicitation label="Draft a governance roadmap for line-of-business AI agent creation" query="Draft an enterprise governance roadmap for enabling line-of-business teams to safely build and deploy AI agents."/>
</ElicitationsGroup>
[17:44](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=1064s) Lessons from Real-World AI Agent Success Stories
Darin Patterson synthesizes the core takeaways from the three case studies discussed throughout the presentation. The key to turning an AI pilot into a production success comes down to mastering three critical areas: adjusting the control dial, establishing distributed governance, and tracking big-picture measurements. Setting the control dial correctly enabled a major automotive retail group to turn its number one customer complaint into its leading Net Promoter Score (NPS) driver and top customer success factor. Governance succeeds when process ownership belongs to the people closest to the business rather than being restricted to a centralized IT perspective; when done right, IT transitions from a central roadblock into an enabling partner. Finally, focusing measurement on the macro picture with iterative improvements gave companies like Dude Wipes the confidence to go all-in with AI agents, scaling revenue from $72 million to $250 million.
- **Cross-Industry Application:** Despite operating in completely different industries with contrasting business models and organizational structures, each company overcame the exact same friction points to bring their proofs-of-concept into production.
- **Ditching Prompt Tweaking:** Successful teams did not waste countless hours "fussing over their system prompts or the evals and writing in all caps" to force agents to cooperate.
- **Business Empowerment:** Triumphant organizations empowered the business teams closest to daily processes to govern and co-build those workflows alongside IT.
[18:49](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=1129s) The Monday Morning Framework for Unblocking AI Pilots
Patterson urges attendees to capture a slide on their phones to review on Monday morning after returning home from Las Vegas. To diagnose why an agentic workflow project might be stalling or to ensure a new initiative stays on track, leaders and project teams must confront three direct diagnostic questions. If leading a team directly, these questions should serve as the blueprint for the very next reporting check-in.
- **Control vs. Flexibility:** "Do we really understand our aptitude for control versus flexibility?"
- **Process Ownership:** "Do we understand who owns the process?"
- **Big-Picture Value:** "Do we have our eye on the bigger prize? What is the big picture value-add that we're going to be focused on in investing?"
[19:29](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=1169s) Bringing It Back to People and Connecting at the Conference
Closing out his talk, Patterson emphasizes that the ultimate line between success and getting trapped in the "proof of concept stage forever" isn't about the tech, but about the people. Reflecting on the current era of artificial intelligence, he notes how much more he appreciates genuine human connections compared to five years ago, expressing his gratitude for being present in person with the audience. He invites everyone to visit the Make team on the showroom floor by following the marked blue line to discuss use cases, share personal insights, and explore the unique challenges and opportunities their organizations are facing.
- **Key Takeaway:** "It's not all about the tech. It's often about the people. This is where it stops and ends."
- **Invitation:** Attendees are encouraged to connect directly with Make's team of "Makers" located on the exhibit floor to continue the conversation.
## Notable quotes
> [02:08](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=128s) They're not dying because the AI is not good enough or they're not stalled because the AI is not good enough. They're dying often or getting stalled because of production roadblocks that are usually solvable by organizational changes. — Darin Patterson
> [11:50](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=710s) If you're building agents only centrally through an IT team and something goes wrong... I suspect your IT team is not going to say, 'Yep, we're the ones accountable.' It's actually owned and best done by the business process owner themselves. — Darin Patterson
> [18:49](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=1129s) Do we really understand our aptitude for control versus flexibility? — Darin Patterson
> [18:49](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=1129s) Do we understand who owns the process? — Darin Patterson
> [18:49](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=1129s) Do we have our eye on the bigger prize? What is the big picture value-add that we're going to be focused on in investing? — Darin Patterson
> [19:29](https://www.youtube.com/watch?v=uwSjLZAxfwU&t=1169s) It's not all about the tech. It's often about the people. This is where it stops and ends. — Darin Patterson
## Video description
Darin Patterson - VP, Market Strategy, Make
Most AI agent projects don't fail because the AI isn't good enough. They stall between proof of concept and production.
Drawing on lessons from thousands of companies building AI-powered workflows and AI agents, this session reviews three gaps that consistently prevent pilots from shipping: the agentic autonomy dial (balancing control and flexibility), governance (decisions on ownership, change control, and decision making authority), and measurement (defining the right success criteria for each AI use case).
We'll frame how these gaps get addressed through real use cases from an enterprise SaaS company that cut finance ticket resolution from two days to five minutes, a major automotive retail service group that turned its #1 customer complaint into a top NPS driver, and a fast-growing CPG brand that scaled from $72M to $250M in revenue while returning 800 hours to employees through AI agents.
Attendees will leave with a practical framework for diagnosing gaps that are blocking their own initiatives and what it takes to close them.
Make副总裁称AI试点停滞源于组织摩擦而非LLM局限
AI智能体项目在生产部署中停滞，是由于自主权配置不当、运营权责模糊以及狭隘的传统指标，而非人工智能技术本身的缺陷。
Darin Patterson解释说，AI智能体项目极少因AI模型本身能力不足而失败，更多是因为企业在从受控试点迈向实际生产时遇到了可解决的运营摩擦。他指出了阻碍部署的三大主要差距：自主权控制配置不当、疏离业务流程负责人的僵化自上而下治理，以及仅聚焦于精简人手的狭隘评估体系。通过Stellantis、Payhawk和Dude Wipes的案例研究，Patterson展示了如何通过平衡确定性工作流与AI、将智能体创建权下放至业务线团队，以及衡量更广泛的业务推进速度，来帮助企业可靠地将AI智能体规模化应用于生产环境。
Make 市场战略副总裁 Darin Patterson 指出，企业 AI Agent 项目停滞或失败的主因并非当前 LLM 能力不足，而是从宽容的 POC 环境跨越到严苛的生产环境时遭遇了组织阻力与治理障碍。 (2:08)
“控制旋钮”（Control Dial）框架强调平衡确定性逻辑与自主性：应将完全可预测的业务步骤硬编码为确定性规则，过度赋予 Agent 自主权不仅会增加运行风险，还会大幅推高 Token 成本。 (4:50)
Stellantis UK 通过 Make 构建了结合确定性触发与嵌入式 AI 的客服 Agent，自动解决超 1.8 万起客户咨询，将原本排名第一的客户投诉转化为 NPS 提升的核心推动力。 (9:51)
有效的 AI 治理应将流程所有权和变更管理权限下放给业务线部门而非完全集中于 IT；Payhawk 财务团队借此自行构建对账 Agent，将工单处理时间从 2 天压缩至 5 分钟。 (13:58)
企业衡量 AI 价值需跳出短期裁员和单纯省时的传统视角，转向整体业务加速与战略敏捷性；消费品牌 Dude Wipes 借助多场景 Agent 协作，在仅 40 人的团队规模下将年营收从 7200 万美元扩展至 2.5 亿美元。 (16:39)
Reading summary
Darin Patterson, VP of Market Strategy at Make, argues that enterprise AI agent pilots stall not because of model limitations, but due to operational and organizational roadblocks across control, governance, and measurement.
The Production Gap Beyond Model Limitations (2:08)
Patterson challenges the widespread assumption that failed AI initiatives stem from modern large language models lacking capability. Addressing Gartner estimates that most enterprise AI projects stall, he claims that the bottleneck lies in the operational leap from proof of concept (POC) to live production. In a pilot environment, workflows remain forgiving because an individual project champion manually monitors execution and catches errors against narrow, predictable inputs.
In contrast, live production environments demand proactive error prevention across unpredictable real-world inputs at scale. Patterson asserts that organizations misdiagnose systemic architecture and process readiness issues as prompt or evaluation failures, leaving teams stuck in perpetual pilot phases when structural adjustments could solve the underlying friction.
Calibrating the Control Dial Against Over-Autonomy (4:50)
The first operational breakdown occurs along the control spectrum, which spans rigid deterministic rules, hybrid workflows with embedded AI, and fully autonomous agents. Patterson notes that teams frequently err by granting agents open-ended autonomy over routine, predictable tasks that require no probabilistic reasoning. Using all-caps prompt directives such as requiring the model to always check account history first is an indicator that a step should be a hardcoded rule instead of an open-ended LLM decision. Forcing models to reason through static steps spikes token expenses and introduces execution risk.
To establish reliability, organizations must embed AI agents within structured, deterministic boundaries. At automotive group Stellantis in the UK, a service department employee used Make to configure fixed triggers for outbound alerts while deploying an embedded agent solely to interpret ambiguous inbound customer messages. Managing over 47,000 customer messages and resolving 18,000 inquiries automatically, the hybrid architecture transformed customer communication from the company's leading complaint into its top driver of positive Net Promoter Score.
Decentralizing Operational Ownership Away From Centralized IT (11:50)
The second critical bottleneck is governance, which Patterson defines not as corporate red tape, but as establishing transparent process ownership, change management, and human-in-the-loop checkpoints. A major operational failure mode occurs when centralized IT departments develop agents in isolation; when edge cases break a workflow, centralized technical teams rarely accept accountability for underlying business outcomes.
Patterson contends that sustainable governance requires empowering line-of-business process owners to build, modify, and audit their own workflows. At financial tech firm Payhawk, engineering leadership provided the finance operations team with visual orchestration tools, enabling them to build an inbound billing triage agent named Ariel. Operating under line-of-business management, the agent automated 30% of incoming finance tickets and reduced backlogged response times from two days down to five minutes while maintaining internal service level agreements.
Reframing Success Metrics Beyond Headcount Reduction (15:06)
The third failure mode involves evaluating agentic automation through legacy software metrics, specifically direct labor reduction and short-term time savings. Patterson warns that relying solely on headcount cuts misses the strategic enterprise value of AI agents, which lies in process acceleration, retention, and operational adaptability.
Consumer packaged goods brand Dude Wipes exemplifies this strategic measurement model. Rather than searching for an all-encompassing single agent, the company integrated targeted agents across operational logistics and sample fulfillment. Handling end-to-end freight distribution and inbound sales email orders returned over 800 hours of manual administrative labor, supporting revenue growth from $72 million to $250 million annually with a lean team of 40 employees.
Patterson concludes that moving beyond stalled pilots requires leaders to assess three diagnostic questions: clarifying their organizational balance between control and autonomy, assigning process ownership to frontline business units, and evaluating investments against strategic enterprise velocity rather than immediate headcount reductions.
[2:08] Darin Patterson: They're not dying because the AI is not good enough or they're not stalled because the AI is not good enough. They're dying often or getting stalled because of production roadblocks that are usually solvable by organizational changes.
[11:50] Darin Patterson: If you're building agents only centrally through an IT team and something goes wrong... I suspect your IT team is not going to say, 'Yep, we're the ones accountable.' It's actually owned and best done by the business process owner themselves.
[18:49] Darin Patterson: Do we really understand our aptitude for control versus flexibility?
[18:49] Darin Patterson: Do we understand who owns the process?
[18:49] Darin Patterson: Do we have our eye on the bigger prize? What is the big picture value-add that we're going to be focused on in investing?
[19:29] Darin Patterson: It's not all about the tech. It's often about the people. This is where it stops and ends.
Original transcript
[0:04] Just briefly before we jump into the meat
uh today, I'll introduce myself. I've been
[0:09] lucky enough to spend the last 20 years working
with companies both small and large all over the
[0:16] world on helping them bring about the future of
work and what it means to change the way we work
[0:22] inside of our organizations to accomplish amazing
outcomes. Of course, increasingly uh lately that's
[0:28] all about how work changes through the use of
agentic processes and capabilities and AI. And
[0:34] that's why I'm very excited today to get a chance
to share some of the insights and learnings that
[0:39] I've had in interacting with customers all over
the world who are trying to take these concepts
[0:44] and move from that place where we all were
a little bit of a year ago, maybe two years
[0:49] ago when we were getting introduced to these new
concepts. We've all sort of understood what an AI
[0:55] agent is now. Many of us in the room, I suspect,
have built them. And at this point in time,
[1:00] we're trying to get to that that next holy
grail moment. Moving from the moment where we
[1:05] saw something work beautifully in that demo or
we saw that proof of concept that the team put
[1:11] together work very effectively and now how do we
take that and actually get it into production so
[1:16] that we can get those big outcomes that we've been
hoping for to pay off in in the big picture. So
[1:24] to help us set the stage just a little bit about
some of the stories that I'm going to share today,
[1:29] I wanted to give you a bit of piece of context.
Most recently in my role, I've been able to work
[1:33] at an organization called Make. Make is an AI
agent and automation orchestration platform that
[1:40] enables business users, those that are closest
to a business process, to be able to identify and
[1:46] automate a core business process in a very visual
way. They do that by interacting with all the
[1:52] different applications that people already use to
run their businesses. and they're able to inspect
[1:57] and audit every single piece of what happened in
a particular automation along the way. So this is
[2:02] the background and the context when uh upon many
of what the customers we're going to talk about
[2:08] today have built in the organizations. So we're
going to start with an uncomfortable number and I
[2:15] have to admit I've attended three or four sessions
and I've already seen this number three or four
[2:19] times in the last three or four days. So the good
news is we're all aware and all on the same page
[2:23] on the same problem and we're seeing stats like
this from reputable organizations like Gartner
[2:28] and other research organizations that identify
that uncomfortable truth that we see. Yes,
[2:34] we see the power. Yes, we see the inspiration of
agent capabilities. But in reality, many of these
[2:39] AI projects are getting scrapped yet this year and
of course into next year. Now it would be easy for
[2:46] all of us to have the same conclusion. the AI
was kind of cool, but it's not quite ready for
[2:50] the business. That would be an easy conclusion to
come out that maybe we're a bit ahead of our skis.
[2:55] That's not actually what I see. I get to travel
all over and meet with customers that actually
[2:59] are moving stuff into production and they're not
dying because the AI is not good enough or they're
[3:04] not stalled because the AI is not good enough.
They're dying often or getting stalled because
[3:10] of production roadblocks that are usually solvable
by organizational ch changes. how you think about
[3:16] the human side of this. They're not technical
problems. They're solvable and they're solvable
[3:21] by the people in these rooms with a little bit
of education about how we can go back and help
[3:25] our organizations work through this leap. And
of course, the leap that we're talking about
[3:31] is how a pilot or a demo is very different
from a production usage. Of course, a pilot,
[3:39] it's forgiving by design. one champion or one team
of people is typically watching every single run
[3:45] of what happens with this aged dentic process.
When something goes sideways that they catch
[3:50] it by hand and of course it's only ever seeing
a handful of known inputs. And of course this
[3:56] jump that we're all interested in making is about
making it sure that we get into a place where in
[4:01] production we're able to identify that agents run
at scale. We're able to make sure that errors can
[4:10] be caught before, not after the fact. They have to
be prevented. And we of course recognize that all
[4:16] the different use cases, no matter how thoughtful
we were in the design of the proof of concept,
[4:22] we're always going to encounter something that
we didn't expect in the real world. And in truth,
[4:27] this has always been true with all software proofs
of concept and going to production. There's always
[4:32] a leap. But this leap is much bigger gap is a
much bigger gap than it was previously because
[4:37] a Gentic by its very design has different
outcomes associated with it. So everything I'm
[4:43] about to say is about how do we successfully
make this jump from pilot into production.
[4:50] And when I look across thousands of different
customers, we've observed a trend and identified
[4:54] three very specific gaps that whenever they're
addressed, they enable you to make that leap
[4:59] into production and of course get that meaningful
impact for your organizations. We'll talk about
[5:04] every single one of these in some level detail. Uh
but first piece that we see is the control dial.
[5:10] Essentially, how much autonomy that we want to
give to AI at each step in the process. Second,
[5:17] we'll talk about governance. I know it's not the
sexiest topic that we all can think about spending
[5:22] our day talking about, but it is definitely a
theme that I've heard at this conference and
[5:26] definitely something that we have to get really
good at in order to apply this technology to
[5:29] our organizations. And third, measurement. Being
extremely clear about what success actually looks
[5:37] like and making sure that we're creating business
cases that enable us to play the long game on
[5:42] how do we really get the return on investment for
these types of initiatives. I'm going to take each
[5:47] one of these in turn and I'll also show you a real
company that is doing it right. They're bringing
[5:52] these concepts uh to bear. So, we'll start first
with what I refer to as the control dial. And the
[6:00] control dial is ultimately all about balance. It's
balancing control against flexibility. And really
[6:08] doing that to understand across how that gets
done in a business process. And when you really
[6:14] back up and you think about like what is the value
proposition of an AI agent, a AI agents ultimately
[6:21] are automating things. And in reality, we've
been automating things for a long time within
[6:26] technology. Agents are simply another way to
automate things. And the way I increasingly think
[6:32] about this is on an automation spectrum. On one
end of an automation spectrum, you have a set of
[6:39] rules, automated workflows that fire consistently
based upon some information, some context, and
[6:45] the rules that have been applied to it. Those are
automated workflows, and they're still incredibly
[6:50] valuable. On the other end of the spectrum, all
the way on the other end is AI agents, where AI is
[6:56] used to determine what steps in a workflow to take
on and how those steps should be reasoned through
[7:02] and what to do next. There's a missing piece that
we often don't talk about enough, which is AI
[7:07] within a workflow. In other words, the workflow
is relatively well-designed and stable. It's
[7:12] deterministic, if you will, but it has AI at the
right steps within that process. This is a good
[7:18] frame of reference for thinking about an entire
business process. And what's really important here
[7:23] is an entire business process is almost never
completely agentic or completely automated and
[7:30] deterministic. It's actually every step within
the process has some level of this control dial
[7:36] and and this automation spectrum and how it gets
executed. Now, here's why so many teams get stuck.
[7:43] They put it they're going to solve a problem that
they have in their organization and a business
[7:47] process and they put an agent all the way over
the completely autonomous end for a decision that
[7:54] honestly doesn't need to be there. If you have
a business process that says always look up the
[7:59] customer's information and then go look up their
refund history and then do this. And if you find
[8:04] yourself writing agent prompts that say always do
this in all caps. If you're writing in all caps,
[8:10] you have a good signal that maybe what you're
actually designing is a deterministic part of
[8:15] the workflow. I should always already provide
that information to my agent. I don't need the
[8:19] agent to think through whether or not to go look
up that customer information. That both increases
[8:26] risk that maybe it won't because the circumstances
in the real world differ from time to time but it
[8:32] also actually increasingly important topic it
increases cost. Every time you ask AI to think
[8:38] through what steps to do and I have to look at
all the tools available I'm actually increasing
[8:42] my token spend at the same time. So what happens
often is people take obvious decisions and they
[8:50] give them to AI agents because AI agents are
the cool new thing and in reality we need to
[8:54] be balancing these different tools in order to
accomplish the automation. So you the way this
[9:01] ultimately shows up is people think of this
as an eval problem. when you put an agent into
[9:05] production or when you're working on the proof of
concept, you go through a lot of evaluations and
[9:10] most often what people have is not an eval problem
or a prompt problem or a reasoning problem. They
[9:16] have this autonomy problem. So thinking through
the fix here, the fix is to make sure that you
[9:22] provide agents with all the right business process
up front and the agents are part of an automated
[9:27] workflow process as opposed to the entire workflow
process. When you provide agents with tools, it's
[9:34] important that you provide them not just here's
access to Salesforce, go do whatever you want,
[9:39] but instead specific tools that accomplish
specific processes within a particular
[9:44] application so that those tools themselves can be
deterministic as part of that business process.
[9:51] Now, let me make this real for you. I want to tell
you about an individual named Connor. He works at
[9:57] a company called Stalantis. And you some of you
are familiar with the brands of cars that this
[10:02] uh company manages, things like Fiat and
Chrysler and Jeep, but ultimately Connor worked
[10:07] in a service department in the UK at Stalantis
and they actually had a particular challenge
[10:12] on their hands at this organization which is
they had an increasing number of communication
[10:20] complaints. If you imagine what it's like to
work in a service dealership at uh a busy uh or
[10:26] service department at a busy dealership, you have
customer communications going on all the time. You
[10:32] have people trying to get your cars in and out as
fast as possible. Connor, who's not a developer,
[10:37] actually identified, hey, there's got to be a
better way to approach this. and he implemented
[10:41] an automated solution to analyze both to automata
automatically send outbound communications to
[10:47] customers about the status of their vehicles
but also to analyze inbound communication and
[10:52] determine what to do with that. Now what's really
important about Connor's approach is he didn't
[10:56] just say man agents are the new cool thing he
said what does the business process look like and
[11:02] what are the steps in the process that I can very
clearly define and what are the steps that I need
[11:06] AI to support. So when a when I say to a customer,
hey, we don't have those uh tires in stock. Do you
[11:13] want to wait till tomorrow? And they write back,
okay? I can use AI to analyze the context of that
[11:18] conversation and then notify the right service
person at the right time effectively. In this
[11:24] case, Connor wasn't giving AI the complete control
of the conversation, just aspects of control of
[11:29] the conversation. And of course, the results mean
that there are over 47,000 messages analyzed.
[11:36] They've had 18,000 of those messages disclosed
automatically and communication became their
[11:41] number one driver of NPS success and positive
NPS scores uh when it was previously one of the
[11:48] negative the things bringing that NPS score down.
So that brings us to the second major gap that
[11:55] people see when they try to make that leap from a
pilot into production and that's governance. And
[12:02] the thing about the word governance is the moment
I say that word, you start to think top-down
[12:06] bureaucracy. And that's actually not what I mean.
But I mean the practical day-to-day things about
[12:11] ownership, change control, and who has authority
and responsibility once a process goes live. Now,
[12:20] the thing about a production use of AI agents is
they actually force you to answer four fundamental
[12:25] questions that you never had to answer within the
pilot. First of all, understanding who really owns
[12:31] this process. And this is an interesting piece
because increasingly I didn't so first of all
[12:37] notice I didn't say who owns this AI agent. I said
who owns the process. And if you're working in a
[12:43] centralized model where it is the only team
building AI agents, they're also going to be
[12:49] only responsible for agents, but they're not
responsible for the process. So understanding
[12:53] and engaging the right stakeholder and the right
business stakeholders for being responsible for
[12:58] owning the process and the outcome that includes
also understanding how the process can change who
[13:04] can change it and determine when it's appropriate
to release those particular changes. What are the
[13:09] appropriate checkpoints that are necessary in a
particular business process? Are we do we have a
[13:15] level of risk within this agentic process that
means we need to evaluate everything before it
[13:19] goes to a customer or are we comfortable uh
having it go ahead and be sent and have an
[13:25] individual review afterwards and of course at the
end of the day this is about accountability who's
[13:32] accountable when something goes wrong I suspect
if you're building agents only centrally through
[13:38] an IT team and something goes wrong in terms of
your context or the tools change I suspect your
[13:44] IT team is not going to say, "Yep, we're the ones
accountable." It's actually owned and best done
[13:49] by the business process owner themselves. And so,
it's important that we empower those individuals
[13:53] to be able to leverage this technology. So,
let me show you what an excellent use case
[13:59] of this looks like. This is a company called
Perk. It's one of the fastest growing travel
[14:05] and spend management platforms in the world.
Gabriel now he's in the senior, he's a senior
[14:10] engineering manager there. His whole focus though
is about empowering others in his organ in their
[14:16] organization to fix and own their own processes.
So he handed the finance team, the people that are
[14:22] chasing down payments and answering billing and
invoice questions, the ability to build their own
[14:25] agents. And they went from a process that they had
a two-day SLA that they were unable to get to. And
[14:33] they had processes that were hard to defi, hard
for Gabriel to define, but easy for the finance
[14:40] team to engage in. So they built a finance team in
make. They named it Ael Ariel. And she's now the
[14:46] first filter on every inbound customer request.
It reads their intent and routes it to the right
[14:51] team. And because of that process, they're able to
actually solve 30% of their tickets automatically
[14:56] and come within their own internal SLAs's. And
what's important here is that the finance team
[15:01] themselves were empowered to own the process and
own that to bring it to production. The third gap
[15:07] is about management. This one's often the silent
killer because most teams have a really hard time
[15:13] really genuinely understanding success upfront
and they can never actually build up the the
[15:18] business case necessary to ship. And here's what
the trap looks like in detail. We could spend a
[15:24] lot of time just talking about how we measure
agentic success, but I would suggest to you
[15:28] there's an old way of thinking about success
in software products. And in the age of AI,
[15:34] I would encourage people to think in the
broader, more holistic long-term value way.
[15:39] If you look at each one of these dimension, you
immediately think, how do I save time? And yes,
[15:46] I love saving time as much as anyone else, but
I think the opportunity for agentic processes
[15:50] is much greater. It's about overall process
acceleration, which means increasing our ability
[15:55] to be competitive. We often think about quality
and error reduction, but actually quality creates
[16:00] better customer experiences. We talk about adding
new revenue resources, but it's really about as
[16:05] your organization is about strategic agility,
being able to respond respond to a fastmoving
[16:09] environment. And I hate it when people first think
about AI and how can we reduce hat headcount,
[16:15] but ultimately how can we create an empowered
workforce that loves their job when they come
[16:20] to work. Thinking about the long-term success of
an agentic initiative in these terms will help
[16:26] you create a business case that your executives
can actually buy into. And at the same time,
[16:31] you can't accomplish this all in one day. And so
it's recognizing that there are many different
[16:35] small steps that help us get to that bigger
business impact. That brings me to an interesting
[16:40] company called Dude Wipes. I kid you not, they
make flushable wipes for men. It's probably not
[16:45] the sexiest company we've ever talked about on the
AI4 stage, but it's an incredible success story.
[16:51] Zach Means, he joined as employee number 14. He
ran their operations team. They were doing about
[16:56] 6 million a month and and now three years later
doing 250 millions a year with just 40 people. And
[17:02] all of that's not because they hired aggressively
and were going to grow at all cost, but they made
[17:06] a deliberate choice. automate every process that
doesn't need human judgment so that the people
[17:12] can focus on the sort of the bigger picture more
strategic work. One agent that they built handles
[17:20] all their shipment planning from manufacturers
to warehouses. Another reads marketing emails,
[17:24] spots sample orders, creates them and sends
tracking updates. And the reason this matters
[17:28] is because they had their eye on the big picture
prize. They knew that there are many small agents
[17:33] that are going to lead up to the big picture to
be able to enable them to grow successfully very
[17:38] quickly. So we put that all together. We had three
companies and three real pro proof points. Again,
[17:44] the control dial that helped set right once it
turned the number one factor of complaints on an
[17:51] NPS driver to the number one success factor for
customers. governance done right by the people
[17:58] that are actually closest to the business who can
actually be those process owners as opposed to
[18:03] only from a centralized IT perspective. Instead,
it moves into an enabling role rather than the
[18:08] central roadblock and measurement focused on the
big picture with iterative movements that gave
[18:15] dube wipes for example the confidence to go allin
and grow from 72 million to 250 million. Now,
[18:22] all those companies, they're completely different
industries, different ways of structure,
[18:25] and their business models, but they all overcame
what's necessary to bring those proofs of concept
[18:29] into production. They didn't spend countless
hours fussing over their system prompts or
[18:35] the evals and writing in all caps to get the
agent to do exactly what I asked them to do.
[18:40] They empowered business teams that are closest to
the business process to govern those processes in
[18:45] addition to helping them build. And they kept
their eye on the prize for the bigger picture
[18:49] wins. If any of this sounds familiar, this is the
moment where I want you to take out your phone and
[18:54] take a picture so you can go back on Monday
morning when you're all energized and rested
[18:58] after enjoying Las Vegas. These are the teams if
you're working on an agentic pro, these are the
[19:03] questions if you're working on an agentic process
you want to go and ask yourselves Monday morning.
[19:08] If you're not working on it directly and you have
a team of people that report to you, these are the
[19:12] questions you want to ask at your next reporting.
Do we really understand the basically our aptitude
[19:18] for control versus flexibility? Do we understand
who owns the process? And do we have our eye on
[19:23] the bigger prize? What are the big picture value
ad that we're going to be focused on in investing?
[19:29] That brings me back to where we started. It's not
all about the tech. It's often about the people.
[19:34] This is where it stops and end. And this is the
difference between success and hanging out in the
[19:40] proof of concept stage forever. Now, in this stage
of AI era, I appreciate human connections way more
[19:48] than I did even 5 years ago. That's why I love
being here in person with you. So, if you want
[19:52] to talk more about these use cases or about the
general the concept or things that you've noticed
[19:57] that are super important, I'd love to hear about
it. We have a fantastic team of makers as well.
[20:02] And all you have to do is follow that blue line
to find us on the showroom floor. We'd be happy
[20:06] to talk about these uh pieces and connect with
you and hear what challenges and opportunities
[20:10] you're facing as well. Thanks so much for your
time. I hope you have a fantastic conference.
