GM, Franklin Templeton Rein In Enterprise AI With Strict Token Caps

The Generative AI Playbook: Setting Your Enterprise Up for Success — Ai4

Ai4

Enterprise leaders warn that automating flawed legacy workflows merely optimizes broken processes, urging strict model routing and pre-execution governance to prevent runaway costs.

AI-assisted summary of The Generative AI Playbook: Setting Your Enterprise Up for Success — original source. Check the original for full context.

Source published:

Detailed notes

## Background Published on YouTube by the channel Ai4 on August 22, 2026, this panel session brings together enterprise leaders from General Motors Financial, UI Health, Concentrix, and Franklin Templeton, moderated by technology journalist Rob Pegoraro. The presentation outlines a practical operational playbook for enterprise executives looking to scale generative AI initiatives through disciplined governance, strategic use-case selection, and technology adoption.

## TL;DR Enterprise technology leaders share concrete deployment case studies, ranging from Franklin Templeton's contrarian investment agent and GM Financial's multi-agent development pipelines to UI Health's EHR clinical automation and Concentrix's agentic HR platform. The discussion stresses that sustainable enterprise value depends on rigorous pre-deployment data hygiene, multi-tier model routing, and pre-execution governance guardrails rather than uncritical frontier model hype. The panelists caution against automating fundamentally broken workflows, urging organizations to focus on domain reimagination, usage-based cost discipline, and revenue expansion rather than pure headcount reduction. Looking ahead, the panel weighs the emerging governance challenges of ephemeral micro-agents and algorithmic middle management against the enduring need for human empathy and nuanced micro-judgments.

## Atmosphere & style The panel maintains a refreshingly pragmatic, war-room candidness as seasoned executives trade real-world deployment battle scars rather than Silicon Valley hype. Rob Pegoraro's lighthearted, self-deprecating moderation keeps the dialogue brisk, fostering an engaging dynamic where cybersecurity gatekeeping and aggressive business innovation playfully clash. The tone feels grounded and intellectually honest, blending vivid analogies like the 'chainsaw principle' with measured optimism about liberating workers from operational drudgery.

## Key takeaways - [03:31](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=211s) Franklin Templeton uses an agentic digital co-worker ('Gromit') to provide contrarian viewpoints in research meetings, restoring productive cognitive friction when junior staff hesitate to challenge leadership. - [06:55](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=415s) GM Financial decomposes software development into sequential multi-agent stages (business specs to Gherkin criteria to code generation) while enforcing mandatory human-in-the-loop validation checkpoints at every stage. - [08:23](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=503s) Data hygiene is the primary operational bottleneck for enterprise AI deployments; unmanaged data bloat, oversharing, and lack of data ownership create severe breach and data exposure risks. - [12:25](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=745s) Cybersecurity controls must balance protection with workflow usability; overly restrictive containment causes end users to reject secure tools in favor of non-compliant workarounds. - [14:09](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=849s) Disciplined token economics requires matching tasks to appropriate model tiers, as only 10% to 20% of enterprise use cases genuinely require expensive frontier foundation models. - [18:41](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=1121s) Governing autonomous systems requires pre-execution 'truth guardians' and architectures where dedicated compliance bots monitor and police other operational agents in real time. - [23:54](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=1434s) Enterprises must build modular orchestration layers that decouple underlying foundation models from business logic to avoid vendor lock-in and enable seamless model swapping as price-performance benchmarks evolve. - [26:53](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=1613s) Applying generative AI to isolated steps of flawed legacy workflows creates 'optimized broken processes'; genuine transformation demands end-to-end domain reimagination. - [31:03](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=1863s) Premature workforce reductions after deploying AI risk catastrophic loss of tacit institutional knowledge, which is essential for training, calibrating, and overseeing autonomous agents. - [33:19](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=1999s) Implementing internal departmental chargebacks for AI token consumption and compute cycles turns theoretical governance into self-regulating financial discipline. - [35:51](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=2151s) Automated model routing should reserve expensive frontier models for high-level architectural decomposition while delegating routine boilerplate code generation to lightweight, cost-effective models. - [37:23](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=2243s) Rolling out AI tools via high-impact beachhead teams creates peer pull-demand across adjacent business units, proving far more effective than forced top-down mandates. - [39:44](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=2384s) Under the 'Chainsaw Principle,' leaders must avoid brandishing AI on problems that simpler deterministic software solves, pivoting focus instead toward unlocking top-line revenue by profitably serving previously cost-prohibitive markets. - [42:10](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=2530s) In high-stakes clinical environments, enterprise AI viability is measured strictly against tangible operational throughput—safely expanding clinician bandwidth without compromising care quality. - [43:08](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=2588s) Dynamically spawned ephemeral sub-agents with temporary access privileges represent an emerging cybersecurity crisis for auditability, identity lifecycle management, and forensic tracking.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=4s) Panel Introductions and Diverse Paths into AI

Moderator Rob Pegoraro opens the session with a nod of gratitude to the audience, joking that he appreciates their "taste and discretion" for picking this specific discussion out of the half-dozen other generative AI panels running concurrently. Acknowledging that the panel will not solve every enterprise problem or answer every question in forty-five minutes, he invites the panelists to introduce their companies and outline how long AI has been part of their professional lives.

Kathryn Harrison begins by introducing Concentrix, a Fortune 500 technology and services giant where she serves as Head of Strategy and AI Commercialization. Her journey in the space stretches back over a decade to working on IBM Watson initiatives, followed by founding four startups before joining Concentrix to focus on how artificial intelligence, human talent, and automation converge within the hybrid workforce. Max Gokhman, Head of AI and Digital Asset Solutions at Franklin Templeton, notes that under a true mathematical definition, AI has underpinned his entire career. A quantitative investor for more than twenty years, Gokhman was engineering Random Forest models and Markov chains prior to the 2008 financial crisis, later ran an AI hedge fund, and now focuses on deploying enterprise-scale AI architectures for global institutional and retail investors.

Pankaj Jain, CIO for International Operations at General Motors Financial, oversees technology strategy and execution across global markets in Mexico, South America, Europe, and the Asia-Pacific region. While GM Financial has leveraged predictive machine learning models for years, Jain notes that generative AI emerged as an intensive strategic focus over the past two years. Murad Dikeidek, Head of Cybersecurity for UI Health (the University of Illinois Hospital and Health Sciences System), brings twenty-five years of technical and executive cybersecurity experience to an academic medical network spanning a central hospital and twenty-six outpatient clinics dedicated to patient care, medical education, and clinical research. Wrapping up the intros, Pegoraro shares his perspective as a veteran technology journalist from Arlington, Virginia, who has reported on AI for three to four years while restricting its practical use in his day-to-day routine strictly to interview transcription: "Never to write. My clients don't like that."

[03:31](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=211s) High-Impact Enterprise Deployments: From Autonomous Counterpoints to Scaled HR

The conversation moves into concrete implementations, with panelists detailing production deployments and operational takeaways. Max Gokhman highlights Franklin Templeton’s first agentic digital co-worker, affectionately named "Gromit" after the cheese-loving claymation character. Gromit was conceived out of an organizational blind spot from Gokhman’s time as Chief Investment Officer of the firm's multi-asset team. Whenever the CIO walked into a room, productive debate tended to shut down because junior analysts hesitated to challenge leadership views. To restore cognitive friction, the firm built Gromit specifically to push back and offer contrarian counter-theses against human analysts. Over time, the agent evolved to ingest proprietary internal research and external market data, formulate independent analyses, and participate directly in live portfolio research meetings. As Gokhman puts it, while Gromit—like any human investor—is not always right, it "always admits when it's wrong," prompting Pegoraro to quip whether that makes Gokhman Wallace ("By definition").

Kathryn Harrison shifts to Concentrix's internal workforce transformation. Because Concentrix manages technology operations and customer experience for 2,000 enterprise clients, it deploys AI externally for customers while dogfooding internally across a massive base of more than 400,000 employees. Facing an annual recruiting requirement exceeding 50,000 hires across 75 global delivery locations and 150 languages, Concentrix introduced an agentic HR platform. The system evaluates resumes and applicant responses, provisions regional payroll accounts, and manages onboarding and training workflows end-to-end. The platform has shaved approximately two weeks off cycle times while elevating recruit satisfaction and output quality.

- **Concentrix HR Agentic Platform Outcomes**: - Global scope spanning 75 locations, 150 languages, and 50,000+ annual hires. - Onboarding velocity increased by approximately 14 days compared to legacy baselines. - New hire Net Promoter Score (NPS) improved significantly. - Output quality during early operational phases increased by roughly 50%.

[06:55](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=415s) Streamlining the Software Lifecycle with Multi-Agent Systems

Pankaj Jain shares GM Financial's flagship generative AI success: an agentic implementation across their Product Development Life Cycle (PDLC). Rather than relying on standard code-completion utilities, the engineering organization established a sequential pipeline using GitHub Copilot where specialized agents tackle distinct phases of the software release cycle. High-level business concepts are systematically decomposed into technical specifications, which are further refined into acceptance criteria written in Gherkin syntax. Subsequent agents generate application code, build unit tests, and compile acceptance test suites.

Jain underscores that the architecture is explicitly not designed to be a fully autonomous black box. Instead, human developers and quality engineers remain firmly in the loop at every transitional checkpoint, inspecting outputs, evaluating logic, and enforcing code quality. Currently deployed across select regional engineering squads, the framework has delivered measurable velocity enhancements, paving the way for wider international enterprise rollouts.

```mermaid flowchart LR accTitle: GM Financial Agentic PDLC Architecture subgraph Discovery["Requirements Phase"] A["High-Level Requirements"] --> B["Agent: Business Specs"] B --> C["Agent: Gherkin Acceptance Criteria"] end subgraph Engineering["Development & Testing"] C --> D["Agent: Code Generation"] D --> E["Agent: Unit & Acceptance Tests"] end subgraph Governance["Human Gatekeeping"] E --> F["Human Code Review & Validation"] F --> G["Production Release"] end ```

- **GM Financial PDLC Metrics**: - Multi-agent decomposition translating abstract requirements directly into Gherkin-formatted criteria. - Productivity improvements reaching up to 20% across participating pilot teams. - Human-in-the-loop checkpoints mandatory at every stage before code execution and merging.

[08:23](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=503s) Clinical EHR Automation and the Labor of Data Hygiene

Murad Dikeidek outlines UI Health’s operational initiatives, which focus on mitigating clinician burnout by embedding generative AI capabilities directly inside the Electronic Health Record (EHR) ecosystem. These tools automate administrative burdens such as clinical documentation, physician note summarization, and discharge summary generation, freeing physicians to spend more face-to-face time with patients. However, Dikeidek cautions that the true operational bottleneck lies not in the algorithmic front end, but in the grueling, behind-the-scenes data hygiene required before tools like Microsoft Copilot can safely touch institutional data.

"Your AI is as smart as your data," Dikeidek explains. Feeding unstructured institutional repositories into an autonomous search or Copilot engine without rigorous boundaries creates severe exposure risks. UI Health has had to wage a labor-intensive campaign against oversharing, legacy data bloat, and orphaned data stores lacking identifiable business owners. To prevent burnout within IT and security teams who cannot manually audit petabytes of clinical and corporate history, the organization deployed automated data security posture tools including Varonis and Microsoft Purview to systematically identify data owners, enforce strict data retention windows, remediate excessive permissions, and archive or permanently purge obsolete files. Dikeidek stresses that proactive data minimization is a non-negotiable prerequisite, because an enterprise breach involving AI-driven ingestion and discovery is orders of magnitude more catastrophic than a traditional database perimeter failure.

[10:36](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=636s) The Friction Between Clinical Demand and Cybersecurity Lockdown

Rob Pegoraro pivots the panel to explore initiatives that were evaluated and deliberately tabled because either the underlying technology was inadequate or the organizational culture was unready. Dikeidek responds with a candid look at the cultural tensions inside an academic medical center. On one side are digital-native younger physicians eager to push operational boundaries and demanding immediate clearance to run patient charts through enterprise tools like Microsoft Copilot; on the other side are senior clinicians who prefer legacy workflows and resist technological changes entirely.

When young clinicians repeatedly pressed cybersecurity for a concrete date when Copilot would be cleared for protected health information (PHI), Dikeidek stood firm: the default Copilot corporate environment simply carries too much oversharing risk, and remediating the sprawl across IT and non-IT business units will take considerable time. Seeking a pragmatic compromise, leadership evaluated specialized, ring-fenced tools isolated entirely from the broader Microsoft 365 tenant, eventually staging a formal proof-of-concept with ChatGPT for Healthcare. During the vendor demonstration, the clinical staff was immediately impressed by the model's capabilities. However, once Dikeidek began interrogating the technical controls and discovered just how tightly locked down the environment was, a sharp ideological divide surfaced. As a cybersecurity leader, Dikeidek was thrilled by the strict containment, but the clinicians found that the very guardrails preventing data exfiltration stripped away the seamless convenience they had envisioned.

[12:25](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=745s) Security as a Balancing Act and Abstracting Agentic Platforms

Continuing his discussion on tool adoption at UI Health, Murad Dikeidek explains that medical teams chose not to adopt ChatGPT for healthcare because they prioritized collaboration, sharing data, creating dashboards, and presenting insights to others. This reinforced a key lesson: no matter how effective or secure a tool is, if it does not fit the organization's workflow needs, end users simply will not adopt it. Dikeidek notes that cybersecurity is inherently a balancing act where security leaders must occasionally compromise on the level of friction to enable the business. While UI Health is temporarily reassessing their strategy, they are looking back at securing Microsoft Copilot to meet user expectations.

Pankaj Jain steps in to discuss why General Motors Financial approaches agentic workflows with deliberate caution. When software agents are empowered to take actions on behalf of users, governance, policy enforcement, and security become critical. Rather than letting individual developers invent disparate workarounds and "proliferate bad practices across the enterprise," the company is focusing on building a centralized agentic AI platform that abstracts the underlying plumbing, guardrails, and controls at scale.

```mermaid flowchart TD accTitle: Enterprise Agentic Governance Architecture accDescription: Diagram showing how agentic actions pass through proactive policy guardians before execution.

subgraph UserLayer["Initiation"] User["User Request"] --> Agent["AI Agent"] end

subgraph PlatformLayer["Central Agentic AI Platform"] Agent --> Guardians["Truth Guardians & Policy Checks"] Guardians --> Engine["Agent Action Execution Engine"] end

subgraph AuditLayer["Governance & Compliance"] Engine --> AuditBot["Compliance Auditing Bot"] AuditBot --> Log["Audit Trail / Human Oversight"] end ```

---

[14:09](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=849s) Token Economics, Frontier Models, and Micro-Judgments

Kathryn Harrison shifts the focus to the critical topic of "token economics," sharing common enterprise pitfalls where organizations blow through their entire annual token budget in a matter of months, weeks, or even two days. She highlights the widespread modern habit of generating a 25-page document in Claude only for the recipient to run it through ChatGPT to generate a summary to avoid reading it. To manage these realities, Concentrix's CIO and CFO treat model consumption as a disciplined capital allocation decision. Across their 400,000-person global workforce, they segment users to match specific tasks with the right model tier, noting that only 10% to 20% of use cases actually require top-tier frontier models.

Harrison admits that "FOMO" frequently tempts people to deploy Anthropic's Claude Opus for complicated analyses when Claude Sonnet would have been entirely sufficient. Moreover, in an organization with a large global operational footprint, human labor is frequently more cost-effective than running high-end AI inference. Harrison stresses that beyond raw ROI, leaders must continue developing their human teams: while AI excels at synthesizing intelligence and information, it currently lacks the nuanced "micro judgments" that humans rely on to run an enterprise at scale.

- **Workforce Scale**: Concentrix manages a 400,000-employee global footprint where unit economics dictate when human labor is cheaper than running high-tier models. - **Model Matching**: Only 10% to 20% of enterprise use cases require true frontier foundation models (e.g., Opus over Sonnet).

---

[16:07](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=967s) Managing Non-Deterministic Risk in Financial Markets and Workforces

Rob Pegoraro asks how organizations navigate the non-deterministic nature of AI and the uncertainty of computing costs. Max Gokhman relates this directly to investment management at Franklin Templeton, observing that dealing with uncertainty is standard practice for investors. Quants have long tried and failed to build a perfect "crystal ball" for financial markets, meaning the goal is not to solve the entire market, but to capture durable edges.

Gokhman explains that whether an organization is developing quantitative strategies or deploying agentic analysts and portfolio managers, backtesting AI remains extraordinarily difficult. Asking a foundation model to simulate past trades without leveraging its pre-existing knowledge is identical to asking a human trader to re-live the financial crisis while pretending not to know the eventual outcome. To mitigate this, teams must build experimental sandbox environments to discover edge cases. Gokhman emphasizes that while leaders accept that human hires will occasionally make mistakes and manage them via governance and compliance, they must now cultivate a similar comfort with agentic co-workers by implementing robust guardrails for critical failure modes.

---

[18:41](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=1121s) Bots Policing Bots and Mitigating the Human "Telephone Game"

Kathryn Harrison shares an experiment where Concentrix deployed an agentic digital worker built on OpenClaw alongside her executive assistant. To manage operational risks, Concentrix implemented a multi-layered system, creating dedicated compliance bots designed specifically to audit and monitor agent actions in real time—establishing an architecture of "bots to police the bots." Harrison illustrates the system's efficiency: when a new executive joined, the agent scheduled 25 meetings across eight distinct time zones in just 10 minutes, a complex puzzle that would have taken hours for a top-tier human assistant. Max Gokhman underscores this principle, noting that "you will need agentic governance systems to control agentic systems of action."

Pankaj Jain highlights the engineering challenge behind this architecture: governance cannot be applied retroactively after an agent has already executed a task or made a blunder. Instead, enterprises must deploy pre-execution "truth guardians" to validate hundreds of policies before an agent takes action. Jain also shares a surprising finding from value stream analyses: fully automated agentic workflows often exhibit higher overall accuracy than human-centric workflows. This disparity is rarely due to individual human error, but rather the compounding loss of fidelity and manual miscoordination that occurs across multiple hand-offs—what is commonly known as the "telephone game."

---

[22:05](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=1325s) Automating Third-Party Risk and Breach Triage in Healthcare

Murad Dikeidek explains that while UI Health does not build custom agents for clinical healthcare, their technical staff actively develop custom GPTs to handle cybersecurity operations, particularly third-party risk management. Healthcare organizations outsource massive shares of their administrative footprint, including patient scheduling and billing. Consequently, roughly 90% of healthcare data is housed in external vendor environments rather than on-premises hospital data centers, making third-party risk far more pressing than internal user activity alone.

To streamline assessments, UI Health uses the Black Kite platform to ingest and analyze SOC 2 and HIPAA compliance audit documents, immediately identifying where vendor security controls are lacking and where remediation is mandatory. Furthermore, when a third-party vendor suffers a security incident, UI Health uses custom GPTs to crawl the web, aggregate incident reports from across multiple technical sources, determine organizational exposure, and synthesize actionable incident response briefs outlining immediate remediation steps.

| Category | UI Health Third-Party Risk Implementation | |:--- |:--- | | **Data Landscape** | ~90% of healthcare data resides externally with outsourced vendors (billing, appointments). | | **Audit Automation** | Ingests SOC 2 and HIPAA documents into Black Kite to evaluate posture and mandate fixes. | | **Breach Response** | Custom GPTs scrape external reports on vendor breaches to assess impact and draft next-step response playbooks. |

[23:54](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=1434s) Avoiding Vendor Lock-In and Building an Orchestration Layer

Addressing the conference floor packed with software vendors showcasing "shiny new things," Max Gokhman warns enterprise leaders above all else not to fall into vendor lock-in. Because underlying artificial intelligence technology evolves at an unprecedented pace, any tool that seems to offer a flawless fit today can quickly become obsolete. Systems must be engineered modularly from day one so individual components can be swapped in and out. Gokhman also urges buyers to scrutinize the post-sale support models of smaller startups. While large enterprises like Franklin Templeton have stringent legal and compliance hurdles that filter out most immature vendors, any enterprise needs a direct human lifeline when deployments fail—unless an organization is advanced enough to have created autonomous agents capable of debugging external vendors' code.

Kathryn Harrison underscores this warning by sharing Concentrix’s own early missteps. The company dove into building internal AI tools during the era of GPT-3 without architecting an abstraction or orchestration layer. When newer foundation models swiftly eclipsed existing capabilities, the engineering team was forced into a painful, ground-up re-architecture—a fate Harrison sees recurring across the industry. Today, enterprise clients frequently find themselves drowning in upwards of 25 disjointed point solutions, each performing an isolated micro-task. Concentrix counters this fragmentation by implementing an orchestration layer that decouples the underlying models, harnesses, and operational tooling from the business logic, enabling companies to swap providers seamlessly as frontier labs leapfrog one another in the ongoing arms race.

```mermaid flowchart TD accTitle: Enterprise AI Orchestration Layer Architecture subgraph Client["Business Operating Model"] A["Business Goals & End Outcomes"] --> B["Flexible Orchestration Layer"] end subgraph Vendors["Interchangeable Modular Tech Stack"] B --> C["Foundation Models\n(GPT, Claude, Open-Source)"] B --> D["Domain Tooling & Harnesses"] B --> E["Vendor Point Solutions"] end ```

- **Vendor Onboarding Thresholds:** Early-stage vendors frequently struggle to satisfy the compliance, security, and legal scrutiny demanded by institutional enterprise procurement. - **Architectural Flexibility:** Building an internal orchestration layer insulates enterprises from model deprecation and rapid industry turnover, letting teams switch foundation models based on evolving price-to-performance benchmarks.

---

[26:53](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=1613s) Managing Expectations: Optimizing Broken Processes vs. Domain Reimagination

Pankaj Jain highlights a pervasive expectations mismatch between executive suites and delivery teams. C-suite leaders frequently demand transformational corporate windfalls from initiatives that do nothing more than apply basic optimizations to isolated steps of existing workflows. Jain cautions that automating a segment of a fundamentally flawed process merely creates an "optimized broken process." To deliver genuine enterprise value, technology leaders must shoulder the responsibility of setting realistic baselines, quantifying expected business benefits up front, monitoring them across the project lifecycle, and explicitly demonstrating post-deployment value in production.

Achieving true transformational ROI requires what Jain terms "domain reimagination"—scrutinizing end-to-end business operations rather than bolting automated widgets onto legacy pathways. In this framing, AI serves as an enabling component within a broader structural redesign rather than an all-encompassing silver bullet. The outsized financial returns originate from fixing and reshaping the end-to-end operational architecture first, with AI acting as a performance multiplier.

- **The Automation Trap:** Applying generative AI to single bottlenecks inside an unoptimized department locks in structural inefficiencies under the guise of modernization. - **Production Value Audits:** Successfully managing leadership expectations requires formal tracking mechanisms that measure tangible business results in live production rather than relying on pilot demonstrations.

---

[28:17](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=1697s) Trust, Human-in-the-Loop, and Navigating Enterprise Messiness

In high-stakes environments like healthcare, Murad Dikeidek explains that automated autonomy is non-viable because clinical errors directly endanger patient lives. At UI Health, adoption within the radiology department relies on AI strictly as an assisting validator rather than a diagnostic decision-maker. Radiologists perform primary image evaluations independently, while the AI software acts as a secondary pair of eyes that surfaces potential anomalies the clinician might have overlooked. Framing the model as an assistive peer rather than an autonomous authority ensures that diagnostic accountability remains anchored with licensed human experts.

Responding to Rob Pegoraro's question on whether AI simply exposes pre-existing dysfunction, Kathryn Harrison agrees that corporate environments are inherently messy networks that traditionally run on informal relationships—relying on staff "knowing the right person in this department" to track down a crucial data point. Attempting to bolt generative models directly onto this chaos backfires. Concentrix developed an "agent assist" platform specifically designed to liberate frontline operators from toggling between dozens of open tabs to synthesize fragmented enterprise knowledge. Instead of displacing human judgment, assistive tooling summarizes live case history and recommends next best actions, converting tedious data retrieval into streamlined operational workflows.

- **Assistive Clinical Deployment:** Radiologists retain primary authority over scans; generative tools act solely as check-and-balance validation layers to catch missed indicators. - **Frontline Workflow Relief:** Enterprise assist tooling directly targets cognitive overload by automating cross-system data retrieval and surfacing recommended next actions.

---

[31:03](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=1863s) Preserving Institutional Knowledge, Workforce Evolution, and Agent Farms

Max Gokhman cautions against the superficial appeal of AI-generated deliverables, noting that executives often review a polished investment report and rashly conclude human analysts are expendable. Because top-level managers do not perform day-to-day modeling in the trenches, they fail to recognize when underpinning assumptions lack sound financial intuition. Some enterprises have aggressively cut staff only to face an operational crisis, forcing them to rehire domain specialists once they realized their proprietary business context had walked out the door. Specialized internal IP does not come bundled with off-the-shelf software; human professionals must actively train and calibrate agents over long periods.

Securing employee buy-in requires articulating rewarding career trajectories rather than asking staff to "train their successor and get fired." At Franklin Templeton, investment analysts are shown that delegating spreadsheet assembly to agents frees them to spend far more time engaging corporate management teams out in the field. Pankaj Jain notes that this collaborative dynamic is already reshaping software engineering scrum teams, where developers now pair-program alongside AI agents rather than human peers. As organizations transition toward deploying multi-agent "agent farms," comprehensive change management becomes vital to help employees cultivate productive, durable working relationships alongside autonomous systems.

- **The Rehiring Rebound:** Premature workforce reductions strip organizations of tacit domain expertise that cannot be replicated by foundational models without extensive human tuning. - **The Evolving Knowledge Worker:** Modern role definitions transition analysts from manual model builders to strategic field evaluators, reframing AI adoption as career elevation rather than role elimination. - **Agent Farms in Software Delivery:** Modern engineering teams are moving from human-to-human pair programming to agentic copilot models, requiring structured organizational change management.

---

[33:19](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=1999s) Data Governance, Workload Routing, and Cost Accountability

Murad Dikeidek revisits the foundational role of information hygiene, asserting that generative AI has forced organizations to confront data governance deficiencies they have neglected for decades. Employees routinely generate unmonitored data silos with zero lifecycle management, undefined ownership, and indefinite retention windows. Because generative models are intensely data-dependent, using low-quality, ungoverned source data inevitably pollutes model outputs.

Addressing how enterprises should divide workloads between massive public LLMs, specialized narrow models, and secure on-premise deployments, Kathryn Harrison outlines the governance architecture powering Concentrix’s ecosystem across 2,000 corporate clients. The company maintains strict guardrails: call center personnel operate within constrained internal interfaces tied strictly to specific client contracts, whereas product developers have access to broader, more powerful development toolsets. Balancing this framework from an operational standpoint, Pankaj Jain argues that model routing and resource discipline are best enforced through direct internal chargebacks. Forcing business units to fund their own API queries and compute cycles transforms theoretical governance into immediate, self-regulating financial discipline.

```mermaid flowchart TD accTitle: Workload Routing and Cost Control Framework subgraph Ingestion["Workforce & Use Case Segmentation"] A["Call Center Agents"] --> B{"Policy Guardrails"} C["Software Developers"] --> B end subgraph Routing["Workload Dispatch Engine"] B -->|"Restricted / Sensitive Data"| D["Narrow Specialized Models\n(Secure / Contract-Bound)"] B -->|"Exploratory / Engineering"| E["Frontier LLMs\n& Tooling"] end subgraph Governance["Cost & Quality Controls"] D --> F["Internal Chargeback Mechanism"] E --> F F --> G["Accountable Business Unit Spend"] end ```

- **Data Governance as a Prerequisite:** AI amplifies the hidden costs of data hoarding and unmanaged storage, demanding rigorous data hygiene before algorithmic implementation. - **Contract-Tiered Guardrails:** Customer service tools are isolated behind access controls governed by customer contractual constraints, while engineering groups receive wider system latitudes. - **Enforcing Discipline via Chargebacks:** Instituting internal cost allocation models ensures department heads consciously optimize model sizing, context windows, and call frequency against departmental budgets.

[35:51](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=2151s) The End of the "Unlimited Buffet": Model Routing and Token Budgets

Addressing the reality of managing enterprise AI consumption over an entire fiscal year, Pankaj Jain stresses that continuous organizational education is essential. With AI capabilities evolving at breakneck speed, tools and techniques considered optimal yesterday quickly become obsolete today. More importantly, the commercial landscape is fundamentally shifting away from all-you-can-eat access: platforms like GitHub Copilot are moving aggressively toward usage-based pricing models. Enterprises can no longer treat generative AI as an "unlimited buffet" without burning through annual budgets within the first quarter.

To curb wasteful consumption without paralyzing technical velocity, Jain highlights automated LLM routing within agentic workflows. In automated pipelines, the orchestrator can mandate routing to lower-cost, purpose-built models based on task type without needing human intervention. Developer workflows, however, require a more nuanced balance. During high-level stages of the product development lifecycle (PDLC)—such as decomposing complex system requirements into technical specifications—frontier models are indispensable because of the high fidelity and contextual reasoning they provide. Once that decomposition is complete and the task turns into straightforward boilerplate code generation, lower-tier models should take over. Jain emphasizes that enterprises cannot simply block developers from accessing frontier models; instead, they must actively educate engineers to choose the right model tier for the right task.

```mermaid flowchart TD accTitle: Enterprise Model Routing and Soft Token Budget Governance subgraph Routing["Automated Agentic Routing"] T["Inbound Engineering Task"] --> C{"Task Fidelity Requirement"} C -- "High Reasoning (Architecture & Decomposition)" --> F["Frontier Model (High Cost)"] C -- "Standard Execution (Code Generation)" --> L["Lower-Tier Model (Cost Efficient)"] end subgraph Governance["Developer Token Governance"] U["Developer Token Usage"] --> S{"Soft Budget Cap Exceeded?"} S -- "No" --> P["Proceed with Task"] S -- "Yes" --> J["Human Permission Gate:<br/>Submit Justification"] J --> G["Grant Token Extension"] end ```

- **Usage-Based Cost Pressures:** Enterprise developer tools are deprecating flat-rate licensing in favor of metered token and usage charges, requiring continuous cost governance across development quarters. - **Fidelity vs. Execution Tiering:** Frontier models are reserved for complex decomposition and requirement definition, while routine code synthesis is offloaded to lighter, economical models.

[37:23](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=2243s) Organic Adoption via Beachheads and Top-Down Executive Accountability

Max Gokhman shares Franklin Templeton’s operational experience deploying their internal tool, Grommet, underscoring that human adoption rarely happens by default. When Grommet was first rolled out to a single pilot investment team, the initial pushback was palpable: "Why do we need to talk to this chatbot?" Gokhman intervened with a mix of direct executive authority and reassurance, telling the team: "Guys, it's not a chatbot, it's actually going to really help you. And also, I'm not asking, I'm telling." Once the team began using Grommet daily, the reaction flipped to genuine enthusiasm as it noticeably improved their job performance. That success sparked organic demand across Franklin Templeton: other investment teams noticed the edge their peers had gained and demanded custom versions of the tool for their own desks, naturally turning the pilot group into an internal sales force.

Turning to developer governance, Gokhman advocates for soft token budgets rather than rigid hard caps. Enforcing a soft cap introduces an intentional human friction point: developers who exceed their quota must articulate why their project requires additional tokens or demands frontier model intelligence. Gokhman enforces this exact discipline upon himself, arguing that having to explain and justify resource requests establishes top-down behavioral restraint.

Kathryn Harrison builds on this principle, emphasizing that organizational AI discipline flows directly from executive behavior. At Concentrix, the global executive committee actively tracks how functional heads across HR, IT, and product development utilize generative tools and communicate AI strategy to their staff. Supported by structured corporate training academies, Concentrix is experimenting with emerging quality and efficiency metrics, though Harrison admits that enterprise measurement remains an evolving discipline: "Anyone who says that they have that figured out, I think is probably lying." Simply being tracked and required to report on AI usage compels leaders and their teams to interact with the technology far more responsibly.

- **The Beachhead Rollout Strategy:** Releasing specialized tooling to a single high-impact cohort creates peer envy and authentic pull-demand across adjacent business units, sidestepping forced enterprise-wide mandates. - **Soft Governance Gates:** Requiring developers and leaders to submit a brief business justification before lifting token caps prevents runaway cloud expenses while keeping critical workflows unblocked.

[39:44](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=2384s) Beyond Cost Cutting: Driving Top-Line Revenue and Avoiding the "Chainsaw" Trap

Moderator Rob Pegoraro observes that selecting the initial beachhead for AI deployment may hinge less on immediate bottom-line savings and more on cultivating credible internal evangelists who can share operational lessons. Harrison agrees, noting that while most organizations reflexively hunt for head-count reductions or automated hours saved, Concentrix found far greater strategic leverage by targeting power users—such as developers and complex data analysts—and deploying their agentic layer directly toward revenue generation. By utilizing autonomous agents, the company can now profitably service customer segments and market tiers that were historically cost-prohibitive to support with human staff alone.

Gokhman validates the pursuit of top-line opportunities alongside baseline productivity gains, but offers a blunt warning forged from past implementation failures: never build a solution searching for a problem. Leaders must build tools either because clients are actively requesting them or because deep domain expertise proves the business genuinely requires them, followed by rigorously vetting whether AI is even the appropriate tool.

To illustrate the danger of generative hype, Gokhman compares AI to a chainsaw: chainsaws are incredibly powerful, impressive, and fun to run, but most everyday household chores do not require a chainsaw, nor do they require any saw at all. Enterprises frequently make the mistake of brandishing AI like a shiny new toy across problems that demand far simpler, deterministic software solutions. Only after verifying that a high-value operational bottleneck exists and confirming that AI is uniquely suited to address it should engineering teams proceed to build prototypes.

- **Top-Line Growth vs. Cost Cutting:** Shifting AI initiatives from defensive cost reductions to offensive revenue generation unlocks new addressable markets by lowering the marginal cost of customer servicing. - **The Chainsaw Principle:** Avoid deploying generative AI simply because the technology is novel; validate client demand and test whether traditional software solves the problem before allocating engineering resources.

[42:10](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=2530s) The Cybersecurity Reality Check and Clear Healthcare Metrics

Murad Dikeidek brings the perspective grounded in enterprise security and clinical healthcare, noting that enterprise enthusiasm frequently outpaces practical clarity. He recounts seeing a humorous industry poster that captured the current corporate mindset perfectly: "Everyone wants AI. What for they do not know and they want it now." From a cybersecurity standpoint, this blind eagerness is a massive vulnerability. Security teams cannot secure an abstract, undefined technology; they require explicit, granular technical use cases to architect appropriate data boundaries, access controls, and threat monitoring.

In a high-stakes clinical environment like UI Health, adoption decisions cannot afford ambiguity or vanity metrics. Dikeidek explains that the hospital evaluates generative tooling against one unambiguous operational benchmark: patient throughput. If an AI platform measurably assists physicians and clinicians in examining, diagnosing, and treating a higher volume of patients without degrading quality of care, the tool delivers unquestioned enterprise value. If it fails to move the needle on clinician capacity and patient care delivery, it is dismissed regardless of how technically sophisticated it claims to be.

- **Granular Security Architecture:** Enterprise defense teams must mandate that business units document specific operational workflows before AI tools receive enterprise data access and environmental approval. - **Throughput as Clinical North Star:** In healthcare settings, tool viability is judged squarely on its ability to expand clinician bandwidth and safely accelerate patient throughput.

[43:08](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=2588s) Future Outlook: Agentic Middle Managers, Ephemeral Audits, and Human Empathy

Wrapping up the discussion, Pegoraro invites the panel to look ahead twelve months and identify the emerging capabilities they find most exciting or most alarming. Gokhman highlights the coming integration of autonomous AI agents directly into corporate organizational charts alongside human workers. Specifically, he foresees agents assuming middle-management responsibilities: while an agent cannot match someone's best human mentor, it easily outperforms their worst toxic boss. An AI agent manager operates completely free of corporate politics, remains accessible 24/7, maintains parallel personalized conversations across an entire department simultaneously, and optimizes workload distributions dispassionately—a shift Gokhman considers fascinating from an organizational psychology perspective.

This vision sparks lively debate across the panel regarding control, safety, and human purpose:

```mermaid flowchart TD accTitle: Future Enterprise Agent Horizons and Strategic Tensions subgraph Opportunities["Transformative Upside"] M["Autonomous Middle Managers<br/>(Objective, 24/7, De-politicized)"] H["Elevation of Human Labor<br/>(Judgment, Empathy, High Job Satisfaction)"] end subgraph Vulnerabilities["Systemic Enterprise Risks"] S["The Skynet Dilemma<br/>(Over-autonomous runaway agent loops)"] E["Ephemeral Agent Sprawl<br/>(Micro-agents created and destroyed in seconds)"] end ```

Pankaj Jain cautions against unchecked autonomy, invoking the cultural specter of Skynet to emphasize that enterprise agents must function strictly as assistants rather than autonomous overlords running organizations. Murad Dikeidek flags a more immediate, acute cybersecurity nightmare: agents autonomously spawning other sub-agents. When an agent spins up daughter micro-agents that possess their own temporary credentials, data access rights, and operational independence—executing tasks for a few seconds or minutes before vanishing entirely—it creates an audit and forensics crisis. If one of those ephemeral agents misbehaves, leaks protected data, or violates compliance, tracing provenance and shutting down the offending identity before it disappears will push conventional security tooling to its absolute limits.

Closing the panel on a resoundingly optimistic note, Kathryn Harrison rejects both the dystopian Skynet framing and widespread fears of occupational doom. She argues that offloading the grueling, repetitive, and hated operational grind to autonomous agents will dramatically elevate employee job satisfaction. Stripping away operational drudgery frees human professionals to focus exclusively on uniquely human gifts: deep judgment, complex creative synthesis, and genuine personal empathy. Pegoraro thanks the panelists for charting a pragmatic playbook centered on sustainable value, realistic governance, and tangible human empowerment.

- **Autonomous Middle Management:** Algorithmic supervisors could eliminate political bias and optimize workflow scheduling across large distributed teams around the clock. - **Ephemeral Micro-Agent Security Risks:** Daughter agents dynamically generated for temporary sub-tasks represent an unprecedented governance challenge for access tracking, compliance auditing, and identity lifecycles. - **Reclaiming Human-Centric Work:** Automating administrative friction promises to refocus enterprise workers on high-value human judgment, strategic problem-solving, and empathetic customer relationships.

## Notable quotes > [00:04](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=4s) Never to write. My clients don't like that. — Rob Pegoraro

> [03:31](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=211s) always admits when it's wrong — Max Gokhman

> [08:23](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=503s) Your AI is as smart as your data — Murad Dikeidek

> [12:25](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=745s) proliferate bad practices across the enterprise — Pankaj Jain

> [18:41](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=1121s) you will need agentic governance systems to control agentic systems of action. — Max Gokhman

> [26:53](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=1613s) optimized broken process — Pankaj Jain

> [26:53](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=1613s) domain reimagination — Pankaj Jain

> [31:03](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=1863s) train their successor and get fired — Max Gokhman

> [37:23](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=2243s) Why do we need to talk to this chatbot? — Max Gokhman

> [37:23](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=2243s) Guys, it's not a chatbot, it's actually going to really help you. And also, I'm not asking, I'm telling. — Max Gokhman

> [37:23](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=2243s) Anyone who says that they have that figured out, I think is probably lying. — Kathryn Harrison

> [42:10](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=2530s) Everyone wants AI. What for they do not know and they want it now. — Murad Dikeidek

## Mentioned Rob Pegoraro, Pankaj Jain, Murad Dikeidek, Kathryn Harrison, Max Gokhman, PCMag, Fast Company, General Motors Financial, UI Health, Concentrix, Franklin Templeton, IBM Watson, Gromit, GitHub Copilot, Gherkin, Microsoft Copilot, Varonis, Microsoft Purview, ChatGPT, OpenClaw, Anthropic, Claude Opus, Claude Sonnet, Black Kite, GPT-3

## Video description Rob Pegoraro - Contributor, PCMag / Fast Company Pankaj Jain - CIO - International Operations, General Motors Financial Murad Dikeidek - Head of Cybersecurity, UI Health Kathryn Harrison - Global Vice President, Strategy & Business Operations, Concentrix Max Gokhman - Head of Artificial Intelligence and Digital Asset Solutions, Franklin Templeton

Generative AI presents a transformative opportunity, but only if executed with clarity and discipline. This panel will present a practical playbook for enterprise leaders looking to scale generative AI initiatives effectively. From identifying the right starting use cases to building governance frameworks and adopting the best technology, panelists will walk through the key ingredients for generative AI success. Learn how to avoid common missteps, prioritize real business value, and set the stage for sustainable innovation. Whether you're early in your journey or scaling aggressively, this panel will offer strategic guidance and operational know-how to help you win with gen AI.

[Source](https://www.youtube.com/watch?v=DsQS-X5vwGc)

中文简报

通用汽车与富兰克林邓普顿设立严格Token上限以管控企业AI 企业领袖警告称,自动化有缺陷的传统工作流只是在优化本就失效的流程,并敦促实施严格的模型路由和执行前治理以防成本失控。 多位企业技术领袖分享了具体的落地部署案例,涵盖富兰克林邓普顿的反向投资智能体、通用汽车金融的多智能体开发流水线、UI Health的EHR临床自动化以及Concentrix的智能体HR平台。讨论强调,可持续的企业价值取决于部署前严格的数据卫生、多层级模型路由和执行前治理护栏,而非盲目跟风前沿模型炒作。与会专家告诫切勿直接自动化从根本上存在缺陷的工作流程,敦促企业聚焦于业务领域重塑、基于用量的成本纪律以及业务增收,而非单纯裁员。展望未来,小组权衡了瞬态微智能体与算法中层管理带来的新兴治理挑战,以及人类同理心与细致微观判断的持久必要性。 Franklin Templeton 部署了名为“Gromit”的智能体数字同事,专门在投研会议中提出逆向观点,有效打破下属不敢质疑领导层意见的职场沉默。 (3:31) GM Financial 将软件生命周期拆解为多智能体流水线,将业务需求转为 Gherkin 验收标准并生成代码,同时在每个环节强制执行人工在环(HITL)验证,试点团队生产力提升高达 20%。 (6:55) UI Health 指出数据治理是 AI 落地的首要瓶颈,数据臃肿和过度共享会带来巨大合规风险,且网络安全防范必须兼顾工作流可用性,否则过于封闭的工具会被业务人员弃用。 (8:23) Concentrix 强调代币经济学(Token Economics)管理,指出仅有 10% 到 20% 的企业用例真正需要昂贵的前沿基座模型,应按任务匹配合适模型层级并建立多层监控机制。 (14:09) 企业必须构建解耦底层模型与业务逻辑的模块化编排层以避免供应商锁定;将 AI 仅用于修补旧流程只会制造“被优化的破损流程”,真正的转型需要端到端的业务重构。 (23:54)

Reading summary

Enterprise technology and cybersecurity leaders argue that scalable generative AI adoption requires modular orchestration architectures, rigorous data hygiene, and proactive token governance rather than indiscriminate model deployment. Realizing measurable production returns depends on establishing pre-execution policy guardrails and human checkpoints while reimagining end-to-end operational processes. Modular Architectures and Controlled Multi-Agent Systems (6:55) Enterprise implementations are transitioning from standalone code-assist utilities to orchestrated, multi-agent frameworks. At General Motors Financial, CIO for International Operations Pankaj Jain describes structuring the product development lifecycle into a sequential pipeline using GitHub Copilot: specialized agents decompose abstract requirements into Gherkin-formatted acceptance criteria, synthesize code, and compile test suites, yielding productivity gains of up to 20% on pilot teams while maintaining mandatory human-in-the-loop review gates. To prevent platform fragility, panelists warn against tight coupling to specific model providers. Kathryn Harrison of Concentrix emphasizes that decoupling business logic from underlying models via an orchestration layer protects enterprises from costly rewrites as foundation models leapfrog one another. Concentrix deployed an agentic platform across 75 global delivery locations and 150 languages, supporting over 50,000 annual hires and reducing onboarding cycle times by approximately 14 days. At Franklin Templeton, Head of AI and Digital Asset Solutions Max Gokhman developed 'Gromit,' an agent engineered to provide contrarian research counter-theses during portfolio meetings to mitigate human deference to executive authority. Data Hygiene, Third-Party Risk, and Clinical Containment (8:23) Deploying generative models in high-stakes environments exposes long-standing information management vulnerabilities. Murad Dikeidek, Head of Cybersecurity at UI Health, cautions that algorithmic utility is limited by underlying data cleanliness. To prevent autonomous engines from exposing sensitive protected health information across corporate tenants, UI Health deployed automated governance tooling, including Varonis and Microsoft Purview, to enforce retention windows, identify data owners, and purge obsolete records before permitting tools like Microsoft Copilot to access internal repositories. Dikeidek highlights that tight containment often introduces severe friction with operational user requirements. In a trial of ChatGPT for Healthcare, clinicians praised the model's analytical capability but rejected adoption because the strict security ring-fencing prevented data sharing and dashboard collaboration. Furthermore, because approximately 90% of healthcare administrative data resides in external vendor environments, UI Health utilizes custom GPTs alongside Black Kite to parse third-party SOC 2 and HIPAA audit reports and automate security incident triage. In clinical diagnostics like radiology, UI Health restricts AI strictly to secondary validation to maintain human accountability. Token Economics, Workload Routing, and Process Reimagination (14:09) Panelists emphasize that the transition of enterprise software to usage-based pricing models marks the end of all-you-can-eat AI consumption. Harrison reports that only 10% to 20% of enterprise use cases warrant top-tier frontier models like Claude Opus over Claude Sonnet, noting that within Concentrix's 400,000-person global footprint, human labor remains more cost-effective than running high-end inference for certain repetitive tasks. To avoid depleting annual token allocations prematurely, GM Financial relies on automated routing pipelines that assign frontier models to high-reasoning tasks—such as requirements decomposition—while delegating boilerplate generation to lower-cost models, reinforcing discipline through departmental chargebacks. Technology leaders also warn against automating flawed legacy pathways. Jain argues that bolting generative tools onto unoptimized steps produces an 'optimized broken process,' urging organizations to measure production benefits against formal baselines. Gokhman compares generative AI to a chainsaw: an effective tool for heavy logging that is fundamentally mismatched for standard household tasks. Franklin Templeton enforces soft token caps requiring developers to submit brief business justifications before exceeding compute thresholds, curbing speculative usage without stalling critical engineering. Agentic Governance, Workforce Continuity, and Ephemeral Risks (18:41) As autonomous agents take direct action across enterprise systems, post-execution audits become insufficient. GM Financial and Concentrix both emphasize pre-execution 'truth guardians' and real-time compliance bots designed to validate organizational policies before agents act. Harrison notes that automated agent workflows can achieve higher fidelity than human workflows by eliminating the compounding communication errors seen across multi-stage manual hand-offs. Concurrently, Gokhman cautions that cutting human analysts based on polished AI outputs risks discarding critical institutional context that foundation models cannot replicate, creating a cycle where firms are forced to rehire specialists. Looking ahead, Gokhman forecasts AI agents operating as dispassionate middle managers capable of allocating workloads and communicating across departments around the clock without interpersonal politics. Conversely, Dikeidek identifies an emerging security vulnerability: ephemeral micro-agents autonomously spawning daughter sub-agents with temporary credentials. Because these dynamic agents execute tasks and dissolve within minutes or seconds, tracing data provenance and investigating compliance violations threatens to outpace conventional enterprise audit and identity governance tooling. While panelists anticipate that delegating administrative friction to generative agents will allow human workers to focus on creative synthesis, judgment, and empathy, unresolved operational tensions remain. Organizations must resolve the trade-off between user friction and data security, while developing new forensic architectures capable of auditing non-deterministic, short-lived autonomous agents before granting them deeper administrative authority.

精炼阅读摘要

在Ai4主办的研讨会上,来自金融、医疗与技术服务领域的企业高管指出,生成式AI的应用重心已从前期的试验性探索全面转向架构解耦、严谨的数据治理与精细化成本控制。与会嘉宾一致认为,企业获取实际投资回报的关键在于重新审视并重塑端到端业务流,而非用AI去缝补本已低效的破碎流程,同时必须在扩大代理自主权的同时建立严格的事前守卫与人在回路机制。 业务流程重塑与多代理落地实践 (3:31) 讨论嘉宾强调,生成式AI创造价值的前提是重构完整业务流程,避免落入给孤立环节打补丁的陷阱。通用汽车金融(GM Financial)国际运营CIO Pankaj Jain介绍了其软件产品开发生命周期(PDLC)的多代理架构:该方案利用GitHub Copilot等工具将宏观需求拆解为Gherkin语法规范,再由专用代理生成代码和验收测试用例,使试点团队效率提升达20%,并在每个节点保留人工审核以确保逻辑与代码质量。富兰克林邓普顿(Franklin Templeton)AI高管Max Gokhman则分享了其内部数字员工“Gromit”,该代理专门通过反向论证来打破投资团队因层级差异形成的思想惯性,直接作为抗辩方参与实盘投研辩论。 在更大规模的运营协同上,Concentrix策略与AI商业化主管Kathryn Harrison指出,公司内部搭建的HR代理平台正服务于全球40万名员工及每年超5万名新员工的招聘,覆盖75个交付地和150种语言。该平台负责简历筛选、薪酬开户及端到端入职培训,不仅将入职周期缩短了约14天,还将初期工作产出质量提升了约50%,净推荐值显著上升。这些实践证明,高价值落地需要将代理深度嵌入多步骤业务链,而非停留在孤立的单点工具层面。 数据治理前置与第三方安全风险 (8:23) 数据成熟度被专家们界定为部署AI不可逾越的前提底线。伊利诺伊大学医院(UI Health)网络安全主管Murad Dikeidek强调,“AI的智能上限完全取决于底层数据”。如果将存在过度共享、权限蔓延和无主状态的企业数据直接接入Microsoft Copilot等检索工具,将引发灾难性的信息外泄隐患。UI Health为此采用Varonis和Microsoft Purview等安全姿态工具排查数据归属并清理冗余归档。此外,临床团队在概念验证后放弃了针对医疗的ChatGPT方案,主因是严格的安全隔离破坏了跨部门数据共享与仪表板协作的便利性,表明过度摩擦会导致终端用户拒绝采用。 Dikeidek进一步指出,医疗机构约90%的数据实际驻留在外部供应商环境(如账单与挂号系统),导致第三方安全风险远高于内部用户威胁。UI Health通过Black Kite平台自动化分析供应商的SOC 2与HIPAA合规审计文档以识别缺陷,并利用定制GPT抓取全网泄露报告以评估机构受损程度。在高风险的临床场景中,技术采纳无法容忍虚荣指标,UI Health将能否安全提高“患者吞吐量”作为衡量AI系统价值的唯一硬性标准。 解耦架构防锁定与Token成本精算 (23:54) 针对市面上层出不穷的技术产品,Max Gokhman与Kathryn Harrison一致告诫企业切勿陷入供应商锁定。Harrison回顾了Concentrix在GPT-3时代因未搭建抽象层而被迫全面推倒重构的教训。为解决客户面临多达数十个孤立点解决方案的混乱局面,Concentrix构建了模块化的编排层,将底层基础模型、领域工具与业务逻辑彻底解耦,使企业能够根据各大模型实验室的技术突破与性价比变动自由切换供应商。 在算力支出方面,与会高管指出软件工具正由固定费率转向按用量计量,企业必须终结无节制调用的“无限自助餐”模式。Harrison表示,在Concentrix庞大的运营体系中,仅有10%至20%的复杂用例需要调用Claude Opus等前沿基础模型,其余场景使用Claude Sonnet等轻量模型即可胜任,甚至某些业务中人工单位成本比高端模型推理更为经济。Pankaj Jain建议在自动化工作流中通过程序将代码编写等任务自动路由给低成本模型,并对开发人员引入“软预算上限”,超出额度需提交业务申请,通过内部成本分摊机制(Chargeback)促使业务部门主动优化调用频次与上下文窗口。 事前合规守护与瞬态代理监管风险 (18:41) 面对大模型的不可预测性与自主代理带来的操作风险,专家们主张采用“以程序监督程序”的前置防御架构。Pankaj Jain提出,治理机制不能在代理执行失误后被动补救,必须在行动发生前设立“真实性守卫”(Truth Guardians)以校验数百项企业策略。Jain分享的价值流分析表明,全自动化工作流的整体准确率往往高于传统人工,原因在于其消除了跨部门多级人工交接中因理解偏差而加剧的“传话游戏”失真。Concentrix也部署了专门的合规审计Bot对日常助理代理进行实时监控,该系统曾仅用10分钟便为新到任高管完成了跨8个时区、涉及25场会议的复杂调度。 展望未来演进,专家们在看到协作机遇的同时也对代理无序扩张表达了深切担忧。Max Gokhman预测自主代理有望进入企业组织架构承担中层管理职能,凭借24小时无休、免受办公室政治干扰以及客观分配工作负载的能力优化团队运作;Pankaj Jain则警告不可任由自主权失控,强调代理应始终恪守副手定位。Murad Dikeidek进一步指出当前最为严峻的网安隐患:当主代理能够自主繁衍仅存活数秒或数分钟的“瞬态子微代理”时,这些临时持有凭证与权限的独立单元一旦发生数据泄露或违规行为,现存的安全审计与溯源工具将很难在它们消亡前完成追责,对企业防御构成了前所未有的挑战。 研讨会最终指出,尽管自主代理在瞬态审计、权限管理及算力成本上面临未决挑战,但其战略价值在于消除机械重复的运营负担,而非简单裁减员工。高管们一致警告,过早裁员会导致企业失去模型无法自主学习的隐性业务领域经验,迫使机构付出高昂代价重新招聘;企业必须建立清晰的员工发展路径,将人类劳动力从繁琐数据搜集中抽离,聚焦于算法无法替代的深度商业判断、策略评估与人际同理心。

Detailed notes — zh

- [00:04](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=4s) 企业在推进生成式 AI 落地时,应优先构建人机协作(Human-in-the-loop)的混合劳动力架构,在提升效率的同时由人类专业人员进行关键把控。 ## Key takeaways - [00:35](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=35s) AI 智能体(Agent)已被引入企业投资研究与决策过程,通过提出反向论证打破权威沉默与从众思维。 - [00:35](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=35s) 生成式 AI 在 HR 招聘与员工入职流程中展现出巨大价值,能跨越多种语言与地区,显著缩短入职周期并提升输出质量。 - [00:35](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=35s) 基于智能体的工作流(Agentic Workflows)可贯穿软件产品开发生命周期,从需求提炼到代码生成和单测自动化,有效提升研发生产力。 - [00:35](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=35s) 医疗等敏感行业的 AI 落地受限于底层数据治理,无用数据清理与权限撤销是发挥 AI 最大效能的前提。 - [10:36](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=636s) AI 安全与业务体验之间存在严峻拉锯:过度封闭的安全隔离环境会导致业务部门拒绝采纳,平衡安全与赋能是落地关键。 - [14:09](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=849s) 企业实施大模型选型需推行“Token 经济学”与模型分层策略,仅将顶尖前沿模型用于10%-20%的高阶复杂场景,其余使用轻量模型以控制算力成本。 - [14:09](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=849s) 面对 AI 系统的不确定性与不可避免的错误,企业需建立护栏机制,甚至利用“合规审计机器人”对智能体行为进行监控追溯。 - [03:19:27](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=11967s) 评估生成式 AI 的价值不应仅局限于降低成本与节省工时,更应致力于扩展业务上限并驱动顶层营收增长。 - [03:19:27](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=11967s) 企业应避免“拿着锤子找钉子”的盲目狂热,AI 必须围绕真实业务痛点和提升实际吞吐量来设计实施。 - [04:03:08](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=14588s) 智能体管理将引发组织心理学与中层管理的深度变革,AI 智能体凭借无政治斗争和高效响应优势将重塑管理形态。 - [04:03:08](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=14588s) 智能体自主生成衍生智能体带来了极大的安全与审计隐忧,临时生成的自主身份和权限可能导致监控失效与失控风险。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=4s) 研讨会开场与嘉宾背景介绍 主持人 Rob Pegoraro 开场并幽默地表示,感谢大家在同一时间段的六场生成式 AI 专题研讨会中选择了本场,这体现了观众的“品味与远见”。虽然本场研讨会无法解决所有人的问题,但希望能为大家带来有价值的启发。 随后,四位来自不同行业的重磅嘉宾依次介绍了各自的业务背景以及 AI 在其职业生涯中的演进历程: - **Kathryn Harrison**(Concentrix 策略与 AI 商业化负责人):Concentrix 是 Fortune 500 强全球技术与服务巨头。她早在 10 多年前就在 IBM 参与 Watson 项目,并曾连续创业成立过四家公司。她强调 Concentrix 处于 AI、人类与自动化融合的最前沿,致力于打造人机协作的混合劳动力架构。 - **Max Gokhman**(Franklin Templeton AI 与数字资产解决方案负责人):拥有超过 20 年的量化投资经验。他指出,如果按 AI 的广义定义,他早在金融危机前就利用随机森林(Random Forest)与马尔可夫链(Markov Chains)等模型构建系统。他曾创立过 AI 机制的对冲基金,如今在 Franklin Templeton 致力于将 AI 应用于大规模企业级投资业务。 - **Pankaj Jain**(General Motors Financial 国际业务 CIO):负责墨西哥、南美、欧洲及亚太等国际市场的技术战略与执行。他指出公司长期使用机器学习,而生成式 AI(GenAI)在过去两年间成为了重点突破方向。 - **Murad Dikeidek**(UI Health 网络安全负责人):拥有 25 年 IT 及网络安全经验。UI Health 包含 1 家医院和 26 家诊所,涵盖临床医疗、医学教育与科学研究。 - **Rob Pegoraro**(科技科技记者/主持人):近 3-4 年持续跟踪报道 AI,日常主要将 AI 用于采访录音转写,并笑称“绝对不敢用它来写稿,因为客户可不喜欢”。 --- ### [00:35](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=35s) 企业级 AI 落地实操案例分享 嘉宾们分享了各自企业最近上线的生成式 AI 落地项目,展现了智能体(Agent)和生成式 AI 在不同领域的突破: - **智能体协同决策(Franklin Templeton)**:Max Gokhman 介绍了他们推出的首个 AI 智能体同事——**Gromit**。该智能体的灵感来源于动画片《超级无敌掌门人》,不仅喜欢吃奶酪,更喜欢“怼”人类分析师。作为前首席投资官,Max 发现过去每当自己进入会议室时,团队的争论就会戛吵然而止,没人敢反驳他的观点。为了打破这种权威沉默,团队开发了 Gromit。Gromit 最初的作用是针对分析师的观点提出反向论证(Counter-thesis),后来逐渐融合内部与外部数据进行自主研究。如今 Gromit 会实时参加所有研究会议与基金经理讨论,且“从不害羞”。Max 调侃称 Gromit 像所有投资者一样“并非总是正确,但它总是敢于承认错误”,主持人顺势打趣称 Max 顺理成章地成了动画里的 Wallace。 - **招聘与入职智能化(Concentrix)**:Concentrix 为全球 2,000 家顶尖企业提供客户支持与技术运营服务,拥有超过 40 万名员工,每年需招聘 5 万多人。今年初,他们上线了一个 HR 智能体平台,覆盖 75 个地区、150 种语言。该平台不仅能解析简历与面试回答,还能在员工入职时加速薪酬设定、培训安排等全流程。 - **智能体产品开发生命周期(GM Financial)**:Pankaj Jain 分享了基于 GitHub Copilot 的 **Agentic PDLC(产品开发生命周期)** 实践。系统从高阶项目需求出发,通过多智能体协作链条,逐步将其提炼为业务需求、低阶需求,最终生成 Gherkin 格式的验收标准,并完成自动编码与单元测试。他强调这并非完全自主,而是一个人机协作(Human-in-the-loop)的流水线,每一步均有工程师进行质量把控。目前试点团队的生产力提升了 **20%**。 - **电子病历与数据治理(UI Health)**:Murad Dikeidek 介绍了在电子健康记录(EHR)系统中集成的 AI 工具,协助医生自动生成临床笔记和出院小结,减轻行政负担。然而,最艰巨的工作发生在 EHR 系统之外——即数据清洗与权限治理。AI 的智力上限取决于数据质量,如果数据不干净、不安全,AI 就无法提供最佳结果。UI Health 正在使用 **Varonis** 和 **Microsoft Purview** 开展大规模的无用数据清理、过度共享权限撤销以及历史数据归档与删除。Murad 强调,涉及 AI 的数据泄露危害远比传统安全事件更为严重。 ``` [高阶项目需求] ──> [业务需求提炼 Agent] ──> [验收标准生成 (Gherkin)] │ [人工审核/质量把控] <── [自动编码/单测 Agent] <────────┘ ``` - **Concentrix HR 智能体平台关键成果**: - **覆盖规模**:75 个全球节点,150 种语言支持。 - **入职速度**:新员工入职流程缩短约 **14 天**。 - **质量与满意度**:新员工 NPS(净推荐值)显著提高,产出质量提升 **50%**。 --- ### [10:36](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=636s) 暂缓或放弃的 AI 项目与合规/安全碰撞 在谈到哪些 AI 项目因技术未成熟或组织未准备好而选择暂缓或放弃时,嘉宾们分享了真实遭遇的阻力与权衡: Murad Dikeidek 分享了一个关于数据安全与业务需求冲突的经典案例。UI Health 的年轻医生们渴望将 Microsoft Copilot 用于患者数据处理,但 Murad 从网络安全角度判定当前环境尚不够安全。为了满足需求,团队安排了与 **ChatGPT for Health** 的演示。该产品提供了完全隔离、高度锁定的安全环境,这让安全部门非常满意。然而,医生们却拒绝采用,因为过度锁定的环境导致他们无法进行跨部门协作、数据共享和可视化看板创建。Murad 感叹:“工具再安全,如果无法满足业务需求,最终用户就不会采纳。安全是一场平衡的艺术,有时必须为了赋能业务而做出适当妥协。” Pankaj Jain 补充道,在推行智能体工作流(Agentic Workflows)时,必须极其谨慎地处理安全、策略定义与治理。GM Financial 决定暂缓盲目开放,转向先构建统一的企业级智能体 AI 平台。通过将底层基础架构与安全护栏抽象到平台层,避免开发者“发明轮子”或将不良实践扩散至整个企业。 - **安全与业务体验的拉锯战**: - 医生诉求:希望利用 AI 处理患者数据(PHI),强调协作、共享与看板展现。 - 安全部门考量:隔离环境、锁定权限,防止敏感健康数据泄露。 - 最终结果:因产品过于封闭影响协作,ChatGPT for Health 项目被挂起,团队重新回到 Copilot 环境的深度加固路径上。 --- ### [14:09](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=849s) Token 经济学与不确定性管理 随着企业 AI 应用的深入,算力成本控制与非确定性系统的管理成为了核心议题: Kathryn Harrison 深入剖析了**Token 经济学(Token Economics)**。她指出几乎每家企业都有过“两周内(甚至两天内)把全年 Token 配额用光”的惨痛经历——比如有人用 Claude 生成了一篇 25 页的长文,接收者又把它塞进 ChatGPT 总结成摘要。Concentrix 目前正促成 CIO 与 CFO 紧密合作,将 AI 资源配置提升至资本预算的高度。他们对 40 万名员工进行分层,其实只有 **10% 到 20%** 的高阶场景需要真正的顶尖前沿模型(Frontier Models)。虽然大家都有 FOMO(错失恐惧),倾向于直接上最贵的模型,但许多日常分析使用轻量模型(如 Sonnet)完全足够。在拥有庞大全球业务的企业中,有时使用人力完成特定任务的成本甚至低于 AI。此外,AI 目前擅长提供情报与信息,却缺乏人类在微观层面做出的审慎判断力。 针对 AI 系统的**非确定性(Non-deterministic)**与风险控制,Max Gokhman 提出了金融投资领域的视角。在投资中,没有人能够制造预测未来的“水晶球”,重点在于寻找可持续的优势。对于智能体分析师,传统的历史回测(Backtesting)非常困难,就像要求一个人在不考虑后续记忆的情况下回忆如何应对金融危机一样。企业领导者习惯于接受人类员工会犯错的事实,并通过治理与合规进行约束;但对于机器的错误,人们的容忍度极低。随着企业走向智能体协同时代,管理者必须学会接受智能体偶尔犯错,并通过护栏(Guardrails)来拦截灾难性失误。 ``` 企业 AI 模型匹配与 Token 预算策略 ┌──────────────────────────────────────────────┐ │ 高阶复杂决策场景 (10%-20%) │ │ ───────> 使用前沿大模型 (如 Claude Opus) │ ├──────────────────────────────────────────────┤ │ 日常分析与自动化场景 (80%-90%) │ │ ───────> 使用轻量级模型 (如 Claude Sonnet) │ └──────────────────────────────────────────────┘ ``` 为了确保智能体的可控性,Concentrix 甚至开发了数字工人(Digital Worker)系统。Kathryn 透露他们利用 Open Claw 机制建立了专门的**合规审计机器人**(Compliance bots),专门用来监控和稽查其他智能体是否在合规护栏内运行,实现了“用机器人监督机器人(Bots all the way down)”。这种审计层让管理层获得了信心。在一个实际案例中,当一位新高管入职需要在 8 个时区内安排 25 场会议时,智能体仅用 **10 分钟** 就完成了排期,而顶级的人类行政助理则需要花费数小时。 - **Token 成本控制与护栏建设总结**: - **模型分层**:将前沿模型限定在 10%-20% 的核心高价值场景,避免算力浪费。 - **ROI 平衡**:综合评估 Token 成本、人力成本与业务风险,警惕“模型越贵越好”的盲目心理。 - **智能体监督机制**:建立 Auditing Bots(审计机器人),对智能体的行为轨迹与合规性进行自动化追溯与监管。 <ElicitationsGroup message="如果您想进一步探讨生成式 AI 的企业落地,可以选择以下方向:"> <Elicitation label="分析 Token 经济学与大模型选型策略" query="请详细分析企业在引入大模型时如何建立 Token 经济学模型与模型分层策略?"/> <Elicitation label="探讨 AI 智能体(Agent)的合规与护栏设计" query="企业在部署自主 Agent 时,应该如何设计合规护栏和审计机制(Bots policing bots)?"/> <ElicitationsGroup> I encountered an error doing what you asked. Could you try again? ### [03:09:27](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=11367s) 从成本削减转向营收增长与理性提效 在评估 AI 的质量与效率指标时,与会嘉宾一致认为,目前全行业在这方面仍处于“探索与完善中”(work in progress)。试图宣称已经完全搞懂这些指标的人多半言过其实,但建立考核与汇报机制本身就能促使团队更加负责。 主持人指出,寻找生成式 AI 的第一个“滩头阵地”(beachhead)至关重要。首个落地场景不一定非要是节省最多成本的地方,而应该是能够让使用者快速看到效果、并自愿向全公司“推销”和分享经验的部门。 Concentrix 的 Kathryn Harrison 补充道,大多数企业最开始习惯于将 AI 视为节省成本或减少工时的“生产力工具”,例如给开发者或复杂分析人员等“超级用户”(power users)配备工具。然而,更具说服力的方式是“翻转剧本”(flip the script),利用生成式 AI 拓展业务上限、直接驱动顶层营收增长(top line),例如去服务那些过去因成本过高而无法覆盖的客户群体。 富兰克林邓普顿(Franklin Templeton)的 Max Gokhman 强调,大型企业完全可以多线并行:既抓基础的生产力提升,又开辟全新业务能力。然而,他从过去的失败经验中汲取了教训:**切记不要“拿着锤子找钉子”(building a solution looking for a problem)**。AI 的应用必须基于客户的真实需求或企业领域专家认定的痛点,然后再评估 AI 是否是合适的工具。 Max Gokhman 将早期 AI 比作电锯(chainsaw):电锯固然酷炫好玩,但并不是所有的工作都需要电锯,有些工作甚至根本不需要任何锯子。AI 不是玩具,必须严肃审视任务的本质。来自 UI Health 的 Murad Dikeidek 也深有同感,他分享道,现实中大家常陷入“每个人都想要 AI,但不知道拿它干嘛,而且立刻就要”的盲目狂热中。作为网络安全负责人,他必须要求各部门明确使用场景才能做好安全防护。在医院场景下,衡量价值的标准非常直接明确:只要工具能提高“患者吞吐量”(patient throughput),帮助医生与临床人员诊断和治疗更多患者,它就具备极高价值。 ### [04:03:08](https://www.youtube.com/watch?v=DsQS-X5vwGc&t=14588s) 未来展望:智能体(Agents)管理、安全隐忧与人类工作的解放 在座谈会的尾声,主持人邀请嘉宾们对未来一年的技术发展与潜在风险进行预测与展望。嘉宾们将焦点集中在了“AI 智能体”(Agents)的兴起及其对组织架构和安全的深刻影响上: * **组织架构与中层管理变革**:Kathryn Harrison 提出,AI 智能体进入组织架构与人类共事将带来巨大改变。她打趣地让大家对比“最差的老板”与“最好的老板”——大多数人会觉得 AI 智能体比最差的老板更好,但不如最好的老板。对于广大“平庸的中层管理者”而言,AI 智能体具备无政治斗争、24/7 全天候响应、可同时与庞大团队对话并高效优化分工等优势,未来中层管理层由智能体驱动将带来极具吸引力的组织心理学变革。 * **安全审计与智能体失控风险**:Murad Dikeidek 提出了他的安全担忧(即所谓的“天网/Skynet”式隐忧)。他特别警惕“智能体生成新的智能体”这一现象:每一个新生成的智能体都拥有独立的身份、访问权限和自主性,这会演变成一场审计噩梦。许多临时智能体可能只存在几秒或几分钟,在系统察觉并关停其不当行为之前就已消失不见。通用汽车金融(GM Financial)的 Pankaj Jain 也强调,AI 应当始终作为辅助工具(assist),而不是剥夺人类控制权。 * **对人类工作体验的乐观愿景**:Kathryn Harrison 表达了乐观态度。她认为 AI 智能体不会变成监控或控制人类的“天网”,而是会把人类从繁琐、痛苦的日常杂务中解放出来。未来的核心趋势是提高人类的工作满意度,让人类专注于发挥独特的判断力、同理心等不可替代的优势。 主持人总结道,非常赞赏本次论坛没有落入“AI 导致大规模职业毁灭”的消极论调中,并对各位专家带来的实战干货与战略指引表示感谢。 ## Video description Rob Pegoraro - Contributor, PCMag / Fast Company Pankaj Jain - CIO - International Operations, General Motors Financial Murad Dikeidek - Head of Cybersecurity, UI Health Kathryn Harrison - Global Vice President, Strategy & Business Operations, Concentrix Max Gokhman - Head of Artificial Intelligence and Digital Asset Solutions, Franklin Templeton Generative AI presents a transformative opportunity, but only if executed with clarity and discipline. This panel will present a practical playbook for enterprise leaders looking to scale generative AI initiatives effectively. From identifying the right starting use cases to building governance frameworks and adopting the best technology, panelists will walk through the key ingredients for generative AI success. Learn how to avoid common missteps, prioritize real business value, and set the stage for sustainable innovation. Whether you're early in your journey or scaling aggressively, this panel will offer strategic guidance and operational know-how to help you win with gen AI. [Source](https://www.youtube.com/watch?v=DsQS-X5vwGc)

Source quotes

[0:04] Rob Pegoraro: Never to write. My clients don't like that. [3:31] Max Gokhman: always admits when it's wrong [8:23] Murad Dikeidek: Your AI is as smart as your data [12:25] Pankaj Jain: proliferate bad practices across the enterprise [18:41] Max Gokhman: you will need agentic governance systems to control agentic systems of action. [26:53] Pankaj Jain: optimized broken process [26:53] Pankaj Jain: domain reimagination [31:03] Max Gokhman: train their successor and get fired [37:23] Max Gokhman: Why do we need to talk to this chatbot? [37:23] Max Gokhman: Guys, it's not a chatbot, it's actually going to really help you. And also, I'm not asking, I'm telling. [37:23] Kathryn Harrison: Anyone who says that they have that figured out, I think is probably lying. [42:10] Murad Dikeidek: Everyone wants AI. What for they do not know and they want it now.

Original transcript

[0:04] Thank you for coming to this Gen AI panel  generative AI panel as opposed to the five [0:08] other ones happening in the same time slot.  We appreciate your your taste and discretion [0:12] in choosing us. Um we're not going to solve  everyone's um problem here, answer every question, [0:21] but let's let's get us started at least.  And let's start that. Why don't each of you [0:26] introduce yourself, talk about your company,  and how long has AI been your primary task? [0:35] Do you want me to start? Go to it. Great.  I'm Katherine Harrison. I'm the head of [0:39] strategy and AI commercialization at  Concentrix. Concentrix is a Fortune 500 [0:48] global technology and services company. Um how  long has AI been part of my life? Is that the [0:56] question? Professional life. Professional life.  Um I was working on Watson initiatives at IBM [1:03] 10-plus years ago. I started four companies, and  then I joined Concentrix because we are right at [1:11] the heart of how AI, humans, and automation are  all coming together for the hybrid workforce. [1:19] Hi everyone. My name is Max Gockman. I'm the head  of AI and digital asset solutions for Franklin [1:24] Templeton, and AI has been a part of my life for  pretty much my entire career if we use the real [1:31] definition of AI. I've been a quant investor for  over 20 years, and things that we would always [1:38] consider AI like random forest models, like Markov  chains, were some of the things I was building [1:44] pre-financial crisis, but then I've started  an AI hedge fund, and now at Franklin, [1:50] I'm really excited to be building AI at  a massive enterprise scale for investors. [1:57] Um hello, I'm Pankaj Jain. I'm CIO for  international operations at General Motors [2:02] Financials. I'm essentially responsible for  technology strategy and execution in all our [2:07] international locations across Mexico, South  America, Europe, and Asia Pacific. And in terms of [2:15] AI, I think we have been using machine  learning for a long time now, right? So, [2:19] because that's part of AI. Um, but generative AI  has been a focus for the last couple of years. [2:26] Good afternoon. I'm Murat Dikayik, the  head of cybersecurity for University of [2:29] Illinois Hospital and Health Sciences System,  commonly known as UI Health. I've been in IT [2:35] and specifically cybersecurity for about 25 years  on the technical side and on the leadership side. [2:42] Uh, University of Illinois Hospital is part of  University of Illinois Medical Center is part [2:46] of University of Illinois System. Um, we're an  academic healthcare organization, one hospital, [2:53] 26 clinics. We provide patient care to local  communities and we educate the next generation of [3:01] healthcare providers and then we also  advance medicine through research. [3:07] Um, AI has been part of my life for 3 years.  I I guess I have to answer my own question. [3:13] By the way, I'm Rob Pegoraro, a freelance  technology journalist from Arlington, Virginia. [3:17] Um, it has been part of what I write about.  Yeah, it's going on about like 3, 4 years. Um, [3:23] I don't actually use it all that much in my daily  day-to-day work. Transcribing interviews, that's [3:26] about it. Never never to write. My clients don't  like that. Uh, but yeah, it's inescapable. It's [3:31] changing so much. So, let let's talk each of you  about a recent implementation you've done of AI. [3:38] Get into details, not so much we get anybody in  trouble, but uh, talk about how you put AI to work [3:43] and what you learned along the way. Who  wants to go first? I'm happy to start. Um, [3:50] most recently we've introduced our first agentic  co-worker at Franklin. His name is Gromit. [3:57] Those of you who are familiar with the cartoon  Yes. may find that Gromit does like cheese, [4:03] but what Gromit likes even more than that  is pushing back on our human analyst. [4:08] The reason Gromit came into existence is  because before my current role, I was chief [4:12] investment officer of our multi-asset team. And  one thing I found is whenever I entered the room, [4:18] all of the debate would die. No one wanted  to push back against my view, and I'm [4:22] fortunately humble enough to know that I am wrong  quite a lot. So, I said, we need to actually build [4:27] an agent to deal with the humans wouldn't do. And  so, we built Gromit to first push back and provide [4:33] counter thesis against our human analyst. And then  Gromit started getting smarter, Gromit started [4:39] getting advanced data, it started synthesizing  information from internal and external sources, [4:45] and doing its own research. And now,  Gromit is actually able to real time [4:52] pipe into debate. Gromit attends all of our  research meetings with our research analyst and [4:56] portfolio managers, and Gromit is never shy. And  while Gromit, like all investors, is never right, [5:03] it always admits when it's wrong. Does  that make you Wallace? By definition. [5:11] I'll jump in. So, Concentrix runs the customer  support and technology operations for 2,000 of the [5:19] world's largest companies. So, we sit in actually  two places on the AI front. We're running AI [5:25] operations for hundreds of our clients, and then  we're also using it to transform our own business. [5:32] As part of that, we have over 400,000 people  worldwide, and we are recruiting more than [5:38] 50,000 people a year. So, earlier this  year we implemented an HR agentic platform [5:47] to help us not only accelerate recruitment but  also the onboarding process. And this is happening [5:54] across 75 different locations, 150 different  languages, and all different skill levels. [6:02] Uh the Agentyc platform not only understands uh  the resume, the the applicant's answers in the [6:10] recruiting process, but as they get on board,  we accelerate their payroll setup, all of the [6:17] details that it takes to onboard someone into our  operation, including training, etc. What we're [6:23] finding is that the NPS of those recruits is ex-  significantly higher than our previous cohorts. [6:31] They're onboarding about 14 days faster than we  were going through our traditional processes. [6:38] Um and the quality of their outputs is about  50% higher than what we were seeing previously. [6:45] So, it's a really exciting, given the the scale  at which we are recruiting, to be able to use that [6:52] to accelerate that journey. Impressive. Who want  Who is next? Okay. I can go next. So, I think um [7:00] the most impactful use case, I would say,  is our Agentyc PDLC implementation. So, [7:05] PDLC stands for product development life cycle.  So, it's essentially leveraging Agentyc AI to [7:13] do your product development. Right. So, you  start with high-level project requirements. [7:18] Right. And then you have agents to distill them  into higher-level business requirements, into [7:23] lower-level requirements, and finally into Gherkin  or acceptance criteria in Gherkin format. Right. [7:28] And then you have a series of multiple agents  coding, writing acceptance tests, unit tests, [7:33] right, and going through the whole cycle. And  uh um the key is like here is not that the whole [7:39] process is fully autonomous. Right. We still have  every step of the way, it's a sequential pipeline [7:46] in GitHub Copilot, right? In every step of  the way, humans are inspecting, evaluating, [7:51] right? Doing quality control. Um So, fast  forward, I think so we have implemented it in [7:58] some of our teams. It's not obviously not global  yet. Um and we found productivity improvements up [8:03] to 20% at this time. And uh we do plan to kind of  take this further into more teams, right? And uh [8:10] hopefully like if you look at it across the  enterprise, that could be significant productivity [8:15] benefits we could realize. So, that that I would  say is one of our most impactful use cases so far. [8:23] Um We're working a lot with our vendor to  implement more and more tools within the [8:28] electronic health record system. We have several  tools that ease the burden on the doctors, [8:37] administrative burden of documenting, summarizing  clinical notes, discharge summary. So, there's AI [8:44] software built into the electronic medical  record to do all that. So, the doctors can [8:49] spend more time seeing patients and treating  patients. But, the real work that's that's [8:54] labor intensive is outside the electronic medical  record is still going on, which is a cleaning up [9:00] the environment where Copilot is going to operate  and the data that Copilot is operating on because [9:09] your AI is as as a smart as your data. If  your data is not the clean, is not secure, [9:17] is not accurate, AI is not going to give you the  best outcome. So, we're spending a lot of time [9:22] removing excessive permissions, oversharing, data  that's overshared, old data. We're trying to put [9:28] in retention policies in place so we don't hold on  to data for so long. We're trying to delete data [9:34] that was created years ago. And we're trying to  go after people to identify who the owners are. [9:41] This is a lot of labor-intensive work that we  cannot do manually, so we bought Varonis and we're [9:48] working with Varonis and Microsoft Purview to do  all this work and at least most of it, so we don't [9:58] burn ourselves because it is almost so hard to get  people to admit that this data needs to be deleted [10:05] or this data is no longer needed, so we try to  archive it if it's not no longer needed, then [10:11] delete it, and we try to limit the access that  Copilot or any AI is going to is going to have [10:18] because any breach that involves AI is worse, far  worse more than traditional security breaches. [10:27] Having just had a former employer breach  some of my data, thank you for your efforts [10:31] in data minimization. I don't think AI was  involved. It was some Oracle database. Um, [10:36] let's now pivot and talk about some case where you  looked at applying AI to a task and then decided, [10:41] "No, the technology isn't there or there isn't  the organizational readiness you just spoke of, [10:47] and either we'll look at this again in a  year or it's just never going to work." [10:54] I can start again. You got it. Well,  since we're talking about Copilot, so [10:58] we have a lot of young doctors who are  always interested in in technology and [11:03] we live in a world where we have young doctors who  wants to push us to approve and deploy more tools, [11:09] and then you have the doctors who've been in  the field for a long time who doesn't want to [11:12] do anything with technology. So, um, they've  been asking us when they can use Copilot for [11:19] patient data, and I keep saying, "I am not sure  if Copilot is the right tool for patient data cuz [11:26] someone else need to decide that.  My focus is on cybersecurity." But, [11:30] I keep telling them that Copilot environment is  not secure enough. And I have really no date when [11:36] we're going to have it secure enough because  there's so much work to do from IT and non-IT. [11:41] So, we said what about looking into a tool that  is isolated, that's not part of Office 365, [11:48] and we can control access to it and control  the environment where we can put PHI. [11:55] They said, "Sounds good." So, we scheduled  a demo with the ChatGPT for health. [12:02] And the doctors looked at the demo, liked  everything about it. They said, "Okay, [12:07] we we we should uh sign up uh for ChatGPT for  health." But when I started asking the questions, [12:14] and they and they started telling me that  the environment is totally locked down, [12:19] I loved it cuz I work in cybersecurity. I want  to lock down everything. They didn't like it [12:25] because they want to collaborate, they want to  share the data, they want to create dashboards, [12:30] they want to show others. So, they decided  not to go for [laughter] ChatGPT for health. [12:39] So, this reminded me that it's not how effective  the tool is if it doesn't meet the needs [12:45] of the organization. And it doesn't matter how  secure it is, the end user is not going to adopt [12:52] it. So, this reminded me of again that security  is really a balancing act that sometimes I have to [12:59] compromise a little bit on the level of  security to enable the business. So, we're [13:03] at a standstill right now. We do not know what  we're going to do, but looks like we're going [13:08] back to securing Copilot. I'm sure there's some  vendor on the floor that will try to help you with [13:13] that. [laughter] Anyone else want to talk about  some opportunities passed on passed by for now? [13:19] So, just on that thread like it's not [clears  throat] that we didn't want to do it. It's so [13:25] when we're are about agentic workflows where  the agents take actions on behalf of the users, [13:30] you have to be extremely careful  with including with security, policy [13:34] definition, governance, and all of that,  right? So, uh we made a decision to kind of [13:39] look into it much more deeply, right? Before  just letting the floodgates loose, right? And [13:44] make sure that we're building the right agentic AI  platform where a lot of this plumbing is handled [13:49] by the platform rather than the developers  inventing it. So, you want you don't want to [13:53] proliferate bad practices across the enterprise.  You want to abstract them into platforms. So, [13:57] that's our focus, right? In terms of how do we  enable agentic workflows that can act on behalf [14:02] of humans and how to do that at enterprise scale  with the right guardrails, controls built in. [14:09] I think the other thing that I would add  is that we're having a really interesting [14:12] conversation about token economics. Um I think a  lot of companies are Well, everyone has a story [14:19] about how they blew through their tokens for  the entire year in 2 weeks, a quarter, 2 days, [14:24] depends who you give it to. I'm sure we've  all received a 25-page document written with [14:30] Claude that you then put into ChatGPT to get the  summary so you don't have to read the whole thing. [14:35] And so, what we're really trying to do is have  our CIO work closely with our CFO to think about [14:41] it much more as a capital allocation decision.  We're segmenting our workforce. We have 400,000 [14:49] employees to make sure that we're giving the right  tools, the right capabilities, the right models [14:56] to the users that need to use them. Really only  10 to 20% of use cases need true frontier models. [15:04] But let's be honest, we all have FOMO. We've  all done some sort of complicated analysis that [15:09] probably would have been fine with Sonnet, but  you're like, "Let me try Opus just in case." And [15:15] so, I think what we're finding is really trying  to match the the work that needs to be done to [15:23] the tools, to the models. Because there are many  many cases where especially in a business like [15:30] ours where we have a significant global footprint,  where it's actually cheaper to do the work with [15:36] humans than it is to do it with AI. And so you've  got to have a really interesting balance of [15:43] what does the ROI look like? What do the economics  look like? And also from a workforce management, [15:49] you need to be developing your team so that  they have the judgment that you need to run your [15:55] business at scale, right? And I think AI right  now is very good at intelligence and information, [16:02] not so good at all those micro  judgments that we humans make. So. [16:07] So let's talk about that. The the fear downsides.  Obviously one of them is widespread job loss, [16:15] um that you chew through a ton of computing  resources, your cost per a lot of control. And [16:21] yeah, and that's something in some ways  because AI is non-deterministic, you [16:26] may not actually know. So how do you deal  with that? You've got the uncertainty of [16:30] you know, you have these tasks to be done, but  it's not quite clear how much computing resources [16:36] will go into getting those done. I can  start with with that one. Um and in part [16:41] of because everything we do as investors, we're  dealing with uncertainty. Like if if believe me, [16:47] there's plenty of quants who have tried to  build the crystal ball that does not exist, [16:50] it will never exist. There's always going to be  someone building a comparable crystal ball with, [16:54] you know, more horsepower behind it. So the key  is to find durable advantages and edges, not [17:00] solve the market. And with that, I think whether  you're building kind of a traditional quantum [17:07] model or now building an agentic analyst or even  teams of agentic analysts and portfolio managers, [17:14] you need to start with a more experimental  approach. Now, backtesting AI is very difficult. [17:23] Telling a model to trade a something in the  past without thinking about what it already [17:31] knows is the same exactly as telling a human,  "Hey, how would you trade the financial crisis, [17:36] but don't think about anything you know about what  happened after." So, it's really about creating [17:43] those kind of environments where we can really  test what will work, find the nuance and the [17:48] context of the edge cases, and then accept what  we think is a reasonable amount of uncertainty. [17:56] I think with humans we're much more comfortable  with that. When we hire a human to do a [18:00] decision-level task, we know about that human  may be wrong. And, you know, as leaders, we [18:06] accept that sometimes our people will not make  the right decisions. We try to make sure that they [18:12] don't make the really bad decisions, and that's  governance, that's constraints, that's, you know, [18:17] in a regulated industry compliance. But, we are  not as comfortable about that with machines, [18:23] and I think as we move into this agentic co-worker  world, we're going to have to be comfortable [18:29] that the agents will sometimes make mistakes,  and some of those mistakes are okay, [18:34] and some of those mistakes are the ones we're  going to need to control for with guardrails. [18:41] I would add we built a digital worker platform  that sits alongside my human executive assistant, [18:47] and I have the best executive  assistant on the entire planet. [18:52] Um but what we did is we used Open Claw we uh  just really to experiment and to learn. For us, [18:59] it's really worth the investment to build  different techno around different technologies. [19:05] And where we really started was around the  guardrails that you mentioned, the compliance, [19:11] um setting what that use what that agent could  do and not do. We've also built compliance bots [19:18] to audit and make sure that what our agents are  doing is in line with the guardrails so that we [19:24] can actually track that. bots to police the bots?  You need bots to police the bots. Bots all the way [19:30] down. Indeed. But I think that auditability layer  starts to give the comfort that we're looking for [19:36] much in the way that I'm sure you have to take  two weeks of vacation every year so they can go [19:40] through your email. Uh similar types of things are  going to need to be set up and run for our agentic [19:45] workforce. And but that agent was able to schedule  so I had a new executive, one of my colleagues [19:53] started. I needed to set up 25 meetings for her  in the first two weeks across eight time zones. [20:00] 10 minutes, the agentic platform had done it. My  best human would have taken hours trying to figure [20:06] out exactly what are the overlaps. So again, I  think it's really starting with the guardrails [20:11] and controlling what they can do, auditing it,  having that compliance layer, and then really [20:16] scoping and scaling the work that you're doing.  There's something you said that I just want to [20:20] underline which is fighting fire with fire or  controlling agents with agents. That is very, [20:26] very valid. You will need agentic governance  systems to control agentic systems of action. [20:35] Yeah, so just on that point, I  think to me like as an engineer [20:40] um that is one of the hardest engineering problems  when it comes to agentic platforms, right? So you [20:45] cannot apply that governance after the agent  has done the job, right? Or bad job, say. [20:51] Um all of those policies need to be run ahead  of time, right? You need to almost have truth [20:55] guardians to validate what the agent is going  to do is actually going to meet the cut. So [21:01] uh so it's a very challenging problem, especially  from an engineering standpoint, right? How How [21:05] you run hundreds of policies, say, right? And how  do you have these two guardians in place? And um [21:11] and it's it's got it's going to be challenging,  but to the original conversation around humans [21:17] versus agents, I think if we do a value stream  analysis and we look at, you know, the process [21:22] which is human-centric today and we look at an  agentic flow, which is completely agent-driven, [21:27] more often than not you will find that  the accuracy at the human level is lower [21:32] than the accuracy at the agentic level. And uh  that's something which was quite surprising to me, [21:37] right? And the reason for that is because there  are so many hand-offs in the human process. [21:42] It's not that an individual human is doing  anything wrong, but as part of the hand-offs, [21:47] there is inaccuracy introduced as part  of that hand-off process. It's the manual [21:52] coordination that is happening. So uh so we  may say that AI has a challenge with accuracy, [21:58] but I think the human challenge is probably even  greater today. The whole playing telephone game. [22:05] Right. So we don't build any agents for  healthcare, but we have enough geeks to play [22:10] around with AI when it's ChatGPT and create  some custom GPTs. So we did some cool stuff [22:17] for third-party risk. So um we bought a platform  called the Black Kite um that made life so easy [22:25] for us. Um we get audit documents, whether it's  SOC 2 documents, HIPAA documents, um and we upload [22:32] them into the Black Kite platform and it tells us  exactly if the vendor has good security in place [22:38] based on the what we have in the document and uh  whether we need to go hard after certain areas [22:46] for them to remediate or not. And then also when  there is a breach that affects one of our vendors, [22:53] and believe it or not, 90% of the data that  healthcare owns is not in their own data center. [22:59] We outsource so much for billing, for  appointments, for for for a lot of things, so [23:05] we worry more about our vendors than we  worry about our users because our vendors has [23:11] a lot more data than our users. So, when  there's a breach, we have custom GPTs that um [23:18] tells us exactly what happened, goes out on the  internet, and it looks at different websites to [23:23] find what other websites and what each website  saying about what happened to that vendor, [23:28] and then it creates a report that this is what  happened on this date. You might be affected, [23:32] you might not be affected, and this is what  you need to do next. So, automating some of the [23:37] process and, you know, the the write-up and um the  summaries and all this is is very helpful for us. [23:44] Now, when you're looking at uh a pitch from a new  AI vendor, what have you learned about evaluating [23:49] that? Clearly, in the last 3 years, you've learned  a lot about the things can work, not work so well. [23:54] What what What advice would you want to share  with the folks here when they're looking, say, [23:58] at some of these exhibitors promising, "Here  is our shiny new thing to solve your problem." [24:06] I would say, first of all, do not lock yourself  in. [snorts] That is the number one thing. [24:12] This technology is moving at a pace that is  unprecedented. So, even if this perfectly [24:18] solves your problem right now, you need to build  everything to be modular and swap in and out. [24:25] And two, consider the support process of smaller  vendors. I think that's really important. Now, [24:35] for a large company like us, we wouldn't be able  to onboard most small companies cuz they just [24:39] wouldn't pass our vendor onboarding process. They  wouldn't have the right legal teams, compliance, [24:45] etc., etc. But, what's really important for  anyone is when things go south, is there [24:52] going to be a human who you can talk to? Cuz you  will at that point need to talk human to human [24:58] unless you're way more advanced and you've built  an agent that can fix another vendor's problem. [25:05] That's so funny because literally we we talked  about one of the best ways to explain what you've [25:10] learned is to share failures. And very we jumped  into the AI product building very very early [25:18] and we built an internal tool. We built it around  ChatGPT 3.0. And then models moved very very [25:26] quickly and we hadn't built our architecture to be  able to be flexible, to be an orchestration layer. [25:34] And so that really forced us to re-architect,  which is what happens to many many many AI product [25:42] projects. Um but it helped us to learn just that  level of flexibility that you need to have in [25:50] platforms. And I think the other thing that we're  seeing on our customer side is that they're ending [25:57] up with 25 different point solutions that do tiny  pieces of all the different tasks that they need. [26:05] And where we're able to add a lot of value  to them is to provide that orchestration [26:10] layer. So whether it's models, whether it's the  harness, whether it's a whole host of things, [26:15] they can swap technology in and out depending  on the use case, what they need, who is suddenly [26:21] leapfrogging the next because obviously there is  just such an incredible arms race at the moment. [26:28] I think we all look at it in terms of models  first and foremost, but there's so many other [26:34] layers that we're going to start to see. So we  often say to our clients, start with the outcome, [26:41] build yourself the freedom and flexibility for  the operating model that your business needs, [26:47] and then work with the tech stack that you have.  So. Yeah, I would also like to add that I think uh [26:53] what I've seen is a there's an expectations  management problem. Right? So, executives [26:59] are expecting transformational benefits when  all you're doing is an optimization. Right? So, [27:04] if you look at how we started with GenAI, we have  all these use cases, right? You you take a part [27:10] of a existing business process and you automate  that. But, have you looked at the entire process [27:15] in itself? Right? So, if you're if your process  itself is not optimized, it's broken. You have You [27:21] have kind of optimized a broken process and then  you're expecting transformation level results. So, [27:25] I think a lot of onus is there on the leaders who  are leading AI to set proper expectations, right? [27:32] And that involves like tracking the defining  the value, the business benefit of the use case, [27:37] tracking the value through the life cycle,  and most importantly showcasing what value [27:43] was achieved in production because I think  that's again I see that sometimes missing, [27:46] right? In organizations. So, uh so I think right  expectations need to be set. The to achieve true [27:53] transformation level benefits, you have to do  domain reimagination. You basically have to go [27:58] after end-to-end business processes, right? Where  AI is a component of it. AI may not be all of it. [28:03] Right? So, the the ROI you're getting is basically  your you're basically transforming an existing [28:08] process and then you're further driving up  the benefits because of the AI technology. [28:17] We don't trust technology, especially in health  in health care because it's going to impact [28:21] the [laughter] the patients very badly. So, I  know our radiology department started looking at [28:28] AI software to help the radiologist look into  something they may have missed. Often gets [28:35] brought up as a good use case for AI. So, cuz I  worried I said, "Now you're you're going to have [28:40] the agent read those images?" I said, "No. The  radiologist will read the images and then AI will [28:46] say, "I look I I read also the image and I saw  five different things. Did you see them or not? [28:52] So, that's what we're using it for. Um validation,  assisting, but not completely a trust. So, [29:00] this one thing I keep hearing is  how you sort of need to start with [29:05] forget for a second that you're going to try to  apply AI, take a look at your existing processes. [29:10] Is AI just going to be the lens that reveals that  oh, we kind of had a broken way of doing things [29:15] all along, but it sort of got lost in the noise?  Yeah. Ooh. Yeah, most enterprises are incredibly [29:21] messy. They rely on you knowing the right person  in this department to get the information or the [29:27] data or the report that you need to get the  answer to your client, to your boss, whatever. [29:33] And I think that so often what we see is that  when people are like, "Okay, just bolt AI on." [29:38] And then they realize, you talked a little bit  about cleaning up the data, about transforming [29:43] the business process. That is as fundamental, and  this allows us to do things that previously were [29:51] so, so challenging, but you still need to have  that human in the loop around the judgments as to [29:58] how do you make sense of that mess? And we've  actually built, in addition to an Agentyc [30:03] platform, we've built uh an agent assist  platform to actually help the humans who [30:09] might normally have 25 tabs open trying to  cross-reference That few? [laughter] [gasps] [30:16] That's on a good day. Um right? Um to really  actually Actually, one of you mentioned kind [30:22] of pulling out the right information and being  able to optimize How do you summarize what's [30:27] happening? What's the next next best action  to take? And that's where you can actually [30:33] make humans' jobs much more enjoyable and much  less let me sort through this morass of data. Um [30:42] so, I think we're seeing that it it really takes  both of those to be successful. And I think that [30:50] the hype cycle is probably even coming down off  its peak um and people are starting to see that [30:56] it is still technology that we have to figure out  how to flow through our businesses. And I want to [31:03] build up on on that a little bit because a lot of  times executives will see an AI report and they'll [31:09] be like, this looks really good. Why do I need all  these humans? Cuz, you know, they're not in the [31:15] trenches doing the task day-to-day. So, they see  it and they're like, this looks just as good like [31:20] this investment model looks great. Well, you're  not an investment analyst, so you may not actually [31:25] know if it's the assumptions underpinning it makes  sense just because it looks accurate, you know? [31:31] Um, and so what I'm seeing across some enterprises  is they fire a lot of people by bringing AI in and [31:38] then they need to hire those people back cuz the  domain knowledge was taken out. That is very key. [31:44] You still need humans to train the agents. And  then, of course, no human wants to train them, [31:51] you know, their successor and get fired. So, you  need to think about how those humans roles evolve. [31:57] And so, part of setting your enterprise up for  success has to be giving your humans who are [32:03] working with these agents a clear career path that  they're excited about. So, for example, one thing [32:09] we've been talking about with our investors is  you're going to have a lot more time in the field [32:13] talking to management teams, less time at your  desk building models. That's for most analysts, [32:20] that sounds really exciting. So, they're happy to  train the agents with everything they know, but [32:26] that internal IP does not ship with an  off-the-shelf agent. You have to build [32:32] it and it takes hard work. Yeah, I  think just to add to that, I think [32:38] one area where you're seeing that is uh our scrum  teams, right? Um so now you're coding with agents, [32:46] you're pair programming with agents, right? So,  that's a shift in terms of earlier we used to pair [32:50] programming with humans, now it's agents, and  then how do we work in that world? And I think [32:56] um there has to be some level of change  management support there in terms of, you know, [33:01] uh for people to feel comfortable working with  agents because that's going to be the reality, [33:04] right? In the future, we will be working with more  and more agents. You're going to have agent farms. [33:10] And uh we need to be able to um uh help the  workforce, right? Manage this transition. [33:19] I'll add my uh security perspective to this.  I start by saying how important the data [33:26] to this, and AI reminded us about the problem that  we've we've been ignoring for a long time, which [33:32] is the urgent need for data governance. People  create data left and right. No one ask them why. [33:40] They ignore ownership. Um we hold on to the data  for too long, so I never seen any technology [33:47] before that is more data-centric than AI.  So, that your data is the main ingredient. [33:54] If you don't have the best ingredient,  you're not going to have the best outcome. [33:59] Yeah. Uh so, another I guess question a lot of  organizations have to deal with is we're talking [34:06] before about, you know, which model should  you apply? Should it be a one of the heavily [34:11] advertised large language models, some much more  narrowly focused thing, should it be an on-device [34:15] system with especially sensitive data? What have  y'all learned about how to sort of decide where [34:21] the workload goes, where it gets handed out,  or does it not go to any AI system at all? [34:28] Well, I alluded to this a little bit earlier,  but we have such a range of employees internally [34:34] and then you can imagine with 2,000 customers  similarly they have a huge diversity of skill, [34:40] expertise, etc. [gasps] So it's really about  routing the the right type of work to the right [34:47] models and systems. And so we've built those  guardrails internally so that our cars call [34:54] center agents have an internal product that really  controls what they're allowed to have access to [35:00] based on the client contracts, etc. Sometimes  it does make sense to use very specific trained [35:07] models. Other times when you're dealing with  your developers who are building new products, [35:13] you might want them to have more robust tools. So  I think it's about being really intentional about [35:19] the outcome that you want from that. And we're  seeing job satisfaction actually go up as they [35:26] as our employees get more access to the  tools that are accelerating their jobs. [35:32] Yeah, I think one good way to um to enforce this  is to make people accountable for their spend. [35:41] So essentially have a chargeback mechanism. You  can spend all you want, but you have to pay for [35:44] it. So if that's that's the start, right? What  happens when that teachable moment happens in [35:51] Q1 and they [clears throat] they have three  more quarters of work to do? So that's one. [35:55] On the other front, I think a lot  of education has to go in, right? [35:59] That's where the the training comes in. You  have to educate the organization in terms of [36:03] how to use what models. And there's a because  that's the space is changing so rapidly, what was [36:09] good yesterday is no longer good today. Right? So  it's a challenge. It's an ongoing challenge. But [36:15] that education aspect is very critical for people  to understand like, you know, there's no more, [36:19] you know, unlimited buffets, right? Now like even  GitHub Copilot Copilot Co-work all all the tools [36:26] are going towards fixed uh you know, usage-based  pricing. So, um so, it's really important for [36:32] people to understand that, right? And uh  you can definitely, like like she mentioned, [36:36] LLM routing is a great mechanism, especially with  agentic workflows, you can force that behavior. [36:42] Yeah, there's no choice there because depending  on the type of the work the agent is doing, [36:46] you it can leverage the right agent. There  There There need not be a choice there. But [36:51] when it comes to developers, as an example, you  cannot prevent developers from using the frontier [36:55] models because for some tasks, like, you know, in  the uh agentic PDLC example I was giving earlier, [37:02] um when they are doing the decomposition into  requirements and uh that requires higher level [37:08] of [snorts] um you know, fidelity from the model.  So, you want to use frontier models there, but [37:12] when it comes to generating the actual code, you  can use a lower-level model. But again, you cannot [37:17] restrict them from using the models, you just have  to educate them. Yeah, I I would say there are two [37:23] parts to this. One, on the end-user case. Um what  we found worked well, like with with Grommet as [37:28] an example, we released Grommet to one team. That  team initially was like, "Why do we need to talk [37:32] to this chatbot?" And I had to say, "Guys, it's  not a chatbot, it's actually going to really help [37:36] you. And also, I'm not asking, I'm telling." And  then, they started using Grommet and were like, [37:42] "Wow, Grommet's actually really helpful and giving  making me do my job better." Then, they started [37:49] talking to other investment teams, and those  teams said, "Well, wait, we want a version of [37:54] this tool." And so, that created this desire, and  as part of, you know, shipping anything to them, [38:01] we make sure they understand the work required  to make it work for them. On the developer side, [38:09] I think it's really important to set the right  token budgets and not have it be a hard cap, [38:16] but more of a soft cap. Like, if And I I  actually I have this for for myself with uh [38:21] some of the models where I have a budget and you  know, I can obviously ask and get more tokens, [38:26] but I will go and say this is why I need more  tokens. And I think setting that discipline from [38:32] the top down is really important. So, justify why  you need those extra tokens, why you need to use [38:40] the frontier model. And I think that simple human  task of asking for permission can be very helpful. [38:49] Just one other thing that I put in my notes.  One other thing that we've done on that front [38:54] is we found that if leaders use AI well  and responsibly, it really trickles down [38:59] to the rest of their teams. So, as a global  executive team, we are actually looking at [39:04] how are the leaders of our organizations,  whether it be HR, IT, product development, etc. [39:11] Are they using the tools? How are they talking  to their teams? We have a whole set of training [39:16] academies, etc. that provide skills. And I think  by just measuring, by just people knowing that [39:22] both their executives and their entire teams  are being tracked, and we're working on new [39:27] metrics around quality and efficiency, which are  I think they're still work in progress. Anyone who [39:32] says that they have that figured out, I think  is probably lying. Um but just even having to [39:38] your point, having to think about it and report  it out makes people be much more responsible. [39:44] One interesting takeaway I'm seeing here  is that you need to think carefully about [39:47] where do you first what what is your  first beachhead for AI? And maybe it's not [39:52] where it save the most money, but where the people  who adopt it can then sell it, wind up selling [39:56] it to the rest of the organization, and have  useful lessons to share. Yeah, I mean, I think [40:02] most people look at it first and foremost as cost  savings or how do I take out hours worked, etc. [40:09] With our pilots, we tried to look at who  were [snorts] the likely power users. For us, [40:13] it was a lot of developers, people who are doing  very complex analysis. We also have flipped the [40:20] script and we're starting to use our our genetic  layer to generate revenue and to drive the top [40:27] line because that's a very compelling story when  you can start to serve customers that previously [40:33] were too expensive, etc. So, I um I think it's  very tempting to start with how do we cut cost, [40:39] how do we save time? Those are the productivity  plays. I think tend to be the most obvious. [40:45] But I would really encourage you to look at how do  I impact more top line metrics cuz I think there's [40:50] huge opportunity there. I I I would agree and  I think you can with a large enough enterprise, [40:55] you can do multiple things in parallel. There  are productivity gains that you can have and some [41:00] of those are you know, if you are starting from  ground zero, very straightforward. And then there [41:07] is the ability to add in brand new capabilities  that didn't exist before. Capabilities that your [41:12] clients are asking for, but that's kind of a key  thing that I learned and speaking of failures, [41:16] I've learned this the hard way. Do not  build a solution looking for a problem. [41:21] Build something that your clients want or you as  a domain expert know your company needs and then [41:30] make sure that AI is the right tool. You know, I  sometimes in the early days would liken AI to a [41:36] chainsaw. In the sense that chainsaws are really  cool and they're fun to play with, but not every [41:41] job requires a chainsaw. Sometimes you don't need  any kind of saw. You don't need a chainsaw. It's [41:45] not a toy. Right. Well, and similarly AI, as  I'm sure a cybersecurity expert will agree, [41:51] is not a toy. So, think about does  this job require AI truly and why? [41:59] Does this job mean that you're going to solve  something that you need to solve? And if you [42:05] answer yes and yes, then you can go and start  building your prototype. You know, this is a [42:10] a question I'm trying to get answer for. Everyone  wants AI. What for they do not know and they want [42:16] it now. Seriously, I've seen a poster like this.  I was like, this is exactly how it is. I was like, [42:21] fine. Tell me exactly how you're going to use  it because I need to know how exactly I need to [42:26] secure it. You can't just secure it for every use  case. But people just excited about it. They want [42:32] to get their hands on it like a you know, a new  toy. So, it's taking a while to get people to [42:39] articulate exactly what they need the tool  for. Uh but in in the hospital setting for us [42:45] it's very simple metric. If the tool increases the  patient throughput consider the tool, you know, [42:54] has lots of value because that's we that's  one of the things we care about a lot. [43:00] Is getting doctors and uh clinicians to to  see more patients and treat more patients. [43:08] Okay, final question. Um We easy way to  get in trouble when we're talking about [43:13] AIs to predict the future. So, we're going  to go there. Um what sort of advances are [43:17] you most looking forward to over the  or most afraid of over the next year? [43:22] Agents inside the org chart with humans I think  it's going to be a big change. Think about your [43:29] worst boss. Would an agent be better than your  worst boss? Most people I think would say yes. [43:35] Yes. What about your best boss? Most people I  think would say no. What about your average boss? [43:42] That's the question we're going to look to  answer. You know, a boss that's not political, [43:46] that's always available 24/7 cuz it can have  conversations with a huge team all at once, [43:52] that can optimize work more efficiently. I  think that layer, that middle management layer [43:58] being actually driven by agents is something  I'm really excited about from an organizational [44:03] perspective and just from an organizational  psychology perspective. Okay. It's the Skynet [44:09] example, right? You don't want [laughter] I  wouldn't go quite that far. I bring that up [44:13] all the time. Not your example, but I'm worried  about the Skynet example. Agents taking over. And [44:18] you don't want that happening, right? You want AI  to be used as an assist, not take over the world, [44:23] right? So. I'm worried about agents creating  other agents. And every new agent is going to have [44:30] some identity, some access, some autonomy, and  it's going to be an audit nightmare for us to know [44:35] who did what. And some of these agents are going  to be created for sometimes seconds, minutes to do [44:40] something and they disappear before you know that  they misbehaved and you need to shut them down. [44:45] Well, and I can't help but be an optimist.  Yes, there's all these things that we need to [44:49] be worried about, but I'm actually really excited  about humans having far better job satisfaction, [44:57] doing things that they love doing that are  uniquely human, that are tied to judgment, [45:02] empathy, etc. And a lot of the really painful  stuff that we all hate doing really starting to [45:10] fall away and getting the agents to do it. I tend  to look at it much less of the Skynet surveillance [45:16] controlling us and actually taking off all of  those really painful tasks that none of us want [45:23] to do. So. We'll end on an optimistic note. I  do appreciate that this panel has not featured [45:28] people talking about widespread occupational  doom. I hope we're all right about that. [45:33] Thank you all for your expertise. Thank  you all for your attention. Thank you.