Cacti Industries Unveils Gesture Systems to Control Mixed-Brand Fleets

Be the Frontier Firm - Talking Machines With Cacti Industries — Ai4

Ai4

Physical AI will generate far greater economic value than text-based generative models by combining domain-specific knowledge graphs with hands-free voice and gesture interfaces.

AI-assisted summary of Be the Frontier Firm - Talking Machines With Cacti Industries — original source. Check the original for full context.

Source published: 2026-08-18

Detailed notes

## Background Published on YouTube by Ai4 on August 18, 2026, this presentation features Viren Shah, Strategic Advisor and former CDIO of AGCO representing Cacti Industries. The talk explores how frontier organizations create tangible value by adopting an AI-native operating model across strategy acceleration, continuous improvement, and citizen development, focusing on voice and gesture interfaces for physical machinery.

## TL;DR To address severe agricultural labor shortages and feed a growing global population, organizations must use physical AI to radically simplify heavy equipment operations. The presentation highlights Valtra's 'talking tractor,' which utilizes LLMs grounded on structured knowledge graphs and real-time telemetry to provide multilingual voice diagnostics and operational coaching directly to operators. Expanding beyond acoustic interfaces, Cacti Industries demonstrates how computer vision-driven gestures and 3D spatial terrain models enable safe, non-verbal multi-robot fleet orchestration across agriculture, defense, and heavy industry. The talk concludes by emphasizing vendor-agnostic OpenTelemetry standards, cost-effective LLM deployment, and the immense real-world value of physical AI over purely text-based tools.

## Atmosphere & style The presentation carries a pragmatic, forward-looking keynote energy that bridges gritty industrial realities with futuristic multi-agent robotics. Shah speaks with accessible enthusiasm and executive clarity, grounding high-concept ideas like sci-fi spatial command tables in the urgent, practical needs of farmers and aging workforces. The atmosphere feels purposeful and inspiring, leaving the audience with an engaging, tangible vision of how physical AI can transform frontline industrial work beyond traditional text chat interfaces.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=4s) Rising global food demand paired with severe agricultural labor shortages requires AI to dramatically simplify heavy machinery operations. - [00:04](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=4s) An AI-native operating model creates enterprise value across three pillars: Strategy Acceleration, Continuous Improvement, and Citizen Development. - [02:17](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=137s) Because a farmer averages only about 40 lifetime harvests, heavy machinery must deliver extreme operational precision without steep learning curves. - [02:17](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=137s) Valtra developed a voice-enabled 'talking tractor' (Valtra Coach) allowing operators to query machine status, diagnostics, and service points in their native language. - [05:30](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=330s) Industrial voice assistants require LLMs grounded on verified engineering knowledge graphs and real-time telemetry rather than raw document ingestion. - [06:30](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=390s) Computer vision gesture recognition serves as a language-agnostic interface for commanding autonomous multi-robot fleets in high-noise environments. - [09:55](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=595s) Real-time 3D spatial table interfaces enable operators to orchestrate autonomous machinery safely beyond visual line-of-sight across defense, mining, and agriculture. - [11:03](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=663s) Heterogeneous multi-agent enterprise systems require vendor-agnostic architectures and OpenTelemetry (OTel) standards for cross-agent tracing and logging. - [11:03](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=663s) Hands-free, multimodal AI tools enable aging expert technicians to diagnose machinery and guide operations remotely, extending their productive careers. - [13:31](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=811s) Physical AI driving autonomous machinery across vast physical environments represents a significantly larger economic value opportunity than generative AI alone. - [14:02](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=842s) Organizations must move beyond text-based interfaces toward voice and gestures while pragmatically reserving costly LLM reasoning only for complex tasks.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=4s) The Agricultural Challenge and Becoming an AI-Native Frontier Firm

The world's population is projected to reach approximately 9 billion people in a couple of decades, creating immense pressure on global food production. Simultaneously, agriculture faces a severe labor shortage—very few people in the younger generation want to become tractor drivers or manual farm operators. To address this crisis, AI must be utilized to dramatically simplify physical machinery and interfaces. While voice communication is a vital modality, natural human gestures represent an equally critical, often overlooked communication interface when acoustic environments fail or voice is impractical.

Creating real economic value from AI remains a widespread industry struggle. Many organizations get bogged down trying to measure and generate direct ROI. An "AI-native operating model" structures organizational value creation across three distinct pillars:

```mermaid flowchart TD accTitle: Three Pillars of the AI-Native Frontier Firm subgraph S1["1. Strategy Acceleration"] A["New Product Innovations"] --> B["Simplified Product Interfaces"] end subgraph S2["2. Continuous Improvement"] C["Sharpening Existing Ops"] --> D["Supply Chain & Telemetry"] end subgraph S3["3. Citizen Development"] E["Democratized AI Tooling"] --> F["Building Personal Agents"] end ```

- **Strategy Acceleration:** Innovating and simplifying complex products to increase market share, onboard workers faster, and accelerate strategic roadmaps. - **Continuous Improvement:** Sharpening existing operational baselines, fine-tuning supply chains, and optimizing legacy workflows. - **Citizen Development:** Empowering every employee across an organization to adopt AI by building their own personal agents, echoing the democratization seen in the early days of Microsoft Excel.

[02:17](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=137s) Simplifying Machinery: Valtra's Talking Tractor

Farming affords remarkably little margin for error: in the United States, an average farmer gets roughly 40 harvests in their entire lifetime. With only 40 opportunities to achieve a successful yield, every piece of machinery must operate with extreme precision and reliability. However, modern agricultural equipment is extraordinarily complex, filled with sophisticated capabilities buried in dense manuals that operators rarely master. Compounding this challenge, global agricultural manufacturers must serve diverse workforces who speak dozens of different languages.

To bridge this operational divide, Valtra introduced the concept of the "talking tractor," embedding conversational voice AI directly into the machine cabin via a mounted tablet interface called Valtra Coach on the Q Series 305 tractor. Through natural voice interaction, an operator can speak in their native tongue to query machine status, telemetry, operational instructions, and maintenance requirements:

- **Performance queries:** Asking *"How much horsepower do you have?"* triggers an immediate verbal response: *"The Valtra Q305 tractor has up to 305 horsepower with boosting engaged."* - **Diagnostic telemetry:** The assistant surfaces real-time metrics and historical telemetry by integrating Valtra Coach directly with the Valtra Connect platform. - **Service point mapping:** Inquiring *"What kind of service points do we have on the machine?"* prompts an exhaustive catalog of maintenance checkpoints, including engine oil and filters, fuel filters, transmission pressure filters, final drive oil, differential oil, front hub oil, and lubrication grease points.

[05:30](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=330s) Building Knowledge Graphs for Mission-Critical LLM Grounding

Deploying an LLM into an industrial vehicle is far more sophisticated than simply uploading operation manuals into ChatGPT. Raw LLM ingestion fails to understand domain-specific operational logic, interdependencies, and physical inferences. Achieving high-precision answers required domain experts to construct verified knowledge graphs from engineering documentation and telemetry streams. The language model was subsequently grounded on top of these curated knowledge graphs, ensuring accurate cross-lingual translations and factual technical answers across multiple tractor product lines.

```mermaid flowchart LR accTitle: Grounded Voice AI Architecture A["Raw Manuals & Engineering Specs"] --> B["Expert Validation & Structuring"] B --> C["Curated Knowledge Graphs"] C --> D["Targeted LLM Layer"] D --> E["Valtra Coach Voice Interface"] F["Valtra Connect Telemetry"] --> E ```

- Valtra's talking tractor received recognition at Agritechnica for pioneering conversational intelligence in heavy agricultural machinery. - The overarching goal is deploying this voice capability across all fleet models to dramatically accelerate operator onboarding and help farmers feed the world.

[06:30](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=390s) Beyond Voice: Multi-Robot Orchestration via Gesture Interfaces

Field environments often involve diverse fleets composed of excavators, tractors, drones, and autonomous implements built by seven different manufacturers. In intensely loud environments or high-stakes field settings where acoustic voice commands fail, alternative non-verbal modalities become mandatory. Gesture recognition serves as a universal, language-agnostic interface that humans have understood throughout history.

Working with Cacti Industries, this interface framework applies computer vision to translate human hand gestures into fleet commands:

- **Point-to-Target Navigation:** An operator points to a specific spot on the ground, commanding an autonomous mobile robot to drive directly to that coordinate. - **Tool Fetching & Hazardous Retrieval:** Workers in dangerous construction or industrial zones use simple waving gestures to call autonomous units carrying required tools, avoiding hazardous transit. - **Dismissal & Return-to-Base:** A universal "dismiss" or "wrap up" hand wave signals multiple co-working robots simultaneously that their assignment is complete and directs them to return to their docking stations.

```mermaid flowchart TD accTitle: Multi-Robot Gesture Orchestration Workflow subgraph HumanInput["Field Operator"] G1["Point to Location"] G2["Summon Tool Gesture"] G3["Dismiss / Return Gesture"] end subgraph VisionEngine["Cacti Gesture Processing"] V1["Vision Tracking & Intent Parsing"] V2["Coordinate & Spatial Mapping"] end subgraph FleetExecution["Autonomous Fleet"] R1["Robot A: Navigate to Point"] R2["Robot B: Tool Delivery"] R3["Fleet: Return to Base"] end HumanInput --> VisionEngine VisionEngine --> FleetExecution ```

[09:55](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=595s) 3D Spatial Interfaces and Defense-Grade Fleet Safety

Beyond simple visual tracking, multi-machine coordination utilizes real-time 3D spatial engines—analogous to holographic command environments seen in films like *Avatar*. Operators manipulate interactive 3D table models representing physical terrain to coordinate multi-robot movements across vast landscapes. This visual command layer allows human supervisors to direct complex machines from a safe standoff distance beyond visual line-of-sight.

- **Defense Applications:** The technology is actively being explored with defense organizations, where voice silence is tactically required and keeping human warfighters physically separated from hazardous combat engineering or scouting operations is a top priority. - **Cross-Domain Utility:** Provides identical safety and coordination benefits across heavy civil construction, forestry, mining, and large-scale agriculture.

[11:03](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=663s) Vendor-Agnostic Architecture, OpenTelemetry, and Remote Expertise

As enterprise AI adoption expands, managing multi-vendor heterogeneity becomes the central architectural bottleneck. Organizations must coordinate disparate proprietary agents—such as Salesforce Agentforce, in-house custom customer agents powered by diverse foundation models, and niche third-party point solutions. Solving this integration requires a vendor-agnostic architecture supported by standardized logging frameworks:

- **OTel (OpenTelemetry) for Agentic AI:** Just as traditional systems relied on standardized application logs, modern multi-agent systems require OpenTelemetry (OTel) standards to trace, decipher, and orchestrate communications and logs across heterogeneous models. - **Extending Skilled Labor Lifespans:** Skilled agricultural technicians and machine operators are aging, with few young replacements entering the trades. Hands-free, multimodal AI tools enable veteran experts to guide operations, diagnose faults, and orchestrate machinery remotely from a safe distance, effectively extending their working careers while keeping them out of hazardous field environments.

[13:31](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=811s) The Rising Value of Physical AI in Agriculture

In the realm of food and agriculture, operations frequently extend far beyond human line of sight, making the integration of physical AI a critical necessity. While industry discourse has overwhelmingly focused on generative AI, physical AI represents a massive, tangible frontier. Physical AI stands to generate significantly more real-world value than pure generative models alone. For anyone operating in the product development space, learning how to build, integrate, and deploy physical AI systems will be foundational to creating long-term business impact.

```mermaid flowchart LR accTitle: Physical AI Value in Industrial Operations subgraph Operations ["Physical Environment"] A["Beyond Line of Sight Operations"] B["Physical AI Integration"] end subgraph Impact ["Value Creation"] C["Generative AI Focus"] D["High-Value Physical AI Deployment"] end A --> B B --> D C -. "Traditional focus".-> D ```

- **Beyond Line of Sight:** Agricultural operations and heavy machinery work at scale across vast terrains where direct human oversight is impossible. - **Physical vs. Generative AI:** While text and image generation capture headlines, physical AI driving autonomous machines and physical systems represents the larger economic value driver.

---

[14:02](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=842s) Moving Beyond Text: Voice, Gestures, and Smarter LLM Utilization

Examining how humans interface with intelligent machines highlights why organizations must rethink their input modalities. While voice and gestures represent the future of machine interaction, not every feature requires an expensive voice model or an overextended Large Language Model (LLM). Today, many applications overuse heavy LLM frameworks for standard chat routines where traditional chat architecture suffices, switching to LLMs only when deep reasoning is genuinely required to keep computing costs sustainable.

At the same time, the industry remains overly reliant on text-based interfaces. In real-world industrial and field environments, natural voice and gesture-based interaction offer far greater efficiency, safety, and value. Embracing voice and gesture controls transforms how frontline workers command complex machinery.

> "We are too dependent on text... and I think voice and gesture actually brings a lot more value."

Wrapping up the session, the speaker describes himself as an AI evangelist eager to connect and collaborate with others in the field—joking that whether people reach out via "voice or gestures, either one works"—and thanks the audience for their time and attention.

- **Pragmatic LLM Deployment:** Manage computational costs by leveraging standard chat mechanisms for simple routing, reserving full LLM capabilities for complex tasks. - **Next-Gen Interfaces:** Transition away from text-heavy screens toward voice and gesture interfaces that integrate naturally into physical work environments.

## Notable quotes > [00:04](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=4s) Strategy Acceleration: Innovating and simplifying complex products to increase market share, onboard workers faster, and accelerate strategic roadmaps. — Viren Shah

> [00:04](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=4s) Continuous Improvement: Sharpening existing operational baselines, fine-tuning supply chains, and optimizing legacy workflows. — Viren Shah

> [00:04](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=4s) Citizen Development: Empowering every employee across an organization to adopt AI by building their own personal agents, echoing the democratization seen in the early days of Microsoft Excel. — Viren Shah

> [02:17](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=137s) How much horsepower do you have?

> [02:17](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=137s) The Valtra Q305 tractor has up to 305 horsepower with boosting engaged. — Valtra Coach

> [02:17](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=137s) What kind of service points do we have on the machine?

> [14:02](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=842s) We are too dependent on text... and I think voice and gesture actually brings a lot more value. — Viren Shah

> [14:02](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=842s) voice or gestures, either one works — Viren Shah

## Mentioned Viren Shah, AGCO, Cacti Industries, Valtra, Valtra Coach, Valtra Connect, Valtra Q Series 305, Agritechnica, Microsoft Excel, Salesforce Agentforce, OpenTelemetry (OTel), Avatar, ChatGPT

## Video description Viren Shah - Strategic Advisor and Ex-CDIO (AGCO), Cacti Industries

Frontier firms create value by becoming AI-native organizations by deploying AI in three work-streams: Strategy Acceleration (New Product Introductions, etc.) Continuous Improvement—sharpening what already exists (supply chain optimization etc.); and Citizen Development putting AI capability directly in the hands of the people (Personal Agents etc.).

Valtra's Talking Tractor "Award Winner at Agritechnica" is a great example of Strategy Acceleration in action: a voice AI agent, trained on machine data which makes a complex machine simple to operate, in the operators own language, ensuring seamless interaction. Natural voice and gesture are becoming the interface to physical machines built to command entire fleets on the field and a platform Cacti is building to enable large landscapes.

If you want to learn how AI can make complex products simple by leveraging voice to solve global problems, such as world hunger, this session is for you.

[Source](https://www.youtube.com/watch?v=NSNOUR-H_uY)

中文简报

Cacti Industries 推出用于控制跨品牌机队的手势控制系统物理 AI 通过将特定领域知识图谱与免提语音及手势界面相结合，将创造远超纯文本生成式模型的经济价值。为应对严峻的农业劳动力短缺并满足不断增长的全球人口粮食需求，各机构必须利用物理 AI 从根本上简化重型设备的操作。本次演讲重点介绍了 Valtra 的“会说话的拖拉机”，该设备利用基于结构化知识图谱和实时遥测数据的大语言模型（LLM），直接为操作人员提供多语言语音诊断和操作指导。除语音交互界面之外，Cacti Industries 还展示了如何借助计算机视觉驱动的手势与 3D 空间地形模型，在农业、国防及重工业领域实现安全、非语言的多机器人机队协同调度。演讲最后强调了独立于供应商的 OpenTelemetry 标准、高性价比的 LLM 部署，以及物理 AI 相比纯文本工具所展现出的巨大现实世界价值。面对全球粮食需求增长及农业劳动力短缺，企业需通过“AI 原生”三大支柱（战略加速、持续改进、全员开发）大幅简化重型机械的操作与交互门槛。 (0:04) 针对农户一生平均仅有约 40 次收获期的极低容错率挑战，Valtra 推出语音智能拖拉机 Valtra Coach，允许操作员通过母语实时查询机械状态、遥测数据与保养点。 (2:17) 工业级语音助手摒弃了直接将技术手册灌入大模型的粗放做法，转而通过构建经工程验证的知识图谱结合实时遥测进行知识对齐，以保障跨语种与技术问答的精确性。 (5:30) 在高噪音或无声作业环境中，计算机视觉手势识别成为跨厂商的多机器人协作界面，使工人能够直观下达指定点位移动、工具调用及返航等指令。 (6:30)

Reading summary

To address severe agricultural labor shortages and feed an expanding global population, heavy industry must move beyond text-based software to deploy physical AI grounded in multimodal voice and gesture interfaces. Demographic Pressures and the AI-Native Framework (0:04) Global population is projected to reach approximately 9 billion within several decades, placing intense pressure on agricultural output at a time when younger generations are largely avoiding careers as manual operators or tractor drivers. Because American farmers average only about 40 lifetime harvests, machinery requires extreme operational reliability. To capture economic returns from AI, organizations are advised to adopt a three-pillar framework: accelerating product strategy through simplified interfaces, driving continuous improvement across supply chains and operational baselines, and supporting citizen development by allowing employees to build personalized agents similar to early spreadsheet adoption. Grounded Conversational Intelligence in Heavy Machinery (2:17) Modern industrial machinery often buries critical capabilities in dense manuals that operators struggle to navigate, especially across linguistically diverse workforces. Valtra addressed this operational bottleneck by introducing Valtra Coach on its Q Series 305 tractor. The tablet-based voice interface allows drivers to query performance metrics, diagnostic telemetry via the Valtra Connect platform, and maintenance checkpoints—such as engine oil, transmission pressure filters, and differential oil—in their native languages. Deploying voice AI into industrial equipment required more than standard document ingestion into a general large language model. Because standard models fail to comprehend physical dependencies and industrial logic, domain experts constructed verified knowledge graphs from engineering specifications and telemetry streams. Grounding the language model on these curated graphs ensured factual responses and accurate cross-lingual translations across product fleets. Non-Verbal Gestures and Spatial Fleet Orchestration (6:30) In loud outdoor environments or defense operations where voice commands are acoustically ineffective or tactically compromised, alternative modalities are necessary. Collaborating with Cacti Industries, computer vision systems interpret natural human gestures to command mixed fleets of excavators, tractors, drones, and implements. Demonstrated capabilities include pointing to target coordinates for autonomous robot navigation, waving to summon tool-delivery units across hazardous zones, and giving dismissal gestures to send multiple machines back to docking stations. Command structures can further scale through interactive 3D spatial engines. By manipulating terrain models representing physical work sites, human supervisors can direct autonomous machinery from safe standoff distances beyond visual line of sight. This remote coordination framework applies to combat engineering, forestry, mining, and large-scale agriculture. Heterogeneous Architecture and Practical Physical AI Deployment (11:03) As multi-agent enterprise deployments proliferate, organizations face integration hurdles across proprietary tools like Salesforce Agentforce, specialized point solutions, and custom foundation models. Managing this heterogeneity requires vendor-agnostic architectures instrumented with OpenTelemetry standards to trace and orchestrate communications across models. These multimodal, hands-free interfaces also permit aging agricultural specialists to diagnose machine faults and oversee operations remotely, extending their careers without requiring physical presence in hazardous conditions. The speaker argues that physical AI operating beyond direct line of sight will ultimately generate substantially greater real-world value than purely generative, text-based software. To remain sustainable, organizations must manage computational costs by routing basic interactions through conventional chat frameworks, reserving complex language models strictly for deep reasoning tasks. Transitioning heavy machinery from text prompts to physical AI, natural voice, and gesture interfaces offers substantial productivity and safety gains. However, successful scaling hinges on disciplined computational cost controls, verified knowledge graphs, and open telemetry standards capable of harmonizing disparate multi-vendor fleets.

精炼阅读摘要

Cacti Industries与行业讲者认为，面对全球人口增长与农业劳动力断层的严峻压力，工业企业必须通过“实体AI（Physical AI）”以及融合语音和手势的多模态交互来大幅降低物理装备的操作门槛。该观点强调，相较于纯生成式AI，将工业知识图谱、开放遥测标准与非语言物理控制深度结合，才能在复杂的超视距作业场景中释放更显著的实体经济价值。劳动力危机与实体AI的战略价值 (0:04) 根据讲者引用的行业预测，全球人口预计在未来几十年内达到约90亿，给粮食供给带来巨大压力；与此同时，年轻一代极少有人愿意从事繁重的农机驾驶，使农业面临深刻的劳动力结构危机。由于美国农民一生平均仅有约40次收获季节的容错窗口，农业机械必须具备极高的作业精度与可靠性。讲者指出，多数企业目前因纠结于直接投资回报率（ROI）的测算而在AI落地中受阻，企业应当构建“AI原生运营模型”，通过战略加速、持续优化以及支持员工构建个人智能体的“全员开发（Citizen Development）”三大支柱来创造系统性价值。在此基础上，讲者主张实体AI代表了远超纯生成式AI的经济增长前沿。在农业及重工业领域，机械作业往往需要超越操作人员的直接视线范围（Beyond Line of Sight），仅凭文本或图像生成无法满足实体生产的需求。将AI深度融入物理机械与自主移动装备中，将成为产品开发领域创造长期商业回报与生产力突破的核心动力。知识图谱赋能工业级车载语音 (2:17) 现代重型农机功能繁多且手册晦涩，加上全球农业劳动力语种多样，极易造成操作断层与误操作。为此，农机品牌Valtra在Q系列305拖拉机驾驶室内集成了名为Valtra Coach的语音交互平板系统，打造出获得德国汉诺威农机展（Agritechnica）认可的“会说话的拖拉机”。农机操作员可以使用母语直接向系统询问机器状态、操作指令与维护信息，系统与Valtra Connect云端遥测平台打通，不仅能即时核实动力参数（例如在动力提升模式下功率可达305马力），还能列举机油滤芯、燃油滤芯、变速箱压力滤清器等全套维护保养检查点。讲者强调，工业装备的语音AI部署绝不能直接依赖未经处理的大语言模型（LLM）。通用大模型缺乏特定工业领域的物理逻辑与工程关联认知，极易产生技术偏差。该方案由领域专家先基于技术说明书与实时遥测数据流构建经验证的领域知识图谱，再通过知识图谱对底层大模型进行严格的物理事实锚定（Grounding），从而消除事实幻觉，确保技术解答的高精度以及跨机型、跨语言环境下的工业级可靠性。复杂现场的多机器人手势控制 (6:30) 在农田与重工业作业现场，作业机队通常由挖掘机、拖拉机、无人机等来自不同制造商的设备共同组成，且强噪音环境常常导致声控语音交互彻底失效。Cacti Industries开发了基于计算机视觉的手势控制架构，将人类手势转化为无需语言翻译的通用操作指令：作业人员通过指向特定地面坐标即可调度自主移动机器人前往该点，在危险作业区通过挥手手势召唤运载工具的机器人，或通过通用的“解散”挥手动作指挥多台协同机器人统一步行返回停泊站。为了实现安全的超视距多机编排，该系统进一步引入了类似科幻电影交互界面的实时3D空间引擎。现场指挥人员通过可交互的3D地形数字沙盘统筹各机动单元的行进路径，使人类操作员能够在危险作业半径之外实施远程监控与决策。讲者透露，该技术正在与国防机构展开应用探索，以满足战场静音战术需求并确保人员免受战斗工兵及侦察任务的高危威胁；同时该交互模式亦可直接迁移至林业、采矿、大规模农场与土木工程等工业领域。跨厂商多智能体治理与算力部署 (11:03) 随着企业内部分别引入Salesforce Agentforce、专有客资模型以及各类第三方智能体，异构多智能体的统一治理成为核心架构痛点。讲者提出，企业需要构建独立于具体厂商的架构体系，并采用开放遥测标准OpenTelemetry（OTel）来追踪、解析与协调来自不同模型的交互日志。此外，免手持的多模态AI工具能够有效缓解行业技能断层：由于经验丰富的老技工日益老龄化而缺乏年轻接班人，老专家无需置身于恶劣危险的现场，即可在后方借助远程AI工具诊断设备故障并指导机械集群运行，从而延长其技术生命周期。在交互形态与算力控制上，讲者主张工业界应当打破对传统文本界面的过度依赖，转向效率与安全性更高的语音及手势形态。同时，讲者告诫企业应采取务实的计算部署策略，避免在常规对话路由中滥用开销高昂的大模型框架，应当通过标准轻量级对话架构处理基础指令，仅在确需深度逻辑推理时调用复杂LLM，以维持工业AI系统的经济可持续性。该节目强调工业智能化应走出纯软件界面的局限，转向融合语音、手势与知识图谱的实体AI系统，以应对老龄化与极端环境作业的现实挑战；不过，在多厂商异构设备广泛并存的现状下，非语言空间控制协议的行业标准化进程、跨机型物理知识图谱的长期维护成本，以及远距离多机协同中的通信抗干扰能力，仍是该方案在更大规模落地应用时面临的关键考验。

Detailed notes — zh

## Background 该内容由行业频道 Ai4 于 2026 年 8 月 18 日发布在 YouTube 平台。视频由 Cacti Industries 战略顾问、农机巨头 AGCO 前 CDIO Viren Shah 主讲，主要介绍了企业如何通过“战略加速、持续改进与全民开发”三大支柱转型为 AI 原生组织，并以在 Agritechnica 获奖的 Valtra“会说话的拖拉机”为例，阐述如何利用语音与手势等物理 AI 交互简化复杂机械操作，进而应对农业劳动力短缺等全球性挑战。 ## TL;DR 演讲围绕全球人口增长与农业熟练劳动力断层的矛盾展开，指出农场主一生仅有约 40 次收获机会，因此大型复杂机械的操作必须极度精密且易于上手。芬兰农机品牌 Valtra 结合专家知识图谱、大语言模型与实时遥测数据打造了 Valtra Coach，让操作员能以自然多语言语音直接调取机况与维保指引。在此基础上，Cacti Industries 进一步推进了针对异构设备集群的手势交互与 3D 数字孪生协同调度，攻克恶劣环境与静默场景的控制痛点。最后，演讲强调实体领域的“物理 AI”价值未来将超越纯文本生成式 AI，企业应构建供应商无关的架构、掌握 Agent 的 OTel 日志能力，并合理兼顾传统交互与大模型以控制成本，推动资深劳动力的技能延寿。 ## Atmosphere & style 整场演讲务实笃定且充满浓厚的工业现场感，演讲者没有陷入空泛的技术炒作，而是将前沿 AI 稳稳扎根于泥土飞扬的田间、高噪音建筑工地与国防复杂作业等严苛现实中。叙事由“农夫一生的 40 次收获”这类极富共情力的微观痛点层层递进至异构机器人集群的宏大编排，沉稳明快又信息密集。现场展示中无声手势调度与实体机械交互的直观碰撞，让整场分享既带着传统重工产业的硬核质感，又洋溢着以多模态物理 AI 破除纯文本局限、重塑人机协作范式的远见与幽默。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=4s) 全球面临农业劳动力严重短缺的挑战，借助 AI 简化复杂的农机操作成为解决粮食安全等全球性问题的关键途径。 - [00:04](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=4s) 企业要成为前沿企业（Frontier Firm），应构建包含“战略加速”、“持续改进”和“全民开发”三大支柱的 AI 原生运营模式。 - [02:17](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=137s) 农场主一生平均仅有约 40 次收获季，容错率极低，因此农机设备不仅要精密可靠，更必须做到极易上手。 - [02:17](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=137s) Valtra 推出的“会说话的拖拉机”结合了知识图谱、领域专家校验与 LLM，实现多语言自然语音交互与遥测数据查询。 - [06:30](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=390s) 在农场、工地或国防等高噪声或无法语音的恶劣场景下，通用手势沟通能打破语言障碍，实现对异构机器人集群的高效调度。 - [06:30](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=390s) 结合 3D 虚拟数字孪生沙盘与多模态手势控制，操作员可在远离危险区域的后方进行异构设备的时空协同编排。 - [11:03](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=663s) 企业在集成各类异构 Agent 时需坚持供应商无关架构，掌握 OpenTelemetry（OTel）遥测日志分析正在成为核心技术能力。 - [11:03](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=663s) 免提操作与超视距多模态远程交互工具，能够让老技师无需亲临一线恶劣环境，有效实现资深人才的技能延寿。 - [13:31](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=811s) 相比纯数字领域的生成式 AI，直接作用于视距外现实世界的“物理 AI”（Physical AI）未来有望撬动更庞大的实体经济价值。 - [14:02](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=842s) 软件与硬件交互应打破纯文本依赖，转向语音和手势等多模态形态，同时避免滥用 LLM，应合理搭配传统对话机制以优化调用成本。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=4s) 农业劳动力短缺与“前沿企业”的 AI 价值创造全球人口预计在未来几十年内增长至约 90 亿，但农业领域正面临严重的劳动力短缺——几乎没有人愿意主动从事拖拉机驾驶等农田重体力劳动。为了应对这一挑战，演讲者提出了如何利用人工智能（AI）简化复杂农机操作的方案，并重点探讨了语音和手势这两种常被忽视的多模态交互方式。特别是在噪声环境或无法使用语音的场景下，通用手势沟通能够作为极佳的人机协作桥梁。当前各行各业都在探索如何通过 AI 实现明确的投资回报（ROI），演讲者提出了打造“AI 原生运营模式”（AI-Native Operating Model）来帮助企业成为“前沿企业”（Frontier Firm）。从 AI 中挖掘价值的核心路径通常涵盖三大支柱： ```mermaid flowchart TD accTitle: 前沿企业 AI 价值创造的三大支柱 subgraph Pillars["前沿企业 AI 价值创造的三大支柱"] A["战略加速 (Strategy Acceleration) 新产品创新与简化、抢占市场份额"] B["持续改进 (Continuous Improvement) 优化现有流程与供应链效率"] C["全民开发 (Citizen Development) 全员采纳 AI、构建个人 Agent"] end ``` - **战略加速（Strategy Acceleration）**：通过打造更优秀、更简化的创新产品，加速战略落地并扩大市场占有率。 - **持续改进（Continuous Improvement）**：利用 AI 持续打磨和优化现有的业务与技术流程。 - **全民开发与全员采纳（Citizen Development）**：推动组织内每个人主动使用并构建自己的 AI Agent（正如当年 Microsoft Excel 的普及过程一样），这是决定 AI 落地成败的最关键环节。 --- ### [02:17](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=137s) 农机操作复杂性与 Valtra“会说话的拖拉机” 在美国，一名农场主在其一生中平均仅有大约 40 次收获季（40 harvests in their lifetime）。这意味着农场主一生中只有 40 次把农田作业做对的机会，因此农机设备不仅必须极度精密、稳定可靠，更必须做到极易上手。现代大型农机功能极多，犹如复杂的高档汽车，普通人很难通读厚重的使用手册；加之全球化制造企业面临多语言本地化的巨大挑战，当一名非英语母语的工人进入驾驶室时，往往连如何启动设备都无从下手。为了彻底解决这一痛点，芬兰农机品牌维尔特拉（Valtra）推出了业界首款“会说话的拖拉机”——配备了 **Valtra Coach** 智能语音辅助系统的 Q 系列 305（Q Series 305）。 ```mermaid flowchart LR accTitle: Valtra Coach 知识图谱与语音交互架构 subgraph DataEngine["数据与知识构建"] M["产品手册与运维文档"] --> E["领域专家介入验证"] E --> KG["结构化知识图谱 (Knowledge Graph)"] T["Valtra Connect 遥测实时数据"] --> KG end subgraph Runtime["交互与推理"] KG --> LLM["大语言模型 (LLM)"] U["操作员语音提问"] --> LLM LLM --> V["Valtra Coach 智能语音答复"] end ``` 在 Agritechnica 农机展的现场演示中，驾驶舱侧边面板安装了一块运行 Valtra Coach 的平板电脑，操作员可以直接进行多轮语音交互： - 询问动力参数：“你的马力有多大？”系统即刻语音回答：“Valtra Q305 拖拉机在开启增压模式（boosting engaged）时，最高可提供 305 匹马力。” - 询问维保点分布：“机器上有哪些保养检修点？”系统能精准列举出发动机机油滤清器、燃油滤清器、变速箱压力滤清器、主减速器机油、差速器机油、前轮毂机油及底盘润滑点等详尽清单。 - 结合 Valtra Connect 系统的实时遥测统计数据，系统将设备过去数天、数周的运行指标和全车功能统一整合至单一交互界面中。该项目并非简单地把说明书上传给 ChatGPT 即可生效，因为专业农机的上下文推理要求极高的准确性。工程团队引入了大量行业专家进行严密校验，先构建出高精度的领域知识图谱（Knowledge Graph），再在其上运行大语言模型（LLM），最终实现了多车型、多语言的跨平台部署，并一举斩获了 Agritechnica 创新大奖。 --- ### [06:30](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=390s) 手势交互与多机器人协同调度在更复杂的真实作业场景中，农场或工地上往往并存着无人机、拖拉机、挖掘机等来自七八家不同制造商的异构设备。操作人员若要在嘈杂、恶劣的环境下统一调度这批机器，单纯依赖语音交互往往难以奏效；在国防与军工前线，语音指令也常常因静默要求或战场噪声而受限。针对这一痛点，Cacti Industries 正在研发基于**通用肢体手势（Gestures）**的多机器人协同调度平台： - **手势的跨语言通用性**：手势是人类最原始的通用交流形式，无论母语为何，特定指向、召唤和挥手告别的肢体动作在全世界具有高度一致的认知。 - **实景作业应用演示**：现场展示的无声演示视频中，操作员只需用手指指向指定作业位置，自主移动机器人（AMR）便会精准行驶至该点；在危险的建筑工地中，工人无需冒险走入险境取工具，只需通过手势即可指挥协同机器人运送工具；完成任务后，操作员做出特定挥手手势，两台协同作业的机器人即刻同步接收指令并返回基地。 - **3D 沙盘协同控制**：该技术不仅支持面对面手势，还结合了 3D 虚拟数字孪生沙盘。操作者可以通过全息/3D 平面在远离危险区域的安全后方对整个异构机器人集群进行精确的时空协同编排。这一方案目前已在国防安全领域深入合作，用于在复杂前线最大程度保障士兵生命安全。 --- ### [11:03](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=663s) 供应商无关架构、OTel 日志与技能延寿随着企业引入的 AI 工具日益复杂，底层架构的解耦与标准化成为关键瓶颈。企业不能被单一厂商或特定模型绑定，必须构建**供应商无关（Vendor-Agnostic）的集成架构**： - **Agent 互联与 OpenTelemetry（OTel）技能**：现实企业环境中，可能既运行着 Salesforce Agentforce，又运行着调用三款不同大模型的自研客户客服 Agent，同时还接入了第三方专属小工具。为了让这些异构 Agent 顺畅协作并厘清调用链，理解并分析各 Agent 生成的遥测日志（即 OTel / OpenTelemetry）正在成为企业技术团队不可或缺的核心技能，其实质是传统应用系统日志在智能体时代的演进。 - **免提操作与资深劳动力技能延寿（延长职业寿命）**：在农业、家电维修与重工业领域，技术人员与熟练操作工普遍面临严重的老龄化问题，且年轻一代鲜有人愿意深入现场从事维修与操作。借助“远距离操作/免提交互”的 AI 工具与超视距（Beyond Line of Sight）多模态技术，老技师无需亲临恶劣现场即可远程输出专业经验与决策，从而有效延长了稀缺专业人才的职业寿命。 ### [13:31](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=811s) 物理 AI 的崛起：在农业与实体产品中释放超越生成式 AI 的价值在食品与农业等实体产业中，大量关键作业和环境要素其实都处于作业人员的“视线之外”（beyond the line of sight）。面对这些复杂的非直观场景，引入“物理 AI”（Physical AI）成为了推动产业变革的关键核心。业界眼下虽然将绝大部分注意力都倾注在生成式 AI（Generative AI）的话题上，但物理 AI 正在切实落地，并且它未来所创造的实际商业与社会价值，极有可能超越当前大家聚焦的生成式 AI。对于任何身处硬件、工业或实体产品领域的团队与企业而言，如何将物理 AI 深度集成并应用到具体的产品流程中，将是决定未来竞争力的重中之重。将智能化能力延伸至物理设备与现实世界，不仅能够化解诸如全球粮食安全等复杂挑战，也为传统重工业装备的智能化升级指明了方向。 - **视距外作业的智能化**：农业与大型机械作业中存在大量视线盲区与环境变量，物理 AI 是感知、解析与决策的核心。 - **价值维度的重估**：相比纯数字领域的生成式 AI，直接作用于物理世界的 AI 系统能够撬动更庞大的实体经济价值。 ```mermaid flowchart TD accTitle: 实体产品智能化演进路径 subgraph PhysicalWorld["物理实体领域 (如农业/装备制造)"] A["复杂现实环境与视距外变量"] --> B["物理 AI (Physical AI) 感知与执行"] end subgraph Interaction["下一代多模态交互"] B --> C["语音 (Voice) 与手势 (Gesture) 交互"] C --> D["简化复杂机械操作 / 赋能机队协同"] end ``` --- ### [14:02](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=842s) 摆脱纯文本依赖：语音、手势交互与务实的大模型成本控制在人机交互界面的设计上，未来的智能化交互绝不仅限于语音，但也绝不应过度固守在文本形式上。当前许多产品在实现聊天功能时陷入了误区，盲目“过度消耗并滥用 LLM（大语言模型）的能力”，而实际上完全可以先借助成熟易用的传统对话机制打底，仅在真正需要复杂语义理解与生成的关键节点切换到 LLM，从而有效控制运营成本。与此同时，当前的软件交互依然“过度依赖文本”（too dependent on text）。在未来的 AI 演进路径中，语音（Voice）与手势（Gesture）必将占据举足轻重的核心地位。无论是操纵复杂的田间农机还是管理整个作业机队，自然流畅的语音与肢体手势交互都能释放出远超传统键盘文本的交互价值。演讲者总结道，作为一名 AI 布道者，非常乐意与各界展开广泛交流，并幽默地表示后续沟通“无论是用语音还是手势都可以”。 - **务实的 LLM 调用策略**：合理搭配传统聊天交互逻辑与大语言模型，避免不必要的算力浪费与调用成本超支。 - **多模态交互范式转移**：逐步打破以文本为核心的输入惯性，转向更符合人机协作直觉的语音和手势多模态交互。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=4s) Frontier firms create value by becoming AI-native organizations by deploying AI in three work-streams: Strategy Acceleration (New Product Introductions, etc.) Continuous Improvement—sharpening what already exists (supply chain optimization etc.); and Citizen Development putting AI capability directly in the hands of the people (Personal Agents etc.). — Viren Shah > [02:17](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=137s) 在美国，一名农场主在其一生中平均仅有大约 40 次收获季（40 harvests in their lifetime）。这意味着农场主一生中只有 40 次把农田作业做对的机会，因此农机设备不仅必须极度精密、稳定可靠，更必须做到极易上手。 — Viren Shah > [02:17](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=137s) Valtra Q305 拖拉机在开启增压模式（boosting engaged）时，最高可提供 305 匹马力。 — Valtra Coach > [02:17](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=137s) 该项目并非简单地把说明书上传给 ChatGPT 即可生效，因为专业农机的上下文推理要求极高的准确性。工程团队引入了大量行业专家进行严密校验，先构建出高精度的领域知识图谱（Knowledge Graph），再在其上运行大语言模型（LLM）。 — Viren Shah > [06:30](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=390s) 手势是人类最原始的通用交流形式，无论母语为何，特定指向、召唤和挥手告别的肢体动作在全世界具有高度一致的认知。 — Viren Shah > [11:03](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=663s) 理解并分析各 Agent 生成的遥测日志（即 OTel / OpenTelemetry）正在成为企业技术团队不可或缺的核心技能，其实质是传统应用系统日志在智能体时代的演进。 — Viren Shah > [11:03](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=663s) 借助“远距离操作/免提交互”的 AI 工具与超视距（Beyond Line of Sight）多模态技术，老技师无需亲临恶劣现场即可远程输出专业经验与决策，从而有效延长了稀缺专业人才的职业寿命。 — Viren Shah > [13:31](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=811s) 业界眼下虽然将绝大部分注意力都倾注在生成式 AI（Generative AI）的话题上，但物理 AI 正在切实落地，并且它未来所创造的实际商业与社会价值，极有可能超越当前大家聚焦的生成式 AI。 — Viren Shah > [14:02](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=842s) 当前许多产品在实现聊天功能时陷入了误区，盲目“过度消耗并滥用 LLM（大语言模型）的能力”，而实际上完全可以先借助成熟易用的传统对话机制打底，仅在真正需要复杂语义理解与生成的关键节点切换到 LLM，从而有效控制运营成本。 — Viren Shah > [14:02](https://www.youtube.com/watch?v=NSNOUR-H_uY&t=842s) 当前的软件交互依然“过度依赖文本”（too dependent on text）。在未来的 AI 演进路径中，语音（Voice）与手势（Gesture）必将占据举足轻重的核心地位。 — Viren Shah ## Mentioned Viren Shah, Cacti Industries, AGCO, Valtra, Valtra Coach, Valtra Q Series 305, Valtra Connect, Agritechnica, OpenTelemetry (OTel), Salesforce Agentforce, ChatGPT, Microsoft Excel ## Video description Viren Shah - Strategic Advisor and Ex-CDIO (AGCO), Cacti Industries Frontier firms create value by becoming AI-native organizations by deploying AI in three work-streams: Strategy Acceleration (New Product Introductions, etc.) Continuous Improvement—sharpening what already exists (supply chain optimization etc.); and Citizen Development putting AI capability directly in the hands of the people (Personal Agents etc.). Valtra's Talking Tractor "Award Winner at Agritechnica" is a great example of Strategy Acceleration in action: a voice AI agent, trained on machine data which makes a complex machine simple to operate, in the operators own language, ensuring seamless interaction. Natural voice and gesture are becoming the interface to physical machines built to command entire fleets on the field and a platform Cacti is building to enable large landscapes. If you want to learn how AI can make complex products simple by leveraging voice to solve global problems, such as world hunger, this session is for you. [Source](https://www.youtube.com/watch?v=NSNOUR-H_uY)

Source quotes

[0:04] Viren Shah: Frontier firms create value by becoming AI-native organizations by deploying AI in three work-streams: Strategy Acceleration (New Product Introductions, etc.) Continuous Improvement—sharpening what already exists (supply chain optimization etc.); and Citizen Development putting AI capability directly in the hands of the people (Personal Agents etc.). [2:17] Viren Shah: 在美国，一名农场主在其一生中平均仅有大约 40 次收获季（40 harvests in their lifetime）。这意味着农场主一生中只有 40 次把农田作业做对的机会，因此农机设备不仅必须极度精密、稳定可靠，更必须做到极易上手。 [2:17] Valtra Coach: Valtra Q305 拖拉机在开启增压模式（boosting engaged）时，最高可提供 305 匹马力。 [2:17] Viren Shah: 该项目并非简单地把说明书上传给 ChatGPT 即可生效，因为专业农机的上下文推理要求极高的准确性。工程团队引入了大量行业专家进行严密校验，先构建出高精度的领域知识图谱（Knowledge Graph），再在其上运行大语言模型（LLM）。 [6:30] Viren Shah: 手势是人类最原始的通用交流形式，无论母语为何，特定指向、召唤和挥手告别的肢体动作在全世界具有高度一致的认知。 [11:03] Viren Shah: 理解并分析各 Agent 生成的遥测日志（即 OTel / OpenTelemetry）正在成为企业技术团队不可或缺的核心技能，其实质是传统应用系统日志在智能体时代的演进。 [11:03] Viren Shah: 借助“远距离操作/免提交互”的 AI 工具与超视距（Beyond Line of Sight）多模态技术，老技师无需亲临恶劣现场即可远程输出专业经验与决策，从而有效延长了稀缺专业人才的职业寿命。 [13:31] Viren Shah: 业界眼下虽然将绝大部分注意力都倾注在生成式 AI（Generative AI）的话题上，但物理 AI 正在切实落地，并且它未来所创造的实际商业与社会价值，极有可能超越当前大家聚焦的生成式 AI。 [14:02] Viren Shah: 当前许多产品在实现聊天功能时陷入了误区，盲目“过度消耗并滥用 LLM（大语言模型）的能力”，而实际上完全可以先借助成熟易用的传统对话机制打底，仅在真正需要复杂语义理解与生成的关键节点切换到 LLM，从而有效控制运营成本。 [14:02] Viren Shah: 当前的软件交互依然“过度依赖文本”（too dependent on text）。在未来的 AI 演进路径中，语音（Voice）与手势（Gesture）必将占据举足轻重的核心地位。

Source quotes

[0:04] Viren Shah: Strategy Acceleration: Innovating and simplifying complex products to increase market share, onboard workers faster, and accelerate strategic roadmaps. [0:04] Viren Shah: Continuous Improvement: Sharpening existing operational baselines, fine-tuning supply chains, and optimizing legacy workflows. [0:04] Viren Shah: Citizen Development: Empowering every employee across an organization to adopt AI by building their own personal agents, echoing the democratization seen in the early days of Microsoft Excel. [2:17] How much horsepower do you have? [2:17] Valtra Coach: The Valtra Q305 tractor has up to 305 horsepower with boosting engaged. [2:17] What kind of service points do we have on the machine? [14:02] Viren Shah: We are too dependent on text... and I think voice and gesture actually brings a lot more value. [14:02] Viren Shah: voice or gestures, either one works

Original transcript

[0:04] So, the topic today is in agriculture, we're going to be about 9 billion people [0:12] in a couple of decades. Um, and what we are finding is that that is not enough labor in [0:20] the farms. Nobody's wanting to become, you know, a tractor driver. So, we'll talk about how we use AI [0:28] to make our products simple, and that will be the story that we'll go through today, but also [0:34] how we are using voice and gestures, because we have kind of forgotten that gesture is another [0:40] form of communication that can be leveraged when voice doesn't work, and hence AI plays a big role. [0:49] So, and I just want to highlight here, what does a frontier firm mean? You know, we are [0:55] all struggling to create value from AI. There are so many sessions that has happened, uh, at least [1:03] yesterday on hey auto I from AI and how do we make auto I from AI? And that is a challenge, and I've, [1:09] you know, I've been fortunate to have a whole, what I call an AI native operating model, uh, that [1:16] I have built, uh, that allows a company to create a value and become the frontier firm. But that's [1:21] another day's conversation. But generally, value creation from AI happens from three things. One [1:30] is, how do you create better products, simplified products, so that you accelerate your strategy, [1:36] increase market share, and different things. The second aspect is continuous improvement, [1:42] which is what most of us are focused on on how do you make the current things better with AI. [1:49] Last but not the least, and most important one, because this relates to adoption, is how are we [1:57] everybody in the organization to actually adopt AI and the one of the best ways to do that, [2:04] if you go back in the Microsoft Excel days, is actually they start using and building agents. So, [2:10] what we're going to talk today is on the strategy acceleration part on new product innovation. [2:17] So, if you have ever been in a farm, right? There is so many tasks that a farmer has to do. Like, [2:27] um just statistically, a farmer gets about 40 harvests in their lifetime, at least in the United [2:35] States. So, imagine you get 40 chances to get your farm operation right in your whole lifetime, [2:45] right? So, the machines that you're using have to be very good, very precise, and they have to [2:52] work all the time. But, most importantly, they should be easy to operate because there's so [2:59] many features in a machine, and we all struggle. If you have a car, like, do we exactly know what [3:06] are all the things that our car can do? Well, we have to read a lot of manuals to understand that. [3:13] And most importantly, we are a we were a global company, so multiple languages around the world, [3:18] right? So, when you make a product, you are now today trying to translate it in many languages, [3:25] uh because you are a global organization, and you need to make sure it can be adopted everywhere. [3:31] So, what we came up with was voice. It's like, how do we make a talking tractor? [3:37] Right? And where voice is the language of choice. Because essentially, what happens is [3:44] imagine a non-English speaking person coming to the United States and trying to operate a tractor. [3:51] Well, I don't know if you're operating a tractor, but you walk in, you really [3:55] don't even know how to turn it on. So, instead of me talking, I will let the tractor talk. [4:02] We are here in the Valtra booth, and behind me we can see a Q series 305. [music] [4:08] And it's the first talking tractor we will launch this year. So, inside the cab we see a lot of [4:13] different [music] options from Unlimited. But, the main project we want to show now is our Valtra [4:18] Coach system. In the side panel of the machine, [music] we have mounted a tablet to use Valtra [4:23] Coach for the tractor. We want to get [music] several parts of data. We want to get information [4:28] about our tractor and how to drive it. So, as an example, [music] we can just ask the machine, [4:33] "How much horsepower do you have?" The Valtra Q305 tractor has up to [4:37] 305 horsepower with boosting engaged. [music] So, quite simple question, but here you can [4:43] see what the system is doing for the driver. [music] Any function we have on the machine, [4:47] any service point we want to use, and of course all the data we have spent the last days, [4:51] the last weeks [music] to get a good statistic combined with Valtra Connect and all the [4:57] telemetrics we have on [music] the machine. What kind of service points do we have on the machine? [5:01] The machine has service points like engine oil and filter, fuel filters, [music] transmission [5:06] pressure filter, final drive oil, differential oil, front hub oil, and lubrication points for the [5:13] So, just one pulse for the system, you can see every information I want [music] to get [5:18] from the tractor is included to the Valtra Coach system. So, for drivers, [5:23] it's getting easier to get all the information they want [music] combined in one system. [5:30] So, essentially, when we had to do this project, now it seems really easier, [5:34] right? Take all of your manuals, load it up, and run chat GDT chat GPT on it. Well, it's not that [5:41] easy because the inferences of the manuals is as critical. So, we actually had to use some experts [5:48] to validate, to create the knowledge graphs. Then only we could run the LLM [5:54] on the knowledge graph that we created. And I think that was a lot of body of work. [6:00] And now you extrapolate that to multiple tractors that we have, multiple languages that we have, [6:07] and we wanted to go live with that. So, this was actually a video. It's about [6:12] 8 months old. We won the award at Technica for the talking tractor. And the goal is [6:18] now we start putting this capability in every machine we have, so that we can start actually [6:24] leveraging the ability to onboard operators pretty fast, helping farmers feed the world. [6:30] So, let's take further a step further. Now we are talking about diverse machines. So, [6:38] I think the panelists that was there just before this last session, they were talking about, "Hey, [6:47] you know, how how do we take a drone? And then there is a tractor, and then there is a, [6:52] you know, I don't know another excavator." There are seven different kinds of machines, different [6:57] manufacturers, right? So, how as a farmer I'm going to be able to orchestrate all of that? [7:05] Secondly, the expense is huge. So, the voice may not even be possible in a very noisy environment. [7:15] So, what do we really do in situations like this? If you take the defense sector, [7:20] right? When you are in the field and you are communicating with multiple machines, [7:25] right? You don't have a choice to use voice. So, how do you actually start communicating [7:32] and creating this problem and solve this problem? And one of the biggest things we have forgotten [7:41] is the old language before you and voice was a thing was called gestures, right? We still know [7:46] what this means even today. Doesn't matter what language we speak in the world. So what we came up [7:53] with and this is where we are working with cacti in the streets is how do we start using gestures [7:59] to orchestrate multiple robotic situations across a a large landscape. So let's look at a video. [8:12] And by the way, this is gestures hence there is no sound. [8:23] So if you look at it, the user pointed the robot to [8:27] come at that specific location and then it came. [8:33] And we all know this gesture. [8:43] So imagine you are in a construction company and you need a tool and you are in a very [8:50] difficult location and safety is always a challenge for you to go get the tool. [9:14] And this is just indicating to both the robots your work is done, you go back home. [9:27] They all know this sign, right? So essentially what we are starting to do is [9:33] how do you start using gestures so that whenever you have a challenge with voice, but how do you do [9:40] multi robot orchestration with gestures, right? So the the You know, the speaker before me said this [9:49] is the beginning and what I'm saying is we are already here. The next frontier is already here. [9:55] On the left on the screen, you see a CGI HoloLens kind of image if you're watching the movie Avatar. [10:03] But on the right, it's actually a 3D model that is kind of created which is a 3D table in real [10:12] and that is a video behind it, but not I didn't have enough time for today to to share or talk [10:17] about it, but the fact that now you can use your 3D plane and you can orchestrate [10:23] on a 3D plane how different machines will operate and orchestrate. And your ability to now be safe, [10:30] but also far away, but have a multi machine machine uh orchestration possible is actually [10:38] real. We are doing this today and this is something we are actually working with the defense [10:45] to say, "Hey, is this something that they can utilize?" Because when we send our soldiers in [10:50] the field, one of the biggest things for them is safety and that is a huge application for a multi [10:56] machine orchestration in the field, especially when you don't have any voice. So, that's kind [11:03] of the presentation for my conversation today is more about as we are looking at more and more AI, [11:11] the vendor agnostic piece is going to be critical. We have talked a lot about, you know, and we hear [11:17] AI and this model, that model, but how are we going to make sure we are able to orchestrate? [11:24] Because that complexity wasn't yesterday, it's already there. We are all struggling with creating [11:30] AI architecture that allows us to do uh you know if you're in the application space how do I [11:37] got salesforce agent force to talk to the customer agents I build that uses three different models [11:44] and by the way you know I've just bought this tool from another small company that does this [11:50] very specific thing you know voice is a big thing that we are talking about. So the vendor agnostic [11:57] architecture is going to be very critical and if you have not uh learned a little bit more there is [12:03] a term called hotel and it's nothing but different logs that each of these agents will generate [12:10] and we will have to ask people in the companies we will have to learn how to work with that [12:16] it's nothing but application logs in the good old days but how do you decipher that and how [12:21] do you start using that is going to be a very important skill in the companies we work for. [12:27] The second aspect of this hands-free. You know I work for appliances for a very long [12:33] time one of the biggest issues if you work in an organization and food and agriculture is [12:38] a a very important part because a lot of people are getting aged and we are not seeing the young [12:45] generation as much as coming to the field of being technicians and being operators and fixing things. [12:52] Hence the ability to prolong the age of the skilled people is going to be very important [12:58] and hence the farther away you are from the action and you're still able to apply your skills [13:05] I think AI is a great tool set if you are in an industry agriculture obviously is one but uh [13:13] that is another place there is so many applications that are available now that [13:18] we don't talk about and think about you know we just talk about the chat stuff and the LLMs a lot. [13:25] Beyond line of sight and you know more stuff the people I'm assuming in the room are probably the [13:31] food and agriculture and there's a lot of things that is beyond the line of sight. And I think [13:35] using physical AI um is a critical aspect of it. We have been talking about generative AI probably [13:43] a lot more. But physical AI is a thing and I think physical AI, I believe, is actually [13:49] going to generate more value than the generative AI that we are focused on. So, if you are in any [13:55] kind of product space, uh integrating and working with physical AI is going to be very critical. [14:02] And last but not the least, you know, what I just shared today is the voice and gesture piece. [14:07] Is not everything has to be voice. And one of the simplest things is even chat functions, [14:15] we are actually overextending the LLM capability versus just using good old chat and then using [14:20] LLM when we need to shift to and just to manage our cost. But voice and gestures actually are one [14:28] of the biggest things that I also believe will be very prominent uh as we go more and more in the AI [14:34] space. So, as you work with organizations, within your organization, please think about that because [14:40] we are too dependent on text. Um and I think voice and gesture actually brings a lot more value. [14:49] So, that's just my presentation. I'm an AI evangelist, happy to extend my network and happy [14:56] to talk, voice or gestures, either one works. Thank you again. Appreciate you all listening.