## Background
Published on YouTube by Ai4 on August 17, 2026, this session features Rcrysta Shaneih, Senior AI/ML Engineer at Ryder Systems Inc., presenting the '360-Degree Split-Brain Spiderweb Architecture.' The presentation outlines how Ryder scales enterprise predictive intelligence by decoupling internal baseline forecasting, macroeconomic influence modeling, and agentic interpretation into modular, production-ready pipelines.
## TL;DR
To manage complex supply chain and logistics operations across North America, Ryder deployed 40 distinct predictive models using a decoupled 'Split-Brain' architecture that separates internal operational drivers from volatile external econometric indicators. Data flows through a multi-hop Medallion architecture in Databricks into aggregated Power BI semantic models and a 'Cognitive Agentic Interpretation Brain' built with Microsoft Copilot Studio. The system automates macroeconomic indicator ingestion with fault-tolerant retries, benchmarks models asynchronously via MLflow to promote champion solutions, and enforces security through AI-driven Databricks Asset Bundle inspection. Shaneih details how operational slicing across tactical and strategic planning horizons dictates whether segments require static annual models or dynamic monthly rolling forecasts.
## Atmosphere & style
The session has the measured, authoritative feel of an advanced enterprise engineering masterclass grounded in hard-won operational experience. Shaneih speaks with calm precision and architectural depth, easily pivoting from high-concept cognitive metaphors to the nitty-gritty of API retry configurations, semantic table consolidation, and MLSecOps gating. The tone is pragmatic, rigorous, and solutions-oriented, giving viewers a transparent, hype-free blueprint for scaling multi-model systems in production.
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=4s) Ryder manages massive North American logistics operations (50k+ employees, 240k+ trucks, 800 maintenance locations), requiring granular multi-model predictive architectures rather than monolithic solutions.
- [02:53](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=173s) The end-state deployment scaled 40 predictive AI solutions combined with a cognitive agentic layer drawing from over 30 external freight, economic, and supply chain sources.
- [04:08](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=248s) The Split-Brain framework decouples internal baseline operational forecasting ('Predictive Brain') from external econometric machine learning ('Influential Brain') to prevent macroeconomic volatility from corrupting internal trend detection.
- [09:05](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=545s) The Cognitive Agentic Interpretation Brain acts as an intelligent reasoning layer on top of model outputs, answering ad-hoc correlation and macroeconomic questions directly in tools like Microsoft Copilot Studio and Teams.
- [11:44](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=704s) A strict abstraction layer is enforced between external APIs and models via Medallion architecture (Bronze, Silver, Gold), ensuring models only train and infer against curated Gold feature tables.
- [11:44](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=704s) Continuous unit testing across each stage of data engineering prevents late-stage schema drifts from breaking parallel downstream model pipelines.
- [13:46](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=826s) To optimize Power BI performance, approximately 100 analytical output tables were consolidated into 4–5 semantic tables, orchestrated dynamically via Power Automate and Databricks SQL Warehouses.
- [19:38](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1178s) Macroeconomic indicator ingestion from FRED requires resilient batch fetching with 2–3 automated retries and non-blocking alerts to handle missing or discontinued indicators.
- [19:38](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1178s) Asynchronous candidate modeling trains multiple model architectures in parallel, using MLflow tracking and automated scoring tables to promote champion models within strict safety thresholds.
- [22:12](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1332s) Databricks Asset Bundles and MLSecOps workflows are gated by an external AI agent that audits deployments against governance, security standards, and credential leak risks before production promotion.
- [25:00](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1500s) Decentralizing into dozens of individual models was mandated by business requirements to slice forecasts across regions, distinct business units, and differing tactical (6–8 month) versus strategic (24 month) planning horizons.
- [26:59](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1619s) Retraining frequency should be dictated by underlying metric volatility: stable, predictable data warrants static annual models, whereas highly cyclical segments require monthly rolling updates with guardrails.
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=4s) Introduction and Ryder's Large-Scale Operations
Rcrysta Shaneih, Senior AI/ML Engineer at Ryder Systems Inc. and a PhD researcher specializing in AI, opens the session by outlining a novel architectural approach designed to scale enterprise predictive pipelines: the Split-Brain Predictive Framework deployed via a "Spiderweb architecture." Drawing on a decade of experience across data science, MLOps, and AI engineering—with over 250 models deployed to production and several award-winning AI projects—Shaneih explains the complex operational realities that demand such a framework.
Ryder operates at a massive logistics scale across North America, employing over 50,000 people and managing 800 maintenance locations, 300 warehouses, and a fleet of more than 240,000 commercial trucks. Annually, the company supports over 340,000 cross-border freight movements spanning Canada, the United States, and Mexico. In an enterprise spanning transportation, warehousing, and dedicated logistics, single monolithic models are ineffective. Decision-makers require forecasts segmented by product lines, operational regions, geographies, and custom classifications to maintain comprehensive operational visibility. This requirement necessitates deploying dozens of simultaneous models in what Shaneih terms "holistic modeling," an approach that introduces unique architectural and deployment challenges.
- Over 50,000 employees, 800 maintenance facilities, and 300 warehouses across North America
- Fleet of 240,000+ commercial trucks managed across the enterprise
- Facilitates over 340,000 cross-border freight movements across the US, Canada, and Mexico each year
- Speaker background includes 10 years in data/AI, a PhD research background in AI, and 250+ production model deployments
---
[02:53](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=173s) End-State Vision: 40 Models and Multi-Modal Market Intelligence
Over an eight-month build, the engineering team scaled 40 distinct predictive AI solutions combining internal operational drivers with macroeconomic indicators. These model outputs feed directly into a centralized demand planning platform, which is augmented by an agentic interpretation layer. Rather than leaving business stakeholders to manually parse complex statistical forecasts, AI agents synthesize output data and contextualize the findings against broader economic realities.
The agentic system incorporates multi-modal industry data, drawing from more than 30 external sources, including market analyses, freight reports, economic trends, and supply chain updates. By synthesizing predictive outputs alongside real-time market reporting, the system functions as a comprehensive market intelligence and decision-support engine capable of validating internal forecasts against broader industry movements.
- 40 predictive AI solutions deployed over an 8-month timeline
- System integrates internal operational data with macroeconomic indicators into a demand planning tool
- AI agent layer connects to 30+ external freight, economic, and supply chain intelligence sources to interpret forecast trends
---
[04:08](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=248s) The Split-Brain Predictive Framework Inspired by Neuroscience
Tracing the project's evolution back to its original September 2025 requirements, the initial scope called for 18 regional forecasting models capable of delivering tactical forecasts (6–8 months) and strategic outlooks (18–24 months) sensitive to economic factors like inflation, diesel fuel pricing shifts, and industrial manufacturing output. While end users interact with what appears to be simple dropdown filters inside a demand planning UI, "in reality, every filter you're selecting is really a different predictive AI solution on the back end that's showcasing the results."
The architecture draws inspiration from the neurosurgical split-brain procedure, where severing the corpus callosum prevents epileptic seizures from spreading across cerebral hemispheres while preserving cognitive function in each half. Similarly, Ryder decouples its forecasting into two isolated systems: the "Predictive Brain" (trained strictly on internal historical inventory, seasonality, and operational data) and the "Influential Brain" (trained on econometric machine learning and external indicators). When the internal models proved exceptionally accurate at sensing localized inventory and product shifts upon their December release, the team opted not to replace them with the macroeconomic models in the spring, but to run both concurrently. Rather than combining the two into a single score or blended model, they are preserved as separate, independent pulses on the business.
- Initial request in September 2025 covered 18 models; expanded to 40 total production solutions
- Internal driver models ("Predictive Brain") deployed in December; external indicator models ("Influential Brain") deployed in spring
- Models support tactical horizons of 6–8 months up to strategic horizons of 18–24 months
- Decoupled design ensures econometric volatility does not corrupt internal baseline trend detection
---
[09:05](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=545s) The Agentic Brain and the Spiderweb Architecture
To address stakeholder questions regarding how forecasts relate to broader market conditions, the team implemented the "Cognitive Agentic Interpretation Brain." Traditional machine learning workflows follow a linear sequence of ingestion, preprocessing, modeling, and deployment. However, scaling dozens of models dependent on varying subsets of economic indicators produces an interconnected "spiderweb architecture" of parallel engineering pipelines.
Ryder unified these workflows using a consolidated enterprise stack of approximately 20 tools, building the modeling pipelines in Databricks and the conversational agents in Microsoft Copilot Studio. Staying within an integrated vendor ecosystem eliminated interoperability friction across systems. The agentic layer queries both the predictive model outputs and raw economic indicator streams, enabling it to answer nuanced correlation questions—such as the direct impact of inflation on diesel fuel costs—even when specific relationships are not explicitly visualized on the primary dashboard.
- Data engineering layer incorporates six core processes feeding specialized model pipelines
- Over 20 enterprise tools coordinated across the end-to-end pipeline
- Databricks manages data engineering and model execution; Copilot Studio powers agentic reasoning
---
[11:44](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=704s) Data Engineering Pipeline, Medallion Architecture, and Abstraction
The back-end data engineering architecture implements a standard Medallion (multi-hop) pattern: raw third-party source data enters the Bronze layer, undergoes validation and cleansing in the Silver layer, and is structured into curated feature tables in the Gold layer. A critical architectural rule is enforcing an abstraction layer between external APIs and the models. Machine learning models should never query third-party APIs directly during inference or training due to differing source refresh cycles, API rate constraints, and potential schema shifts.
Shaneih highlights the importance of continuous unit testing throughout the data lifecycle rather than validating exclusively at the pipeline's end. In holistic deployments with dozens of downstream models, a late-stage schema failure forces engineers to trace data lineage across the entire graph to locate the break and propagate fixes. Continuous validation catches inconsistencies immediately at the ingestion and intermediate staging layers.
- Medallion architecture (Bronze raw ingestion, Silver cleansing/quality checks, Gold feature tables)
- Strict abstraction layer ensures models train and infer solely from Gold feature tables, never live APIs
- Continuous unit tests prevent schema drift across 20+ downstream model pipelines
---
[13:46](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=826s) Integration and Serving via Power BI, Power Automate, and MCP
Although original stakeholders requested a bespoke web application for forecast consumption, the engineering team determined that Power BI could satisfy all user interface requirements, reserving custom application development for the AI agents. With 40 models generating extensive analytical outputs—including backtesting results, correlation matrices, cosine similarity scores, and causal analyses—ingesting output data directly would require over 100 individual database tables. To maintain performance, the team aggregated the data into four to five consolidated tables within the Power BI semantic layer.
The semantic layer connects directly to Databricks via a Databricks SQL Warehouse. To overcome standard Power BI scheduling limitations, Power Automate was layered on top to orchestrate and trigger dynamic refreshes. For agentic access, Copilot Studio connects to the Databricks SQL Warehouse using Service Principal IAM credentials or Personal Access Tokens (PAT), or interfaces via Model Context Protocol (MCP) servers to expose model tables directly as tool-accessible knowledge bases across Microsoft Teams and dedicated web portals.
- Consolidated ~100 analytical output tables (backtesting, cosine similarity, causal analysis) into 4–5 Power BI semantic tables
- Databricks SQL Warehouse provides native data retrieval to the Power BI semantic layer
- Power Automate manages customized refresh execution schedules
- Copilot agents connect to SQL Warehouses and MCP servers to serve responses in Teams and custom web interfaces
---
[16:34](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=994s) Engineering Best Practices: Reusability, Temporal Modeling, and Economic Indicators
Scaling numerous predictive models within tight delivery timelines relies on pipeline reusability. Because internal data sets share common regional and product segment hierarchies, data cleansing and feature engineering transformations can be standardized into reusable code templates. Feature engineering for temporal operational data incorporates rolling windows, lag features, seasonality modeling, and mathematical smoothing to account for historical disruptions such as the COVID-19 pandemic.
Integrating macroeconomic data requires substantial domain research rather than indiscriminate API ingestion. Shaneih notes that inflation can be calculated in nearly 12 distinct ways, while unemployment has six primary definitions alongside numerous regional variations. When ingesting series from public repositories like FRED (Federal Reserve Economic Data), engineers map specific Series IDs within configuration dictionaries to automate programmatic API retrieval. Parallel model evaluation and experiment tracking are orchestrated throughout both release cycles using MLflow to manage model registry and performance comparisons across internal and econometric feature sets.
- Reusable data engineering templates accelerate model development across shared regional/product schemas
- Time-series feature engineering incorporates lags, rolling windows, seasonal adjustments, and smoothing for COVID-era anomalies
- Economic indicators mapped via FRED Series IDs stored in dictionaries for automated API ingestion loops
- MLflow tracks parallel experiments and manages artifact lifecycle across iterative December and spring releases
[19:38](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1178s) Indicator Ingestion, API Batch Handling, and Asynchronous Candidate Modeling
When building predictive pipelines with external data, data freshness and historical depth are critical criteria. It is common to identify indicators that initially appear ideal for modeling, only to discover that they are not updated or refreshed regularly. When ingesting hundreds of indicators from external sources like the Federal Reserve Economic Data (FRED) API, ingestion should be handled in batches with two to three automatic retries. The API ingestion logic should be resilient: if an indicator that is usually available fails or goes missing, the system should log an alert and record the specific missing indicator without failing the entire batch run. Once pulled, indicators are amalgamated, up-sampled or down-sampled to match internal datasets, and fed into the modeling layer.
```mermaid
flowchart TD
accTitle: Indicator Ingestion and Automated Candidate Modeling Pipeline
subgraph Ingestion["Ingestion & Data Prep"]
A["FRED API / External Sources"] --> B["Batch Fetch with Retries"]
B --> C{"Check Indicator"}
C -->|Failed| D["Send Alert & Continue Batch"]
C -->|Success| E["Amalgamate & Resample"]
end
subgraph CandidateModeling["Candidate Modeling & Champion Selection"]
E --> F["Asynchronous Candidate Training"]
F --> G["Track Runs in MLflow"]
F --> H["Rank & Score Output Table"]
H --> I{"Thresholds Met?"}
I -->|Yes| J["Deploy Champion to Production"]
I -->|No| K["Trigger Alert for Manual Assessment"]
end
```
To evaluate and train models efficiently, a strategy called "asynchronous candidate modeling" is implemented to evaluate multiple candidate model architectures simultaneously. Reusable workflow patterns automate model training while MLflow runs under the hood to track every experiment run and parameter set, enabling quick rollbacks to prior iterations if necessary. Candidate outputs are collected in a structured scoring table that ranks models according to defined business and performance logic. In an automated retraining pipeline, the top-performing candidate model is automatically promoted to production as the new champion; if performance falls below predefined threshold guardrails, the deployment halts, and an alert is dispatched for manual intervention and troubleshooting.
- Configure external indicator ingestions with 2–3 automated retries per batch to prevent intermittent network or API failures from stopping the pipeline.
- Resilient batch logic allows successful batch processing even when individual indicators drop out, raising non-blocking notifications instead.
- MLflow experiment tracking provides backend visibility and repeatable rollbacks across all asynchronous model evaluation runs.
- Scoring tables rank candidate models dynamically to automate champion promotion or escalate threshold failures to engineering teams.
---
[22:12](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1332s) MLOps Capabilities, Bundle Deployment, and MLSecOps Integration
Production AI deployments depend on comprehensive MLOps foundations, spanning upwards of 21 core capabilities including feature stores, experiment tracking, pipeline orchestration, and end-to-end automation. Orchestration remains one of the largest hurdles for data scientists, especially when transitioning between building end-to-end pipelines and inheriting legacy models strictly for deployment. At Ryder Systems Inc., Databricks serves as the primary platform for managing workflow execution and deployment patterns.
```mermaid
flowchart LR
accTitle: Secure Bundle Deployment Architecture
subgraph Dev["Packaging"]
A["Databricks Code & Assets"] --> B["Asset Bundling"]
B --> C["Embedded Unit Tests"]
end
subgraph Governance["Governance & Security"]
C --> D["External AI Agent Monitor"]
D --> E{"Validate Against Best-Practice Knowledge Base"}
end
subgraph Deploy["Deployment"]
E -->|Approved| F["Deploy Across Environments"]
E -->|Flagged| G["Block & Remediate Sensitive Data/Code"]
end
```
A core differentiator in this deployment architecture is the addition of an external AI agent that inspects Databricks Asset Bundles before promotion across staging and production environments. Standard unit tests run throughout the CI/CD pipeline, but deployments frequently risk leaking sensitive credentials or proprietary code. The external AI agent references a curated organizational knowledge base of ideal deployment patterns, auditing every bundle for security posture and compliance. Furthermore, MLSecOps is embedded directly into the deployment lifecycle, reflecting the evolving role of AI engineers into infrastructure and security architects responsible for safeguarding enterprise generative AI applications.
- MLOps incorporates up to 21 distinct capabilities, centering around Databricks for workflow orchestration, feature storage, and automated deployment.
- Specialized AI agent acts as an automated release gatekeeper, scanning deployment bundles against governance and security standards.
- MLSecOps enforces strict data privacy, credential protection, and governance policies across enterprise-scale AI pipelines.
---
[25:00](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1500s) Business Slicing vs. Single Models: Managing Forecast Granularity
During the audience Q&A, a question is raised regarding why so many individual models are trained rather than relying on a single, global decision-tree-like model that segments conditions internally. From an engineering and simplicity standpoint, minimizing total models is always preferred, but the multi-model architecture was a direct operational requirement from the business. An aggregated model combining all regions and product categories failed to provide granular operational visibility across Ryder's distinct business units—specifically supply chain, warehousing, and dedicated transportation.
subgraph Application["End-User Consumption"]
A & B & C & D --> E["Segmented Forecasting Models"]
E --> F["Interactive Business Intelligence Dashboard\n(Custom Filter Views & Horizon Selection)"]
end
```
Model volume expands further when forecasting across varying planning horizons:
- Tactical forecasts spanning 6 to 8 months provide high near-term accuracy for immediate operational decisions.
- Strategic forecasts spanning 24 months project macro trends further out, requiring distinct assumptions and feature sets.
- Slicing data across separate horizons, geographical regions, and product verticals multiplies the total number of models, powering an interactive BI application where business stakeholders can filter down to specific, tailored predictive views.
---
[26:59](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1619s) Volatility-Driven Retraining: Rolling vs. Static Forecast Strategies
A second audience question addresses champion model volatility—specifically, the challenge of automated selection flipping rapidly between model types each month, introducing wide variations across forecast iterations. In production systems, frequent retraining is not universally required; it should be determined directly by business volatility and data drift.
When underlying business metrics are stable, flat, or follow standard predictable seasonality without structural drift, static forecasting models are preferable. These can be trained once and safely relied upon for 12 months, or retrained on a standard annual schedule. In contrast, highly volatile business lines subject to macroeconomic shifts and cyclical disruptions necessitate monthly rolling forecasts. Analyzing historical trend patterns reveals whether a static or rolling framework is warranted: flat, standard, predictable histories suit static models, while volatile, cyclical series require dynamic rolling pipelines with built-in guardrails to prevent erratic model switching.
- Static forecasts retrained annually are optimal for low-volatility, non-drifting operational metrics.
- Monthly rolling retraining is reserved for business units exposed to severe market and economic cyclicality.
- Historical volatility and drift analysis dictate whether a business segment requires continuous model re-evaluation or fixed horizon planning.
## Notable quotes
> [04:08](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=248s) in reality, every filter you're selecting is really a different predictive AI solution on the back end that's showcasing the results. — Rcrysta Shaneih
## Mentioned
Rcrysta Shaneih, Ryder Systems Inc., Ai4, Databricks, Microsoft Copilot Studio, Power BI, Power Automate, MLflow, Federal Reserve Economic Data (FRED), Microsoft Teams, Model Context Protocol (MCP)
## Video description
Rcrysta Shaneih - Sr. AI/ML Engineer, Ryder Systems Inc.
Inspired by neuroscience, the 360-Degree Split-Brain Spiderweb Architecture introduces a novel way to scale holistic predictive intelligence pipelines by decoupling forecasting, influence modeling, and interpretation into specialized but connected components. At Ryder Systems Inc., this architecture was implemented to scale multi-model pipelines across platforms, leveraging modular deployment, automation, and orchestration to support production forecasting and market intelligence use cases.
Session attendees will learn how to build enterprise-ready AI architectures that mimic cognitive specialization: deploying stable forecasting models as the “Predictive Brain,” pairing them with influence and explainability models as the “Influential Brain,” and extending interpretation through a “Cognitive Agentic Interpretation Brain” that helps translate analytical outputs into decision-ready intelligence.
This session will equip AI/ML engineers, data scientists, and AI leaders with a practical blueprint for building scalable, interpretable, and production-ready predictive intelligence systems that move beyond single-model forecasting toward holistic AI architectures.
Rcrysta Shaneih, Senior AI/ML Engineer at Ryder Systems Inc., details how the logistics enterprise scaled 40 predictive AI solutions using a decoupled 'Split-Brain' architecture and an agentic interpretation layer. The framework isolates internal operational baselines from external macroeconomic volatility to meet granular operational demands across fleets, warehouses, and cross-border corridors.
Operational Granularity Over Monolithic Forecasting (25:00)
Ryder's logistics network encompasses over 50,000 employees, 240,000 commercial trucks, 800 maintenance facilities, and 300 warehouses managing more than 340,000 annual cross-border freight movements across North America. Shaneih states that while deploying a single global model is simpler from an engineering perspective, it fails to provide the granular visibility required by individual business units. Combining operational lines—specifically dedicated transportation, warehousing, and supply chain logistics—masks critical localized trends.
To deliver actionable operational visibility, Ryder decomposed forecasts across business units, geographies, and distinct planning horizons. The system provides tactical forecasts spanning 6 to 8 months for immediate operational precision, alongside strategic forecasts extending 18 to 24 months for macroeconomic planning. Slicing data across these dimensions caused the engineering team to expand its initial September 2025 scope of 18 regional models into 40 distinct production solutions.
Isolating Baselines via Split-Brain Modeling (4:08)
Inspired by neurosurgical split-brain procedures that isolate cerebral hemispheres to prevent disruptions from spreading, Shaneih decoupled forecasting into two isolated computational tracks: the 'Predictive Brain' and the 'Influential Brain.' The Predictive Brain trains strictly on internal historical data, including inventory levels, seasonality, and operational cycles. The Influential Brain handles external econometric machine learning, modeling variables such as diesel pricing, industrial output, and inflation metrics sourced from public repositories like FRED.
Shaneih notes that when internal models proved highly accurate at sensing localized shifts upon their December release, the engineering team opted not to blend them with the macroeconomic models deployed the following spring. Combining internal and external features into a single composite score risks allowing econometric volatility to distort stable internal baseline signals. Running both model families concurrently allows business stakeholders to view independent operational and macroeconomic perspectives side by side in a demand planning interface.
Agentic Synthesis and Semantic Data Serving (9:05)
Managing parallel modeling pipelines across disparate economic inputs generated an interconnected 'spiderweb architecture' spanning approximately 20 enterprise tools across Databricks and Microsoft stacks. To prevent business stakeholders from having to manually parse statistical outputs, Ryder introduced a 'Cognitive Agentic Interpretation Brain' built in Microsoft Copilot Studio. The agentic system links model outputs to more than 30 external market, freight, and supply chain reports, allowing it to answer nuanced cross-correlation queries—such as the direct impact of inflation trends on diesel costs—without requiring bespoke visualization changes.
To maintain interface performance, the engineering team aggregated analytical outputs from approximately 100 back-end tables—containing backtesting, causal analysis, and cosine similarity metrics—into four to five consolidated tables within a Power BI semantic layer. Power Automate triggers dynamic refreshes directly against a Databricks SQL Warehouse. Copilot agents access the warehouse data via Service Principal credentials, personal access tokens, or Model Context Protocol (MCP) servers to deliver conversational insights across Microsoft Teams and internal web portals.
Resilient Ingestion, Candidate Promotion, and MLSecOps (19:38)
To handle third-party data reliably, Ryder implements a standard Medallion architecture (Bronze, Silver, and Gold), enforcing a strict abstraction layer where models query only curated Gold feature tables rather than live APIs. When pulling external data from sources like the FRED API, pipelines execute batch requests with two to three automated retries. If an individual indicator drops out, non-blocking alerts log the failure while permitting the broader batch run to complete.
Candidate model architectures are trained simultaneously using asynchronous candidate modeling, with experiment parameters tracked in MLflow. Dynamic scoring tables rank candidate models against business performance thresholds, automatically promoting top candidates to production as champions or halting deployment for manual intervention if thresholds are missed. Deployment bundles are further governed by MLSecOps: an external AI agent audits Databricks Asset Bundles against organizational security benchmarks to prevent credential leaks and code vulnerabilities prior to production promotion.
Shaneih emphasizes that retraining cadence must be dictated by empirical metric volatility rather than arbitrary schedules. Predictable, low-drift operational metrics rely on static annual models, whereas cyclical business units exposed to macroeconomic shifts require monthly rolling retraining with threshold guardrails to prevent erratic champion switching.
精炼阅读摘要
Ryder Systems 高级 AI/ML 工程师 Rcrysta Shaneih 提出，面对超大规模物流网络的复杂性，单一庞大模型无法满足精细化决策需求，企业必须采用解耦内部运营基线与外部宏观经济指标的“双脑蛛网架构”（Split-Brain Spiderweb Architecture），并借助多模态智能体解释层与严密的工程管道支撑全方位预测智能。
双脑预测架构：解耦运营基线与宏观波动 (4:08)
Ryder 在北美管理超过5万名员工、800个维护网点、300座仓库以及逾24万辆商业卡车，每年协调超过34万次跨境货运。针对供应链、仓储和专用运输等多重业务，工程团队在8个月内将最初规划的18个区域模型扩展为40个生产级预测方案。为了防止宏观经济剧烈波动干扰内部底层业务趋势，架构借鉴了神经外科的脑裂手术理念，构建两个相互独立的核心系统：“预测脑”（Predictive Brain）纯粹基于内部历史库存、季节性规律与运营数据训练，擅长捕捉本地化微观波动；“影响脑”（Influential Brain）则依托计量经济学机器学习算法，接入柴油价格、通胀率和工业产出等外部宏观指标。两套系统保持并行独立运作，而非融合成单一加权分值，确保业务人员能够同时获取客观的内部运行基线与宏观经济冲击对冲视角。
蛛网管道拓扑与认知智能体解释层 (9:05)
当数十个预测模型各自依赖不同的经济指标子集时，传统的线性流水线会演变为高度交织的“蛛网架构”。Ryder 整合了约20种企业级工具栈，统一由 Databricks 驱动底层数据管道与模型执行，并在消费端引入由 Microsoft Copilot Studio 构建的“认知智能体解释脑”。业务决策者通常难以直接解读复杂的统计预测结果，该智能体层接入了超过30个涵盖货运报告、宏观趋势与供应链动态的外部数据源，不仅能够自动归纳预测产出，还能回答未在基础仪表盘中直接可视化的细粒度归因问题（例如特定通胀指标对柴油成本的传导效应）。智能体通过服务主体身份凭证或模型上下文协议（MCP）安全连接 Databricks SQL 仓库，在 Microsoft Teams 及定制化网页应用中提供交互式决策支持。
数据工程抽象隔离与弹性批量摄取 (11:44)
底层数据架构采用标准的奖牌架构（Medallion Architecture），原始外部数据流入铜层（Bronze），经校验清洗后进入银层（Silver），最终在金层（Gold）固化为精选特征表。Shaneih 强调了一条关键架构铁律：机器学习模型在训练与推理过程中绝不允许直接调用外部第三方 API，必须严格从金层特征表读取，以规避外部数据源刷新周期不一、速率受限及数据架构漂移（Schema Drift）带来的系统性风险。全流程实施持续单元测试以替代仅在末端验证的做法，避免几十个下游模型因上游微小结构变动而陷入排查困境。在从 FRED 等公共库摄取上百个经济指标时，系统采用2至3次自动重试的批处理逻辑；当个别指标缺失或调用失败时，管道仅触发非阻塞式预警并跳过记录，防止偶发异常中断整体数据流。
异步候选建模与 MLSecOps 安全网关 (19:38)
为了在严格的时限内高效交付，团队构建了“异步候选建模”机制，能够同时评估多种候选模型架构。整个训练流程与实验参数均由 MLflow 进行追踪和版本控制，候选输出汇入统一的打分表以执行自动化排序；当最优模型达到预设指标时会自动晋升为生产冠军模型，一旦指标低于安全阈值则立即阻断部署并转交人工排查。在贯穿特征存储、管道编排等多达21项 MLOps 核心能力的基础上，架构特别引入了 MLSecOps 治理机制。系统部署了一个专用的外部 AI 治理智能体，在 Databricks 资产包（Asset Bundles）推向准生产与生产环境之前充当安全门禁，严格依据企业知识库规范审查代码合规性，有效阻断敏感凭据泄露与专有代码缺陷。
业务粒度切分与波动驱动的重训策略 (25:00)
针对为何不采用单一全局模型（如全局决策树）的疑问，Shaneih 指出高度切分是业务视角的刚性诉求：高度聚合的模型会抹杀专用运输、供应链管理和仓储之间的本质差异，无法提供实操层面的可见性。此外，规划周期的长短决定了建模假设的完全不同——涵盖6至8个月的战术预测侧重近期的精准操作响应，而18至24个月的战略预测则依赖更长周期的宏观推演。为了在前端交互中承载这套多维模型矩阵，团队在 Power BI 语义层中将约100张原始分析表聚合为4至5张核心表，并通过 Power Automate 实现动态编排刷新。针对模型重新训练的频率，团队认为应完全取决于业务波动性与数据漂移程度：趋势平稳、周期性明确的业务线适用静态预测，每年仅需重训一次；而高度受制于宏观周期震荡的业务线才需要每月滚动重训，并配置防范剧烈振荡的防抖护栏。
Shaneih 总结表明，预测智能在大型物流企业的成功落地依赖于架构层面的解耦与模块化复用，证明了通过解耦内外部驱动因素、引入智能体综合解释以及严格落实数据工程抽象，能够可靠地维系数十个专用模型的并发运转；但这也对跨异构环境下的特征治理、API 弹性容错与持续代码安全审计提出了更高的基础设施要求。
Detailed notes — zh
## Background
本视频由 Ai4 频道于 2026 年 8 月 17 日发布在 YouTube 平台上，主讲人为 Ryder Systems Inc. 高级人工智能工程师 Rcrysta Shaneih。视频简介指出，本场演讲介绍了受神经科学启发的“360度裂脑蜘蛛网架构”，探讨如何通过解耦预测脑、影响脑与智能体解读脑，构建企业级可规模化、高可解释性的端到端预测智能流水线。
## TL;DR
演讲围绕北美物流巨头 Ryder 的实际业务落地展开，详细阐述了团队如何在 8 个月内将预测模型群从 18 个规模化扩展至 40 个并投产上线。Rcrysta 深度拆解了让内部业务趋势（预测脑）与外部宏观经济指标（影响脑）并行运行的“裂脑架构”，以及上层接入 30 多个外部研报数据源的“智能体解读脑”。此外，她还系统分享了基于 Databricks、奖章架构与 FRED API 容错管道的蜘蛛网工程实践，并针对海量模型切分逻辑与滚动预测防抖机制给出了明确的实战选型指南。
## Atmosphere & style
整场演讲呈现出严谨扎实、高度凝练的工业级技术工程风格，讲者将精妙的医学隐喻与硬核的 MLOps 架构设计举重若轻地结合在一起。叙事节奏干脆利落、条理清晰，不仅有自顶向下的宏观系统设计蓝图，更充满了一线工程踩坑与避坑的经验细节；现场问答互动直击算法边界与系统抖动的深水区痛点，传递出极高信息密度的实战交流氛围。
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=4s) Ryder 面对横跨运输、仓储与供应链的复杂实体网络，单一通用模型无法满足决策需求，必须按业务线与地理维度精细化切分开展整体性建模（Holistic Modeling）。
- [02:53](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=173s) 系统在 8 个月内将 40 个预测性 AI 解决方案推向生产，并接入 30 多个权威外部行业数据源，构建起多模态市场决策支持体系。
- [04:08](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=248s) 前端 UI 表现为简洁的多维下拉筛选器，而底层每个滤镜标签实际上都直连着一个独立运行的专用预测 AI 解决方案。
- [05:56](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=356s) 受神经科学裂脑手术启发，系统解耦为基于内生数据的“预测脑”与融合宏观指标的“影响脑”，二者独立运行以提供双重感知视角而不做强行加权合并。
- [08:27](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=507s) 在预测层之上部署“认知智能体解读脑（Agentic Brain）”，能够理解因果关联与外部研报，突破传统 BI 固定图表的限制，提供动态交互问答。
- [10:05](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=605s) 多源异构数据与模型的复杂依赖构成“蜘蛛网架构”，选型上坚持收敛至单一厂商生态（如核心计算在 Databricks、智能体在 Copilot Studio）以降低互操作摩擦。
- [11:44](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=704s) 采用奖章架构（Bronze/Silver/Gold）并在特征与模型间建立特征表抽象层，严禁模型直接调用外部 API，并在数据生命周期各阶段前置嵌入单元测试。
- [13:46](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=826s) 将模型生成的上百张输出表深度整合为 4-5 张核心宽表注入 Power BI，并通过 Power Automate 实现事件驱动的自适应语义层刷新，同时支持 MCP 协议供智能体调用。
- [16:34](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=994s) 利用底层业务数据的同源性沉淀可复用工程模板加速多模型开发，并对 COVID-19 等黑天鹅异常周期进行专门的特征平滑处理。
- [18:36](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1116s) 接入 FRED 等宏观经济指标前需进行严谨的口径与机理研究，管道设计上需抽象 Series ID 字典并配备批次加载与 2-3 次重试的容错机制。
- [20:33](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1233s) 基于可复用工作流模板开展异步候选模型评估，利用 MLflow 追踪实验全生命周期，并在再训练管道中全自动评选和更新“冠军模型”。
- [23:05](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1385s) 在 Databricks Asset Bundles 部署流程中引入专属“AI 审查 Agent”，在跨环境推进前核查配置规范与安全隐患，践行 ML SecOps 理念。
- [25:00](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1500s) 海量模型切分源于业务实体特性与 6-8 个月战术预测 vs 24 个月战略规划的视距差异，“时间视距 × 业务区域 × 产品线”的笛卡尔积自然导致模型数量膨胀。
- [26:59](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1619s) 滚动预测与静态预测的选择取决于数据波动率：低波动平稳场景坚决使用静态预测以防抖动，而强周期剧烈波动业务才需承担月度滚动再训练的成本。
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=4s) 讲者背景与 Ryder 的超大规模物流场景
会议在下午四点半准时开场，讲者 Rcrysta Shaneih 向坚持到当天下半场的所有听众致谢。作为 Ryder Systems Inc. 的高级人工智能工程师兼专注于 AI 领域的人工智能博士研究员，Rcrysta 在过去十年中深耕数据与 AI 领域，职业轨迹跨越了数据科学家、MLOps 工程师再到当下的 AI 工程师角色。在 Ryder 内部，她不仅负责构建跨部门的核心业务用例，更主导着多支团队将解决方案规模化推向生产环境。在其职业生涯中，她已经将 250 多个模型成功投入生产，并主导或协同参与了多个屡获殊荣的人工智能项目。她本次分享的核心主题，正是如何借助神经科学启发的“裂脑预测框架”（Split-Brain Predictive Framework）与“蜘蛛网架构”（Spiderweb Architecture）来规模化扩展端到端的预测智能流水线。
随后，Rcrysta 详细介绍了 Ryder Systems 的业务底色。作为北美领先的运输和物流服务商，Ryder 拥有庞大而厚重的实体资产与运营网络：全公司拥有超过 50,000 名员工、800 个维护保养网点和 300 处大型仓库；其直接管理的商用卡车车队规模超过 240,000 辆，每年承担着超过 340,000 次穿梭于美国、加拿大和墨西哥之间的跨境货运移动。面对横跨运输、仓储和供应链物流的复杂业务矩阵，Rcrysta 指出，任何单一的通用 AI 模型都无法真正满足业务决策的需求。为了获得全链路的全局业务可视性，模型必须按照产品线、业务区域、地理区划以及特定分类标准进行精细化切分，这就导致生产环境必须同时运行大量预测系统。这种跨维度、多视角的规模化模型群部署，正是她所定义的“整体性建模”（Holistic Modeling），而它也对传统的系统架构提出了极其独特的挑战。
- **全职规模与资产**：北美员工超 50,000 人，拥有 800 个维护点、300 个仓库以及逾 240,000 辆商用卡车车队。
- **跨境运营**：每年支持美、加、墨三国之间超过 340,000 次跨境货运运输。
- **工程师经验**：个人拥有 10 年数据与 AI 经验，累计向生产环境交付超 250 个模型。
---
### [02:53](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=173s) 全景预测智能：从40个模型到多模态决策支持
在展开技术细节前，Rcrysta 首先亮出了这套系统在过去 8 个月内实现的最终落地形态。团队不仅完成了从基础数据到顶层应用的建设，更将原本仅处于规划阶段的多项能力直接规模化扩展至 40 个正式投产的预测性 AI 解决方案。这些方案深度结合了 Ryder 内部的业务驱动数据与宏观经济指标，所有预测结果被统一汇入一个核心的需求规划工具（Demand Planning Tool）中。更为关键的是，团队在这些预测输出之上构建了一套 AI 智能体（AI Agents）网络，使机器不仅能够给出冷冰冰的数字走势，更能对海量预测数据展开深度归因与商业解读。
为了让 AI 智能体具备与真实市场对齐的能力，系统打通了大量外部非结构化与多模态数据源。在北美货运与供应链体系中，行业充斥着海量的市场研报、宏观经济动向与运力分析文件。系统最终接入了超过 30 个权威行业数据源，赋予智能体阅读并综合这些信息的能力。由此，AI 智能体不仅可以解释模型预测结果产生的底层逻辑，还能横向对比这一预测与外部大宗市场判断是否吻合，最终形成了一套多模态的市场情报与决策支持系统。
- **模型规模**：8 个月内将 40 个预测性 AI 解决方案全部推进至生产状态。
- **市场对齐**：AI 智能体接入超过 30 个外部行业数据源，全面覆盖货运、运输和供应链趋势。
- **业务价值**：打通宏观预测与微观运营，形成可直接用于商业决策支持的多模态市场情报系统。
---
### [04:08](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=248s) 需求回溯与“滤镜即模型”的系统雏形
为了理清整套架构的设计脉络，Rcrysta 带领听众进行了一场“时间旅行”，将时钟拨回至项目立项的 2025 年 9 月。当时的业务需求远没有如今 40 个模型这么宏大，业务部门最初提出的需求仅包含 18 个模型。然而，从项目启动的第一天起，数据接入的巨大挑战就已显露无遗——企业内部并没有一套成熟健壮的流水线能够持续拉取各色外部宏观经济指标。为了实现业务诉求，团队必须首先打通这些外部数据断点，再根据不同地理区域构建分层模型。这些模型被赋予了双重使命：既要提供 6 至 8 个月的战术性短期预测，又要支持 18 至 24 个月的战略性长期规划。
除了预测时间跨度，模型更核心的目标在于厘清外部宏观变量对 Ryder 各个业务产品细分板块的具体冲击机制。在供应链物流业务中，通货膨胀波动、柴油价格升跌以及工业制造指数的起伏，都会直接或间接地传导至运力需求和库存流转之中。为了让业务人员在前端易于上手，系统在前端展现上做出了极具欺骗性的简化：“当业务用户打开需求规划应用程序时，从 UI 视角看，他们可能觉得自己只是在下拉菜单中简单切换几个滤镜；但实际上，他们点击的每一个筛选标签，在后台对应的都是一个独立运行的预测性 AI 解决方案。”
- **立项基准（2025年9月）**：初始需求仅为 18 个预测模型，终态扩展至 40 个。
- **预测跨度覆盖**：6-8 个月的战术级预测与 18-24 个月的战略级远期规划。
- **前端抽象设计**：前端 UI 表现为极其清爽的多维下拉滤镜，底层则直连几十个专用的 AI 预测服务。
---
### [05:56](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=356s) 神经科学启发：裂脑预测框架（Split-Brain Framework）
随着系统研发的深入，一个脱胎于 Rcrysta 个人博士研究课题的全新架构雏形开始浮现——这就是“裂脑预测框架”。在项目推进过程中，业务原本要求直接构建能够融入宏观经济指标的“影响脑”（Influential Brain）。但由于外部经济数据工程链路极长，为了尽早为业务交付价值，团队在 12 月份率先基于 Ryder 内部历史业务数据（包括历史库存、运营数据、季节性周期和内在趋势）将 18 个内部模型推向生产，构建起了“预测脑”（Predictive Brain）。原定计划是等到来年春季宏观经济数据流水线跑通后，直接用“影响脑”将第一阶段的“预测脑”完全替换掉。
然而实际业务表现却打破了研发团队的预设计划。第一版“预测脑”在投产后展现出了极高的准确度，它对内部库存变动、业务细分板块的内生周期有着极强的感知灵敏度。到了春季，当融合了外部指标的“影响脑”也成功交付并能够清晰揭示外部变量对业务的因果驱动时，团队并未如期“移花接木”，而是做出了一个关键决策：让两大模型体系同时并行在线。至此，生产环境中形成了多达 40 个针对不同业务切面的模型矩阵。Rcrysta 用医学神经科学中的“裂脑手术”（Corpus Callosotomy）来比喻这一架构：医学上为了治疗难治性严重癫痫，外科医生会切断连接大脑左右半球的胼胝体；这样一来，即便一侧大脑半球发生异常放电癫痫发作，健康的一侧仍能维持患者正常的认知和生命功能。
```mermaid
flowchart TD
accTitle: 裂脑架构与业务决策支持流
accContext: 展示预测脑与影响脑相互独立运行并最终由智能体解读脑统一综合的架构图
subgraph InternalSystem["内部系统 (预测脑)"]
A1["历史库存数据"] --> B1["内部趋势分析"]
A2["日常运营数据"] --> B1
B1 --> C1["预测脑模型群<br/>(Predictive Brain)"]
end
subgraph ExternalSystem["宏观环境 (影响脑)"]
D1["FRED 经济指标"] --> E1["计量经济特征工程"]
D2["柴油价格/通胀"] --> E1
E1 --> C2["影响脑模型群<br/>(Influential Brain)"]
end
subgraph AgenticLayer["认知解译层 (智能体脑)"]
F1["30+ 外部行业研报"] --> G["智能体解读脑<br/>(Cognitive Agentic Brain)"]
C1 --> G
C2 --> G
end
G --> H["多模态市场决策支持体系"]
```
在系统设计中，经常有人问 Rcrysta：“既然两个脑都构建出来了，你们为什么不把它们加权相加，或者合成一个综合评分卡呢？”她的回答是：“绝对不要合并。我们将它们作为感知我们行业的两个独立脉搏予以保留。”这种保持独立演进与独立测算的机制，反而为企业利益相关者带来了前所未有的双重视角。
---
### [08:27](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=507s) 计量经济学门槛与智能体解读脑（Agentic Brain）
在构建“影响脑”的过程中，团队遇到了一个行业普遍存在的技能短板：绝大多数 AI 工程师或数据科学家并未受过严格的计量经济学（Econometrics）机器学习建模训练。将宏观经济指标与业务预测结合属于极其垂直的专业利基领域，传统建模团队往往更习惯于处理结构清晰、来源受控的企业内部数据。然而，一旦 Ryder 啃下了这块硬骨头、构建起标准化的宏观经济特征流水线，下游模型孵化的巨大想象空间便被瞬间激活了。
随着两个预测半脑就位，业务端随即抛出了新的痛点：“我们该如何把眼前这些预测数字与最新的货运市场大盘做深度比对？”为了回答这一问题，第三个关键脑区——“认知智能体解读脑”（Cognitive Agentic Interpretation Brain）应运而生。工程师在模型输出层之上搭建了 AI Agent。这套智能体不仅能深入内部模型抓取预测结果，还能直接理解经济数据间的深层关联。如果在需求规划仪表盘中某些特定关系没有现成的可视化图表（例如业务主管临时想知道“通货膨胀对柴油价格的具体关联度是多少”），智能体脑便能凭借底层打通的统一数据湖与知识库，自主进行动态关联推导与深度问答，突破传统固定 BI 界面能力的边界。
- **技能壁垒**：AI 工程师通常缺乏计量经济学建模沉淀，宏观与微观特征融合具有极高门槛。
- **智能体定位**：跳出固定仪表盘图表限制，提供深入到因果关联层面的自然语言交互问答。
- **认知整合**：既能向下理解多达 40 个数学模型的数理输出，又能横向对照非结构化行业大盘研报进行逻辑验证。
---
### [10:05](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=605s) 蜘蛛网架构（Spiderweb Architecture）与统一技术选型
谈到系统级工程架构时，Rcrysta 对比了不同场景下的流水线复杂度。在许多常规 AI 部署中，工作流往往呈现单向线性推进——从数据接入、预处理、特征工程到模型训练与部署上线。但当企业需要吸纳大量异构外部数据源，且每个源头都需要独立的数据工程清洗与特征转换时，架构图就会演变成一张错综复杂、相互交织的“蜘蛛网”（Spiderweb Architecture）。在 Ryder 的实践中，不同的预测模型对宏观经济因子的敏感度截然不同：某些产品线高度受制造业指数牵引，而另一些则与油价强绑定，导致多条并行数据工作流与不同模型节点之间呈现密集的网状交错关系。
为了支撑这套拥有 6 大核心工程处理流程、横跨数十个模型与智能体的复杂系统，团队在底层动用了超过 20 种不同的技术工具。Rcrysta 特别强调了企业级 AI 技术选型中的一个核心原则：**尽量收敛至单一厂商生态体系内部**。在 Ryder 的架构落地中，所有核心预测模型均部署在 Databricks 平台之上，而顶层的 AI Agent 则统一依托微软 Copilot Studio 构建。在大规模系统工程中，跨异构工具链部署极易遭遇组件间互操作性差、通信开销高昂等暗坑，而选用具备原生集成能力的成熟工具栈，是项目能够在短短几个月内实现规模化交付的关键防线。
- **蜘蛛网拓扑成因**：多源数据摄入、不同模型对经济驱动因子的异构敏感性，导致数据工程与模型计算产生网状交叉。
- **架构组件规模**：涵盖 6 大核心数据工程环节，整体流水线联动超过 20 种底层工具。
- **核心选型法则**：核心计算依托 Databricks，智能体生态依托 Copilot Studio，借助统一技术家族降低异构互操作性阻力。
---
### [11:44](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=704s) 数据工程与特征抽象：奖章架构与全程单元测试
在拆解后端数据工程管线时，Rcrysta 展示了经典的数据流向：数据从最初的接入层开始，经过严格的数据质量检验，随后注入标准的大数据工程管道。Ryder 全面采用了行业成熟的“奖章架构”（Medallion / Multi-hop Architecture）：
1. **Bronze 层（青铜层）**：原汁原味接入来自外部 API 与内部业务系统的原始源数据；
2. **Silver 层（白银层）**：完成数据脱敏、异常清洗、缺失值填补以及模式标准化；
3. **Gold 层（黄金层）**：产出高度凝练的业务聚合指标，供下游模型直接使用。
```mermaid
flowchart LR
accTitle: 奖章架构与特征抽象数据流
accContext: 展示从原始API数据经由青铜、白银、黄金层到特征表，并最终分发至模型与测试的全流程
subgraph DataEngineering["数据工程管道 (奖章架构)"]
direction TB
Raw["原始 API / 内部源数据"] --> Bronze["Bronze 层 (原始摄入)"]
Bronze --> Silver["Silver 层 (数据清洗)"]
Silver --> Gold["Gold 层 (业务聚合)"]
end
subgraph AbstractionLayer["抽象特征解耦"]
Gold --> FT[("统一特征表 Feature Tables")]
end
subgraph ModelServing["模型构建与消费"]
FT --> M1["预测脑模型群"]
FT --> M2["影响脑模型群"]
end
subgraph QualityControl["全生命周期测试"]
Bronze -.-> UT1["Schema 单元测试"]
Silver -.-> UT2["转换单元测试"]
FT -.-> UT3["漂移与特征测试"]
end
```
针对宏观经济数据建模，Rcrysta 提出了一条极其关键的生产避坑指南：**绝对不要让机器学习模型直接调用第三方外部 API**。外部数据源存在不可控的更新周期、网络波动甚至 Schema 漂移风险。最佳工程实践是在数据源与模型之间建立一道“特征表”（Feature Tables）物理抽象层，由定时作业将外部数据拉取沉淀至特征表中，再由模型从中读取特征。此外，不同于多数工程团队习惯在流水线末端统一执行测试的做法，Rcrysta 坚持在流水线的各个环节嵌入全流程单元测试：“当你在下游部署了多达 20 个模型时，如果直到最后才发现某一个模型的底层 Schema 发生错误，那种溯源排查的痛苦是难以忍受的；你必须一路沿着数据血缘回溯到最初的数据源，修复错误后再逐级向下传播更正。”因此，在数据生命周期的每一步设置断言与测试守门，是控制规模化模型维护成本的唯一解法。
---
### [13:46](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=826s) 预测数据接入：Power BI、SQL Warehouse 与 MCP 协议
在前端数据消费层，业务最初倾向于定制一套专门的独立 Web 应用程序。然而，架构团队在全面评估实际需求后断定，大部分预测分析界面的呈现完全可以通过 Power BI 深度实现，而将独立的交互式 Web 应用界面留给顶层 AI 智能体。然而，当多达 40 个模型投入运转时，每个模型在回测（Backtesting）、相关性分析（Correlation Analysis）、余弦相似度计算（Cosine Similarity）以及因果分析（Causal Analysis）阶段都会产生大量派生表，下游总数据表瞬间攀升至上百张之多，直接打爆传统的商业智能承载力。
为了解决这一问题，工程团队实施了深度的表数据归并整合策略，将原本散落在系统中的百余张输出表清洗汇聚成 4 至 5 张高维度的核心表，统一推入 Power BI 的语义模型层（Semantic Layer）。整个系统底层打通了 Databricks 的 SQL Warehouse，通过原生连接器实现预测数据的高吞吐流转。为了突破 Power BI 自带调度器的时间触发局限性，架构引入了 Microsoft Power Automate 作为高阶流程调度编排器：由 Power Automate 根据上游模型的跑批结束状态，自适应触发 Power BI 的语义层刷新，Power BI 再通过 SQL Warehouse 快速将预测输出拉取到报表层，实现端到端的事件驱动闭环。
在连接顶层的 Copilot 智能体时，系统设计展现了极高的扩展性。开发团队探索了两种打通模型知识与智能体认知的通路：
1. **直接知识库模式**：智能体通过服务主体（Service Principal）或个人访问令牌（PAT），直接通过 SQL Warehouse 以语义数据源形式挂载模型表；
2. **工具扩展模式**：利用 MCP 服务（Model Context Protocol Server），将模型推理输出封装为标准的外部工具插件，供智能体在推理链路上灵活调用。
- **数据合并降维**：将模型产生的上百张因果、回测及相似度表深度整合为 4-5 张核心大宽表注入语义层。
- **编排协同闭环**：通过 Power Automate 监听上游任务状态并编排刷新，替代简单的定时刷新机制。
- **双模接入支持**：支持 SQL Warehouse 直接查询鉴权挂载，或通过 MCP Server 标准化协议将数据作为 Tool 暴露给大语言模型。
---
### [16:34](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=994s) 预测脑的工程落地与时间序列特征构建
针对外界经常问及的“如何在极短时间内将规模如此庞大的模型群快速推向生产”这一疑惑，Rcrysta 剖析了整体性建模（Holistic Modeling）的核心秘诀。在工业界，构建模型最艰苦的环节永远是数据清洗与特征治理。在 Ryder 的架构下，不同模型的核心维度（如业务区域、细分产品等）原本就高度聚集在少数几个核心业务数据源中。由于多数模型只是对同一个主干数据集按照不同区域或业务线进行逻辑切片（Filtering），其底层绝大部分数据清洗、模式校验等预处理逻辑具有天然的可复用性。
因此，Rcrysta 极力推崇在多模型开发中沉淀“可复用工程模板”（Reusable Templates）。在构建“预测脑”的时间序列特征时，团队重点挖掘了滚动窗口统计量（Rolling Windows）、滞后变量（Lags）以及复合季节性周期。Rcrysta 特别提到了历史异常周期的处理：针对诸如 COVID-19 疫情大流行等特殊时期引发的非典型业务断崖与异常峰值，特征管线必须对这些结构性异常展开平滑降噪（Smoothing）处理，才能确保模型学到的是正常的底层商业运行规律，而非被极端黑天鹅事件引入漂移。在训练阶段，架构依托 Databricks 支持多模型并行训练与横向评估，并强依赖 MLflow 记录跨模型的实验版本、参数及指标，确保了从 12 月首次投产到次年春季宏观扩展的平滑迭代。
- **模板化提效**：利用底层业务数据特征的同源性，通过参数化切片与可复用代码模板支撑数十个并行模型的开发。
- **时间序列特征工程**：重点构建多尺度滚动窗口、时间滞后项，并对 COVID-19 等黑天鹅事件引发的周期性畸变实施专门的特征平滑。
- **生命周期追踪**：全程使用 MLflow 严密追踪多模型并行实验的指标走向、超参数与制品版本。
---
### [18:36](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1116s) 影响脑构建：宏观经济指标的复杂性与 FRED API 数据抓取
在转入“影响脑”的工程实战细节时，Rcrysta 直言，将宏观经济数据引入工业系统绝非简单地从公共 API 批量抓取几张现成表格那么轻松，前期需要耗费数月进行极其严谨的学术与商业机理研究。她举例说明了经济学统计口径的多样性与复杂性：“单就通货膨胀（Inflation）而言，业界就有多达 12 种截然不同的统计与核算方法；而在失业率（Unemployment）指标上，核心统计口径也至少有 6 种主流变体，此外还衍生出大量辅助分支指标。”在确定技术方案前，她花费了数月时间通读海量行业权威报告，并与外部供应商及业内专家深入研讨，才最终锁定与 Ryder 物流生态关联最紧密的代表性指标。
在具体的 API 自动化摄取工程上，Rcrysta 以圣路易斯联邦储备银行经济数据库（FRED API）为例，分享了底层抓取脚本的构建逻辑。在调用这类公开金融经济数据接口时，核心是精确定位每个指标所对应的“序列标识符”（Series ID）。工程团队将精挑细选后的各类经济指标 Series ID 整理为标准的系统字典配置，底层数据爬取脚本则遍历这一字典配置，通过自动化循环以高容错的方式定时拉取最新发布的数据序列，将其平稳输送至上游的数据湖 Bronze 摄入层中，为后续整套蜘蛛网架构提供高质量的外部血液。
- **指标复杂性壁垒**：通胀指标存在近 12 种计算口径，失业率存在 6 类核心统计变体，盲目抓取极易引入系统性偏差。
- **FRED 数据管道设计**：抽象指标 Series ID 配置字典，以驱动底层自动化批处理任务循环抓取外部开放经济数据。
- **工程设计思想**：前置严密机理分析，将经济学逻辑沉淀为参数配置，降低下游特征工程的维护难度。
### [19:38](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1178s) 宏观经济数据接入策略与容错管道设计
在处理外部数据源（例如美联储经济数据库 FRED）时，首要关注的是历史数据的覆盖跨度与刷新频率。在实际工程中，团队经常会遇到一些从理论建模角度看似极佳的经济指标，但由于其发布或更新缺乏规律性与及时性，最终无法在生产环境中使用。因此，数据更新的稳定性是选型评估的核心门槛。
当需要从外部 API 批量拉取成百上千个经济指标时，健壮的管道机制不可或缺。必须采用批次加载（Batch Loading）策略，并为每个请求配置至少 2 至 3 次重试机制。同时，API 访问逻辑需要设计精细的容错处理：如果某个平时存在的指标临时出现缺失或获取失败，管道绝不会直接让整个批次崩溃报错，而是捕获异常、记录具体缺失的指标名称并触发告警通知维护人员，其余数据则继续正常完成批次加载。采集入库后，数据通过聚合（Amalgamation）、上采样（Up-sampling）或下采样（Down-sampling）统一时间颗粒度，与企业现有的内部数据对齐合并，从而为下游模型训练做好准备。
```mermaid
flowchart LR
accTitle: 外部数据接入与容错处理管道流程
subgraph S1["数据获取与重试"]
A["FRED / 外部 API"] --> B["批次加载器<br/>(支持 2-3 次重试)"]
end
subgraph S2["容错与监控"]
B --> C{"指标是否可用?"}
C -- 否 --> D["记录缺失指标<br/>发送告警通知"]
C -- 是 --> E["合并成功批次"]
D -. 批次不中断.-> E
end
subgraph S3["特征工程与对齐"]
E --> F["上/下采样对齐<br/>(Up/Down-sampling)"]
F --> G["合并内部数据<br/>进入建模准备"]
end
```
---
### [20:33](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1233s) 异步候选模型评估与自动化再训练管道
在实际建模阶段，核心策略是“异步候选模型评估（Asynchronous Candidate Modeling）”。工程设计的基石是最大程度的自动化，日常运转由自动化管道全权负责，仅在指标异常或告警触发时才由工程师介入排查。通过可复用的工作流模板，系统能够同时并行评估多个候选算法方案。
整个评估链路深度集成了 MLflow 进行实验追踪。每次候选模型运行后，背后的参数、指标和制品都会被统一记录，以便工程师在必要时一键回滚到任意历史迭代版本。候选模型全部跑通后，系统通过输出评分表（Output Table）自动依据预设业务规则与指标进行加权排名，并自主选出最优的“冠军模型（Champion Model）”推向生产环境。
在常规的模型探索阶段，系统仅作实验记录而不需要强制注册冠军；但在生产环境的自动化再训练管道（Retraining Pipeline）中，整套机制形成了全自动闭环：定期拉取新数据、并行多模型跑分、筛选冠军模型、校验是否达到上线阈值。一旦所有候选模型均未达到性能基准，系统会立即触发预警，由工程师接入进行人工诊断和调优。
---
### [22:12](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1332s) MLOps 核心能力框架与端到端编排挑战
从方案探索走向生产化部署，必须依托系统化的 MLOps 实践手册。从 MLOps 的成熟度视角来看，支撑规模化运行的核心能力维度多达 21 项以上，包括此前架构中展现的基于 MLflow 的实验追踪，以及用于特征统一治理的特征存储库（Feature Store）等。
在实际跨团队协作中，许多数据科学家往往精于建模，但最薄弱、最具挑战性的环节是工作流编排（Workflow Orchestration）与流水线自动化（Pipeline Automation）。AI 工程师在日常工作中有时需要端到端自主完成“建模到部署”的全生命周期，但在更多场景下，往往是接手由其他算法科学家构建好的模型成果。在这种模式下，工作重心完全聚焦在工程落地与部署架构上，必须投入大量精力打磨稳固、标准化的自动化流水线。
- **MLOps 核心支撑**：涵盖模型注册、特征存储、实验追踪、CI/CD 编排等 21 项以上核心能力。
- **职责侧重转移**：AI 工程师需针对继承而来的分散模型，构建标准化、自动化且高容错的部署管线。
---
### [23:05](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1385s) Databricks 部署流水线、AI 审查 Agent 与 ML SecOps 安全治理
在 Ryder 系统的生产部署中，Databricks 构成了最为关键的基础底座。但在部署实施环节，团队引入了一项与传统 CI/CD 截然不同的创新机制：引入外部专门的“部署监督 AI Agent”。
通常团队在流水线内部都会编写单元测试，但部署阶段往往最容易发生配置漂移或敏感数据/代码泄露事故。针对通过 Databricks Asset Bundles 打包的部署产物，这名专属的 AI Agent 配备了包含最佳部署架构规范的内部知识库。在跨环境（开发、测试、生产）推进代码前，Agent 会对 Bundle 内容进行严格的规范核查与安全审查，只有获得该 Agent 的放行指令，流水线才允许推向下一环境。
这一实践紧密契合了当前前沿的 ML SecOps 理念。随着 AI 工程师的职责逐步向“兼顾底层基础设施与安全治理的架构师”转变，单纯保证模型上线已远远不够，企业级生成式 AI 与预测系统必须将合规与安全作为首要前提。在 Ryder，针对 Generative AI 应用场景的安全审查机制，已经与标准 MLOps 管道完成了深度融合。
---
### [25:00](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1500s) 问答一：海量模型切分是业务诉求还是算法刚需？
在进入观众问答环节时，一位现场参会者提出了一个充满技术共鸣的疑问：在过往业务中经常看到业务方不断施压要求细分，但从数据科学本质来看，一个基于决策树或聚类特性的全局大模型本身就能根据条件分支拟合不同客群的行为表现。那么在项目中构建多达数十上百个细分模型，究竟是“业务方的非理性胜利”，还是算法层面确实有此必要？
演讲者明确回应道：“作为一名 AI 工程师，我本能的选择永远是极力追求自动化和系统简洁化”，而海量模型切分的格局完全是由 Ryder 的实际业务特性驱动的：
1. **单一全局模型的失效**：早期团队曾使用聚合了全部区域与产品线的单一预测模型，但该模型完全无法为具体业务单元提供可操作的精细化洞察。Ryder 旗下涵盖供应链管理、仓储运营、车队运输等多个业务形态迥异的板块，数据特征差异巨大，必须进行横向切分。
2. **预测时间视距（Horizon）的差异化**：从商业规划角度看，6 至 8 个月的战术性短期预测，其准确率和特征依赖与 24 个月的长期规划预测截然不同，两者必须分别独立建模。
3. **多维笛卡尔积膨胀**：当“时间视距 × 业务区域 × 产品线”进行交叉组合后，模型数量迅速成倍放大。由此支撑的应用界面允许业务决策者通过筛选条件随意穿透查看，而前端每个视角的背后，实际都由针对该细分场景专门训练的模型独立驱动。
---
### [26:59](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1619s) 问答二：冠军模型频繁摇摆与滚动预测的防御机制
另一位参会者提出了一个深入工程细节的困惑：在其自身的工作流中，每个月都会自动化再训练并刷新冠军模型，但某些数据切片组合在月度更新时会在不同模型架构间“剧烈摇摆（Flip）”，导致不同版本的预测数值出现大幅震荡波动。参会者询问是否有现成的防护栏（Guardrails）机制，或者滚动预测本身是否能平抑此类抖动。
演讲者针对该问题拆解了滚动预测（Rolling Forecast）与静态预测（Static Forecast）的选型分水岭：
1. **静态预测优先原则**：若所处业务环境本身波动较低（Low Volatility）、不存在显著的数据漂移（Data Drift），盲目追求高频刷新只会适得其反。在此类平稳场景下，应采用静态预测，一次训练即可支撑未来 12 个月的视距；对于传统机器学习系统，只要未监测到模型漂移，通常一年重新训练一次即可。
2. **滚动预测的适用场景与波动根源**：Ryder 的主营业务与宏观经济周期深度绑定，环境剧烈波动，因此必须采用按月触发的滚动预测。面对宏观大宗变化，不同模型对拐点的捕获能力不同，自然会导致冠军模型的更迭。
3. **历史数据判定法则**：要判断自身场景是否需要滚动预测，关键在于审查历史序列数据。如果曲线剔除季节性后整体保持平缓且具规律性，坚决使用静态模型；如果历史数据呈现频繁的波峰波谷，对宏观经济与周期轮动高度敏感，才需要承担维护滚动再训练管道的成本。
## Notable quotes
> [04:08](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=248s) 当业务用户打开需求规划应用程序时，从 UI 视角看，他们可能觉得自己只是在下拉菜单中简单切换几个滤镜；但实际上，他们点击的每一个筛选标签，在后台对应的都是一个独立运行的预测性 AI 解决方案。 — Rcrysta Shaneih
> [05:56](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=356s) 绝对不要合并。我们将它们作为感知我们行业的两个独立脉搏予以保留。 — Rcrysta Shaneih
> [10:05](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=605s) 尽量收敛至单一厂商生态体系内部。 — Rcrysta Shaneih
> [11:44](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=704s) 绝对不要让机器学习模型直接调用第三方外部 API。 — Rcrysta Shaneih
> [11:44](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=704s) 当你在下游部署了多达 20 个模型时，如果直到最后才发现某一个模型的底层 Schema 发生错误，那种溯源排查的痛苦是难以忍受的；你必须一路沿着数据血缘回溯到最初的数据源，修复错误后再逐级向下传播更正。 — Rcrysta Shaneih
> [18:36](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1116s) 单就通货膨胀（Inflation）而言，业界就有多达 12 种截然不同的统计与核算方法；而在失业率（Unemployment）指标上，核心统计口径也至少有 6 种主流变体，此外还衍生出大量辅助分支指标。 — Rcrysta Shaneih
> [25:00](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1500s) 作为一名 AI 工程师，我本能的选择永远是极力追求自动化和系统简洁化 — Rcrysta Shaneih
> [26:59](https://www.youtube.com/watch?v=sSgOf1Vys9I&t=1619s) 若所处业务环境本身波动较低（Low Volatility）、不存在显著的数据漂移（Data Drift），盲目追求高频刷新只会适得其反。在此类平稳场景下，应采用静态预测，一次训练即可支撑未来 12 个月的视距；对于传统机器学习系统，只要未监测到模型漂移，通常一年重新训练一次即可。 — Rcrysta Shaneih
## Mentioned
Rcrysta Shaneih, Ryder Systems Inc., Ai4, Databricks, Microsoft Copilot Studio, Power BI, Microsoft Power Automate, MLflow, FRED API (Federal Reserve Economic Data), Databricks Asset Bundles, Model Context Protocol (MCP)
## Video description
Rcrysta Shaneih - Sr. AI/ML Engineer, Ryder Systems Inc.
Inspired by neuroscience, the 360-Degree Split-Brain Spiderweb Architecture introduces a novel way to scale holistic predictive intelligence pipelines by decoupling forecasting, influence modeling, and interpretation into specialized but connected components. At Ryder Systems Inc., this architecture was implemented to scale multi-model pipelines across platforms, leveraging modular deployment, automation, and orchestration to support production forecasting and market intelligence use cases.
Session attendees will learn how to build enterprise-ready AI architectures that mimic cognitive specialization: deploying stable forecasting models as the “Predictive Brain,” pairing them with influence and explainability models as the “Influential Brain,” and extending interpretation through a “Cognitive Agentic Interpretation Brain” that helps translate analytical outputs into decision-ready intelligence.
This session will equip AI/ML engineers, data scientists, and AI leaders with a practical blueprint for building scalable, interpretable, and production-ready predictive intelligence systems that move beyond single-model forecasting toward holistic AI architectures.
[Source](https://www.youtube.com/watch?v=sSgOf1Vys9I)
[4:08] Rcrysta Shaneih: in reality, every filter you're selecting is really a different predictive AI solution on the back end that's showcasing the results.
Original transcript
[0:04] Really glad that you guys decided to
stick around for our 4:30 session.
[0:08] Um so, I'm Ark Kristashonai and I'm a senior AI
engineer for Ryder Systems Inc. And today we're
[0:15] going to highlight a novel approach uh that was
built out for scaling our predictive pipelines
[0:21] leveraging the Split Brain Predictive Framework.
And we're also going to showcase how we deploy
[0:26] that leveraging Spiderweb architecture. So, uh
I have you guys for about 35 minutes with Q&A
[0:34] at the end. So, let's just get started. So,
a little bit about me. I'm a PhD researcher
[0:41] where I'm specializing within AI. I've worked
within the data AI space for the past decade.
[0:47] Um in that time frame, I've held
positions like being a scientist to
[0:51] ML Ops engineer and now as an AI engineer, um
I help multiple teams within Ryder scale their
[0:58] solutions into production while also building
out use cases for several of our departments.
[1:04] Um over the years I've also deployed over 250
models in a production and I've had the pleasure
[1:10] of working on several collaborative and individual
projects that have won AI awards and recognition.
[1:18] So, a little bit about Ryder for those of you
that might not know about about the company. So,
[1:23] Ryder's been around for a while. It's actually
one of the leading providers of logistics and
[1:29] transportation services for North America. The
company currently has over 50,000 employees.
[1:36] We have 800 maintenance locations, 300
warehouses. And when you think about all of
[1:42] the fleet that we manage, we manage over 240,000
commercial trucks. And within a single year,
[1:49] Ryder is responsible for supporting more than
340,000 cross-border freight movements throughout
[1:57] Canada, US, and Mexico. So, when you're
thinking about a large-scale operations
[2:03] such as Ryder that has divisions which span
across transportation, warehousing, logistics,
[2:09] often times when we build models or AI solutions,
we can't just support the business with one
[2:15] solution. Sometimes we need that solution to be
able to be segmented by product lines, by regions,
[2:21] by geographies, by different classifications to
provide a holistic visibility of the operations.
[2:29] And that often times means that we're building
multiple predictive solutions. And whenever
[2:34] we have to build and scale multiple solutions
instead of just one, that's what I call holistic
[2:41] modeling because the deployment for that is
little bit different. The architecture for those
[2:46] kind of patterns have some unique challenges,
and so we're going to dive into that as well.
[2:53] So, this is pretty much the end result. So,
I'm going to peel back the architecture in
[2:58] upcoming slides. So, over the past 8 months, 40
predictive AI solutions were actually scaled,
[3:06] and these were predictive forecasts based on our
internal drivers and incorporating macroeconomic
[3:12] indicators. So, all of those solutions fed
into a demand planning tool, and we also build
[3:18] AI agents on top of that. Now, our AI agents
were able to synthesize all of that output data
[3:24] to provide interpretation for the business, but
it also brought in other modalities as well.
[3:30] So, in our industry, we often have industry
reports, market-related analysis, economic trends,
[3:38] anything related to freight transportation supply
chain. So, over 30 different industry sources
[3:44] is what we leverage, and our agents have access
to many of those sources. So, now our agents are
[3:50] able to explain the predictive output results
and assess how it matches against the market.
[3:57] And that's where we get into our multi-modal
market intelligence and decision support system.
[4:08] So, now that you've seen the end state, let's
kind of reverse engineer this back, okay?
[4:13] Uh so, you kind of seen what our goal was, but
when we start back 8 months ago um from the time
[4:19] that we deployed So, almost a year at this point.
Um the original requirements were in September
[4:23] 2025. So, this is where we're going to kind of
time travel back and look at the architecture.
[4:30] So, pretty much this was the original
request. So, as you saw in the previous slide,
[4:35] 40 solutions were deployed. However, 18 were the
original request. So, in starting this project,
[4:41] it was already known that there would be a huge
data ingestion component that's there because
[4:46] many of the external economic indicators we had
to pull from various source systems that we didn't
[4:52] really have the robust pipelines for. And so, the
idea was to bring that in and then build these
[4:57] holistic models that were segmented by region.
And so, not only were the models uh tasked with
[5:03] being able to provide tactical 6-8 month forecasts
upwards to strategic forecasts of 18-24 months,
[5:11] their goal was also to highlight how different
indicators impacted various product segments.
[5:18] So, you can imagine for our supply chain
um anything related to inflation um shifts
[5:24] within diesel prices uh industrial manufacturing
directly or indirectly, that might have an impact
[5:32] on our underlying business. And so, that was the
goal of the models. And when you're looking at
[5:37] the demand planning application um from the
UI perspective, if you're the business user,
[5:43] you just think you're selecting filters. Uh
but in reality, every filter you're selecting
[5:48] is really a different predictive AI solution
on the back end that's showcasing the results.
[5:56] And while we were developing this, this is
where a novel framework actually emerged. And
[6:02] much of this pulls from um my existing um PhD
research. So, originally we were tasked with
[6:09] building the influential brain, which was
really the macroeconomic forecast models.
[6:14] However, we actually scaled models based on
our internal drivers. So, we moved 18 of those
[6:21] models into production. And so, the internal
driver models pulled all of our historical
[6:26] uh inventory, operational data, any seasonality,
all of our trends. And pretty much we were able
[6:33] to give this to the business by December
while we work on building out some of the
[6:39] economic pipelines so that we could release
the influential brain models in the spring.
[6:45] So, the original plan was just swap out with
iteration number two, the predictive brain. But
[6:51] what we found was that the model was pretty much
spot-on. The predictive brain models were capable
[6:58] of really sensing shifts within our inventory, um
shifts within our product segmentations. And so,
[7:05] that's where we kept them in production.
Once we deployed the influential brain,
[7:09] now we were able to see how the indicators
influence the forecast. And what kind of
[7:16] you know, associations were made from there. And
that's where we did not plan this, but we had two
[7:22] different types of predictive solutions, which
totaled upwards of 40 models by the end state,
[7:29] that targeted different aspects of our business.
And when you get into looking at where this pulls
[7:36] from, this pulls from neuroscience. So, to treat
severe epilepsy, uh there is a technique that's
[7:43] called split brain where the corpus callosum
is actually severed and it separates two
[7:49] sides of the hemisphere. And in the medical
world for patients who have severe epilepsy,
[7:55] instead of having seizures spread throughout
the brain, when they do that surgical procedure,
[8:00] now it creates two separate hemispheres. So, if
they're seizing on one side of the hemisphere,
[8:06] it doesn't impact the other that's healthy and
allows them to still have cognitive function.
[8:11] And this is much how our models work. Um sometimes
people think, "Well, now you have the two models.
[8:17] Do we add them together? Do we create a
scorecard?" No, we keep them operating as
[8:21] separate pulses on our industry. And that's what
our stakeholders have found absolutely beneficial.
[8:27] So, this was just something that we noticed. Now,
one of the challenges with the influential brain
[8:33] models is that most AI engineers aren't really
trained or don't have a lot of experience building
[8:39] a kind of metric machine learning models. That
is a distinctive niche. And so, typically when we
[8:45] see traditional modeling solutions, most of the
solutions that we're building out is leveraging
[8:50] our internal data. But, once the pipelines are
built in to bring in all this economic data,
[8:55] it really opened up tremendous opportunities for
additional models that we [clears throat] could
[8:59] build downstream. Now, when it came to serving
this to the business, one of the requests was,
[9:05] "Wow, how does this compare to all of our market
industry data?" So, that created the Agentyc
[9:11] Brain. Well, we can build our AI agents on top.
It can assess our um internal models that we built
[9:19] and then also marry that with the market. Now,
our AI agents, they're able to go a little bit
[9:25] deeper and are more customized to answer inquiries
because some aspects we didn't build in our demand
[9:31] planning tool that AI agent is able to capture
because it now has all of the data. So, where
[9:37] our models really focus on economic indicators
paired with our own internal data, because we
[9:43] built that pipeline, if our stakeholders wanted
to understand, well, how does inflation impact,
[9:51] you know, diesel prices? What's the association?
Or any of the indicators that's there, the model
[9:58] um agentic brain layer is able to
actually go and make those associations.
[10:05] So, when it comes to the architecture here.
So, in my years of deploying solutions,
[10:11] sometimes you'll deploy models where the workflow
is just linear. You know, you have ingestion,
[10:16] pre-processing, modeling, and then you have
deployment. Other times you're going to work
[10:21] with workflows where you have distinctive work
streams. And this is common when you have multiple
[10:27] uh data sources coming in, and they all
have their data engineering pipelines.
[10:32] What you start to see in the architectural
workflow is this interconnected web of spider
[10:37] webs, where you have this work stream that's here
and connected to this model, and not all of the
[10:44] uh models that we built respond to the same um
economic indicators. So, with this, whenever I
[10:50] see a complex workflow or workflow and many work
streams, I call it a spider web, because when you
[10:56] look at it, that's kind of what it's forming. And
so, in our in engineering layer, which we knew
[11:00] that that was a big factor here, we have about
six processes. And then, overall, when it comes
[11:06] to scaling this, where you have models and agents
that were upwards of 20 different tools. So, our
[11:12] models were built out within Databricks, and then
our AI agents were built out in Copilot Studio.
[11:18] And one of the reasons why, when you're looking
at most of the solutions there, you could see that
[11:22] they're predominantly under one vendor family,
whenever you're building out multiple models,
[11:27] it helps when you're leveraging tools that have
natural integration capabilities. So, that's why
[11:34] uh of all the enterprise solutions and tools
we have, we leverage the ones where we know
[11:38] we wouldn't have any interoperability as
we scale and deploy them across systems.
[11:44] And so, when you're looking at the back-end
development architecture, so for any of you that
[11:49] have deployed models, much of this architecture is
pretty much familiar. So, you have the ingestion
[11:55] layer, you have all the quality checks, and then
you have the actual data engineering pipeline.
[12:01] If you've ever done data engineering work, seeing
a medallion or multi-hop architecture where the
[12:07] raw source system data comes in into the bronze
layer, you're cleansing it in the silver layer,
[12:12] and you're making it available for downstream
modeling in the gold layer won't be new for you.
[12:18] But, for the most part, you have the models
that are built out, you have evaluation, and
[12:23] then you have unit testing and serving into the
application. A couple things that's distinctive
[12:28] about building economic models is you can see
where we have the abstraction layer between the
[12:34] source systems versus the models. The models are
fed feature tables. So, when you're modeling with
[12:42] third-party APIs, you never want your models to
pull directly from the APIs because that's just
[12:50] not efficient. And sometimes with source systems,
the data could change because it is, you know,
[12:56] they have different refresh cycles. So, best
practice is to actually pull the data in, store it
[13:03] as a feature table, and then build your models on
top. When it comes to unit testing, sometimes you
[13:09] see unit testing performed at the end. When you're
building out holistic models in these pipelines,
[13:15] I like to embed different unit tests throughout
the pipeline instead of waiting till the end
[13:21] because it's quite painful when you have, you
know, upwards of 20 different models that you're
[13:26] deploying and you notice that a schema is off on
one model downstream. Now you have to trace the
[13:32] lineage all the way back to where that originally
began and fix that error and then propagate that
[13:38] change downstream. So I find it helpful to do
unit testing throughout fastest the life cycle.
[13:46] So when we look at the architecture for embedding
all and ingesting all this data into a demand
[13:51] planning tool, the original requirement was
looking for some kind of a web app and when
[13:57] we looked at the requirements, pretty much
everything we could build out within Power BI.
[14:02] We did save the app interface for the AI agents,
however. And so when you're dealing with upwards
[14:08] of 40 models that all have output tables that
highlight backtesting, correlation analysis,
[14:15] cosine similarity, causal analysis, all
of the the tables that we pulled out,
[14:22] you're pretty much going to climb into upwards
of a hundred different tables that need to be
[14:26] ingested into data in data, I'm sorry, into Power
BI. So what we did was amalgamate that data as
[14:32] much as possible to about four or five tables
that were loaded into Power BI semantic layer.
[14:39] When it comes to integrating the tools with Data
Bricks, you're leveraging their SQL warehouse
[14:45] which natively lets you bring in the data into
Power BI. You load that into the semantic layer
[14:51] and then you build your UI on top of that.
Now if you want more control over scheduling,
[14:58] which that was one of the things we wanted,
you can bring Power Automate on top and you can
[15:03] customize your schedule refreshes. So we found
that we had some limitations with leveraging
[15:09] Power BI scheduler, but once we put Power
Automate, now we could customize that downstream.
[15:14] And so how that works is where Power Automate
triggers the refreshes of Power BI. Power BI
[15:20] pulls from the SQL Warehouse to retrieve
the predictive outputs from Databricks.
[15:28] Now, when we look at this architecture for
Copilot, now, once again, that same SQL Warehouse,
[15:35] you can also leverage that to integrate with
Copilot. You also have another way you can
[15:40] leverage an MCP server. So, if you want to
bring in your tables as a knowledge base,
[15:46] you can leverage um pretty much just the direct
connectivity through uh a SQL Warehouse with,
[15:52] of course, the service principal IM credentials.
Even the personal access tokens will let you bring
[15:57] that data in. Or, you can leverage and bring it in
as a tool using MCP server. So, our architecture
[16:04] for the solution we deploy takes a little bit
of a different approach than what you see here,
[16:10] but this was one of our earlier architectural
model. So, we wind up building multiple agents,
[16:15] some deployed through Teams, some deployed
through an actual web app interface. But,
[16:21] the use case kind of comes to the same result.
We are giving the business a holistic interface
[16:27] where it can access the predictive results
and it can access the market data as well.
[16:34] So, in deploying these solutions, there were
several best practices for how you actually build
[16:40] some of the use cases that I showed in the prior
slides. So, when it comes to the predictive brain,
[16:47] um one of the questions I've been asked before
is, "How do you scale so many holistic models
[16:52] in such such a short window of time?" Well, one
of the beauties of holistic models is that for
[16:57] the most part, the hardest part of our job is
really getting the data right. And often times,
[17:03] when you're building holistic models, much of your
classifications are predominantly in a single data
[17:09] set. So, we had to use only a few internal data
sets. The The difficulty was bringing actually the
[17:14] economic data in. But, for the most part,
most of our data already had like regional
[17:20] classifications, product, and that was just kind
of filtering it. So, a lot of your cleansing steps
[17:25] are reusable. And so, one of the things when
you're building these types of solutions where
[17:30] you're maybe just filtering on a different region
or product and then building a different model,
[17:35] um I do recommend reusable templates. Um when it
comes to the feature engineering, for those of
[17:41] you that maybe haven't worked with building out
temporal economic data, and this is actually not
[17:46] the economic model at this point. You want to take
into consideration seasonality, rolling windows,
[17:53] any kind of lags. Um if you have to treat cyclical
abnormalities like the COVID period, you smooth
[18:00] all of that down and then your models are able
to learn normal business patterns and activities.
[18:06] When it comes to modeling, I typically like to
uh run models in parallel where I can evaluate
[18:12] across them. And then at the end of that, you
can see where MLflow is absolutely essential.
[18:18] So, for this deployment, there were two
iterative deployment cycles. Our internal
[18:23] models were deployed in December, and then in
the spring we had the uh macroeconomic models,
[18:29] and MLflow was essential as we were running
through multiple experiments across the models.
[18:36] So, when we look at the influential brain here, um
some tidbits if you've never worked with economic
[18:41] data, it's not just pulling a plethora of data
from the API. You do want to conduct research
[18:48] on this as well. So, when you get into inflation,
there's nearly 12 different ways you can calculate
[18:54] inflation. When you get into unemployment, there's
about six different core variations of that,
[18:59] and then you have additional variation
variants of that. So, you really want
[19:03] to conduct a lot of research. So, I spent a
couple months looking at industry reports,
[19:08] uh talking to a lot of our vendors to figure
out which were the um indicators that were more
[19:14] associated to bring in. When you're bringing it in
from Fred, for example, you want to pay attention
[19:20] to the series ID. A lot of these open public APIs,
whatever indicator you're bringing in, you're
[19:26] going to take that series ID and then you're going
to put it in a dictionary and your code is going
[19:31] to loop through that API and bring that in. Um for
every indicator, you also want to look at and see
[19:38] how far back the data goes, how frequently
is it refreshed. There's a lot of indicators
[19:43] that would be that we thought were awesome for
modeling, but it just wasn't refreshed regularly.
[19:48] So, that's one thing you want to look at. Um
when you're pulling from a source like Fred,
[19:54] if you're pulling in hundreds of indicators,
you typically want to load that in batches
[19:59] and you want to at least have at least two
to three retries when you're loading that in.
[20:03] So, and in your logic, you also want to test
that API. So, the logic I have here is that if,
[20:11] let's say, indicator that's usually available
is not available, it will not fail the batch,
[20:17] it will just let me know which indicator
wasn't available and send me an alert,
[20:21] but it will still have a successful batch. And
once you bring that in, you just amalgamate them
[20:25] and then down or up sample them to merge with your
existing data and then you're ready for modeling.
[20:33] Now, one of the techniques that I use a lot
when I'm actually building my models, um I love
[20:40] like just evaluating multiple candidate solutions
simultaneously. And this is where I just call it
[20:46] asynchronous candidate modeling. Um so, pretty
much I love to automate as much as possible. I
[20:52] automate if there's alerts, then I go back and I I
assess things manually. But because a lot of these
[20:59] worked on reusable workflow patterns, in the
patterns you can see where in my code there, I
[21:05] have multiple models. And this is not, of course,
our actual code. This is just samples here for the
[21:09] presentation. Um but I have multiple models that
are being evaluated. I also nest MLflow in there.
[21:17] So every time a model is run, it's tracking that
on the back end. So I can assess and if I wanted
[21:24] to roll back to a different model iteration, I can
do so. And once it goes through all the candidate
[21:31] models, I have an output table which ranks and
scores based on my logic and it automatically
[21:39] selects the most candidate model and move that
into production. So this is a similar to what
[21:44] I would how I would build out of a retraining
pipeline. Um for just model testing, I wouldn't
[21:49] have it necessarily log the champion model,
but when I'm um building a retraining pipeline,
[21:54] as the the model refreshes, I typically have
it evaluate multiple models, um put the ideal
[22:00] champion in production. If the thresholds are not
met to be put in production, I receive an alert
[22:06] and I go back and then I and I kind of test
it out and and see what I can fix manually.
[22:12] And in deploying a lot of solutions in
production, um this is essential when you
[22:18] think about like a MLops playbook, what are
some core factors that you want to consider
[22:23] when you're scaling and deploying your solution.
So there's upwards of 21 core capabilities from
[22:30] a MLops perspective. Many of them you've already
seen in our previous architectural diagrams where
[22:36] you've seen me talk about experiment tracking
through MLflow or leveraging feature stores.
[22:41] One of the challenges I see amongst a lot of
scientists is really the workflow orchestration
[22:46] or pipeline automation. Sometimes in my work, I'm
building the models into and and deploying them.
[22:52] Other times I'm coming in and inheriting models
and just focus on the deployment aspect. So I
[22:57] do have to spend a lot of time with just the
structures of of pipeline automation that's there.
[23:05] And then as we take a look at our last slide
here, this is the actual deployment workflow
[23:11] as I mentioned, Databricks is the predominant
tool that's used. What is different in how I
[23:17] do my deployments apart from the previous slide
where you can see all the facets that I look for
[23:23] is that I also have an external AI agent that
monitors the actual bundles that I deploy. So
[23:31] unit testing is embedded throughout the pipelines
but then sometimes people forget about how we
[23:36] deploy and often times we are exposing a lot of
sensitive data or sensitive code as we deploy.
[23:43] So when I bundle and get ready
to deploy across environments,
[23:48] my AI agent actually has a knowledge base on
how our ideal deployment pattern should look and
[23:54] it evaluates the bundles and if it gets an okay
then we're able to move across the environments.
[24:00] One of the thing that you will see lastly is ML
SecOps is incorporated into our ML Ops pipeline
[24:08] and not a lot of organizations have this but
as the role of the AI engineer is consistently
[24:14] changing where we're moving more into architects,
focus on infrastructure security. I'm fortunate
[24:21] that at Writer I've been able to be heavily
involved in a lot of our security assessments
[24:27] when it comes to enabling GenAI use cases across
our applications. And so we're not just deploying
[24:34] solutions, we're also deploying them with
governance and security and as a forefront.
[24:42] And now, you know, you've seen the blast from the
past and I've bringing you back to where we are in
[24:48] 2026 and this concludes my presentation. We have
several minutes left if there are any questions.
[25:00] Thank you. Uh the my question was uh that
you know, in in a very similar I mean,
[25:06] it's a complete deja vu for me, but in that
company as well, uh the business pushed a lot,
[25:11] but if you look [snorts] at it from a data science
perspective, uh you know, it's a like a decision
[25:15] tree, right? You're basically saying that okay, I
have a segment that if these conditions are true,
[25:20] then it should predict more or less. So, when
you said right that that would apply for any
[25:24] model you build that okay, that you know,
there's a cluster that ends up in behaving
[25:29] based on these conditions. So, so still uh
you know, obviously, sometimes business wins,
[25:33] but I was just curious that is it the business
winning or was it data science that actually
[25:39] you know, requires building so many models? Sure.
So, I'm an AI engineer, so I'm always going to
[25:43] automate and do as simplistic as possible.
It was actually requested by the business.
[25:48] Um so, originally, there was a one holistic model.
One I'm sorry, not holistic. It was one model
[25:54] that aggregated regions and product segments,
and that was not giving operational visibility
[26:00] across the different products. So, for our
business, which which is very distinctive across
[26:06] like supply chain, warehouse, transportation, you
really need to cut and slice that differently.
[26:13] So, that was actually a request by the business.
Now, one other thing that inflated the numbers
[26:19] is when you're looking at forecast, sometimes from
a business planning decision, you have a tactical
[26:24] forecast, which is 6 to 8 months. That's typically
going to be more accurate than like a 24-month
[26:30] forecast where you go forward out. However, when
you're building out like a 6-month or 24-month
[26:36] forecast, those are two separate models. So, when
you start slicing it up across horizon, region,
[26:41] product, that's where it starts adding um because
you're just giving different views. And so, when
[26:47] where that powered the actual application, now it
gives a holistic visibility where the business can
[26:53] click and filter and see these things and it's a
different model view that's reflected. Thank you.
[26:59] I think I have a follow-up on the championship
model part cuz I will also work on something
[27:06] similar. My struggle like this might be too much
like a little bit into the technical side but
[27:15] personally like I refresh the championship
model every month probably for certain combo
[27:21] is going to flip between different models
really quick and sometimes the variation
[27:28] like the different versions of forecast could
fluctuate a lot. So is there any guardrails you
[27:35] implemented to like prevent these from
happening or like the rolling forecast
[27:42] you mentioned will be able to address that?
Sure. So the rolling forecast addresses that but
[27:48] as I mentioned previously if your business is not
volatile you can just pull off a static forecast
[27:54] and you don't have to have the retraining
that frequently. So that's the the rule of
[27:58] thumb and even I'd even outside of like forecast
models if you have no drift if you don't have
[28:04] fluctuations in your business where the data is
changing you don't have to do a frequent refresh
[28:09] or retrain or update. Typically when I build
traditional machine learning models in production
[28:15] if I don't see signs of drift if it's still there
I try to retrain at least once a year but because
[28:22] our business is very volatile and of course with a
lot of things is happening with the economy that's
[28:27] why we have a rolling forecast where we train
monthly but we do have other models in production
[28:33] that have less volatility where they're modeling
different aspects of the business those are static
[28:38] forecast. We build them and then the business can
rely on that for you know whatever the horizon is
[28:44] the next 12 months because we don't see a lot of
shifts in volatility. If you want to test whether
[28:50] your data needs a rolling forecast or static, just
look at the historicals. If you start to see where
[28:56] it's pretty much flat, and of course you're going
to see seasonality, but if it's pretty much flat
[29:01] and standard and predictable, that's where you
know you can build a static forecast. If you start
[29:06] seeing ups and downs and dips and it's really
responding to cyclical movements in the in the you
[29:12] know economy, that's when you might need a rolling
forecast. If you see that volatility historically.
