Meibel Touts Structured Checkpoints to Fix Enterprise LLM Failures
Ai4
Pure end-to-end language models fail on arithmetic and compliance, requiring enterprises to use AI solely for intermediate schema extraction while offloading analysis to deterministic code.
## Background
Published on YouTube by the Ai4 channel on August 27, 2025, this presentation features Berk Ekmekci, Research Scientist at Meibel. According to the description, the talk introduces the concept of 'structurable data' to show how modern AI techniques can bridge the divide between structured and unstructured formats to maximize data value in enterprise workflows.
## TL;DR
Berk Ekmekci introduces 'structurable data,' defining it as unstructured or semi-structured information from which machine-usable, structured representations can be systematically derived. He evaluates traditional ETL, pure end-to-end LLM chains, and hybrid AI structuring pipelines across manufacturing, defense, and healthcare use cases, showing that intermediate checkpoints optimize arithmetic accuracy, cost, and auditability. Addressing schema selection and Richard Sutton's 'Bitter Lesson,' the presentation argues that hybrid structured interfaces deliver essential near-term business value and governance while general-purpose AI continues to mature.
## Atmosphere & style
The presentation carries a polished, analytical, and pragmatic conference tone centered on enterprise systems engineering. Ekmekci speaks with clear, deliberate pacing, methodically breaking down pipeline trade-offs and historical computing shifts without relying on generic AI hype. The session feels like an authoritative architectural briefing designed to reassure engineering leaders that messy legacy data can be tamed through disciplined, inspectable AI checkpoints.
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=9a1lhytoqpk&t=4s) Structurable data bridges the gap between rigid structured databases and raw unstructured files, treating structurability as a continuous spectrum enabled by modern AI.
- [00:04](https://www.youtube.com/watch?v=9a1lhytoqpk&t=4s) Data 'messiness' is not an intrinsic property of files or storage formats, but rather a reflection of an organization's historical inability to structure that information.
- [02:33](https://www.youtube.com/watch?v=9a1lhytoqpk&t=153s) Data structuring has evolved from brittle, upfront relational schemas and manual ETL in the 1990s–2000s to multimodal extraction and dynamic NoSQL schemas in the 2010s.
- [05:03](https://www.youtube.com/watch?v=9a1lhytoqpk&t=303s) Pure end-to-end LLM pipelines suffer from major production drawbacks, including arithmetic hallucinations, high token inference latency, opaque evaluation, and escalating costs.
- [05:03](https://www.youtube.com/watch?v=9a1lhytoqpk&t=303s) A hybrid AI structuring architecture leverages LLMs solely to extract raw data into a stable intermediate schema, offloading mathematical and analytical logic to deterministic code.
- [05:03](https://www.youtube.com/watch?v=9a1lhytoqpk&t=303s) Intermediate schema checkpoints provide critical human- and machine-readable boundaries that ensure regulatory compliance, auditability, and seamless integration with legacy databases.
- [09:44](https://www.youtube.com/watch?v=9a1lhytoqpk&t=584s) Echoing George Box's aphorism on models, 'All data is structurable, not all structures are useful,' emphasizing that target schema selection requires domain-specific prioritization.
- [09:44](https://www.youtube.com/watch?v=9a1lhytoqpk&t=584s) Modern AI structuring lowers transformation costs, enabling engineering teams to rapidly prototype, A/B test, and iteratively refine intermediate schemas for optimal information capture.
- [09:44](https://www.youtube.com/watch?v=9a1lhytoqpk&t=584s) While Richard Sutton's 'Bitter Lesson' shows general compute-driven methods win over long horizons, practical enterprises capture immense commercial value today within the multi-year transition window using structured checkpoints.
- [13:56](https://www.youtube.com/watch?v=9a1lhytoqpk&t=836s) Future enterprise AI systems will rely on dynamic, multimodal ingestion pipelines coupled with intelligent routing layers that direct queries to optimized structural representations.
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=9a1lhytoqpk&t=4s) Introducing Structurable Data Across Industry Domains
Berk Ekmekci, a research engineer at Meibel, introduces Meibel as the runtime platform for "confident AI." A central challenge in production AI is meeting enterprise data where it actually lives and transforming existing data formats into representations required for downstream intelligence tasks. Rather than viewing data strictly as an absolute binary—structured versus unstructured—modern AI structuring methods unlock a new operational paradigm called **structurable data**. Structurable data is defined as less structured or completely unstructured data from which a more structured, machine-usable representation can be derived.
```mermaid
flowchart TD
accTitle: Structurable Data Processing Workflow
subgraph Inputs["Diverse Raw Inputs"]
A["Document Scans & Images"]
B["Supplier Spec Sheets / Spreadsheets"]
C["Domain Codes & Technical Notes"]
end
subgraph Structuring["AI Structuring Layer"]
D["AI Structuring Engine"]
end
subgraph Target["Intermediate Target Representation"]
E["Defined Human/Machine Checkpoint"]
end
The concept of structurable data cuts across critical, high-stakes industries:
- **Manufacturing and Heavy Industry:** Organizations frequently manage a constellation of disorganized document types—supplier Excel workbooks, scanned spec sheets, and technical diagrams—where critical operational parameters reside in isolation.
- **Clinical Research and Auditing:** Datasets often contain hyper-specific or overly complex billing and diagnosis codes across disparate clinical documents, requiring harmonization into a unified, accessible structure.
- **Defense Logistics and Mission Readiness:** Analysts must reconcile granular component telemetry alongside high-level strategic summaries to maintain tactical visibility.
Data "messiness" is not merely an intrinsic property of the files or storage mediums; it is fundamentally a function of an organization's ability—or historical inability—to structure that information meaningfully.
---
[02:33](https://www.youtube.com/watch?v=9a1lhytoqpk&t=153s) The Evolution of Data Structuring: From SQL to Modern AI
Data structuring and schema transformation have historical precedents spanning decades of computing. In the 1990s and 2000s, tabular schemas and relational databases dominated enterprise workflows. SQL gained massive adoption, Microsoft Excel became standard, and tools like Microsoft Access attempted to bridge relational schemas and desktop end users ("trying being the operative word"). However, during this period, machine usability demanded rigid, upfront schema definitions. Moving data into these formats required intensive manual data entry or brittle extract, transform, and load (ETL) pipelines.
```mermaid
flowchart LR
accTitle: Historical Evolution of Data Structuring
subgraph Era1["1990s–2000s"]
A["Relational DBs & SQL"] --> B["High Machine Usability Threshold<br/>(Rigid ETL / Manual Entry)"]
end
subgraph Era2["2010s"]
C["GPU Advances & CNNs"] --> D["Multimodal Extraction<br/>(Vision, Audio, Video)"]
E["NoSQL Databases"] --> F["Dynamic Schemas<br/>(Lower Entry Threshold)"]
end
subgraph Era3["Present (2020s+)"]
G["Modern AI Structuring"] --> H["Continuous Spectrum from Unstructured to Structured"]
end
```
The paradigm shifted substantially across two technical waves in the 2010s:
1. **GPU Acceleration and Convolutional Neural Networks:** Neural network concepts dating back decades became computationally feasible for widespread commercial training and inference. This enabled flexible extraction across unstructured modalities like images, audio, and video.
2. **The Rise of NoSQL Databases:** Non-relational stores lowered the threshold for machine ingest by allowing semi-structured documents with dynamic or unknown schemas to be stored in collections and queried immediately.
Today, structurability exists along a continuous spectrum ranging from totally unstructured artifacts to strictly modeled tables. AI systems can ingest, retrieve, and transform data at any point along this spectrum, raising foundational design questions for practitioners constructing automated pipelines.
---
[05:03](https://www.youtube.com/watch?v=9a1lhytoqpk&t=303s) Comparing ETL, Pure End-to-End LLMs, and AI Structuring Pipelines
To illustrate architectural trade-offs, Berk examines a technical manufacturing report workflow where engineers seek to extract subcomponent hierarchies, cascade costs, and calculate complexity metrics across multiple suppliers.
| Architecture Paradigm | Setup Complexity & Flexibility | Execution Speed & Cost | Mathematical & Analytical Precision | Evaluability & Governance |
|:--- |:--- |:--- |:--- |:--- |
| **Traditional Deterministic ETL** | Low flexibility; breaks easily when input layout changes. | Extremely fast and cheap once compiled. | Highly reliable, deterministic calculations via SQL/code. | High auditability; verifiable checkpoint schemas. |
| **End-to-End LLM Chaining** | High setup flexibility; ingest prompts without intermediate schemas. | Slow runtime latency; high cumulative token inference costs. | Prone to arithmetic errors; requires tool calling or MCP schemas to calculate reliably. | Opaque black box; difficult to benchmark as inputs grow in complexity. |
| **Hybrid AI Structuring Checkpoint** | High flexibility on varied inputs; maps diverse formats into stable schemas. | Optimized costs; LLM used only for schema extraction, deterministic code for math. | High; offloads arithmetic to deterministic downstream engines. | High; intermediate schema serves as a human- and machine-readable checkpoint. |
While a pure end-to-end LLM pipeline sounds attractive for rapid prototyping, it introduces severe production failure modes. Language models struggle with arithmetic, and attempting to resolve this with tool-calling or Model Context Protocol (MCP) services still requires defining agreed-upon data schemas for model-tool interaction. Furthermore, evaluating black-box LLM systems becomes nearly impossible as document complexity scales.
The optimal strategy separates the ingestion into an **AI structuring stage** that extracts raw input into a defined intermediate checkpoint, followed by targeted analysis. This provides key architectural advantages:
- Accommodates varied and shifting supplier document formats without breaking downstream logic.
- Creates an inspectable checkpoint that is both human-readable and machine-friendly.
- Seamlessly integrates with existing enterprise databases, analytical models, and compliance pipelines.
- Preserves regulatory and audit validity in restricted industries, restricting AI review to the ingestion layer.
---
[09:44](https://www.youtube.com/watch?v=9a1lhytoqpk&t=584s) The Art of Schema Selection and Sutton's "Bitter Lesson"
A central challenge when designing structurable data workflows is schema discovery: when handling heterogeneous inputs, how do engineers evaluate whether a chosen target schema is optimal? As a remix of statistician George Box’s famous aphorism ("All models are wrong, some are useful"), Berk emphasizes:
> "All data is structurable, not all structures are useful. Only some are."
Because the space of potential data schemas is infinite, there is no deterministic recipe or step-by-step formula for structuring data. However, because modern AI structuring is significantly faster and more flexible than legacy coding, engineers can leverage rapid A/B testing:
- Select a candidate intermediate schema based on engineering intuition and existing downstream requirements.
- Run AI-assisted extraction and evaluate information capture rates.
- Iteratively refine the representation using domain expertise to capture the most salient business variables.
```mermaid
flowchart TD
accTitle: The Bitter Lesson Timeline vs Business Value
subgraph LongTerm["Richard Sutton's Bitter Lesson"]
A["General Learning Systems + Compute"] -->|Eventually Outperforms| B["Human-Engineered Heuristics"]
end
subgraph NearTerm["Enterprise Pragmatism"]
C["Defined Checkpoint Architecture"] --> D["Immediate Business Value<br/>(1- to 5-Year Operational Window)"]
C --> E["Deterministic Verification & Auditing"]
end
```
Addressing AI researcher Richard Sutton’s influential 2019 essay *The Bitter Lesson*—which demonstrates that general learning methods leveraging compute consistently surpass human-engineered domain heuristics over long horizons—Berk notes that the essay provides no timeline for when general methods will conquer specific applied domains:
- Breakthroughs in pure end-to-end reasoning may take 1, 2, or 5 years for specialized enterprise tasks.
- In commercial operations, substantial business value and revenue are generated inside that transition window.
- Implementing structured checkpoints does not impose brittle hand-coded reasoning rules; it establishes a verifiable interface between AI structuring and downstream execution.
---
[13:56](https://www.youtube.com/watch?v=9a1lhytoqpk&t=836s) The Future of Structurable Data Architecture
The emerging architecture for enterprise AI workflows relies on dynamic, multi-modal ingestion: diverse upstream data sources are mapped into distinct, optimized structural representations and database engines. An intelligent retrieval and routing layer then selects the appropriate schema and analysis method based on the incoming query.
- **Research Inquiries:** Further reading on structurable data is published in a blog post by Meibel Principal Scientist Spencer Tin on `maybel.ai` (referenced during the Ai4 presentation).
- **Conference Presence:** Attendees were invited to visit Meibel at booth 342 in the expo hall for technical deep dives on structurable runtime platforms.
## Notable quotes
> Data is messy, and always has been. — Berk Ekmekci
> Your data doesn’t need to stay messy. — Berk Ekmekci
> [00:04](https://www.youtube.com/watch?v=9a1lhytoqpk&t=4s) confident AI — Berk Ekmekci
> [00:04](https://www.youtube.com/watch?v=9a1lhytoqpk&t=4s) Data 'messiness' is not merely an intrinsic property of the files or storage mediums; it is fundamentally a function of an organization's ability—or historical inability—to structure that information meaningfully. — Berk Ekmekci
> [02:33](https://www.youtube.com/watch?v=9a1lhytoqpk&t=153s) trying being the operative word — Berk Ekmekci
> [05:03](https://www.youtube.com/watch?v=9a1lhytoqpk&t=303s) While a pure end-to-end LLM pipeline sounds attractive for rapid prototyping, it introduces severe production failure modes. — Berk Ekmekci
> [09:44](https://www.youtube.com/watch?v=9a1lhytoqpk&t=584s) All models are wrong, some are useful — George Box
> [09:44](https://www.youtube.com/watch?v=9a1lhytoqpk&t=584s) All data is structurable, not all structures are useful. Only some are. — Berk Ekmekci
> [09:44](https://www.youtube.com/watch?v=9a1lhytoqpk&t=584s) Because the space of potential data schemas is infinite, there is no deterministic recipe or step-by-step formula for structuring data. — Berk Ekmekci
> [09:44](https://www.youtube.com/watch?v=9a1lhytoqpk&t=584s) Implementing structured checkpoints does not impose brittle hand-coded reasoning rules; it establishes a verifiable interface between AI structuring and downstream execution. — Berk Ekmekci
## Mentioned
Berk Ekmekci, Meibel, Ai4, Spencer Tin, George Box, Richard Sutton, The Bitter Lesson, SQL, Microsoft Excel, Microsoft Access, NoSQL, Model Context Protocol (MCP)
## Video description
Berk Ekmekci, Research Scientist, Meibel
Abstracts
Data is messy, and always has been. Structured and unstructured have been two bookends when thinking about data and how it can be used, but modern AI techniques present new opportunities in way of thinking about data: Structureable. In this talk, we’ll introduce the concept of Structurable Data, where to find it, effective ways to think about and work with it, and the unique value both the mindset and result can provide AI workflows small or large. Your data doesn’t need to stay messy.
Meibel主张采用结构化检查点解决企业LLM失效问题
纯端到端语言模型在算术和合规方面容易失效，要求企业仅将AI用于中间模式提取，而将分析工作交由确定性代码处理。
Berk Ekmekci介绍了“可结构化数据”（structurable data）的概念，将其定义为能够系统推导出机器可用结构化表示的非结构化或半结构化信息。他评估了传统ETL、纯端到端LLM链以及混合AI结构化流水线在制造、国防和医疗领域的实际用例，表明中间检查点能够优化算术精度、成本与可审计性。针对模式选择以及Richard Sutton提出的“苦涩的教训”（Bitter Lesson），本次演讲指出，在通用AI持续发展成熟的过程中，混合结构化接口能够提供不可或缺的近期商业价值与治理能力。
Meibel 提出“可结构化数据（Structurable Data）”概念，认为数据的“混乱”并非文件格式的固有属性，而是组织历史处理能力的体现，现代 AI 已将数据结构化推向连续光谱形态。 (0:04)
与易产生算术幻觉、高延迟和黑盒评估难题的端到端 LLM 流程相比，混合架构将 LLM 仅用于将原始输入提取至稳定的中间模式（Schema），将数学与分析逻辑交由确定性代码执行。 (5:03)
中间模式检查点构建了人机皆可读的审查边界，既能确保高合规要求行业的审计合规性，又能与现有企业级数据库和传统分析系统无缝集成。 (5:03)
针对 Schema 设计难题，演讲引用变体格言“所有数据皆可结构化，但并非所有结构皆有用”，主张利用 AI 降低转换成本的优势进行敏捷 A/B 测试与迭代优化。 (9:44)
Reading summary
Berk Ekmekci of Meibel argues that enterprise AI architectures should abandon the strict structured-versus-unstructured dichotomy in favor of 'structurable data'—using AI to transform disorganized inputs into verifiable intermediate schemas before executing downstream tasks.
The Concept of Structurable Data Across Industries (0:04)
Rather than treating data format as a rigid binary, Ekmekci defines structurable data as less structured or completely unstructured information from which machine-usable structures can be derived. Under this view, data messiness is not an unchangeable property of files, but reflects an organization's historical inability to structure its information meaningfully. This operational challenge cuts across sectors: manufacturing plants manage fragmented supplier workbooks, scanned spec sheets, and technical diagrams; clinical researchers must harmonize complex billing and diagnosis codes across disparate files; and defense logistics teams reconcile granular telemetry with high-level summaries.
Architectural Trade-Offs: ETL, Pure LLMs, and Hybrid Structuring (2:33)
Computing history moved from rigid SQL schemas and brittle manual ETL in the 1990s and 2000s to flexible multimodal extraction via GPUs, CNNs, and NoSQL databases in the 2010s. For modern workflows—such as analyzing manufacturing reports to calculate cascade costs and component hierarchies—Ekmekci contrasts three architectural choices: traditional deterministic ETL, end-to-end LLM chaining, and hybrid AI structuring checkpoints. Traditional ETL breaks easily when document layouts shift, while pure end-to-end LLM pipelines suffer from high token inference costs, slow latency, black-box auditability, and poor arithmetic precision.
Intermediate Schemas as Verifiable Checkpoints (5:03)
Meibel advocates for an AI structuring checkpoint architecture that separates raw data ingestion from analytical execution. In this hybrid model, AI handles the flexible extraction of shifting formats into an inspectable, human- and machine-readable intermediate schema. Downstream mathematical calculations and business logic are then offloaded to deterministic code, databases, and compliance pipelines. This boundary reduces token costs, avoids arithmetic errors inherent in language models, and preserves regulatory and audit validity in restricted industries by confining AI uncertainty to the ingestion layer.
Schema Discovery and Pragmatism Under the Bitter Lesson (9:44)
Because the space of potential target schemas is infinite, Ekmekci notes there is no fixed formula for structuring data; engineers must iteratively test candidate schemas using domain expertise to capture salient variables ('all data is structurable, not all structures are useful'). Addressing Richard Sutton's 2019 essay 'The Bitter Lesson'—which shows that general compute methods consistently defeat hand-engineered heuristics over the long run—Ekmekci contends the essay provides no timeline for enterprise adoption. Whether general reasoning requires one, two, or five years to master specialized domains, intermediate structured checkpoints capture immediate commercial value without enforcing brittle reasoning rules.
Looking forward, Meibel envisions enterprise architectures utilizing multi-modal ingestion to route diverse data sources into optimized schemas and databases, queried through an intelligent retrieval layer. However, determining the optimal intermediate representation remains an empirical challenge requiring domain intuition and rapid iterative testing.
精炼阅读摘要
Meibel 研究工程师 Berk Ekmekci 指出，生产环境中的企业数据并非简单的结构化与非结构化二元对立，而是存在“可结构化数据”光谱；企业应采用 AI 抽取中间检查点与下游确定性计算相结合的混合架构，以平衡系统的灵活性、算术准确性与合规审计要求。
可结构化数据概念与行业痛点 (0:04)
在传统认知中，数据常被简单划分为结构化和非结构化两大类，但 Ekmekci 提出了“可结构化数据”（Structurable Data）的新视角，将其定义为那些原始形式不规则或完全非结构化、但能够提炼出机器可用结构化表征的数据资产。Ekmekci 强调，数据的杂乱程度并非文件或存储介质本身的固有属性，而是取决于企业在特定历史阶段对其进行有效结构化处理的技术能力。
这一数据挑战广泛存在于对精准度要求极高的关键领域。例如，重工业与制造业需要处理零散的供应商 Excel 表格、扫描版技术规格书和工程图纸，关键工艺参数往往彼此割裂；临床医学研究与医疗审计面临跨文档且高度复杂的诊断与账单编码，需要统一清洗与对齐；国防物流和任务战备分析则需要将底层设备遥测数据与宏观战略报告相互关联。在这些场景中，单纯依赖人工录入或静态数据库难以满足实时智能任务的需求。
数据结构化演化与技术范式转变 (2:33)
回顾企业计算历史，数据结构化的技术手段经历了数次范式迁移。在 20 世纪 90 年代至 21 世纪初，关系型数据库与 SQL 占据主导地位，Excel 和 Microsoft Access 推动了表格普及，但这类技术对机器可用性有着严苛的预设 Schema 要求，高度依赖人工录入或极易断裂的脆弱 ETL 数据流水线。
到了 2010 年代，随着 GPU 计算加速与卷积神经网络（CNN）走向商业化落地，系统具备了对图像、音频和视频等非结构化多模态数据的灵活抽取能力；与此同时，NoSQL 数据库的兴起打破了严格 Schema 的限制，支持半结构化文档的直接入库与动态检索。演进至 2020 年代，结构化程度已变成一条连续的光谱，AI 可以在该光谱的任意位置接入并转换数据，推动了自动化流水线系统设计方法的重构。
纯 LLM 与混合架构的权衡对比 (5:03)
针对复杂的工程分析场景（例如解析多供应商报告中的子组件层级、成本级联和复杂度指标），Ekmekci 对比了传统 ETL、纯端到端大语言模型（LLM）串联以及混合 AI 结构化架构。传统 ETL 在输入版式变动时极易失效；而端到端 LLM 链条虽然原型搭建灵活，但在生产环境中存在推理延迟高、累积 Token 成本昂贵、黑盒系统难以量化评估等严重缺陷，且模型自身极易出现算术错误。即便引入工具调用或 Model Context Protocol（MCP），系统依然需要明确预设的数据 Schema 才能正常协作。
更为可行的工程方案是采用“AI 结构化检查点”（Hybrid AI Structuring Checkpoint）架构：首先利用 AI 将异构输入转化为可检查的标准化中间 Schema，再将算术逻辑交给下游确定性代码或数据库引擎执行。这种设计既保留了应对多样化输入格式的弹性，又利用可读的中间层满足了受监管行业对审计追溯与合规治理的刚性要求。
Schema 探索实践与苦涩教训启示 (9:44)
在设计目标 Schema 时，工程师面临着“可能的数据结构无限多”的现实难题。Ekmekci 改写了统计学家 George Box 的名言，提出“所有数据都是可结构化的，但并非所有结构都有用，只有部分有用”。由于没有确定性的固定公式来构建 Schema，团队应依托工程直觉和现有业务需求定义候选表示，利用 AI 抽取进行快速 A/B 测试，并通过捕获率指标与领域专业知识持续迭代。
针对 AI 学者 Richard Sutton 在 2019 年发表的《苦涩的教训》（The Bitter Lesson）——即长期来看通用学习加算力必然超越人类手工设计的启发式规则——Ekmekci 评论指出，该理论并未给出通用方法在特定应用领域完全成熟的时间表。在未来的 1 至 5 年商业过渡期内，业务价值仍需落地。采用结构化中间检查点并不是在模型内部硬编码脆弱的先验规则，而是在 AI 摄入与确定性执行之间建立一个可验证的工程接口，兼顾当前的商业实用价值与审计确定性。
未来企业 AI 的演进方向在于动态多模态摄入：将各类上游异构数据映射至多种针对性优化的存储引擎与结构表示中，再通过智能检索与路由层匹配分析逻辑。不过，如何在无限的 Schema 空间中持续实现自动评估，以及通用端到端模型在特定企业垂直任务中的落地节奏，仍是有待行业进一步探索的工程课题。
Detailed notes — zh
## Background
本内容由 Ai4 于 2025 年 8 月 27 日发布在 YouTube 平台上，演讲者为来自 Meibel 的研究科学家 Berk Ekmekci。视频简介指出，数据历来充满混乱，而在结构化与非结构化的传统二分法之外，现代 AI 技术催生了“可结构化数据”（Structurable Data）这一全新视角，演讲旨在探讨其定位、有效处理方式以及为 AI 工作流赋予的独特价值。
## TL;DR
Berk Ekmekci 阐述了将非结构化或低结构化数据提炼重构为“可结构化数据”的核心理念，回顾了从 SQL、NoSQL 到现代 AI 结构化全光谱的技术演进。针对传统刚性 ETL 扩展困难与端到端纯 LLM 成本高、计算弱、难以评估的痛点，他倡导引入“AI 结构化中间检查点”解耦架构，既兼顾了人类可读可验与机器友好性，又能直接复用成熟合规的下游分析资产。此外，他还探讨了基于 A/B 测试的结构迭代法与领域知识的重要性，并回应了理查德·萨顿的《苦涩的教训》，指出在通用模型彻底通晓一切前，中间检查点是当下确保高置信度商业落地与人机协同契约的关键。
## Atmosphere & style
整场演讲呈现出典型的前沿技术分享风格，务实、严谨且充满工程理性。演讲者语言条理清晰，以娓娓道来的叙事节奏梳理技术简史，并在剖析纯 LLM 方案与《苦涩的教训》等业界争议时展现出客观辩证的思辨态度。聆听体验如同置身一场高质量的工业界技术研讨会，既没有夸大其词的浮躁宣讲，也没有深奥晦涩的理论堆砌，而是紧扣实际落地痛点，给工程师与技术决策者带来兼具方法论与现实启发性的冷静思考。
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=9a1lhytoqpk&t=4s) 构建高置信度 AI（Confident AI）的核心在于深入数据现状，并将现有数据结构转换为下游所需的目标数据结构。
- [00:04](https://www.youtube.com/watch?v=9a1lhytoqpk&t=4s) Meibel 提出了“可结构化数据”（Structurable Data）概念，指那些处于较低结构化或非结构化状态，但能被提炼重构成高价值结构化表达的数据。
- [02:43](https://www.youtube.com/watch?v=9a1lhytoqpk&t=163s) 从 1990 年代刚性的 SQL/ETL，到 2010 年代 GPU 算力驱动的多模态 CNN 与 NoSQL，现代 AI 实现了贯穿全光谱的数据结构化与灵活摄入。
- [05:08](https://www.youtube.com/watch?v=9a1lhytoqpk&t=308s) 传统确定性 ETL 架构过于刚性且难以应对格式变更，而端到端纯 LLM 方案则面临高延迟、高成本、复杂数学计算薄弱及端到端评估困难等缺陷。
- [08:04](https://www.youtube.com/watch?v=9a1lhytoqpk&t=484s) 推荐的工程范式是在流程中引入“AI 结构化”作为中间检查点（Checkpoint），解耦输入多样性与下游业务逻辑。
- [08:04](https://www.youtube.com/watch?v=9a1lhytoqpk&t=484s) 中间目标结构兼具人类可读性、可验证性与机器友好性，能够最大化复用受监管行业中已验证且合规的现有分析系统与计算资产。
- [09:37](https://www.youtube.com/watch?v=9a1lhytoqpk&t=577s) “所有数据都是可结构化的，但并非所有结构都是有用的”，结构设计不存在固定配方，需要结合领域认知（Domain Expertise）通过 A/B 测试快速迭代。
- [11:49](https://www.youtube.com/watch?v=9a1lhytoqpk&t=709s) 针对萨顿《苦涩的教训》（The Bitter Lesson）的质疑，设立中间检查点并非限制通用算法演进，而是在通用模型完全赶超前的商业窗口期内建立人机协作验证契约。
- [13:56](https://www.youtube.com/watch?v=9a1lhytoqpk&t=836s) 未来 AI 分析范式将演进为智能网络架构：异构数据自动沉淀为最适合的结构形态，由上层智能检索与分析代理自主调度执行精准计算。
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=9a1lhytoqpk&t=4s) 什么是“可结构化数据”（Structurable Data）
Meibel 的研究科学家 Berk Ekmekci 提出了一个核心观点：在现代 AI 工作流中，构建“高置信度 AI”（Confident AI）的关键在于“深入数据现状，并将现有数据结构转换为所需的数据结构”（meeting your data where it is and transforming the data structures that you have into the data structures that you need）。随着 AI 结构化技术的演进，评估数据质量已不再局限于数据本身是否干净，同样重要的是明确掌握手头拥有什么结构、最终需要什么结构。
为此，Meibel 提出了一个全新的概念——**“可结构化数据”（Structurable Data）**。可结构化数据是指那些较低结构化程度或完全非结构化的数据，但人们可以从中提炼并推导出更加结构化的表达形式。数据的“混乱性”（messiness）不仅取决于数据本身的形态及其分散来源，很大程度上取决于人们是否有能力将其重构成有意义的形式。
这种思维方式在诸多垂直业务领域中展现出直接价值：
- **制造业与工业（Manufacturing and Industry）**：初始数据通常如同杂乱无章的“文档星系”（constellation of documents），混合了扫描件、图像以及某个供应商特有的 Excel 规格说明书。核心挑战在于如何将这些散落的信息整合进统一视图中。
- **临床研究与审计（Clinical Research and Auditing）**：面临的问题往往是数据结构过于专有或过度复杂，不同文档中使用的高度特异性编码可能指代同一事物，需要对其进行重构与统一化处理以供访问。
- **国防后勤与任务战备（Defense Logistics and Mission Readiness）**：数据跨越多个粒度层级，既需要保留极其精细的具体细节，又要能够随时汇总生成顶层宏观全局视图。
---
### [02:43](https://www.youtube.com/watch?v=9a1lhytoqpk&t=163s) 数据结构化的历史演进：从 SQL、NoSQL 到现代 AI
数据结构化与重构并非凭空出现的新事物，回顾过去几十年的技术演进有助于理清当下的技术坐标：
```mermaid
flowchart LR
accTitle: 数据结构化技术的历史演进
subgraph S1["1990s - 2000s: 关系型时代"]
direction TB
A1["SQL 与关系型数据库"] --> A2["Excel & Access 普及"]
A2 --> A3["刚性 ETL / 高机器可用门槛"]
end
subgraph S2["2010s: 模态与灵活度突破"]
direction TB
B1["GPU 算力突破 / CNN 兴起<br>(图像/音视频多模态)"]
B2["NoSQL 数据库流行<br>(灵活 Schema / 快速上手)"]
end
subgraph S3["现代: AI 结构化全光谱"]
direction TB
C1["全光谱数据结构化"]
C2["多切入点摄入与分析"]
end
S1 --> S2 --> S3
```
- **1990 年代至 2000 年代**：关系型数据库与表格数据占据统治地位，SQL 大行其道，Microsoft Excel 成为家喻户晓的工具，Microsoft Access 等工具试图架起连接二者的桥梁。然而，该时期的“机器可用性门槛”（threshold for machine usability）非常高，数据要么依赖人工录入与整理，要么依赖极其刚性的软件 ETL 管道。
- **2010 年代**：发生了两大关键技术变革。其一是 GPU 算力的飞跃，使得数十年前就已提出概念的卷积神经网络（CNN）在训练和推理上变得经济可行，解锁了对图像、视频、音频等全新数据模态的处理能力；其二是 NoSQL 数据库的普及，降低了机器可用门槛，允许开发者在无需严格预设 Schema 的情况下快速汇聚、分析与检索多源文档。
- **现代 AI 时代**：可结构化性（Structurability）贯穿了从完全非结构化到高度结构化的“全光谱”，工程师可以在光谱的任意节点作为起点或终点进行数据摄取、转换与分析。
---
### [05:08](https://www.youtube.com/watch?v=9a1lhytoqpk&t=308s) 传统刚性 ETL 与端到端纯 LLM 方案的局限
在实际工程场景中（例如制造业的技术报告与零件明细表分析，需要回答“哪个组件包含最多子组件”或进行制造复杂度/成本穿透汇总），开发者通常会面临两种极端方案的抉择：
1. **传统确定性 ETL + 算法分析管道**：
- **优势**：具备完全的确定性（Deterministic），执行路径稳定可重复，测试与验证相对简单。
- **劣势**：架构高度刚性。一旦上游供应商调整文档格式，或者业务方提出了全新的分析指标，整个 ETL 规则、中间结构及下游分析脚本都需要重写或大改。
2. **端到端纯大语言模型（End-to-End LLM / LLM Chain）**：
- **优势**：初始搭建和原型迭代极具吸引力，开发极度灵活。
- **核心缺陷**：
- **速度与成本瓶颈**：全流程串联多个 LLM 调用不仅会导致延迟大幅增加，推演成本也十分高昂。
- **复杂计算能力薄弱**：LLM 本身并不擅长数学运算。尽管可以通过工具调用（Tool Calling）和 MCP 服务（Model Context Protocol）缓解，但要稳定使用这些服务，仍然需要前后端遵循明确约定的数据结构与交互接口。
- **端到端评估极度困难**：随着业务问题复杂度提升，仅凭输入和最终输出很难精准判定模型在哪个环节表现良好或失效。
---
### [08:04](https://www.youtube.com/watch?v=9a1lhytoqpk&t=484s) 破局方案：AI 结构化检查点与现有系统复用
为了兼顾灵活性与确定性，推荐的工程范式是将流程解耦，引入“AI 结构化”作为中间的关键检查点（Checkpoint）：
```mermaid
flowchart LR
accTitle: 基于 AI 结构化检查点的混合工作流架构
subgraph In["输入层"]
DocA["多样化技术文档 A"]
DocB["变体文档 / 新供应商 B"]
end
subgraph Process["AI 结构化与检查点"]
AIStruct["AI 结构化步骤<br>(灵活适应文档差异)"]
Checkpoint["【定义的目标中间结构】<br>可读 / 可验证 / 机器友好"]
end
subgraph Out["下游消费与分析"]
AIAnalysis["AI 分析工作流"]
Legacy["现有传统分析管道<br>(已合规/已验证资产)"]
end
DocA --> AIStruct
DocB --> AIStruct
AIStruct --> Checkpoint
Checkpoint --> AIAnalysis
Checkpoint --> Legacy
```
- **定义目标表示（Target Representation）**：在流水线最前端利用灵活的 AI 结构化技术，将多来源、多版式的文档统一解析并沉淀为你主动定义的标准数据结构中。
- **双重优势检查点**：这个中间结构既具备良好的“人类可读性与可验证性”（human readable and verifiable），又保持了高度的“机器友好性”（machine friendly），为系统调试和评估提供了可靠基准。
- **最大化复用现有系统资产**：业务中原有的成熟分析管道本身就是针对特定数据结构构建的。将数据转入该结构后，现有的计算引擎、报表系统乃至在受监管行业（Regulated Industries）中已经通过合规审计与验证的下游系统均可无缝复用，大幅节省工程重构成本并确保输出置信度。
### [09:37](https://www.youtube.com/watch?v=9a1lhytoqpk&t=577s) 可结构化数据的评估与“所有模型皆有错，唯部分有用”
当数据处理流程跑通后，接下来的核心任务是核查 AI 的结构化过程，确保它确实满足了业务的所有检查项。然而，当输入不再是单一类型的文档，而是面对多种不同形态的输入文档时，一系列复杂的问题便随之浮现：我们最初为技术文档选定的目标结构，是否依然是统揽所有文档的最佳范式？如果我们从另一个起点切入，是否会得到更优质的全局结构？我们究竟该如何衡量一个结构的优劣？
这是一个需要辩证看待的深刻问题。原因在于——“尽管所有数据都是可结构化的（all data is structurable），但并非所有结构都是有用的（not all structures are useful）”。演讲者借用了统计学家乔治·博克斯（George Box）的传世名言“所有模型都是错的，但有些是有用的（All models are wrong, some are useful）”来进行类比。在设计结构时，工程师拥有无限的选择空间，这既令人振奋，又因为“无限”本身的浩瀚而令人望而生畏。在构建结构化数据流水线的目标选型中，存在着坏消息与好消息：
- **坏消息**：结构化数据并不存在固定的“万能配方”（no recipe），无法给出一套刻板的逐步操作指南来机械选定理想的中间检查点。
- **好消息**：基于 AI 的结构化流程比传统方法更敏捷、更具弹性。团队可以凭直觉或依托现有系统先拟定一个候选结构，再通过快速迭代与 A/B 测试（A/B test）来评估优劣：对比信息捕获的广度与深度是否有所提升。
```mermaid
flowchart LR
accTitle: 结构化数据迭代与评估流水线
subgraph Input["多样化输入"]
D1["技术文档"]
D2["多源异构文档"]
end
subgraph Structuring["AI 弹性结构化"]
C["初始候选结构\n(基于领域知识/经验)"]
AB{"A/B 测试与评估\n(信息捕获/业务表现)"}
end
subgraph Output["落地应用"]
U["高价值结构化数据"]
end
D1 --> C
D2 --> C
C --> AB
AB -- "持续优化迭代" --> C
AB -- "验证有效" --> U
```
- **领域认知（Domain Expertise）的关键作用**：至少在现阶段，深刻理解业务问题本身以及明确哪些信息对下游分析最具备价值，仍然是驾驭可结构化数据工作流最至关重要的核心竞争力。
---
### [11:49](https://www.youtube.com/watch?v=9a1lhytoqpk&t=709s) 架构中间检查点与“苦涩的教训”（The Bitter Lesson）
面对引入人工设计中间结构的做法，业内常会提出质疑：这是否违背了著名 AI 学者理查德·萨顿（Richard Sutton）在 2019 年发表的经典文章《苦涩的教训》（The Bitter Lesson）？该理论回顾了数十年 AI 研究史，指出试图将人类对问题的先验理解与求解方法硬编码进系统的做法，最终无一例外都会被纯粹依赖通用算力与学习框架、自主摸索解题路径的通用系统所击败。
然而，在商业与工程实践中设立“中间结构检查点”有着坚实的现实理由：
1. **时间窗口与商业价值**：通用方法超越特定方法需要时间，这个周期可能是 1 年、2 年甚至 5 年。在通用模型彻底赶超之前的这段窗口期内，利用结构化流水线所创造的业务收益与用户价值是极其巨大且真实的。
2. **界定契约而非限制算法**：制定中间结构并非在强行规定模型“具体如何去解析”或“具体如何进行下游推理”，而是在人类与 AI 系统之间确立一个达成共识的“检查点”（agreed upon checkpoint），以便双方协同验证执行结果的有效性。
---
### [13:56](https://www.youtube.com/watch?v=9a1lhytoqpk&t=836s) 可结构化数据驱动的未来 AI 分析范式
展望未来，面向可结构化数据的 AI 分析架构将演进为高度协同的智能网络：多源异构数据将自动映射并沉淀到最适合其形态的各种可结构化形态与数据库中；上层的智能检索分析代理（Intelligent Retrieval & Analysis）在面对具体业务问题时，能够自主调度，前往正确的结构形态与存储位置执行精准分析。
- **关于 Meibel 的探索与资源**：Meibel 团队正致力于打造这一新范式的落地基础设施。
- 关注可结构化数据前沿探索的参会者可访问 `meibel.ai`（或活动专属链接）阅读 Meibel 首席科学家 Spencer Tin 撰写的专题深度博文。
- 演讲者 Berk Ekmekci 邀请现场观众前往展厅 342 号 Meibel 展位或在演讲结束后直接交流探讨。
## Notable quotes
> [00:04](https://www.youtube.com/watch?v=9a1lhytoqpk&t=4s) 深入数据现状，并将现有数据结构转换为所需的数据结构 — Berk Ekmekci
> [00:04](https://www.youtube.com/watch?v=9a1lhytoqpk&t=4s) meeting your data where it is and transforming the data structures that you have into the data structures that you need — Berk Ekmekci
> [00:04](https://www.youtube.com/watch?v=9a1lhytoqpk&t=4s) 评估数据质量已不再局限于数据本身是否干净，同样重要的是明确掌握手头拥有什么结构、最终需要什么结构 — Berk Ekmekci
> [00:04](https://www.youtube.com/watch?v=9a1lhytoqpk&t=4s) 可结构化数据是指那些较低结构化程度或完全非结构化的数据，但人们可以从中提炼并推导出更加结构化的表达形式 — Berk Ekmekci
> [00:04](https://www.youtube.com/watch?v=9a1lhytoqpk&t=4s) 数据的“混乱性”不仅取决于数据本身的形态及其分散来源，很大程度上取决于人们是否有能力将其重构成有意义的形式 — Berk Ekmekci
> [02:43](https://www.youtube.com/watch?v=9a1lhytoqpk&t=163s) 可结构化性贯穿了从完全非结构化到高度结构化的“全光谱”，工程师可以在光谱的任意节点作为起点或终点进行数据摄取、转换与分析 — Berk Ekmekci
> [08:04](https://www.youtube.com/watch?v=9a1lhytoqpk&t=484s) 这个中间结构既具备良好的“人类可读性与可验证性”，又保持了高度的“机器友好性”，为系统调试和评估提供了可靠基准 — Berk Ekmekci
> [09:37](https://www.youtube.com/watch?v=9a1lhytoqpk&t=577s) 尽管所有数据都是可结构化的，但并非所有结构都是有用的 — Berk Ekmekci
> [09:37](https://www.youtube.com/watch?v=9a1lhytoqpk&t=577s) All models are wrong, some are useful — George Box
> [09:37](https://www.youtube.com/watch?v=9a1lhytoqpk&t=577s) 结构化数据并不存在固定的“万能配方”，无法给出一套刻板的逐步操作指南来机械选定理想的中间检查点 — Berk Ekmekci
> [09:37](https://www.youtube.com/watch?v=9a1lhytoqpk&t=577s) 深刻理解业务问题本身以及明确哪些信息对下游分析最具备价值，仍然是驾驭可结构化数据工作流最至关重要的核心竞争力 — Berk Ekmekci
> [11:49](https://www.youtube.com/watch?v=9a1lhytoqpk&t=709s) 制定中间结构并非在强行规定模型“具体如何去解析”或“具体如何进行下游推理”，而是在人类与 AI 系统之间确立一个达成共识的“检查点”，以便双方协同验证执行结果的有效性 — Berk Ekmekci
## Mentioned
Meibel, Berk Ekmekci, Spencer Tin, Richard Sutton, George Box, 《苦涩的教训》（The Bitter Lesson）, SQL, NoSQL, Microsoft Excel, Microsoft Access, CNN (卷积神经网络), Model Context Protocol (MCP)
## Video description
Berk Ekmekci, Research Scientist, Meibel
Abstracts
Data is messy, and always has been. Structured and unstructured have been two bookends when thinking about data and how it can be used, but modern AI techniques present new opportunities in way of thinking about data: Structureable. In this talk, we’ll introduce the concept of Structurable Data, where to find it, effective ways to think about and work with it, and the unique value both the mindset and result can provide AI workflows small or large. Your data doesn’t need to stay messy.
[Source](https://www.youtube.com/watch?v=9a1lhytoqpk)
Source quotes
[0:04] Berk Ekmekci: 深入数据现状，并将现有数据结构转换为所需的数据结构
[0:04] Berk Ekmekci: meeting your data where it is and transforming the data structures that you have into the data structures that you need
[0:04] Berk Ekmekci: 评估数据质量已不再局限于数据本身是否干净，同样重要的是明确掌握手头拥有什么结构、最终需要什么结构
[0:04] Berk Ekmekci: 可结构化数据是指那些较低结构化程度或完全非结构化的数据，但人们可以从中提炼并推导出更加结构化的表达形式
[0:04] Berk Ekmekci: 数据的“混乱性”不仅取决于数据本身的形态及其分散来源，很大程度上取决于人们是否有能力将其重构成有意义的形式
[2:43] Berk Ekmekci: 可结构化性贯穿了从完全非结构化到高度结构化的“全光谱”，工程师可以在光谱的任意节点作为起点或终点进行数据摄取、转换与分析
[8:04] Berk Ekmekci: 这个中间结构既具备良好的“人类可读性与可验证性”，又保持了高度的“机器友好性”，为系统调试和评估提供了可靠基准
[9:37] Berk Ekmekci: 尽管所有数据都是可结构化的，但并非所有结构都是有用的
[9:37] George Box: All models are wrong, some are useful
[9:37] Berk Ekmekci: 结构化数据并不存在固定的“万能配方”，无法给出一套刻板的逐步操作指南来机械选定理想的中间检查点
[9:37] Berk Ekmekci: 深刻理解业务问题本身以及明确哪些信息对下游分析最具备价值，仍然是驾驭可结构化数据工作流最至关重要的核心竞争力
[11:49] Berk Ekmekci: 制定中间结构并非在强行规定模型“具体如何去解析”或“具体如何进行下游推理”，而是在人类与 AI 系统之间确立一个达成共识的“检查点”，以便双方协同验证执行结果的有效性
Source quotes
Berk Ekmekci: Data is messy, and always has been.
Berk Ekmekci: Your data doesn’t need to stay messy.
[0:04] Berk Ekmekci: confident AI
[0:04] Berk Ekmekci: Data 'messiness' is not merely an intrinsic property of the files or storage mediums; it is fundamentally a function of an organization's ability—or historical inability—to structure that information meaningfully.
[2:33] Berk Ekmekci: trying being the operative word
[5:03] Berk Ekmekci: While a pure end-to-end LLM pipeline sounds attractive for rapid prototyping, it introduces severe production failure modes.
[9:44] George Box: All models are wrong, some are useful
[9:44] Berk Ekmekci: All data is structurable, not all structures are useful. Only some are.
[9:44] Berk Ekmekci: Because the space of potential data schemas is infinite, there is no deterministic recipe or step-by-step formula for structuring data.
[9:44] Berk Ekmekci: Implementing structured checkpoints does not impose brittle hand-coded reasoning rules; it establishes a verifiable interface between AI structuring and downstream execution.
Original transcript
[0:04] Uh good morning everyone. My name is
[0:05] Burke. I'm a research engineer at Maybel
[0:08] and we're building the runtime platform
[0:10] for confident AI. An important part of
[0:13] that is meeting your data where it is
[0:15] and transforming the data structures
[0:17] that you have into the data structures
[0:20] that you need. And that with the modern
[0:24] advances in AI structuring techniques
[0:26] has become something that is just as
[0:28] important the quality of the data that
[0:29] you have as knowing what structures you
[0:32] have and what structures you want to get
[0:34] to. And so we've come up with a new term
[0:37] to embody that quality. We call it
[0:39] structurable data.
[0:41] Structurable data is less structured or
[0:44] unstructured data from which a more
[0:46] structured representation can be
[0:48] derived. And the value of this sort of
[0:51] way of thinking is something that spans
[0:53] across multiple domains.
[0:56] Mabel works in several of these domains,
[0:58] but in the manufacturing and industry
[1:00] industries, you might have your starting
[1:03] data be a constellation of very many
[1:05] different document types coming from
[1:07] different places and they might be very
[1:09] disorganized. You might have scans of
[1:11] documents of images. You might have a
[1:14] product specification Excel worksheet.
[1:16] that is the only place where certain
[1:18] information from a supplier comes in.
[1:20] And the challenge is in bringing all of
[1:22] that stuff into a unified view.
[1:26] In the clinical research and auditing
[1:28] space, the problem is a little bit
[1:29] different. You might have data
[1:31] structures that are a bit too specific
[1:33] or too complex. Um, highly specific
[1:36] codes that might mean the same thing in
[1:38] multiple documents. Your job might be to
[1:40] structure it and restructure it in such
[1:42] a way that the underlying information
[1:44] becomes unified and accessible. In
[1:48] defense logistics and mission readiness,
[1:49] there's a similar analog, but one that
[1:51] spans multiple levels of granularity.
[1:54] Information that is highly specific
[1:56] needs to be available alongside
[1:57] information that is giving a tople view.
[2:00] So if you look at just that center
[2:02] column, the idea of all of these
[2:04] different input structures, it's easy to
[2:06] think of that as being overwhelming, of
[2:08] being messy inherently.
[2:11] But messiness of your data is something
[2:13] that is not only a function of what that
[2:16] data is and the variety of places you
[2:18] get it from and store it, but also your
[2:21] ability or in some cases inability to
[2:24] structure it in a way that's meaningful.
[2:26] And that's what we're going to talk
[2:27] about is taking the information you know
[2:29] is there, the value you know is there
[2:31] and bringing it all together. And while
[2:33] modern AI structuring techniques are
[2:35] very powerful, they do build on a
[2:37] foundation and history of existing
[2:40] techniques. Structuring your data,
[2:41] restructuring it, it's nothing new. And
[2:43] it helps to set the stage by taking a
[2:45] brief view to the past. So in the 1990s
[2:48] and 2000s, it was really the world of
[2:51] relational databases and tabular data.
[2:54] SQL was gaining a lot of popularity.
[2:56] Microsoft Excel was becoming a household
[2:58] name and Microsoft Access and similar
[3:00] tools were trying to sort of span the
[3:03] world between those tools. Trying being
[3:04] the operative word but with the tooling
[3:07] beginning at that s such level of
[3:09] structure for it to be useful the
[3:11] threshold for machine usability was
[3:13] quite high. And similarly in order to
[3:16] get your data into that level of
[3:18] structure the options were relatively
[3:20] limited. It had to be either manually
[3:22] structured and restructured or you had
[3:24] rigid uh data pipelines and software
[3:27] methods to bring a certain input into a
[3:30] certain output. Some of that changed in
[3:32] the 2010s in two major ways. One,
[3:36] advances in GPU technology meant that
[3:38] convolutional neural nets which were
[3:41] decades old in concept could now be much
[3:43] more feasible for training and inference
[3:45] for a lot more people in businesses. And
[3:47] what this meant is that new modalities
[3:49] of data, modalities of image, of video,
[3:52] of audio that were previously extracted
[3:55] in different ways could now be could now
[3:58] be done much more flexibly and with a
[4:00] lot more variety in the types of outputs
[4:02] that we could process. The second of
[4:04] these was the popularization of NoSQL
[4:07] databases. Uh this reduced the threshold
[4:10] of machine usability in such a way that
[4:12] you could take different documents that
[4:14] didn't necessarily have the same schema
[4:15] or maybe you didn't even know the fields
[4:17] of entirely, place them into a
[4:19] collection and start analyzing,
[4:21] retrieving and working with it. That
[4:23] sort of uh free form approach had its
[4:25] issues then and and still does, but it
[4:28] did give a quicker start to things. And
[4:30] we're seeing that much more accelerated
[4:32] in the modern space with modern AI
[4:35] systems. Structurability runs this full
[4:37] spectrum of structure from unstructured
[4:40] all the way to highly structured and you
[4:41] can begin and end at any point here.
[4:44] Furthermore, machine usability in terms
[4:46] of the systems that do analysis inest
[4:49] retrieval, they can begin at multiple
[4:51] points along the spectrum as well. So
[4:53] the key question as a practitioner then
[4:55] is how do I go about doing this? I've
[4:58] got so many possibilities, so many
[4:59] options, so much flexibility. What are
[5:02] the guiding principles I should use? and
[5:03] we'll talk through that in the form of
[5:05] an example. So here we have a document
[5:08] processing example where we have some
[5:10] technical report documents. Imagine a
[5:12] manufacturing case where uh there's some
[5:14] technical tables maybe descriptions and
[5:17] we have several of these and we want to
[5:19] get to certain answers about our
[5:20] manufacturing process. what's the
[5:22] component with the most subcomponents or
[5:25] in terms of manufacturing complexity or
[5:27] cost? If I cascade that down, can I roll
[5:29] that up in some kind of report?
[5:32] In a traditional approach to this, there
[5:34] would be a specific ETL process that
[5:37] would be configured to load this
[5:38] specific document type into a structured
[5:41] representation and then an algorithmic
[5:43] analysis that takes that structured
[5:45] representation and answers specific
[5:47] questions. Now with the rigidity of
[5:49] these algorithmic analyses, each
[5:51] question would need its own pipeline for
[5:53] analysis. You can think of that as being
[5:55] as simple as a SQL query, but it can get
[5:57] much more complicated based on the type
[5:59] of uh analysis that you're doing. And
[6:01] the advantage here is this is a
[6:03] deterministic system. It runs the same
[6:05] way each time. Once you have it working,
[6:08] you can verify that it's working pretty
[6:10] easily, but it's also at the same time
[6:12] quite rigid. If you have any changes or
[6:14] differences in your input documents, you
[6:16] need to adjust your ETL process. You
[6:18] need to ask yourself the question, is
[6:20] that rigid structure I have in the
[6:22] middle still relevant? Do I need to
[6:24] change that? If you want to add a new
[6:26] question to be answered, not only do you
[6:27] need to change the analysis, but ask
[6:29] yourself the question of does my
[6:31] structure and ETL process support this?
[6:34] So, given we have such powerful LMS,
[6:37] can't we just replace this whole thing
[6:38] end to end with an LLM call or a chain
[6:41] of LLM M calls? And from the perspective
[6:44] of flexibility and ease of setup getting
[6:47] working with this and iterating for the
[6:48] first time, this is very attractive. But
[6:51] it has a handful of uh major drawbacks
[6:54] that make it infeasible or less
[6:55] feasible. The first of which is speed
[6:57] and cost. These are talked about quite
[6:59] often, but um chaining your entire
[7:02] process as a sequence of LOM calls is
[7:04] going to be rather expensive, relatively
[7:06] speaking at least. And in terms of
[7:08] runtime speed, you're going to see that
[7:11] it's going to take quite a long time.
[7:12] But it goes beyond this. LMS alone are
[7:15] quite bad at math. And while things like
[7:18] tool calling and MCP services do help
[7:21] with this problem, if you want to make
[7:23] the most out of those, if you want to
[7:25] actually use them in your processes, you
[7:27] need an agreed upon structure of data
[7:30] and interface with which your LM
[7:32] communicates with these services. And
[7:35] last, it's quite hard to evaluate
[7:37] end-to-end processes in terms of inputs
[7:39] and outputs when those inputs and
[7:41] outputs get somewhat complex. Um, you
[7:43] can imagine that as your input
[7:45] complexity, not just in terms of the
[7:46] data that you're working with, but the
[7:48] problems you're looking to solve become
[7:50] more varied, more complicated, more
[7:52] nuanced, simply looking at your inputs
[7:54] and outputs and asking, did this do a
[7:55] good job? Did it not? Becomes more
[7:57] difficult. And while this is an active
[7:59] area of research for Maybel, it's a
[8:01] difficult task. So what might we do
[8:04] instead? Well, we begin with these same
[8:06] types of documents, but we do an AI
[8:09] structuring step at the very beginning
[8:11] to get to a known defined structure.
[8:13] This is a structure that you define.
[8:15] What is your target representation of
[8:18] the data? You can use flexible AI
[8:20] structuring techniques to do this. And
[8:22] as a result, if you have a different
[8:23] form of document, maybe a slightly
[8:25] different uh variety of that document,
[8:28] maybe a completely different
[8:29] manufacturer provides you a document of
[8:32] the same information in a different way,
[8:34] you can work with that without changing
[8:36] too much of how you treat those input
[8:38] documents. And then from there, you can
[8:40] use AI analysis to get to the answers to
[8:43] your questions. Once again, quite
[8:44] flexibly. But that intermediate
[8:46] structure, it serves as a checkpoint for
[8:48] you. a checkpoint that is not just
[8:50] relatively human readable,
[8:53] understandable, verifiable, but also
[8:54] very machine friendly. And that dual
[8:56] advantage is quite critical. It's why it
[8:59] makes it a good checkpoint. Another
[9:01] really big benefit to splitting out your
[9:03] way of working with structurable data in
[9:05] this way is that you gain access to a
[9:07] lot of the value of your existing
[9:08] systems. All of your existing analysis
[9:11] pipelines, they're built to work with a
[9:13] certain structure of data. So if you
[9:15] choose to move into that structure
[9:17] perhaps in addition to some other
[9:19] structures, you get all the value of
[9:20] that going forward. And that's not just
[9:22] business value or value in terms of uh
[9:24] engineering time saved to not have to
[9:26] rebuild a certain process. It also means
[9:28] that if you're in a regulated or
[9:30] restricted industry, you can have
[9:32] confidence that everything that has been
[9:34] approved, vetted, verified downstream of
[9:37] that is working. And now you just need
[9:38] to check up on your AI structuring and
[9:40] making sure that's checking the boxes
[9:42] that you needed to. So what happens if
[9:45] we then have not just one type of input
[9:48] document, but rather several input
[9:50] documents that we're structuring? Is
[9:52] that structure that we chose for our
[9:54] original technical documents the best
[9:56] representative structure to be targeting
[9:58] for all of these? Is it perhaps that if
[10:00] we had chosen a different starting
[10:02] point, we might have gotten a better
[10:03] overall structure? How do we evaluate
[10:05] the quality of a structure overall? And
[10:08] this is a great question and one with a
[10:10] nuanced answer. That's because while all
[10:13] data is structurable,
[10:15] not all structures are useful. Only some
[10:17] are. And this is my take or remix on
[10:20] George the statistician George Box's
[10:22] famous data proverb. All models are
[10:24] wrong, some are useful. And this is
[10:27] because you have infinite choice in
[10:29] terms of the structures you can choose.
[10:30] This is sort of encouraging and a little
[10:32] bit daunting because infinity is a very
[10:35] daunting concept to think about. Uh but
[10:37] you have some good intuition. There's
[10:38] some good news and bad news in this art
[10:41] that you need to develop in terms of
[10:42] choosing the data structures to target
[10:44] in your structural data pipelines. So
[10:47] the good news and bad news, we'll start
[10:48] with the bad news. Uh there is no recipe
[10:52] for structuring data. I can't give you a
[10:53] step by step for choosing those ideal
[10:56] intermediate checkpoints along the way.
[10:58] Uh there is good news though.
[11:00] AI powered structuring is faster and
[11:03] more flexible than existing methods or
[11:05] methods beforehand. And that means you
[11:07] can choose a candidate. Go with your gut
[11:09] on what that first structure should be
[11:11] or might be. Maybe it's an existing
[11:12] structure that use that you're using
[11:14] existing systems and then iterate from
[11:16] there. AB test. Is this better worse? Am
[11:19] I getting similar results? Am I able to
[11:21] capture more types of information from
[11:23] my structurable data than I was before?
[11:26] And this process is one that draws a lot
[11:28] on your domain knowledge and expertise.
[11:30] And for the time being at least, domain
[11:33] understanding what you know about your
[11:35] problem and what you believe is valuable
[11:37] to capture to present to your structured
[11:40] analysis techniques. That's very very
[11:42] important. Perhaps the most important
[11:44] component of working with structural
[11:46] data and structural workflows.
[11:49] So at this point you might be thinking
[11:50] to yourself, well Burke, aren't you
[11:52] forgetting the bitter lesson? And it
[11:55] seems like any AI talk in 2025 is
[11:57] incomplete without a reference to the
[11:59] bitter lesson. For those unfamiliar, it
[12:01] was a 2019 essay by AI rich uh AI
[12:04] researcher Richard Sutton. And I've got
[12:07] a QR code here for the link to the
[12:09] essay. Uh but to summarize, it's the
[12:11] idea that when you have uh when you take
[12:15] a look, a historical look back to AI
[12:17] research over the past several decades,
[12:20] specific methods that try to encode an
[12:22] understanding of how a human might go
[12:25] about solving a problem or methods or
[12:27] techniques that a human might have to
[12:29] approaching a problem. Eventually always
[12:32] get out competed and outperformed by
[12:34] general systems that give a learning
[12:37] system. the framework in which to
[12:40] evaluate its performance and allow it to
[12:42] freely learn its way of solving the
[12:44] problem. So in the long run, generalized
[12:46] systems that are sort of unguided in how
[12:48] they go about the task simply evaluate
[12:50] on their ability to perform the task win
[12:52] out in performance. So why are we
[12:54] talking about these checkpoints? Well,
[12:56] the key thing there is that that lesson
[13:00] doesn't give a specific timeline on
[13:02] which it's true. So it might be for a
[13:04] given problem set two years, 5 years,
[13:07] maybe just one year in between the time
[13:09] where you have a certain specific method
[13:11] and when the generalized method catches
[13:13] up and outperforms it. And in terms of
[13:15] business,
[13:17] you can get a lot of value out of your
[13:19] data pipelines in those one years, two
[13:22] years. That can create a lot of very
[13:23] real benefit to you, your customers,
[13:25] your users, the people that you serve.
[13:28] And so in that case, it's important to
[13:29] think about how you go about working
[13:31] with this. And sometimes checkpointing
[13:33] is a good thing to do. And another piece
[13:36] that I want to pay attention to is that
[13:38] when you're choosing that intermediate
[13:40] structure for your AI workflows, you're
[13:42] not saying how to structure that point
[13:44] or how to analyze from that structure.
[13:46] You're giving an agreed upon checkpoint
[13:48] that you and your AI systems can use
[13:51] together and verify that what you're
[13:52] doing is working well. So what does the
[13:56] near future of AI analysis look like for
[13:58] structurable data? Well, it looks like
[14:00] having multiple different data sources
[14:02] that come in different structures that
[14:04] are then placed into different
[14:06] structurable forms, different databases
[14:08] that they exist in and intelligent
[14:10] retrieval analysis that for the problem
[14:12] at hand goes to the right structure in
[14:14] the right place and performs the right
[14:16] analysis. And this is what we're
[14:17] building at Maybel. If you're interested
[14:20] in reading more about structuring and
[14:22] structurable data, um you can check out
[14:24] our blog post at maybel.ai4.
[14:26] Follow that QR code and you'll see a
[14:29] recent blog post by our principal
[14:31] scientist Spencer Tin about this very
[14:33] topic. If you have any questions, please
[14:35] feel free to come by the Maybel booth in
[14:37] the expo hall at 342 is the number of
[14:40] the booth. Uh or follow up with
[14:42] questions with me after the talk. Uh
[14:44] thank you very much and I hope you have
[14:45] a good rest of your conference.
