Demanding Formal Explanations Cuts Multimodal AI Accuracy, Study Finds

Panel 7 - 2: Explaining is Harder Than Predicting: Evaluating Multimodal LLMs in ICL — Ai4

Ai4

Forcing multimodal models to produce machine-verifiable, structured reasoning degrades visual classification accuracy from 93.8% to 90.1%, exposing an operational trade-off for enterprise deployments.

AI-assisted summary of Panel 7 - 2: Explaining is Harder Than Predicting: Evaluating Multimodal LLMs in ICL — original source. Check the original for full context.

Source published: 2026-08-14

Detailed notes

## Background Published on YouTube by the Ai4 channel on August 14, 2026, this presentation features Leticia Lorena Rodriguez from Universidad de Buenos Aires and TEG AI Consulting. According to the page description, the talk examines why forcing Multimodal Large Language Models (MLLMs) to output formally structured, concept-based explanations in in-context learning unexpectedly degrades their predictive accuracy, highlighting formal explainability as a major hurdle for enterprise-ready autonomous agents.

## TL;DR Drawing from the foundational pillars of trustworthy AI, Rodriguez introduces an evaluation pipeline that tests multimodal models across five distinct explanation formats, ranging from unprompted classification to machine-verifiable Description Logic. Using image benchmarks alongside an LLM-as-a-Judge protocol validated by human auditors, the research demonstrates a significant explainability penalty: direct prediction achieved 93.8% accuracy, while demanding formal logical explanations caused performance to drop to 90.1%. To make autonomous agents viable for critical real-world deployments, she argues that industry practitioners must bridge this gap through specialized supervised fine-tuning and rigorous independent auditing.

## Atmosphere & style The presentation carries a pragmatic, academically rigorous, and authoritative tone, informed by Rodriguez's decades of experience straddling university research and big tech engineering. It feels like an incisive conference briefing that directly challenges prevailing industry optimism surrounding Chain-of-Thought prompting. The narrative moves deliberately from ethical principles to empirical reality, delivering a sobering, data-backed warning about the hidden costs of demanding transparency from current generative models.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=tYxeBijkeRs&t=4s) Leticia Lorena Rodriguez introduces her research on AI explainability, highlighting responsible AI's five fundamental pillars: fairness, accountability, safety, transparency, and privacy. - [00:04](https://www.youtube.com/watch?v=tYxeBijkeRs&t=4s) Formal transparency and explainability are critical bottlenecks in preventing unpredictable model behavior and hallucinations in enterprise AI agent deployments. - [03:18](https://www.youtube.com/watch?v=tYxeBijkeRs&t=198s) Standard Chain-of-Thought prompting exhibits limitations for formal verification, prompting the development of a 5-level explanation taxonomy ranging from no explanation to machine-verifiable Description Logic. - [03:18](https://www.youtube.com/watch?v=tYxeBijkeRs&t=198s) The experimental framework tests frozen Multimodal LLMs (Gemini, Gemma, Qwen, LLaMA) using In-Context Learning across datasets like CIFAR-10, DTD, Oxford Flowers, and Oxford-IIIT Pet. - [08:59](https://www.youtube.com/watch?v=tYxeBijkeRs&t=539s) Evaluation employs an LLM-as-a-Judge alongside a custom Human-in-the-Loop auditing interface to evaluate predictions and generated visual explanations across nine dimensions. - [08:59](https://www.youtube.com/watch?v=tYxeBijkeRs&t=539s) A specialized concordance formula tolerating minor score divergences confirmed strong alignment between human auditors and the LLM judge (>0.8 score). - [13:23](https://www.youtube.com/watch?v=tYxeBijkeRs&t=803s) Experiments uncovered an inverse relationship between explainability and performance: direct prediction achieved 93.8% accuracy, whereas requiring formal Description Logic dropped accuracy to 90.1%. - [14:37](https://www.youtube.com/watch?v=tYxeBijkeRs&t=877s) Current Multimodal LLMs are primarily instruction-tuned for end-result prediction rather than formal reasoning, creating an accuracy penalty when structured explanations are required. - [14:37](https://www.youtube.com/watch?v=tYxeBijkeRs&t=877s) Future research must focus on supervised fine-tuning for machine-verifiable transparency and establishing continuous auditing to close the accuracy-explainability gap in production systems.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=tYxeBijkeRs&t=4s) Introduction and the Pillars of Trustworthy AI

Leticia Lorena Rodriguez introduces herself, noting that she has traveled all the way from Argentina to present her research on artificial intelligence explainability. She shares that she wears "many hats": she is a computer science professor at the University of Buenos Aires (UBA), has conducted academic research at CONICET (the National Scientific and Technical Research Council of Argentina), and has spent nearly 30 years in the industry with tenures at companies like Google and Mercado Libre. She emphasizes that understanding the inherent risks of AI, particularly large language models (LLMs), is crucial for educators, students, and enterprises alike.

Building on this, she outlines the five fundamental pillars required for responsible and trustworthy AI: fairness, accountability, safety, transparency, and privacy. She stresses that every software developer, user, and organization must keep these dimensions in mind. Her research zeros in on transparency and explainability—uncovering the exact reasons behind a model’s decisions to combat unpredictable behavior and hallucinations.

```mermaid flowchart TD accTitle: The Five Pillars of Trustworthy AI Pillars["Trustworthy & Responsible AI"] Pillars --> Fair["Fairness"] Pillars --> Acc["Accountability"] Pillars --> Safe["Safety"] Pillars --> Trans["Transparency & Explainability"] Pillars --> Priv["Privacy"] ```

[03:18](https://www.youtube.com/watch?v=tYxeBijkeRs&t=198s) Moving Beyond Chain-of-Thought: The 5-Level Explanation Pipeline

While Chain-of-Thought (CoT) prompting ("think step by step") is widely used, Rodriguez notes that recent literature demonstrates its clear limitations in formal verification. To systematically assess explainability, her team developed an evaluation framework categorized across five formats ranging from unstructured text to machine-verifiable formal logic:

1. **No explanation** (pure classification) 2. **Natural language** 3. **Feature-based** 4. **Value-based** 5. **Description Logic / Formal Ontologies** (a machine-verifiable, concept-based logical language)

The core experimental pipeline feeds multimodal LLMs a few-shot In-Context Learning (ICL) prompt containing support images and a query image. The model must output both the predicted class and an explanation structured according to the requested format. A secondary model acting as an "LLM-as-a-Judge" then evaluates the quality, completeness, and accuracy of both the prediction and the explanation across standardized dimensions.

```mermaid flowchart LR accTitle: Multimodal In-Context Learning Evaluation Pipeline subgraph Input["Prompt Construction (ICL)"] A["Support Set Images & Labels"] B["Target Query Image"] C["Requested Explanation Format"] end

subgraph Target["MLLM Under Test"] D["Frozen Multimodal LLM\n(Gemini, Gemma, Qwen, LLaMA)"] end

subgraph Output["Predictions & Adjudication"] E["Predicted Class +\nStructured Explanation"] F["LLM-as-a-Judge Evaluation"] G["Human-in-the-Loop Audit"] end

A --> D B --> D C --> D D --> E E --> F E --> G ```

- **Supported Datasets**: CIFAR-10, Describable Textures Dataset (DTD), Oxford Flowers, and Oxford-IIIT Pet. - **Model Palette**: Experiments were executed on frozen baseline models including Google Gemini, Gemma, Alibaba Qwen, and Meta LLaMA. - **Example Flow**: In an Oxford Flowers test case, the model correctly predicts class `Clematis` and generates visual descriptors: *"Five petals, purple petals, light colored center with multiple stamens."*

[08:59](https://www.youtube.com/watch?v=tYxeBijkeRs&t=539s) Human-in-the-Loop Auditing and Alignment Scoring

Because relying entirely on automated LLM judges carries risks of compounding errors, the team designed a custom auditing interface to bring humans into the evaluation loop. In this setup, human annotators view the target image (e.g., a cat), the support set examples, and the LLM's generated explanation, scoring them across nine distinct evaluation dimensions.

To measure concordance between the LLM judge and human reviewers, the team introduced a specialized scoring formula designed to evaluate alignment while tolerating minor variance:

- **Exact Match**: When the human score and LLM judge score are identical, the score is $1.0$. - **1-Point Difference**: A minor divergence (e.g., human rates $4$ and LLM rates $5$) is lightly penalized, yielding a score of $0.75$. - **Large Divergence**: Differences greater than one point (e.g., a $5$ vs. a $2$) drop immediately to $0.0$.

Empirical results demonstrated strong alignment between the LLM judge and human reviewers across all nine evaluation metrics, with alignment scores consistently exceeding $0.8$.

[13:23](https://www.youtube.com/watch?v=tYxeBijkeRs&t=803s) Experimental Findings: The Penalty of Explainability

The central finding of the research reveals a striking inverse relationship between explainability and predictive accuracy:

Forcing models to articulate and formalize their visual reasoning systematically degrades their task performance. Requiring models to construct formal Description Logic representations led to the steepest drop, falling nearly four full percentage points from the unprompted baseline.

[14:37](https://www.youtube.com/watch?v=tYxeBijkeRs&t=877s) Strategic Takeaways and Future Work

Rodriguez concludes by emphasizing the practical implications for enterprise AI deployments. If demanding formal explanations harms task accuracy, companies must recognize this hidden trade-off when deploying autonomous visual agents. She outlines three core pillars for future research:

- **Bridging the Explainability-Accuracy Gap**: Developing techniques where generating rationale does not cannibalize reasoning performance. - **Supervised Fine-Tuning (SFT) for Explainability**: Modern models are tuned strictly for end-result accuracy; they require fine-tuning on machine-verifiable explanation datasets to reason transparently. - **Independent Auditing**: Establishing continuous auditing mechanisms to guard against unseen hallucination risks in mission-critical applications.

The paper was accepted and presented at ECML PKDD in the 2nd Workshop on Compositional Learning. Rodriguez closes by inviting attendees to connect with her through TEG AI Consulting.

## Notable quotes > formal explainability has become the ultimate bottleneck between functional prototypes and safe, enterprise-ready systems. — Leticia Lorena Rodriguez

> forcing Multimodal Large Language Models (MLLMs) to generate formally structured, concept-based explanations surprisingly degrades their predictive accuracy. — Leticia Lorena Rodriguez

> explaining is genuinely harder than predicting alone and that current models lack the instruction-tuning required for machine-verifiable transparency. — Leticia Lorena Rodriguez

> [03:18](https://www.youtube.com/watch?v=tYxeBijkeRs&t=198s) Five petals, purple petals, light colored center with multiple stamens. — MLLM Under Test

## Mentioned Leticia Lorena Rodriguez, Universidad de Buenos Aires (UBA), CONICET, Google, Mercado Libre, TEG AI Consulting, ECML PKDD, Gemini, Gemma, Qwen, LLaMA, CIFAR-10, Describable Textures Dataset (DTD), Oxford Flowers, Oxford-IIIT Pet

## Video description Leticia Lorena Rodriguez, Independent Researcher / Founder, Universidad de Buenos Aires / TEG AI Consulting, Argentina

As the industry shifts towards autonomous AI agents, formal explainability has become the ultimate bottleneck between functional prototypes and safe, enterprise-ready systems. While Chain-of-Thought is popular, our research reveals that forcing Multimodal Large Language Models (MLLMs) to generate formally structured, concept-based explanations surprisingly degrades their predictive accuracy. The talk will explore this phenomenon using a rigorous evaluation pipeline, demonstrating that explaining is genuinely harder than predicting alone and that current models lack the instruction-tuning required for machine-verifiable transparency. I will explain why overcoming this evaluation hurdle is fundamental for deploying robust, auditable agents in real-world industrial applications. Join this session to uncover the hidden limits of in-context learning and learn what it takes to build the trustworthy visual classifiers the industry demands.

[Source](https://www.youtube.com/watch?v=tYxeBijkeRs)

中文简报

研究发现：要求提供形式化解释会降低多模态AI准确率强制多模态模型生成机器可验证的结构化推理，会使视觉分类准确率从93.8%降至90.1%，揭示了企业部署中的实际权衡。立足于可信AI的核心支柱，Rodriguez介绍了一套评估流程，在从无提示分类到机器可验证的描述逻辑（Description Logic）等五种不同解释格式下测试多模态模型。该研究利用图像基准数据集以及经人工审核员验证的LLM-as-a-Judge协议，揭示了显著的“可解释性代价”：直接预测达到了93.8%的准确率，而要求提供形式逻辑解释则导致性能下滑至90.1%。她认为，为使自主智能体能够在关键的现实部署中切实可行，业界从业人员必须通过专门的有监督微调和严格的独立审计来弥合这一差距。 Leticia Lorena Rodriguez 阐释了可信 AI 的五大核心支柱（公平性、问责制、安全性、透明度和隐私），并指出形式化可解释性是避免企业级 AI 智能体出现不可预测行为与幻觉的关键瓶颈。 (0:04) 针对标准 Chain-of-Thought 的局限性，团队提出了涵盖无解释到机器可验证描述逻辑（Description Logic）的五级解释分类法，并在 CIFAR-10 和 Oxford Flowers 等数据集上对 Gemini、Gemma、Qwen 和 LLaMA 等冻结多模态模型进行了上下文学习评估。 (3:18) 评估体系结合了 LLM-as-a-Judge 与定制的人在回路（Human-in-the-Loop）审计界面，特制的一致性评分公式证实人类审计员与 LLM 裁判在九个评估维度上的对齐得分稳定超过 0.8。 (8:59) 实验揭示了可解释性与准确率之间的反比关系：直接分类的准确率为 93.8%，而在要求输出形式化描述逻辑时准确率下降近 4 个百分点至 90.1%，表明现存多模态模型尚缺乏面向机器可验证透明度的指令微调。 (13:23)

Reading summary

Requiring multimodal large language models to articulate formal explanations for their predictions causes a systematic degradation in classification performance, according to research presented by Leticia Lorena Rodriguez. Structuring Explainability Beyond Chain-of-Thought (3:18) While Chain-of-Thought prompting is common, Rodriguez notes that recent literature highlights its inadequacy for formal verification. To measure transparency within trustworthy AI frameworks, her research team constructed a five-level evaluation pipeline for multimodal in-context learning (ICL). The formats range in formality: no explanation (pure classification), natural language, feature-based, value-based, and machine-verifiable Description Logic grounded in formal ontologies. The framework tested frozen multimodal models—including Google Gemini, Gemma, Alibaba Qwen, and Meta LLaMA—across datasets such as CIFAR-10, Describable Textures Dataset (DTD), Oxford Flowers, and Oxford-IIIT Pet. Models received support images and a query image, and were tasked with outputting both a predicted class and an explanation matching the requested format, which was subsequently evaluated by an automated LLM-as-a-judge. Validating LLM Evaluators with Human Audits (8:59) To mitigate the risk of compounding errors from an automated LLM judge, the researchers integrated a human-in-the-loop auditing interface. Human reviewers evaluated the target images, context examples, and generated model rationales across nine standardized dimensions. Alignment between the automated judge and human reviewers was scored using a non-linear scale: an exact score match received 1.0, a single-point deviation received 0.75, and any discrepancy greater than one point was scored as 0.0. The team reported strong concordance across all nine dimensions, with alignment scores consistently exceeding 0.8. The Accuracy Penalty of Explaining Decisions (13:23) The experimental results demonstrated an inverse relationship between explanation rigor and predictive performance. When executing direct classification with no explanation requested, the baseline models achieved 93.8% accuracy. Demanding explanations led to performance declines that worsened as structural rigor increased. Intermediate representations showed gradual degradation, while requiring formal Description Logic ontologies produced the steepest penalty, dropping accuracy by nearly four percentage points to 90.1%. Forcing models to verbalize their visual reasoning in a formal syntax impaired their underlying classification capability. Strategic Imperatives for Enterprise Deployments (14:37) Rodriguez cautions that enterprises deploying autonomous visual agents must account for this hidden performance penalty when requiring audit trails. Current foundational models are trained primarily for end-result accuracy rather than verifiable rationale generation. To address this limitation, Rodriguez identifies three key priorities for future research: conducting supervised fine-tuning (SFT) explicitly on machine-verifiable explanation datasets, developing techniques that bridge the explainability-accuracy gap, and implementing independent, continuous auditing systems to detect hallucinations in mission-critical environments. The research, presented at ECML PKDD's 2nd Workshop on Compositional Learning, shows that current multimodal LLMs cannot provide rigorous, verifiable explanations without compromising task accuracy. Until models are specifically fine-tuned for structural reasoning, organizations face an operational trade-off between predictive accuracy and decision transparency.

精炼阅读摘要

布宜诺斯艾利斯大学计算机科学教授 Leticia Lorena Rodriguez 在 Ai4 峰会上指出，在多模态大语言模型的上下文学习中，要求模型生成解释会导致预测准确率下降，解释的形式化程度越高，性能损失越显著。可信人工智能的支柱与透明度诉求 (0:04) Rodriguez 结合其在学术界与工业界的长期经历指出，理解人工智能（特别是大语言模型）的固有风险对教育者、学生和企业至关重要。负责任且值得信赖的人工智能建立在五大核心支柱之上：公平性、问责制、安全性、透明度以及隐私保护。软件开发者、用户和组织在设计与部署系统时都必须兼顾这些维度。在这些支柱中，该项研究专门聚焦于透明度与可解释性。Rodriguez 强调，厘清模型决策背后的确切原因对于应对不可预测的行为和幻觉现象至关重要。仅依赖最终输出无法满足关键任务对可信度的要求，必须建立能验证模型推理逻辑的评估体系。多模态五级解释评测框架的构建 (3:18) 针对思维链（CoT）在形式化验证中的局限性，研究团队提出了一个包含五个解释层级的系统性评测框架，涵盖从纯文本到机器可验证的形式逻辑：第一级为无解释（纯分类），第二级为自然语言，第三级为基于特征的解释，第四级为基于属性值的解释，第五级为基于描述逻辑与形式本体论（Description Logic / Formal Ontologies）的概念化机器可验证语言。在实验流程中，研究人员采用少样本上下文学习（ICL）方法，向冻结的基础多模态大模型（包括 Google Gemini、Gemma、阿里 Qwen 和 Meta LLaMA）输入支持集图像、目标查询图像以及指定的解释格式指令。测试数据集涵盖 CIFAR-10、DTD 纹理数据集、Oxford Flowers 和 Oxford-IIIT Pet。模型输出的预测类别与结构化解释随后交由评判大模型（LLM-as-a-Judge）以及人工审计进行质量评估。人机回环审计与评分对齐验证 (8:59) 为了防止单纯依赖评判大模型导致错误累积，研究团队设计了人机回环审计界面。人类标注员需要同时查看目标图像（例如猫的图像）、支持集样例以及模型生成的解释，并围绕九个独立的评估维度进行人工打分。为衡量评判模型与人类评审之间的一致性，团队制定了一套容忍微小偏差的对齐评分规则：当模型评分与人类评分完全一致时，得分为 1.0；相差 1 分时予以轻微惩罚，得分为 0.75；当分歧超过 1 分（如 5 分与 2 分）时，得分直接降为 0.0。实验结果显示，在全部九个评估维度上，评判模型与人类评分的对齐分数均持续超过 0.8，证实了自动化评判与人类判断之间具有高度一致性。可解释性带来的任务准确率惩罚 (13:23) 实验的核心结论揭示出可解释性与预测准确率之间存在明显的反向变动关系。在不需要提供任何解释的纯分类设置下，多模态模型的基线分类准确率达到 93.8%。当要求模型使用自然语言或中间形式给出推理依据时，准确率呈现逐渐下降的趋势。当要求模型构建高度严格的描述逻辑形式化表示时，准确率降至 90.1%，相比基线下降了近 4 个百分点。数据表明，强行要求模型形式化表达其视觉推理过程，会系统性地损害模型在核心分类任务上的表现，形成了不可忽视的性能代价。 Rodriguez 总结认为，企业在部署自主视觉智能体时必须认识到强制解释可能削弱任务准确率这一权衡关系。她提出未来研究应着眼于缩小可解释性与准确率之间的差距、利用机器可验证的解释数据集进行监督微调（SFT），以及在关键业务中建立持续的独立审计机制，该成果已入选 ECML PKDD 的第二届组合学习研讨会。

Detailed notes — zh

## Background 本视频由 YouTube 频道 Ai4 于 2026 年 8 月 14 日发布，演讲者为布宜诺斯艾利斯大学教授兼 TEG AI Consulting 创始人 Leticia Lorena Rodriguez。内容聚焦于多模态大语言模型（MLLM）的形式化可解释性研究，探讨强制模型生成基于概念的结构化解释为何反而会降低其预测精度，以及如何为工业级多模态智能体构建机器可验证的透明审计体系。 ## TL;DR 演讲围绕论文《解释难于预测：评估上下文学习中的多模态大语言模型》展开，系统评估了多模态模型在图像分类少样本上下文学习（ICL）中的表现。研究团队构建了从“无解释”到严密“描述逻辑”（Description Logic）的五级解释体系，并在开源与闭源主流模型上结合 LLM 裁判与人在回路（Human-in-the-Loop）机制进行了严谨测试。实验发现，强制模型输出严谨的符号逻辑解释会导致分类准确率从基线的 93.8% 跌落至 90.1%，直观证实了形式化解释对视觉表征的认知挤占效应。讲者据此提出了面向可解释性的监督微调（SFT）等未来方向，以构建兼顾准确率与可审计性的可信多模态智能体。 ## Atmosphere & style 整场演讲洋溢着严谨、务实且兼备深厚工业洞察的学术氛围。讲者凭借跨越近 30 年的学界研究与工业界落地经验，以沉稳清晰的节奏层层拆解实验设计与数据流水线，有力反驳了思维链技术普遍适用的盲目乐观认知。整体叙事兼具对负责任 AI 风险的清醒警示与对形式化逻辑前沿探索的专业笃定，给人以严谨可信、发人深省的现场感受。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=tYxeBijkeRs&t=4s) 构建可信人工智能（Trustworthy AI）需聚焦公平性、问责性、安全性、透明度与隐私保护五大核心维度，其中透明度与可解释性是高风险工业场景落地的关键瓶颈。 - [03:18](https://www.youtube.com/watch?v=tYxeBijkeRs&t=198s) 针对传统思维链（CoT）自然语言输出模糊且难以自动验证的问题，研究提出了从无解释、自然语言、特征级、属性值级到描述逻辑（Description Logic）的五级形式化解释评估体系。 - [04:27](https://www.youtube.com/watch?v=tYxeBijkeRs&t=267s) 评估采用上下文学习（ICL）少样本视觉分类范式，要求冻结参数的多模态大模型在完成图像分类预测的同时，同步输出符合形式化规范的决策依据。 - [04:27](https://www.youtube.com/watch?v=tYxeBijkeRs&t=267s) 构建了 LLM-as-a-Judge 自动化裁判系统，从 9 个独立维度对模型生成的解释完整度、准确性与逻辑严密性进行量化打分。 - [07:39](https://www.youtube.com/watch?v=tYxeBijkeRs&t=459s) 实验广泛覆盖 Gemini、Gemma、Qwen 和 Llama 等主流多模态模型，并在 CIFAR-10、DTD、Oxford Flowers 等多个少样本基准上进行系统化评测。 - [07:39](https://www.youtube.com/watch?v=tYxeBijkeRs&t=459s) 引入人在回路（Human-in-the-Loop）机制与阶梯式惩罚评分公式，验证了人类专家与大模型裁判在解释评估上的对齐一致性（对齐度均在 0.8 以上）。 - [13:23](https://www.youtube.com/watch?v=tYxeBijkeRs&t=803s) 实验揭示反直觉的核心现象：解释比单纯预测更困难，强制模型生成结构化解释会导致分类准确率单调递减（无解释为 93.8%，而描述逻辑形式跌至 90.1%）。 - [13:23](https://www.youtube.com/watch?v=tYxeBijkeRs&t=803s) 强制遵循复杂的符号逻辑与形式本体会挤占模型在视觉特征辨识上的注意力与表征容量，从而对最终决策的准确率产生反噬。 - [14:37](https://www.youtube.com/watch?v=tYxeBijkeRs&t=877s) 未来推动可解释多模态智能体落地需将形式化解释能力直接纳入监督微调（SFT），并构建兼顾透明度与预测精度的机器可验证审计流水线。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=tYxeBijkeRs&t=4s) 讲者背景与可信 AI 的核心维度来自阿根廷的布宜诺斯艾利斯大学（University of Buenos Aires, UBA）计算机科学教授 Leticia Lorena Rodriguez 开启了本次演讲。她不仅在阿根廷国家科学与技术研究委员会（CONICET）从事学术研究，还在工业界深耕了近 30 年，曾先后在 Google 和拉美电商巨头 Mercado Libre 任职，目前兼任 TEG AI Consulting 创始人。Rodriguez 教授指出，随着大语言模型（LLM）与多模态大模型（MLLM）在工业界的广泛落地，人工智能带来的潜在风险不容忽视，无论是学界研究者、一线软件工程师还是企业决策层，都必须对这些系统性风险保持高度警惕。针对“可信人工智能”（Trustworthy AI）的构建，演讲强调了业界公认的五大核心维度：公平性（Fairness）、问责性（Accountability）、安全性（Safety）、透明度（Transparency）以及隐私保护（Privacy）。本次研究将重心聚焦于“透明度”与“可解释性”（Explainability）。由于大语言模型普遍存在幻觉（Hallucination）和输出不可靠的问题，单纯获取一个分类结果已无法满足高风险工业场景的需求。研究的核心目标是探究模型做出每一项决策、生成每一段文本背后的真正依据。 - 讲者身兼布宜诺斯艾利斯大学教授、CONICET 研究员及 TEG AI Consulting 创始人，具备近 30 年工业界与学术界跨界经验。 - 强调可信 AI 的五大支柱：公平性、问责性、安全性、透明度与隐私保护。 - 本项研究的主题为《解释难于预测：评估上下文学习中的多模态大语言模型》（Explaining is Harder Than Predicting: Evaluating Multimodal LLMs in ICL）。 --- ### [03:18](https://www.youtube.com/watch?v=tYxeBijkeRs&t=198s) 评估框架设计：从自然语言到描述逻辑的五级解释体系在探讨模型可解释性时，目前最主流的方法是思维链（Chain of Thought, CoT）提示词技术，即通过引导模型“一步步思考”（think step by step）来输出推理过程。然而，近期的诸多研究表明思维链存在明显的局限性，其生成的自然语言解释往往模糊且难以被机器自动化验证。为了系统化评估多模态模型的解释能力，研究团队构建了一套包含五个梯度的形式化解释评估框架： 1. **无解释（No Explanation）**：基线对照组，仅要求模型给出最终的分类预测标签。 2. **自然语言解释（Natural Language）**：类似于常规思维链，输出自由文本描述。 3. **基于特征的解释（Feature-based）**：提取目标的显式属性或视觉特征。 4. **基于属性值的解释（Value-based）**：进一步量化和结构化特征的取值。 5. **形式本体 / 描述逻辑（Description Logic, DL）**：使用严谨的形式逻辑语言表达概念关系，能够直接被计算机程序进行形式化验证。 ```mermaid flowchart LR accTitle: 五级解释结构层级 subgraph Levels["解释形式化程度阶梯"] L1["1. 无解释 (No Explanation)"] L2["2. 自然语言 (Natural Language)"] L3["3. 特征级 (Feature-based)"] L4["4. 属性值级 (Value-based)"] L5["5. 描述逻辑 (Description Logic)"] end L1 --> L2 --> L3 --> L4 --> L5 ``` --- ### [04:27](https://www.youtube.com/watch?v=tYxeBijkeRs&t=267s) 实验流程与上下文学习（ICL）视觉分类任务实验采用了标准的上下文学习（In-Context Learning, ICL）范式进行少样本视觉分类。在 Prompt 中，研究团队向多模态大模型提供若干张带有类别标签的示例图片（Support Set），随后给出一张查询图片（Query Image，例如一朵紫色的花）。模型不仅需要预测出该查询图片的正确分类，还必须严格按照指定的五种解释格式之一，同步输出其做出该决策的形式化推理依据。在具体的分类示例中，模型成功将目标图片识别为“铁线莲”（Clematis），并输出了结构化特征解释：“5片花瓣、紫色花瓣、浅色中心伴有多枚雄蕊”（Five petals, purple petals, light colored center with multiple stamens）。为了对生成的解释质量进行自动化和规模化评估，团队采用了“大模型作为裁判”（LLM-as-a-Judge）的技术方案，由另一个独立的大模型从 9 个不同维度对解释的完整性、准确性与逻辑严密程度进行打分。 ```mermaid flowchart TD accTitle: 实验与评估流水线 subgraph ICL["上下文学习推理阶段"] A["支持集示例 (Support Set) + 解释格式要求"] --> C["待测 MLLM (Frozen)"] B["查询图片 (Query Image)"] --> C C --> D["分类预测 (Class) + 形式化解释 (Explanation)"] end subgraph Eval["双重评估流水线"] D --> E["LLM 裁判 (LLM-as-a-Judge) 9 维度自动打分"] D --> F["人类专家审核系统 UI 界面独立标注"] E --> G["对齐度一致性计算 (Score Formula)"] F --> G end ``` - 任务设定：给定上下文少样本示例与解释格式模板，多模态模型需同时完成“图像分类预测”与“生成对应格式的推理链”。 - 评估机制：引入 LLM-as-a-Judge 针对解释的 9 项评估指标进行自动化质量裁决。 --- ### [07:39](https://www.youtube.com/watch?v=tYxeBijkeRs&t=459s) 测试模型、基准数据集与人在回路（Human-in-the-Loop）对齐评估为了保证实验结论的通用性，研究涵盖了主流的多模态开源与闭源基准模型，包括 Google Gemini、Gemma、阿里千问（Qwen）以及 Meta Llama 等，所有测试均在模型参数完全冻结（Frozen）的状态下进行。测试基准涵盖了 CIFAR-10、可描述纹理数据集（DTD）、Oxford 102 鲜花数据集（Oxford Flowers）以及 Oxford-IIIT 宠物数据集（Oxford Pets）。实验还设计了多种少样本配置（Few-shot Settings），例如每类 1 个样本（1-shot）或每类 5 个样本（5-shot），以探索上下文样本密度对推理的影响。针对负责任 AI 倡议中对纯自动化裁决的质疑，团队坚持引入“人在回路”（Human-in-the-Loop）机制。他们自研了标注审核软件，界面直观呈现待测查询图片、模型生成的解释文本以及原始支持集图片，由人类专家在 9 个维度上独立打分。为了量化人类评审员与 LLM 裁判之间的对齐程度，团队设计了一套阶梯式惩罚评分公式： - **完全一致（Exact Match）**：当人类评分与 LLM 评分一致时，对齐得分为 $1.0$。 - **微小分歧（1 分差距）**：若两者仅相差 1 分（例如人类打 4 分，LLM 打 5 分），评分设定为 $0.75$，给予适度容忍与轻微惩罚。 - **严重分歧（差异 $\ge 2$ 分）**：若两者差距达到 2 分及以上（例如 LLM 给出满分 5 分，而人类仅打 2 分），对齐得分直接归零（$0$）。实验统计显示，在 9 个评估维度上，人类专家与 LLM 裁判的对齐度均维持在 0.8 以上的高位水平，证明了自动化裁判流水线具备可靠的评估效度。 --- ### [13:23](https://www.youtube.com/watch?v=tYxeBijkeRs&t=803s) 核心实验发现：解释为何会拖累预测准确率？汇集全部实验数据后，研究团队揭示出了一个反直觉的关键现象：**迫使模型生成结构化解释反而显著降低了其原本的预测分类准确率**。在基线实验中，当模型仅执行分类任务而不提供任何解释（No Explanation）时，平均预测准确率达到了 **93.8%**。然而，一旦要求模型同步输出决策依据，准确率便随着解释形式化程度的加深呈现出单调递减的趋势。当要求模型使用表达最为严密、最便于机器自动化验证的“描述逻辑”（Description Logic）输出本体解释时，模型的分类准确率跌至全场最低的 **90.1%**。 | 解释形式层级 | 形式化特征 | 任务分类准确率 | |:--- |:--- |:--- | | **无解释（No Explanation）** | 仅输出类别标签，无额外认知负荷 | **93.8%**（最高） | | **自然语言 / 属性特征解释** | 自由文本或半结构化属性，逻辑约束较松散 | 准确率逐步下降 | | **描述逻辑（Description Logic）** | 严格的形式本体与可验证符号逻辑 | **90.1%**（最低） | 这一发现有力地证实了“解释比单纯预测更加困难”（Explaining is harder than predicting）。在现有的上下文学习机制下，强制模型遵循复杂的符号逻辑规则生成解释，会挤占其视觉特征辨识的注意力与表征容量，从而对最终的决策正确性产生反噬。 --- ### [14:37](https://www.youtube.com/watch?v=tYxeBijkeRs&t=877s) 结论与未来展望：构建可解释多模态智能体 Rodriguez 教授在总结中向工业界提出了明确预警：企业在部署自动化多模态系统时，必须认识到强行要求模型自圆其说可能带来隐蔽的精度损失风险，建立严格的多模态审计机制势在必行。针对当前的技术瓶颈，她提出了推动可信多模态智能体落地的三大支柱： - **弥合解释性与准确率的鸿沟（Bridge the Gap）**：深入探索模型认知架构，消除形式化逻辑输出对主任务特征分类的干扰。 - **面向可解释性的监督微调（SFT for Explainability）**：目前的微调大多只关注“预测结果是否正确”，未来必须将“推理过程的形式化解释能力”直接纳入监督微调目标。 - **构建机器可验证的审计流水线（Auditable Agents）**：利用局部判别度（Local Discrimination）等指标，打造兼顾透明度与鲁棒性的工业级视觉分类器。在演讲尾声，Rodriguez 教授宣布该项由布宜诺斯艾利斯大学与 TEG AI Consulting 联合完成的研究成果已被欧洲机器学习会议（ECML 2026）第二届组合学习研讨会（Workshop on Compositional Learning）正式录用。她向在场听众致谢，并欢迎学术界与工业界同行通过线上渠道建立联系，共同推动多模态模型可信度与形式化解释技术的演进。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=tYxeBijkeRs&t=4s) 来自阿根廷的布宜诺斯艾利斯大学计算机科学教授 Leticia Lorena Rodriguez 开启了本次演讲。她不仅在阿根廷国家科学与技术研究委员会从事学术研究，还在工业界深耕了近 30 年 — Leticia Lorena Rodriguez > [00:04](https://www.youtube.com/watch?v=tYxeBijkeRs&t=4s) 针对“可信人工智能”的构建，演讲强调了业界公认的五大核心维度：公平性、问责性、安全性、透明度以及隐私保护。 — Leticia Lorena Rodriguez > [03:18](https://www.youtube.com/watch?v=tYxeBijkeRs&t=198s) 近期的诸多研究表明思维链存在明显的局限性，其生成的自然语言解释往往模糊且难以被机器自动化验证。 — Leticia Lorena Rodriguez > [04:27](https://www.youtube.com/watch?v=tYxeBijkeRs&t=267s) 为了对生成的解释质量进行自动化和规模化评估，团队采用了“大模型作为裁判”（LLM-as-a-Judge）的技术方案，由另一个独立的大模型从 9 个不同维度对解释的完整性、准确性与逻辑严密程度进行打分。 — Leticia Lorena Rodriguez > [07:39](https://www.youtube.com/watch?v=tYxeBijkeRs&t=459s) 针对负责任 AI 倡议中对纯自动化裁决的质疑，团队坚持引入“人在回路”（Human-in-the-Loop）机制。 — Leticia Lorena Rodriguez > [13:23](https://www.youtube.com/watch?v=tYxeBijkeRs&t=803s) 汇集全部实验数据后，研究团队揭示出了一个反直觉的关键现象：迫使模型生成结构化解释反而显著降低了其原本的预测分类准确率。 — Leticia Lorena Rodriguez > [13:23](https://www.youtube.com/watch?v=tYxeBijkeRs&t=803s) 当模型仅执行分类任务而不提供任何解释（No Explanation）时，平均预测准确率达到了 93.8%。然而，一旦要求模型同步输出决策依据，准确率便随着解释形式化程度的加深呈现出单调递减的趋势。 — Leticia Lorena Rodriguez > [13:23](https://www.youtube.com/watch?v=tYxeBijkeRs&t=803s) 当要求模型使用表达最为严密、最便于机器自动化验证的“描述逻辑”（Description Logic）输出本体解释时，模型的分类准确率跌至全场最低的 90.1%。 — Leticia Lorena Rodriguez > [13:23](https://www.youtube.com/watch?v=tYxeBijkeRs&t=803s) 这一发现有力地证实了“解释比单纯预测更加困难”（Explaining is harder than predicting）。 — Leticia Lorena Rodriguez > [14:37](https://www.youtube.com/watch?v=tYxeBijkeRs&t=877s) 企业在部署自动化多模态系统时，必须认识到强行要求模型自圆其说可能带来隐蔽的精度损失风险，建立严格的多模态审计机制势在必行。 — Leticia Lorena Rodriguez ## Mentioned Leticia Lorena Rodriguez, Universidad de Buenos Aires, CONICET, TEG AI Consulting, Google, Mercado Libre, Gemini, Gemma, Qwen, Llama, CIFAR-10, DTD, Oxford Flowers, Oxford Pets, ECML 2026, Workshop on Compositional Learning, LLM-as-a-Judge, Chain of Thought, Description Logic ## Video description Leticia Lorena Rodriguez, Independent Researcher / Founder, Universidad de Buenos Aires / TEG AI Consulting, Argentina As the industry shifts towards autonomous AI agents, formal explainability has become the ultimate bottleneck between functional prototypes and safe, enterprise-ready systems. While Chain-of-Thought is popular, our research reveals that forcing Multimodal Large Language Models (MLLMs) to generate formally structured, concept-based explanations surprisingly degrades their predictive accuracy. The talk will explore this phenomenon using a rigorous evaluation pipeline, demonstrating that explaining is genuinely harder than predicting alone and that current models lack the instruction-tuning required for machine-verifiable transparency. I will explain why overcoming this evaluation hurdle is fundamental for deploying robust, auditable agents in real-world industrial applications. Join this session to uncover the hidden limits of in-context learning and learn what it takes to build the trustworthy visual classifiers the industry demands. [Source](https://www.youtube.com/watch?v=tYxeBijkeRs)

Source quotes

[0:04] Leticia Lorena Rodriguez: 来自阿根廷的布宜诺斯艾利斯大学计算机科学教授 Leticia Lorena Rodriguez 开启了本次演讲。她不仅在阿根廷国家科学与技术研究委员会从事学术研究，还在工业界深耕了近 30 年 [0:04] Leticia Lorena Rodriguez: 针对“可信人工智能”的构建，演讲强调了业界公认的五大核心维度：公平性、问责性、安全性、透明度以及隐私保护。 [3:18] Leticia Lorena Rodriguez: 近期的诸多研究表明思维链存在明显的局限性，其生成的自然语言解释往往模糊且难以被机器自动化验证。 [4:27] Leticia Lorena Rodriguez: 为了对生成的解释质量进行自动化和规模化评估，团队采用了“大模型作为裁判”（LLM-as-a-Judge）的技术方案，由另一个独立的大模型从 9 个不同维度对解释的完整性、准确性与逻辑严密程度进行打分。 [7:39] Leticia Lorena Rodriguez: 针对负责任 AI 倡议中对纯自动化裁决的质疑，团队坚持引入“人在回路”（Human-in-the-Loop）机制。 [13:23] Leticia Lorena Rodriguez: 汇集全部实验数据后，研究团队揭示出了一个反直觉的关键现象：迫使模型生成结构化解释反而显著降低了其原本的预测分类准确率。 [13:23] Leticia Lorena Rodriguez: 当模型仅执行分类任务而不提供任何解释（No Explanation）时，平均预测准确率达到了 93.8%。然而，一旦要求模型同步输出决策依据，准确率便随着解释形式化程度的加深呈现出单调递减的趋势。 [13:23] Leticia Lorena Rodriguez: 当要求模型使用表达最为严密、最便于机器自动化验证的“描述逻辑”（Description Logic）输出本体解释时，模型的分类准确率跌至全场最低的 90.1%。 [13:23] Leticia Lorena Rodriguez: 这一发现有力地证实了“解释比单纯预测更加困难”（Explaining is harder than predicting）。 [14:37] Leticia Lorena Rodriguez: 企业在部署自动化多模态系统时，必须认识到强行要求模型自圆其说可能带来隐蔽的精度损失风险，建立严格的多模态审计机制势在必行。

Source quotes

Leticia Lorena Rodriguez: formal explainability has become the ultimate bottleneck between functional prototypes and safe, enterprise-ready systems. Leticia Lorena Rodriguez: forcing Multimodal Large Language Models (MLLMs) to generate formally structured, concept-based explanations surprisingly degrades their predictive accuracy. Leticia Lorena Rodriguez: explaining is genuinely harder than predicting alone and that current models lack the instruction-tuning required for machine-verifiable transparency. [3:18] MLLM Under Test: Five petals, purple petals, light colored center with multiple stamens.

Original transcript

[0:04] Hi everyone. Hello. I have traveled a long distance. I come from Argentina. [0:10] So I am truly honored to be here and I am going to present first. [0:17] I'm professor in computer science. My career at University of Wenocidis is professor [0:24] and I've been teaching a lot but I had like many hats right. I worked in industry for almost [0:34] 30 years and also I worked in research in Koniset that is an academic institution in Argentina. [0:42] Um and I worked for Google and also in Marcado Li. So now I am bringing this presentation that is [0:52] like um my first research and it is regarding explanability in artificial intelligence. [1:01] This is a super important topic because um as we were talking we need to know we need to talk about [1:09] the risk of artificial intelligence at it has many. We need to let our students to know all the [1:16] risk. We need to have companies that are aware of the risk of artificial intelligence in particular [1:23] large language models. So the title of my work is explaining abilities harder than predicting [1:30] evaluating multimodel in in context learning. Let me see if I can pass this. This is [1:40] green button at the top. Okay. I got it. Thank you so much. Oh, I have a problem in this slide, but [1:49] the check circle uh is an image story. So, basically when we think about trustworthy AI, [1:57] we are thinking in five dimensions. F first we need to the AI be a have fairness. Then [2:06] we need to make it accountable. We need to make it safe. And also we need to make it transparent [2:14] and at the end we need to have privacy. These are the core five principles of responsible AI [2:22] and any company any user every person should be aware of this and mostly software developers. [2:32] So our work is focusing in transparency. In particular we want to make models that are [2:39] explainable. We want to understand why they are taking their decision. [2:45] What what are the reasons behind each decision, each text that they gave us? [2:53] Yes, you know that there is a common problem in large language model that the models hallucinate. [2:59] They are not always trustworthy. They can give us bad information. So explanability is related [3:07] to that to understand why they are given us and the reasoning behind each answer. So um [3:18] so a common technique is called chain of thought. Chain of thought is asking to the large language [3:24] model in the prompt to think a step by step and explain its reasonings. But re recently there [3:33] were a lot of research that say that chain of thought has a limitation and our proposal is to [3:41] um provide different kind of explanabilities using decryption logic and try to understand that. [3:49] So we done an evaluation framework. The framework has five dimensions. First we try with no [3:55] explanation then using lateral language then using feature base value base and at the end we wants to [4:03] use formal ontologies that is description logic. It's a kind of of logical language that can be [4:11] easily verifiable by a computer. So we want the LLM to provide us explanations in one of these [4:19] five dimensions going from not explanations to explanations in a logical language. Yes. [4:27] So we created this experimentation pipeline that include a classification where [4:34] we are going to provide um different examples to a large language model and an image and a new image [4:44] and we are going to ask the large language model to explain its reasoning. The reasoning [4:51] is going to be in one of the five formats that we had provided, right? You're following? Yeah. [5:00] So, we send that prompt to the large language model and then we use another LLM as a SH. [5:09] We use an LLM to evaluate all the plannations. Yes, it's going to be clear in next slide. [5:17] And we have done h we have used a different evaluation metrics for this search. [5:26] [snorts] So here there is an example of how we run this experimentation. [5:31] So we use a technique called in context learning. We pass the examples inside the prompt of the LLM. [5:40] So we we have we asked the LLM to classify the query the the flower that is in violet. Yes. [5:48] The task of the LLM is to provide the class of the flower that is in violet [5:55] and we provided two examples with the names of the classes. Yes. [6:01] And then we also provided the format of the plability of the planation that we want to get. [6:09] Yeah. So the task of the multimodel large language model is going to be [6:16] guess the class of the of the flower in violet and explain why she has decided that. Yeah. So [6:30] check out the output. It says class Clementes. She got it right. And then explanation. Five petals, [6:40] purple petals, light colored center with multiple estments. Yes. So he [snorts] she gave us the [6:49] glass and the explanation. The explanation is important because if we can explain all the [6:56] decisions that an LLM does, it won't be so risky. Yes. That's why explanability is so important. [7:05] And then we put that to put in a charge to tell us how accurate was the LLM. Yes. And also um [7:18] if the explanation is complete or it is a good explanation if it is missing something. [7:25] We evaluated the classification and explanation in five dimensions using another LLM. [snorts] [7:34] This is called it's a technique called LLMASH. [7:39] So for conducting this research we use different large language model. We use Gemini, [7:45] we use shemma, we used Quen, we use Lama. Yeah. And also we we use different data sets because we [7:55] we need to be aware that this is going to happen in in many many different settings, [8:02] right? We use C4 10, DTD, O for flower and O for pet as data sets and we use a technique that is [8:10] very common in future learning in this kind of classification that is selecting one example. [8:19] Yeah, check here. This is uh two classes, one image per class. Yes. So that we we we set up [8:29] different settings. What happen if we we put five examples for each class or one [8:35] example for each class? We try to do different configurations, few short configurations. Yeah. [8:44] But we hadn't changed the large language model. The large language model are the [8:48] same that we are using. They are frozen is the sheina or the shea as you know. So um [snorts] [8:59] so because we are working in explanability and for us uh being responsible is important. [9:08] We cannot let all the decisions to the NLM. We cannot judge our work everything using a large [9:17] language model. We need to have humans. We need to have humans in the loop. So what we have done is [9:24] we created the software here. It shows the image uh that is the query image that is the cut that is [9:32] uh on your right. The cut on your right is the the the image that we are evaluating and then we have [9:42] the explanability in the middle. The explanation generated by the large language model and then the [9:48] support set. all the images that we have shown about that how they have to to classify. Yes. [9:57] And a human was annotating in the last column if the um if if the the what they think about the [10:08] the [clears throat] result of the plannability in the nine dimensions that we are evaluating [10:14] each explanations. So we want to check with this pipeline if this are just with what the [10:22] shash have done. People is checking if the sh is doing a good work or not. Yes. Uh and we find [10:30] that uh there is a kind of misalignment. We we are not working exactly like the large language model [10:38] and and even the people is not workingly exactly the same. we have different reviewers and all [10:46] the reviews uh were different between the people that were assessing our work. So we we create this [10:55] sorry there is a little bit of math on there in this slides but usually in research paper [11:01] we we put the mouth uh we created a a formula that is trying to to gather the difference between the [11:10] trash and the human reviewer and for doing that we created this formula. This formula works [11:18] as anore as a way of writing how how was it and if the human agree with the large language model [11:28] it gets a score of one because it is a good agreement they are aligned. [11:34] But if the human and the and the large language model have one point of difference, for example, [11:43] the human rated four and the large language model rated five h is going to be 0 75 [11:54] because we want to penalize a bit that they have one point difference, right? like a 54 32 they [12:03] have it's very little the difference between the human and the large language model evaluator but [12:12] we want we don't want to be so hard on them but if they defer a lot we are going to send it to zero [12:20] we don't want to to have any match any relation if the large language model um rated a five and [12:28] the person rated a two then h it's going to have zero score because there is a lot of difference [12:37] right they hadn't agree so we find out that there is a kind of that is called alignment [12:44] you you have to align your reviewers and align the large language model to the criteria and [12:52] there is very similar alignment if you see the graph everything is almost over 0.8. [13:02] So they have a good alignment between the reviewers and the LLM. [13:07] And the dimensions that are in the X-axis are the nine different evaluator [13:15] evaluation metrics that we used for sashing if the explanation was good or bad. [13:23] So our result right we we took all this work and all this experimentation but what we have [13:30] find out with all this and basically we have seen that if we do um a classification without [13:40] explanability we get 93.8 Eight accuracy right the accuracy is very good if we don't ask the LM how [13:52] how she has arrived to that conclusion yes if we don't if if it doesn't provide any explanation [14:01] we get that accuracy but if we ask to to provide explanation we see that the accuracy is declining [14:10] if she has to explain why she's taking that decision the accuracy goes goes down [14:16] is it's is it's not uh it's not predicting well it's going worse each time. So if we do the [14:26] LLM to use uh decryption logic it is it is being the worst scenario. It is the the 90.1% accuracy. [14:37] So what are our conclusions? Um we think that this is important to know to the companies. [14:45] Yes. Beside the accuracy if we try to explain the classification or if we try to explain a decision [14:52] of a large language model it could be worse. It could be it could hurt the classification. [14:58] It could hurt the results. Uh also that these are risk that are not uh visible. uh we strongly think [15:08] that we need to audit the result of large language models. [15:13] They hallucinate there are we don't know what they are going to uh give us a result [15:20] and that could be dangerous in many in many different areas. We also found out that um [15:29] that local discrimination could be a metric for achieving accuracy [15:35] and uh we think that there are three pillars to work. One is uh to brings the gap between [15:44] explanability and accuracy. The other we think that we need to fine-tune the large language [15:50] model using supervised finetuning to provide explanability. Today we are fine-tuning the [15:56] large language model for making right prediction right we say okay this is a problem this is the [16:02] result and we need to get this but we also need to include explanability to fine-tune models to [16:09] be explainable this is going to help us in many areas because if we want to expand the use of [16:16] large language model and make them more clear more predictable we need to to fine-tune them [16:24] This is our team. Uh this is a mixed work with Univers. Um I have my own consultancy in company [16:34] that is the AI consulting. Uh the paper was accepted in ACML 2026 to the sec in the second [16:43] workshop of compositional learning. So this is a a very huge achievement for us [snorts] [16:52] and if you want to connect with us to know more about my company about me [16:58] about what we are doing and if you need anything you can reach me in the link. Thank [17:04] you so much for your attention. Sorry that I got without there but thank you. Thank you so much.