Central Bank of Egypt Uses Knowledge Graphs to Curb AI Hallucinations

埃及中央银行利用知识图谱抑制AI幻觉 由于训练数据匮乏，大语言模型在面对区域性专业规则时易产生幻觉，企业级部署亟需结构化知识图谱来提供可验证、可解释的推理路径。 Basant Mounir 解释道，知识图谱将现实世界的数据表示为相互关联的实体与关系，在关系表力不能及之处，能够有效模仿人类逐步推理的过程以得出清晰、可解释的结论。她详细阐述了知识图谱如何作为提供事实依据的外部真实源泉，解决大语言模型在专业或区域性主题（例如小众经济分类系统）上的幻觉问题。结合埃及中央银行开发的监管AI助手 Ma'at.AI，她展示了一套端到端的实施框架：识别具体的知识盲区、选定权威来源手册、构建层级与横向关系模型、将模式导入 Neo4j Desktop，并通过专家人工评分衡量性能提升。 知识图谱将信息建模为通过显式关系相连的离散实体，直接映射了人类的认知推理与决策路径。 (0:48) 通过遍历相连的实体-关系路径，知识图谱能为机器得出特定结论提供显式且自洽的解释。 (0:48) 关系型数据库表专为扁平行和数学聚合而设计，在对多跳语义网络建模时显得僵化且不够适用。 (4:20) 大语言模型主要在训练数据代表性不足的专业或本地化主题上产生幻觉，宛如学生在考前仓促翻看课本。 (7:32) 知识图谱充当大语言模型的外部事实锚定大脑，以针对企业事实的确定性图遍历取代参数记忆中的猜测。 (7:32)

Basant Mounir, Senior Data Scientist at the Central Bank of Egypt, argues that knowledge graphs provide an explainable reasoning structure that resolves large language model hallucinations on specialized domains. Drawing from her work on the regulatory assistant Ma'at.AI, she outlines an operational blueprint for identifying knowledge gaps, modeling relational entities, and measuring performance uplift. Modeling Step-by-Step Reasoning Beyond Relational Tables (0:48) Knowledge graphs structure information as discrete entities (nodes) connected by explicit relationships (edges), mirroring human cognitive processes. Using a personal scenario of selecting a concert to attend around speaking commitments and seat preferences, Mounir illustrates how traversing linked statements produces both a logical outcome and an explainable rationale for why a decision was reached. Relational database tables, by contrast, are engineered for flat rows, foreign-key links, and numerical aggregations like sums and averages. Mounir characterizes tabular formats as rigid and fundamentally ill-suited for multi-hop semantic traversal across interconnected, real-world objects. Grounding Large Language Models on Specialized Knowledge (7:32) Although knowledge graphs have existed for decades, Mounir attributes their current resurgence to the need to mitigate Large Language Model (LLM) hallucinations. Drawing an analogy to a student who masters ten widely read books but skims an eleventh right before an exam, she notes that LLMs answer well on heavily documented global topics but fabricate answers on localized, specialized domains. Querying an LLM on international historical events yields accurate results due to broad representation across training corpora, whereas inquiries about niche economic structures cause hallucinations. External knowledge graphs address this gap by acting as a fact-grounding layer, enabling models to retrieve verified entities and edges from private or specialized datasets rather than relying on parametric memory. Identifying Domain Gaps and Modeling Hierarchical Taxonomies (12:17) To build an effective graph, Mounir highlights the Central Bank of Egypt's proof of concept for Ma'at.AI, an AI research assistant designed for legal and regulatory tasks. Implementation begins by benchmarking a baseline LLM to isolate where knowledge drops off: international standards generally perform well and regional frameworks moderately, while localized jurisdiction frameworks show high failure rates. Teams then select authoritative, comprehensive documentation to populate the graph without complex scraping. In Ma'at.AI's evaluation of the Hong Kong Standard Industrial Classification (HSIC), the data model was mapped directly from an official manual. The resulting schema captures four hierarchical tiers—from Section (Level 1) down to Level 4—linked vertically by derivation rules and laterally through explicit inclusion and exclusion relationships across domestic and international industry classes. Tooling, Ingestion, and Quantifying Graph Uplift (17:57) For graph construction and visual querying, Mounir recommends Neo4j Desktop, which ingests cleaned tabular data like CSV files straight into the predefined schema. The platform supports local development as well as remote hosting, accommodating the strict security perimeters required for sensitive central bank records. Validation requires benchmarking the graph-augmented model against the unaugmented baseline using domain-specific questions. Domain experts score both outputs on a 1-to-5 scale to quantify the performance uplift and confirm that the system provides sufficient traceability for operational deployment. Mounir emphasizes that teams should not pursue complete factual perfection when building knowledge graphs. Because conceptual blind spots will always remain, the operational objective is narrowing the domain gap to a level that is good enough to deliver reliable, verifiable answers in production.

埃及中央银行高级数据科学家巴桑特·穆尼尔（Basant Mounir）指出，知识图谱通过节点与关系的显式网络模拟人类多步可解释推理，可作为外部事实基座弥合大语言模型在专有与本土化领域的幻觉缺陷。结合埃及央行监管AI项目Ma'at.AI的实践，构建图谱需经历界定知识缺口、提炼权威参考源、形式化层级建模以及通过专家评分量化提升效果的全流程。 人类思维模拟与传统关系数据库局限 (0:48) 穆尼尔指出，人类大脑在复杂决策中通常遵循离散的单步因果逻辑。她以自身差旅中的看演出决策为例：根据艺术偏好（Bruno Mars或Lady Gaga）、演讲差旅日程（开罗、虚拟会议或拉斯维加斯）、巡演排期及票档权益（仅接受前排，即Platinum或Diamond票档），认知过程逐步排查多重约束，最终得出在拉斯维加斯购买Platinum票观看Bruno Mars的决策路径。知识图谱将现实世界抽象为离散实体（节点）与明确关系（边），使计算机能够沿着路径逐步遍历，生成透明且具有自解释性的决策依据。 相比之下，穆尼尔认为传统关系型数据库表在表达复杂、多跳语义网络时存在结构性刚性。关系表的设计专长在于处理扁平单层记录、外键关联以及求和、均值等纯数学统计聚合，但在跨越多个层级的真实世界实体之间缺乏原生的语义表达能力。当应用场景依赖丰富的上下文多步推导时，图结构构成了更符合人类逻辑的自然底层模型。 抑制大模型幻觉与专有事实基座 (7:32) 尽管知识图谱已有数十年发展历史并在谷歌提出“事物而非字符串（things, not strings）”后广为人知，但大语言模型（LLM）的兴起为其带来了新的落地动力。穆尼尔以备考学生作比喻：若学生将十本书精读五次并完成作业，答题时便极具把握；若第十一本书在考前半小时才匆匆扫过，面对相关问题时便容易编造情节以求蒙混过关。大语言模型的机制与此完全一致：在面对“大萧条”等广泛记录于公开训练集的高曝光度事件时回答精准，但在面对极其专门或本土化的行业知识时，由于训练数据稀疏，模型极易产生严重幻觉。 知识图谱在此扮演了LLM外部事实接地（fact-grounding）扩展脑的角色。通过将LLM接入经过验证的专业知识图谱，系统不再依赖模型易失且脆弱的参数化记忆检索冷门事实，而是引导模型通过实体与关系路径检索真实基准数据，从而在私有企业数据与高度监管的专业场景中输出具有解释依据的确定性回答。 监管实践中的知识缺口与分类建模 (12:17) 穆尼尔结合埃及中央银行在研发法律与监管研究AI助手“Ma'at.AI”（以古埃及代表真理与平衡的神祇命名）的经验，阐述了概念验证（POC）的落地路径。首要任务是与领域专家合作建立评估基准以量化知识缺口。基线LLM测试表明，模型对全球通用框架应答良好，对欧盟等区域性框架准确率中等，而对香港经济活动分类系统等国家和地区级的本土化规范则出现严重幻觉。针对这一缺口，团队选取了官方权威、结构完备且无需复杂爬虫清洗的《香港标准行业分类》（HSIC）手册作为单一数据源。 在数据建模阶段，HSIC手册所定义的四级体系被直接转化为图谱Schema：顶层部门（Section/Level 1）逐级向下派生出第二级、第三级和第四级（Level 2至Level 4）。除了纵向树状派生（derives）关系外，各行业类别之间还通过“包含（includes）”与“排除（excludes）”构建横向交叉引用规则。手册中属于国际标准的通用部分与香港本地特色行业（如电子制造与贸易）并存，共同构成了支撑知识图谱运转的形式化本体架构。 Neo4j落地部署与专家量化评估 (17:57) 在数据导入与工程实现层面，穆尼尔推荐使用Neo4j Desktop进行本地原型设计和可视化建模。清洗后的标准CSV或图兼容格式数据可直接按Schema映射导入，供开发人员进行交互式拓扑图核查与图查询。对于中央银行等对合规要求极高的金融监管机构，该工具既支持本地隔离安装，也支持远程服务器协同部署，可确保受限监管数据和机构专有资产完全保留在安全边界之内。 构建图谱的闭环在于验证其弥合知识缺口的实际成效。评估机制将知识图谱增强后的算法输出与缺乏图谱的基线模型进行成对测试，由领域专家依据最初设计的测试集，在1至5分的标准化量表上对两者的回答质量进行打分。穆尼尔总结道，尽管知识图谱驱动的系统在专家评分和溯源证据链上表现优异，但实践哲学并非追求全知全能：“知识缺口永远存在，团队的目标是找到一个‘足够好’的缺口边界，掌握足以支持业务运行的充分知识即可。” 穆尼尔倡导的知识图谱构建方法强调从实际业务痛点出发，通过清晰界定LLM的本土化盲区、选取权威标准化源数据、形式化构建派生与排他关系，并在受控环境中量化专家提效收益。该技术路径的核心价值在于为高风险机构提供可验证、可审计且防幻觉的决策支持，而非片面追求消除所有认知不确定性。

## Background 该视频由 YouTube 频道 Ai4 于 2025 年 8 月 27 日发布，主讲人为来自埃及中央银行（The Central Bank of Egypt）的高级数据科学家 Basant Mounir。视频简介指出，本次演讲旨在系统拆解知识图谱的核心定义、在现代 AI 工作流中的关键价值，并结合埃及央行监管 AI 助手 Ma’at.AI 的真实法律场景，帮助从业者掌握识别落地场景与规划构建知识图谱的实用技能。 ## TL;DR 演讲围绕知识图谱的“是什么、为什么、怎么做”三大核心问题展开。主讲人首先借助生动的生活决策推演，阐明了图谱基于实体与关系建模的高语义丰富度与天然可解释性；随后指出大语言模型（LLM）因长尾垂直语料匮乏而极易产生幻觉，必须借助知识图谱作为“外挂大脑”进行事实锚定（Grounding）。最后，她以埃及央行 Ma'at.AI 的香港经济分类实践为例，系统拆解了从识别知识鸿沟、筛选权威数据源、Schema 建模，到导入 Neo4j 数据库以及专家打分闭环评估的完整工程落地方法论。 ## Atmosphere & style 整场演讲亲切幽默、深入浅出，主讲人以“出差巡讲是否该顺道看演唱会”的生活抉择开篇，瞬间拉近了前沿图论算法与普通听众的距离。随着议题从日常逻辑自然过渡至严谨的央行级监管 AI 实践，叙事风格在生动具象的比喻与务实克制的工程心法间自如切换，营造出兼具启发性与极高实操价值的技术分享氛围。 ## Key takeaways - [00:47](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=47s) 人类大脑做复杂决策时依赖离散事实与偏好的逐层串联，知识图谱正是模拟这种人类思维，让计算机通过实体与关系的网状连接完成链式逻辑推导。 - [04:11](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=251s) 知识图谱具备两大核心优势：一是具备能够表达深层网状逻辑的极高语义丰富度（Semantically Rich），二是每一步推理都有事实支撑的天然高可解释性（Highly Explainable）。 - [05:30](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=330s) 传统关系型数据库表格擅长数值求和、极值与计数等聚合操作，但在处理跨实体跳转与深层网状语义时显得笨拙，缺乏行与行之间的内生关联。 - [07:32](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=452s) 大语言模型产生幻觉的根本原因在于长尾与垂直领域知识在预训练语料中的低表征度，模型面对未充分学习的知识极易顺理成章地编造虚假故事。 - [10:08](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=608s) 在 LLM 时代，知识图谱的核心使命是充当大模型的“外挂大脑”，将模型生成锚定（Grounding）在客观事实之上，弥补通用模型在企业私有与垂直细分领域的盲区。 - [12:17](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=737s) 构建垂直图谱的首要步骤是识别“知识鸿沟”，通过与领域专家设计基准测试题，可以清晰发现模型在国际通用知识上表现极佳，但在本地细分规则上缺陷严重。 - [13:40](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=820s) 埃及央行在研发监管 AI 助手 Ma'at.AI 过程中，将模型频繁产生幻觉的“香港经济活动分类”作为典型切入点，针对性攻坚本地化知识盲区。 - [15:07](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=907s) 筛选图谱数据源应综合权衡易访问性、易解析度、内容完整度与权威性，结构严谨的官方分类手册远优于杂乱难抓取的非结构化新闻网页。 - [16:20](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=980s) 图数据建模（Schema 设计）不仅需要构建多层级的实体派生关系，还需明确提炼出“包含（Includes）”与“排除（Excludes）”等关键逻辑交互规则。 - [17:57](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=1077s) 选用 Neo4j 等现代图数据库不仅支持图模式的可视化探索与 Cypher 查询，还能在本地断网环境下部署，保障高密金融数据的安全性。 - [18:25](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=1105s) 评估图谱引入价值需组织领域专家对原生基线模型与图谱增强模型的回答进行标准化双盲打分，以此量化准确度与可用性的实质提升。 - [18:45](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=1125s) 知识图谱落地的工程心法是追求“足够好”的知识收窄边界——不盲目追求全知全能，而是精准收敛并弥补业务痛点所需的关键事实。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=4s) 开场与个人背景介绍 来自埃及央行（Central Bank of Egypt）的高级数据科学家 Basant Mounir 开启了本次关于知识图谱（Knowledge Graphs）的演讲。她幽默地表示，这是她唯一一场不需要列议程目录（Agenda）的分享，因为演讲的标题——“知识图谱：是什么、为什么以及怎么做（What, Why, and How?）”本身就已经把内容交代得一清二楚了。 Basant 介绍了自己的专业背景：她来自埃及，不仅在埃及央行担任高级数据科学家，还在 MIT 的面向职场人士的应用数据科学项目（MIT's Applied Data Science Program）中担任导师。在过去的大约六个月里，她一直在深度研究并实践图技术。她坦言，在起步阶段，想要找到系统性入门并掌握如何搭建知识图谱的优质资料并不容易。因此，她希望将自己摸爬滚打学到的经验提炼出来，带给所有对这一领域感到好奇的听众，直面回答三个核心问题：知识图谱到底是什么？为什么现在每个人都在谈论它？以及我们究竟该如何着手构建一个知识图谱？ - **主讲人身份**：Basant Mounir，埃及央行高级数据科学家、MIT 应用数据科学项目导师。 - **研究经历**：在过去六个月中专注于图结构与相关研究（Research Graphs/Knowledge Graphs）。 --- ### [00:47](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=47s) 推演实录：我该在出差巡讲时去看演唱会吗？ 为了通俗直观地讲透“知识图谱究竟是什么”，Basant 并没有直接抛出晦涩的数学图论概念，而是拿自己这次出差途中真实纠结的一个生活抉择作为引子——“我应该在这次外出巡讲期间去顺便看一场演唱会吗？”她邀请台下观众一起跟着她进行“出声思考”（thinking out loud），观察人类大脑在做决策时一步一步的推理过程。人类大脑在回答复杂问题时，并不是一下子跳出终极答案的，而是“一个陈述接着一个陈述（one statement at a time）”，抽丝剥茧地召回相关信息。 她首先把这个决策拆解成几个具体的前提条件： 1. **喜好条件**：“既然要花时间和精力去演唱会，那必须是我喜欢的歌手。”她最喜欢的歌手是 Bruno Mars 和 Lady Gaga。 2. **日程与地点**：看演唱会必须和出差行程吻合。Basant 梳理了自己的演讲日程：她要在埃及开罗举办的 Engineer X 大会演讲；还要在 DSC（Data Science Conference）演讲，不过那是线上的（Virtual），地点无关紧要；最后则是今天正在参加的这场大会——在拉斯维加斯举办的 AI4。 3. **演出安排匹配**：歌手们都在哪里开唱？Lady Gaga 会在纽约演出（只能错过了）；而 Bruno Mars 不仅在纽约演出，巧合的是他刚好也在拉斯维加斯开演唱会。 4. **票务与体验要求**：Basant 强调，如果要去看，她绝不想马虎，必须坐“第一排（front row）”。Lady Gaga 的门票分为钻石票（Diamond）和黄金票（Gold）；而 Bruno Mars 提供的是黄金票和白金票（Platinum）。进一步挖掘票务座位规则发现：黄金票安排在中间（Middle），白金票和钻石票则能坐在第一排。 ```mermaid flowchart TD accTitle: 演唱会决策推理链路图 subgraph SG1["行程与偏好"] Me["Basant (我)"] -->|"喜欢"| BM["Bruno Mars"] Me -->|"出差演讲于"| AI4["AI4 大会 (拉斯维加斯)"] end subgraph SG2["演出与场地"] BM -->|"举办演出地点"| LV["拉斯维加斯"] BM -->|"提供票务"| Plat["白金票 (Platinum)"] end subgraph SG3["观演体验"] Plat -->|"座位安排"| Front["第一排 (Front Row)"] end AI4 -.->|"同城匹配"| LV ``` 当把这些离散的想法、事实串联成一条逻辑链条（String of thoughts）时，答案自然浮现：Basant 应该去拉斯维加斯看 Bruno Mars 的演唱会，并且购买白金票。更妙的是，这种图式推导不仅仅给出了“Yes”的结论，还附带了完整且无懈可击的因果解释——因为我喜欢 Bruno Mars、我刚好在拉斯维加斯演讲、他刚好在此地演出、白金票刚好能让我坐到第一排。 --- ### [04:11](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=251s) 知识图谱的本质与优势：语义丰富度与高解释性 Basant 由此引出了知识图谱的本质定义：在现实世界中存在着各种实体（Entities），例如“我”、歌手、会议、城市、门票等；实体与实体之间存在着复杂的互动与关系（Relationships），比如“我喜爱 Bruno Mars”、“大会在拉斯维加斯举办”。知识图谱正是一种对真实世界进行高度抽象与建模的工具。 大脑在思考问题时，是在联想并串联各个想法；而计算机通过知识图谱，也能以这种类似人类思维的方式进行推理（Reasoning），顺着实体间的关系网络一步一步检索信息。这种机制赋予了知识图谱两大核心优势： - **语义极度丰富（Semantically Rich）**：当业务场景跨越多个实体、包含多层跳转逻辑时，知识图谱能够天然表达网状关系。 - **天然的可解释性（Highly Explainable）**：相比于许多“黑盒（Black Box）”算法直接甩出一个预测结果，基于知识图谱的推理路径清晰可见，每一步都有明确的事实陈述作为支撑。 谈到这里，Basant 顺带对比了传统的关系型数据库表格（Tables）。为了做实验，她曾尝试用表格来存储上面那套演唱会决策数据，结果发现“丑陋不堪（really hideous）”，以至于她在幻灯片里特意把表格背景虚化调淡。表格本质上假定数据之间没有复杂的深层语义，擅长处理的是一对一、一对多关系，以及求和（Summing）、取极值（Max）、计数等聚合（Aggregation）统计操作；但在表格的各行之间并没有内生性的关联。一旦面对需要沿着实体链条层层探索的复杂语义场景，知识图谱才是正确的工具。 - **图的核心构成**：实体（Entities，即节点）与关系（Relationships，即连线）。 - **表格的局限**：缺乏行间内在语义，擅长数值聚合与简单关联，难以优雅表达深层网状逻辑。 --- ### [07:32](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=452s) 为什么知识图谱在 LLM 时代重新爆火？ Basant 回顾道，知识图谱其实早在上个世纪就已经存在，并不是什么新鲜发明。在 2012 年左右（演讲中口误提到 2014 或 2016 年），Google 提出了著名的口号“Things, not strings（是事物，而不是字符串）”，推动了现代知识图谱的一次爆发。那为什么在当下这个节点，知识图谱又重新成为了全行业热议的风口？答案毫无疑问指向了大型语言模型（LLMs）。 为了讲明白两者的结合点，Basant 首先剖析了 LLM 的学习机制与局限性。AI 实验室在预训练模型时，会喂给它海量的通用互联网数据（通用常识、历史事件、智力谜题、《纽约时报》等公开语料）。但这种训练方式存在一个致命痛点——长尾知识与冷门专业数据的严重匮乏。 她打了一个极其生动的期末考试比方：假设你要备考一门课，考试范围一共有 11 本书。前面的 10 本书你反复读了五遍，做了演讲报告，还认真完成了所有作业；但是第 11 本书你一直找不到，直到考前 30 分钟你才好不容易拿到，匆匆扫了一眼（skimmed it very quickly）。走进考场后会发生什么？ - **面对前 10 本书的内容**：你成竹在胸，答题详尽且令人惊艳，能拿超高分。 - **面对最后那本书的内容**：你的记忆非常模糊，但作为人类，你为了不交白卷，很可能会开始胡编乱造一些听起来头头是道的“漂亮故事”，试图去忽悠助教（TAs）。 大语言模型也是一模一样的道理。如果你问大模型“1929 年大萧条（Great Depression）发生了什么”，这是一个举世瞩目的历史大事件，语料库里有无数材料，它能回答得滴水不漏；但如果你问它非常具体、晦涩的专业领域问题——例如“香港经济活动分类详情（Classification details for Hong Kong economic activities）”，这主要是经济学家才会关注的冷门领域，在训练集中的数据表征度极低（Low representation）。模型由于见得太少，回答时就极易像考前突击的考生一样，开始煞有介事地产生幻觉（Hallucination）与瞎编。 ### [10:08](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=608s) 知识图谱：弥补大语言模型幻觉的“外挂大脑” 大语言模型（LLM）在面对缺乏训练数据的长尾、专业或本地化话题时，往往会给出浮于表面的回答甚至产生严重的“幻觉”（Hallucination）。尽管近期的某些工具（例如 Perplexity）开始学会回答“请查阅手册”，这种保守策略算是一种进步，但本质问题依然存在：训练数据中对某一领域的覆盖度越低，模型产生幻觉的概率就越大。LLM 掌握的是通识性的世界观——它熟知大众话题中的通用实体与关系，但无法凭空知晓那些未被充分记录的内容。 引入知识图谱（Knowledge Graph）正是为了弥补这种世界观上的盲区。通过为大模型在训练期间未曾大量接触的领域构建专属知识图谱，图谱便成为了 LLM 的“外挂大脑”（Extension of the Brain）。以香港的经济分类为例，当向模型提问相关问题时，它不再靠臆测编造，而是借助图谱检索实体与关系，沿着图中的思维路径（Path in the Graph）进行逻辑推理，并输出具备事实依据且高度可解释（Explainable）的答案。这正是知识图谱当下备受瞩目的核心价值：将大语言模型锚定在真实事实之上（Grounding in facts）。 无论是企业内部绝密的机密数据，还是极其垂直的专业领域，LLM 原生模型都不可能事先获得这些数据。知识图谱搭建了一座桥梁，它打破了通用 LLM 的局限，让模型能够跨越到制造者尚未涉足的特定现实世界中，实现从“通才”向“专精专家”的质变。 - 幻觉根源：训练数据中话题的表征度与大模型的幻觉概率成严格反比。 - 核心功能：知识图谱为 LLM 提供事实锚定、可解释的推理路径与结构化记忆扩展。 - 适用场景：企业机密数据利用、高度垂直细分行业、缺乏公开网络语料的特定知识领域。 --- ### [12:17](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=737s) 实战框架：识别知识鸿沟与埃及央行 Ma'at.AI 的法律用例 为了帮助从业者判断知识图谱是否契合自身业务场景，演讲者根据数月来的实践提炼了一套方法论框架，并结合埃及央行正在研发的法律研究 AI 助手“Ma'at.AI”（“Ma'at”在古埃及语中意为“法律”或“真理”）中的真实任务展开说明。该项目的关键任务之一，是解答与本地经济活动及监管分类相关的具体问题。 构建流程的第一步是“识别知识鸿沟”（Identify the Knowledge Gap）。团队必须明确：大语言模型到底缺失了什么？需要补足哪些内容才能遏制幻觉？标准做法是与领域专家共同设计一套包含不同难度梯度的基准测试题集，用以覆盖实际业务场景。随后，利用原生基线模型（Vanilla LLM）进行测试，再由专家评估其回答质量。 在对经济分类体系的测试中，模型回答的质量呈现出明显的地域层级差异：对于“国际分类系统”（International Classification System），模型表现极佳，因为全球讨论广泛、互联网语料极其充沛；对于“欧盟”（EU）等区域级系统，模型回答尚可，但未达到专家认可的业务深度；而一旦下沉到香港等本地层级（Hong Kong's Economic System），由于公开语料稀缺，模型便开始大范围幻觉。由于概念验证（PoC）阶段不可能穷尽全球所有知识，团队最终选定“香港经济分类”作为典型切入口，以此攻坚本地化知识盲区。 ```mermaid flowchart TD accTitle: 知识图谱构建全流程 accDescr: 展示从识别差距到最终评估的五个核心阶段 subgraph Step1["阶段一：差距与数据"] A["定义评估问卷<br/>(与领域专家共创)"] --> B["评估基线模型 (LLM)<br/>(国际 vs 区域 vs 本地)"] B --> C["识别知识鸿沟<br/>(聚焦本地知识缺失)"] C --> D["筛选权威数据源<br/>(全面性/易解析/权威度)"] end subgraph Step2["阶段二：建模与导入"] D --> E["定义数据模型<br/>(实体层级/包含/排除关系)"] E --> F["导入 Neo4j Desktop<br/>(清洗为 CSV / 图模式构建)"] end subgraph Step3["阶段三：效果评估"] F --> G["对比专家打分 (1-5分)<br/>(基线 LLM vs KG 增强模型)"] G --> H{"差距是否收窄至可用？"} H -- 是 --> I["上线/拓展应用"] H -- 否 --> E end ``` --- ### [15:07](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=907s) 数据源甄选与经济活动数据建模（Data Modeling） 确定了知识鸿沟之后，第二步是“识别所需数据”（Identify the Needed Data）。专家在面对此类业务时通常会参考三种材料：政府发布的标准工业分类手册（例如《香港标准行业分类》）、官方新闻通稿，以及专家主观意见。在资源有限的 PoC 阶段，团队必须依据可获取性（Easy to Access）、处理成本（Easy to Parse）、覆盖完整度（Comprehensive）以及权威性来权衡取舍。新闻通稿需要编写网络爬虫解析非结构化网页，而官方分类手册则是一份完整、标准、详尽阐述了香港所有市场经济活动的单一参考文档，因此顺理成章地被选为最佳数据源。 第三步是“抽取实体与定义关系”（Identify Entities + Relationships），即设计知识图谱的 Schema（数据模型，Data Model）。《香港标准行业分类手册》天然具备严密的结构：每个条目代表一项经济活动，附带代码，分为从 Level 1 到 Level 4 的四级层级结构（从最高层级的 Section 一路向下派生）；同时包含中英文的双语详细描述，以及明确标注的“包含”（Includes）与“排除”（Excludes）规则。手册中浅灰色部分代表沿用国际标准的通用行业（全球一致），其余部分则体现了香港特有的产业特色（例如电子行业）。 - 数据源筛选四准则：易访问性、易解析性（结构化程度）、内容全面性、专家参考依赖度。 - 核心实体与层级：大类（Section/Level 1）、中类（Level 2）、小类（Level 3）及细类（Level 4）。 - 关系定义：层级派生关系（Derives）、跨类别的逻辑交互关系（如包含 `Includes` 与排除 `Excludes`）。 --- ### [17:57](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=1077s) 图数据落库（Neo4j）、知识增益评估与工程心法 在完成数据清洗并导出为 CSV 等友好格式后，第四步是将数据导入图数据库。演讲者强烈推荐使用 Neo4j Desktop 工具，它支持直观的可视化图模式设计，无论是连接远程服务器还是处理本地敏感保密数据，都能提供开箱即用的图谱导入、交互式可视化检索与 Cypher 语句查询体验。 最后一步是“评估知识增益”（Evaluate the Knowledge Gain）。知识图谱的引入是否真正收窄了模型的知识鸿沟？团队通过最初制定的专家题集，让领域专家以 1 到 5 分的打分标准，分别对基线 LLM 的回答与经图谱增强（Graph-augmented）模型的回答进行双盲打分，计算效果提升（Uplift）。结果表明，图谱增强模型的得分显著优于基线模型。 针对落地心态，演讲者强调了务实的工程思维：“世界上永远会存在知识盲区，你的目标并不是穷尽一切，而是找到一个‘足够好’的知识收窄边界（A knowledge gap that's good enough）——你永远不可能无所不知，但你的模型可以掌握足够多的必要知识。”演讲最后展示了一份梳理上述完整方法论的速查表（Cheat Sheet），并提供了 LinkedIn 二维码以供同行深入交流。 | 评估维度 | 原生基准模型 (Vanilla LLM) | 知识图谱增强模型 (KG-augmented) | |:--- |:--- |:--- | | **国际通用知识** | 极佳（全球语料海量支持） | 极佳 | | **本地细分知识** | 极差（语料匮乏导致频繁幻觉） | 优秀（锚定行业手册结构化知识） | | **推理过程** | 概率黑盒预测，不可解释 | 基于图路径与实体关系，完全可解释 | | **专家评分标准** | 1 至 5 分（分数偏低，浮于表面） | 显著提升，达到业务上线许可阈值 | ### [19:36](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=1176s) 演讲结语与致谢 在系统性地阐述了知识图谱的核心价值、在现代 AI 工作流中的构建方法，以及埃及中央银行（The Central Bank of Egypt）基于监管 AI 助手 Ma'at.AI 的落地实践后，主讲人 Basant Mounir 以一句诚挚的“Thank you so much”（非常感谢大家）向全场观众致谢，正式结束了本次主题演讲。 ## Notable quotes > [00:47](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=47s) 一个陈述接着一个陈述（one statement at a time） — Basant Mounir > [07:32](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=452s) Things, not strings（是事物，而不是字符串） — Basant Mounir > [07:32](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=452s) 面对前 10 本书的内容：你成竹在胸，答题详尽且令人惊艳，能拿超高分。 — Basant Mounir > [07:32](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=452s) 面对最后那本书的内容：你的记忆非常模糊，但作为人类，你为了不交白卷，很可能会开始胡编乱造一些听起来头头是道的“漂亮故事”，试图去忽悠助教（TAs）。 — Basant Mounir > [10:08](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=608s) 训练数据中对某一领域的覆盖度越低，模型产生幻觉的概率就越大。 — Basant Mounir > [10:08](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=608s) 将大语言模型锚定在真实事实之上（Grounding in facts）。 — Basant Mounir > [12:17](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=737s) 识别知识鸿沟（Identify the Knowledge Gap）。 — Basant Mounir > [15:07](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=907s) 识别所需数据（Identify the Needed Data）。 — Basant Mounir > [15:07](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=907s) 抽取实体与定义关系（Identify Entities + Relationships） — Basant Mounir > [17:57](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=1077s) 世界上永远会存在知识盲区，你的目标并不是穷尽一切，而是找到一个‘足够好’的知识收窄边界（A knowledge gap that's good enough）——你永远不可能无所不知，但你的模型可以掌握足够多的必要知识。 — Basant Mounir > [19:36](https://www.youtube.com/watch?v=y_N0KZu2DnM&t=1176s) Thank you so much — Basant Mounir ## Mentioned Basant Mounir, 埃及中央银行 (Central Bank of Egypt), MIT (麻省理工学院), Ma'at.AI, Bruno Mars, Lady Gaga, AI4, Engineer X, DSC (Data Science Conference), Google, Perplexity, Neo4j Desktop, 香港标准行业分类 ## Video description Basant Mounir, Senior Data Scientist, The Central Bank of Egypt Abstracts Knowledge graphs are gaining renewed attention alongside LLMs, but what’s driving this momentum? This talk breaks them down clearly: what knowledge graphs are, why they matter in modern AI workflows, and how you can begin building them. To solidify our learnings, we’ll walk through a hands-on legal use case from Ma’at.AI, a regulatory AI research assistant developed at the Central Bank of Egypt. You’ll acquire two foundational skills: (1) recognizing opportunities where knowledge graphs can provide value, and (2) planning their development effectively. If you’ve been curious but haven’t had the time to explore, this session is your on-ramp. [Source](https://www.youtube.com/watch?v=y_N0KZu2DnM)

[0:47] Basant Mounir: 一个陈述接着一个陈述（one statement at a time） [7:32] Basant Mounir: Things, not strings（是事物，而不是字符串） [7:32] Basant Mounir: 面对前 10 本书的内容：你成竹在胸，答题详尽且令人惊艳，能拿超高分。 [7:32] Basant Mounir: 面对最后那本书的内容：你的记忆非常模糊，但作为人类，你为了不交白卷，很可能会开始胡编乱造一些听起来头头是道的“漂亮故事”，试图去忽悠助教（TAs）。 [10:08] Basant Mounir: 训练数据中对某一领域的覆盖度越低，模型产生幻觉的概率就越大。 [10:08] Basant Mounir: 将大语言模型锚定在真实事实之上（Grounding in facts）。 [12:17] Basant Mounir: 识别知识鸿沟（Identify the Knowledge Gap）。 [15:07] Basant Mounir: 识别所需数据（Identify the Needed Data）。 [15:07] Basant Mounir: 抽取实体与定义关系（Identify Entities + Relationships） [17:57] Basant Mounir: 世界上永远会存在知识盲区，你的目标并不是穷尽一切，而是找到一个‘足够好’的知识收窄边界（A knowledge gap that's good enough）——你永远不可能无所不知，但你的模型可以掌握足够多的必要知识。 [19:36] Basant Mounir: Thank you so much

[7:32] Google: things, not strings [7:32] Basant Mounir: make up some stuff and have a nice story [18:34] Basant Mounir: There will always be a knowledge gap, but the goal is to find a knowledge gap that's good enough. You will never know everything, but you can know enough.

[0:04] So, we're going to be talking about [0:05] knowledge graphs. What, why, and how? I [0:07] think that's the only talk I don't need [0:09] to make an agenda for because the title [0:10] is pretty self-explanatory. [0:13] So, I want to give you a quick [0:14] introduction about me. So, my name is [0:16] Basant. I'm a senior data scientist at [0:18] the Central Bank of Egypt. Yeah, I come [0:20] from Egypt and I'm an educator for MIT's [0:23] applied data science program for working [0:25] professionals. And I've been working [0:28] with research graphs for the past six [0:30] months or so. And it hasn't been easy [0:31] finding information and how to get [0:33] started. So I hope to give you this [0:35] still what I learned to you and answer [0:38] the three questions. What are graphs? [0:40] Why is anyone everybody talking about [0:42] them? And how do we go about starting to [0:45] build one? [0:47] All right. So the first one is what is [0:48] our knowledge graphs? [0:51] So this is a question I actually [0:52] considered you know my way here. Should [0:54] I attend a should I attend a concert [0:56] while I travel for speaking? So, I'm [0:59] gonna be thinking out loud and I want [1:01] you to pay attention to my reasoning [1:03] process. [1:04] So, first of all, of course, for me to [1:06] attend the concert, it has to be a [1:07] singer I like, right? I'm not going to [1:09] invest my time and energy in a concert I [1:11] don't enjoy. So, what singers do I like? [1:16] So, I like Bruno Mars. I love Bruno Mars [1:18] and I love Lady Gaga. [1:21] So [1:23] this is the should I part right because [1:25] if it's someone I love I will probably [1:26] want to attend. Now I know who I like [1:30] but [1:31] where will I travel? What's my speaking [1:33] and travel schedule right? So [1:38] I will be speaking at Engineer X which [1:39] is hosted in Egypt Cairo. I'll be [1:42] speaking at DSC which is virtual so [1:44] location isn't relevant here. And [1:45] finally today I'm speaking at AI4 which [1:48] is in Las Vegas. Right. So that being [1:53] said, I know who I love. I know where [1:55] I'm going to go. So it's natural next to [1:58] think about [2:00] where the singers will go, right? Where [2:02] will they perform? So let's see the [2:05] attend the concert part. So as you can [2:07] see, I'm taking the question bit by bit. [2:08] I'm trying to recall relevant [2:10] information one thought at a time. So [2:12] one statement at a time. Our brain [2:14] thinks in what statement at a time to [2:17] answer the question fully. Okay, part by [2:20] part. [2:23] So, Lady Gaga is going to perform in New [2:24] York. Too bad. I'm going to miss her. [2:26] She accepts diamond tickets and gold [2:28] tickets. Okay. [2:31] Bruno Mars is going to also perform in [2:33] New York. However, he's going to perform [2:34] in Las Vegas, too. So, analyzing what [2:38] I'm remembering so far, it seems like [2:40] Bruno Mars could be a good fit so far. [2:42] Right. The final part of the question. [2:44] Oh, yeah. He offers golden uh platinum [2:47] tickets. The final part of the question [2:49] is it in the front row? If I want to go [2:51] to a concert, I really want to enjoy it. [2:52] So, it needs to be in the front row. So, [2:55] let's see now the concerts, the tickets. [2:59] What seats do they offer? [3:03] So, gold sits he seats you in the [3:06] middle, platinum in the front, and [3:08] diamond in the front. So now we have [3:11] everything we need to answer the [3:12] question who I love, where I'm going to [3:14] be, where are they going to be, and [3:16] whether or not I'll get to sit in the [3:18] front row. So these are my thoughts. So [3:21] what is the answer to the question? We [3:24] analyzing the thoughts, the related [3:26] thoughts to arrive at the answer, right? [3:31] So, I want you to look at this the these [3:33] thoughts and find the string of [3:38] statements or the string of thoughts [3:39] that answers the question. [3:44] It's this one. So, if you if you thought [3:46] about this one, you're right. So, this [3:49] answers the question. And the best part [3:51] about this is that it doesn't just [3:53] answer the question, it explains the [3:55] answer. So yes, I should go for Bruno [3:57] Mars concert, platinum ticket, because I [4:00] love Bruno Mars. I'm going to be [4:02] speaking in the same concert, Las Vegas, [4:04] where he'll host his concert and I get I [4:06] get a platinum ticket to sit in the [4:08] front row. [4:11] So as you can see, reasoning is a string [4:14] of thoughts and every thought is one [4:16] statement. [4:20] So what we call this you know collection [4:23] of entities entities being for example [4:26] me the singers uh the locations [4:30] uh the the conferences we call those [4:32] entities and we're studying how they [4:34] interact in the world what are the [4:35] relationships among the entities for [4:37] example I love Bruno Mars so I'm [4:40] performing an action which is love for [4:42] Bruno Mars so that's what knowledge [4:44] graphs are about they try to model the [4:46] world you can very abstractly think [4:48] worlds are you know entities and [4:51] interactions between them you can think [4:53] we can simplify our view of the world [4:54] like that so I'm trying to capture [4:57] information in the world that will be [4:58] able to answer my questions or help me [5:00] think in this case the kinds of [5:03] information entities I needed were the [5:05] concerts the singers the tickets and so [5:07] on and of course the possible [5:09] relationships between them okay so a [5:12] knowledge graph helps you uh find all of [5:17] the possible thoughts Right? It links [5:19] the possible thoughts and it helps you [5:21] based on these linked thoughts make a [5:23] decision or answer a question. So this [5:26] is actually it it kind of functions like [5:28] our brain you know when we're we're [5:30] recalling thoughts and then linking them [5:31] together to answer the question. So this [5:33] can be used by computers right we can [5:35] give them this knowledge graph and they [5:36] will be able to reason with it you know [5:39] remember one thought at a time until it [5:42] has all the relevant information to [5:43] answer the question. So that's the power [5:45] of knowledge graphs. It simulates the [5:47] world and our reasoning process. Parsing [5:50] it is like our reasoning process in a [5:52] sense. Okay. [5:54] All right. So when do we use knowledge [5:56] graphs? So that what knowledge graphs [5:58] are a network of entities and [6:00] relationships to model the world that [6:02] also can be used to think for computers. [6:06] So when can we use the knowledge graphs? [6:08] So as you can see here this question was [6:10] very semantically layered, right? We had [6:13] to explore several relationships and [6:15] then arrive at the answer. So knowledge [6:17] graphs are great for any semantically [6:19] rich application. [6:21] And of course, as you can see here, [6:22] they're highly explainable. So maybe [6:24] you're asking an algorithm and it's a [6:25] black box. So you don't know the the [6:27] answer the explanation, right? But [6:29] knowledge graphs are very [6:30] straightforward. As you can see here, [6:33] this is the question and these are the [6:34] thoughts that happened one by one for me [6:37] to reach the final answer. [6:39] I want to go back and touch on tables. [6:42] So I tried to model tables uh this [6:44] information in tables and it's it's [6:45] really hideous. I don't want to show you [6:47] but it's faded in the background for [6:49] that reason. Um the thing with tables is [6:52] that they assume no complex semantics [6:55] right as we all know tables are one to [6:57] one one to many. There isn't an inherent [7:00] meaning in the relationships between [7:01] tables right and the objects in the [7:04] tables. So the rows they don't have [7:06] relationships among each other. So [7:07] tables are great for simple operations [7:09] like aggregations, summing, getting the [7:12] maximum value for these kind of [7:14] operations. So aggregations and very [7:16] simple basic linking. But when I need to [7:18] see entity linked to another entity and [7:20] the relationships it can have with other [7:22] entities and so on and so forth to [7:23] answer my question, then I'm having a [7:25] semantically rich problem. So knowledge [7:27] graphs are a good uh uh tool to try. [7:30] Okay. [7:32] So that is the what. Now why are people [7:35] talking about knowledge graphs? So [7:37] knowledge graphs have actually been [7:38] around for a while since the last [7:40] century and they've become very popular [7:42] in 2014 or 16 I'm not sure when Google [7:45] published the paper knowledge graphs [7:48] things not strings. [7:50] So why are we are they getting traction [7:52] now? So of course it has to do with LMS [7:54] as you might have guessed. [7:57] So let's first think how do LLMs uh [7:59] learn. So the labs curate the data [8:05] data general data right so maybe general [8:08] knowledge uh past events you know [8:10] reasoning some popular problems like [8:11] puzzles or maybe connections in the New [8:13] York Times and so on and then we give [8:16] this information to the LLM and it [8:18] learns and then it's able to answer your [8:20] questions [8:22] but what happens I want you to imagine [8:24] what happens if I give you if you're [8:26] studying for an exam right and you have [8:29] 11 books to study [8:31] 10 of them. You read them five times and [8:34] you present about them and you have [8:35] assignments about them and then one you [8:38] couldn't find the missing book. You [8:40] cannot you cannot find that missing book [8:42] and then you find it just 30 minutes [8:43] before the exam and you skim it very [8:45] quickly. How are you going to answer the [8:48] questions about the first 10 books [8:50] versus the last book? So the first book [8:52] you're going to be very confident. Yeah, [8:53] I know that. I've studied that very [8:55] well. I know how to how to answer these [8:58] questions. is your answer are going to [8:59] be complete impressive. You're going to [9:00] get amazing grades. But then, oh, okay. [9:04] So, I need now to answer questions about [9:05] the last chapter. You barely remember [9:08] the words that were there. But, you [9:10] know, we're humans. So, we know we can [9:13] make up some stuff and have a nice story [9:15] to convince the TAs of our answers, [9:17] right? But we're not sure because at the [9:19] end of the day, we just saw the uh [9:21] chapter very superficially, right? And [9:23] when you see something superficially, [9:24] the less the more superficial you go [9:27] through something, the more likely you [9:29] will make up your answers. So I think [9:32] we've all done that at some point. So [9:34] the LMS are pretty much the same thing. [9:36] So when you ask an NLM, what happened in [9:38] the Great Depression? So that's a very [9:39] notable event. We all know about it. [9:42] It's it's probably going to answer you [9:43] well. It has a lot of sources. Okay. [9:46] However, what if I ask it about [9:48] classification details for the Hong Kong [9:50] economics uh mark like activities? Very [9:53] specific. Maybe you didn't even [9:54] understand it because actually it's econ [9:56] economists who care about this [9:58] information. And because this is a very [10:00] specific question, it has very low [10:02] representation in the training data. And [10:05] because it has low representation, the [10:06] LLM doesn't see it as much as the other [10:08] topics. So when I ask questions about [10:10] it, it hallucinates. It gives [10:12] superficial answers and it hallucinates. [10:15] though perplexity recently has been [10:17] saying you know what's referred to the [10:18] manual. It's an improvement but you get [10:21] the point. The less you see some kind of [10:23] data source the more likely the LLM will [10:25] hallucinate. So the less your data the [10:28] less your topic is represented in the [10:29] training data the more likely your model [10:31] is going to hallucinate. And of course [10:33] that's one of the major problems with [10:34] LLMs. So LLMs have a view of the world a [10:38] general view right they know the [10:39] entities and the relationships among [10:41] popular general things in the world. So [10:43] how can we complement this world view [10:47] with knowledge graphs? Okay. So we [10:49] created knowledge graphs about the [10:50] topics that the didn't get to train on [10:52] much data from and it becomes an [10:54] extension of what they already know. So [10:56] they know the general information plus [10:58] some you know economics [11:00] of Hong Kong. So when you ask it a [11:02] question about Hong Kong's economics [11:04] it's going to be able to refer to the [11:05] knowledge graph as extension of the [11:07] brain. It's going to be able to reason [11:09] with it with the entities and the [11:10] relationships, reason with a string of [11:12] thoughts or a path path in the graph and [11:15] give you the answer and it's explainable [11:16] best of all. So knowledge graphs are [11:20] helping you ground your LLM in facts [11:22] when it comes to topics that it doesn't [11:24] know enough about and that's why they're [11:27] that's why they're very popular right [11:28] now. [11:30] Okay. So finally, so we know what what [11:32] knowledge graphs are. We know why [11:34] everybody's talking about them because [11:35] basically they're an extension of the [11:37] brains of LLMs. You can think of it like [11:39] that. You can extend it in very creative [11:41] ways. So if you have confidential [11:42] information, probably the models of [11:45] course they weren't trained on your [11:46] confidential data. So when you create a [11:47] knowledge graph out of your confidential [11:49] data, it it's uh you you augment your [11:52] LLM to answer you know your questions [11:54] about your confidential data maybe [11:56] inside in-house or if it's specialized [11:58] knowledge you enable it to empower it to [11:59] answer questions about in-house uh sorry [12:02] about the specialized field. So [12:05] knowledge graphs are really taking us [12:06] from you know general LLM to LLMs that [12:09] specialize in things or LLMs that know [12:11] about sees other parts of the world that [12:14] it creators haven't taught it yet. Okay. [12:17] Now, how do we go about building our [12:19] first LLM? So, I want to show you a [12:21] process to, you know, like a framework I [12:23] devised based on what's my experiments [12:26] uh the past months uh to see if [12:28] knowledge graphs are a good fit for your [12:30] problem. So, maybe you know you have a [12:31] question answering robot uh chatbot [12:33] sorry and you uh noted that it [12:36] hallucinates, right? So, maybe it h [12:39] maybe your LLM doesn't have enough of [12:41] the view of the world to answer the [12:42] question. So I actually talk about uh [12:46] one of the tasks in our project. So it's [12:48] called math.ai. It's in the central [12:50] bank. So it's a legal research AI [12:53] assistant. It still works in progress. [12:56] Matt is law in ancient Egyptian. [13:02] And [13:03] one of the tasks we had to do is to [13:05] answer questions about local economic [13:07] activities. Okay. So the first thing we [13:10] naturally want to know is we want to [13:13] identify the knowledge gap, right? What [13:15] does the LLM not know? What do I wish [13:17] the LLM knew so it can stop [13:18] hallucinating or stop answering the [13:20] questions poorly? So here I I would [13:23] advise you to have a set of questions [13:25] you develop with the expert with various [13:27] levels of difficulty and topics. So it's [13:30] representative of the task at hand. and [13:32] then you uh get the answers from the LLM [13:36] and with the expert you evaluate you [13:38] know where's the gap what does the LLM [13:40] not know so for example here we have [13:43] what are the main divisions of the [13:44] international classification system so [13:46] it's international right and then these [13:49] are the levels and then the second one [13:50] is about EU right so EU is regional [13:53] European Union and then finally the last [13:55] one is about Hong Kong so local on the [13:58] country level right so then we studied [14:00] the answers of the LLM the vanilla LLM, [14:03] the baseline LLM. And we noticed that [14:06] when it's asked about the international [14:08] system, it does a great job because you [14:10] know the international system is the [14:12] subject of discourse of you know people [14:14] all over the world. All the countries [14:16] care about the international system for [14:18] the EU um so and so. So EU is a region. [14:21] So there is discourse but not enough [14:22] discourse to say that the answers are [14:24] good enough by the experts. Okay. And [14:26] then finally Hong Kong. So who cares to [14:29] you know uh discuss local or like Hong [14:31] Kong's class economic classification [14:34] system probably you know um [14:38] Hong Kong pe people in Hong Kong [14:40] themselves so it doesn't have enough [14:42] information so it hallucinates okay so [14:45] of course we we see that on the uh the [14:48] more local we go the worse it gets but [14:50] we can't you know model every we cannot [14:53] check all the possible knowledge grafts [14:55] in the world so because we're doing a [14:57] proof of concept to try to identify one [14:58] one part of the knowledge gap. So let's [15:00] say yes, we're going to take Hong Kong [15:01] as an example of a local u knowledge gap [15:04] in the economic system. Okay. [15:07] The second thing you need to do is to [15:08] identify the needed data. So for the [15:10] questions you've identified, what are [15:12] the sources that the experts use [15:13] themselves to answer this questions? So [15:16] for example, in our case, they use the [15:17] Hong Kong standard industrial [15:19] classification manual. So this basically [15:20] describes all of the possible economic [15:22] activities in a hierarchy that is in [15:24] Hong Kong. We have the official press [15:26] releases. So the government releases [15:28] press and of course there are expert [15:29] opinions. So which one of these that's [15:33] so we can't again we cannot do for all [15:34] the data sources. It's a proof of [15:36] concept. So which one of these is it's [15:38] easy to access. [15:41] It's easy to process. It's [15:42] comprehensive. It covers most of the [15:43] aspects we care about. It's important [15:45] for the experts. They they reference [15:47] this source uh quite frequently and [15:49] maybe other business specific criteria [15:51] like translations. So raise your hand if [15:54] you think it's the first one. [15:58] Okay. So, it is the first one. [16:01] Yeah. So, because you know the the [16:03] manual is just one document. I don't [16:05] need to parse it. So, the other sources [16:07] I probably need to parse the web uh [16:08] scrape the web. Sorry. The manual is [16:10] just one document. It's a standard. It's [16:12] complete. It describes all of the [16:14] economic activities in Hong Kong. Um, of [16:16] course, it's a basic reference document [16:18] for experts. They they have to reference [16:20] this document because it it describes [16:22] all the it describes the market of Hong [16:24] Kong basically. [16:26] Okay, so now that we know what's the gap [16:28] and we gathered data to fill the gap, we [16:30] need to identify entities plus [16:33] relationships, right? We want to see the [16:34] world of the econ uh e economic system [16:36] of Hong Kong. What are the entities and [16:38] what are the um relationships? [16:41] So I just want to show you how the each [16:44] each entry is an activity. It has a [16:46] code. It goes from level one to level [16:48] four. It has the details description and [16:52] the includes excludes in Chinese and [16:54] English. So as you can see here the [16:56] hierarchy is already relationships right [16:58] the the first level derives the second [17:00] level which deres the third level and so [17:02] on. Also if you see here this is from [17:06] the manual itself. The classes are [17:08] referencing each other. So different [17:10] classes are also interacting includ [17:12] inclusion exclusion. [17:14] So this is from the manual. The parts [17:17] the parts in gray are actually [17:18] international standards. So they're for [17:20] they're present in all world manuals, [17:22] all country manuals and Hong Kong is [17:24] known for for example electronics right [17:28] so as we can see in this example what [17:31] types of entities do we have so we have [17:33] level one level two and level three [17:34] there's also level four section the [17:37] section is the height level highest [17:39] level it deres the second derives the [17:41] third and of course the classes interact [17:42] so inclusion or exclusion we call this [17:45] the data model the data model describes [17:47] what kind of entities we have and what [17:49] relationships are possible among them. [17:52] It's you can think of it as a schema for [17:54] a knowledge graph. [17:57] Now that you have your knowledge graph [17:58] and your data, you want to import your [18:00] data to the knowledge graph. So I highly [18:02] recommend you use Neo4j desktop. It's [18:04] excellent. So you can use it to upload [18:06] the data after it's cleaned of course in [18:08] CSV or any other uh knowledge graph [18:10] friendly uh file and then you're able to [18:13] load it into the schema. You can put [18:16] this the data model. You can create it [18:17] graphically as you can see here. And [18:20] then finally you can visualize with the [18:21] knowledge graph after the data has been [18:23] loaded and interact with it. You can [18:24] query it. So it's really excellent to [18:26] get started and you can do it for remote [18:28] or local in case you have sorry remote [18:30] or yeah local in case you have sensitive [18:32] data. [18:34] So finally we need to evaluate the [18:35] knowledge gain. So when I use the [18:37] knowledge graph did the gap get you know [18:40] narrower? So I want to evaluate the [18:41] baseline versus uh the knowledge graph [18:44] the the algorithm that uses the [18:45] knowledge graph and we do that with the [18:48] set of questions we identified in the [18:50] beginning. So we ask the expert to rate [18:52] the the answers of the baseline from one [18:54] to five the answers of the knowledge [18:56] graph from 1 to five and then we see the [18:58] uplift. So as you can see in the [19:00] knowledge graph it's better but now is [19:02] it good enough? So we ask the experts is [19:04] the knowledge gap narrow enough? There [19:06] will always be a knowledge gap but the [19:08] goal is to find a knowledge gap that's [19:09] good enough. You will never know [19:10] everything, but you can know enough. You [19:12] know, that should be the mindset. [19:15] So, yeah, that's my gift to you guys. [19:17] This is a cheat sheet summarizes [19:19] everything we just said today. You can [19:21] refer to it. I hope you um yeah, I hope [19:25] you go back to this and reflect on how [19:26] you can use knowledge graphs. And the QR [19:28] code is my LinkedIn if you want to ask [19:30] me any questions and we can uh talk of [19:32] course outside. Uh yeah, thank you very [19:35] much. [19:36] >> Thank you so much.