Former AIG Engineer Chen Charts LLM Credit Risk Shift

Leveraging LLMs in Credit Risk Modeling: Unlocking Deeper Insights With AIG and Fannie Mae — Ai4

Ai4

Integrating large language models into credit risk pipelines enables institutions to process unstructured alternative data and surpass hard traditional model performance ceilings.

AI-assisted summary of Leveraging LLMs in Credit Risk Modeling: Unlocking Deeper Insights With AIG and Fannie Mae — original source. Check the original for full context.

Source published:

Detailed notes

## Background Published by the Ai4 channel on YouTube on August 27, 2025, this presentation features financial manager Liliang (Daniel) Chen, formerly of AIG and Fannie Mae. The session explores how large language models can enhance credit risk modeling in finance and banking by unlocking unstructured data, improving model performance, and strengthening interpretability.

## TL;DR Chen breaks down the fundamental mechanics and historical evolution of credit risk modeling, illustrating why standardized bureau scores fall short without holistic frameworks like the 5 Cs of credit. He explains that traditional tabular algorithms face hard performance ceilings, making multimodal LLMs and RAG pipelines essential for mining unstructured alternative data such as utility statements, social profiles, and property imagery. The talk highlights five transformative LLM capabilities for risk teams: default prediction, automated data extraction, balanced synthetic dataset generation, step-by-step decision reasoning, and prompt-chained model interpretability for regulatory compliance.

## Atmosphere & style The presentation maintains an authoritative, technical, and educational tone that reflects Chen's extensive institutional background across Fannie Mae, Freddie Mac, and AIG. The narrative flows methodically from foundational statistical underwriting mechanics to live, forward-looking demonstrations of multimodal AI in action. It feels like an executive-level masterclass, bridging traditional credit risk engineering with cutting-edge generative AI workflows in a practical, pragmatic manner.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=4s) Credit risk management fundamentally evaluates the probability of borrower default to price risk-adjusted terms after screening for identity fraud. - [02:41](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=161s) Traditional credit bureau scores are heavily weighted toward payment history and revolving credit utilization (~65%), making them insufficient on their own for holistic borrower evaluation. - [05:00](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=300s) Holistic credit modeling relies on the 5 Cs framework—Credit history, Capacity, Capital, Conditions, and Collateral—to accurately differentiate borrowers with identical or inverted credit scores. - [08:56](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=536s) In classical modeling, finding an optimal separating hyperplane via non-linear ensembles like Random Forest and XGBoost yields higher predictive performance than linear logistic regression. - [14:01](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=841s) Because structured data represents only 20% of enterprise information, breaking through predictive performance ceilings requires unlocking unstructured alternative data sources. - [16:56](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1016s) Multimodal LLMs and RAG architectures allow institutions to extract actionable credit indicators directly from raw utility bills, social profiles, and property photographs. - [22:02](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1322s) LLMs can generate statistically consistent synthetic default records that respect domain constraints, mitigating the historical class imbalance problem in credit datasets. - [22:02](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1322s) LLM tabular classification benchmarks have rapidly improved, but institutional adoption in banking remains dependent on strict regulatory auditability and model interpretability. - [24:17](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1457s) Conversational prompt chaining enables risk modelers to unpack underlying risk drivers and discover missing external features, such as macroeconomic trends and industry headwinds. - [26:55](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1615s) The strategic value of LLMs in financial risk centers on five core capabilities: classification, unstructured extraction, synthetic data generation, reasoning, and governance explainability.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=4s) Introduction and Credit Ecosystem Fundamentals

Daniel Liliang Chen introduces himself as a financial engineer with career experience spanning three Fortune 100 financial institutions: Fannie Mae, Freddie Mac, and AIG. Opening his presentation to the audience, Chen conducts an informal show of hands to gauge how many attendees work in financial modeling and how many have already integrated large language models (LLMs) into their day-to-day workflows.

Chen explains that within traditional financial institutions, credit risk management exists fundamentally to ensure that money moves safely. The credit decision ecosystem operates as an intricate network of interdependent stakeholders, encompassing lenders, borrowers, regulators, shareholders, and even bad actors attempting fraud. Within this ecosystem, credit risk is strictly defined as the probability that a borrower will fail to repay their debt obligation to the lender. To properly price this risk, institutions rely heavily on credit risk modeling throughout the underwriting process.

```mermaid flowchart TD accTitle: Lending Decision and Underwriting Workflow subgraph Ingestion["1. Fraud & Identity Screening"] B["Borrower Application"] --> Q1{"Is the customer who they say they are?"} Q1 -- No --> F["Fraud Investigation / Rejection"] Q1 -- Yes --> Underwriting end

subgraph Underwriting["2. Credit Risk Assessment"] Q2{"Do we do business, and at what terms?"} --> CRM["Credit Risk Model"] CRM --> Offer["Terms & Interest Rate Pricing"] end

subgraph Execution["3. Loan Issuance & Servicing"] Offer --> Agreement{"Borrower Accepts Offer?"} Agreement -- Yes --> Payout["Disburse Funds (Cash Flow)"] Payout --> Repay["Regular Debt Service"] end ```

When an application is submitted, the lender addresses two sequential business questions: 1. **Identity & Authenticity:** *"Is the customer who they say they are?"* This phase isolates legitimate applicants from fraudsters via fraud investigation protocols. 2. **Risk & Pricing:** *"Do I want to do business with the borrower? If so, at what term?"* Here, the credit risk model determines an appropriate risk-adjusted price. Once terms are mutually agreed upon, funds are disbursed and the borrower begins regular amortized payments, completing the formal underwriting process.

---

[02:41](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=161s) The Evolution of Credit Risk Modeling

Chen charts the chronological evolution of credit risk evaluation from manual heuristics to generative artificial intelligence:

* **1950s–1960s (Early Credit Systems):** Credit determinations were purely manual, constrained by scarce documentation, subjective personal judgment, and localized evaluation. * **1970s (Centralized Credit Bureaus):** Credit bureaus emerged to centralize consumer financial histories, standardizing record access. * **1980s (Standardized Credit Scoring):** The introduction of the credit score represented a massive industry game-changer, distilling consumer risk profiles into a single standardized numerical metric. * **1990s–2000s (Multi-Factor Risk Engines):** Decision engines evolved beyond single-score reliance into multi-factor, rule-based algorithmic models. * **2000s–2020 (Machine Learning & Advanced Risk Modeling):** Adoption of statistical learning and traditional AI models capable of processing large-scale historical datasets. * **2022–Present (Generative AI & LLMs):** The generative AI era introduces unprecedented speed, scale, and semantic sophistication into model pipelines.

To illustrate how credit bureau scores are structured, Chen displays an Equifax report featuring an exemplary credit score of 743. He details the primary score composition:

| Component Category | Share of Total Score | Core Drivers | |:--- |:--- |:--- | | **Primary Determinants** | ~65% | Payment history and revolving credit utilization ratios | | **Secondary Determinants** | ~35% | Length of credit history, credit mix/experience, and recent credit inquiries |

---

[05:00](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=300s) Why Credit Scores Alone Are Not Enough: The 5 Cs of Credit

Addressing the common question of why institutions still need dedicated credit risk models if standard bureau scores already exist, Chen walks through three comparative scenarios between two hypothetical borrowers:

* **Scenario 1:** Borrower A has a 700 score; Borrower B has a 650 score. With no other context, Borrower A obviously presents lower credit risk. * **Scenario 2:** Borrower A has a 700 score, a \$100,000 annual salary, and \$50,000 in annual debt. Borrower B has a 650 score, an \$80,000 salary, and \$60,000 in debt. Borrower A remains distinctly lower risk across all metrics. * **Scenario 3 (The Inversion):** Borrower A holds the higher 700 score but earns a lower salary with higher debt obligations. Borrower B holds a lower 650 score but possesses a significantly higher salary and lower existing debt.

Determining the safer counterparty in the third scenario is impossible through a credit score alone, underscoring the absolute necessity of holistic credit risk models. Lenders evaluate these multidimensional profiles using the foundational framework known as the **5 Cs of Credit**:

* **Credit History:** The historical track record of borrowing and repayment discipline. * **Capacity to Repay:** Cash flow and earnings relative to current liabilities (debt-to-income). * **Capital:** Liquid financial assets, personal investment reserves, or savings available to withdraw in an emergency. * **Conditions of the Loan:** The requested loan amount, repayment term length, interest rate, and designated borrowing purpose. * **Collateral:** Pledged tangible assets that can be claimed by the lender upon default to mitigate loss given default (LGD).

Chen maps these 5 Cs directly to open-source LendingClub loan data:

``` LendingClub Dataset Mapping: ├── Loan Conditions: Loan Amount, Term (36/60 mo), Interest Rate ├── Capacity to Repay: Employment Length, Home Ownership Status ├── Capital: Annual Income, Debt-to-Income (DTI) Ratio ├── Credit History: Earliest Credit Line, FICO Score, Revolving Balance └── Target Variable: Loan Status (Fully Paid vs. Charged Off / Default) ```

---

[08:56](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=536s) Classical Modeling Approaches and Optimization Mechanics

When constructing predictive default models, practitioners choose algorithms across varying trade-offs of interpretability and complexity. Baseline logistic regression models provide full transparency and simple production implementation. In contrast, non-linear ensemble models like Random Forest, gradient boosting (XGBoost), and deep neural architectures capture complex feature interactions and ingest massive feature spaces.

Mathematically, the fundamental objective of credit risk modeling is to isolate a separation function—specifically an optimal separating hyperplane within a multi-dimensional feature space:

$$w_0 + w_1 x_1 + w_2 x_2 + \dots + w_n x_n = 0$$

Finding this hyperplane allows the institution to effectively segment default-bound loans from non-defaulting loans.

```mermaid flowchart LR accTitle: Feature Predictive Power Validation subgraph FeatureAnalysis["Target vs. Feature Distribution Analysis"] direction TB F1["Feature Value Distribution<br/>(e.g., Interest Rate, DTI, FICO)"] P["Fully Paid Loans<br/>(Blue Density Curve)"] D["Charged Off Loans<br/>(Orange Density Curve)"] F1 --> P F1 --> D end

subgraph Decision["Separation Metric"] P & D --> Comp{"Distribution Curves Diverge?"} Comp -- Significant Divergence --> High["Strong Predictive Power<br/>(Retain Feature in Model)"] Comp -- Curves Overlap --> Low["Weak / No Predictive Power<br/>(Drop from Model)"] end ```

To validate predictive power before model training, Chen plots distribution curves comparing fully paid loans against charged-off loans across variables such as interest rate, credit score, and debt-to-income (DTI) ratio. If the two probability density curves overlap heavily, the independent variable offers little predictive power. If the distributions diverge significantly, the variable contains strong predictive signal.

After feature selection, practitioners conduct rigorous feature engineering—normalizing continuous variables, encoding categorical fields, and optimizing transformation functions. Chen notes that while logistic regression delivers baseline performance, non-linear machine learning ensembles (Random Forest, XGBoost, CNNs) consistently achieve superior F1 scores because they capture multi-factor non-linearities:

$$\text{Precision} = \frac{\text{True Positives}}{\text{True Positives} + \text{False Positives}} = \frac{\text{True Positives}}{\text{Forecasted Positives}}$$

$$\text{Recall} = \frac{\text{True Positives}}{\text{True Positives} + \text{False Negatives}} = \frac{\text{True Positives}}{\text{Actual Positives}}$$

$$\text{F}_1 = 2 \cdot \frac{\text{Precision} \cdot \text{Recall}}{\text{Precision} + \text{Recall}}$$

---

[14:01](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=841s) Expanding Data Dimensionality with Unstructured Alternative Data

Chen emphasizes that regardless of algorithm complexity, credit modeling always encounters hard performance ceilings dictated by data quality. "No matter how sophisticated your model is, the improvement of your algorithm always have some limits." The most viable path toward significant model lift is expanding the dimensionality of the underlying data.

Historically, credit modeling has relied exclusively on structured tabular data, which represents only about 20% of all enterprise data. The remaining 80% consists of unstructured data—images, audio, video, and text documents—which traditional pipelines could not systematically process.

```mermaid flowchart TD accTitle: Data Taxonomy in Credit Modeling subgraph Traditional["Traditional Space"] SD["Structured Traditional Data<br/>(FICO, Bureau Reports, W2)"] end

subgraph Alternative["Alternative Data Frontiers"] SAD["Structured Alternative Data<br/>(Digital payment logs, rent registries)"] UAD["Unstructured Alternative Data<br/>(Utility bills, social media, property images, text reviews)"] end

SD -.->|Explored & Saturated| Models["Credit Risk Engine"] SAD -.-> Models UAD ==>|High Value / Unexplored Frontier| LLM["LLMs & Multimodal Embeddings"] LLM ==> Models ```

The greatest untapped opportunity in modern finance lies in the intersection: **unstructured alternative data**. Chen categorizes the key alternative data types applicable to credit risk:

* **Transactional Data:** Granular utility bills, continuous rent payment histories, and mobile micro-payment transactions. * **Behavioral Data:** Digital shopping patterns, frequency of merchant transactions, and categorical spending allocation. * **Social & Public Presence:** Verified social media profiles, public business posts, customer reviews, and professional affiliations. * **Property & Real Estate Data:** High-resolution exterior/interior property imagery, real estate listings, and municipal tax assessment documents.

---

[16:56](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1016s) Implementing Multimodal RAG and Extraction Workflows

To ingest and query unstructured documents, financial institutions deploy **Retrieval-Augmented Generation (RAG)** systems powered by multimodal LLMs.

```mermaid flowchart LR accTitle: Multimodal Retrieval-Augmented Generation Workflow subgraph Ingestion["Multimodal Ingestion Pipeline"] Raw["Unstructured Source Data<br/>(PDF Bills, Social Feeds, Images)"] --> MM["Multimodal Embeddings"] MM --> VecDB[("Vector Index Database")] end

subgraph QueryEngine["Query & Retrieval Engine"] Prompt["Credit Officer / LLM Query"] --> VecDB VecDB --> Context["Context Retrieval"] Context --> LLMEngine["Google Gemini / LLM Engine"] LLMEngine --> Output["Structured Attribute Output<br/>(JSON / Credit Signals)"] end ```

Chen showcases real-world implementations using Google Gemini to extract credit insights from unstructured artifacts:

* **Utility Bill Parsing:** An LLM processes an image of an electric utility bill, successfully extracting the exact statement period, total amount due, previous balance (\$223), and payment received (\$223). Verifying that previous balances were paid in full confirms day-to-day cash flow management and recurring debt discipline without waiting for bureau reporting. * **Social Media Profile Verification:** Gemini parses public professional profile data, identifying the applicant's name, verified job title, corporate affiliation, and follower count. Chen explains that large follower counts act as a fraud defense—fraudulent shell identities rarely maintain established followings—while also serving as a proxy for social capital and professional stability. * **Visual Property Inspection:** An LLM inspects a residential exterior photograph, identifying it as a two-story colonial-style home with vinyl siding, a red front door, and well-maintained landscaping. When prompted for property parameter estimates, the model estimates square footage (2,400–2,800 sq ft), lot dimensions, and a build year (~2010), providing explicit architectural reasoning. This visual data enables lenders to accurately project property tax obligations, homeowner insurance premiums, and future deferred maintenance expenses.

---

[22:02](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1322s) Synthetic Data Generation and LLM Performance Trajectory

Beyond data extraction, LLMs resolve severe class imbalance issues inherent to credit data, where loan defaults represent a small fraction of total historical records. To avoid model overfitting on skewed classes, practitioners traditionally employ mathematical resampling. Chen demonstrates prompting an LLM to generate 10 plausible synthetic charged-off loan records. The model generates synthetic records adhering strictly to realistic feature distributions, variable correlations, data types, and logical behavioral patterns.

``` Synthetic Generation Parameter Constraints: - Maintains plausible domain combinations (e.g., high DTI paired with low revolving balance limits) - Adheres to statistical feature distributions of historical default cohorts - Injects controlled stochastic randomness to prevent direct record duplication ```

Evaluating the predictive benchmark trajectory of LLMs, Chen reviews recent academic literature tracking zero-shot and fine-tuned models on tabular credit tasks. Early iterations of ChatGPT scored an F1 metric of 0.44, whereas GPT-4 rapidly reached 0.70. Chen projects that within the next five years, frontier multimodal LLMs will likely match or outperform standalone logistic regression and ensemble baseline models directly.

However, Chen emphasizes that predictive power represents only one half of institutional adoption. Financial institutions operate under strict regulatory scrutiny and must provide transparent, mathematically auditable model interpretability to explain exact decision rationale to banking regulators.

[24:17](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1457s) Interactive Prompt Chaining for Model Interpretability and Feature Insights

Large language models can be prompted interactively to unpack the underlying drivers of credit risk predictions, serving as an intuitive explainability layer on top of complex credit data. Working with a dataset exhibiting a baseline default rate of 0.14, the LLM is prompted to identify the primary factors driving default probability. In response, the model highlights key financial metrics, identifying interest rate, FICO score, monthly installment amount, earliest credit line, and revolving credit utilization as the most influential variables.

```mermaid flowchart TD accTitle: Interactive LLM Prompt Chaining for Credit Explainability subgraph Baseline["1. Data & Drivers"] A["Default Rate (0.14)"] --> B["Extract Top Factors\n(Interest Rate, FICO, Installment, etc.)"] end subgraph Reasoning["2. Chained Inquiries"] B --> C["Model Mechanics\n(Statistical model & directional relationships)"] C --> D["Training Logic\n(Pattern search & historical experience)"] end subgraph Expansion["3. Feature Discovery"] D --> E["External Context\n(Macroeconomics, Industry factors, Human oversight)"] end ```

By feeding the model's previous outputs back into subsequent prompts, users can drill down further into the mechanics of the credit assessment. When asked what kind of modeling approach it is leveraging and how those variables interact, the LLM explains that it applies a statistical framework where "higher interest rate will increase the default" while a "better credit score will decrease the default." Continuing the prompt chain to ask how the system learns, the model clarifies that it processes historical datasets by "searching for the pattern" and fundamentally learning from past credit performance. While these explanations represent foundational, intuitive arguments, they provide clear, transparent explanations of baseline credit risk dynamics.

- **Baseline Metric**: Dataset default rate analyzed at 0.14. - **Top Influential Drivers**: Interest rate, FICO credit score, installment amount, earliest credit line history, and revolving credit utilization. - **Directional Risk Insights**: Transparently articulates that elevated interest rates correlate with higher default likelihood, whereas stronger credit scores reduce default probability.

---

[25:56](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1556s) Broadening Feature Discovery and External Reasoning Beyond the Dataset

Prompting the LLM beyond the boundaries of the immediate training dataset reveals strategic variables that human underwriters and risk managers evaluate in practice. When queried about what additional factors influence final lending decisions beyond the core quantitative model, the LLM identifies critical contextual dimensions, including macroeconomic variables, industry-specific considerations, and human oversight.

Because these external variables are not part of the initial tabular dataset, the LLM acts as an ideation engine for feature engineering. An insight such as "industry consideration" prompts the modeler to investigate sector-specific metrics that could be ingested to enrich the baseline model. Leveraging the LLM's chain-of-thought capabilities allows risk practitioners to systematically refine their analytical logic and uncover valuable external data sources.

- **External Decision Factors Identified**: Macroeconomic conditions, sector/industry-specific headwinds, and manual underwriting or human oversight. - **Feature Engineering Expansion**: Uses conversational reasoning to pinpoint missing features outside the original dataset for iterative model refinement.

---

[26:55](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1615s) Summary: Five Core Capabilities of LLMs in Credit Risk Modeling

The presentation concludes with a synthesis of the overarching value propositions that large language models bring to credit risk modeling and financial assessment workflows:

1. **Prediction and Classification**: Performing supervised credit default predictions and risk classification comparable to traditional machine learning algorithms. 2. **Unstructured Data Extraction**: Parsing, structuring, and extracting meaningful risk signals from textual and non-tabular financial documentation. 3. **Synthetic Data Generation**: Creating realistic synthetic credit records to augment sparse training sets and balance underrepresented cohorts. 4. **Generative AI-Based Reasoning**: Providing logical, step-by-step reasoning chains to support decision-making frameworks. 5. **Model Interpretability and Governance**: Translating complex statistical relationships into human-understandable narratives for compliance, underwriting, and risk validation.

## Notable quotes > The shift toward LLM-enhanced credit risk modeling presents new opportunities in finance and banking, including processing large volumes of unstructured data, enhancing model performance, and improving interpretability. — Daniel Liliang Chen

> [00:04](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=4s) Is the customer who they say they are? — Daniel Liliang Chen

> [00:04](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=4s) Do I want to do business with the borrower? If so, at what term? — Daniel Liliang Chen

> [14:01](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=841s) No matter how sophisticated your model is, the improvement of your algorithm always have some limits. — Daniel Liliang Chen

> [24:17](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1457s) higher interest rate will increase the default — Daniel Liliang Chen

> [24:17](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1457s) better credit score will decrease the default — Daniel Liliang Chen

> [24:17](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1457s) searching for the pattern — Daniel Liliang Chen

> [25:56](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1556s) industry consideration — Daniel Liliang Chen

## Mentioned Liliang (Daniel) Chen, Ai4, AIG, Fannie Mae, Freddie Mac, Equifax, FICO, LendingClub, Google Gemini, ChatGPT, GPT-4, XGBoost, Random Forest

## Video description Liliang (Daniel) Chen, Financial Manager, Formerly of AIG and Fannie Mae

Abstracts

The shift toward LLM-enhanced credit risk modeling presents new opportunities in finance and banking, including processing large volumes of unstructured data, enhancing model performance, and improving interpretability. This talk will explore the evolving landscape of LLM integration in finance and highlight opportunities for impactful system design and development.

[Source](https://www.youtube.com/watch?v=bq0Wt2hXmv0)

中文简报

前AIG工程师Chen解析大模型驱动的信用风险变革 将大语言模型整合至信用风险流程，使机构能够处理非结构化替代数据并突破传统模型的刚性性能上限。 Chen剖析了信用风险建模的基本机制与历史演进，阐释了在缺乏信贷“5 C”等全面评估框架时，标准化征信机构评分存在局限性的原因。他指出传统表格算法面临难以逾越的性能瓶颈，因此多模态LLM和RAG流程对于挖掘公用事业账单、社交资料及房产图像等非结构化替代数据至关重要。演讲重点探讨了LLM为风控团队带来的五大变革性能力：违约预测、自动化数据提取、均衡合成数据集生成、逐步决策推理，以及面向监管合规的提示链模型可解释性。 信用风险管理的核心是在通过欺诈筛查后，评估借款人的违约概率以确定经风险调整的贷款定价。 (0:04) 传统征信机构评分约65%取决于还款历史与循环信贷使用率，必须结合5C框架（信贷历史、还款能力、资本、贷款条件、抵押品）才能全面评估借款人画像。 (5:00) 非线性集成模型如Random Forest与XGBoost能够捕捉多维特征交互，但受制于仅占企业信息20%的结构化数据，突破模型性能上限必须引入非结构化替代数据。 (14:01) 多模态LLM与RAG架构可直接从水电账单、社交档案及房产照片中提取信用指标，并能生成统计一致的合成违约记录以解决样本类别不平衡问题。 (16:56) 虽然LLM在表格数据分类基准测试中F1分数已大幅提升（GPT-4达0.70），但银行机构的实质性落地依然取决于严格的监管可审计性与模型可解释性。 (22:02)

Reading summary

Financial engineer Daniel Liliang Chen asserts that traditional credit risk modeling has hit a performance ceiling by relying exclusively on structured tabular data, but integrating multimodal large language models unlocks the 80% of enterprise data that is unstructured while improving data balancing and explainability. Limitations of Bureau Scores and Structured Data (5:00) Standard credit bureau scores—derived approximately 65% from payment history and revolving credit utilization, and 35% from credit history length, credit mix, and recent inquiries—cannot resolve conflicting borrower profiles on their own. When applicants present diverging metrics, such as a higher credit score paired with higher debt and lower income, single metrics fail to assess counterparty risk. To bridge this gap, institutions employ credit models structured around the 5 Cs of credit: credit history, capacity to repay, capital, loan conditions, and collateral. However, Chen notes that statistical optimization on traditional tabular data faces hard limits regardless of algorithm complexity. Structured tabular records account for only about 20% of enterprise information, meaning baseline logistic regressions and non-linear machine learning ensembles (such as Random Forest and XGBoost) leave a vast majority of potential risk signals unexamined. Extracting Risk Signals from Unstructured Alternative Data (16:56) The remaining 80% of enterprise data exists in unstructured formats, including transactional records, social profiles, and property imagery. Financial institutions can deploy multimodal retrieval-augmented generation (RAG) pipelines and vector databases to parse and structure these inputs into standardized credit signals. Chen demonstrates real-world implementations using Google Gemini to extract underwriting attributes from non-traditional artifacts: parsing electric utility bills to verify statement periods and consistent cash-flow management without waiting for bureau reporting, validating professional social media profiles to confirm corporate affiliations and guard against fraudulent shell identities, and inspecting residential exterior photographs to infer square footage, build period, and upcoming maintenance liabilities. Synthetic Data Generation and Predictive Trajectory (22:02) Credit modeling frequently suffers from severe class imbalance because historical loan defaults represent a small proportion of overall portfolios. To prevent predictive models from overfitting on skewed distributions, practitioners can prompt LLMs to generate synthetic charged-off records. These generated profiles maintain realistic domain combinations, respect historical default correlations, and introduce controlled stochastic variations to avoid direct record duplication. Evaluating predictive benchmarks, Chen highlights academic literature where LLM performance on tabular credit classification advanced from an F1 score of 0.44 in early ChatGPT iterations to 0.70 in GPT-4. Based on this trajectory, Chen projects that frontier multimodal LLMs could match or exceed standalone logistic regression and ensemble models within the next five years. Interactive Model Governance and Feature Discovery (24:17) Beyond direct classification, LLMs offer an interactive explainability layer through prompt chaining. When analyzing a sample dataset with a baseline default rate of 0.14, an LLM prompted on default drivers isolated core variables—including interest rate, FICO score, installment amounts, and revolving utilization—while articulating directional dynamics, such as how elevated interest rates correlate with increased default probability. Furthermore, querying the model about factors omitted from the quantitative dataset surfaces critical contextual drivers, such as macroeconomic indicators, sector-specific industry headwinds, and human underwriting discretion. Chen emphasizes that risk practitioners can use these conversational reasoning steps as an ideation engine for ongoing feature engineering. While LLMs demonstrate expanding utility across data extraction, synthetic augmentation, and automated reasoning, Chen emphasizes that institutional adoption remains constrained by regulatory oversight, requiring lenders to maintain mathematically auditable and explainable decision frameworks before deploying models into production.

精炼阅读摘要

金融工程师Daniel Liliang Chen指出，传统信贷风险模型因过度依赖结构化表格和单一信用评分已面临性能天花板，而多模态大语言模型（LLM）能够挖掘占据企业80%体量的非结构化替代数据。通过将LLM引入多模态特征提取、合成样本生成、交互式归因解释与特征工程构思，金融机构有望突破传统建模的数据维度限制，并为信贷风控决策提供更透明的逻辑支持。 信用分局限与多维风控模型 (5:00) 陈指出，仅凭征信机构的信用分难以准确刻画借款人的综合资信。以典型信用分为例，约65%的权重由还款历史与循环信贷利用率决定，约35%由信贷历史年限、资产组合类型及近期查询构成；然而在诸如高信用分但低收入、高负债的逆转情境下，传统单维评分将彻底失灵。因此，金融机构必须围绕涵盖信用历史（Credit History）、还款能力（Capacity）、资本储备（Capital）、贷款条件（Conditions）以及抵押资产（Collateral）的“信贷5Cs”框架，构建多变量信贷风控模型。 在传统建模实践中，机构利用Logistic回归或非线性集成算法（如XGBoost、随机森林）在多维特征空间中寻找最优分离超平面，以区分正常履约与违约借款人。但陈强调，无论算法结构多么精密，其性能改善终究存在上限，核心瓶颈在于传统风控长期仅依赖结构化表格数据，而这部分数据仅占企业总数据资产的约20%，剩余80%的非结构化数据始终处于未被充分利用的状态。 非结构化替代数据与多模态RAG (14:01) 为拓展数据维度，陈建议系统引入非结构化替代数据，包括公用事业账单等交易数据、消费行为数据、公开社交档案以及不动产物理图像。金融机构可通过部署结合多模态大语言模型（如Google Gemini）的检索增强生成（RAG）管道，对异构文档与多媒体资料进行向量化检索与结构化指标提取。 陈列举了具体实践场景：在电费账单分析中，多模态LLM能准确提取账单周期、应付总额以及前期结清记录（如账单所显示的223美元历史欠费全额还清），从而在不依赖征信报告时滞的前提下验证借款人的短期现金流韧性；在社交档案核验中，模型解析认证职位与关注者规模，将其作为防范空壳虚假身份及评估职业声誉的辅助参考；在住宅外观图像评估中，LLM可判定建筑架构（如双层殖民风格）、推断使用面积（2400至2800平方英尺）与建造年份（约2010年），为物业估值、税费预测及后续维护成本测算提供前置依据。 合成数据生成与预测性能演化 (22:02) 针对信贷历史样本中违约借款人占比极低导致的严重类别不平衡问题，传统重采样手段容易造成模型过拟合。陈演示了利用大语言模型生成逼真违约样本的路径：在设定参数边界与受控随机性的前提下，模型可生成10条与历史违约特征分布、变量相关性及业务逻辑相符的合成贷款记录（如高债务收入比与低循环授信额度的特定组合），有效缓解训练样本的倾斜现象。 在底层预测能力上，文献测试显示LLM在表格信贷任务上的准确率提升迅速，早期ChatGPT的F1评分仅为0.44，而GPT-4已攀升至0.70。陈据此预测，前沿多模态大模型在未来五年内有望达到甚至超越独立的Logistic回归及机器学习集成基线。然而他也指出，纯粹的预测能力仅占模型投产考量的一半，受制于金融监管，模型决策必须具备可追溯、可审计的数学可解释性。 提示词链可解释性与特征工程 (24:17) 在监管合规层面，大语言模型可借助交互式提示词链（Prompt Chaining）充当复杂风控模型的解释层。在基线违约率为0.14的数据集测试中，模型通过提示词逐步提炼出借款利率、FICO评分、分期金额、授信历史年限与循环信用利用率等关键驱动因子，并能以直观语言阐明变量方向性逻辑——高利率抬升违约概率，而优质信用评分则有效抑制违约风险。 除了归因解释，提示词链还能引导风控人员跳出既有表格特征的局限。当被追问数据之外的影响因素时，模型主动提示了宏观经济指标、行业周期逆风以及人工核保复核等多重视角。陈认为，这种多步推理能力赋予了大模型作为特征工程“构思引擎”的角色，能协助建模团队在初始数据集外发掘潜在高价值特征，从而驱动风控体系的迭代优化。 陈将大语言模型在信贷风控中的核心价值归纳为五大支柱：违约预测分类、非结构化数据抽取、合成数据生成、生成式逻辑推理以及模型可解释性治理。然而，尽管大模型在处理非结构化数据与拓展特征维度方面展现出明显优势，银行与金融机构面临的严苛法规、决策可解释性要求与数学审计标准，仍然是大模型全面替代传统判别模型前必须解决的关键难题。

Detailed notes — zh

## Background 该视频由 Ai4 于 2025 年 8 月 27 日发布在 YouTube 平台。演讲者为前 AIG 和房利美（Fannie Mae）财务经理 Daniel Liliang Chen，页面简介指出本次演讲旨在探讨大语言模型（LLM）在金融信贷风险建模中的演进，展示其在处理非结构化数据、提升模型性能和增强可解释性方面的系统设计与实践机遇。 ## TL;DR 演讲系统梳理了信贷风险建模的演进历程与“5C”评估原则，指出单一信用评分和传统线性模型难以充分捕捉多维借款人风险。为突破数据瓶颈，讲者重点阐释了如何结合多模态 LLM 与 RAG 技术，从公用事业账单、社交媒体画像及房产图片等非结构化替代数据中高效提取关键风控特征。此外，大模型在合成样本生成以缓解数据不平衡、直接违约预测以及多轮交互式因果归因与思维链（Chain of Thought）拓展中展现出显著价值，为金融机构打通了黑盒模型与监管合规之间的沟通壁垒。 ## Atmosphere & style 整场演讲呈现出高度专业、务实且兼具前瞻性的金融科技大师课风格。讲者凭借财富 100 强金融机构的实操经验，将复杂的信贷风控理论与最前沿的生成式 AI 技术有机结合，叙事逻辑清晰严密。通过真实的账单与房产案例演示，原本抽象的算法和数据架构被具象化呈现，带给听众一种既紧跟技术浪潮又扎实落地的沉浸式专业体验。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=4s) 信贷决策生态系统涵盖放贷机构、借款人、监管机构、股东及潜在欺诈者，其承保流程包含反欺诈身份验证与量化违约风险定价两个核心环节。 - [02:41](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=161s) 信贷风险评估经历了从人工经验、信用局兴起、信用评分机制到多因子决策引擎及机器学习的演进，当前正迈向大语言模型（LLM）赋能阶段。 - [02:41](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=161s) 单一信用分无法全面反映借款人的真实财务状况，放贷机构必须结合收入、负债等多维度变量构建模型进行综合评估。 - [06:39](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=399s) 信贷审查围绕“5C原则”（信用历史、还款能力、资本实力、贷款条件、抵押品）展开，模型本质上是在高维空间中寻找区分违约与正常样本的决策超平面。 - [12:39](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=759s) 在传统算法优化边际递减的背景下，提升模型性能最切实可行的途径是增加数据维度，其中占据企业数据80%的非结构化替代数据最具挖掘潜力。 - [16:56](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1016s) 借助多模态嵌入与 RAG 架构，LLM 能高效解析水电账单、社交媒体信息及不动产外观图片，提取还款纪律、反欺诈特征与隐性财务负债。 - [21:59](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1319s) LLM 可依据特征分布与逻辑生成高质量合成数据，有效扩充极少数的违约样本，解决信贷建模中长期存在的样本不平衡问题。 - [21:59](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1319s) 前沿大模型在标准信贷数据集上的直接预测性能持续攀升，GPT-4 的 F1 分数已达 0.70 左右，未来有望追平或超越传统逻辑回归与随机森林模型。 - [24:17](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1457s) LLM 具备强大的多轮交互与归因解释能力，可将复杂的黑盒模型输出转化为通俗的业务逻辑与因果关系，满足严苛的监管合规审查。 - [25:56](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1556s) LLM 能基于行业通识知识库，突破原始数据集局限，启发风控人员引入宏观经济、行业特性及人工复核等外部特征维度以优化决策体系。 - [26:55](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1615s) LLM 赋能信贷风控的核心价值体现在：预测分类、非结构化数据抽取、合成数据增强、逻辑推理与模型可解释性五大维度。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=4s) 个人背景与信贷决策生态系统概述 讲者 Daniel Liliang Chen 曾先后在房利美（Fannie Mae）、房地美（Freddie Mac）和 AIG 等三家财富 100 强金融机构担任金融工程师与财务经理。在演讲开场，他通过现场互动了解听众在金融建模以及在日常工作中使用大语言模型（LLM）的普及情况，随后切入信贷风险的核心议题。在传统金融机构中，信贷风险管理是保障资金安全流动的基石。信贷决策生态系统本身是一个高度复杂的协同网络，涵盖了放贷机构（Lender）、借款人（Borrower）、监管机构（Regulator）、股东（Shareholder）乃至潜在的欺诈者（Fraudster）。 所谓信贷风险，本质上是指借款人未能按期向放贷机构履行偿债义务的可能性。为了给这种风险合理定价，放贷机构必须依赖精准的信贷风险模型。在具体的业务流程中，放贷机构需要连续解答两个核心问题：第一，“客户是否真实为其所声称的身份？”放贷机构首先需要通过反欺诈调查，从正常借款人中排查并剔除欺诈者；第二，“我是否愿意与该借款人开展业务？如果愿意，应以何种条款开展？”此时，放贷机构依靠信贷风险模型对违约风险进行量化定价。一旦双方就借款价格和条款达成一致，放贷机构便会向借款人发放资金，借款人则按期偿还本息，这一以确定价格发放贷款的完整业务流程被称为“承保”（Underwriting）。 ```mermaid flowchart TD accTitle: 信贷决策与承保业务流程图 subgraph S1["身份与欺诈筛查"] A["借款人提出申请"] --> B{"身份验证与反欺诈调查"} B -- "识别为欺诈者" --> C["拒绝申请 / 阻断交易"] end subgraph S2["风险评估与承保定价"] B -- "真实借款人" --> D["信贷风险模型评估"] D --> E["风险定价与设定贷款条款"] end subgraph S3["放款与履约"] E --> F{"借款人接受要约？"} F -- "是" --> G["承保放款与现金流交付"] G --> H["借款人定期偿还本息"] F -- "否" --> I["流程终止"] end ``` ### [02:41](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=161s) 信贷风险建模的历史演进与信用分局限 信贷风险评估体系在过去数十年间经历了显著的范式变迁。在 20 世纪 50 至 60 年代的早期信贷系统中，信贷决策完全依赖人工操作，受限于极度匮乏的数据和高度主观的个人经验判断。进入 70 年代，信用局（Credit Bureau）开始兴起，实现了借款人信用信息的集中化并逐步对外公开。80 年代迎来了行业最大的一次变革——信用评分（Credit Score）的诞生，它将复杂的个人资信压缩为一个标准化的单一数值指标。90 年代至 2000 年代，基于多因子的风险决策引擎崛起；2000 年至 2020 年，人工智能与机器学习全面融入风险决策体系；而自 2022 年以来，随着生成式 AI 的爆发，大语言模型（LLM）为信贷风险建模注入了前所未有的处理速度、规模化能力以及复杂非线性关系的解析度。 以 Equifax 出具的典型信用报告（如评分为 743 分）为例，其分数构成中还款记录（Payment History）与信用额度使用率（Credit Utilization）合计占据了接近 65% 的绝对权重，其余 35% 则分散在信用历史长度、信用类型组合以及近期的硬查询记录上。面对“既然已有标准化信用分，为何还需要信贷风险模型”的疑问，Daniel 通过三组对比案例给出了直观解答： - **案例一**：借款人 A 信用分为 700，借款人 B 为 650，显然 700 分的违约风险更低； - **案例二**：在相同信用分差距下，若 A 的年薪为 10 万美元、年负债为 5 万美元，而 B 的年薪为 8 万美元、年负债为 6 万美元，A 凭借更高的收入、更低的负债以及更高的信用分，风险依旧明显较低； - **案例三**：若将两者的财务状况反转——A 虽有 700 分的高信用分，但年薪仅 8 万、负债达 6 万；B 虽只有 650 分，但年薪高达 10 万、负债仅 5 万。此时仅凭单一信用分已无法判断谁的风险更低，必须依赖多维度变量构建的信贷风险模型进行综合量化评估。 ### [06:39](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=399s) 评估信贷的“5C”原则与传统模型构建 放贷机构在评估贷款申请时，通常围绕信贷“5C 原则”展开全方位审查： 1. **信用历史（Credit History）**：借款人过去的借贷与还款履约表现； 2. **还款能力（Capacity to Repay）**：借款人的实际收入水平与已有债务负担之间的配比； 3. **资本实力（Capital）**：除工资外，借款人可随时变现或支取的储蓄与投资资产规模； 4. **贷款条件（Condition of Loans）**：借贷金额、偿还期限、借款用途及宏观利率环境； 5. **抵押品（Collateral）**：借款人承诺在发生违约时代为偿还债务的抵押资产。 借款人在 5C 维度上的综合表现越强，其违约风险就越低，从而能够获得更具竞争力的贷款条件（例如更低的贷款利率）。以公开的 LendingClub 数据集为例，模型特征涵盖了贷款金额、期限、利率等贷款条件，工作年限、住房所有权等还款能力指标，年收入、债务收入比（DTI）等资本实力指标，以及最早信贷账户年限、FICO 分数、循环信用余额等信用历史指标；模型的目标变量（Target Variable）通常定义为贷款状态，即“全额还清（Fully Paid）”与“核销违约（Charged Off）”。 在建模数学本质上，信贷风险模型的核心在于寻找一个能够在高维空间中有效分离违约与未违约案例的决策超平面（Hyperplane）： $$w_0 + w_1 x_1 + w_2 x_2 + \dots + w_n x_n = 0$$ 为了筛选具备强预测能力的特征，建模人员会将数据集按目标变量切分为“已还清”（蓝色）与“核销违约”（橙色）两组，并分别绘制自变量的概率分布曲线。若两组样本在某一变量（如贷款利率、信用评分、DTI）上的分布曲线重合度低且形态差异显著，则表明该变量具有极强的违约预测能力。在完成变量筛选后，还需通过特征工程（Feature Engineering）对原始数据进行变量转换、归一化处理及分类变量编码，最终提炼出最有效的特征组合输入模型。 ```mermaid flowchart LR accTitle: 传统信贷风险特征工程与建模流程 subgraph S1["数据准备与变量探索"] A["LendingClub / 资信原始数据"] --> B["样本分组<br/>(已还清 vs 核销违约)"] B --> C["概率分布曲线对比分析"] end subgraph S2["特征工程"] C --> D["特征变换与归一化"] D --> E["分类变量编码"] E --> F["特征重要性评估与筛选"] end subgraph S3["模型构建与超平面划分"] F --> G["线性/非线性算法拟合"] G --> H["高维空间决策超平面<br/>(分离违约与正常样本)"] end ``` ### [12:39](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=759s) 传统模型性能对比与替代数据的引入 在评估信贷模型性能时，业界普遍采用兼顾精确率（Precision，预测为正例中实际为正例的比例）与召回率（Recall，实际正例中被成功预测出的比例）的 F1 分数（F1 Score）作为核心评估指标。传统的逻辑回归（Logistic Regression）因具备极强的可解释性与部署便捷性而被广泛应用，但由于其主要捕捉线性关系，F1 分数通常落后于随机森林（Random Forest）、XGBoost 以及卷积神经网络（CNN）等能够捕捉复杂非线性特征交互与海量高维数据的机器学习模型。 然而，无论算法架构如何演进，信贷风险建模始终面临三大瓶颈：数据质量、模型性能上限与模型可解释性。在算力与算法优化遭遇边际递减效应的情况下，“提升模型性能最切实可行的途径是增加数据的维度”。这正是引入大语言模型（LLM）的契机所在——LLM 能够高效处理传统信贷建模中难以利用的“替代数据”（Alternative Data）。 值得注意的是，替代数据不能与非结构化数据混为一谈。两者之间的分类维度与特性包括： - **数据结构形态**：结构化数据（如传统表格）仅占企业数据的约 20%；非结构化数据（图像、音频、视频、文本）占据了企业数据的 80%，但因其未按结构化组织，一直难以被信贷模型直接消化； - **替代数据范畴**：既可以表现为结构化形态，也可以是非结构化形态； - **核心机遇空间**：最具挖掘潜力的是两者的交集——**非结构化替代数据**。这类数据目前尚未被主流金融机构充分开发，涵盖了公用事业缴费、房租、移动支付等交易数据，线上购物与消费偏好等行为数据，社交媒体公开言论与评论数据，以及房产外观图片、房产挂牌描述和税务评估报告等不动产数据。 | 数据分类 | 常见形态与来源 | 传统利用程度 | LLM 赋能带来的新价值 | |:--- |:--- |:--- |:--- | | **传统结构化数据** | FICO 分数、年收入、债务收入比（DTI）、信贷账户年限 | 极高（主流信贷模型核心特征） | 自动化特征提取与清洗 | | **结构化替代数据** | 公用事业缴费记录表、标准化租金还款账单 | 中等（需特定数据集成渠道） | 辅助拓宽资信薄弱人群的评估维度 | | **非结构化替代数据** | 水电燃气账单扫描件、房产外观图像、社交媒体公开档案 | 极低（难以规模化结构化解析） | 结合多模态 RAG 实现信息秒级提取与量化分析 | ### [16:56](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1016s) 基于 RAG 的多模态非结构化替代数据提取 为了有效处理非结构化数据，金融系统依赖检索增强生成（RAG）技术与多模态嵌入（Multimodal Embedding）。在 RAG 工作流中，用户通过 Prompt 发起查询，系统若对接结构化数据则进行标准向量嵌入；若面对图像、音频或视频等非结构化数据，则调用多模态模型将其转换为高维向量索引并存储。当大语言模型接收到检索指令时，能从索引中迅速召回最相关的信息并生成解析结果。 Daniel 以 Google Gemini 为例，演示了利用多模态 LLM 提取三种非结构化替代数据的具体应用场景： - **公用事业账单（Utility Bills）解析**：给定一份水电账单图片，LLM 能够准确提取应缴金额、前期欠费（Previous Balance）以及已收付款（Payment Received）。例如账单显示前期欠费 223 美元、本期已收到付款 223 美元，这意味着借款人在上个月全额按期还款。该细节能帮助放贷机构直接穿透评估借款人的日常财务纪律与还款习惯； - **社交媒体公开信息解析**：LLM 可以从个人主页截图中提取借款人的姓名、工作单位、职位头衔以及粉丝关注量。庞大的粉丝群不仅是一道天然的反欺诈屏障——“欺诈者几乎不可能拥有如此庞大的粉丝关注量”，而且在当今数字化经济中，粉丝量往往直接关联个人变现能力或无形资产价值，可作为放贷机构核实身份与评估个人综合实力的参考； - **不动产图片（Property Images）智能分析**：向模型输入一张房屋外观照片，LLM 能精准识别出这是一座“双层殖民风格住宅，配有乙烯基壁板、红色入户大门以及维护良好的绿化景观”；当进一步追问时，模型能够合理推断出房屋室内面积约为 2400 至 2800 平方英尺、建造年份约为 2010 年，并详细列出推论依据。这一信息对放贷机构至关重要，因为房屋维护状况和真实体量直接决定了未来的房屋保险支出与潜在维修成本，有助于机构更精准地测算借款人的隐性财务负债。 ```mermaid flowchart TD accTitle: 多模态 RAG 替代数据提取工作流 subgraph Ingestion["非结构化多模态数据摄入"] A1["水电账单扫描件"] A2["社交媒体主页截图"] A3["房产外观照片"] end subgraph Embedding["多模态嵌入与索引"] A1 & A2 & A3 --> B["多模态嵌入引擎<br/>(Multimodal Embedding)"] B --> C[("多维向量数据库 / 索引库")] end subgraph Retrieval["RAG 检索与推理 (以 Gemini 为例)"] D["承保分析 Prompt 指令"] --> E["LLM / 多模态推理"] C --> E E --> F["结构化信贷特征输出<br/>(还款纪律 / 反欺诈 / 隐性负债)"] end ``` ### [21:59](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1319s) 大语言模型在样本生成、直接建模与可解释性上的前沿探索 除了从非结构化数据中提取特征，大语言模型在信贷风险建模中还展现出多维度的前沿应用潜力： - **合成数据生成与样本不平衡解决（Synthetic Data Generation）**：在传统信贷数据集中，违约（Charge-off）案例往往属于极少数的稀缺样本，导致严重的“样本不平衡”（Imbalanced Data）问题。以往通常采用重采样技术，而现在可以通过向 LLM 下发指令，例如要求其“依据底层数据类型分布、合理的特征组合逻辑与受控随机性，自动生成 10 条逼真的违约贷款合成样本”，并输出其合成推导逻辑，从而高效扩充少数派样本以辅助模型训练； - **LLM 直接参与信贷预测的性能跃升**：学术前沿文献显示，大语言模型自身的判别预测能力正呈现爆发式增长。在标准信贷风险数据集上，早期 ChatGPT（GPT-3.5）的 F1 分数仅约为 0.44，而 GPT-4 的 F1 分数已迅速攀升至 0.70 左右。Daniel 预测，在未来五年内，新一代前沿大模型（如 GPT-5 等）的直接预测表现完全有可能追平甚至超越传统的逻辑回归和随机森林模型； - **模型可解释性与监管合规赋能（Model Interpretability）**：在信贷风险建模领域，“构建一个高精度的模型仅仅完成了工作的一半”。金融机构面临着极为严格的合规与监管审查，建模人员必须向监管机构清晰阐述模型选择特定特征的经济学逻辑、因果关联及决策合理性，而具备自然语言生成与复杂逻辑归纳能力的 LLM 正成为打通黑盒模型与合规审计沟通的重要桥梁。 ### [24:17](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1457s) 基于大语言模型的多轮交互与信用违约可解释性分析 在信贷风控建模的实际落地中，大语言模型（LLM）展现出了强大的交互式归因与模型可解释性能力。演讲者以具体的数据记录为例进行演示：当输入一条预测违约率为 0.14（即 14%）的样本记录并向大模型提问影响该违约概率的核心因素时，模型迅速给出了排名靠前的关键特征，包括利率（Interest Rate）、FICO 信用评分、分期还款额（Installment）、最早信用额度/信用历史（Early Credit Line）以及循环授信使用率（Revolving Utilization）。 ```mermaid flowchart TD accTitle: LLM-Driven Multi-Turn Interpretability in Credit Risk subgraph DataInput ["样本与特征输入"] A["违约记录<br/>(违约率: 0.14)"] end subgraph LLMAnalysis ["LLM 多轮归因与解释"] B["识别关键特征<br/>(利率, FICO, 分期额, 循环使用率)"] C["解释因果逻辑<br/>(高利率推高违约, 高信用分降低风险)"] D["阐释模型机理<br/>(基于历史经验搜索数据模式)"] end subgraph FeatureExpansion ["特征拓展与决策"] E["引入外部维度<br/>(宏观经济, 行业考量, 人工审核)"] end A --> B B --> C C --> D D --> E ``` 演讲者随后展示了如何通过多轮提示词（Prompt Chaining）将上一轮模型的回复作为上下文继续深入追问。当被问及“用于信用风险评估的模型类型”时，大模型不仅指明其背后采用的统计学模型，还用通俗直观的逻辑进行了业务解释——例如“较高的利率会增加违约风险，而更好的信用评分则会降低违约概率”，从而有效辅助业务人员理解统计结果背后的业务因果关系。当进一步追问“该模型是如何训练的”时，大模型准确归纳出模型是通过分析数据、寻找潜在模式并“本质上从过往历史经验中学习”的。演讲者指出，尽管这些回答看似中肯而直白，但对于信贷风控业务的可解释性而言，已经提供了非常清晰且有价值的解释支持。 --- ### [25:56](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1556s) 突破原始数据局限：利用大模型拓展特征维度与思维链推理 大语言模型在信贷风控中的另一项独特价值在于能够超越既有数据集的边界，提供启发式的特征挖掘与思维链（Chain of Thought）推理支持。当演讲者向模型抛出更具开放性的决策问题——“在模型之外，还有哪些因素会进入最终的贷款审批决策？”时，大模型列出了贷款机构在实际决策中需要兼顾的额外维度，如宏观经济变量（Economic Variables）、行业考量（Industry Considerations）以及人工审核介入（Human Oversight）。 这一回复的关键意义在于，这些维度完全不在最初喂给模型的结构化原始数据集中，而是大模型基于其广博的行业知识库所给出的洞察。以“行业考量”为例，这启发风控建模人员去思考并深入探究该特定领域的细节，从而评估能否引入原本数据集中所缺失的新特征与变量。演讲者强调，这种基于大模型的思维链交互不仅能帮助优化风控模型本身，更能有效完善风控人员的分析逻辑与决策体系。 --- ### [26:55](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1615s) 大语言模型在信贷风控建模中的核心价值总结 演讲者在演讲尾声对其全套演示文稿的核心要点进行了系统性回顾，全面总结了大语言模型赋能信贷风控建模的五大关键能力： - **传统算法对标的预测与分类能力**：LLM 能够执行与传统机器学习算法相媲美的高精度信用预测与风险分类任务。 - **非结构化数据信息提取**：能够直接从海量文本、财报及其他非结构化风控材料中高效抽取结构化关键信息。 - **生成合成数据进行样本增强**：可生成高质量的合成数据（Synthetic Data），有效扩充与平衡训练数据集，缓解风控样本不均衡问题。 - **生成式 AI 的逻辑推理能力**：提供基于 Generative AI 的逻辑推理与因果探索，辅助风控专家进行全方位的思维拓展。 - **强化与支持模型可解释性**：将复杂的黑盒统计输出转化为业务人员与监管层易于理解的自然语言归因与决策依据。 在提炼完大模型在金融与银行业信贷风控领域的落地价值后，演讲者结束了本次分享，并宣布进入现场观众提问与互动交流环节。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=4s) 客户是否真实为其所声称的身份？ — Daniel Liliang Chen > [00:04](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=4s) 我是否愿意与该借款人开展业务？如果愿意，应以何种条款开展？ — Daniel Liliang Chen > [12:39](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=759s) 提升模型性能最切实可行的途径是增加数据的维度 — Daniel Liliang Chen > [16:56](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1016s) 欺诈者几乎不可能拥有如此庞大的粉丝关注量 — Daniel Liliang Chen > [16:56](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1016s) 双层殖民风格住宅，配有乙烯基壁板、红色入户大门以及维护良好的绿化景观 — Google Gemini > [21:59](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1319s) 依据底层数据类型分布、合理的特征组合逻辑与受控随机性，自动生成 10 条逼真的违约贷款合成样本 — Daniel Liliang Chen > [21:59](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1319s) 构建一个高精度的模型仅仅完成了工作的一半 — Daniel Liliang Chen > [24:17](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1457s) 较高的利率会增加违约风险，而更好的信用评分则会降低违约概率 — Daniel Liliang Chen > [24:17](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1457s) 本质上从过往历史经验中学习 — Daniel Liliang Chen > [25:56](https://www.youtube.com/watch?v=bq0Wt2hXmv0&t=1556s) 在模型之外，还有哪些因素会进入最终的贷款审批决策？ — Daniel Liliang Chen ## Mentioned Liliang (Daniel) Chen, 房利美 (Fannie Mae), 房地美 (Freddie Mac), AIG (美国国际集团), Ai4, Equifax, LendingClub, FICO, Google Gemini, ChatGPT, GPT-4, GPT-5, XGBoost, 随机森林 (Random Forest), 逻辑回归 (Logistic Regression), 检索增强生成 (RAG) ## Video description Liliang (Daniel) Chen, Financial Manager, Formerly of AIG and Fannie Mae Abstracts The shift toward LLM-enhanced credit risk modeling presents new opportunities in finance and banking, including processing large volumes of unstructured data, enhancing model performance, and improving interpretability. This talk will explore the evolving landscape of LLM integration in finance and highlight opportunities for impactful system design and development. [Source](https://www.youtube.com/watch?v=bq0Wt2hXmv0)

Source quotes

[0:04] Daniel Liliang Chen: 客户是否真实为其所声称的身份？ [0:04] Daniel Liliang Chen: 我是否愿意与该借款人开展业务？如果愿意，应以何种条款开展？ [12:39] Daniel Liliang Chen: 提升模型性能最切实可行的途径是增加数据的维度 [16:56] Daniel Liliang Chen: 欺诈者几乎不可能拥有如此庞大的粉丝关注量 [16:56] Google Gemini: 双层殖民风格住宅，配有乙烯基壁板、红色入户大门以及维护良好的绿化景观 [21:59] Daniel Liliang Chen: 依据底层数据类型分布、合理的特征组合逻辑与受控随机性，自动生成 10 条逼真的违约贷款合成样本 [21:59] Daniel Liliang Chen: 构建一个高精度的模型仅仅完成了工作的一半 [24:17] Daniel Liliang Chen: 较高的利率会增加违约风险，而更好的信用评分则会降低违约概率 [24:17] Daniel Liliang Chen: 本质上从过往历史经验中学习 [25:56] Daniel Liliang Chen: 在模型之外，还有哪些因素会进入最终的贷款审批决策？

Source quotes

Daniel Liliang Chen: The shift toward LLM-enhanced credit risk modeling presents new opportunities in finance and banking, including processing large volumes of unstructured data, enhancing model performance, and improving interpretability. [0:04] Daniel Liliang Chen: Is the customer who they say they are? [0:04] Daniel Liliang Chen: Do I want to do business with the borrower? If so, at what term? [14:01] Daniel Liliang Chen: No matter how sophisticated your model is, the improvement of your algorithm always have some limits. [24:17] Daniel Liliang Chen: higher interest rate will increase the default [24:17] Daniel Liliang Chen: better credit score will decrease the default [24:17] Daniel Liliang Chen: searching for the pattern [25:56] Daniel Liliang Chen: industry consideration

Original transcript

[0:04] Good afternoon everyone. My name is [0:05] Daniel Leang Chen. I'm a financial [0:07] engineer. Uh I have been working in [0:10] three Fortune 100 financial institution [0:12] including Fanny May, Freddy Mack and [0:15] AIG. Today I want to uh share my [0:19] insights about how to use LM LM in [0:23] credit risk modeling. Before I start, I [0:25] want to see how many of you are involved [0:28] in the financial modeling. Can you raise [0:31] your hands? [0:33] Okay. How many of you have used the LRM [0:36] for your for your for your work purpose? [0:39] Can you raise up your hand? Great. [0:47] So in a traditional financial [0:49] institution, credit risk is very [0:51] important because is about to make sure [0:53] the money moving safely. [0:56] Uh the credit decision ecosystem is a [0:59] complex network including the lender, [1:04] borrower, regulator, [1:06] shareholder and even foster. [1:11] The credit risk is the possibility that [1:14] a borrower will fail to repay a DAP [1:17] obligation to the lender. In order to [1:20] price this risk, the lender needs a [1:23] credit risk model [1:27] to price the risk [1:30] and how does the credit risk model fits [1:32] in the business process. Here we have a [1:36] borrower and the lender. The first [1:39] question the lender will ask is is the [1:42] is the customer who they say they are. [1:47] This is where the lender will screen out [1:49] the the borrower from a froster through [1:53] a invest fraud investigation. [1:57] The second question the lender will ask [2:01] do I want to do the business with the [2:03] borrower? If so, at what term? [2:08] So, the lender needs a credit risk model [2:11] in order to make the correct price for [2:14] the risk. If the borrower and the lender [2:18] agree on the price and the borrower [2:21] accept the offer, he will get the money [2:24] or cash flow from the lender and repay [2:27] the loan on a regular basis. [2:31] The process of issuing the loan at the [2:34] uh uh determined price is called [2:37] underwriting. [2:41] Before I jump in the credit risk model, [2:43] I want to introduce a brief history of [2:47] credit risk modeling back to 1950 to [2:51] 1960 [2:53] where we call the early credit system. [2:56] At that time the decision is made manual [2:59] and based on limited information and [3:02] personal judgment. [3:05] In 1970 the credit builder start to [3:08] emerge centralize the credit the [3:10] borrowers information and make the [3:13] information available to the public. [3:16] 1980s has brought the credit credit [3:20] score the brought the biggest game [3:23] changer to the markets which is the [3:26] credit score which is the single [3:28] standardized the number [3:32] from 1990 to 2000s [3:36] we have seen the rise of credit credit [3:39] based risk based credit decision engine [3:43] which is based on multiple factor [3:45] instead instead of based on a single [3:46] number. [3:48] From the 2000 to 2020, [3:52] the AI based and the AI and the risk [3:55] based decision [3:58] became available [4:00] and uh since 2022 we have reached the [4:04] stage of [4:06] generative AI where LLM has brought the [4:11] speed, scale and sophistication to the [4:15] credit risk modeling to the modeling [4:17] side. Here [4:21] we can see a example of the credit risk [4:24] report from the Equifax. It report a [4:27] borrower's credit score of 743. [4:30] The credit score majorly come from the [4:34] two factor. [4:36] One is your [4:39] payment history. The second one is the [4:41] credit utilization. Together, these two [4:45] made up of almost 65% of your credit [4:48] score. The remaining of 35 are coming [4:51] from your credit history, credit [4:53] experience, and recent inquiry. [5:00] Some some of you might ask, if we [5:03] already have a credit score, why do we [5:05] still need a credit risk modeling? [5:08] To answer that, let me show you an [5:10] example. [5:12] Here we have a two borrower. One has a [5:14] credit score of 700. The other one has a [5:17] credit score of 650. Which one has less [5:20] risk? It's very simple, right? The first [5:22] one has less risk. [5:26] Next one with the same credit score as [5:28] before. The borrow A has an annual [5:31] salary of 100 100,000s per year and the [5:34] annual depth of 50,000 per year. But the [5:37] borrow B has 80 80,000 of the salary and [5:42] 60,000 of deps. Which one has a less [5:45] risk? [5:47] Because the borrow A has higher salary [5:49] and lower deps. It has a better credit [5:53] performance and less risk. Also, it has [5:55] a higher score. So, this borrow A should [5:57] have a less risk. [6:00] The third one, the credit score is the [6:03] same. If we flip over their salary and [6:06] the depths. So credit A the borrow A has [6:10] lower salary with higher depths but the [6:14] credit B has a higher salary with lower [6:17] depths. Which one has a which one has [6:21] less risk? It's very difficult, right? [6:24] Because you cannot just tell by looking [6:26] at the credit score. That's why in this [6:29] sense we still need a model to evaluate [6:32] what's the credit risk of the borrower. [6:39] So when a lender are evaluating the [6:42] application [6:43] normally they will focus on a few [6:46] factors which I call 5C. The credit [6:50] history. [6:52] Credit history is [6:55] how much you is the performance you will [6:58] borrow and repaying your money. [7:03] Capacity of repay is how much you have [7:08] earned and how much you own. [7:12] Capital is the money you either your [7:17] salary or your investment you can [7:20] withdraw if you need it. Condition of [7:24] loans [7:26] is how much the amount you borrow, [7:32] how much how soon you can pay back and [7:34] the purpose of your loan. Collateral is [7:39] something valuable you can promise to [7:42] pay to the lender if you cannot repay [7:45] the loan. [7:49] The general rule is the more stronger in [7:52] these five category you have less risk. [7:57] Uh less risk will normally translate [8:00] into a better offer such as you know [8:03] lower interest rate offer. Here [8:08] is a sample data from the landing club. [8:11] I use just landing club in case you know [8:13] I don't use some other pri proprietary [8:16] information. So here you can see the [8:18] loan condition including the [8:22] loans loan amounts loan term and [8:24] interest rate. The capacity to repay [8:27] including the employment lens home [8:30] ownership. The capital includes the [8:32] annual income. your adapt to income [8:35] ratio, the credit history include the [8:37] earliest credit line, your FICO score or [8:41] credit score and your revol balance and [8:44] also the target variable normally is our [8:47] loan status either is a fully pay or [8:49] it's a charge off. Normally we call [8:51] charge off as a default. [8:56] Then once we try to build a model, we [8:59] want to choose either a very simple [9:02] logistic regression model which is [9:04] easily it interpretable and easy to be [9:07] implemented or we can choose a more [9:11] complic complex model such as the random [9:14] forest model which can capture more [9:17] nonlinear behavior of the data or neural [9:21] network [9:23] which can process mass Massive feature. [9:32] The goal of credit risk modeling is to [9:36] find [9:38] a function of functions [9:41] which can split the default case from [9:44] the undefault loans. [9:49] This function is normally is a linear [9:52] combination of different feature [9:56] and the solving of the optimization is [9:59] normally done through finding this hyper [10:02] plan [10:04] in the hyper dimension space. [10:08] This hyper plan can be expressed as w + [10:14] w1 x1 w2 x2 all the way to wxn equal to [10:19] zero. So finding this hyper plan it can [10:24] help us to divide the case between [10:28] default case versus non-default case. [10:34] After we determine the after we have [10:37] decided data we still want to understand [10:40] which variable might be related to the [10:42] target variable. [10:44] Here I split the data into two groups. [10:49] The fully paid show in blue and the [10:54] charge off show in orange. And then I [10:58] plot out their distribution curve. [11:01] Here I plot out their distribution curve [11:03] for interest rate for credit score and [11:07] the dep to income ratio. The idea is [11:10] very simple. [11:12] If the target variable is related to the [11:14] independent variable, these two [11:16] distribution curve should [11:20] uh if the target variable is not related [11:23] to the independent variable, these two [11:27] curve should be close. But here you can [11:30] see these two curve, they are not [11:32] closed. They are quite different. So [11:33] that difference signify or indicates [11:37] there's a predictive power between the [11:40] independent variable versus the target [11:43] variable. It make us think about that's [11:45] a a good candidates to leave in the [11:49] model. [11:54] Once we have the data, the data is only [11:56] half the battle. The feature engineering [12:00] will help us to transform the raw data [12:04] into more meaningful signal. It involve [12:07] with transforming the variable, [12:12] normalizing the data, normalizing the [12:14] variable and also sometimes we need to [12:17] encoding the the categorical data. [12:21] After that we want to compare the [12:26] the difference of each variable in terms [12:29] of their importance and then decide [12:32] which variable we want to leave and [12:33] which variable we want to remove from [12:35] the model. [12:39] Here is a credit risk model performance [12:42] compared between different uh models. [12:45] For example, here I show the logistic [12:46] regression, random forest, XG boost and [12:49] the convolution neural network. [12:52] And uh for you uh here I show the matrix [12:55] using F1 score. For you if you are not [12:58] familiar with F1 score, F1 score is a [13:01] combination of precision and recall. The [13:04] precision is defined by true positive [13:06] divide by true positive plus [13:10] false positive. Which means the true [13:12] positive divided by the [13:15] the forecasted positive and the recall [13:18] is defined by true positive divided by [13:21] true positive plus the false negative [13:23] which is the true positive divided by [13:26] the actual positive [13:28] and the f_sub_1 is the combination of [13:31] these two. So normally the higher f1 [13:34] score means the the higher performance. [13:37] The why logistic regression has lower F1 [13:41] score than the machine learning model. [13:43] That's because logistic regression model [13:45] normally they can only capture the [13:47] linear relationship of the data or maybe [13:49] they are not that complex but some of [13:53] the machine learning they can capture [13:55] more more nonlinear relationship of the [13:57] data. [14:01] There are some challenges in credit risk [14:04] modeling including the data quality, [14:06] model performance and the model [14:08] interpretability. [14:12] One of the biggest hurdle in the credit [14:14] modeling [14:16] regardless of your algorithm is your [14:19] data quality is your data. [14:23] No matter how sophisticated your model [14:26] is, [14:27] the improvement of your algorithm always [14:30] have some limits. [14:33] So in order to a a more [14:38] practical [14:40] way to boosting your model performance [14:43] is by increasing the dimensionality of [14:46] your data. So with LLM [14:50] LM can help us to bring some additional [14:52] data which are not traditionally used in [14:55] the credit risk modeling. [14:59] Those untraditional data which is not [15:02] used in the standard way is called [15:04] alternate data. [15:06] Uh but it's important not to confuse [15:10] alternate data with unstructured data. [15:15] Unstructured data is opposite to [15:17] structured data. Unstructured data [15:20] normally is in a format of some like [15:23] some form of the image, audio, video. [15:28] They are probably they are 80% of your [15:30] data and the structure made data maybe [15:32] only your 20% of your data. But we still [15:35] rarely see unstructured data are widely [15:37] used for enterprise purpose. Why? [15:40] Especially for credit risk modeling. [15:42] Why? Because this data is not convenient [15:46] to be consumed since they are not [15:48] sitting in a organized way. [15:52] Alternative data actually can be either [15:54] structured data or unstructured data. [15:58] The promising opportunity lies for us is [16:01] using the unstructured alternative data [16:04] which is the overlapping between the [16:06] alternate data and unstructured data [16:09] because they are not widely explored by [16:13] the current financial institution. [16:20] So what's the alternative data we could [16:22] use in the credit risk modeling that [16:25] including some of the transactional data [16:27] which including the utility payments [16:30] rent payments and mobile payments and [16:33] also some of the behavior data including [16:36] the online shopping patterns the [16:38] spending category. [16:41] some of the social media data including [16:43] the public post, online review [16:46] and some of the property data including [16:48] the property image, real estate listing [16:51] and the tax assessment. [16:56] How can [16:58] use unstructured data? It rely on a [17:01] techniques called rack which stand for [17:04] retrieval augmented generation. How does [17:08] rack works? When a user want to make a [17:12] query to the database through a LM [17:15] prompt, the LM will go in will will go [17:18] into the database [17:21] to retrieve the relevant information. [17:24] If the data is structured data, LLM will [17:28] make the embedding. [17:30] If the original data is unstructured [17:32] data, it will rely on [17:36] multimodel embedding to handle the [17:39] formats like image, audio, video and uh [17:43] after embedding it will the data will be [17:46] indexed and the later LRM will retrieve [17:51] the information from index information [17:54] and then send back the response back to [17:57] the user. Here is an example of using [18:01] the Google Gymni to to make the [18:05] to make the rack core. First step we try [18:09] to use we try to initialize the Gemini [18:13] AI model and then [18:17] we are pointing to the source data. [18:19] Right now we are point to a image and [18:22] later we want to ask oh can you extract [18:24] some information from from the source [18:27] data and the later and the last one the [18:30] LRM will give us on the response. [18:35] So here is an example how LM use the [18:38] utility data. The left side is a sample [18:41] utility bill and if I ask the LM can you [18:46] extract the information from the utility [18:48] bill. So the right side the the LM will [18:52] tell me oh I can extract this [18:53] information for you. [18:56] What's important here is the the amount [19:00] due previous balance and the payment [19:03] received. You see the previous balance [19:06] 223 and the payment received 223. That [19:10] means the borrower has paid back the the [19:12] the utility in the last month. So this [19:15] in information is very relevant for the [19:18] for the lender because they can use this [19:20] information to evaluate your financial [19:22] habits and your financial capability [19:30] and uh LM also can extract information [19:34] from the social media. So here just an [19:36] example LM can pass the a person's name, [19:42] affiliation, job title and uh the [19:46] followers to the to you. So here what's [19:50] important to us for example the [19:52] followers number [19:54] here the followers number can identify [19:56] if you have so many followers it can [19:59] prevent the froster right you it's [20:02] almost impossible to have a froster to [20:04] have so many followers. The second one [20:06] is if you have so many followers that [20:09] might signify some sometime the follower [20:12] might be related to your personal wealth [20:14] or maybe followers means some intangible [20:17] value or intangible asset for a [20:19] particular person. So when lender [20:21] consider that they can also incorporate [20:23] this information into their into their [20:25] credit risk information consideration [20:30] and this is a property data we can use [20:36] to extract the property data from from [20:38] from the the picture if I asked what's [20:43] the information from the property the [20:47] model will tell me this is a two-story [20:48] building with colonial style home with [20:51] vinyl sh siding red front front door it [20:54] also tell me it's wellmaintained the [20:55] landscaping [20:58] and if I keep ask the the LR model can [21:02] you estimate what's the house size lot [21:04] size number number of story building [21:07] year garages capacity it can even tell [21:10] me his estimation of a house size 240 [21:13] 200 240 400 to 2800 square ft with [21:18] different lo price unit year is 20210. [21:22] It also tell me the reasoning why he [21:24] think that way. So with this information [21:28] why this one is important to us [21:31] because when the when we think about the [21:32] ported information the property [21:35] information might not uh is very [21:38] important for a person's home insurance [21:40] and increasing increasing [21:44] uh maintenance costs. [21:48] So a lender understand that they can [21:49] better evaluate what's the financial [21:51] obligation for a borrower. So they can [21:54] capture that in their consideration. [21:59] Besides the data instruction, the alm [22:03] can also help us to generate a synthetic [22:05] data. So in credit risk model sometimes [22:10] we have a issue called imbalanced data [22:12] issue. So because in this original data [22:15] the default case is rare case and the [22:18] undefa case as is is a majority case. [22:22] So in order to to move away from the [22:26] imbalanced data normally we need to [22:28] resample the the default case to make [22:32] that more. So I can ask the LM model I [22:36] would like to generate 10 more charge of [22:39] loans. Can you help me to generate that [22:40] that? So this is LM can do for me [22:43] because they will generate the 10 more [22:46] actual D4 loan for me and it also tell [22:50] me what's the reason he is generating [22:52] these loans. So right now he can tell [22:55] you he he is based on this data type [22:58] distribution plausible combination [23:01] randomness to generate this some [23:04] synthetic data [23:06] and also LRM model can help us in the [23:09] model performance. [23:12] we we just have this one as the original [23:14] model performance and uh also there's [23:18] some literature in they show the chart [23:21] GBT and the GBT4 can have this F1 score [23:26] even right now the CH GP GBT we can see [23:29] within this few years there's a lot a [23:32] big improvement from the performance of [23:34] CH GPT to GPT4 the F1 score increased [23:37] from 0.4 four to all all the way to 0.7 [23:41] and we can see maybe in the next five [23:43] years maybe even CH GBT or GBT4 five [23:46] they may even beat logistic regression [23:48] model or random forest model that's [23:50] could be possible so this is the [23:53] literature you can see [23:56] oh if you want to take a picture also [24:01] model can help us to improve the model [24:04] interpretability in the credit modeling [24:06] the model to build a good model is only [24:10] one side also we also face the [24:12] regulation we have to explain the model [24:14] to the regulator why we want to choose [24:17] certain particular model so our model [24:20] can also help us on that for that front [24:26] so since I already [24:28] uh have a data records and which has a [24:31] default default rate of 0.14 so as the [24:35] our model can tell me how what's the top [24:40] factors affecting this this default [24:43] probability. So the R model will tell me [24:46] the biggest factor he's he's getting is [24:50] the interest rate, FICO score, [24:52] installment, early credit line and [24:54] revolving utility. And if I keep asking [24:58] and use the previous response as a [25:00] prompt and keep asking with oh can you [25:03] tell me what kind of model you are using [25:05] for the credit risk assessment [25:09] and uh the model is tell me he's using [25:11] the statics model and also he explained [25:14] you know the higher interest rate will [25:16] increase the default the better credit [25:18] score will decrease the default so it [25:20] actually it help me to explain something [25:24] and also if I keep Using the previous [25:27] promp previous prompt and I keep asking [25:30] how is the model trained [25:34] the L model will tell me the model will [25:37] analyze the data and searching for the [25:39] pattern and the model essentially is [25:42] learn from the past experience. Of [25:44] course, this is just maybe some neutral [25:47] very honest arguments, but I think this [25:51] as a list is explain something. [25:56] The last one I will ask what other [25:58] factors besides the model go into the [26:01] final application decision and uh the [26:05] model will tell me besides the model [26:09] when lender make the decision they will [26:11] also consider several other factors such [26:13] as the economic variable industry [26:15] consideration human oversightes what's [26:18] interesting here is this is not from the [26:22] original data right this is what the LR [26:25] is telling me. So for example, when I [26:28] look at the number two in industry [26:29] consideration so maybe I'm thinking [26:31] about maybe I can look into more detail [26:34] about this particular area to see what [26:37] other variable what other features I can [26:39] bring in which is not originally in my [26:41] original data set. So that's the [26:43] additional capability which L model are [26:46] actually helping me to improve my model [26:49] or or use this channel of thought to [26:51] help me to refine my thinking logic. [26:55] to summarize to summarize my deck. So [26:59] the LR model can you know perform the [27:02] prediction and the classification [27:04] similar to a traditional machine [27:06] learning algorithm. It can extract [27:09] meaningful information from an structure [27:11] data. It can generate a synthetic data [27:14] to augment training data sets. It can [27:17] provide generative AI based reasoning. [27:20] It can enhance and support model [27:22] interpretability. [27:24] So this conclude my my talk today. Yeah. [27:29] If you if you have any question [27:32] they feel free to ask.