Bloomberg Deploys DelFin AI to Outperform Classical Financial Models
Ai4
Deep generative time-series architectures and neural factor models beat classical linear baselines by capturing non-linear market dependencies and dynamically adjusting hedge ratios during volatility shocks.
AI-assisted summary of Deep Generative Modeling of Financial Time Series With Bloomberg LP — original source. Check the original for full context.
Source published:
Detailed notes
## Background Published by the Ai4 channel on YouTube on August 29, 2025, this talk features Arun Verma, Head of Quantitative Research Solutions at Bloomberg LP. According to the page description, the session explores generative machine learning methods for financial time series forecasting, financial data imputation, and collaborative filtering for illiquid asset pricing using deep generative architectures.
## TL;DR Arun Verma outlines how Bloomberg is translating machine learning breakthroughs into numerical finance, moving from domain-specific language models like BloombergGPT to financial time series foundation models. He introduces DelFin, a pre-trained foundation model built to handle tasks ranging from economic nowcasting to long-horizon intraday volume prediction. Verma also details Neural Factor Modeling, which extracts non-linear latent stock embeddings to optimize portfolios, as well as Neural Beta, a deep architecture that models dynamic CAPM beta and adaptively handles extreme market shocks like the COVID-19 volatility spike.
## Atmosphere & style The presentation begins with self-deprecating humor about presenting in the post-lunch conference slump before shifting into a polished, highly technical discourse. Verma speaks with the pragmatic authority of a quantitative researcher, contrasting the forgiving nature of conversational text AI with the razor-thin error margins required in live financial markets. The talk carries a steady, analytical momentum, bridging classical econometrics with modern deep learning architectures in an engaging and accessible style.
## Key takeaways - [00:04](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=4s) Bloomberg maintains a 500-person quantitative research and AI division within an engineering workforce of roughly 10,000, cultivating an open research culture alongside proprietary finance technology. - [03:51](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=231s) Financial time series modeling demands a vastly lower error tolerance than textual NLP, as inaccuracies directly degrade portfolio optimization, risk estimation, and alpha generation. - [07:22](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=442s) Bloomberg developed DelFin, a pre-trained foundation model for financial time series designed to replace bespoke, fragmented models across tasks like risk forecasting, imputation, and anomaly detection. - [07:22](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=442s) DelFin enables economic nowcasting by integrating alternative data sources, such as granular consumer credit card transactions, to project company earnings ahead of official disclosures. - [07:22](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=442s) The foundation model captures structural intraday microstructure patterns, successfully forecasting 7-day rolling intraday volume curves at 5-minute resolution. - [16:34](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=994s) Neural Factor Models move beyond fixed Fama-French characteristics by jointly learning dynamic factor exposures and latent factor trajectories directly from raw asset return series. - [19:20](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=1160s) Rather than producing deterministic point predictions, deep generative asset pricing models output full multivariate return distributions with explicit uncertainty bands. - [22:15](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=1335s) Log-likelihood calibration identifies an optimal capacity of 64 latent factors, capturing complex non-linear market interactions that traditional linear PCA fails to detect. - [22:15](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=1335s) Unsupervised embeddings naturally organize stocks into intuitive economic clusters, allowing illiquid instruments to borrow pricing signals from closely related liquid neighbors. - [25:31](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=1531s) Neural Beta replaces rolling OLS regressions with non-linear deep neural networks optimized via maximum likelihood to track dynamic, time-varying market beta. - [29:02](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=1742s) During extreme market dislocations like the 2020 COVID shock, Neural Beta intelligently down-weights transient extreme spikes to maintain stable, realistic hedge ratios.
## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=4s) Bloomberg’s R&D Culture and AI Journey
Opening during the post-lunch conference session, Arun Verma jokes about being assigned to the "lunch slump hour" before outlining his role within Bloomberg's Chief Technology Office (CTO), where his group leads innovation and incubation for emerging products. Connecting his session to earlier events, he notes Geoffrey Hinton’s keynote conversation with Bloomberg reporter Shireen Ghaffari. Verma emphasizes that while Bloomberg is known for journalism, it operates fundamentally as a global technology powerhouse serving 350,000 customers worldwide.
Verma underscores that roughly 40% of Bloomberg’s workforce—between 9,000 and 10,000 employees—are software engineers, including a dedicated cadre of 500 PhD-level quantitative finance researchers, machine learning scientists, and AI researchers. Unlike conventional Wall Street institutions that closely guard intellectual property, Bloomberg embraces a Silicon Valley research culture by actively contributing back to the open-source ecosystem.
Tracing the company's 15- to 20-year history with artificial intelligence, Verma points to Bloomberg News as their earliest production machine learning deployment. Every story published by internal journalists or received from third-party wire services was ingested into a real-time natural language processing (NLP) pipeline that tagged corporate entities, key executives, topic themes, and sentiment analysis. This long-standing foundation culminated two years prior with the release of BloombergGPT, demonstrating that domain-specific large language models trained on Bloomberg's proprietary financial data—spanning analyst reports, news, and market filings—could outperform generic models on specialized financial workflows.
- Bloomberg operates with 350,000 global financial clients and a technical staff of 9,000–10,000 engineers. - The company maintains a 500-person scientific research division holding PhDs in quantitative finance, machine learning, and AI. - The real-time NLP news pipeline established the company's early ML infrastructure by automatically tagging sentiment, named entities, and topics. - BloombergGPT established the blueprint for training domain-tailored autoregressive transformer models on specialized financial corpuses.
---
[03:51](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=231s) Adapting Transformers to Financial Time Series
Verma explains how researchers in quantitative finance are translating NLP transformer paradigms to numerical time series. Rather than tokenizing subwords, numerical observations across time steps are converted directly into continuous or discretized tokens and processed through standard attention and encoder-decoder mechanisms. However, the performance tolerance in quantitative modeling is far more unforgiving than in conversational NLP. While users tolerate minor hallucinations or vague answers in ChatGPT, financial risk management, alpha generation, and portfolio optimization demand strict precision:
> "In the textual setting we have accepted a little bit of more margin of error and the vagueness of response... but when you're trying to solve a financial time series question—for example, estimation of risk or projection of returns or optimizing your portfolio—there is very little room for error. You really want to make sure you are able to build a model that can definitely beat the benchmarks, otherwise it's not useful at all."
Verma charts the mathematical lineage of time series and asset pricing models. Early economic modeling began with William Sharpe's single-factor Capital Asset Pricing Model (CAPM) in the 1960s, which earned a Nobel Prize, followed by multi-factor paradigms such as the Fama-French models. Over the past century, statistical forecasting evolved from classical autoregressive techniques like ARMA and Robert Engle's Nobel Prize-winning GARCH family to LSTMs, attention-based architectures, and modern diffusion models repurposed from image generation to simulate continuous financial distributions.
```mermaid flowchart LR accTitle: Evolution of Financial Time Series Modeling
subgraph Classical["Classical Econometrics"] A["1960s: CAPM<br/>(Sharpe)"] B["ARMA & GARCH<br/>(Engle)"] C["Fama-French Factor Models"] end
subgraph ModernML["Machine Learning & Deep Architectures"] D["Recurrent Models<br/>(LSTM / GRU)"] E["Time Series Transformers<br/>(Attention / Encoders)"] F["Generative Diffusion<br/>& Foundation Models"] end
A --> B --> C --> D --> E --> F ```
---
[07:22](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=442s) DelFin: A Foundation Model for Financial Time Series
To replace the fragmented paradigm where data scientists build handcrafted, isolated models for individual tasks (e.g., separate custom architectures for 5-minute price predictions, anomaly detection, one-day horizon forecasting, or data imputation), Bloomberg developed **DelFin**—a foundation model named by blending the mythological oracle Delphi with finance. DelFin is pre-trained across diverse cross-domain numerical series, including macroeconomic indicators, equities, order book data, and non-financial data such as climatological and scientific series.
```mermaid flowchart TD accTitle: DelFin Foundation Model Architecture and Task Execution
subgraph Ingestion["Pre-Training Modalities"] D1["Financial Time Series<br/>(Equities, FX, Yields)"] D2["Macro & Alternative Data<br/>(Receipts, News)"] D3["Scientific & Climate Data"] end
subgraph Core["Foundational Core"] FM["DelFin Pre-Trained Backbone"] end
subgraph Execution["Downstream Execution"] FT["Task Fine-Tuning<br/>(Labeled Examples)"] ZS["Zero-Shot Inference"] end
subgraph Tasks["Target Financial Use Cases"] T1["Variance & Risk Forecasting"] T2["Economic Nowcasting"] T3["Intraday Volume Forecasting"] T4["Missing Data Imputation"] end
D1 & D2 & D3 --> FM FM --> FT & ZS FT --> T1 & T2 & T3 & T4 ZS -. Lower Accuracy.-> T1 & T2 ```
Verma outlines DelFin's operational performance across distinct benchmarks:
1. **Variance and Volatility Forecasting:** Multivariate risk modeling for correlated multi-asset portfolios over weekly or 3-month horizons. Fine-tuned DelFin strictly outperforms classic GARCH baselines, whereas zero-shot inference remains competitive but less robust. 2. **Economic Nowcasting:** Synthesizing non-synchronous, lagging economic releases (e.g., staggered housing starts vs. monthly nonfarm payroll prints). DelFin incorporates Bloomberg’s granular receipt-level retail credit card transactions across the United States to produce real-time quarterly revenue projections for public consumer companies well ahead of official earnings reports. 3. **Long-Horizon Intraday Volume Forecasting:** Predicting equity trading activity over a rolling 7-day period (projecting out roughly 500 steps of 5-minute bars, comprising 78 bars per trading day). The model reproduces the classic intraday "U-shaped" curve—characterized by high volume spikes during the 9:30 AM opening market auctions and 4:00 PM closing crosses, with subdued mid-day trading lulls.
While general-purpose time series foundation models exist (such as Chronos, TimesFM, MOMENT, and Moirai), DelFin is specifically engineered for financial dynamics. Verma highlights ongoing technical challenges, such as modeling long context windows for currency pairs (e.g., JPY/AUD) across multiple years of macro cycles, imputing missing data before downstream forecasting, and managing experimental overhead during fine-tuning.
- Traditional pipelines require distinct pipelines and bespoke datasets for 5-minute forecasting, anomaly detection, daily returns, and imputation. - DelFin uses a single pre-trained multi-modal backbone adapted to downstream tasks via few-shot or full fine-tuning. - Intraday volume forecasting spans 7 trading days at 5-minute resolution (78 intraday bars/day, totaling ~500 forecast steps). - The model captures standard market microstructure patterns, including morning order imbalance absorption and market-on-close execution surges.
---
[16:34](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=994s) Deep Learning and Factor Modeling Paradigms
Verma transitions from universal foundation models to specialized deep learning formulations for asset pricing, focusing on **Neural Factor Modeling**. In quantitative finance, asset returns are traditionally decomposed into factor exposures ($\beta$), systematic factor returns ($F$), and idiosyncratic residual noise ($\epsilon$):
$$R_{i,t} = \beta_{i,t} F_t + \epsilon_{i,t}$$
Verma reviews the three distinct structural methodologies historically used to solve this equation:
| Approach | Known Variables (Observable) | Unknown Variables (To Estimate) | Mechanism / Description | |:--- |:--- |:--- |:--- | | **Fama-French Style (Style Portfolios)** | Factor returns ($F$) | Factor loadings / Betas ($\beta$) | Pre-defined long/short systematic factor portfolios (e.g., Value: High Book-to-Market vs. Growth; Size: Mega-Cap vs. Micro-Cap). | | **Characteristic-Based (Explicit Betas)** | Factor loadings / Betas ($\beta$) | Factor returns ($F$) | Observable firm characteristics (e.g., historical dividend payouts, balance sheet metrics) act as fixed $\beta$, while regression learns factor returns $F$. | | **Latent / Deep Factor Models** | None (Returns $R$ only) | Both Betas ($\beta$) and Factor Returns ($F$) | Both factor trajectories and asset-specific sensitivities are treated as unobserved latent states, extracted simultaneously using non-linear deep architectures. |
[19:20](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=1160s) Neural Factor Models and Stock Time Series Embeddings
Moving beyond predefined fundamental or statistical factors, Arun Verma introduces a latent factor approach where the model operates in a data-driven regime, learning both latent factors and their implied returns simultaneously in an iterative manner. Rather than imposing rigid factor definitions or fixed exposures, the framework constructs dense representations of equity time series analogous to modern text embeddings. The neural factor architecture compresses the historical time series of an entire stock universe into an embedding space and pairs it with a factor return model, postulating that individual stock returns can be expressed as their factor exposures ($\beta$) multiplied by the latent factor returns. The entire joint system is trained end-to-end via standard neural network backpropagation to minimize reconstruction error and maximize data representation fidelity.
The objective is framed probabilistically by maximizing the posterior likelihood of observing the actual historical return realizations conditional on the latent factors. By optimizing this posterior distribution, the network finds an optimal representation of the stock universe within the learned embedding space. Crucially, the model does not merely generate deterministic point forecasts—such as predicting that "Apple's return for tomorrow is going to be 2%"—but instead outputs a full multivariate return distribution with explicit uncertainty estimates and confidence bands. This probabilistic formulation allows portfolio managers and risk systems to quantify the degree of estimation confidence under shifting market regimes.
```mermaid flowchart LR accTitle: Neural Factor Model Architecture subgraph DataPrep["Stock Universe Representation"] A["Stock Time Series History"] --> B["Neural Time Series Embedding"] end subgraph FactorEngine["Joint Optimization"] B --> C["Dynamic Factor Exposures (Beta)"] D["Latent Factor Returns"] --> E["Predicted Return Distribution"] C --> E end subgraph Output["Probabilistic Outputs"] E --> F["Point Estimates & Confidence Bands"] E --> G["Multivariate Risk & Uncertainty"] end ```
[22:15](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=1335s) Factor Dimensionality, Non-Linear Dependencies, and Downstream Applications
Examining the model calibration across different latent factor dimensions reveals a distinct U-shaped log-likelihood curve. The loss drops sharply as factor capacity increases, bottoming out at an optimal dimensionality of 64 factors before increasing slightly at higher dimensions. This optimal number of 64 latent factors is substantially larger than what traditional Principal Component Analysis (PCA) identifies. Because PCA is strictly a linear dimension-reduction technique, it fails to pick up the nuanced, conditional non-linear dependencies present in real market data. The non-linear neural network framework successfully captures these complex dependencies, translating into superior distributional accuracy.
Once calibrated, these generative representations unlock several downstream quantitative workflows, including dynamic variance forecasting, Value at Risk (VaR) estimation, and portfolio optimization. Projecting the high-dimensional stock embeddings down to two dimensions using UMAP confirms that the latent space naturally organizes the market into economically intuitive clusters: banking institutions cluster together near the top, market vendors align across the center, and technology companies gather in a dedicated corner. For asset pricing and risk tasks, an illiquid target stock can borrow informational strength directly from adjacent neighbors in the embedding manifold. When plugged into a Sharpe ratio optimizer, the generative return and risk distributions produce systematically higher realized P&L compared to conventional baseline portfolio optimization strategies.
- **Optimal Latent Capacity**: Log-likelihood optimization peaks at 64 latent dimensions, capturing non-linear interactions invisible to standard linear PCA. - **Natural Market Clustering**: 2D UMAP projections show self-organized clusters of banking, technology, and vendor sectors without explicit manual tagging. - **Enhanced Portfolio Performance**: Feeding generative return distributions and uncertainty metrics into Sharpe ratio optimizers yields improved risk-adjusted P&L over standard mean-variance baselines.
---
[25:31](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=1531s) Neural Beta: Estimating Dynamic CAPM Beta with Neural Networks
Verma transitions to a related initiative called "Neural Beta," which reimagines the classic Capital Asset Pricing Model (CAPM). In the standard CAPM framework, an asset's expected excess return is governed by its market beta ($\beta$). In practice, beta serves as the primary metric for hedging single stocks against index fluctuations—where an investor holding a long equity position shorts $\beta$ times the market index—and plays a foundational role in portfolio construction. However, standard CAPM estimation struggles because market beta is inherently dynamic and fluctuates significantly over time rather than remaining constant.
```mermaid flowchart TD accTitle: Evolution of Beta Calibration Models A["Standard OLS Regression\n(Static Historical Window)"] --> B["Weighted Least Squares\n(Exponential Alpha Decay)"] B --> C["Neural Beta Architecture\n(Non-linear Likelihood Maximization)"] ```
To capture these temporal shifts, the objective is structured to minimize the discrepancy between the realized next-day stock return and the model-implied return. Traditional quantitative approaches rely on Ordinary Least Squares (OLS) over rolling lookback windows as a baseline strawman, or extend it to Weighted Least Squares (WLS) with an exponential alpha decay parameter to place heavier emphasis on recent data points. Neural Beta replaces these linear heuristics with a deep neural network that directly models the non-linear relationship between underlying market features and dynamic beta. Furthermore, the loss function shifts from simple Root Mean Squared Error (RMSE) to maximum likelihood estimation, simultaneously minimizing prediction errors while maximizing the true observation probability.
---
[29:02](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=1742s) Synthetic Evaluation and Real-World COVID-19 Adaptation
Evaluating beta estimation on real market data poses a fundamental challenge: true underlying beta is a latent parameter with no observable ground truth. To establish rigorous empirical validation, the team evaluates the models in controlled synthetic environments generated across various test patterns, including constant betas, abrupt step functions, and oscillating sine waves. Under step-function shifts, standard OLS exhibits substantial lag, taking extended periods to adjust, while WLS suffers from heavy dampening and sluggish reaction times. In contrast, Neural Beta tracks step changes almost immediately and adapts rapidly to oscillating regimes while handling stochastic noise.
Applied to historical market disruptions like the COVID-19 shock in early 2020, Neural Beta exhibits intelligent adaptive weighting. During the peak volatility of the pandemic, the network automatically slashed the weighting assigned to the most immediate recent observations. Because the COVID market crash was highly idiosyncratic, relying strictly on short-term trailing data would lead traditional rolling models to drastically overestimate beta, resulting in severely distorted hedge ratios once the market stabilized. By automatically down-weighting anomalous short-term spikes and extending its effective lookback horizon, the model prevented overreaction and preserved hedging efficiency. Verma concludes the presentation by noting that published research papers covering all three topics—collaborative filtering for illiquid pricing, generative financial data imputation, and neural factor/beta models—are available for deeper study.
- **Synthetic Step-Function Tracking**: Neural Beta reacts rapidly to sudden regime jumps, whereas rolling OLS lags behind and WLS dampens high-frequency dynamics. - **Adaptive COVID Shock Weighting**: During the 2020 volatility shock, the model systematically reduced weights on immediate trailing observations to prevent beta overestimation during post-shock stabilization. - **Published Research**: Full theoretical formulations and empirical results for these deep generative frameworks are detailed in Bloomberg LP quantitative research publications.
## Notable quotes > [03:51](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=231s) In the textual setting we have accepted a little bit of more margin of error and the vagueness of response... but when you're trying to solve a financial time series question—for example, estimation of risk or projection of returns or optimizing your portfolio—there is very little room for error. You really want to make sure you are able to build a model that can definitely beat the benchmarks, otherwise it's not useful at all. — Arun Verma
> Generative ML methods are becoming more popular for use cases like imputation of financial data, as well as forecasting asset prices. — Arun Verma
> Generative models are especially suited for estimating point values and uncertainty, giving us a measure of confidence in estimation. — Arun Verma
> Collaborative filtering methodologies for illiquid asset pricing, using generative deep gaussian processes to estimate behavior of a target instrument from the observations on more liquid, closely-related instruments — Arun Verma
> Machine learning techniques for financial data imputation, e.g., estimating companies' granular geographical exposure given sparse & aggregate disclosure in financial statements. — Arun Verma
> [19:20](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=1160s) Apple's return for tomorrow is going to be 2% — Arun Verma
> [00:04](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=4s) Unlike conventional Wall Street institutions that closely guard intellectual property, Bloomberg embraces a Silicon Valley research culture by actively contributing back to the open-source ecosystem.
> [07:22](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=442s) DelFin is pre-trained across diverse cross-domain numerical series, including macroeconomic indicators, equities, order book data, and non-financial data such as climatological and scientific series.
> [22:15](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=1335s) Because PCA is strictly a linear dimension-reduction technique, it fails to pick up the nuanced, conditional non-linear dependencies present in real market data.
> [29:02](https://www.youtube.com/watch?v=fpyAzMJy-Ko&t=1742s) During the peak volatility of the pandemic, the network automatically slashed the weighting assigned to the most immediate recent observations.
## Mentioned Arun Verma, Bloomberg LP, Bloomberg News, BloombergGPT, DelFin, Ai4, Geoffrey Hinton, Shireen Ghaffari, ChatGPT, William Sharpe, Robert Engle, Capital Asset Pricing Model (CAPM), Fama-French models, ARMA, GARCH, Chronos, TimesFM, MOMENT, Moirai, Neural Beta, UMAP, Apple
## Video description Arun Verma, Head of Quantitative Research Solutions, Bloomberg LP
Abstracts
Generative ML methods are becoming more popular for use cases like imputation of financial data, as well as forecasting asset prices. Generative models are especially suited for estimating point values and uncertainty, giving us a measure of confidence in estimation.
In this talk, we will cover topics like: Collaborative filtering methodologies for illiquid asset pricing, using generative deep gaussian processes to estimate behavior of a target instrument from the observations on more liquid, closely-related instruments Machine learning techniques for financial data imputation, e.g., estimating companies' granular geographical exposure given sparse & aggregate disclosure in financial statements.
[Source](https://www.youtube.com/watch?v=fpyAzMJy-Ko)