AstraZeneca Cuts Cancer AI Annotation Hours to Zero with Foundation Models
Ai4
AstraZeneca deploys pathology foundation models across its drug portfolio to eliminate manual tissue slide annotations and accelerate targeted oncology trial transitions.
## Background
This talk titled "Transformation of Oncology R&D through AI With AstraZeneca" was presented by Jorge Reis-Filho (Chief of AI for Science Innovation at AstraZeneca), published on YouTube by Ai4 on August 20, 2026. The page description highlights how multimodal foundation models and autonomous agents are accelerating oncology drug design and clinical development, emphasizing the need for fit-for-purpose data, rigorous benchmarks, and grounding in biological reality.
## TL;DR
Jorge Reis-Filho outlines AstraZeneca's strategy to transform oncology R&D by integrating multimodal foundation models, agentic frameworks, and causal AI across the drug development pipeline. Rather than pre-training models from scratch, AstraZeneca augments domain-specific foundation architectures with 15 years of proprietary clinical and translational data to solve key challenges in patient selection, trial transition, and combination design. Real-world case studies—such as Quantitative Continuous Scoring (QCS) in pathology and the ESMO Delphi consensus on AI biomarkers—demonstrate how foundation models drastically reduce annotation bottlenecks, establish prospective validation standards, and reshape clinical decision-making.
## Atmosphere & style
The presentation carries a highly authoritative, visionary, and intellectually rigorous tone, blending deep biomedical expertise with forward-looking artificial intelligence strategy. Jorge Reis-Filho speaks with measured confidence and urgency, framing current technological shifts not as incremental improvements, but as a paradigm shift redefining oncology R&D. Watching the talk feels like an insider look into the executive scientific strategy of a major pharmaceutical leader, where complex theoretical frameworks are grounded in pragmatic clinical applications and regulatory realities.
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=4s) The AI paradigm shift in oncology R&D moves from handcrafted annotations on single-use cases to multimodal foundation models leveraging unstructured raw data.
- [00:04](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=4s) Single-vector patient representations created across temporal trajectories serve as the technical equivalent of clinical digital twins.
- [03:23](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=203s) Agentic frameworks and continuous model improvements are driving rapid domain-specific intelligence, rendering generic benchmarks quickly obsolete.
- [06:49](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=409s) AstraZeneca's core strategy emphasizes data augmentation and fine-tuning external foundation models with proprietary datasets over expensive pre-training from scratch.
- [12:22](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=742s) Quantitative Continuous Scoring (QCS) uses computer vision to evaluate tumor tissue at a subcellular level, unlocking novel biomarkers like the Normalized Membrane Ratio for ADCs.
- [18:21](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=1101s) Adopting pre-trained pathology foundation models dramatically reduced required manual annotation time from 40,000 hours down to zero.
- [20:53](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=1253s) Combining multimodal foundation models with causal AI architectures is essential for counterfactual reasoning and predicting patient responses to novel drug combinations.
- [25:34](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=1534s) A successful AI transformation strategy requires a multi-layered framework separating the Model, Solution, and Application layers across an ecosystem of partners.
- [29:09](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=1749s) The ESMO Delphi Consensus classifies AI biomarkers into three tiers and establishes that multi-dataset validation takes precedence over model explainability.
- [34:42](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=2082s) Regulatory submission for multimodal foundation models focuses on domain-specific task applications rather than the broad base model itself.
- [36:38](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=2198s) Overcoming pathology adoption barriers requires addressing digital slide infrastructure, establishing clear reimbursement codes, and conducting extensive multi-center validations.
- [38:04](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=2284s) Multimodal models applied to spatial biology and multiplex imaging compress weeks of manual slide analysis into hours of automated, reproducible insights.
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=4s) Reimagining Oncology R&D and the AI Paradigm Shift
Jorge Reis-Filho opens by emphasizing that the AI transformation of clinical development at AstraZeneca extends far beyond simple system automation or operational optimization. The core thesis at AstraZeneca is that AI will fundamentally transform every stage of the pipeline—accelerating drug discovery, revolutionizing clinical development, optimizing operations, and delivering seamless, impactful patient journeys. The evolution of oncology R&D directly mirrors the rapid advancements made in computer science and data science over recent years.
Historically, biomedical R&D required highly curated, standardized, and harmonized data to build predictive tools. Today, the field can leverage unstructured, multimodal raw data. Models previously depended on manual, handcrafted human annotations to solve single indications or isolated use cases. The industry has now shifted to self-supervised learning, enabling foundation models to extract representations, logic, and patterns directly from vast quantities of unlabeled data.
```
HISTORICAL APPROACH MODERN AI PARADIGM
+-------------------------------+ +-------------------------------+
| Highly Curated & Standardized | | Unstructured Multimodal Data |
| Raw Data | | (Raw Inputs) |
+---------------+---------------+ +---------------+---------------+
| |
v v
+-------------------------------+ +-------------------------------+
| Handcrafted Annotations | | Self-Supervised Learning |
| (One Model per Use Case) | | (Foundation Models) |
+-------------------------------+ +-------------------------------+
```
- Foundation models extract patterns and reasoning directly from raw, unlabeled patient data.
- Over the last three years, the industry evolved from domain-specific, single-modality models to multimodal foundation models that construct a single vector representing a patient's entire temporal trajectory.
- These single-vector patient representations serve as the closest technical approximation to true "digital twins."
[03:23](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=203s) Agentic Frameworks and the Emergence of Domain Intelligence
In parallel with foundation models, agentic frameworks have emerged, allowing autonomous models to interact, delegate tasks, and perform complex operations. In software engineering, the standard for continuous agentic execution jumped from a few minutes to 30 consecutive hours of autonomous coding within two years. Looking forward from 2026, advances like automated alignment researchers allow weaker models to train stronger ones, alongside delegated awareness and recursive self-improvement.
This rapid acceleration underpins AstraZeneca's strategic planning. Back in late 2024, the team presented a premise to executive leadership: between October 2024 and December 2028, frontier AI models would outperform humans at reasoning and cognitive tasks. At that time, frontier models taken offline were already scoring above 140 on standard IQ tests and improving by an order of magnitude every eight months on complex benchmarks like Humanity's Last Exam.
- Domain-specific superintelligence is already a reality in narrow fields like software engineering, coding, and mathematics.
- Rapid model capability gains mean generic benchmarks quickly become obsolete, requiring weekly benchmark updates.
- The National Cancer Institute (NCI) spearheaded a specialized initiative in oncology to build task-specific benchmarks to evaluate clinical model performance.
- "If two years ago, getting a few minutes of agentic work in coding was a prowess, now 30 hours of work for coding... is table stakes."
[06:49](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=409s) AstraZeneca’s Core Strategy: Data Augmentation Over Pre-Training
AstraZeneca's overarching strategy leverages its primary competitive advantage: 15 years of rich, intentionally built clinical and translational data across a diverse drug portfolio. Rather than engaging in the capital-intensive task of pre-training base models from scratch, AstraZeneca partners with frontier AI developers to reinforce, fine-tune, and augment external domain-specific foundation models using its proprietary datasets.
AstraZeneca prioritizes four critical clinical development challenges:
1. **Patient Selection:** Applying the principle of maximum parsimony to post-train models, determining the minimal set of multimodal parameters required to identify the largest patient group that will benefit from a drug.
2. **Accelerating Phase I/II Transitions:** Utilizing the inherent ability of foundation models to generalize from small sample sizes, ensuring high-confidence transitions into Phase III trials without needing expanded 60-patient cohorts.
3. **Optimizing Phase III Success:** Deriving robust predictive insights from sparser data matrices typical of late-stage global trials to improve the probability of technical success.
4. **Designing Combination Regimens:** Moving past modest single-agent response rates toward transformational curative combinations tailored to specific patient populations.
- Pre-training success scales with data volume and diversity across modalities; leveraging external base architectures optimizes resource allocation.
- Maximum parsimony constraints ensure models select the simplest diagnostic panel needed for accurate patient stratification.
[12:22](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=742s) Case Study: Quantitative Continuous Scoring (QCS) in Pathology
AstraZeneca pioneered AI-driven patient selection prior to 2020 under the leadership of Susan Galbraith, formalizing the project around 2022–2023. While Antibody-Drug Conjugates (ADCs) represent cutting-edge targeted chemotherapy—delivering potent payloads via targeted antibodies—the industry paradoxically assessed target expression using 19th-century technology: pathologists visually eyeballing tissue slides under a microscope.
To modernize target assessment, AstraZeneca developed the **Quantitative Continuous Scoring (QCS)** system using computer vision. Initial QCS iterations relied on fully supervised computer vision models trained on handcrafted annotations to evaluate tumor tissue at a single-cell level across subcellular compartments (nucleus, cytoplasm, and cell membrane).
When applied to the TROP2-targeting ADC program in clinical trials (such as TROPION-Lung01 and TROPION-Lung02), QCS uncovered an unexpected biomarker. Instead of raw membrane expression, the predictive biomarker was the **Normalized Membrane Ratio**—the target expression on the cell membrane divided by the total expression in the membrane plus cytoplasm. Preclinical data confirmed this ratio acts as a surrogate for target internalization, reflecting the drug's true mechanism of action.
- QCS provides a dynamic measurement range far exceeding human visual perception.
- The QCS TROP2 biomarker became the first predictive AI pathology biomarker for a therapeutic agent to receive FDA Breakthrough Device Designation (April 2024).
- Developing the initial supervised QCS model for a single indication required an astounding 40,000 pathologist annotation hours.
- "Can you believe that we are developing antibody-drug conjugates... but then we are using 19th-century technology to do target assessment?"
[18:21](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=1101s) Foundation Models Break the Annotation Bottleneck
Faced with executive requests to scale QCS across the entire oncology pipeline, relying on 40,000 manual annotation hours per indication was unfeasible. The team established a benchmark sandbox to test pre-trained foundation models for zero-shot and single-shot target quantification and tumor segmentation.
By leveraging pre-trained pathology foundation models, AstraZeneca slashed required annotation time from 40,000 hours to 4,500 hours, and ultimately to zero hours. QCS models were deployed across the entire portfolio for all targets involving cell-surface proteins, including multi-cancer cross-indication QCS tools used across R&D sites.
- Transitioning to pre-trained foundation models enabled rapid deployment of QCS across all cell-surface protein targets in the portfolio.
- AstraZeneca built end-to-end evaluation frameworks, benchmarks, and real-time leaderboards to monitor agent and model drift.
- Insights from evaluating foundation model downstream performance led directly to AstraZeneca's partnership with and acquisition of Modella AI.
[20:53](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=1253s) Causal AI, Counterfactuals, and the Future of Clinical Interception
Multimodal models are rapidly advancing outcome predictions. At the AACR conference, researchers presented **Apollo**, a multimodal foundation model trained on 7.2 million patients across 28 data modalities using a BERT backbone, utilizing time as a primary dimension to build unified patient vectors.
While outcome prediction within learned training distributions is now achievable, the next frontier requires early clinical interception and counterfactual reasoning. To answer "what-if" questions—such as predicting how a specific patient will respond to an unstudied novel drug combination—foundation models must integrate with causal AI architectures.
- Outcome prediction models excel at historical distributions, but novel therapeutic interventions require true causal reasoning.
- Current causal models lack the massive data scale necessary to combine seamlessly with foundation models, representing a major open challenge in AI-driven oncology R&D.
Here is a detailed, faithful walkthrough based on the transcript provided:
### [22:53](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=1373s) Building an Ecosystem of Partnerships
To create actionable causal models in drug discovery, organizations must establish continuous feedback loops between predictive modeling and testing spaces, whether in the wet lab or the clinic. Without these active learning loops, AI initiatives remain stagnant. Understanding every intricate phase—from target identification and drug design to clinical development, operations, commercialization, and patient journeys—exceeds the capacity of any single institution or pharmaceutical firm.
To address this challenge, AstraZeneca has constructed an interconnected partner ecosystem structured around five core pillars:
* **Hyperscalers:** Providing foundational computing infrastructure.
* **Frontier AI Labs:** Supplying cutting-edge base models and algorithm research.
* **Specialized AI Companies:** Delivering domain-specific technical capabilities.
* **Academic Partnerships:** Exploring frontier science to anticipate future developments.
* **Patient Advocates:** Integrating real-world patient perspectives and priorities directly into AI strategy.
```
+-------------------------------------------------+
| AstraZeneca Innovation Ecosystem |
+-------------------------------------------------+
| | | | |
v v v v v
Hyperscalers Frontier Specialized Academic Patient
Labs Techs Partners Advocates
```
### [25:34](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=1534s) A Multi-Layered AI Strategy and Stakeholder Alignment
To move beyond presentation slide decks and deliver operational clinical solutions, AI strategy must operate across a three-layered framework:
```
+-----------------------------------------------------------------+
| Model Layer | Large-scale pre-training via partnerships |
+-----------------------------------------------------------------+
|
v
+-----------------------------------------------------------------+
| Solution Layer | In-house domain specialization & task adapt |
+-----------------------------------------------------------------+
|
v
+-----------------------------------------------------------------+
| Application Layer | External delivery with diagnostic/tech bios |
+-----------------------------------------------------------------+
```
- **Model Layer:** Focuses on pre-training large models. Because the required volume and diversity of training data exceed what a single company holds, this layer relies heavily on external partnerships, alongside robust internal evaluation and benchmarking suites.
- **Solution Layer:** Adapts and connects these base models to specific domain areas to answer defined scientific questions, developed internally at pace.
- **Application Layer:** Brings AI applications directly to external clinical users, diagnostic companies, tech bio partners, and patients.
Realizing this framework requires bringing all key industry stakeholders together. Academic partners help recognize and solve biomedical benchmarking challenges, alongside initiatives like the National Cancer Institute's AI benchmark efforts. Tech bio, biotech, and pharmaceutical companies supply industry execution. Payers address structural hurdles such as diagnostic billing codes, while regulators collaborate on forward-looking evaluation standards.
### [29:09](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=1749s) The ESMO Delphi Consensus on AI Biomarkers
Translating AI concepts into clinical reality is already underway. Led by academic partners like Jakob Kather from Heidelberg, the European Society for Medical Oncology (ESMO) convened a 37-panelist Delphi consensus to establish practical requirements for deploying AI in clinical decision-making.
```
+-------------------------------------------------------------+
| ESMO AI Biomarker Framework |
+-------------------------------------------------------------+
| | |
v v v
[Class A] [Class B] [Class C]
Pathological Inference Prognostic /
Quantification Biomarkers Predictive
| | |
Technical Genetic Validation
Alignment Comparison Required
```
- **Class A (Pathological Quantification):** Automates standard routine pathology tasks, such as quantifying PD-L1 levels or identifying cancer types. Technical validation requires demonstrating concordance with pathologist consensus.
- **Class B (Inference Biomarkers):** Extracts hidden features directly from standard H&E stains to predict specific genetic mutations. Validation requires comparing AI predictions directly against gold-standard molecular or genetic testing.
- **Class C (Prognostic and Predictive Biomarkers):** Forecasts patient disease trajectories or specific treatment responses. Prognostic features can be validated retrospectively using prospective study datasets, whereas predictive claims require prospective clinical trial validation.
A critical conclusion of the ESMO panel is that validation across multiple, sufficiently powered datasets is mandatory, whereas model "explainability" is no longer considered a strict prerequisite. Prioritizing explainability can encourage human "abductive thinking" and post-hoc narratives. Ensuring consistent performance across diverse real-world datasets provides a more reliable foundation for patient care.
### [32:49](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=1969s) Acceleration of AI Timelines and Closing Ambition
The development pace of biomedical AI models has accelerated rapidly:
```
Decades Months to Weeks Immediate
+------------+ +-----------------+ +------------+
| Big Data | ---------> | Foundation | ----> | Recursive |
| Insights | | & Multimodal | | Self- |
+------------+ +-----------------+ | Improvement|
+------------+
```
While early big data insights required decades to mature, the transition from single-modality foundation models to domain-specific multimodal architectures unfolded over months or weeks. Upcoming recursive self-improvement paradigms will compress these development cycles further. Succeeding in this evolving environment requires cross-disciplinary teams, open collaboration, and comfort with rapid change. The ultimate goal remains leveraging AI to generate novel biological insights that help eliminate cancer as a cause of death.
### [34:42](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=2082s) Q&A: Regulatory Frameworks for Multimodal Foundation Models
In response to an audience question regarding regulatory approval for general multimodal foundation models within disease-specific frameworks, the speaker clarified that base models are pre-trained for broad capability rather than zero-shot deployment.
```
+------------------+ +-------------------+ +------------------+
| Pre-trained Base | --> | Task Application | --> | FDA Submission |
| Foundation Model | | (Domain-Specific) | | & Review |
+------------------+ +-------------------+ +------------------+
```
Developers build task-specific applications on top of foundation models and submit those specific applications to regulators like the FDA. Much like replacing older supervised computer vision backbones (such as ResNet-50) with foundation models, applicants must document pre-training data provenance, pre-training methodologies, and validation metrics. Demonstrating regulatory readiness, PathChat received Breakthrough Device Designation from the FDA in January 2025.
### [36:38](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=2198s) Q&A: Overcoming Adoption Barriers in Pathology
When asked how to address caution among pathologists regarding AI adoption, the speaker described the journey as a "formidable" task requiring extensive education and change management. Pathologists must transition from traditional light microscopy to trusting digital screens and machine embeddings. Key operational hurdles being actively addressed through early access programs and key opinion leader engagement include:
* Lack of digital slide scanners and underlying digital infrastructure.
* Absence of standardized billing and reimbursement codes.
* Need for extensive multi-center performance validations to ensure clinical safety.
### [38:04](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=2284s) Q&A: Future Data Sources and Spatial Biology
Addressing a question on future data modalities beyond standard 2D H&E slides—such as spatial genomics, tissue clearing, or 3D imaging—the speaker noted rapid progress in spatial biology. Recent multimodal foundation models handling spatial proteomics and multiplex immunotyping (such as work from Faisal Mahmood's lab and Sizun Jiang at Harvard) can compress three to four weeks of manual image analysis into a couple of hours. These models provide automated, reproducible cell segmentation, tissue division, and semantic feature extraction across tissue sections.
## Notable quotes
> [03:23](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=203s) If two years ago, getting a few minutes of agentic work in coding was a prowess, now 30 hours of work for coding... is table stakes. — Jorge Reis-Filho
> [12:22](https://www.youtube.com/watch?v=G1e3Y-P9e10&t=742s) Can you believe that we are developing antibody-drug conjugates... but then we are using 19th-century technology to do target assessment? — Jorge Reis-Filho
## Mentioned
AACR, Ai4, Apollo, AstraZeneca, ESMO, FDA, Faisal Mahmood, Heidelberg, Jakob Kather, Jorge Reis-Filho, Modella AI, National Cancer Institute (NCI), PathChat, QCS (Quantitative Continuous Scoring), ResNet-50, Sizun Jiang, Susan Galbraith, TROP2, TROPION-Lung01, TROPION-Lung02
## Video description
Jorge Reis-Filho - Chief of AI for Science Innovation, AstraZeneca
The AI transformation of drug design and clinical development is already under way.
Multimodal foundation models and increasingly capable agents are increasing R&D capabilities at a scale that were inconceivable not a few years ago. Realizing their potential will require fit-for-purpose data, rigorous evals and benchmarks and deep grounding in biology and clinical reality.
Getting this right will help us to not only move faster, but also generate novel insights to enable us to ask better questions, design more discriminating experiments, and deliver more meaningful advances for patients.
阿斯利康利用基础模型将癌症AI标注工时降至为零
阿斯利康在其药物管线中部署病理学基础模型，以消除人工组织切片标注并加速靶向肿瘤试验推进。
Jorge Reis-Filho概述了阿斯利康通过在整个药物研发流程中整合多模态基础模型、智能体框架和因果AI来变革肿瘤研发的战略。阿斯利康并非从零开始预训练模型，而是利用15年积累的专有临床与转化医学数据增强特定领域的基础架构，以解决患者筛选、试验过渡及联合用药方案设计等关键挑战。真实世界案例研究——例如病理学中的定量连续评分（QCS）以及关于AI生物标志物的ESMO德尔菲（Delphi）共识——展现了基础模型如何大幅消除标注瓶颈、确立前瞻性验证标准并重塑临床决策。
肿瘤研发的 AI 范式正从针对单一场景的人工特征标注，转向利用非结构化原始数据的多模态基础模型，构建贯穿患者全时序轨迹的单向量表征以逼近真实“数字孪生”。 (0:04)
阿斯利康采取数据增强而非从头预训练的基础战略，利用其积淀15年的专有临床数据微调外部前沿模型，专注于精准患者筛选、加速 I/II 期向 III 期的推进及联合疗法设计。 (6:49)
定量连续评分（QCS）利用计算机视觉在亚细胞层级评估组织，为抗体偶联药物（ADC）发掘出关键生物标志物“标准化细胞膜比率”并获 FDA 突破性认定；病理基础模型的应用更使人工标注耗时从4万小时降至零。 (12:22)
突破既有训练分布以预测患者对全新联合疗法的反应，要求将多模态基础模型与因果 AI 架构结合，通过反事实推理模拟不同治疗假设。 (20:53)
AI 战略落地依托模型层、解决方案层与应用层的多层生态体系；ESMO 共识亦明确指出，跨多中心充足数据集的泛化验证是临床落地的核心前提，而模型可解释性已不再是硬性要求。 (29:09)
Reading summary
AstraZeneca argues that oncology R&D is shifting from handcrafted, single-task models to multimodal foundation models and causal frameworks, enabling biological target discovery and compressed clinical development cycles.
From Handcrafted Annotations to Multimodal Foundation Models (0:04)
Historically, biomedical machine learning relied on heavily curated, standardized datasets and human-annotated labels tailored to individual use cases. According to AstraZeneca's Jorge Reis-Filho, the sector has transitioned to self-supervised learning on raw, unstructured multimodal data. Pre-trained foundation models can now capture patterns directly from unlabeled inputs and represent a patient's temporal trajectory within a unified vector, approximating a digital twin.
Fine-Tuning Strategy and the QCS Biomarker Case (6:49)
Rather than pre-training base models from scratch, AstraZeneca's strategy focuses on fine-tuning external foundation models using its 15 years of proprietary clinical trial and translational data. The company applies this approach to four priorities: patient selection via parsimonious diagnostic panels, accelerating Phase I/II transitions without expanding cohorts to 60 patients, optimizing late-stage trial success, and designing combination therapies.
Eliminating Annotation Bottlenecks in Computational Pathology (12:22)
To modernize visual target evaluation for antibody-drug conjugates, AstraZeneca developed Quantitative Continuous Scoring (QCS) to measure target expression across subcellular compartments. In TROP2 trials, QCS identified the Normalized Membrane Ratio as an internalization surrogate, earning an FDA Breakthrough Device Designation in April 2024. While the initial supervised model required 40,000 pathologist annotation hours, adopting foundation models cut required annotations to 4,500 hours and ultimately zero, allowing cross-indication deployment and prompting the acquisition of Modella AI.
Causal Modeling and Multimodal Clinical Trajectories (20:53)
Multimodal models such as Apollo, trained across 7.2 million patients and 28 data modalities, can predict patient outcomes within historical data distributions. However, Reis-Filho emphasizes that evaluating novel drug combinations and counterfactual scenarios requires pairing foundation models with causal inference architectures. A major ongoing technical barrier is that existing causal inference models currently lack the large-scale data required to integrate seamlessly with foundation models.
Delphi Standards and Clinical Translation Hurdles (29:09)
Translating models into clinical adoption requires cross-sector alignment among hyperscalers, frontier labs, regulators, and diagnostic partners. A 37-panelist ESMO Delphi consensus led by academic researchers classified AI biomarkers into pathological quantification (Class A), genetic inference (Class B), and prognostic/predictive indicators (Class C). The panel determined that validation across sufficiently powered independent datasets is mandatory, whereas model explainability is not an absolute prerequisite. Routine adoption still faces operational hurdles, including slide scanner availability, reimbursement codes, and digital workflow changes.
AstraZeneca's long-term objective is deploying autonomous, recursively improving models to uncover novel cancer biology. However, realizing this potential requires resolving data-scaling constraints in causal AI, establishing standard diagnostic reimbursement codes, and conducting multi-center clinical validations.
[3:23] Jorge Reis-Filho: If two years ago, getting a few minutes of agentic work in coding was a prowess, now 30 hours of work for coding... is table stakes.
[12:22] Jorge Reis-Filho: Can you believe that we are developing antibody-drug conjugates... but then we are using 19th-century technology to do target assessment?
Original transcript
[0:04] It's such a distinct privilege to be here with you
today and to discuss with you the transformation
[0:09] of clinical development through AI. I don't think
that we should be focusing only on automation and
[0:15] optimization of systems. As a matter of fact,
there is so much that we can gain by reimagining
[0:21] how we deliver our drugs to our patients. But to
begin with, at Astrazenica, our core thesis is
[0:31] that AI is going to transform everything that we
do from accelerating drug discovery, transforming
[0:37] clinical development, optimizing clinical
operations, and actually delivering journeys
[0:43] for patients that are more meaningful, impactful,
and more seamless. Our uh core thesis here is
[0:52] then grounded on the in an observation actually
that we've made that if we stop for a second and
[0:59] think about oncology the critical developments in
oncology actually have mirrored developments in
[1:07] data science and computer science. And if we think
about the way we make discoveries in oncology and
[1:14] deliver the drugs, then we we we have to consider
that actually in the last few years we have gone
[1:21] from needing extremely curated standardized and
harmonized data to being able to use unstructured
[1:31] raw data that's multimodal in nature. That's has
that has happened already. Second, we have gone
[1:39] from the perspective of the development of AI from
the needs of the need of handcrafted annotations
[1:46] to develop one model for one indication, one use
case to actually doing self-supervised learning,
[1:54] extracting the representations and creating
highly capable foundation models. And what
[1:59] do I mean by a foundation model? It's
that very principle of having unlabeled
[2:07] data and allowing the model to extract the
features, the representations, the patterns,
[2:13] the logics and eventually create reasoning from
that process. or through that process and the very
[2:20] development of self-supervised self-supervised
learning allowed us to move to a new space as
[2:26] well that is that data no longer needs to be
labeled vast amounts of unlabeled data can
[2:34] be extremely useful in the context of developing
foundation models and in the space of foundation
[2:40] models what has happened over the last three years
is quite incredible. We went from the development
[2:46] of the first generation of foundation models that
were domain specific and single data modality to
[2:52] highly capable domain specific foundation models
in pathology to then multimodal foundation models
[2:58] that create single patient representations with
time as a first dimension. And we've seen three
[3:05] this year already being shipped that actually can
create a single vector that represents a patient
[3:12] in its in his or her trajectory and this is the
closest approximation to a term that I don't
[3:19] like very much that's digital twins that part of
the previous discussion but that's exactly what
[3:23] it's beginning to create and in parallel with this
we have seen the birth of the agentic frameworks
[3:30] where we get these models to interact with each
other in the ecosystem and perform increasingly
[3:37] more complex tasks. And the tasks that we can
perform now are incredible in coding. If two
[3:45] years ago, getting a few minutes of agentic work
in coding was a prowess, now [snorts] 30 hours
[3:52] of work for coding under in within a highly
competent agentic framework is table stakes.
[4:00] We are moving now to an age in 26 and forward with
the development of automated alignment researchers
[4:08] that allow weaker models to train stronger models
to get a performance that's even higher. In the
[4:15] agentic space, the idea of delegated awareness
where basically you drop the agent to figure out
[4:21] the data that's available, the tools available
and starts actually integrating and asking the
[4:27] right questions. has already come to become
a reality and we're moving into the space of
[4:33] space of recursive self-improvement and this may
sounds all extremely bullish. Yeah. But again,
[4:41] if we stop for a second and just accept that
domain specific super intelligence has already
[4:49] come to become a reality in some domains, very
narrow domains like software engineering and
[4:56] coding itself. often we're beginning to see this
happening in mathematics. So with that in mind,
[5:02] back in 2024, this is a long time ago. Yeah, it
sounds like an eternity in AI years. Yeah, 2024
[5:10] we came to our leadership and that was our premise
on the basis of what was happening at that time
[5:15] that between October 2024 and December 2028 the
frontier models would outperform us first at any
[5:25] and eventually at every reason and cognition task.
And at that time the reason for us to believe that
[5:32] was that the first models at that time when they
were taken offline they would perform in terms of
[5:39] offline IQ tests incredibly well and would sit
comfortably above 140. It's it's a weak test.
[5:48] I granted it's not a great way of of of showing
this but it's one piece of evidence. The second
[5:54] is when we look at extremely sophisticated bench
benchmarks such as the humanities last exam.
[6:01] Again, the the models were increasing one order of
magnit magnitude every 8 months and they continue
[6:10] to break the destroy the benchmarks constantly to
the point that we have to be generating benchmarks
[6:17] weekly now [laughter] and there's an an initiative
in oncology that the NCI has spearheaded to start
[6:25] creating benchmarks that are really fit for
purpose because we need to be able to assertain
[6:30] the performance of these these models really
well, not in a generic based way, but actually
[6:35] in a task specific way that is representative
of what we need to execute and and then finally
[6:41] as we discussed the agentic workflows are and
frameworks are becoming incredibly capable. How
[6:49] do we translate that into then a strategy within
Astrazenica? So, Astrazenica has has been known as
[6:58] a company that has many brought many breakthroughs
in in the form of novel classes of agents. So,
[7:07] with the development in terms of frontier AI
models and domain specific foundation models,
[7:14] our strategy was to leverage then our key
differentiator, our data. We have developed
[7:22] over the last 15 years a uniquely diverse and
intentionally built portfolio that you can see
[7:28] here on the left side of the slide with this color
wheel. On the right hand side of the color wheel,
[7:34] um there are five classes of agents uh that
target the cancer cells directly. Antibbody
[7:40] drug conjugates, radio conjugates, um agents
that target DNA damage response. Then we have
[7:49] the agents that target drivers like ES like ER
signaling or EGFR mutations. And we also have
[7:57] drugs that target the genetics. The way
the states that the cells get locked in,
[8:02] they are all complimentary. And on the left side
of this color wheel, you can see the classes
[8:09] of agents that we use to leverage the immune
system to target the cancer cells. traditional
[8:14] imunoconcology approaches in the form of bi
specifics. We have engages to attracts the tea
[8:21] cells and cell therapy that actually we are going
into cell therapy now even for solid malignancies.
[8:29] So the idea is that we have all these these
agents we have been generating translational
[8:35] and clinical data over the last 15 years. So what
if we use this data not to pre-train models? We're
[8:43] not in the business of pre-training models, but
actually we use this data to reinforce, fine-tune,
[8:50] augment frontier models and domain specific
foundation models to address questions that
[8:57] are really critical to our clinical development.
And the way the reason for us not to pre-train
[9:05] models was a really simple one. The performance
of these models depends upon the diversity of the
[9:13] data and the volume of data. So if you have more
data across modalities that's really diverse. The
[9:20] more diversity we have in terms of modalities and
the more diversity we have within a modality, the
[9:26] better the performance of the model if we keep the
same architecture and training strategy. So we've
[9:31] created partnerships with uh multiple companies
that are actually developing these frontier models
[9:39] and we use our data to reinforce, fine-tune
and augment them. There are four flavors of
[9:44] questions that we've have prioritized uh in terms
of creating those solutions. First the challenge
[9:54] that we face in oncology in particular that
is patient selection. So the the way we phrase
[10:01] framed it for ourselves was what would be the most
parsimonious set of multimodal parameters that
[10:12] would be required and sufficient to identify the
largest patient population who would benefit from
[10:17] a drug or combination. And actually it may sound
complex but applying the principle of maximum
[10:23] parsimony to post- train a model is something that
has already been demonstrated in other contexts.
[10:30] So we have borrowed from those examples to apply
to patient selection strategy and actually we
[10:37] have already started seeing this in some of our
clinical development programs identification of
[10:42] those populations. Second, to accelerate phase
one, phase two studies in particular to render
[10:50] the transition from phase one, phase two to phase
three studies in a much more informed way using
[10:55] one of the properties of these foundation models
that is to generalize from smaller sample sizes.
[11:01] That's a one of the inherent properties of these
highly capable multimodal foundation models. Um
[11:10] the last thing that I want to happen going forward
is to end uh an expanded phase one trial with 60
[11:16] patients and not having certainty in terms of what
the transition should be in terms of the patient
[11:23] population in terms of probability of success.
The third application to revisit our phase three
[11:31] studies and launch new phase three studies with
a much better probability of technical success.
[11:38] And again here borrowing another property
of the foundation models that is to gen to
[11:43] derive insights from sparser data in phase three
studies. We don't have the density of data that
[11:49] we have in phase one, phase two and but all of
these three are to serve the ultimate goal here
[11:56] that is to develop the combination regimens that
actually will be transformational for patients.
[12:02] Uh we have to move away from the type of objective
response rates and clinical benefit rates that
[12:10] we get with our single agents today to a a place
where we can actually transform patients journeys
[12:17] on the basis of the patient selection strategy
that we take the way we deliver the combination
[12:22] of treatments. This may all sound sci-fi. We may
sound really futuristic, but the reality is that
[12:32] artificial intelligence for patient selection for
us as Astroenica is not something that we're not
[12:37] familiar with. To be perfectly honest, thanks
to the visionary leadership of Susan Galbright,
[12:43] we went into um AI for patient selection a number
of years ago, pre pre202. So it actually started
[12:53] back in 2017 but it crystallized really around
22 23 just bear here with me for one second I
[13:02] think all of us here or most of us here around
70% are into um biomedical research or here
[13:08] or biomedical sciences okay u we did a show of
hands earlier today so antibbody drug conjugates
[13:17] um where we have an antibbody a link and a payload
are transformational treatments. Antibbody drug
[13:25] conjugates are targeted chemotherapy. Yeah.
So we have the antibbody and the payload. So
[13:29] it should be targeted chemotherapy. If it is
targeted chemotherapy target assessment is a
[13:35] critical question. Would you agree?
So if you start with that premise,
[13:39] can you believe that we are developing antibbody
drug conjugates frontier technology in terms of
[13:44] therapy development but then we are using 19th
century technology to do target assessment. It's
[13:52] a microscope with someone like me a pathologist
eyeballing the slides. What if we reimagine how
[13:59] we do target assessment using AI? And one of the
major developments for AI was the application of
[14:06] AI to computer vision of pathology images.
Yes. So we went in with high uh convection
[14:15] and we developed what we call our quantitative
continuous scoring system. Uh we at initially
[14:24] um we utilized a fully supervised methods. So
handcrafted annotations to extract seven human
[14:32] interpretable features in relation to the extract
extract expression of a target. So what does it
[14:38] do? It identifies there are two steps. One, it
identifies the tumor areas by by itself and then
[14:47] it quantifies the expression of the target
within subcellular components, the nucleus,
[14:53] the cytoplasm and the membrane and it does it at
single cell level. It can assess the hetrogenity
[14:59] of the target and it does with a dynamic range
that far exceeds what the human eye allows for.
[15:09] It's computer vision. Yeah. Again I think we
all would all agree here. So the way we develop
[15:15] it is first we develop a model okay trained with
specific data sets. This is for us to be able to
[15:25] quantify the target really well sub at subcellular
level. And then we go to the second part. Once
[15:31] the model is locked we go into the biomarker
development. Right? The biomarket development then
[15:38] requires very specific uh prospective studies that
we analyzed retrospectively or even a prospective
[15:47] trial testing than the biomarker itself. We
applied QCS first to our trop targeting ADC
[15:56] and we did this with the first study that was
J101 and then with the tropion lung 01 trial
[16:03] and actually we observed two things that we could
identify a biioarker for our trop target in ADC
[16:14] but what was really interesting and I think most
surprising is that actually it was not the obvious
[16:21] uh assessment rather than being the expression of
the target on this on the membrane itself. What
[16:29] we found to be the biomarker was the normalized
membrane ratio which is the expression of the
[16:36] target on the cell membrane divided by the
expression of the target on the membrane plus
[16:40] cytoplasm. in our preclinical studies indicate
very strongly that this is a surrogate marker for
[16:48] one of the properties that actually is related to
the mechanism of action of the the agent that is
[16:55] the internalization of the target itself. So the
pull of the target that's internalized we tested
[17:01] this also in the tropial lung O2 and [snorts] the
results held and actually it has come to become
[17:09] the first AI applied to pathology biomarker that's
predictive of response to a therapeutic agent
[17:19] uh our trop targeting ADC we received FDA
breakthrough device designation in April last
[17:25] year for this But I'm afraid to say that we had a
challenge then we we presented this in September
[17:34] 2024 got FDA breakthrough deviceation in April
2025 wonderful for us to develop that particular
[17:42] biioarker for just for the model part the first
part yeah we needed handcrafted annotations.
[17:50] Can can you guess how many hours of pathologist
annotating those slides we needed for one model
[17:59] one indication one tumor type take a guess perhaps
more 40,000 40,000 hours so when leadership came
[18:09] to us and said oh now we would like to have
this across our portfolio I said chat is this
[18:14] real [laughter] because it was very very very
hard So with the development in foundation models
[18:22] then we saw an asymmetric opportunity because
those models are that are pre-trained um they
[18:30] can perform multiple downstream tasks. So what we
did was we set up a set of benchmarks in a sandbox
[18:38] and evaluated all the potential foundation models
back then that were available in terms of doing
[18:44] the tumor segmentation and the quantification of
the targets single shot. And um there was a clear
[18:53] set of foundation models that performed really
well. And what happened next was we went from
[18:59] 40,000 hours to 4 and a half thousand to zero. And
basically we did we have now QCS models across our
[19:09] portfolio of therapeutic agents all the all the
agents that have cells that target cells surface
[19:17] proteins. And we've even developed now um multic
cancer cross indication QCS that we're using for
[19:24] in our R&D site. Our idea now here is actually
to borrow from this concept and go from pathology
[19:31] and the quantification of the target excuse me to
multimodel uh to address those four [snorts] types
[19:39] of questions that I mentioned in the previous
slide and we are doing this also creating the
[19:44] agentic framework to connect our data which is our
key differentiator with the models to address the
[19:51] questions and to create these agents it has been
a labor of love because it's not only having the
[20:00] results and I have to say AI is fantastic to
create AI hypothesis lop. It's fantastic. It
[20:07] sounds really sophisticated. It sounds really cool
but once you start testing it breaks. So we had to
[20:14] create a whole set of evaluation evaluations and
benchmarks in an end toend way task specific and
[20:22] subtask specific. and we have leaderboards and we
we keep constant monitoring of the performance of
[20:31] the models and agents. Um and in that process it
also informed us in terms of how we could assess
[20:39] the most capable foundation models for specific
downstream tasks and that was one of the reasons
[20:44] for the partnership with model AI that happened
earlier this year and the subsequent acquisition.
[20:53] Now
[20:56] this idea of using multimodal foundation models
for prediction is coming to become a reality. And
[21:02] if we wind our clocks back in eternity again
ACR this year American Association for Cancer
[21:09] Research meeting there was a presentation
of one um new multimodal foundation model
[21:15] Apollo trained with 7.2 2 million patients, 28
data modalities using a a birth backbone. So,
[21:25] and time as a first dimension to create single
patient representations. And I have to say
[21:31] that the ability to predict outcomes of the
distributions that we have in the pre-training
[21:37] data is is quite incredible. There are many
other um developments in academia and in the
[21:44] tech bio space where they have developing
these models that are becoming incredibly
[21:50] capable. The challenge that we face now is one
step further is if you can predict outcomes.
[21:58] The next set of questions is can we intercept
earlier so that we can so that we can intervene
[22:06] and once we get there we need to start asking
ourselves the questions about the counterfactuals
[22:12] the whatifs and why the are the counterfactuals
important because we want to use these engines
[22:20] to predict what will be the response of a patient
to a new treatment. It's not only to say that the
[22:25] patient will have with the constraint probability
this as an outcome and to do that we need to move
[22:32] into the space of causal models together with the
foundation models. This is a problem that has not
[22:39] been solved yet. It's incredibly hard and most of
the causal models at present do not have the scale
[22:46] of data to be able to get to that point. I would
submit another point here that's very important
[22:53] when we consider this that is that when we think
about this causal models we need to create a loop
[23:02] of the model in the testing space be it wet lab
or clinical that allows for the continual learning
[23:11] because if you don't enable that you stuck in
the same at the same point that we are today.
[23:19] This will require and I would submit here that
this will require much more than what we have
[23:26] in a single institution company at present. I I
I am afraid to say that I I I cannot see anyone
[23:37] at present that can comprehend all the intricate
complexities of every step required from target
[23:47] identification, drug design, clinical development,
clinical operations, the commercial side and the
[23:54] patient journeys in a single entity. So we talk
about end to end but we have to recognize that
[24:01] there are int intricate complexities at each
one of the steps and here at Astroenica what
[24:08] we decide to do was to create an ecosystem of
partnerships that would help us in our journey.
[24:14] So we have partnerships with hyperscalers with
frontier AI labs with specialized AI companies.
[24:22] The fourth pillar is really important, the
academic partnerships. And I have to say that
[24:28] at Astroenica probably have had more meetings with
academics than I used to have when I was the chief
[24:34] of experimental pathology at MSK. Yeah. Because
I need to see around the corner from around the
[24:40] corner. So for us to understand what how we
can adjust our path going forward to develop
[24:46] to derive new insights we need to be aware of
what is the bleeding edge of the bleeding edge
[24:51] and it's been incredibly fruitful extremely
intellectually rewarding and impactful for
[24:57] us and finally and I have to thank Nathan who is
here in the audience because he brought the fifth
[25:04] pillar we realized that we needed to bring the
patient advocates into this ecosystem them. I was
[25:12] really humbled by the quality and sophistication
of the solutions that they were making available
[25:17] to patients and I also was really fascinated by
the types of questions that they wanted answer
[25:25] through AI. So that type of um insight that
they bring to the discussion is really unique.
[25:34] The other aspect that I we've learned in the
hardest possible way was that we needed to
[25:40] approach AI strategy from a multi-layered
system. So from our perspective within AI
[25:47] for science innovation what we we focus
on on three layers. first the model
[25:53] layer um where actually as I mentioned to you
the volumes of data and the diversity of data
[26:00] required for the pre-training of those models far
exceed what we have in within a single company.
[26:08] So we are doing this largely through partnerships
creating the evals and benchmarks required for us
[26:13] to internalize those solutions at base. We build
some models but we internalize many of them.
[26:21] The second layer is the solution layer where
we basically connect those models to specific
[26:26] domain areas to address specific questions.
Uh and this we basically develop inhouse at
[26:34] pace. Then comes the the third layer that's the
application where we are going to bring this to
[26:40] life to patients or to um specific teams that have
a surface area outside our company. And he here
[26:49] we we also do it through partnerships through
diagnostic companies and through tech bios.
[26:58] The other lesson that we learned from this
collaborative um ecosystem is that for us
[27:06] to be able to bring this to real life not to be
a really nice slide deck that we present to our
[27:13] board was that we needed to have all the different
stakeholders around the table. We as I mentioned
[27:21] before the academics the academics provide some
of the models and frontier but also they have
[27:26] they had recognized the benchmarking crisis in
the in biomedical research way earlier than we
[27:32] did and actually we started building benchmarks
with them they cut across industry so it's a
[27:39] really great opportunity and seeing the national
cancer institute now spearheading this initiative
[27:45] of benchmarks for oncology AI and oncology simply
fantastic uh we need all the industry partners
[27:53] from tech tech bio biotech and pharma. uh we need
to have the payers because one of the challenges
[28:00] in the uh adoption of computational pathology is
that for instance the uh the building codes are
[28:08] still very challenging and and fourth of course
the regulators and I have to say that the thought
[28:14] partnership with the regulators over the last
18 months has been simply incredible uh they
[28:21] have really forwardlooking they are asking us for
information we exchange insights it's it's been
[28:26] incredible to be fruitful. And when we do that, we
have this ecosystem in place and we have the evals
[28:33] and benchmarks that have really fit for their
purpose. Then we can move the patients to the
[28:38] very center of this innovationdriven ecosystem and
start delivering the solutions that will be most
[28:44] impactful for them because ultimately we change
patients patient the outcomes of patients. Then
[28:50] we have the opportunity of actually um bringing
the solutions that AI has brought much promise
[28:59] and hype to into real life into something that's
really impactful. And again you may say that this
[29:09] is theoretical George it's a conceptual framework
actually it's already happening and thanks to
[29:14] the leadership of one of our closest academic
partners Jakob Kata [snorts] from H Highleberg
[29:21] um asmo the European Society of Medical Oncology
went back to leading academics and uh oncologists
[29:31] in other settings and created created two deli
processes to evaluate what would be required and
[29:40] sufficient for them to deploy AI solutions for
patient selection for treatment decision making
[29:46] in the clinic. Let's stop with the discussion
of what's theoretical what actually is needed
[29:52] and they had 37 uh panelists. They carried out
all those deli deli processes and they came up
[30:02] with a conceptual framework that's actually
really pragmatic and applicable. So there are
[30:08] three types of AI biomarkers. The class A is very
simple is to do what we pathologists do. Quantify
[30:17] PDL1 or two or render a diagnosis of a a given
cancer type. For this basically the validation
[30:27] is really technical. It's just matching the
accuracy of what pathologists do in AI and
[30:32] then adjucate the differences and there you go.
The second is the inference type of biomarker
[30:41] with just extracting the features from H& you can
predict some types of mutations in cancer. So this
[30:49] requires a different type of validation. You need
to compare what you derive with AI from pathology
[30:56] with the gold standard the genetic test educate
the differences and come with your validation.
[31:04] Now the third the class C is where I find really
interesting because we can now use pathology or
[31:11] multimodal approaches to predict patient prognosis
or even treatment response prognostication and
[31:19] prediction. And here the the validation is
really differentiated for prognostication.
[31:25] You can do retrospective analysis of prospective
studies that suffices for predicted biomarkers. So
[31:31] whether drug A is effective in a given population,
it requires a prospective study still. But I think
[31:38] the biggest take-h home message which I found I
found really gratifying I have to say because it
[31:44] was a discussion point from back in 2022 2023.
The conclusion of this panel was that validation
[31:52] with sufficiently powered data sets that were
acred in the right manner is absolutely essential
[31:59] and multiple times not a single validation. But
explanability that many had touted as a condition
[32:07] that was cynical known for biomarkers is no longer
essential. It's no longer mandatory. It's perhaps
[32:15] desirable. And I find this really important
for one reason. If you focus on explanability,
[32:22] our brains are fantastic for human slop. We can
retrofit a a really great narrative and explain a
[32:29] biomarker actually with really sufficient data
using abductive thinking that's fundamentally
[32:34] flawed. Moving from that to one where it needs to
perform consistently across data sets, I find it
[32:41] as as a physician and as a patient as well, I find
this much more palatable and much more impactful.
[32:49] So we go back to the our very beginning the
very beginning of this discussion today.
[32:55] AI is advancing at an unprecedented pace. I think
we could foresee some of the developments, but
[33:02] I think the privilege of hindsight now grants us
something really phenomenal because this the pace
[33:10] of development is incredible and the timelines
are being compressed every time. If it took us
[33:18] multiple decades to go from that idea of big data
to generating insights multimodally. Now to go
[33:25] from that to foundation models that were single
modality then domain specific then multimodal
[33:34] uh has taken months to weeks. It's it's quite
incredible. With the advent now of recursive
[33:43] self self-improvement, we'll see an even greater
acceleration. It behooves us to accept that we may
[33:50] not have all the answers. We also we also need to
learn to be be comfortable in being uncomfortable
[33:59] because it's evolving an incredible pace. But
if you assemble these multi-disiplinary teams,
[34:07] if we focus on the collab, the the synergistic
impact of collaboration ecosystems, I'm certain
[34:16] that AI will help us not only automate
tasks but actually derive new insights
[34:22] and those new insights will help us advance
in our journey. That's our bold ambition
[34:28] that is to eliminate cancer as a cause
of death. With that, I'd like to thank
[34:33] you for your attention. I'll happily take any
questions you might have. Thank you. [applause]
[34:42] Um it's great cover uh um introduction here.
So would you mind elaborate a little more on
[34:47] this multimodel fundamental fund foundation
models because in the current system particular
[34:55] regulatory perspective it's all condition like
disease-based. So if we move into that direction,
[35:01] do you see the current framework is there to
support uh something like this even though
[35:07] technically it's totally doable? The answer is
absolutely because remember you're pre-training
[35:13] a model to make it the most capable. We not
going to use this model in a zero short manner.
[35:20] You create then a solution on the base of this
model in an application. You go to the FDA with
[35:26] the application. So still even it's a multimodel
one but you still deal with uh each modality one
[35:33] by one regulations you create the application the
application eventually is is something that you
[35:39] will deploy the model itself if you look at the
AI approved models that are imaging based okay
[35:48] they includes ResNet or REGNET 50 or 23 just
imagine here you swap that model this general
[35:56] supervised model that is used in most of the um
AI uh biomarkers into um a foundation model you
[36:07] have to provide all the aspects related to the
pre-training of the model this data provenence
[36:13] it's and so on so forth but is an entirely
doable process to be honest path chat uh received
[36:21] breakthrough device designation in January 2025 so
you can see that the FDA is prepared to entertain
[36:29] the utilization of pre-trained foundational
models. Okay, great to hear that. Thank you.
[36:38] Pathologists traditionally are, I'll call
it, cautious in adopting new tool sets and
[36:43] and in particular with AI. I'm curious, are
are you taking that into consideration? Are
[36:48] you building that out? How are you getting the
adoption from them? It's it's been a labor of
[36:53] love. Yeah. Okay. There are many barriers.
You may you made a very important point uh
[37:00] here. That's the cultural barrier is an important
one. The cultural and actually the the directly
[37:05] professional barrier because you have to go
from looking down the microscope and rendering
[37:09] decisions uh by humans to a system where actually
the pathologist may not even comprehend the the
[37:16] embedding that was used for that decision to be
made and rendered by a machine on a screen. Yeah.
[37:23] So that required a lot of engagement, learn and
development and education. First um there were
[37:31] four other types of hurdles including the lack
of scanners, the lack of adoption of the basic
[37:37] infrastructure needed, lack of billing codes, lack
of sufficient validation for it to be sufficiently
[37:45] performant to be introduced in clinical practice.
We've been addressing this head on through early
[37:51] access programs, through engagement with key
opinion leaders and community practitioners and
[37:58] it's been a phenomenal journey. Oh perhaps
I should qualify it better as a formidable
[38:04] journey. Hello. Thank you so much for that talk.
Uh quick question. You a lot of these models are
[38:11] built on a lot of existing traditional 2DHC
data. What do you think are the uh what do
[38:17] you call future directions of data sources? Are
you thinking spatial genomics or more like even
[38:22] tissue clearing type 3D approaches to this? Yes,
it's advancing so quickly and absolutely and we've
[38:30] seen the first generation of the highly capable
uh multimodal models including spatial proteomics
[38:39] or multiplex imenotyping just shipped in the
last three months and I have to say that I was
[38:47] skeptic I was skeptical yeah I I thought spatial
biology will always remain extremely let's say
[38:55] handcraft crafted in required human feedback. Um
what I've seen I'm not going to give commercial
[39:02] names here but I've seen from two companies
and one academic partner academic partner I
[39:06] can mention Pais Mahmood's team and Sizun Jang
both from Harvard what they've done is quite
[39:12] incredible. It compresses three to four weeks
of work into a couple of hours stops and with
[39:21] really repeatable patterns of cell segmentation,
tissue segmentation, semantic extraction of what's
[39:30] represented on the section. Yes, absolutely.
The core registration is is still it's evolving
[39:39] really quickly. I'd like to see even more capable
solutions to be honest. All right, thank you so
[39:45] much for your time today, George. If we can give
him a round of applause. We appreciate your time.
