Genentech Embeds AI in Clinical Units to Cut Trial Drafting Time

Driving Innovation: How to Build and Manage AI Products in the Biotech Industry With Genentech — Ai4

Ai4

Centralized AI units fail in pharma because model adoption requires embedding engineers directly into existing clinical workflows like Microsoft Word rather than deploying standalone platforms.

AI-assisted summary of Driving Innovation: How to Build and Manage AI Products in the Biotech Industry With Genentech — original source. Check the original for full context.

Source published:

Detailed notes

## Background Published on YouTube by the Ai4 channel on August 27, 2025, this talk features Amir Emadzadeh, Director of Software Engineering at Genentech. According to the description, Emadzadeh shares insights and strategies from his journey building AI teams and innovative AI-driven products across major pharma companies, focusing on cross-functional management and real-world biotech impact.

## TL;DR Amir Emadzadeh outlines a practical blueprint for developing and managing AI and Generative AI products within biotech, drawing from his experience at Genentech and Roche. He argues strongly for embedding engineering teams directly into clinical business units rather than utilizing centralized AI departments, allowing developers to build solutions that fit existing workflows. The talk details modern AI team composition, emphasizing full-stack product delivery over isolated notebooks, disciplined requirements management, and multi-dimensional success metrics. Emadzadeh illustrates these principles with production RAG-powered tools integrated into Microsoft Word for clinical trial document workflows, including protocol redactions, amendment insights, and automated informed consent form drafting.

## Atmosphere & style The presentation has a grounded, pragmatic, and engineering-centric tone, delivered with the clear, methodical cadence of an experienced technical leader. Emadzadeh cuts through generative AI hype to focus on the unglamorous realities of change management, user interface friction, and stakeholder alignment in regulated environments. Listening to the talk feels like an actionable, insider briefing on bridging high-tech software engineering practices with complex pharmaceutical operations.

## Key takeaways - [01:09](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=69s) Embedding AI engineering teams directly within business units is far more effective in biotech than centralized AI models, as algorithmic knowledge cannot substitute for deep clinical and scientific domain context. - [02:00](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=120s) Treating AI like a general utility without domain integration leads to low adoption and operational friction when detached teams try to impose disruptive workflows on domain specialists. - [06:22](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=382s) Modern Generative AI product delivery emphasizes full-stack engineering and intuitive UI/UX design to move models out of isolated Jupyter notebooks and into production applications. - [08:00](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=480s) Empowering technical lead engineers to act as semi-product managers creates direct lines of communication with domain experts, streamlining requirement translation and accelerating delivery. - [10:17](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=617s) Securing explicit, documented sign-offs on baseline MVP requirements is essential for managing scope creep among biotech stakeholders unaccustomed to iterative agile lifecycles. - [11:49](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=709s) Evaluating AI product success requires balancing a triad of business metrics (cost and turnaround), technical metrics (accuracy and latency), and operational metrics (adoption and throughput). - [12:19](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=739s) Integrating AI tools directly into existing user workflows, such as Microsoft Word add-ins for medical writers, drives higher user adoption compared to forcing adoption of external platforms. - [13:00](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=780s) Keeping a human-in-the-loop to review, accept, or reject AI-generated recommendations ensures compliance and builds trust in clinical document operations. - [13:42](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=822s) Generative AI can automate multi-tiered clinical documentation, such as drafting Informed Consent Forms (ICFs) from global study protocols down to country- and site-specific adaptations. - [14:28](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=868s) A standardized Retrieval-Augmented Generation (RAG) architecture—combining semantic chunking, vector retrieval, and structured JSON generation—serves as a reusable blueprint for complex enterprise document workflows.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=4s) Introduction and Background

Amir Emadzadeh introduces himself as a Director of Software Engineering at Genentech (part of Roche). While clarifying that he is not a professional product manager by trade, he has spent significant time building software and AI products that are actively used across Genentech and Roche. His academic and professional roots lie in applied mathematics and software engineering, having previously worked at prominent tech companies including Qualcomm, Waymo, and Nvidia. Roughly five years ago, he transitioned into the biotech and pharma space to focus on designing and deploying both internal and external-facing software products.

While acknowledging the broad spectrum of artificial intelligence, Emadzadeh notes that his talk focuses predominantly on Generative AI (GenAI), though the core principles and management strategies apply equally well to traditional machine learning systems and newer large language model (LLM) products.

- **Speaker Background:** Applied mathematics and software engineering with experience at Qualcomm, Waymo, Nvidia, Genentech, and Roche. - **Context:** Transitioned into biotech/pharma five years prior to develop impactful software products. - **Focus Area:** GenAI-based products alongside traditional ML systems in pharmaceutical settings.

---

[01:09](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=69s) Centralized AI Teams vs. Embedded AI Teams

When tasked with building an AI engineering organization inside a biotech company, engineering leaders are faced with two primary architectural models: establishing a central AI team serving the entire enterprise, or deploying embedded teams directly within specific business units. Emadzadeh works within Genentech Research and Early Development (gRED), specifically inside Early Clinical Development (ECD), which oversees Phase I and Phase II clinical trials. His primary stakeholders include clinical operations specialists (who manage and run trials), clinical scientists (who draft clinical trial protocols), medical writers (who draft regulatory documents and extract insights for compliance authorities and trial sites), and compliance personnel.

```mermaid flowchart TD accTitle: Embedded vs Centralized AI Team Model in Biotech subgraph Centralized["Centralized Model (Less Effective)"] CA["Central AI Core"] -->|"Disconnected from day-to-day"| U1["Clinical Operations"] CA -->|"Lacks domain nuance"| U2["Medical Writing"] CA -->|"Generic tools"| U3["Drug Discovery"] end

subgraph Embedded["Embedded Model (Recommended)"] EA["Embedded AI Engineering"] <-->|"Direct partnership & workflow integration"| BU["ECD Domain Units\n(Protocol Drafting, Trial Ops, Compliance)"] end ```

Emadzadeh strongly argues against the centralized model, using the analogy of creating a "central math team." Because mathematics is simply a foundational tool applied differently across physics, chemistry, or home construction, having a generic math department makes little sense. Similarly, AI techniques applied in drug discovery differ vastly from the tools required for automated medical document generation. Simply knowing AI does not qualify an engineer to design drug compounds or build rockets without deep domain partnership. An embedded team, by contrast, sits right next to the domain experts, learning their day-to-day bottlenecks, observing their real-world workflows, and building solutions backwards from user needs rather than inventing tools in a vacuum. This integration minimizes resistance and reduces the heavy change-management friction that occurs when disconnected teams try to impose disruptive, unfamiliar workflows.

- **Primary Stakeholders in ECD:** Clinical operations, medical writers, clinical scientists, and regulatory/compliance specialists. - **The "Central Math Team" Analogy:** AI is an enabling tool; general algorithmic knowledge does not replace domain-specific context (e.g., drug discovery vs. protocol writing). - **Embedded Strategy Advantage:** Facilitates backwards design from existing user workflows, avoiding low adoption rates caused by dramatic, unwanted operational shifts.

---

[06:22](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=382s) Core Personas and Team Composition for GenAI Products

To build an adaptable and high-performing AI team, leaders should identify repetitive engineering patterns and establish a core set of key personas before scaling outward. In the modern GenAI and LLM landscape, team composition has shifted markedly compared to five years ago:

| Persona / Role | Modern GenAI Role Focus | Contrast with Traditional ML Era (~5 Years Ago) | |:--- |:--- |:--- | | **Full Stack Developers** (Front End & Back End) | Critical for packaging models into usable user interfaces, delivering software directly to end users instead of leaving code stranded in Jupyter notebooks. | High focus used to be strictly on ML modeling algorithms, whereas UI/UX delivery was often secondary. | | **ML / LLM Ops & DevOps** | Automating CI/CD pipelines, model fine-tuning, observability, runtime performance tracking, and continuous improvement. | Focused heavily on manual train/test/validation splits, feature engineering, and deploying static models (e.g., SVM, Naive Bayes). | | **Automation & QA Engineers** | Building automated testing suites, output verification, and regression pipelines. | Standard software and manual model evaluation. | | **Agile Leads & Technical Program / Product Managers** | Facilitating agile sprints, managing delivery timelines, and structuring product milestones. | Standard project tracking. |

Emadzadeh highlights that developers no longer need to dissect every granular inner mathematical mechanism of base foundation models the way they once had to implement Support Vector Machines (SVMs) or Naive Bayes classifiers from scratch. However, they must bridge the gap between technical infrastructure and real user interaction. To accelerate delivery, he advocates giving lead developers a "semi hat of a product manager," empowering them to interact directly with end users without intermediary translation layers, allowing them to translate messy, non-technical business bottlenecks straight into code.

- **Delivery Over Notebooks:** AI solutions must move out of Jupyter Notebooks or prototype sandboxes and into intuitive full-stack web applications. - **Hybrid Engineering Roles:** Assigning technical leads direct stakeholder-facing responsibilities improves feature alignment and prevents miscommunication.

---

[10:17](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=617s) Managing Requirements and Aligning with Biotech Stakeholders

The foundational pillar of any successful AI product is rigorous requirements gathering. Teams must clearly define and document both functional and non-functional requirements and secure formal sign-offs from stakeholders. In biotech and pharma environments, domain experts are rarely accustomed to standard agile software development lifecycles. They frequently struggle to articulate their precise technical needs and may forget previous feature requests.

To mitigate scope creep and project stall, engineering teams must invest time in educating stakeholders about development iterations. While requirements will naturally evolve, establishing fixed milestones and securing documented sign-offs is mandatory for shipping a viable Minimum Viable Product (MVP). Any incoming requests that emerge after baseline sign-off are routed into the backlog pipeline for future iterations rather than derailing the active release timeline. Emadzadeh transitions to the next critical component of product governance: establishing measurable success metrics to evaluate whether an AI solution actually delivers business impact.

> "To be able to commit to any MVP and then be able to release new features, of course you have to fix your requirements first."

[11:49](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=709s) Defining Metrics and Clinical Trial Document Workflows

To determine whether an AI initiative is truly successful, teams must be exceptionally clear about what they are measuring. Success cannot rely on a single metric; it requires establishing a balanced triad of business metrics, technical metrics, and operational metrics. Being "crisp on what you're aiming for" ensures that engineering efforts stay directly aligned with real-world biotechnology outcomes.

Within clinical trial operations, the engineering team focused heavily on document-centric AI applications. Clinical development involves massive quantities of complex paperwork, making it a prime area for natural language processing and generative AI techniques such as document manipulation, sentiment analysis, summarization, drafting, and redaction.

- **Business Metrics**: Evaluating financial impact, cost savings, and turnaround acceleration. - **Technical Metrics**: Measuring model accuracy, retrieval precision, and latency. - **Operational Metrics**: Tracking user adoption, error rates, and workflow throughput across medical writing teams.

---

[12:19](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=739s) Protocol Redaction and Protocol Amendment Insights

The first major clinical use case tackled protocol redaction. Medical writers historically redacted clinical trial protocols through tedious manual workflows, often outsourcing the downstream software engineering work—such as rendering the final sanitized PDF—to third-party vendors. Recognizing that medical writers spend their workday inside Microsoft Word, the team avoided forcing them onto an external platform. Instead, they delivered an integrated Microsoft Word add-in. The tool surfaces AI-driven redaction recommendations directly in the document, keeping a human in the loop to accept, modify, or reject every suggestion with simple button clicks.

A closely related tool addresses protocol amendment insights. Protocol amendments happen frequently throughout trial execution and routinely cost biotech and pharmaceutical companies millions of dollars. By automating the extraction and analysis of amendment changes within the familiar Word interface, the solution delivers substantial cost and time savings. Because of its simplicity and measurable utility, this suite of document tools is being deployed across the entire Roche organization.

---

[13:42](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=822s) Automated Informed Consent Form (ICF) Drafting

The team also developed an AI solution for drafting Informed Consent Forms (ICFs)—the critical legal and ethical documents that prospective patients must review and sign prior to enrolling in a clinical trial. Drafting an ICF traditionally requires cross-referencing complex clinical protocols against standardized organizational templates.

```mermaid flowchart TD accTitle: Informed Consent Form (ICF) Drafting Workflow subgraph Ingestion["1. Document Ingestion"] P["Clinical Protocol"] T["ICF Master Template"] end

subgraph Processing["2. AI Processing"] LLM["LLM Drafting Engine"] end

subgraph Hierarchy["3. Multi-Level Generation"] Study["Study-Level ICF Draft"] Local["Country & Site-Level Adaptations"] end

P --> LLM T --> LLM LLM --> Study Study --> Local ```

Using Large Language Models (LLMs), the system ingests the new protocol alongside the ICF template, automatically locating relevant clinical parameters in the protocol and populating the template accordingly. This automated drafting occurs first at the global study level when a new trial is initiated, after which writers can tailor the output down to specific country-level and site-level regulatory requirements.

---

[14:28](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=868s) Core RAG Architecture and Enterprise Integration

Underpinning these clinical tools is a standardized Retrieval-Augmented Generation (RAG) framework designed to handle large-scale regulatory documents reliably. The engineering pipeline breaks long-form clinical protocols into smaller semantic chunks, indexes them inside vector databases, and performs semantic retrieval to fetch the precise context required for a given query.

```mermaid flowchart LR accTitle: Generic RAG Architecture for Clinical Products subgraph DataPrep["Data Preparation & Retrieval"] Doc["Source Documents"] --> Chunk["Semantic Chunking"] Chunk --> VDB[("Vector Database")] VDB --> Ret["Context Retrieval"] end

subgraph Generation["LLM & Transformation"] Ret --> Prompts["Prompt Engineering"] Prompts --> LLMNode["LLM Processing"] LLMNode --> JSON["Structured JSON Output"] end

subgraph UserInterface["Client Layer & Storage"] JSON --> WordPlug["MS Word Plugin\n(Human in the Loop)"] WordPlug --> PDFGen["Automated PDF Generation"] PDFGen --> FinalDB[("Final Archive Database")] end ```

Once retrieved, the context is paired with tailored prompts and passed to the LLM. The resulting responses are structured into JSON objects and formatted for the Microsoft Word UI. End users can review, edit, and approve the generated content directly in their workflow. Upon final approval, a single click triggers downstream systems to compile the validated output into a final PDF and automatically upload it to the enterprise database. This generalizable RAG framework serves as the core blueprint across most of the team's production AI products.

## Notable quotes > AI is transforming the biotech industry, but building successful AI products requires more than just technical expertise. — Amir Emadzadeh

> [01:09](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=69s) Simply knowing AI does not qualify an engineer to design drug compounds or build rockets without deep domain partnership. — Amir Emadzadeh

> [01:09](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=69s) An embedded team, by contrast, sits right next to the domain experts, learning their day-to-day bottlenecks, observing their real-world workflows, and building solutions backwards from user needs rather than inventing tools in a vacuum. — Amir Emadzadeh

> [06:22](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=382s) AI solutions must move out of Jupyter Notebooks or prototype sandboxes and into intuitive full-stack web applications. — Amir Emadzadeh

> [06:22](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=382s) Assigning technical leads direct stakeholder-facing responsibilities improves feature alignment and prevents miscommunication. — Amir Emadzadeh

> [10:17](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=617s) To be able to commit to any MVP and then be able to release new features, of course you have to fix your requirements first. — Amir Emadzadeh

> [11:49](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=709s) Success cannot rely on a single metric; it requires establishing a balanced triad of business metrics, technical metrics, and operational metrics. — Amir Emadzadeh

> [12:19](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=739s) Protocol amendments happen frequently throughout trial execution and routinely cost biotech and pharmaceutical companies millions of dollars. — Amir Emadzadeh

## Mentioned Amir Emadzadeh, Genentech, Roche, Qualcomm, Waymo, Nvidia, Genentech Research and Early Development (gRED), Early Clinical Development (ECD), Microsoft Word, Jupyter Notebook, Ai4

## Video description Amir Emadzadeh, Director, Software Engineering, Genentech

Abstracts

AI is transforming the biotech industry, but building successful AI products requires more than just technical expertise. In this talk, I’ll share my journey of building an AI team at two major pharma companies, and the key challenges and successes we’ve experienced while developing innovative AI-driven products. I’ll dive into the strategies for managing cross-functional teams, navigating complex industry requirements, and creating AI solutions that have real-world impact. Attendees will leave with actionable insights on how to build and manage AI products in the biotech space. Don’t miss this opportunity to learn how to drive innovation and make a meaningful impact in AI-powered healthcare.

[Source](https://www.youtube.com/watch?v=tvzSf4V2ANU)

中文简报

Genentech将AI嵌入临床部门以缩短试验起草时间 集中式AI部门在药企难以奏效，因为模型的落地采用需要将工程师直接嵌入到Microsoft Word等现有的临床工作流程中，而不是部署独立平台。 Amir Emadzadeh结合其在Genentech和Roche的从业经验，概述了在生物技术领域开发与管理AI及生成式AI产品的实用蓝图。他极力主张将工程团队直接嵌入临床业务部门，而非设立集中式AI部门，以便开发人员能够打造出契合现有工作流程的解决方案。该演讲详细阐述了现代AI团队的构成，强调相较于孤立的Notebook实验，更注重全栈产品交付、严格的需求管理以及多维度的成功指标。Emadzadeh还以集成到Microsoft Word中、用于临床试验文档工作流的生产级RAG工具为例说明了这些原则，包括临床方案修订遮蔽、修订洞察分析以及知情同意书的自动化起草。 在生物医药行业中，将 AI 工程团队直接嵌入具体业务部门比集中式团队更有效，因为算法专长无法替代深度的临床试验与科研领域背景。 (1:09) 生成式 AI 产品交付需依托全栈工程与直观的 UI/UX 设计将模型移出 Jupyter Notebook 进入生产，同时赋能技术主管兼任部分产品经理职责以直连业务专家加速迭代。 (6:22) 针对不熟悉敏捷开发的药企利益相关者，获取书面确认的 MVP 基线需求是遏制范围蔓延的关键，产品成效需从业务、技术和运营三重指标进行综合评估。 (10:17) 将基于 RAG 的 AI 工具直接集成至用户日常的 Microsoft Word 工作流中，并在方案脱敏与知情同意书（ICF）起草等任务中保留人机协同（Human-in-the-loop），能显著提升用户采纳率并保障临床合规。 (12:19)

Reading summary

Amir Emadzadeh, Director of Software Engineering at Genentech, argues that successful artificial intelligence deployments in biotechnology depend on embedding engineering teams directly inside domain units, aligning with established user workflows, and enforcing disciplined software delivery cycles. Embedded Teams versus Centralized AI Units (1:09) When organizing AI engineering in biotechnology, Emadzadeh strongly advises against centralized enterprise AI groups, comparing them to a hypothetical 'central math team.' Because mathematical and algorithmic tools apply fundamentally differently to drug discovery than to protocol generation, generic technical expertise cannot replace deep domain context. Within Genentech's Early Clinical Development (ECD) unit, engineering teams are embedded alongside clinical scientists, operations specialists, and medical writers. Emadzadeh states that this direct proximity allows engineers to observe operational bottlenecks and design backward from existing routines, reducing the change-management resistance that typically occurs when external teams introduce disruptive, unfamiliar platforms. Shifting Team Roles toward Full-Stack Delivery (6:22) Modern generative AI product development requires a different personnel mix than the traditional machine learning projects of five years ago. Emadzadeh highlights that engineers no longer need to build foundational algorithms like Support Vector Machines or Naive Bayes classifiers from scratch. Instead, the priority has shifted toward packaging models into accessible web applications, moving functionality out of exploratory Jupyter notebooks and directly into end-user interfaces. To achieve this, engineering teams rely heavily on full-stack developers, ML/DevOps specialists managing continuous integration and observability, and automated QA engineers. Emadzadeh also advocates assigning lead developers the 'semi hat of a product manager,' giving them direct access to business stakeholders so they can translate non-technical operational pain points directly into software features without intermediary layers. Requirements Governance and Multi-Tiered Metrics (10:17) Biotechnology domain experts are frequently unaccustomed to agile software lifecycles, which can cause scope creep and project stagnation. To mitigate this risk, engineering teams must educate stakeholders on iterative development and secure documented, formal sign-offs on functional and non-functional requirements before committing to a Minimum Viable Product (MVP). Subsequent requests are routed to future backlog cycles rather than altering the active release timeline. Project evaluation must also move beyond isolated model performance. Emadzadeh stresses defining a clear triad of success criteria: business metrics (such as financial impact and turnaround acceleration), technical metrics (including model precision, retrieval accuracy, and latency), and operational metrics (such as software adoption rates and medical writing throughput). In-Workflow Document Automation and RAG Architecture (12:19) Because clinical trials generate high volumes of documentation, Genentech prioritized natural language processing tools for protocol redaction, protocol amendment analysis, and Informed Consent Form (ICF) generation. Rather than requiring medical writers to navigate separate software, the team integrated redaction and amendment features directly into Microsoft Word via an add-in, preserving a human-in-the-loop mechanism to review and accept AI suggestions. These tools run on a standardized Retrieval-Augmented Generation (RAG) framework. Source protocols are segmented into semantic chunks, stored in vector databases, and retrieved using semantic search before passing through customized LLM prompts. The model generates structured JSON outputs that populate the Word interface, allowing writers to finalize documents, compile compliant PDFs, and archive records to enterprise storage with a single click. Genentech is scaling this document tool suite across the broader Roche enterprise, demonstrating that generative AI adoption in clinical settings relies less on proprietary model creation and more on domain-embedded delivery, strict requirements baselining, and frictionless workflow integration.

精炼阅读摘要

Genentech（基因泰克）软件工程总监 Amir Emadzadeh 指出，生物制药领域成功落地人工智能产品的核心，在于摒弃脱离业务的中心化组织、建立直面业务痛点的嵌入式工程团队，并依托标准化检索增强生成（RAG）架构与无缝集成既有工作流的交互设计来驱动实际业务价值。 嵌入式工程架构优于中心化团队 (1:09) Emadzadeh 结合在 Genentech 早期临床开发（ECD）部门的实践强调，在生物医药企业内设立服务全公司的通用“中心化 AI 团队”通常效果欠佳。他以“中心数学团队”为类比指出，数学作为基础工具在物理、化学或建筑工程中的应用方式大相径庭，AI 算法亦是如此——仅掌握算法基础的工程师，无法在缺乏深厚业务沉淀的情况下独立完成药物研发或临床试验方案设计。ECD 涉及临床运营、临床科学家、医学撰稿人及合规人员等多类角色，不同群体的技术痛点存在显著差异。 Emadzadeh 极力倡导采用嵌入式团队模式，让软件工程师直接融入具体业务单元，与领域专家并肩工作并实时观察日常工作流中的真实堵点。这种以终为始、由用户真实诉求逆向推导技术方案的路径，能够最大限度降低外部团队强推陌生系统所引发的抵触情绪与变革管理成本，从而有效提升业务部门对智能化工具的采纳意愿。 生成式AI时代的工程角色与需求管理 (6:22) 随着生成式 AI 与大语言模型（LLM）的发展，工程团队的人员结构和分工相比五年前传统的机器学习时代发生了显著演变。Emadzadeh 指出，开发者当前已不再需要像过去从底层推导支持向量机（SVM）或朴素贝叶斯模型那样深究每一个细粒度数学公式，团队重心已转向全栈开发、LLM Ops、自动化测试验证与敏捷交付。其中，全栈工程师的作用尤为关键，其核心职责是将算法能力封装为直观可用的应用，避免代码停留在 Jupyter Notebook 实验原型阶段；同时，让核心研发人员兼具部分产品经理职责，直接与终端用户对话以消除中间沟通损耗。 然而，生物医药领域的领域专家通常不熟悉敏捷软件开发流程，往往难以清晰定义技术指标，并容易在开发周期中变更或遗忘初期诉求。为防止项目停滞与范围蔓延，工程团队必须投入精力向业务方普及迭代交付逻辑，并在进入研发前正式签署确认功能与非功能性需求基线。Emadzadeh 强调，固化需求是按时交付最小可行产品（MVP）的基石，所有在签署后新增的业务诉求均须统一导入需求池（Backlog）为后续版本排期，不得直接扰乱当前迭代计划。 三维指标与临床文档自动化落地 (11:49) 为准确衡量 AI 工具是否产生实质影响，Emadzadeh 提出团队必须设立平衡的三维指标体系，即结合商业指标（财务回报、成本节约与周转提速）、技术指标（模型准确率、检索精确度与响应延迟）以及运营指标（用户采纳率、系统错误率与撰写吞吐量）。早期临床试验涉及极其繁复的合规文书，这使得自然语言处理与大语言模型在文档操作、摘要提炼、自动起草与信息脱敏等文本密集型场景中具备巨大的落地空间。 团队由此聚焦并推出了数个关键临床文档工具。其一是临床方案审查脱敏工具（Protocol Redaction），将过去繁琐的手工涂抹以及外包给第三方软件厂商渲染合规 PDF 的低效模式收敛为内部自动化处理；其二是临床试验方案修订洞察工具（Protocol Amendment Insights），针对业内极其频繁且每次变更成本常达数百万美元的方案修改流程，自动化提取差异并分析潜在影响，显著压缩了跨版本核对的时间与资金成本；其三是知情同意书（ICF）自动化起草系统，利用大语言模型将复杂的临床试验方案参数自动填充至标准模板中，并支持从研究级版本向下分发至国家及研究中心层级的本地化适配。 标准化RAG与原生工作流深度融合 (14:28) 在底层技术实现上，上述文档智能化工具均依托一套通用的检索增强生成（RAG）管道架构。该系统将长篇临床试验方案拆解为语义文本块，通过向量数据库进行语义索引与检索；当获取到相关背景信息后，配合定制化提示词输入大语言模型，并输出规范的结构化 JSON 数据对象。 在交付与交互层面，团队并未要求医学撰稿人离开其日常工作环境去使用独立的外部系统，而是将整套 AI 能力直接内嵌于医学撰稿人高度依赖的 Microsoft Word 插件中。系统生成的脱敏建议与起草内容会直接显示在文档编辑界面内，保持人机协同（Human-in-the-Loop）机制，供专业人员点击按钮完成采纳、修正或否决。确认无误后，系统后台一键自动化将其编译为合规的 PDF 文件并上传至企业数据库。凭借这种极低侵入性的产品形态与可量化的实用价值，该套工具已被推广部署至整个 Roche 集团内部。 Emadzadeh 的分享表明，在严监管与强专业属性的制药行业中，AI 产品的商业化成功不仅取决于底层模型的先进程度，更取决于团队是否真正深入临床业务场景、利用无缝嵌入传统办公生态（如 Word）的交互形态来降低使用门槛。同时，通过嵌入式协作与严格的需求基线管理化解业务专家的敏捷协作鸿沟，是确保前沿 AI 技术平稳转化为实际研发效益的决定性因素。

Detailed notes — zh

## Background 本内容由 YouTube 频道 Ai4 于 2025 年 8 月 27 日发布，主讲人为基因泰克（Genentech）软件工程总监 Amir Emadzadeh。演讲简介指出，Amir 基于自己在两家大型药企搭建 AI 团队的实战历程，深入探讨了在生物医药领域构建和管理 AI 产品所面临的关键挑战、跨职能团队协作策略以及让 AI 方案产生现实影响的实践经验。 ## TL;DR Amir Emadzadeh 结合自身在罗氏与基因泰克的落地经验，系统拆解了生物制药行业中 AI 产品的构建之道。他明确反对设立脱离业务的“中心化 AI 团队”，主张采用深入临床运营与科学家工作流的“嵌入式团队”，并推崇全栈开发配合“半个产品经理”思维的高效敏捷模式。针对药企非技术人员沟通成本高的痛点，他强调了严格的需求工程管理与多维衡量指标。最后，他展示了基于通用 RAG 架构的 Word 插件产品矩阵，包括协议脱敏、方案变更洞察及知情同意书自动起草等，实现了人机协同并已在罗氏全集团推广。 ## Atmosphere & style 整个分享务实而凝练，洋溢着资深工程管理者直面复杂行业现实的清醒与笃定。Amir 的表达条理清晰、毫不拖泥带水，巧妙地用“数学家不会直接造火箭”等接地气的比喻切中传统组织架构的弊病。现场散发着浓厚的工业界工程实战气息，没有浮夸的概念炒作，而是专注于从业务痛点、代码落地到真实降本增效的闭环，给观众一种严谨、沉稳且极具启发性的专业沉浸感。 ## Key takeaways - [02:09](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=129s) 反对在药企建立中心化 AI 团队，技术专家无法仅凭算法知识搞药物研发，必须与一线业务专家深度结对。 - [02:09](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=129s) 推崇将 AI 团队嵌入具体业务部门（如早期临床开发 ECD），直接与临床运营、临床科学家等终端用户密切协作。 - [02:09](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=129s) 采用逆向工作法（Working Backward），顺应业务人员既有流程设计工具，避免推行颠覆式流程带来过大的组织变革阻力。 - [06:22](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=382s) Gen AI 时代核心工程力量由传统 ML 算法研究员转向全栈开发工程师，重点是将原型真正封装交付为可用系统。 - [06:22](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=382s) 工程师需要承担“半个产品经理”的角色，由技术负责人直接对接业务干系人，防止层层转述导致痛点失真。 - [10:17](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=617s) 因药企人员多缺乏敏捷认知，需求管理必须区分功能与非功能规范，并在开发前强制业务方书面确认（Sign-off）。 - [10:17](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=617s) 实行严格的需求管道化机制，当前版本聚焦 MVP 交付，新增变动一律排入下一阶段管道，保证项目落地确定性。 - [10:17](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=617s) 必须确立业务、技术和运营三个维度的衡量指标，全方位评估 AI 产品为药企带来的真实降本增效价值。 - [12:01](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=721s) 针对医学撰写员高频使用 Word 的习惯，开发原生 Word 插件形态的脱敏工具，以人机协同方式一键生成合规 PDF。 - [12:01](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=721s) 方案变更洞察工具针对频繁变更导致的数百万美元高昂成本，通过比对提取痛点，已获罗氏集团全域推广上线。 - [12:01](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=721s) 知情同意书（ICF）起草工具结合方案与模板自动抽取填充，支持从研究项目级逐级向下微调至国家及试验中心级别。 - [12:01](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=721s) 多个临床文档处理产品底层基于通用 RAG 架构，打通文档切片、向量检索、Prompt 增强推理及 Word 渲染回填闭环。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=4s) 个人背景与生物医药行业构建 AI 的核心挑战 演讲者 Amir Emadzadeh 现任基因泰克（Genentech）软件工程总监。他开门见山地指出，虽然自己并非职业产品经理，但他在基因泰克及其母公司罗氏（Roche）内部打造了多款已落地使用的产品，本次演讲旨在分享在大型药企构建与管理 AI 产品的实战经验。Amir 拥有应用数学与软件工程背景，曾在高通（Qualcomm）、Waymo 以及英伟达（Nvidia）等高科技企业工作，并在过去五年决定转向生物科技与制药领域，探索为内部及外部客户构建软件产品。 关于本次分享的范畴，Amir 表示虽然大家对 AI 的定义都不陌生，但他将重点放在生成式 AI（Gen AI）上，不过所涉及的经验与原则同样适用于传统机器学习（ML）与新型 Gen AI 产品。在生物医药公司构建 AI 团队时，管理者首先面临一个组织架构维度的基本问题：是建立一个服务于整个公司的大一统“中心化团队”（Centralized Team），还是采取其他形式？在基因泰克内部，Amir 隶属于 gRED（Genentech Research and Early Development，研究与早期开发）组织，具体处在 ECD（Early Clinical Development，早期临床开发）部门，主要负责临床一期和二期试验。 ```mermaid flowchart TD accTitle: 基因泰克早期临床开发协作关系图 subgraph Org["gRED（研究与早期开发）/ ECD（早期临床开发）"] A["嵌入式 AI 产品团队"] end subgraph Stakeholders["跨职能业务干系人"] B["临床运营 (Clinical Operations)<br/>负责实际临床试验运作"] C["临床科学家 (Clinical Scientists)<br/>负责撰写临床方案 (Protocols)"] D["医学撰写员 (Medical Writers)<br/>负责草拟文档与洞察提取"] E["法规合规团队 (Compliance Folks)<br/>负责合规审查与机构申报"] end A <--> B A <--> C A <--> D A <--> E ``` --- ### [02:09](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=129s) 团队组织架构：中心化团队 vs 业务嵌入式团队 在生物医药环境中，AI 团队需要服务的业务干系人背景差异极大：临床运营团队负责临床试验的日常推进，临床科学家负责撰写临床试验方案（Protocols），医学撰写员负责起草专业文件并提炼洞察以提交给合规机构或临床试验机构（Sites）。面对如此多样化的需求，Amir 明确表达了自己对“中心化 AI 团队”模式的反对态度。他形象地打了个比方：设立中心化 AI 团队就像是在公司设立一个“中心化数学团队”一样不合理。数学只是一种工具，物理学家、化学家、建筑工人都在使用数学，但使用的数学类型和应用场景完全不同，掌握数学并不代表一个人就能造火箭成为宇航员。同理，仅仅精通 AI 技术绝不意味着可以直接搞药物研发（Drug Discovery），技术人员必须与领域专家深度结对。 因此，基于五年的探索经验，Amir 极力推荐“嵌入式团队”（Embedded Teams）模式，即将 AI 团队直接植入到各个具体的业务组织中。这种模式的核心优势在于： - **贴近真实工作流**：AI 团队无需全知全能，而是与业务最终用户紧密结对，深入理解其日常业务流程和实际瓶颈。 - **避免闭门造车**：不是在真空中臆想功能，而是依据用户的核心诉求与高频痛点量身定制自动化或 AI 工具。 - **降低变革阻力（Change Management）**：遵循“逆向工作法”（Working Backward），打造最大程度贴合用户现有业务流程的产品，而不是强迫业务人员大幅改变习惯去适应一套全新系统。经验表明，要求业务线全盘接受颠覆式新流程往往难以推行。 --- ### [06:22](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=382s) AI 产品团队的核心角色演变与全栈化要求 在团队搭建方面，Amir 建议首先识别业务中反复出现的业务模式（Repetitive Patterns），围绕这些模式构建核心团队画像（Personas），后续再随业务发展动态增减成员。与五年前传统的机器学习团队相比，当前生成式 AI 与大语言模型（LLM）时代的团队构成发生了显著变化： | 团队角色 / 职能 | 传统 ML 时代（约5年前） | 现今 Gen AI / LLM 时代 | |:--- |:--- |:--- | | **核心工程力量** | 重度依赖 ML 工程师，专注训练/验证/测试（Train/Test/Validation）、特征工程与传统模型实现（如 SVM、朴素贝叶斯）。 | **全栈开发工程师（Full Stack）**成为绝对主力（前端、后端不可或缺），因为原型不能停留在 Jupyter Notebook 中，必须交付成可用产品；底层 LLM 细节无需像以往那样从零实现。 | | **运维与工程化** | 传统 DevOps、MLOps 负责模型部署与维护。 | **LLMOps、MLOps 与 DevOps 并重**，负责模型微调（Fine-tuning）、可观测性（Observability）、性能监控与全链路自动化。 | | **质量与测试** | QA 工程师以常规手动和自动化功能测试为主。 | **自动化工程师（Automation Engineers）与 QA** 紧密配合，实现高度自动化的测试流水线与 CI/CD。 | | **敏捷管理与产品** | 标准 Scrum Master、TPM、PM 分工明确。 | 采用敏捷开发，开发者需具备“半个产品经理”（Semi-PM）思维，技术负责人直接与终端用户对接，减少信息传递层级。 | Amir 强调，在自己的团队中，他倾向于让每个产品的技术负责人（Lead）以及开发人员直接面对干系人交流。因为业务人员提出的痛点往往不是结构化的技术需求，而是日常工作中的具体卡点，如果中间隔着层层传话，诉求极易失真。让工程师戴上“半个产品经理的帽子”，能够极大加速沟通与产品迭代。 --- ### [10:17](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=617s) 需求管理与衡量指标在制药企业的落地法则 在生物医药行业推进软件产品时，需求管理（Requirements Engineering）是决定项目成败的核心环节。Amir 指出，生物医药领域的同事普遍缺乏软件开发的认知体系，不熟悉敏捷迭代与发布周期，经常出现“提不清需求”或“遗忘前期诉求”的情况。 为此，团队必须建立严格的需求管理流程： 1. **区分功能性与非功能性需求**：与业务用户紧密合作，将模糊的痛点沉淀为清晰的书面规范。 2. **强制书面确认（Sign-off）**：在正式编码构建前，必须由业务方签字确认需求基准。 3. **需求变更管道化**：虽然需求发生变更是常态，但必须先锁定当前版本的最小可行产品（MVP）范围，后续新增的需求一律放入排期管道（Pipeline）等待下一轮迭代，以确保项目交付的确定性。 4. **多维衡量指标（Metrics）**：必须清晰设定三类指标来评判产品是否成功，包括业务指标（Business Metrics，如降本增效幅度）、技术指标（Technical Metrics，如准确率、延迟）以及运营指标（Operational Metrics）。 --- ### [12:01](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=721s) 落地案例拆解：协议脱敏、方案变更洞察与知情同意书起草 Amir 介绍了团队在早期临床阶段落地的三个核心文档处理产品，涵盖文本抽取、情感分析、自动化起草、摘要及脱敏等能力： ```mermaid flowchart LR accTitle: 临床试验智能文档处理 RAG 架构 subgraph Input["输入层"] A["Word / PDF 临床方案与模板"] end subgraph CoreEngine["RAG 检索增强生成引擎"] B["文档分块 (Chunking)"] C["向量数据库 (Vector DB)"] D["语义检索 (Retrieval)"] E["Prompt 工程 + LLM 推理"] end subgraph Presentation["交互与交付层 (Human-in-the-loop)"] F["Word 插件建议呈现"] G["用户审核: 接受 / 编辑 / 拒绝"] H["转为 JSON / 一键生成 PDF 归档入库"] end A --> B --> C --> D --> E --> F --> G --> H ``` - **临床试验方案脱敏工具（Protocol Redaction）**： 医学撰写员此前需要手动对试验方案进行脱敏，并将最终 PDF 制作外包给第三方供应商。团队发现撰写员日常主要在 Microsoft Word 中工作，因此直接开发了 Word 插件。系统通过 AI 自动识别并高亮建议脱敏的内容，采用“人机协同”（Human-in-the-loop）机制，用户只需点击按钮即可执行接受、编辑或拒绝，一键完成脱敏并生成合规 PDF。 - **方案变更洞察分析（Protocol Amendment Insights）**： 临床试验方案的变更极其频繁，每次变更都会给制药企业带来数百万美元的额外成本。该工具采用与脱敏插件一致的 Word 交互体验，自动化提取和比对变更洞察，因其实用性强且能直接节省巨额成本，目前已在整个罗氏集团（Roche）全面推广上线。 - **知情同意书自动起草（ICF Drafting）**： 知情同意书（Informed Consent Form, ICF）是患者入组试验前必须签署的重要法律文件。该工具利用 LLM 读取标准模板与新的临床试验方案，自动从方案中提取对应信息填充到模板中。流程先在研究项目级别（Study level）生成基准草案，随后可根据具体国家（Country level）及试验中心（Site level）的要求进一步微调。 - **通用底层架构（Generic RAG Framework）**： 上述产品均基于通用的 RAG（检索增强生成）架构实现。系统将文档进行切片分块（Chunking）并存入向量数据库，根据任务检索最相关的片段，连同精心设计的提示词（Prompts）喂给 LLM，将生成的结果转换为结构化 JSON 格式，最后重塑渲染回 Microsoft Word 界面，供业务人员审核批准后一键生成 PDF 并归档入库。 ## Notable quotes > [02:09](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=129s) 设立中心化 AI 团队就像是在公司设立一个“中心化数学团队”一样不合理。 — Amir Emadzadeh > [02:09](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=129s) 数学只是一种工具，物理学家、化学家、建筑工人都在使用数学，但使用的数学类型和应用场景完全不同，掌握数学并不代表一个人就能造火箭成为宇航员。 — Amir Emadzadeh > [02:09](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=129s) 仅仅精通 AI 技术绝不意味着可以直接搞药物研发（Drug Discovery），技术人员必须与领域专家深度结对。 — Amir Emadzadeh > [02:09](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=129s) 遵循“逆向工作法”（Working Backward），打造最大程度贴合用户现有业务流程的产品，而不是强迫业务人员大幅改变习惯去适应一套全新系统。 — Amir Emadzadeh > [06:22](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=382s) 全栈开发工程师（Full Stack）成为绝对主力，因为原型不能停留在 Jupyter Notebook 中，必须交付成可用产品；底层 LLM 细节无需像以往那样从零实现。 — Amir Emadzadeh > [06:22](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=382s) 业务人员提出的痛点往往不是结构化的技术需求，而是日常工作中的具体卡点，如果中间隔着层层传话，诉求极易失真。 — Amir Emadzadeh > [06:22](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=382s) 让工程师戴上“半个产品经理的帽子”，能够极大加速沟通与产品迭代。 — Amir Emadzadeh > [10:17](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=617s) 在正式编码构建前，必须由业务方签字确认需求基准。 — Amir Emadzadeh > [10:17](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=617s) 虽然需求发生变更是常态，但必须先锁定当前版本的最小可行产品（MVP）范围，后续新增的需求一律放入排期管道（Pipeline）等待下一轮迭代，以确保项目交付的确定性。 — Amir Emadzadeh > [12:01](https://www.youtube.com/watch?v=tvzSf4V2ANU&t=721s) 临床试验方案的变更极其频繁，每次变更都会给制药企业带来数百万美元的额外成本。 — Amir Emadzadeh ## Mentioned Amir Emadzadeh, Genentech (基因泰克), Roche (罗氏), Qualcomm (高通), Waymo, Nvidia (英伟达), gRED (Genentech Research and Early Development), ECD (Early Clinical Development), Microsoft Word, Jupyter Notebook, Ai4 ## Video description Amir Emadzadeh, Director, Software Engineering, Genentech Abstracts AI is transforming the biotech industry, but building successful AI products requires more than just technical expertise. In this talk, I’ll share my journey of building an AI team at two major pharma companies, and the key challenges and successes we’ve experienced while developing innovative AI-driven products. I’ll dive into the strategies for managing cross-functional teams, navigating complex industry requirements, and creating AI solutions that have real-world impact. Attendees will leave with actionable insights on how to build and manage AI products in the biotech space. Don’t miss this opportunity to learn how to drive innovation and make a meaningful impact in AI-powered healthcare. [Source](https://www.youtube.com/watch?v=tvzSf4V2ANU)

Source quotes

[2:09] Amir Emadzadeh: 设立中心化 AI 团队就像是在公司设立一个“中心化数学团队”一样不合理。 [2:09] Amir Emadzadeh: 数学只是一种工具，物理学家、化学家、建筑工人都在使用数学，但使用的数学类型和应用场景完全不同，掌握数学并不代表一个人就能造火箭成为宇航员。 [2:09] Amir Emadzadeh: 仅仅精通 AI 技术绝不意味着可以直接搞药物研发（Drug Discovery），技术人员必须与领域专家深度结对。 [2:09] Amir Emadzadeh: 遵循“逆向工作法”（Working Backward），打造最大程度贴合用户现有业务流程的产品，而不是强迫业务人员大幅改变习惯去适应一套全新系统。 [6:22] Amir Emadzadeh: 全栈开发工程师（Full Stack）成为绝对主力，因为原型不能停留在 Jupyter Notebook 中，必须交付成可用产品；底层 LLM 细节无需像以往那样从零实现。 [6:22] Amir Emadzadeh: 业务人员提出的痛点往往不是结构化的技术需求，而是日常工作中的具体卡点，如果中间隔着层层传话，诉求极易失真。 [6:22] Amir Emadzadeh: 让工程师戴上“半个产品经理的帽子”，能够极大加速沟通与产品迭代。 [10:17] Amir Emadzadeh: 在正式编码构建前，必须由业务方签字确认需求基准。 [10:17] Amir Emadzadeh: 虽然需求发生变更是常态，但必须先锁定当前版本的最小可行产品（MVP）范围，后续新增的需求一律放入排期管道（Pipeline）等待下一轮迭代，以确保项目交付的确定性。 [12:01] Amir Emadzadeh: 临床试验方案的变更极其频繁，每次变更都会给制药企业带来数百万美元的额外成本。

Source quotes

Amir Emadzadeh: AI is transforming the biotech industry, but building successful AI products requires more than just technical expertise. [1:09] Amir Emadzadeh: Simply knowing AI does not qualify an engineer to design drug compounds or build rockets without deep domain partnership. [1:09] Amir Emadzadeh: An embedded team, by contrast, sits right next to the domain experts, learning their day-to-day bottlenecks, observing their real-world workflows, and building solutions backwards from user needs rather than inventing tools in a vacuum. [6:22] Amir Emadzadeh: AI solutions must move out of Jupyter Notebooks or prototype sandboxes and into intuitive full-stack web applications. [6:22] Amir Emadzadeh: Assigning technical leads direct stakeholder-facing responsibilities improves feature alignment and prevents miscommunication. [10:17] Amir Emadzadeh: To be able to commit to any MVP and then be able to release new features, of course you have to fix your requirements first. [11:49] Amir Emadzadeh: Success cannot rely on a single metric; it requires establishing a balanced triad of business metrics, technical metrics, and operational metrics. [12:19] Amir Emadzadeh: Protocol amendments happen frequently throughout trial execution and routinely cost biotech and pharmaceutical companies millions of dollars.

Original transcript

[0:04] I'm Amir Ahmad Zada. Um I am a director [0:06] of software engineering at Genentech. Uh [0:10] so I'm not a professional product [0:12] manager but um I've built a few products [0:14] that are being used in my company GenEnt [0:17] RO and I want to share my personal [0:21] experience um in uh building these uh [0:25] products. My background is in um applied [0:29] math and software. I used to work for [0:31] high-tech companies um like uh Qualcomm [0:36] uh Whimo, Nvidia and then over the past [0:40] five years I decided to move to Biotech [0:42] Pharma and try out building software [0:44] products for um uh internal and external [0:49] facing customers there. [0:52] Um, we all know what AI is, so I'm just [0:54] going to skip this. I'll be focusing [0:56] mainly on Gen AI and um, but I think a [1:00] lot of the stuff I'll be talking about [1:01] applies to both traditional MLbased and [1:05] and also new newer genai based products. [1:09] Um, so I guess the first question is, so [1:14] the problem statement is we I want to [1:16] build an AI team inside a biotech [1:17] company. That's what I was asked to do. [1:20] Uh so um there are two options [1:24] basically. Uh first is to build a [1:27] centralized team that serves the whole [1:29] company or a bigger organization. To [1:32] give you some background, I uh belong to [1:35] um GRE um organization in Genentech [1:38] which is research and development and [1:40] within GED I'm in my team is in ECD [1:43] early clinical development which is [1:44] phase one and two um basically early [1:47] stage of physical um clinical trials. [1:50] Now um the people I work with generally [1:54] they are clinical operations folks um [1:57] clinical writers medical writers um [2:00] clinical scientists uh compliance folks. [2:03] So those are the type of uh people that [2:05] I um potentially have to build products [2:07] for. [2:09] Now um the question is if there is a [2:12] central AI team within Greed let's say [2:16] or a broader organization um would that [2:19] be a good choice for um this kind of [2:23] environment where you have all these [2:24] different groups and organizations and [2:27] you want to build products for um people [2:29] um your stakeholders with different [2:31] backgrounds like clinical operations [2:34] their job is to run uh clinical trials [2:37] to operate basically clinical trials [2:39] Clinical scientists do things like um [2:42] writing protocols and clinical uh or [2:44] medical writers do things like um [2:47] drafting specific documents or [2:50] extracting insights from specific [2:52] documents and um and basically serving [2:55] them to um the authorities which could [2:58] be compliance or their end users which [3:02] which could be for example um clinical [3:04] trial sites. [3:07] there's um so this is one one again one [3:09] approach is to have that central team um [3:12] I'm not a big fan of it because um [3:15] what's going to happen is you have this [3:17] to me uh the analogy is to have um a [3:20] central math team in any company and I [3:23] don't think that's a good approach [3:25] because math is just a tool and you [3:27] would end up using math for different [3:30] things as a physicist you use math um [3:33] and as a chemist you use math as well [3:35] and you can use math also for building [3:39] homes. So these are different use cases [3:41] and you use different types of math. [3:42] There's different types of math and um [3:45] each um end user of math will use it for [3:48] their own good. So um because of that I [3:52] um I don't think these kind of [3:54] organizations are very efficient because [3:56] if you are to work with let's say [3:58] clinical writers um you have to use [4:02] specific um let's say uh AI tools that [4:07] are more suitable for that use case [4:09] versus something like drug discovery. [4:11] Like for drug discovery um just because [4:14] you know AI very well it doesn't mean [4:16] you can do drug discovery using AI. you [4:19] have to kind of either partner with [4:21] someone who is very good in drug [4:22] discovery and see where AI can come in [4:25] or um you have to know both but uh just [4:28] just using math doesn't make you for [4:30] example an astronaut or something or you [4:32] cannot build brackets just just because [4:34] you know math um the same I think the [4:37] same thing applies to AI so the other [4:39] approach which I my personal experience [4:42] based on my personal experience I would [4:45] recommend is um having embedded teams [4:48] within the um different organizations [4:51] that um uh want to build AI products. So [4:55] the good thing there is uh you don't [4:57] have to have all these different um [5:01] expertise. you can partner with your end [5:04] users and they c you can understand [5:07] their day-to-day um practices and based [5:10] on that um you can come in as an AI um [5:13] organization who's expert in building AI [5:17] products and software and then um build [5:22] automation tools or whatever AI based [5:25] products uh they need um according to [5:28] their um uh day-to-day problems and [5:32] according according to their main um [5:35] requests as opposed to building [5:36] something in vacuum. [5:40] The key there is to work with the [5:42] stakeholders directly um and understand [5:44] their problems and u work backward as [5:48] the other um uh speakers were talking [5:51] about here. The main goal is to [5:53] understand what the end user wants and [5:55] build something that is as close as [5:58] possible to their current business flow [6:01] versus something that um you could build [6:05] um but then it needs a lot of change [6:07] management or they have to um they have [6:10] to basically change their workflows [6:12] dramatically. Uh usually I would say [6:15] based on my experience they would not be [6:17] open to adopting something completely [6:19] new. [6:22] So there are some now the question is [6:24] what kind of team uh you want to build. [6:28] So I would go for finding out repetitive [6:32] patterns that you may have to deal with [6:35] and then try to have at least a core [6:37] team who addresses these repetitive [6:39] patterns that will come up later and um [6:43] build up the team based on those [6:45] personas. And then over time of course [6:48] things change and then you can start [6:49] modifying the team and add or remove [6:52] member members. But to me there is a [6:54] core uh team uh the core set of personas [6:58] you would need for building an AI team [7:01] in this era. You uh you would need uh [7:05] definitely full stack developers because [7:07] at the end of the day you want to serve [7:09] your end users. So whatever you build, [7:11] you cannot keep it in your Jupyter [7:13] notebooks or notebook LM or whatever [7:15] that you are developing your prototypes [7:18] at. You have to deliver it to the end [7:20] user and it has to be in a way that [7:23] they again they want to use it. So front [7:27] end, back end and full stack are super [7:29] important compared to what we used to do [7:31] 5 years ago. Um back then you would need [7:34] more ML engineers uh people who could um [7:38] do things like train test validation and [7:41] then um work on the more traditional [7:44] machine learning models. But again [7:45] things are changing really fast. So when [7:47] you're working with genai and LM LLM uh [7:51] things are um much different and I think [7:54] you would need to know less about what's [7:57] going on behind the scenes. for example, [7:59] what's the details of LLMs? Um, of [8:02] course, um, so um, you have to [8:05] understand it, but again, you don't have [8:07] to go as deep as what you used to do [8:09] like five years ago when you wanted to [8:10] implement some support vector machine or [8:14] naive base or those kind of traditional [8:16] machine learning models. Um, the other [8:19] personas that you would have to have is [8:22] again the people that you need to build [8:23] a software product which is I think [8:26] generic enough. uh you would need a QA [8:29] team, you would need um a good CI/CD uh [8:32] pipeline in place um like MLOps people, [8:36] uh DevOps people traditionally and of [8:38] course now LLM ops people, you want to [8:41] have uh you need them because you want [8:42] to be able to fine-tune your models. You [8:45] want to be um doing things like [8:47] observability and monitoring the [8:49] performance and then fine-tuning your [8:51] models and improving them. So you would [8:52] need to automate it as much as you can. [8:56] Um of course um we follow agile [9:00] development. So you would need scrum [9:01] masters um and also um what you need is [9:06] um [9:07] again um automation engineers because [9:10] you try to automate your testing and QA [9:13] as much as you want as as much as you [9:15] can and of course you need a technical [9:16] program at least a uh technical program [9:19] manager and a product manager. One thing [9:23] I have done in my teams is I because I [9:26] have to work with my stakeholders and [9:28] they do on they do different products [9:30] and I I will share a few of them in a [9:32] bit. Um it would really help if your [9:36] developers interact with your [9:38] stakeholders directly. At least you have [9:39] a lead for each one of the products that [9:41] you work with so that they have close [9:44] interface with the end users as opposed [9:47] to having another layer where they would [9:50] um pass on um whatever the ask is from [9:53] the end users. And again those ask [9:56] usually are not technical. they face [9:58] some issues um some bottlenecks on their [10:01] day-to-day lives and they want a [10:03] solution for it. So it would really help [10:05] if your developers have this um semi hat [10:09] of like product manager as well so that [10:12] they can have uh direct interactions [10:14] with the end users [10:17] and of course as as any other ML AI [10:20] product um what I would say is um the [10:25] most important thing is requirements. [10:26] You have to really be clear about what [10:28] you're building, what the um functional [10:31] requirements are, what non-functional [10:33] requirements are. Uh you have to partner [10:36] with your end users to understand what [10:39] they want and then you have to make sure [10:41] they sign off on those requirements. [10:43] Another observation I had I've had is um [10:47] again in biotech pharma they are not [10:49] used to building uh software products. [10:51] So what happens is they don't know any [10:54] of these cycles that need to you you [10:56] need to go through. So you have to do [10:58] some work in training them. Um also what [11:00] happens is they don't know exactly what [11:02] they want and they forget what they told [11:03] you or what they wanted before. So it's [11:06] good to document it. It's good to have [11:08] them sign off before you build anything. [11:11] Um because and again requirements change [11:13] that's totally fine but then to be able [11:15] to deliver anything and of course you [11:18] have to somehow uh you have to fix your [11:20] requirements at some point and then you [11:22] can if there is any new requirements [11:23] coming you can put in the pipeline and [11:24] bring them up later but but to be able [11:27] to commit to any MVP um and then be able [11:30] to release new features of course you [11:32] have to fix your requirements uh first. [11:36] Then um the other um point I wanted to [11:40] make was um of course you need metrics [11:43] and metrics are to define basically if [11:46] you're if you have succeeded um or you [11:49] are successful you have you need metrics [11:51] and you need different types of metrics [11:53] um you need business metrics technical [11:55] metrics and also operational metrics so [11:57] you have to be crisp on what you're [11:59] aiming for [12:01] um for clinical trial use cases I'll be [12:04] talking about a few products that I have [12:06] buil in my team. Um, and it's all around [12:09] documents, manipulation of documents, [12:11] sentiment analysis, drafting, [12:13] summarization, things like redaction. [12:15] Um, I'll quickly show you some of the [12:17] stuff uh we've done. The first one is [12:19] redaction. Um, here what we have done, [12:23] we have [12:25] identified this problem where um the [12:28] medical writers wanted to ret redact [12:30] protocols. They were doing this manually [12:33] and they would outsource some good part [12:35] of it. the engineering software [12:36] engineering part where you would create [12:38] the final PDF to some third party. Uh [12:41] here what we realized is uh they work [12:44] Microsoft Word documents. So instead of [12:46] having them move to a new platform, we [12:49] added a plugin to uh their Microsoft [12:52] Word environments where they could but [12:54] by clicking a few buttons they can um [12:57] redact their protocols and um there's [13:00] always this human in the loop um in our [13:03] processes. So we suggest them where what [13:06] they need to redact the protocol and [13:08] then they could accept, edit or reject. [13:11] The other use case is very similar um [13:14] protocol amendment insights where um [13:16] amendments happen a lot for protocols [13:18] and they cost millions of dollars to [13:20] biotech pharma companies. Um we have [13:22] automated the process um very similar [13:25] user experience. Um and this uh these [13:28] products actually being are being rolled [13:30] out to entire RO um because again they [13:34] are um simple and um they're very useful [13:38] and they save money. [13:42] The third one I wanted to talk about is [13:43] ICF drafting. So um ICFs um are informed [13:48] consent forms and they are documents [13:51] that um patients need to sign off on [13:54] before being able to join a trial. So [13:56] here on a high level you have a template [13:59] you have a new protocol you want to you [14:01] want to draft a new ICF for your new [14:03] trial uh or new protocol. So we have [14:05] used LLMs to read the protocol and then [14:08] the template and then fill up the [14:10] template using what they um find in the [14:12] new protocol. Um [14:15] this happens at the um study level which [14:19] is basically where the new study come in [14:22] and then it can be modified to uh [14:25] country and site level as well. [14:28] Um high level again architecture is um [14:31] maybe a little too much. We do prompt [14:33] engineering. What we do is basically uh [14:36] we do rags. So we chunk down our [14:38] documents into uh smaller pieces. we [14:41] create um [14:43] vector databases off of them and then so [14:46] we um basically retrieve the specific [14:50] part that we need to um uh find the uh [14:53] the the results at and then we pass it [14:56] on to our um LLM using our prompts and [15:00] then we generate the results. we change [15:03] them to some JSON format and should [15:05] reshape it into the final um uh format [15:09] which will be consumed by um our product [15:12] for example for the um like amendment [15:15] insights it's going to be shown on the [15:17] Microsoft word uh document that the end [15:19] user uh wants to use and then uh again [15:23] they could edit it they could approve it [15:25] they would send it to a different system [15:26] where by clicking a button and then [15:28] recreate a PDF and provide and basically [15:30] upload the PDF into the final database. [15:33] Um, but this gen like a rag generic [15:37] framework is what we have been using for [15:40] most of our products. [15:42] And I believe that was the end of it. [15:46] Thank you.