Yahoo Engineer Cuts AI Latency to 150ms with AWS S3 Vectors Switch

Building Scalable Enterprise AI Systems Solo, LLMs, AWS Bedrock & Serverless Architecture With Yahoo — Ai4

Ai4

Migrating vector similarity search from DynamoDB to native Amazon S3 Vectors reduces AI query latency from three seconds to 150 milliseconds, eliminating backend compute bottlenecks in serverless architectures.

AI-assisted summary of Building Scalable Enterprise AI Systems Solo, LLMs, AWS Bedrock & Serverless Architecture With Yahoo — original source. Check the original for full context.

Source published: 2025-08-28

Detailed notes

## Background Published by the Ai4 channel on YouTube on August 28, 2025, this talk features Soujanya Vullam, a Software Development Engineer II at Yahoo. According to the session description, it explores how a solo developer independently designed and deployed a secure, production-ready AI Q&A platform using LLMs, AWS Bedrock, AWS serverless architecture, and AI-assisted tools like Cursor and Eraser.io.

## TL;DR The presentation chronicles the end-to-end design of 'Epic Q&A,' an enterprise documentation and code Q&A platform built by a single engineer. An initial serverless setup storing vector embeddings in Amazon DynamoDB resulted in a 3-second query latency bottleneck due to calculating cosine similarity inside AWS Lambda functions. Migrating to the newly released Amazon S3 Vectors resolved the compute overhead and enabled sub-second ANN similarity searches with bundled chunk attribution, slashing response times down to 150 milliseconds. The session illustrates how combining AWS managed services, structured evaluations, and AI-assisted developer tooling allows solo engineers to build cost-effective systems that meet rigorous enterprise design and security standards.

## Atmosphere & style The talk is brisk, technical, and pragmatic, delivering a focused architectural post-mortem packed with system diagrams, CloudWatch metrics, and concrete engineering trade-offs. Vullam presents with earnest enthusiasm and methodical clarity, making complex serverless workflows and retrieval-augmented generation concepts feel accessible and actionable. The session maintains an intense, information-dense momentum right up until the end, when the conference moderator abruptly steps in to yield the stage for the next presenter.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=4s) Building enterprise-grade AI requires extending traditional system design principles to include observability and continuous evaluations (evals) for measuring model accuracy, relevance, and coherence. - [03:21](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=201s) Storing high-dimensional embeddings in DynamoDB introduces severe latency bottlenecks because Lambda compute instances must pull raw arrays and perform cosine similarity math in memory. - [07:10](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=430s) AWS Bedrock provides a fully managed, serverless interface to decouple embedding generation from LLM text generation on a pay-per-use model without provisioning GPU clusters. - [07:10](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=430s) Pairing Amazon S3 with CloudFront serves as a cost-effective dual-purpose foundation for both static React web asset delivery and long-term raw document storage. - [09:56](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=596s) Standard relational and NoSQL databases lack native indexing algorithms to perform sub-second approximate nearest neighbor (ANN) searches across high-dimensional vector spaces. - [12:45](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=765s) Migrating to Amazon S3 Vectors streamlines RAG workflows by returning matched text chunks and metadata directly with vector similarity responses, eliminating secondary data queries. - [12:45](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=765s) A modular serverless architecture allows vector backend components to be swapped cleanly with localized code and IAM permission updates rather than full system re-architecting. - [15:38](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=938s) Pairing automated CloudWatch dashboards with LLM tracing tools like LangSmith and TensorBoard provides complete operational and accuracy telemetry across development iterations. - [18:50](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=1130s) Adopting native vector search and serverless optimization drops end-to-end API response latency from nearly 3 seconds down to approximately 150 milliseconds. - [18:50](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=1130s) Enterprise generative AI risks such as hallucinations, bias, and context gaps are mitigated through strict RAG grounding, document-boundary isolation, and Bedrock content filters. - [18:50](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=1130s) Solo engineers can build and operate scalable, highly secure enterprise architectures for nominal infrastructure costs by leveraging AI-assisted tools like Cursor, Eraser.io, and Gemini.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=4s) Building Enterprise-Grade AI Systems Solo: The Epic Q&A Web App

The presentation opens with a fundamental question that challenges conventional startup and engineering assumptions: "Can a solo developer build enterprise-grade AI systems?" For an independent engineer, bridging the gap between a quick proof-of-concept prototype and a truly production-ready, secure, modular, and scalable enterprise platform often seems out of reach. Driven by this challenge over several months, the speaker developed the Epic Q&A web application—an AI-powered platform capable of parsing massive books and code repositories, automatically generating documentation, and returning context-aware, intelligent answers to user queries in natural language.

Creating an enterprise-grade application requires adhering strictly to standard system design principles like modularity, scalability, security, and reliability. However, in the context of modern AI architectures, two additional pillars take center stage: observability and evaluations (evals). Observability allows engineers to inspect the internal state of an application via generated telemetry—logs, metrics, and distributed traces—to understand how the large language model makes its decisions, evaluate performance trends over time, and ensure stringent compliance with security protocols.

Evals diverge sharply from traditional unit testing by systematically benchmarking model outputs against targeted metrics and guidelines. By continually measuring dimensions such as accuracy, relevance, coherence, completeness, and helpfulness, evals safeguard system reliability, illuminate specific performance bottlenecks, and empower engineers to carry out informed, repeatable iterations across the entire AI development lifecycle.

- **Primary Challenge**: Determining whether a solo developer can progress beyond simple prototypes to architect a production-ready, enterprise-grade AI system. - **Core Use Case**: The Epic Q&A application processes dense codebases and voluminous books into structured documentation and answers natural language questions using semantic context. - **AI Evaluation Criteria**: System performance is measured against metrics including accuracy, relevance, coherence, completeness, and helpfulness.

---

[03:21](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=201s) Initial Serverless Architecture and the DynamoDB Vector Bottleneck

To satisfy enterprise standards, the initial implementation was built entirely around managed AWS serverless infrastructure. When an end user submits a request, traffic is routed through Amazon Route 53 for DNS resolution and handled by a content delivery network (CloudFront/CDN) that serves the compiled React front-end assets stored in an Amazon S3 bucket. Authentication is enforced through Amazon Cognito; once authenticated, the client attaches JSON Web Tokens (JWT) to request headers, which are validated downstream at the Amazon API Gateway.

The API Gateway provides throttling and rate limiting, dispatching validated requests to specific AWS Lambda functions mapped to individual endpoints—such as listing library content, uploading raw source files or books, and handling user search queries. During document ingestion, Lambda splits the source text into distinct chunks, submits them to Amazon Bedrock to generate vector embeddings, and writes those embeddings directly into Amazon DynamoDB.

``` [User] ---> [Route 53] ---> [CDN / CloudFront] ---> [S3: Web UI] | (JWT via Cognito) v [API Gateway] | +-----------------+-----------------+ | (List / Query / Upload Endpoints) | v v [Lambda Functions] <===============> [AWS Bedrock] | (Embeddings / LLM) v [DynamoDB Storage] (Overhead: Cosine similarity & sorting computed in Lambda) ```

While DynamoDB provided high availability, automated scaling, and encryption at rest and in transit, it quickly exposed a critical architectural flaw. Because DynamoDB is a traditional NoSQL database lacking native vector indexing and vector similarity search primitives, the compute-heavy operations—namely calculating cosine similarity and sorting/ranking candidates against query vectors—had to be processed entirely inside the Lambda functions.

- **DNS & Asset Delivery**: Amazon Route 53 routes inbound web traffic; CloudFront CDN serves cached React UI assets stored in S3. - **Identity & Ingress**: Amazon Cognito issues JWTs for authentication, which are validated by Amazon API Gateway before triggering backend compute. - **Microservices via Lambda**: Granular Lambda functions govern distinct operations such as listing books, file uploads, and Q&A inference. - **Compute Bottleneck**: Storing high-dimensional vectors in DynamoDB forced Lambda functions to pull candidates and run cosine similarity calculations in memory, degrading query performance.

---

[07:10](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=430s) Core Serverless Building Blocks and Early Design Decisions

Designing an enterprise system as a solo developer demands rigorous architectural choices early on to avoid runaway infrastructure maintenance and unmanageable operational overhead. Choosing AWS Bedrock provided access to foundation models for both natural language response generation and embedding synthesis, packaged as an auto-scaling, pay-per-use managed service with multi-nine availability. This allowed the platform to leverage specialized, distinct models optimized separately for semantic vector embedding generation versus conversational question answering without managing underlying GPU clusters.

Similarly, pairing Amazon S3 with CloudFront created a dual-purpose storage foundation. S3 served as the repository for raw document uploads and codebase archives, while also acting as the origin server for front-end static files (HTML, JavaScript, and CSS). CloudFront cached these UI assets at global edge locations to ensure low latency for global users, backed by S3's near-unlimited storage tier and built-in security controls.

At the compute layer, the combination of Amazon API Gateway and AWS Lambda provided an elastic backend with pay-per-use billing that completely eliminated idle server costs. Each operational path—listing items, document querying, and ingestion uploads—mapped directly to an isolated Lambda handler. API Gateway handled JWT verification against Cognito natively alongside request throttling and rate limiting, ensuring the backend remained protected against traffic spikes while maintaining strict access boundaries across endpoints.

- **AWS Bedrock**: Fully managed LLM infrastructure used for both vector embedding generation and Q&A text generation, chosen for its serverless pay-per-use billing and high availability. - **S3 & CloudFront Pairing**: S3 serves as an origin for UI code and long-term object storage for raw source documents, fronted by edge caching for rapid delivery. - **Compute Isolation**: Individual Lambda functions scale independently per endpoint without incurring idle infrastructure expenses. - **Built-in Governance**: API Gateway manages rate limits, burst throttling, and automatic JWT authorization checks before dispatching calls to Lambda handlers.

---

[09:56](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=596s) Diagnosing the 3-Second API Latency and the Need for Vector Databases

Once the baseline implementation was running, CloudWatch metrics exposed a major latency issue: API responses were taking nearly 3 full seconds. Profiling traces pinpointed AWS Lambda as the primary bottleneck. Because DynamoDB stored the vector arrays merely as raw attributes, the Lambda compute layer had to retrieve the records, parse the numerical arrays, execute cosine similarity math across every record, and sort the ranked results before constructing the model context.

This performance wall prompted an evaluation of dedicated vector databases. Standard relational and NoSQL databases lack indexing strategies capable of executing approximate nearest neighbor (ANN) searches across millions of high-dimensional vectors at sub-second speeds. Vector databases solve this by storing data within specialized vector spaces and utilizing purpose-built indexing algorithms that navigate dense numerical representations in milliseconds.

Beyond raw speed, vector databases offer native support for operations essential to enterprise retrieval-augmented generation (RAG) pipelines, including metadata filtering and hybrid search (combining sparse keyword search with dense semantic retrieval). These capabilities are crucial across diverse domains, including chatbots, recommendation engines, media retrieval, and anomaly detection, where systems must quickly filter and retrieve the most relevant semantic context without burdening downstream compute instances with manual ranking algorithms.

- **Latency Baseline**: Early CloudWatch telemetry showed query response times hovering around 3 seconds due to Lambda compute overhead. - **Algorithmic Limitations of NoSQL**: Standard database indexes cannot natively execute vector distance metrics (such as cosine similarity or dot product) at scale. - **Role of Vector Databases**: Purpose-built vector engines store embeddings in indexed vector spaces to enable sub-second nearest-neighbor lookups, metadata filtering, and hybrid search.

---

[12:45](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=765s) Migrating from DynamoDB to S3 Vectors

To resolve the compute bottleneck without introducing complex third-party infrastructure, the speaker turned to Amazon S3 Vectors—a feature newly introduced on July 15, 2025. Although not yet rolled out to every global region at the time of evaluation, it was accessible in the `us-east-1` (US East) region. Adopting S3 Vectors preserved native AWS integration, enabling infrastructure-as-code management via AWS CloudFormation and unified monitoring through Amazon CloudWatch without provisioning external clusters.

S3 Vectors provided significant operational and architectural advantages. Because vectors are stored alongside their raw chunk text and metadata directly within the database engine, an ANN similarity query returns the matched text chunks directly in the search response payload. This eliminated an entire secondary round-trip query to fetch the raw text from secondary storage. Passing these attribution chunks directly to the final API response introduced clear explainability, allowing developers and end users to inspect the exact source chunk used to ground the LLM's answer.

The architectural migration from DynamoDB to S3 Vectors highlighted the advantages of modular serverless design. The migration required no architectural overhaul: the system boundaries remained intact, and changes were confined to the query Lambda functions and IAM execution roles to grant access to the S3 Vectors API. Shifting the vector similarity math inside the managed database immediately unlocked substantial performance gains and simplified backend compute logic.

``` +------------------------------------------------------+ | Revised Architecture: S3 Vectors | +------------------------------------------------------+

[Client App] ---> [Route 53] ---> [CloudFront CDN] | v [Amazon Cognito] <-----------> [API Gateway] | v [AWS Lambda] / \ (Generate Emb) (Query Vectors + Chunks) / \ v v [AWS Bedrock] [S3 Vectors] - Embedding Model - Native Vector Search - Q&A LLM - Bundled Text & Metadata ```

- **Feature Release Date**: Amazon S3 Vectors launched on July 15, 2025, and was evaluated in the `us-east-1` region. - **Native AWS Ecosystem**: Deployed cleanly using AWS CloudFormation and monitored out of the box using CloudWatch metrics. - **Embedded Chunk Metadata**: S3 Vectors returns matched source text chunks alongside vector similarity scores, eliminating secondary data retrieval queries. - **Frictionless Migration**: Swapping out DynamoDB for S3 Vectors required only localized updates to Lambda handlers and IAM role policies, showcasing modular system design.

---

[15:38](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=938s) System Telemetry, Evals, and Mapping Enterprise Design Principles

Following the migration to S3 Vectors, CloudWatch metrics demonstrated sweeping improvements: API latency dropped sharply, Lambda execution durations were slashed, error rates fell, and overall throughput climbed significantly. Concurrently, offline evals showed marked improvements in end-to-end response accuracy and retrieval precision, confirming that native vector operations delivered higher-quality context to the underlying Bedrock language models.

To evaluate the broader maturity of the build, the speaker mapped foundational enterprise system design principles to concrete technologies and business impacts across the stack. Reliability was anchored by managed auto-retries in Lambda and continuous redundancy in S3. Security followed the principle of least privilege using granular IAM execution roles for Bedrock and S3 access, alongside end-to-end encryption for data at rest and in transit.

Operational observability was achieved by combining automated CloudWatch dashboards with LangSmith (or Langfuse/LangChain tracing) to trace granular model execution paths and monitor system health. On the evaluation side, TensorBoard was utilized alongside custom eval scripts to visualize performance trends and measure answer accuracy across continuous development cycles.

- **CloudWatch Telemetry**: Tracked Lambda runtimes, invocation error rates, request throughput, and latency drops after vector migration. - **Observability Stack**: Integrated tracing to capture LLM chains alongside automated CloudWatch dashboards for DynamoDB, Lambda, and S3. - **Evaluation Tooling**: Used TensorBoard to graph accuracy trends and evaluate retrieval performance across iterations. - **Infrastructure as Code**: Managed complete environment provisioning through AWS CloudFormation and automated updates via the AWS CLI.

---

[18:50](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=1130s) AI Risk Mitigation, Sub-Second Performance, and AI-Assisted Tooling

Operating enterprise AI systems requires proactively mitigating specific generative AI risks. To eliminate hallucinations, the application strictly adheres to the RAG pattern, grounding every generation within retrieved context and injecting the source chunk metadata into the final API payload for end-to-end source attribution. Semantic drift and context gaps are resolved through vector similarity matching inside S3 Vectors. To protect user privacy and corporate data, network paths are secured via VPC isolation, least-privilege IAM controls, and multi-layered encryption at rest and in transit.

Bias is minimized by maintaining strict document boundary isolation, ensuring searches pull only from the specific book or codebase explicitly selected by the user. On the safety front, custom content filtering implemented across AWS Bedrock and Lambda filters out profanity, inappropriate language, and sensitive data leaks. Optimizing these layers brought end-to-end query response times down from the initial 3-second crawl to roughly 150 milliseconds. When audited against the AWS Well-Architected Framework whitepaper, the architecture earned a rigorous score of 54 out of 60.

Reflecting on velocity, the speaker credited an AI-assisted development workflow powered by tools like Cursor, Eraser.io (for architectural diagrams), and Google Gemini. Cursor's inline editing, agentic workflows, and codebase indexing allowed a solo engineer to work at the pace of an entire team—building an enterprise-grade, highly secure AI platform for "less than a cup of coffee a day" that scales to thousands of concurrent users in seconds. Just as the speaker begins outlining final takeaways on clearly defining the problem space, the session moderator steps in to close the presentation for the next incoming speaker.

- **150ms Query Latency**: Native vector operations and optimized serverless compute reduced total query execution time from ~3,000ms to 150ms. - **Hallucination & Bias Safeguards**: RAG architecture forces grounded responses with full chunk attribution, while strict document-level isolation prevents cross-tenant data leakage. - **Enterprise Safety**: Bedrock content filters and Lambda validation guardrails actively screen out profanity, adult language, and sensitive data. - **AWS Well-Architected Review**: Scored 54 out of 60 against AWS Well-Architected Framework best practices. - **Solo Developer Toolkit**: Leveraged Cursor, Eraser.io, and Google Gemini to dramatically accelerate architecture, coding, and debugging cycles. - **Session Conclusion**: "I'm so sorry to have to cut you off. Unfortunately we have another session starting at 11:30. So let's give her a round of applause."

## Notable quotes > [00:04](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=4s) Can a solo developer build enterprise-grade AI systems? — Soujanya Vullam

> [00:04](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=4s) Observability allows engineers to inspect the internal state of an application via generated telemetry—logs, metrics, and distributed traces—to understand how the large language model makes its decisions, evaluate performance trends over time, and ensure stringent compliance with security protocols. — Soujanya Vullam

> [03:21](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=201s) Because DynamoDB is a traditional NoSQL database lacking native vector indexing and vector similarity search primitives, the compute-heavy operations—namely calculating cosine similarity and sorting/ranking candidates against query vectors—had to be processed entirely inside the Lambda functions. — Soujanya Vullam

> [07:10](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=430s) Designing an enterprise system as a solo developer demands rigorous architectural choices early on to avoid runaway infrastructure maintenance and unmanageable operational overhead. — Soujanya Vullam

> [07:10](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=430s) Choosing AWS Bedrock provided access to foundation models for both natural language response generation and embedding synthesis, packaged as an auto-scaling, pay-per-use managed service with multi-nine availability. — Soujanya Vullam

> [09:56](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=596s) Because DynamoDB stored the vector arrays merely as raw attributes, the Lambda compute layer had to retrieve the records, parse the numerical arrays, execute cosine similarity math across every record, and sort the ranked results before constructing the model context. — Soujanya Vullam

> [09:56](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=596s) Standard relational and NoSQL databases lack indexing strategies capable of executing approximate nearest neighbor (ANN) searches across millions of high-dimensional vectors at sub-second speeds. — Soujanya Vullam

> [12:45](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=765s) Because vectors are stored alongside their raw chunk text and metadata directly within the database engine, an ANN similarity query returns the matched text chunks directly in the search response payload. — Soujanya Vullam

> [12:45](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=765s) The migration required no architectural overhaul: the system boundaries remained intact, and changes were confined to the query Lambda functions and IAM execution roles to grant access to the S3 Vectors API. — Soujanya Vullam

> [18:50](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=1130s) To eliminate hallucinations, the application strictly adheres to the RAG pattern, grounding every generation within retrieved context and injecting the source chunk metadata into the final API payload for end-to-end source attribution. — Soujanya Vullam

> [18:50](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=1130s) less than a cup of coffee a day — Soujanya Vullam

> [18:50](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=1130s) I'm so sorry to have to cut you off. Unfortunately we have another session starting at 11:30. So let's give her a round of applause. — Session Moderator

## Mentioned Soujanya Vullam, Yahoo, Ai4, Epic Q&A, AWS, AWS Bedrock, Amazon Route 53, Amazon CloudFront, Amazon S3, Amazon Cognito, Amazon API Gateway, AWS Lambda, Amazon DynamoDB, Amazon S3 Vectors, AWS CloudFormation, Amazon CloudWatch, AWS Well-Architected Framework, LangSmith, Langfuse, LangChain, TensorBoard, Cursor, Eraser.io, Google Gemini, Perplexity, ChatGPT, React

## Video description Soujanya Vullam, Software Development Engineer II, Yahoo

Abstracts

Can a solo developer build enterprise-grade AI systems? In this session, I’ll share how I independently designed and deployed a secure, scalable AI Q&A platform using LLMs, AWS Bedrock, and AWS serverless architecture. The talk will illustrate real-world techniques to accelerate developer velocity, reduce infrastructure costs through serverless system design, and co-develop with AI tools like Cursor, Eraser.io, Perplexity, and ChatGPT. You’ll walk away with actionable strategies for turning lightweight prototypes into production-ready applications that scale. Join me to explore what’s possible when generative AI and cloud-native architecture come together.

[Source](https://www.youtube.com/watch?v=vv3ydGd9Anc)

中文简报

雅虎工程师改用 AWS S3 Vectors 将 AI 延迟降至 150 毫秒将向量相似度搜索从 DynamoDB 迁移至原生 Amazon S3 Vectors，使 AI 查询延迟从 3 秒缩短至 150 毫秒，消除了无服务器架构中的后端计算瓶颈。本次演讲记录了由单名工程师构建的企业级文档与代码问答平台“Epic Q&A”的端到端设计过程。最初的无服务器架构将向量嵌入存储在 Amazon DynamoDB 中，但由于需要在 AWS Lambda 函数内部计算余弦相似度，导致了长达 3 秒的查询延迟瓶颈。迁移至新发布的 Amazon S3 Vectors 解决了这一计算开销，不仅实现了具备分块归因绑定的亚秒级 ANN 相似度搜索，还将响应时间大幅降至 150 毫秒。该演讲展示了结合 AWS 托管服务、结构化评估以及 AI 辅助开发工具，如何让单名工程师也能构建出符合严格企业设计与安全标准的高性价比系统。构建企业级 AI 需要扩展传统系统设计原则，纳入可观测性以及用于衡量模型准确性、相关性和连贯性的持续评估（evals）。 (0:04) 在 DynamoDB 中存储高维嵌入会引入严重的延迟瓶颈，因为 Lambda 计算实例必须拉取原始数组并在内存中执行余弦相似度数学计算。 (3:21) AWS Bedrock 提供了完全托管的无服务器接口，以按量计费模式将嵌入生成与 LLM 文本生成解耦，无需预置 GPU 集群。 (7:10) 将 Amazon S3 与 CloudFront 结合使用，可作为兼顾静态 React Web 资产分发与长期原始文档存储的高性价比双重基础。 (7:10) 标准关系型和 NoSQL 数据库缺乏原生索引算法，无法在高维向量空间中执行亚秒级的近似最近邻（ANN）搜索。 (9:56)

Reading summary

A solo developer can construct a production-ready, enterprise-grade AI platform by pairing serverless cloud primitives with managed foundation models and dedicated vector databases. According to the presentation, building the Epic Q&A application on AWS demonstrated that modular serverless design and continuous AI evaluations allow an individual engineer to achieve low-latency, scalable retrieval-augmented generation without maintaining dedicated GPU clusters or servers. Initial Serverless Ingress and DynamoDB Bottlenecks (3:21) The Epic Q&A application was initially architected entirely on managed AWS serverless infrastructure to process voluminous books and codebases into context-aware answers. User traffic reaches React front-end assets in Amazon S3 via Route 53 and CloudFront, while Amazon Cognito provides JWT authentication validated downstream by Amazon API Gateway. Validated requests trigger isolated AWS Lambda functions for distinct tasks such as document ingestion and querying, interacting with AWS Bedrock to synthesize vector embeddings and conversational text. This modular, pay-per-use design eliminated idle server expenses and leveraged auto-scaling. However, storing high-dimensional embeddings in Amazon DynamoDB exposed a major architectural bottleneck. Because DynamoDB lacks native approximate nearest neighbor vector indexing, Lambda functions were forced to pull raw candidate records, parse numerical arrays, calculate cosine similarity in memory, and sort ranked results before passing context to Bedrock. CloudWatch metrics revealed that this compute overhead drove end-to-end API response times up to nearly three full seconds. Eliminating Compute Walls with S3 Vectors (12:45) To bypass the three-second latency bottleneck without provisioning complex external database clusters, the speaker adopted Amazon S3 Vectors, which launched on July 15, 2025, and was evaluated in the us-east-1 region. S3 Vectors executes native similarity searches and bundles raw text chunks directly alongside vector scores in the query response. This capability eliminated the secondary database query previously required to fetch source text, injecting verifiable attribution chunks directly into the API response payload for answer explainability. Migrating away from DynamoDB required only localized changes to the query Lambda functions and IAM execution roles, validating the benefits of modular serverless microservices. Offloading vector similarity math directly to the managed database reduced end-to-end query times from roughly 3,000 milliseconds down to approximately 150 milliseconds, while enabling CloudFormation deployment and native CloudWatch monitoring. Observability, Telemetry, and Evaluation Standards (15:38) The project emphasized that enterprise AI architectures require two foundational pillars beyond standard design: observability and evaluations (evals). Operational observability combined automated CloudWatch dashboards with LangSmith (or Langfuse/LangChain tracing) to trace granular model execution paths and monitor system health. For evaluations, the developer replaced conventional unit tests with targeted benchmarks measuring accuracy, relevance, coherence, completeness, and helpfulness, visualizing iterative accuracy trends using TensorBoard. The speaker mapped these components across ten enterprise design principles: modularity (Lambda), scalability (serverless primitives), reliability (managed auto-retries and S3 redundancy), security (least-privilege IAM and transit/rest encryption), flexibility (Bedrock model selection), maintainability (CloudFormation IaC), testability (isolated unit tests), reusability (Lambda Layers), observability (CloudWatch and LangSmith), and evaluation (TensorBoard). This systematic architecture scored 54 out of 60 against the AWS Well-Architected Framework. Risk Mitigation and Developer Workflow Acceleration (18:50) To satisfy enterprise safety and compliance requirements, the architecture implements proactive guardrails against typical generative AI risks. Grounding answers within retrieved S3 Vector chunks mitigates hallucinations, while strict document boundary isolation restricts queries to the user's selected codebase or book to prevent cross-tenant data leaks and minimize bias. Additional safety layers across Bedrock and Lambda actively filter profanity, inappropriate language, and sensitive information, supported by VPC network isolation and multi-layer encryption. The speaker asserted that an individual engineer can sustain an enterprise pace using AI-assisted tooling such as Cursor for codebase indexing and agentic workflows, Eraser.io for architectural diagrams, and Google Gemini. The presenter claimed these tools allowed the system to be developed and operated for 'less than a cup of coffee a day' while scaling to thousands of concurrent users in seconds. The presentation concluded abruptly when the session moderator intervened to vacate the stage for an incoming speaker, cutting off the presenter's final remarks regarding problem-space definition.

精炼阅读摘要

演讲者通过独立开发知识问答应用 Epic Q&A 证明，单人开发者借助 AWS 全托管无服务器架构与现代向量检索技术，同样能够构建出兼具高安全性、毫秒级延迟与可观测性的企业级生成式 AI 系统。单人构建企业级 AI 的架构原则与目标 (0:04) 演讲者提出，单人开发者若想跨越概念验证（PoC）阶段、交付真正的企业级系统，必须严格遵循模块化、可扩展性、安全性与可靠性等传统设计原则。针对现代生成式 AI 架构，系统还需要额外引入可观测性（Observability）与持续评估（Evals）两大支柱。通过收集日志、性能指标和分布式链路追踪，团队可以清晰洞察大语言模型的决策轨迹，确保系统合规；而通过系统性衡量准确率、相关性、连贯性、完整度与可用性等评估维度，开发者能够精确定位性能瓶颈，摆脱低效的传统测试模式。在这一理念下孵化的 Epic Q&A 平台，旨在对体量庞大的代码仓库和长篇书籍进行文本切分、自动生成文档，并依据语义上下文回答自然语言查询。演讲者指出，这种兼具海量知识解析与精准语义检索的复杂场景，对单人开发者的架构精简度与基础设施运维成本提出了极高要求。早期 DynamoDB 向量方案导致的算力瓶颈 (3:21) 为了压降闲置服务器成本并满足企业级运维标准，该应用最初完全依赖 AWS 托管无服务器套件。用户流量经由 Amazon Route 53 解析，通过 CloudFront CDN 获取存储在 Amazon S3 上的 React 前端静态资源；用户身份由 Amazon Cognito 生成 JWT 并在 Amazon API Gateway 处完成校验与限流，进而将请求路由至独立的 AWS Lambda 函数。在数据摄取阶段，Lambda 将文本切块提交给 Amazon Bedrock 生成向量嵌入，并存入 Amazon DynamoDB。然而，早期的 CloudWatch 监控指标暴露了严重的延迟缺陷：一次 API 查询竟耗时接近 3 秒。链路追踪排查显示，DynamoDB 属于传统 NoSQL 数据库，缺乏原生的向量索引与近似最近邻（ANN）检索能力。系统不得不依靠 Lambda 从数据库拉取全部向量数组并在内存中逐一执行余弦相似度计算与排序，这一重型计算瞬间打满了 Lambda 的运行开销，直接演变为阻碍系统落地的核心瓶颈。迁移至 S3 Vectors 达成毫秒级延迟突破 (12:45) 为从根本上解决算力瓶颈，演讲者对专用向量数据库的作用进行了评估。与传统关系型或 NoSQL 数据库相比，向量数据库能够在高维向量空间内实现亚秒级的近似最近邻搜索，并原生支持元数据过滤和混合检索（结合关键词与语义匹配）。演讲者并未引入复杂的三方自建集群，而是转向评估了 2025 年 7 月 15 日新推出的 Amazon S3 Vectors 功能（在 us-east-1 区域进行验证）。得益于无服务器架构的高模块化设计，系统仅修改了查询相关的 Lambda 函数与 IAM 权限策略，便通过 AWS CloudFormation 顺畅完成了迁移。S3 Vectors 支持在数据库引擎内部直接完成向量相似度检索，并将向量关联的原始文本切块及元数据一同打包在响应中返回，消除了二次回表查询文本的开销。架构优化后，整体端到端查询耗时从最初的约 3,000 毫秒锐降至 150 毫秒，Lambda 执行耗时与调用错误率大幅下降，系统吞吐量显著提升。全面风控策略与 AI 辅助开发提效 (18:50) 在企业风险防范层面，该系统严格执行 RAG 检索增强生成模式，强制模型仅依据检索出的上下文作答，并将命中的文本切块元数据直接注入 API 最终响应，从而保障内容可解释性并遏制幻觉。针对语义漂移、租户隔离与数据泄露隐患，系统依靠 VPC 网络隔离、最低权限 IAM 访问控制、传输及静态数据端到端加密，并限制检索范围仅作用于用户指定的单一书籍或代码库。此外，AWS Bedrock 与 Lambda 层内嵌了定制内容过滤规则，用于拦截不当言论与敏感数据泄漏。经评估，该系统在 AWS 良好架构框架（Well-Architected Framework）白皮书审查中获得了 54/60 的高分。在开发效率方面，演讲者表示依靠 Cursor、Eraser.io 和 Google Gemini 等现代 AI 辅助研发工具，单人即可完成全套架构设计、代码编写与调试。该系统的日常运行开销被控制在“低于每天一杯咖啡的费用”，且具备数秒内横向扩展以承载数千并发用户的弹性。该系统证明了单人借助无服务器与托管向量工具构建低成本、高并发企业级系统的可行性。不过，Amazon S3 Vectors 在演讲时仍受限于特定区域（仅限 us-east-1 等），全球范围的多地域支持与实际落地稳定性仍有待进一步验证；此外，由于现场议程限制，关于问题空间定义的总结讨论未及展开便被主持人中断。

Detailed notes — zh

## Background 本内容由频道 Ai4 于 2025 年 8 月 28 日发布在 YouTube 平台上。视频记录了来自雅虎（Yahoo）的软件工程师 Soujanya Vullam 的技术演讲，主要探讨独立开发者如何利用大语言模型（LLMs）、AWS Bedrock 和无服务器（Serverless）架构，独立设计并部署一套安全、可扩展的企业级 AI 问答平台。 ## TL;DR 演讲围绕名为 Epic Q&A 的企业级智能问答系统展开，系统化复盘了一名独立开发者从原型构建到生产落地的全过程。针对早期由于在 Lambda 中计算向量余弦相似度导致的性能瓶颈，系统重构并迁移至 AWS 原生 S3 向量存储方案，成功将查询延迟优化至 150 毫秒并显著提升了答案可解释性。作者结合 AWS 卓越架构框架（Well-Architected Framework），详细阐述了在无服务器体系下实现模块化、安全性、可观测性（Observability）与评测（Evals）的具体路径，并分享了如何通过 RAG、内容过滤及权限隔离应对幻觉与数据泄露等核心 AI 风险。最后，作者展示了如何联动 Cursor 等 AI 编程工具实现极高研发效能，以极低运行成本交付高并发、高弹性的生产级应用。 ## Atmosphere & style 整场演讲呈现出极为务实、精炼且充满极客底色的技术分享氛围。主讲人以清晰利落的工程视角抽丝剥茧，毫无浮夸辞藻，而是通过指标监控数据、架构演进对比与踩坑经验娓娓道来，给观众带来极其扎实且可落地的实战代入感。伴随着“一人成军”并以一杯咖啡的日成本撬动企业级架构的生动事实，整个演讲洋溢着现代 AI 开发者掌控全栈生产力的自信与高效节奏，最后在现场热烈而意犹未尽的掌声中圆满收尾。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=4s) 独立开发者构建企业级 AI 系统的关键在于超越概念验证（Prototype），同时达成高可扩展性、模块化、强安全性与生产就绪状态。 - [00:04](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=4s) AI 系统与传统系统的核心差异在于可观测性（Observability）与评估体系（Evals），需建立专门指标衡量模型回答的准确性、相关性与连贯性。 - [02:50](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=170s) Epic Q&A 初始架构采用全套 AWS Serverless 技术栈，通过 Route 53、Cognito 鉴权、API Gateway 与微服务化 Lambda 驱动业务流转。 - [02:50](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=170s) 早期在 Lambda 内存中对 DynamoDB 提取的向量做余弦相似度计算，导致了严重的算力瓶颈与接近 3 秒的高接口延迟。 - [07:08](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=428s) AWS Bedrock、S3 与 Lambda 等无服务器选型具备极高的可用性与弹性伸缩能力，同时按量计费实现零闲置成本。 - [10:00](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=600s) 引入云原生 S3 Vectors 向量数据库，支持毫秒级近似最近邻（ANN）检索、元数据过滤及基础设施即代码（IaC）部署。 - [14:00](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=840s) 从 DynamoDB 迁移至 S3 向量存储实现了单次检索同时返回切片文本与向量，将计算下沉至 AWS 原生环境并大幅提升 API 响应性能。 - [14:00](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=840s) 在 API 响应中直接附带文档切片溯源数据，有效解决了大模型黑盒输出问题，赋予系统企业级的可解释性（Explainability）。 - [15:35](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=935s) 依托 CloudWatch 与链路追踪工具建立全方位评估指标，系统在迁移后吞吐量成倍提升且延迟与错误率显著下降。 - [16:33](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=993s) 企业级架构原则在无服务器设计中全面落地，通过 IAM 最小权限、端到端加密及 Lambda Layers 达成高内聚低耦合与安全合规。 - [18:48](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=1128s) 构建纵深防御安全体系，结合 RAG 事实接地、上下文物理隔离与 Bedrock 护栏内容过滤，端到端查询耗时优化至 150 毫秒。 - [18:48](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=1128s) 系统在《AWS 卓越架构框架》自评中斩获 54/60 高分，证明了 Serverless AI 架构在企业级合规上的可行性。 - [21:07](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=1267s) 借助 Cursor、Eraser.io 等 AI 辅助工具，单个开发者即可实现秒级千并发弹性伸缩且日均运行成本不足一杯咖啡的高效企业级系统。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=4s) 独立开发者构建企业级 AI 系统与核心评估维度来自雅虎（Yahoo）的软件工程师 Soujanya Vullam 开门见山地提出了一个驱动她数月探索的核心问题：“单个开发者能否独立构建出企业级的 AI 系统？”在她看来，许多 AI 项目往往止步于概念验证（Prototype），而要跨越到真正的“企业级”，系统必须同时具备高可扩展性（Scalability）、模块化（Modularity）、强安全性（Security）以及真正的生产就绪（Production-ready）状态。为此，她独立开发了一款名为 **Epic Q&A** 的 Web 应用，该系统能够解析大型书籍以及复杂代码库并生成技术文档，进而为用户提供具备上下文感知能力（Context-aware）的智能问答服务。在系统架构设计原则中，Soujanya 特别强调了 AI 系统与传统系统最显著的区别在于**可观测性（Observability）**与**评估体系（Evals）**： - **可观测性（Observability）**：通过分析日志（Logs）、指标（Metrics）和调用链路追踪（Traces）来理解系统内部状态，精准洞察大语言模型（LLM）是如何做出决策的，监控其长期性能表现，并严格满足企业安全合规要求。 - **评估体系（Evals）**：不同于传统的单元测试或集成测试，AI 评估是一套专门衡量模型输出质量的框架。它依据具体的指标和规范，系统化检验答案的**准确性（Accuracy）**、**相关性（Relevance）**、**连贯性（Coherence）**、**完整性（Completeness）**以及**实用性（Helpfulness）**，从而保障系统可靠性，指导开发生命周期中的持续迭代。 ```mermaid flowchart TD accTitle: Epic Q&A 初始无服务器架构流程图 accDescription: 展示用户从前端通过 Route 53、CDN、Cognito、API Gateway 到 Lambda、Bedrock 及 DynamoDB 的交互链路。 subgraph ClientAndEdge["接入与边缘分发层"] U["用户客户端"] --> R53["AWS Route 53 (DNS)"] R53 --> CDN["CloudFront / CDN (静态资源缓存)"] U -.-> Auth["AWS Cognito (身份验证/JWT)"] end subgraph ApiAndCompute["网关与计算服务层"] CDN --> APIGW["API Gateway (JWT鉴权/限流/路由)"] APIGW --> L_List["Lambda: 资源列表"] APIGW --> L_Upload["Lambda: 上传与切块"] APIGW --> L_QA["Lambda: 问答与相似度计算"] end subgraph StorageAndAI["存储与模型服务层"] L_Upload --> S3["Amazon S3 (原始文件存储)"] L_Upload & L_QA --> Bedrock["AWS Bedrock (Embedding / LLM 生成)"] L_Upload --> DDB[("DynamoDB (存储向量与元数据)")] DDB --> L_QA end ``` --- ### [02:50](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=170s) Epic Q&A 初始架构设计与数据流转 Soujanya 详细拆解了 Epic Q&A 的端到端请求流转链路。系统完全基于 AWS 现代化无服务器（Serverless）云原生组件搭建： 1. **网络路由与前端分发**：用户的网络请求首先经过 AWS Route 53 进行 DNS 解析，静态 React 前端资源托管在 Amazon S3 上并通过 CDN（CloudFront）进行全球边缘缓存与加速分发。 2. **身份认证与鉴权**：用户登录时通过 AWS Cognito 完成认证，前端将签发的 JWT 令牌附加到每个 HTTP 请求头中；API Gateway 负责校验 JWT 的有效性，并提供请求限流（Rate Limiting）和防抖（Throttling）保护。 3. **微服务化 API 分发**：API Gateway 根据请求路径路由到专属的 AWS Lambda 函数（如列出文件 `List`、上传文件/代码 `Upload`、发起问答 `Query` 等）。 4. **文本切块与嵌入**：上传的代码或书籍文本在 Lambda 中被切分成语义块（Chunks），随后调用 AWS Bedrock 生成向量嵌入（Embeddings）。 5. **初始存储与检索计算**：在第一版架构中，这些向量嵌入直接存入 DynamoDB。然而，由于 DynamoDB 本质上是非向量数据库，**余弦相似度（Cosine Similarity）匹配与候选排序操作全部被迫在 Lambda 内存中执行**，这也为后续的性能优化埋下了伏笔。 --- ### [07:08](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=428s) 独立开发者的技术选型考量与无服务器优势作为一名独立开发者，架构选型的容错率极低，必须兼顾高可用性、极低运维负担与成本控制： - **AWS Bedrock**：提供 99.999%（Five Nines）的高可用性与自动弹性伸缩，采用按量计费（Pay-per-use）模式，零闲置成本。更关键的是支持按需解耦模型选型——文本向量生成与最终问答推理分别调用两款最适配的专属基础模型。 - **Amazon S3 + CDN**：S3 提供近乎无限且高持久度的存储能力，既用来保存用户上传的大型原始文档/代码，也承载 React 构建生成的 HTML/JS 静态资源；配合 CDN 边缘缓存实现极速本地化访问。 - **Amazon API Gateway + AWS Lambda**：具备 99.95% 的可用性与自动伸缩特性，天然集成 Cognito JWT 校验，无需维护服务器集群即可根据访问负载自动伸缩，实现极致的成本效益。 - **DynamoDB**：初期选用其作为主存，看重其开箱即用的静态与传输加密、高可用性以及无缝自动扩展能力。 --- ### [10:00](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=600s) 性能瓶颈与向量数据库演进：引入 S3 Vectors 在第一版架构上线后，Soujanya 借助 Amazon CloudWatch 监控指标对系统的延迟（Latency）、吞吐量（Throughput）与接口响应速度进行了全方位评测。数据暴露了严峻的性能瓶颈：**API 平均响应时间高达近 3 秒**。瓶颈的根源正是 Lambda——在无向量索引支持的普通数据库架构下，每次检索都需将全量向量读入 Lambda 并在函数内逐一计算余弦相似度和排序。为了化解算力瓶颈，必须引入专用的向量数据库（Vector Database）。向量数据库将高维数组形式的语义特征进行专业索引（如近似最近邻 ANN 算法），支持在毫秒级内对海量数据执行向量检索、元数据过滤（Metadata Filtering）与混合检索（Hybrid Search，即语义检索与关键词检索结合）。经过调研，Soujanya 锁定了于 2025 年 7 月 15 日全新发布的 **S3 Vectors** 技术，并在支持该特性的 `us-east` 区域进行了深度评估： - **AWS 原生集成**：支持通过 CloudFormation 进行基础设施即代码（IaC）部署，且指标天然接入 CloudWatch，极大降低了运维复杂度。 - **成本与扩展性**：继承了 S3 的按量付费和无上限存储特性，避免了常驻向量数据库集群的高昂闲置开销。 - **元数据协同存储**：文本切块与元数据直接同向量保存在一起，单次查询即可同时完成相似度检索与上下文元数据拉取，彻底解放了 Lambda 计算瓶颈。 ### [14:00](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=840s) 从 DynamoDB 迁移至 S3 向量存储与可解释性提升在谈及检索架构的迭代时，讲师详细说明了用 S3 向量存储方案替代原有 DynamoDB 方案的技术动机与具体收益。在原有的实现中，向量运算和文本块（Chunks）的获取存在割裂，可能需要进行二次查询；而在 S3 向量方案中，系统在一次查询中即可同时返回与用户 Query 最相关的切片文本。这种设计避免了多余的网络往返，不仅降低了系统开销，更直接将切片数据顺畅透传至最终的 API 响应中。 ```mermaid flowchart LR accTitle: RAG Architecture Evolution subgraph Old["旧架构"] O1["API 请求"] --> O2["Lambda 函数"] O2 --> O3["DynamoDB (外部向量计算)"] O3 --> O2 end subgraph New["新架构"] N1["API 请求"] --> N2["Lambda 函数"] N2 --> N3["S3 向量存储 (AWS 原生向量计算)"] N3 -->|返回 Chunks 与嵌入| N2 N2 -->|携带溯源上下文| N4["API 响应 (高可解释性)"] end ``` 这种将文本切片直接封装到 API 响应中的机制，为系统带来了至关重要的“可解释性（Explainability）”。无论对内部开发者还是终端用户而言，系统都能清晰证明“当前答案是基于哪些具体文档切片生成的”，彻底告别了黑盒问答。此外，性能得到了“极其可观的飞跃”，主要原因在于向量相似度计算被直接放在 AWS 原生环境内部执行，而非在数据库外部处理。得益于无服务器架构的高内聚低耦合特性，这次底层存储的架构重构仅仅修改了少部分 Lambda 代码和对应的 IAM 执行角色权限，就实现了“性能成倍提升”，充分印证了系统良好的模块化与灵活性。 - **单次检索返回文本切片**：一次查询直接拉取相关上下文文本，免去二次查询开销。 - **可解释性贯穿链路**：API 响应直接附加事实依据切片，增强企业级输出的可信度。 - **计算下沉至云原生底座**：利用 AWS 原生向量能力替代外部处理，大幅缩减运算延迟与传输成本。 - **极低改造成本**：仅需微调 Lambda 业务逻辑与 IAM 权限策略，便完成核心存储与算力引擎的平滑替换。 --- ### [15:35](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=935s) 系统可观测性评估与关键性能指标对比任何生产级 AI 系统的落地，都离不开严密的可观测性（Observability）与评测（Eval）体系。在数据从 DynamoDB 全面迁移至 S3 向量存储后，讲师利用 AWS 原生的 CloudWatch Metrics 对系统进行了全方位性能压测与运行监控。监控的核心指标覆盖了请求延迟（Latency）、吞吐量（Throughput）、错误率（Error Rate）以及 Lambda 的运行时长（Duration）。监控数据显示，迁移后的系统呈现出了全方位的性能优化：Lambda 函数的单次执行时长和调用延迟显著降低，整体吞吐量成倍攀升，错误率得到了有效收敛；更为核心的是，模型检索生成的准确率（Accuracy）与精确度（Precision）因更优质的向量处理机制获得了明显改善。 --- ### [16:33](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=993s) 企业级系统设计原则在 AWS Serverless 上的全面落地讲师回顾了此前制定的企业级架构原则，并逐一将其映射到具体的技术选型与实际业务收益上。这种设计确保了系统既轻量敏捷，又完全符合企业对于高可用、高合规与高弹性的严苛要求： | 设计原则 | 对应技术与实现方式 | 业务价值与实际影响 | |:--- |:--- |:--- | | **模块化 (Modularity)** | 各 API 路由严格映射至独立的 AWS Lambda 函数 | 业务解耦，单一功能改动与升级互不干扰 | | **可扩展性 (Scalability)** | 全托管 Serverless 架构 | 面对高并发请求实现秒级指数级扩容 | | **高可靠性 (Reliability)** | AWS 托管服务 + Lambda 自动重试 + S3 自动备份与多可用区冗余 | 消除单点故障，保障服务高可用 | | **安全性 (Security)** | IAM 最小权限原则（Least Privilege）+ S3 静态与传输中全链路加密 | 严格管控 Bedrock 与数据访问权限，符合合规标准 | | **灵活性 (Flexibility)** | Bedrock 支持按需动态切换底层基础模型；Lambda 支持多语言与灵活重构 | 业务算力与技术栈可敏捷调整，避免供应商锁定 | | **高可维护性 (Maintainability)** | CloudFormation 基础设施即代码（IaC）+ AWS CLI 快速打补丁 + CloudWatch 自动化监控 | 运维操作代码化、自动化，变更安全可追溯 | | **可测试性 (Testability)** | 隔离组件单元测试（Unit Test）与独立 Lambda 调试 | 降低测试复杂度，加速 CI/CD 流水线验证 | | **高复用性 (Reusability)** | Lambda Layers 共享依赖与通用逻辑 | 核心 RAG 架构可无缝复用于 Chatbot 等多种业务前端 | | **可观测性与评测 (Observability & Eval)** | LangSmith / Langfuse 链路追踪 + CloudWatch 监控大盘 + TensorBoard 评估可视化 | 全链路追踪调用细节，量化评估模型表现与系统健康度 | --- ### [18:48](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=1128s) 企业级 AI 核心风险防控与安全护栏实践在企业生产环境中部署大语言模型，必须直面幻觉、数据泄露、偏见及安全性等严峻挑战。讲师系统性阐述了针对各类潜在 AI 风险所构建的纵深防御与缓解策略： ```mermaid flowchart TD accTitle: Enterprise AI Risk Mitigation Framework subgraph Risks["AI 系统核心风险"] R1["模型幻觉 (Hallucinations)"] R2["上下文缺失 (Lack of Context)"] R3["安全与隐私泄露 (Security & Privacy)"] R4["内容偏见 (Bias)"] R5["黑盒输出 (Lack of Explainability)"] R6["不良内容与违规 (Profanity & Sensitivity)"] end subgraph Strategies["架构级缓解方案"] S1["RAG 检索增强 + 事实接地"] S2["向量数据库语义检索 (原生余弦相似度)"] S3["VPC 隔离 + IAM 最小权限 + S3 端到端加密"] S4["文档上下文物理级隔离"] S5["切片追踪 (Chunk Tracking) 溯源标注"] S6["AWS Bedrock 护栏 + Lambda 内容过滤"] end R1 --> S1 R2 --> S2 R3 --> S3 R4 --> S4 R5 --> S5 R6 --> S6 ``` - **幻觉抑制与可解释性**：采用 RAG 架构将大模型锚定在企业专有知识库上；API 响应同步返回切片溯源信息，彻底实现回答可溯源、可核查。 - **语义上下文增强**：依托向量数据库的原生余弦相似度（Cosine Similarity）进行精准语义匹配，攻克上下文语义偏差。 - **数据安全与隐私保护**：通过 AWS VPC 虚拟私有云隔离网络流量，执行严格的 IAM 最小权限分配，并对 S3 数据实施传输与静态全流程加密。 - **消除算法与数据偏见**：将各业务领域知识库与代码文档进行逻辑与物理层面的彻底隔离，检索严格限定在当前文档域内。 - **性能与实时响应**：持续优化后，端到端查询耗时被压缩到了极其出色的 150 毫秒（150 ms）。 - **内容合规与安全过滤**：在 AWS Bedrock 与 Lambda 之间集成了内容过滤（Content Filtering）机制，前置拦截脏话污语（Profanity）、成人内容及敏感企业数据。 - **合规高分表现**：对照《AWS 卓越架构框架（Well-Architected Framework）》白皮书进行严格自评，系统取得了 54 / 60 的优异高分。 --- ### [21:07](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=1267s) 一人团队的极致研发效能与结语在演讲尾声，讲师分享了个人开发者借助现代化 AI 研发工具实现“一人成军（Building Big with a Team of One）”的实战心法。在整个系统的架构设计与代码构建过程中，讲师深度联动了 Cursor、Eraser.io 以及 Google Gemini 等 AI 工具。通过充分利用 Cursor 的智能补全与上下文代码生成能力，独立开发者能够以极高的人效跨越原型构建与生产落地的鸿沟。最终建成的企业级问答系统不仅具备“日均运行成本不足一杯咖啡钱”的极致成本控制力，还能在秒级实现数千并发的指数级弹性伸缩，同时具备顶级的企业级安全性。正当讲师准备总结“明确定义业务问题（Clearly define the problem）”等核心建议时，主持人因下一场会议时间临近（11:30 开始）礼貌介入主持，现场以热烈的掌声为这场兼具深度与实战价值的分享圆满收尾。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=4s) Can a solo developer build enterprise-grade AI systems? — Soujanya Vullam > [00:04](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=4s) 单个开发者能否独立构建出企业级的 AI 系统？ — Soujanya Vullam > [10:00](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=600s) API 平均响应时间高达近 3 秒 — Soujanya Vullam > [14:00](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=840s) 当前答案是基于哪些具体文档切片生成的 — Soujanya Vullam > [14:00](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=840s) 极其可观的飞跃 — Soujanya Vullam > [14:00](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=840s) 性能成倍提升 — Soujanya Vullam > [21:07](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=1267s) 一人成军（Building Big with a Team of One） — Soujanya Vullam > [21:07](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=1267s) 日均运行成本不足一杯咖啡钱 — Soujanya Vullam > [21:07](https://www.youtube.com/watch?v=vv3ydGd9Anc&t=1267s) 明确定义业务问题（Clearly define the problem） — Soujanya Vullam ## Mentioned Soujanya Vullam, Yahoo, Ai4, Epic Q&A, AWS Bedrock, AWS Lambda, Amazon S3, Amazon DynamoDB, Amazon API Gateway, AWS Cognito, AWS Route 53, Amazon CloudFront, Amazon CloudWatch, AWS CloudFormation, S3 Vectors, LangSmith, Langfuse, TensorBoard, Cursor, Eraser.io, Perplexity, ChatGPT, Google Gemini ## Video description Soujanya Vullam, Software Development Engineer II, Yahoo Abstracts Can a solo developer build enterprise-grade AI systems? In this session, I’ll share how I independently designed and deployed a secure, scalable AI Q&A platform using LLMs, AWS Bedrock, and AWS serverless architecture. The talk will illustrate real-world techniques to accelerate developer velocity, reduce infrastructure costs through serverless system design, and co-develop with AI tools like Cursor, Eraser.io, Perplexity, and ChatGPT. You’ll walk away with actionable strategies for turning lightweight prototypes into production-ready applications that scale. Join me to explore what’s possible when generative AI and cloud-native architecture come together. [Source](https://www.youtube.com/watch?v=vv3ydGd9Anc)

Source quotes

[0:04] Soujanya Vullam: Can a solo developer build enterprise-grade AI systems? [0:04] Soujanya Vullam: 单个开发者能否独立构建出企业级的 AI 系统？ [10:00] Soujanya Vullam: API 平均响应时间高达近 3 秒 [14:00] Soujanya Vullam: 当前答案是基于哪些具体文档切片生成的 [14:00] Soujanya Vullam: 极其可观的飞跃 [14:00] Soujanya Vullam: 性能成倍提升 [21:07] Soujanya Vullam: 一人成军（Building Big with a Team of One） [21:07] Soujanya Vullam: 日均运行成本不足一杯咖啡钱 [21:07] Soujanya Vullam: 明确定义业务问题（Clearly define the problem）

Source quotes

[0:04] Soujanya Vullam: Can a solo developer build enterprise-grade AI systems? [0:04] Soujanya Vullam: Observability allows engineers to inspect the internal state of an application via generated telemetry—logs, metrics, and distributed traces—to understand how the large language model makes its decisions, evaluate performance trends over time, and ensure stringent compliance with security protocols. [3:21] Soujanya Vullam: Because DynamoDB is a traditional NoSQL database lacking native vector indexing and vector similarity search primitives, the compute-heavy operations—namely calculating cosine similarity and sorting/ranking candidates against query vectors—had to be processed entirely inside the Lambda functions. [7:10] Soujanya Vullam: Designing an enterprise system as a solo developer demands rigorous architectural choices early on to avoid runaway infrastructure maintenance and unmanageable operational overhead. [7:10] Soujanya Vullam: Choosing AWS Bedrock provided access to foundation models for both natural language response generation and embedding synthesis, packaged as an auto-scaling, pay-per-use managed service with multi-nine availability. [9:56] Soujanya Vullam: Because DynamoDB stored the vector arrays merely as raw attributes, the Lambda compute layer had to retrieve the records, parse the numerical arrays, execute cosine similarity math across every record, and sort the ranked results before constructing the model context. [9:56] Soujanya Vullam: Standard relational and NoSQL databases lack indexing strategies capable of executing approximate nearest neighbor (ANN) searches across millions of high-dimensional vectors at sub-second speeds. [12:45] Soujanya Vullam: Because vectors are stored alongside their raw chunk text and metadata directly within the database engine, an ANN similarity query returns the matched text chunks directly in the search response payload. [12:45] Soujanya Vullam: The migration required no architectural overhaul: the system boundaries remained intact, and changes were confined to the query Lambda functions and IAM execution roles to grant access to the S3 Vectors API. [18:50] Soujanya Vullam: To eliminate hallucinations, the application strictly adheres to the RAG pattern, grounding every generation within retrieved context and injecting the source chunk metadata into the final API payload for end-to-end source attribution. [18:50] Soujanya Vullam: less than a cup of coffee a day [18:50] Session Moderator: I'm so sorry to have to cut you off. Unfortunately we have another session starting at 11:30. So let's give her a round of applause.

Original transcript

[0:04] So thanks for joining me here. Uh the [0:06] session today is about building [0:08] enterprisegrade AI systems solo with [0:12] LLMs and AWS. Can a solo developer build [0:16] enterprisegrade AI systems? So this is [0:19] the question I always wondered and uh [0:22] can I go beyond a prototype and build [0:25] enterprisegrade AI system uh which means [0:28] which is scalable, secure, modular and [0:32] um which can be production ready. So [0:36] this has driven me for the past few [0:38] months and I've been working on this um [0:41] prototype for um uh developing an [0:45] application for a question and answer uh [0:48] application related to um any large [0:52] language um that could parse any large [0:55] books and code into documentation and [0:59] give contextaw aware intelligent [1:01] answers. So today I'm going to go over [1:04] the use case [1:08] uh Epic Q&A web application. It is a AI [1:11] powered uh Q&A web application. Uh so [1:16] firstly what makes an um what makes a [1:20] system enterprise grade? [1:23] Let's go over this and we'll go back to [1:25] the other slide. uh so enterprise grade [1:29] as I mentioned previously these are all [1:31] the system design principles that are [1:33] need to be followed for an enterprise [1:35] grade uh enterprise grade system [1:37] modularity scalability I'm not going to [1:39] go into detail of each of these but in [1:42] the context of AI systems observability [1:44] and evals are very important so uh let's [1:48] go over that uh briefly so observability [1:51] to understand the internal state of a [1:54] system by analyzing the data it [1:56] generates like logs, metrics and traces [2:00] u to understand how an LLM is making a [2:03] decision and to perform uh how it's [2:06] performing over time and also complying [2:08] to security. So that that is [2:11] observability and evals they are [2:13] frameworks used for understanding and uh [2:17] measuring how LLM is performing. uh [2:20] unlike traditional testing this is going [2:23] to uh conform to uh specific uh [2:27] guidelines and specific metrics and we [2:30] are going to uh check the accuracy [2:33] relevance and coherence completeness [2:35] helpfulness of the system of the AI [2:38] system. So they are going to be useful [2:40] for ensuring reliability and identifying [2:43] areas of impu uh improvement and [2:45] enabling informed iterations through AI [2:48] development life cycle. So let's go back [2:50] to the previous one now. So this is the [2:54] um application that I have built. Um so [2:58] users can ask natural language questions [3:00] and receive intelligent answers that are [3:03] context aware and uh based on the [3:06] questions as well as the context which [3:08] is a large book or a code base and the [3:12] documentation is generated from the [3:13] codebase and the relevant answers are um [3:17] are produced based on the context. Uh so [3:21] this is the architecture diagram. If you [3:23] look closely, [3:25] let's look into the architecture. So I [3:27] considered all the system design [3:29] principles that I mentioned previously [3:31] and uh I came up with this um so I came [3:35] up with architecture diagram after I [3:37] considered all the system design [3:38] principles for enterprisegrade quality [3:41] and u when user logs into the epic [3:45] library. Um so let's look at the [3:48] architecture. [3:50] So when the user request comes in um the [3:54] DNS routing is done by uh AWS uh route [3:58] 53 and the request goes through to CDN [4:04] which has the front end uh uh the front [4:07] end code the static uh files are hosted [4:10] on CDN and [4:13] then the authentication happens [4:16] using Cognto and uh Cognto uh um and CDN [4:22] adds the front-end code adds JWT tokens [4:26] to the header to the request and uh the [4:30] JWT tokens that are added to the header [4:33] are validated by API gateway [4:38] and the appropriate lambda function is [4:41] called based on the uh based on the API [4:45] endpoint. So there are different [4:47] endpoints for each lambda operation like [4:50] list uh listing the books, uploading the [4:53] books or uploading the code um or asking [4:56] questions about the code. So each of [4:59] these operations have a different lambda [5:01] endpoint and a specific lambda is called [5:05] based on the API endpoint. [5:08] And um [5:11] then the uh the text that comes into the [5:18] uh lambda is split into chunks and those [5:23] chunks are used uh to generate [5:26] embeddings [5:28] by uh using AWS bedrock and those [5:31] embeddings are inserted into Dynamo [5:34] database. [5:36] So this is the first round of [5:39] architecture that I came up with. Um and [5:42] you can see that each of these um AWS [5:47] services have uh separate [5:51] um [5:53] description as written over there. DNS [5:56] the route 53 for DNS, CDN for static web [6:00] hosting and content delivery. uh asset [6:04] caching for uh the UI files and uh O the [6:09] cognto for authentication [6:11] uh API gateway for rate limiting [6:14] throttling lambda integration [6:16] um and lambdas for all the things that I [6:19] said before the qu question answer [6:21] processing file uploading and all those [6:23] things and also here lambda does the [6:26] cosine similarity and sorting because [6:29] Dynamo database is not a vector database [6:32] So we have this overhead of uh [6:35] performing cosine similarity and sorting [6:37] in lambda functions. So that was the [6:41] first pass of architecture that we came [6:43] up with. And you also see that there is [6:45] S3 in the middle over there that is for [6:48] static web hosting. So all the files [6:51] that are generated by react which is the [6:54] front end are uh loaded into uh S3 and [6:58] uh these are cached on CDN for global [7:01] content uh delivery. So that's the first [7:04] pass of the architecture and so for any [7:08] architecture [7:10] um especially for a solo developer I had [7:13] to be very careful with the first uh [7:16] important design decisions. So these are [7:19] some of the design decisions that I've [7:21] taken. So bedrock uh so bedrock [7:25] I've used this to generate question [7:28] answer responses and to also generate [7:31] the semantic uh semantic embeddings for [7:35] uh sorry the embeddings for semantic [7:37] search and um it is um it offers [7:42] autoscaling fine lines of availability [7:45] and um it has paper use which is perfect [7:47] for cost efficiency and uh I can pick [7:51] and choose any model I need. So I have [7:53] different models for different use cases [7:55] which is one for uh text uh uh the u [7:59] question answer generation and uh [8:02] response and the other for uh [8:04] embeddings. So two different models used [8:07] um and the next is S3 CDN. S3 as I said [8:12] is used to store the uh raw files uh [8:16] that are uploaded by the users and also [8:20] it is used to uh save the static um the [8:24] static UI files that are generated by [8:26] the front-end code uh the HTML and [8:28] JavaScript files. Um and uh CDN is used [8:32] to uh cache those files so that they are [8:36] available locally and even this [8:39] combination is uh scalable exponentially [8:42] because um S3 has uh unlimited storage [8:46] and it is um finance of availability and [8:50] offers security and so on so forth. So [8:54] then we have the combination of API [8:56] gateway and lambda. We also briefly [8:58] spoke about this. Every API endpoint has [9:01] a corresponding lambda function and um [9:05] that is list, query, upload and uh so on [9:08] so forth and it has uh the same uh [9:12] characteristics as bedrock and uh S3. Um [9:16] it it is um finance of avail it has [9:20] 99.95 availability and autoscaling and [9:24] uh it has inbuilt u authentication uh [9:28] integration uh that I have integrated [9:30] with cognto right now um and uh JWT [9:34] tokens uh it is pay-per-use just as [9:37] other services that I picked so that we [9:40] don't pay for idle time [9:43] Dynamob uh this I picked in the [9:46] beginning for the same purpose as I [9:48] mentioned before encryption uh at rest [9:50] and uh transit and uh finance of [9:54] availability autoscaling and uh so on so [9:56] forth and the next slide um vector [10:00] databases. So after after the first pass [10:04] of the architecture, [10:07] I measured some um uh I measured the [10:11] cloudatch metrics of latency, throughput [10:15] um and um um performance, the speed of [10:19] APIs. it was um almost like 3 seconds [10:24] the speed of APIs because the lambda uh [10:28] was uh the bottleneck because the cosine [10:31] similarity operations and uh the sorting [10:34] was happening in lambdas. So I looked up [10:37] a vector database but I needed something [10:39] that is cost efficient. Uh so I started [10:42] looking into vector databases and what [10:45] is a vector database for most people uh [10:48] probably know about this but in short it [10:51] stores and indexes and searches through [10:54] vectors. Vectors are nothing but [10:56] numerical representation of data uh in [10:59] arrays uh so that it is enabling faster [11:02] semantic search. And why do we need [11:05] vector database? We need vector database [11:08] because uh not all databases have [11:12] indexes that can uh scale and also be [11:16] used for this purpose and that are [11:19] efficient for searching millions of [11:21] records. So uh [11:25] just as I used Dynamob which is also a [11:28] powerful database and has fast u it is [11:32] scalable but it's not a vector database. [11:34] So the operations like cosine search and [11:38] dot search and similarity uh the uh [11:41] sorting ranking had to be done outside [11:44] of the database and the the way the data [11:48] is stored itself is not uh similar to [11:52] the regular database. It is in a vector [11:54] space in a numerical format and uh these [11:59] embeddings are stored in the database [12:01] and they are intended to be searched for [12:03] a large scale in milliseconds and they [12:06] have specialized indexing which follows [12:09] different algorithms. Um and they have [12:12] they provide nearest neighbor search u [12:16] and also metadata filtering and uh [12:18] hybrid search which is semantic search [12:21] with uh metadata with u with keywords. [12:25] So why do we have to use when do we use [12:27] this? We use this for semantic searches, [12:30] uh, rag, chat bots, recommendation [12:34] searches, uh, image search, video [12:37] search, anomaly detection, and the list [12:39] goes on. So, uh, [12:45] so I looked into S3 vectors. So, this is [12:48] fairly new. It just came up in July, uh, [12:52] July 15th on July 15th, 2025. I started [12:55] looking into it a few weeks back. It's [12:57] not yet available in all regions but it [12:59] is available in US East uh fortunately. [13:03] So I was able to evaluate it and um [13:09] so good thing about S3 vectors is it's [13:11] native AWS integration. So I was able to [13:14] use cloud formation for installation and [13:16] I was able to use the cloudatch metrics [13:19] for checking the uh latency and duration [13:23] and all these things uh that I've used [13:25] for other services and uh the [13:27] operational simplicity of course because [13:29] of the native AWS integration cost [13:32] efficiency because it's paper use and uh [13:35] scalability because it is uh it scales [13:37] exponentially and it has unlimited [13:40] storage and metadata storage here the [13:45] data is stored the chunks after they're [13:48] split they're stored along with the uh [13:51] metadata so when I query for the uh [13:55] metadata u when I'm making a query to s [14:00] uh S3 vectors I get the chunks that are [14:03] relevant to the query in the query [14:05] itself so I can opt to do that so that I [14:08] don't have to make another query to get [14:10] the text that is relevant So this text [14:13] can be passed all the way to the API [14:15] response. So there is explanability and [14:18] uh the developer or the user can [14:20] understand why this response is made [14:23] based on what chunk this response is [14:25] made and uh performance SD vectors [14:30] performance was uh there was a very um a [14:35] considerable improvement. I will go over [14:37] that in the next slides. And the [14:40] improvement is because of the um uh the [14:44] vector operations that are happening [14:46] native to AWS uh within the database uh [14:50] as opposed to outside the database pre [14:53] in the previous implementation and it [14:55] has the native security compliance of [14:57] AWS. So this was the old architecture [15:00] which had the Dynamo DB and this is the [15:03] new architecture which has the S3 [15:05] vectors. So now I replaced the S3 uh the [15:08] Dynamob with S3 vectors and all I had to [15:11] change was lambdas and uh some lambda [15:15] execution rules to give access to S3. So [15:19] this is an a perfect example of [15:21] flexibility and modularity because it [15:23] was able to just change a small portion [15:26] of the code to get access to uh S3 [15:29] vectors and increase the performance of [15:32] the system multiffold. [15:35] So let's go over that next. So [15:38] observability and eval. So as I said [15:40] previously these are very important um [15:43] for any AI system or any system for the [15:46] matter of fact and u cloudatch metrics. [15:50] So I've I was able to use the cloudatch [15:53] metric since it is native and um I I [15:57] checked the latency, throughput, error [16:00] rate and duration of Lambda's uh runtime [16:03] and I I was able to see reduction of [16:06] latency and increase in throughput and [16:10] reduction in error rate and duration for [16:13] lambdas [16:14] and I was able to see an improvement in [16:18] accuracy and precision of the uh model [16:22] based on by using the S3 vectors. [16:27] This is after the migration of data from [16:30] Dynamo DB to S3 vectors [16:33] achieving enterpriseg grade design [16:35] decisions. So these were all the uh [16:38] system design principles that we had [16:40] looked into previously and I mapped them [16:42] to technologies that I have used and the [16:46] um effect that it had made the business [16:49] impact it had made based on the system [16:52] design principle. So if you see [16:53] modularity is mapped to the lambda [16:55] functions we have already seen that each [16:57] API uh is mapped to a lambda function [17:00] and scalability exponential scaling we [17:03] have already seen that reliability uh we [17:05] are um on AWS u manage services and [17:10] lambda has automatic retries and S3 has [17:13] automatic backups and redundancy [17:15] security um IM roles we're using IM [17:18] roles to um give access to AWS bedrock [17:22] on S3 vectors we are using least [17:24] privilege uh principle here and uh S3 [17:27] data encryption uh at rest and in [17:30] transit uh flexibility we can use any [17:33] model we choose based on the operation [17:36] on the bedrock and uh lambda uh is [17:39] modular and we can change the code we [17:41] can change the language we can change uh [17:44] stuff inside lambdas and cost [17:47] effectiveness as we saw all these before [17:50] maintainability infrastructure as code [17:52] I'm using cloud formation here and AWS [17:55] CLI for patches um automated moni [17:58] monitoring using cloudatch [18:00] testability I'm using uh unit test and [18:04] um isolated components lambdas can be [18:06] tested separately reusability lambda [18:10] layers um can be reused the entire [18:13] architecture can be reused for different [18:15] purposes like rags chatbot it can be [18:17] reused for multiple purposes by changing [18:19] the front end uh observability ility. I [18:22] use lang to trace the requests and uh I [18:26] use cloudatch uh logs and the automatic [18:29] dashboards it provides for lambdas and [18:31] dynamob and s3 monitoring. Um evaluation [18:35] I use tensorboard for the uh [18:37] visualization and um act to measure [18:40] accuracy uh and uh as I mentioned I used [18:44] cloudatch metrics for the other for the [18:46] other measurements um AI risks. So for [18:50] the enterprise AI systems when we are [18:53] building that we need to keep in mind [18:55] all these risks and the mitigation [18:57] strategies. So I mentioned some of the [19:00] mitigation strategies that I have used [19:02] here. Uh for hallucinations I've used [19:05] rack pattern. So the responses are [19:08] grounded and as I mentioned the chunks [19:10] are in the API responses. So we are able [19:12] to uh determine which uh why this API [19:17] response is returned. lack of context we [19:19] using semantic search here with a vector [19:21] database which manages uh native cosign [19:24] similarity security and privacy there is [19:27] VPC isolation there is least privilege [19:29] AM principle that we are following there [19:32] is encryption at rest and transit for S3 [19:35] as I mentioned and bias we are uh the [19:38] documents are isolated completely um [19:41] when the search is made it's made based [19:44] on the book or the code that we are [19:46] referring to So um there is no bias [19:50] explanability as I mentioned there is uh [19:52] source attribution with the uh chunk [19:55] tracking sorry um and uh performance as [19:59] I mentioned there was a considerable [20:01] hike of performance and uh we are still [20:04] improving the system uh however the [20:06] performance has increased uh to 150 [20:10] milliseconds uh for the uh query and um [20:15] reliability um As we see there, there is [20:18] a content filtering that I added uh [20:21] using AWS bedrock and lambdas uh for [20:24] ensuring the um that the AI responses [20:28] are um are without any profanity and [20:33] adult language and sensitive data and [20:35] stuff. [20:36] Um and we went through the AWS well [20:41] architected framework best practices [20:44] and this is [20:48] um best practices and scored um 54 out [20:51] of 60 based on the white paper AWS well [20:55] architected framework. Um we already [20:57] went through each of these previously in [21:00] the previous slide so I'm not going to [21:02] go over them again. [21:07] Yeah. Uh so I've used these tools for um [21:11] accelerating velocity um for my [21:14] development as you see uh cursor uh [21:18] eraser and Google Gemini. And these are [21:20] the cursor features that are um that [21:24] accelerated my development. [21:28] And yeah uh building big with a team of [21:30] one cost less than a cup of coffee a day [21:33] scales to thousands in a seconds and [21:36] enterprise grade security co-uild with [21:38] AI tools from prototype to production [21:40] ready key takeaways clearly define the [21:44] problem. [21:44] >> I'm so sorry to have to cut you off. [21:45] Unfortunately we have another session [21:47] starting at 11:30. So let's give her a [21:49] round of applause. Thank you so much for [21:50] that beautiful session. Thank you.