## Background
This video was published on YouTube by the channel Ai4 on August 18, 2026. The page description indicates that Ariel Shulman, Chief Product Officer at Bright Data, presents on how modern AI agents often get blocked or misled by stale and deceptive web data, and introduces solutions for providing them with reliable, real-time web eyes.
## TL;DR
Ariel Shulman outlines how the internet has shifted from human traffic to autonomous AI agent traffic that frequently falls victim to web deception, CAPTCHAs, and rapid data staleness. He demonstrates how websites actively fight back against scrapers by serving fake prices or false availability, which AI agents lack the human intuition to detect before automating costly errors. To solve this, Bright Data provides robust web data infrastructure—utilizing millions of IPs, headless browsers, and AI-driven self-healing collectors—to deliver accurate, real-time web data to major frontier AI models and developers.
## Atmosphere & style
The presentation maintains an engaging, witty, and authoritative tone, seamlessly blending nostalgic tech history with practical modern engineering challenges. Shulman uses lighthearted pop-culture references—ranging from Orson Welles' classic radio hoax to Blade Runner and BattleBots—to make complex web scraping infrastructure feel accessible and fun. The audience interaction and practical Q&A session create an informative, dynamic atmosphere focused on solving real-world AI reliability issues.
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=4s) Historically, early web portals relied on manual categorization, but Google later established an unprecedented data monopoly spanning text, images, video, and real-world telemetry.
- [02:05](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=125s) A 2025 DOJ antitrust court opinion ruled that Google must share limited data while retaining core IP like PageRank, forcing software developers to build their own web crawlers.
- [04:07](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=247s) Unconnected AI applications act like static books, requiring dynamic and fresh web data to operate as reliable agentic tools in production.
- [05:37](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=337s) Simple, unshielded web scraping scripts work in local tests but fail at scale due to sophisticated anti-bot blocks, rate limits, and IP throttling.
- [05:37](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=337s) AI agent traffic on the internet has officially surpassed human web traffic, generating exponentially more page views per task.
- [07:25](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=445s) Target websites now actively defend against scraping by serving plausible-looking fake data or artificially inflated pricing rather than simple HTTP errors.
- [10:33](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=633s) Inaccurate web data destroys customer trust and leads to severe AI failure modes, as agents execute bad inputs at automated speeds without human intuition.
- [10:33](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=633s) Web data decays rapidly; social media feeds and e-commerce stock become stale within a day, whereas news and financial data decay over several days.
- [14:33](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=873s) Bright Data leverages 200+ million IPs and 250,000+ headless browsers to power real-time search capabilities for 75% of leading frontier AI models.
- [14:33](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=873s) Landmark court victories against Meta and Elon Musk affirmed the legal right to scrape publicly available internet data.
- [20:04](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=1204s) To verify data accuracy, automated systems cross-validate suspicious data using alternate IPs, browser user agents, and device fingerprints.
- [21:43](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=1303s) Bright Data provides an integrated product stack ranging from raw proxies and web-unlocking algorithms to self-healing APIs and AI-assisted scraper builders.
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=4s) Historical Evolution of the Web: From Orson Welles to Tim Berners-Lee and Search Portals
Ariel Shulman opens the presentation with an engaging historical comparison, pointing back to 1938 when there was no internet or television, only radio. He recalls how Orson Welles broadcasted a radio show that famously fooled listeners into believing Martians were invading Earth, triggering mass hysteria. Fast-forward nearly 90 years, and it is no longer humans being fooled—it is AI agents. Before diving into how AI agents get duped, Shulman sets the stage with a quick historical perspective on how web infrastructure developed.
He highlights Sir Tim Berners-Lee, the inventor of the World Wide Web, who single-handedly created the URL, HTTP, HTML, the first web browser, and the first web server. Shulman emphasizes that the open-sourcing of these technologies in 1993 made the modern internet possible. The narrative moves to 1994, when the internet began expanding rapidly, leading to early attempts at indexing web content. Stanford University students manually categorized 23,000 websites to create Yahoo!, followed by early engines like Ask.com (which recently announced the cessation of its operations) and the "cringey" early era of web portals. Shulman pokes fun at the chaotic early interfaces like MSN.com—shipped on every new computer around 1998 to 2000—pointing out nostalgic details like a smiling Ben Affleck and noting how the Nasdaq was sitting at 1,940 back then compared to around 26,000 today.
- Orson Welles' 1938 *War of the Worlds* radio broadcast caused mass hysteria by fooling listeners.
- Sir Tim Berners-Lee open-sourced the internet technology stack (URL, HTTP, HTML, browser, server) in 1993.
- Early Yahoo! manually categorized 23,000 sites item by item.
- Early portals bundled search, news, and financial indicators on a single dense homepage.
[02:05](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=125s) Google's Data Monopoly and the DOJ Antitrust Ruling
The true leap forward in web technology arrived with Google. Shulman polls the audience, asking who has used YouTube, Android, Google Maps, or Gmail in the last 24 hours, noting that virtually every hand goes up. Google possesses an unprecedented data advantage: indexed web text, Google Photos images, YouTube video and audio, e-commerce data, local Google Maps information, sensor inputs from Android phones, health data, and even real-world physical telemetry from Waymo autonomous vehicles. Google leverages this vast pool of data to train its Gemini AI models and power agentic features like Spark that execute actions on behalf of users.
This immense concentration of information did not go unnoticed by regulators. The U.S. Department of Justice (DOJ) brought a major antitrust case against Google. Shulman clarifies that while he is not a lawyer, he reviewed the 2025 court opinion. The ruling required Google to share certain data once, but allowed them to keep their core proprietary algorithms—such as PageRank—secret. Consequently, the judge concluded that software builders must still construct their own crawlers, index web pages, and innovate independently if they want to compete with Google.
- Google holds massive proprietary data across text, video, audio, location, mobile sensors, and Waymo telemetry.
- Google uses this dataset to train Gemini and build action-oriented agents like Spark.
- The 2025 DOJ antitrust opinion ruled Google must share limited data once while retaining core intellectual property like PageRank.
- The judicial ruling explicitly notes that competitors must build their own web crawlers and extraction pipelines.
[04:07](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=247s) Web Data Infrastructure and the Scaling Problem
For software developers and AI builders, live internet access is essential. Without real-time access to web data, a product is static—like a printed book. However, web data collection must be reliable, fresh, and scalable. Bright Data provides this underlying "web data infrastructure" to enable developers to retrieve public data reliably and compete effectively.
Shulman compares web scraping at scale to quantum mechanics: while physics operates predictably at a macro scale, things behave strangely at quantum dimensions. Similarly, simple web access logic works fine in pilot projects or local development, but once a system scales up in production, traditional scraping scripts break down. Systems run into immediate blocking, rate-limiting, and anti-bot measures, making dedicated web data infrastructure necessary to maintain reliable access.
- Unconnected AI applications behave like static books rather than dynamic tools.
- Web data collection breaks at production scale due to anti-bot measures, rate limits, and structural blocks.
- Bright Data provides web data infrastructure to ensure consistent and scalable data extraction.
[05:37](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=337s) Vibe Coding, "Blade Runner" Bots, and the Rise of Agentic Web Traffic
Shulman warns against the temptation to rush unvetted scraping scripts into production. In today's era of "vibe coding," a founder or CEO might build a hotel pricing scraper over the weekend and urge the team to ship it immediately. While the UI and basic logic may function initially, unshielded scripts are guaranteed to fail at scale in production.
Referencing his favorite movie, *Blade Runner*, Shulman reflects on the shift toward an internet populated by autonomous bots. The web was originally built for human navigation, but automated agents are taking over. Citing Cloudflare's CEO, Shulman notes that automated agent traffic on the web has officially surpassed human traffic. The math explains why: a human looking for a camera might check 3 or 4 web pages, whereas an AI agent evaluating options might navigate through 100 sites, clicking links and gathering specifications automatically.
- "Vibe coded" weekend scraping projects typically pass local tests but fail immediately in production environments.
- Cloudflare telemetry confirms that AI agent traffic on the internet now exceeds total human traffic.
- Agents generate exponentially more page views per task compared to human shoppers.
[07:25](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=445s) When the Web Fights Back: CAPTCHAs, Throttling, and Fake Data
As bot traffic surges, the traditional human web is pushing back using defensive mechanisms like CAPTCHAs, rate-limiting, and IP throttling. While these defenses were designed to stop malicious actors—such as bots attempting to register thousands of fake accounts—they end up collateralizing legitimate user-backed AI agents acting as extensions of individual users.
More dangerously, websites are transitioning from simple blocking to active deception. Bright Data has observed a growing trend where anti-bot systems detect automated crawlers and silently feed them garbage data or "disinformation" rather than blocking them outright. The scraper receives an apparently normal response, but the content inside is completely wrong—the web has effectively "learned to lie."
- Sites use CAPTCHAs and throttling to shield server resources from automated traffic.
- Anti-bot systems now actively detect scrapers and serve plausible-looking fake data rather than throwing 403 or 429 error codes.
- "Garbage in, garbage out" occurs silently when agents process fake data feeds.
[09:04](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=544s) Concrete Examples of Web Deception: Ticket Outages and Hotel Surge Pricing
Shulman illustrates web deception with two concrete examples:
1. **Concert Ticket Aggregator:** A developer creates a site to help users find event tickets. When a human visits `tickets.com`, available seats display normally. When the developer's agent scrapes the exact same site, the target server identifies it as a bot and returns "No tickets available." The aggregator reports zero availability, leaving users frustrated over false information.
2. **Hotel Price Comparison:** In a price tracking app (like the fictional "Wiz Hotel"), an agent scrapes a major hotel chain (`hilton.com`). Because hotels prefer direct bookings to avoid paying third-party commissions, the site detects the scraper and artificially inflates the room rate from $199 to $399. When the user checks the direct site and sees $199, they lose total confidence in the agentic app.
- Ticket websites return empty seat inventories to automated agents while serving valid tickets to human browsers.
- Hotel websites output inflated prices (e.g., displaying $399 to bots instead of the real $199 rate) to penalize third-party scrapers.
- Inaccurate agent data immediately destroys customer trust and application utility.
[10:33](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=633s) Failure Modes of AI Agents vs. Humans and Data Decay Rates
AI agents fail far more dangerously than human users. If a human encounters an unexpected $399 price tag on a budget hotel, intuition kicks in—they pause, double-check, and verify before taking action. AI agents lack this self-correcting intuition; they process bad inputs and execute automated workflows at blazingly high speeds. For instance, an agent receiving a false $399 hotel rate might instantly blast out 5,000 automated emails to customers reporting price hikes, forcing the business to handle mass customer backlash.
Beyond intentional web deception, stale data quietly degrades AI performance. E-commerce prices, product stock, and local business statuses change continuously. Shulman gives an example of an AI chat recommending a restaurant, only for the user to arrive and find a "Temporarily Closed" sign on the door.
Bright Data analyzes this phenomenon through real-world telemetry, scraping roughly 50 billion URLs per day (about 35 million per minute across 20,000 customers)—roughly three times the advertised daily volume of Google searches worldwide. Their empirical data tracking "time to staleness" reveals clear decay rates across different categories:
```
[Fresh / Reliable Data]
│
├── Social Media & E-commerce Stock/Prices ──> Stale within < 1 Day
├── News Articles ───────────────────────────> Stale within 1 - 2 Days
├── Financial Information ───────────────────> Stale within 5 - 7 Days
├── Retail Descriptions & Reviews ───────────> Slower Decay
└── General Web & Blogs ─────────────────────> Very Slow Decay
│
[Stale / Misleading Data]
```
- AI agents lack human intuition and rapidly propagate errors across automated downstream tasks (e.g., sending 5,000 erroneous emails instantly).
- Bright Data scrapes ~50 billion URLs daily (35 million URLs per minute) for over 20,000 customers.
- Social media feeds and e-commerce inventories become stale and misleading in less than 24 hours.
- News content decays in 1 to 2 days, whereas financial data decays over 5 to 7 days.
[14:33](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=873s) Inside Bright Data: Architecture, Unconventional Culture, and Legal Victories
To overcome blocks, fake data, and stale content, Bright Data operates an extensive scraping infrastructure. Their network leverages over 200 million IP addresses across the globe, with over 250,000 headless browsers active at any given second. These browsers perform realistic human-like actions—clicking elements, entering text into input boxes, and mimicking natural interaction flows—to ensure target websites register legitimate sessions rather than bot interactions.
Behind this infrastructure sits a remarkably unconventional company of about 500 employees. Bright Data operates with:
- **Only one meeting room** across the entire company (reserved strictly for external vendor meetings).
- **Zero QA staff**, relying entirely on automated testing to prevent quality assurance bottlenecks from slowing development.
- **~100 production code deployments per day** to continuously adapt to changing anti-bot measures on the web.
Bright Data powers the web search capabilities for 75% of the leading frontier AI models and chatbots available today. When a user sees a chatbot display "Searching the web...", the request is frequently routing through Bright Data's infrastructure behind the scenes.
This aggressive pace of innovation has occasionally put Bright Data ahead of existing legal frameworks. The company faced high-profile lawsuits from Meta (Facebook) and X (formerly Twitter, led by Elon Musk), both claiming that collecting publicly available web data was unauthorized. Bright Data refused to settle, spending millions of dollars to litigate both cases. They achieved decisive court victories in both cases, establishing legal precedents confirming that scraping publicly available web data is entirely lawful. These landmark rulings are now cited across the industry in ongoing AI data training lawsuits.
- Bright Data utilizes a pool of 200+ million IP addresses and 250,000+ active headless browsers.
- The company structure features 500 employees, 0 QA staff, 1 physical meeting room, and ~100 daily software releases.
- Bright Data infrastructure serves 75% of major frontier AI models and LLM chatbots for real-time web retrieval.
- Landmark courtroom victories against Meta and Elon Musk legally affirmed the right to scrape publicly accessible web data.
- Bright Data provides unified API endpoints supporting everything from 1 request/second to 1,000+ requests/second, structured extractions, search indexing, deep crawling, and MCP/SDK integrations.
<ElicitationsGroup message="Explore further details from this presentation:">
<Elicitation label="Summarize Part 2 of Bright Data's AI presentation" query="Can you provide a detailed walkthrough for Part 2 of this Bright Data presentation?"/>
<Elicitation label="Compare web scraping approaches for AI agents" query="Compare Bright Data's approach to web data extraction against standard headless browser automation frameworks."/>
</ElicitationsGroup>
[19:02](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=1142s) Summarizing the Agent Era and BattleBots Fun
Ariel Shulman summarizes the core takeaway of his talk: while modern AI models are highly capable and know how to act, the web has effectively "learned how to lie." To build a scalable, reliable system that users can trust, AI agents must be fed accurate, real-time data—which is precisely where Bright Data comes in. To conclude on a fun note, he mentions Bright Data's sponsorship of the BattleBots League and invites attendees to visit their conference booth to play with the robots and "get some violence out of your system without hurting anyone."
- **Conference Booth**: Attendees are invited to Booth 813 to try out the BattleBots.
- **Evening Event**: Bright Data is hosting an evening event inside the BattleBots arena for conference attendees.
---
[20:04](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=1204s) Data Reliability, Scraping Verification, and Responsible Data Collection
During the Q&A session, an audience member asks how Bright Data evaluates and ensures the accuracy of its scraped web data given the prevalence of fake information. Shulman explains that operating at scale allows Bright Data to inspect data from multiple angles. If an anomaly is detected—such as a sudden price jump on an e-commerce page—the system runs automated verification checks by querying the target site using different country IPs, browser user agents, and device fingerprints to determine if the change is a localized glitch or a widespread update. Furthermore, Shulman emphasizes responsible and ethical data collection; for instance, Bright Data actively monitors target website response times and backs off or pauses requests if a site begins slowing down so real users are never impacted.
- **Verification Techniques**: Cross-validating suspicious web data via alternate geolocation IPs, browser user agents, and device fingerprints.
- **Ethical Scrape Controls**: Real-time target site response time monitoring to automatically throttle or pause scraping before impacting real user traffic.
---
[21:43](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=1303s) Product Tiering, Scraper Studio, and Self-Healing Maintenance
An audience member asks about Bright Data's primary product offerings, specifically inquiring whether they handle end-to-end data extraction or simply supply rotating residential proxies for client-side scraping. Shulman outlines Bright Data's vertically integrated stack, which spans basic proxy infrastructure at the bottom, website-unlocking algorithms in the middle, and structured parsing APIs at the top. Customers can purchase access at any layer, ranging from raw IP proxy pipes to fully managed JSON/CSV data feeds. Additionally, major search engines and AI companies rely on Bright Data's foundational network to access web data.
For custom or unfamiliar websites, Shulman highlights **Scraper Studio**, an AI-powered feature within their control panel. Users simply input a domain name, and the system fetches the HTML, extracts the schema (such as pricing, product descriptions, or user reviews), and engages in a brief conversational dialogue with the user to confirm the schema. Without writing any code, users can build custom scrapers that function like standard Bright Data-maintained collectors.
- **Product Layers**:
- Raw proxy networks (residential, mobile, data center).
- Web-unlocking algorithms (handling CAPTCHAs and anti-bot defenses).
- Structured data extraction APIs (delivering clean JSON/CSV feeds).
- **Pre-built & Custom Collectors**: Out-of-the-box scrapers exist for major platforms like Zillow, while Scraper Studio allows non-technical users to build scrapers for new domains via an AI chat workflow.
- **Self-Healing Technology**: AI-driven self-healing layers automatically adapt extraction code when a target website updates its DOM layout or HTML structure, ensuring unbroken downstream data delivery.
## Notable quotes
> For decades, the web was Google's domain, organizing the chaos for humans. Now AI agents are taking over - but they're only as good as what they can see, and they often get blocked or work off stale data.
> [14:33](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=873s) Searching the web... — Ariel Shulman
> [19:02](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=1142s) get some violence out of your system without hurting anyone — Ariel Shulman
## Mentioned
Ariel Shulman, Bright Data, Ai4, Google, YouTube, Orson Welles, Tim Berners-Lee, Yahoo!, Ask.com, MSN.com, Ben Affleck, Android, Google Maps, Gmail, Google Photos, Waymo, Gemini, Spark, U.S. Department of Justice, PageRank, Cloudflare, Blade Runner, Wiz Hotel, Hilton, Meta, Facebook, X, Twitter, Elon Musk, BattleBots, Scraper Studio, Zillow
## Video description
Ariel Shulman - Chief Product Officer, Bright Data
For decades, the web was Google's domain, organizing the chaos for humans. Now AI agents are taking over - but they're only as good as what they can see, and they often get blocked or work off stale data. They then make confidently wrong decisions, faster than any human could. This session is about fixing that - giving agents real-time, reliable eyes on the world, so they stop automating mistakes and start delivering results you can actually trust.
赢得数据抓取诉讼后，Bright Data 致力于对抗针对 AI 的网络虚假信息
当目标网站不再拦截请求，而是隐蔽地提供虚假信息时，自主 AI Agent 面临大规模执行高代价错误的风险。
Ariel Shulman 概述了互联网如何从人类流量转向自主 AI Agent 流量，而这些 Agent 往往会沦为网络欺骗、验证码以及数据快速失效的受害者。他指出，网站正通过提供虚假价格或虚假的库存信息来主动反击抓取行为，而 AI Agent 缺乏人类直觉，在自动化执行并造成高代价错误前难以察觉此类欺骗。为解决这一问题，Bright Data 提供了强大的网络数据基础设施——利用数百万个 IP、无头浏览器以及 AI 驱动的自愈式采集器——为主要前沿 AI 模型和开发者提供准确、实时的网络数据。
从历史上看，早期门户网站依赖人工分类，但 Google 随后建立了跨越文本、图像、视频和现实世界遥测数据的前所未有的数据垄断。 (0:04)
2025 年美国司法部的一项反垄断法院裁决判定 Google 必须共享有限数据，同时保留 PageRank 等核心知识产权，这迫使软件开发者构建自己的网络爬虫。 (2:05)
未联网的 AI 应用就像静态书籍，需要动态且新鲜的网络数据才能在生产环境中作为可靠的 Agent 工具运行。 (4:07)
简单且缺乏防护的网络抓取脚本在本地测试中有效，但由于复杂的反爬虫拦截、速率限制和 IP 节流，在大规模运行时会失败。 (5:37)
互联网上的 AI Agent 流量已正式超过人类网页流量，每个任务所产生的页面浏览量呈指数级增长。 (5:37)
Reading summary
Bright Data speaker Ariel Shulman argues that as autonomous AI agent traffic overtakes human browsing, websites increasingly deploy anti-bot defenses and deliberate data deception, making real-time, resilient web extraction infrastructure essential to prevent downstream operational failures.
Surge of Agent Traffic and Infrastructure Scaling (2:05)
Citing Cloudflare telemetry, Shulman notes that automated agent traffic has surpassed human traffic on the internet. Because AI agents evaluate dozens or hundreds of pages per task compared to a human's few clicks, bot interactions place unprecedented load on web hosts. While engineers increasingly deploy quick, "vibe-coded" scraping prototypes, Shulman warns that local scripts reliably break when deployed into production due to IP blocks, structural changes, and rate limits.
This infrastructure challenge is heightened by platform centralization. Shulman points to a 2025 Department of Justice antitrust ruling concerning Google: although the court required Google to share specific data once, it allowed the company to keep core algorithmic intellectual property like PageRank secret. The judicial ruling observed that competing software builders must construct their own crawling and indexing infrastructure rather than rely on search incumbents.
Anti-Bot Defenses and Intentional Web Deception (7:25)
Standard defensive mechanisms like CAPTCHAs, IP throttling, and rate limits originally designed to block malicious actors now routinely obstruct legitimate, user-backed AI agents. More critically, Shulman observes that anti-bot architectures are shifting from simple 403 or 429 error rejections to active deception, returning plausible-looking disinformation to automated crawlers.
Shulman illustrates this with real-world examples: ticket platforms detecting bots return zero inventory while serving available seats to humans, and hotel websites inflate room rates from $199 to $399 to deter third-party aggregators and protect direct booking margins. Unlike human users who pause and verify anomalous data, AI agents lack intuitive sanity checks; they immediately process false inputs and execute automated workflows at high speeds, risking rapid reputational damage such as broadcasting incorrect price alerts to thousands of customers.
Accelerated Data Staleness Across Web Categories (10:33)
Beyond deliberate deception, agents are vulnerable to data decay. Drawing on Bright Data's telemetry—tracking approximately 50 billion URLs daily (35 million per minute across 20,000 customers)—Shulman maps out empirical rates of web data staleness. Highly volatile information, such as social media posts and e-commerce inventory or pricing, becomes stale and misleading in under 24 hours.
Other information classes decay at different intervals: news reporting degrades within one to two days, while financial information shifts over five to seven days. General web copy and reviews degrade more slowly. Shulman stresses that without live web infrastructure, AI applications remain static like printed books, causing agents to output obsolete facts, such as recommending restaurants that have temporarily closed.
Verification Architecture, Self-Healing, and Legal Precedent (14:33)
To counter active blocking and data degradation, Bright Data deploys an infrastructure network of over 200 million IPs and 250,000 active headless browsers mimicking human navigation patterns. Shulman states that Bright Data powers web search capabilities for 75% of leading frontier AI models and chatbots. When suspicious anomalies occur, the platform cross-verifies data across differing geolocation IPs, browser fingerprints, and user agents, while throttling requests if a target site slows down.
The stack spans raw proxies to AI-powered Scraper Studio tools that auto-generate schemas and deploy self-healing routines when target DOM structures shift. Addressing legal boundaries, Shulman highlights Bright Data's court victories against Meta and X (formerly Twitter), which established precedent confirming the legality of collecting publicly accessible web data.
While modern AI models can execute sophisticated multi-step workflows, their real-world utility remains constrained by an internet infrastructure actively defending against automation. As websites increasingly serve fabricated or stale data to bots, managing verified, scalable data pipelines emerges as a critical operational prerequisite for reliable agentic applications.
## Background
本内容由 YouTube 频道 Ai4 于 2026 年 8 月 18 日发布，主讲人为 Bright Data 首席产品官（CPO）Ariel Shulman。视频主要探讨在 AI 智能体时代，网络反爬机制与虚假数据如何导致 Agent 做出错误决策，并介绍了 Bright Data 如何通过构建实时、可靠的 Web 数据基础设施为 AI 赋能。
## TL;DR
演讲回顾了互联网从早期门户到谷歌数据垄断的演进历程，指出当前 AI 智能体在规模化联网访问时面临着严峻的反爬阻断、时效性衰减以及网站主动投毒（喂送虚假数据）等挑战。Ariel Shulman 详细阐述了 Bright Data 依托全球分布式代理网络、无头浏览器集群以及自底向上的垂直技术栈，为市面上多数前沿大模型提供实时数据支撑。最后，演讲介绍了 AI 驱动的 Scraper Studio 与代码自愈技术，并结合多维交叉验证与负责任采集机制，确保为智能体生态输送真实可信的结构化数据。
## Atmosphere & style
整场演讲风格兼具极客硬核与轻松幽默，讲者语言干练自信且直击技术痛点，从经典的广播剧恶作剧讲到现代网络反爬对抗，叙事生动且富有张力。现场氛围务实而开放，问答环节展现了极高的工程专业度与严谨的伦理思考，穿插的 BattleBots 展位花絮更是为硬核的技术分享增添了趣味与互动感。
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=4s) 从早年广播剧误导大众到如今 AI 智能体受骗，现代 AI Agent 高度依赖外部网络数据，却面临被虚假与陈旧数据误导的严重困境。
- [02:05](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=125s) 谷歌凭借全方位生态构建了庞大的数据垄断；司法裁决明确表明，竞争对手不能直接依赖谷歌红利，必须自建爬虫与数据提取能力来构建产品。
- [04:48](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=288s) 网络数据抓取存在“量子力学效应”，在本地或低并发下运行顺畅的原型代码，推向生产环境大规模并发时会立刻遭遇 WAF 与反爬机制的全面阻断。
- [04:48](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=288s) AI 智能体访问流量已正式超越人类流量，为人类设计的传统万维网正通过频繁触发 CAPTCHA 验证码和限流反击自动化工具。
- [08:09](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=489s) 目标网站开始主动向机器人“喂送虚假数据”（如虚高价格或伪造售罄），而 AI 智能体缺乏常识直觉，会高速且规模化地将错误结果分发给终端用户。
- [11:41](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=701s) 各类网络数据存在明确的时效性半衰期（如电商价格与社媒动态极速衰减），不具备实时更新能力的应用会因陈旧数据迅速破坏用户体验。
- [14:33](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=873s) Bright Data 拥有超 2 亿代理 IP 与 25 万台在线无头浏览器，每日抓取约 500 亿个 URL，为市面上 75% 的前沿大模型提供联网搜索底层支撑。
- [14:33](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=873s) Bright Data 践行无会议制度、零专职 QA 的极客工程文化，依靠全自动化系统支撑 SaaS 平台实现每日约 100 次的高频线上发布。
- [16:32](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=992s) Bright Data 在针对 Meta 和马斯克旗下公司的诉讼中取得完全胜诉，从法律判例上确立了公开网络数据采集的合法性。
- [20:04](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=1204s) 通过跨国家、IP 和浏览器指纹的多维交叉验证，系统能有效识别局部异常与数据造假，并在采集导致目标网站响应变慢时主动降速以确保合规。
- [21:43](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=1303s) 产品提供从底层网络代理、自动解封（Web Unlocking）到结构化数据交付（API / CSV / JSON）的自底向上全栈垂直整合服务。
- [23:17](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=1397s) 借助 Scraper Studio 对话式 AI 工具与代码自愈（Self-Healing）技术，开发者可零代码快速生成爬虫，并在目标网站前端改版时自动适应而无需手动维护。
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=4s) 从火星人入侵到 AI 智能体受骗：万维网的演进与信息索引的起源
1938 年，在没有互联网和电视的时代，大众唯一的实时信息渠道是收音机。奥森·威尔斯（Orson Welles）通过一场广播剧让全美民众误以为火星人正在入侵地球，引发了全社会的恐慌。近九十年后的今天，被虚假信息愚弄的对象变成了 AI 智能体（AI Agents）。为了理解智能体现今面临的困境，需要回顾互联网构建的起源。蒂姆·伯纳斯-李（Tim Berners-Lee）一手发明了 URL、HTTP、HTML、首个网页浏览器以及首台 Web 服务器，并在 1993 年将整套技术开源，奠定了现代互联网的基础。
随后自 1994 年起，互联网进入了爆炸式增长与信息索引的早期探索阶段。斯坦福大学的两名学生创立了 Yahoo!，最初依靠纯人工逐个网站审核，整理出包含 23,000 个站点的分类目录。接着出现了 Ask.com（近期已宣布停止运营）以及如今看来界面繁杂得让人尴尬的门户网站（Portal）时代。在那个纳斯达克指数还在 1,940 点（如今已攀升至约 26,000 点）的时代，MSN 成为每台个人电脑开机默认的主页，门户页面塞满了搜索框、资讯模块以及好莱坞影星本·阿弗莱克（Ben Affleck）灿烂的微笑。这些早期尝试虽然简陋，却拉开了人类组织网络信息的序幕。
### [02:05](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=125s) 谷歌的数据垄断、反垄断诉讼与开发者面临的挑战
互联网信息组织的真正飞跃来自谷歌。无论是 YouTube、Android、Google 地图还是 Gmail，谷歌构建了极其庞大的数据帝国。它拥有全网索引文本、Google 相册的图像、YouTube 的音视频、电商与 Google 地图的本地商户数据，以及来自 Android 传感器、健康监测乃至 Alphabet 旗下 Waymo 自动驾驶汽车的物理世界数据。谷歌正利用这些海量专有数据训练 Gemini 模型，并发布了名为“Spark”、可代表用户自主执行复杂任务的操作智能体。
这种对信息的高密度垄断引起了监管层面的震动。美国司法部（DOJ）对谷歌发起了反垄断诉讼，2025 年公布的司法裁决认定谷歌拥有过多信息优势。然而判决仅要求谷歌单次共享部分数据，其核心竞争优势（如 PageRank 排序算法等商业机密）依然完全保留。法官在裁决中明确指出："竞争对手必须自行构建爬虫、抓取网页并提取信息，必须通过自身的投资与创新来与谷歌竞争。" 对于构建产品的开发者而言，脱离了实时互联网访问的应用就像一本静态的书；要让产品具备实用价值，所获取的网络数据必须具备可靠性、高时效性与弹性扩展能力。
- **谷歌数据广度**：涵盖网页文本、多媒体（YouTube/相册）、地理与商户（Google Maps）、移动端行为传感器（Android）与物理世界动态（Waymo）。
- **司法裁决核心影响**：竞争者无法直接继承谷歌的基础设施红利，必须自建网页采集与数据抽取能力。
- **产品核心诉求**：AI 应用必须具备实时联网能力，且要求输入数据保持极高新鲜度与稳定性。
### [04:48](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=288s) 规模化访问的量子力学效应与反爬阻断机制
获取网络数据在概念上看似简单，但其底层逻辑存在类似于量子力学的规模化效应：在微观尺度下物理定律会失常，而网络数据抓取则会在大规模并发（生产环境）下彻底失效。很多开发者在本地或针对少数用户进行“氛围编码（vibe coding）”测试时运行顺畅，例如周末快速写出一个抓取希尔顿等各大酒店价格的比价智能体，逻辑与界面看似完美无缺，但一旦推向生产环境就会立刻遭遇大规模封禁与请求崩溃。
```mermaid
flowchart TD
accTitle: 目标网站区分人类与智能体访问的处理逻辑
subgraph ClientLayer["客户端访问请求"]
Human["人类用户访问"]
Agent["AI 智能体访问 (大规模并发)"]
end
subgraph DefenseLayer["目标网站防御与识别系统"]
WAF{"访问特征检测 (WAF / 反爬)"}
end
subgraph ResultLayer["网站返回结果"]
NormalView["返回真实正常数据\n(如: 门票充足 / 房价 $199)"]
DeceptionView["阻断或反制响应\n(CAPTCHA / 限流 / 注入假数据 $399)"]
end
Human --> WAF
Agent --> WAF
WAF -- "判断为人类" --> NormalView
WAF -- "判定为自动化机器人" --> DeceptionView
```
科幻电影《银翼杀手》（Blade Runner）曾探讨智能仿生人与人类的界限，如今的互联网正在经历类似的演变。互联网最初完全是为人类设计的，但如今自动化系统与 AI 智能体的访问量已出现爆发式增长。Cloudflare 首席执行官曾公开证实，来自智能体系统的流量已经正式超过了人类流量。人类购买相机通常只会浏览几个页面，而一个代表用户决策的 AI 智能体可能会瞬间并发访问 100 个网站并触发大量操作。面对这种冲击，旧有的“人类互联网”开始全力反击，频繁触发 CAPTCHA 验证码与流量限制（throttling）。虽然这些防御最初是为了抵御黑客批量注册等恶意攻击，但如今却将作为人类延伸工具的正常 AI 智能体阻绝在外。
### [08:09](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=489s) 网站开始“说谎”：虚假数据污染与 AI 智能体的高速规模化失败
相比传统的阻断或限流，过去几个月到几年间出现了一种更隐蔽且危险的新趋势——网站开始主动向机器人“喂送虚假数据”（Disinformation）。当目标网站通过行为特征判定访问者并非人类时，它不再直接返回错误码，而是返回排版工整但数值完全错误的垃圾数据，即“互联网学会了说谎”。
* **门票聚合场景**：真实人类访问票务网站（如 tickets.com）能看到余票充足；而比价智能体访问时，网站会伪造“已售罄”的信息，导致聚合平台向用户误报无票。
* **酒店比价场景**：酒店为了鼓励用户直接在其官网预订以节省渠道佣金，人类访问官网能看到 $199 的真实优惠价；而第三方比价智能体抓取时，网站故意返回 $399 的虚高价格，导致用户比对后对比价产品产生信任危机并彻底流失。
这种虚假数据之所以极具破坏性，是因为 AI 智能体与人类的失败机制截然不同。人类员工如果看到离谱的数据会有直觉并停下来核实，但 AI 智能体缺乏这种常识直觉，它只会高速且大规模地犯错（"Fail fast and at scale"）。智能体在读取到错误的 $399 价格后，可能会在几秒内自动向 5,000 名客户群发降价/调价通知邮件，引发极其严重的“垃圾进，垃圾出”（Garbage In, Garbage Out）连锁灾难。
### [11:41](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=701s) 数据腐化与时效性衰减：陈旧数据如何摧毁用户体验
即使抓取到的数据真实无误，如果缺乏持续的实时更新，陈旧的数据（Stale Data）同样会迅速毁掉智能体产品的用户体验。例如，购物比价智能体如果保存着过时数据，向用户展示某款耳机售价为 $29.99，当用户点击跳转到零售商实际页面时发现已涨至 $44.99，或者智能体推荐了一家在 Google 地图上已标记为“暂时歇业”的餐厅，用户到店后面对紧闭的大门，都会对产品产生极大的挫败感。
```mermaid
flowchart LR
accTitle: 各类网络数据半衰期与时效性衰减速度对比
subgraph FastDecay["极速衰减区 (1 - 2 天内失效)"]
Social["社交媒体动态"]
Ecom["电商价格与库存"]
News["新闻时事资讯"]
end
subgraph MidDecay["中速衰减区 (5 - 7 天)"]
Finance["金融市场信息"]
RetailDesc["产品规格与评价"]
end
subgraph SlowDecay["慢速衰减区 (长周期变化)"]
GeneralWeb["通用网页 / 博客文章"]
end
FastDecay --> MidDecay --> SlowDecay
```
数据衰减（Data Decay）是客观存在的规律。Bright Data 每日抓取约 500 亿个 URL，这一吞吐量约为谷歌全球公开搜索量的三倍，对应着每分钟处理约 3500 万个 URL，这使得公司能够精确量化不同数据类型的时效性半衰期（Time to Staleness）：
- **社交媒体与电商价格/库存**：更新极快，在极短时间内（甚至 1 天内）就会失效并沦为误导信息。
- **新闻资讯**：生命周期通常维持在 1 到 2 天。
- **金融市场信息**：有效周期通常在 5 到 7 天左右。
- **零售产品描述与用户评论**：变动相对平缓，衰减周期较长。
- **通用网络内容（博客等）**：更新频率极低，衰减速度最慢。
### [14:33](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=873s) Bright Data 的基础设施底座与极客公司文化
为了解决网络阻断、数据造假及数据老化问题，必须建立专业的 Web 数据基础设施（Web Data Infrastructure）。Bright Data 构建了涵盖全球超过 2 亿个住宅与数据中心 IP 的网络，并且在任意时间点都有超过 250,000 台无头浏览器在线运行，模拟真实人类点击网页组件、输入文本框等交互行为，从而确保目标站点将请求识别为合法的人类操作。
在组织运作上，Bright Data 展现出高度工程驱动的极客文化：
- **无会议制度**：全公司约 500 名员工，仅设有一间会议室，且该会议室仅限接待外部供应商，所有内部协作均通过临时自发沟通完成。
- **零 QA 团队**：全公司没有任何专职 QA（测试）人员，测试流程全部由自动化系统接管，团队坚信传统 QA 会大幅拉低研发迭代速度。
- **极高发布频率**：SaaS 平台每天进行约 100 次线上生产代码发布。
- **核心模型支撑**：目前市面上 75% 的前沿大模型与主流 AI 聊天机器人，在执行“正在搜索网络（searching the web...）”指令时，底层均通过 Bright Data 的数据基础设施进行实时检索与内容抓取。
### [16:32](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=992s) 法律诉讼胜利与新一代 Agent 基础设施架构
在高速创新的过程中，数据采集技术往往会走在现有法律框架之前。Bright Data 曾先后遭到 Facebook（Meta）和埃隆·马斯克（Elon Musk）旗下公司的起诉，指控其抓取公开网络信息。公司投入数百万美元应诉，最终在两起诉讼中均取得了毫无争议的完全胜诉（非庭外和解）。这一系列司法判例在法律层面上明确确立了“抓取网络公开数据属于合法行为”的行业准则，成为当前诸多 AI 企业在公开数据采集诉讼中引用的判例基石。
面对上层激增的 AI 智能体生态与底层旧有人类万维网之间的摩擦，Bright Data 将自身定位为承上启下的结构化数据中间层。该系统提供高并发 API 端点（支持从单秒 1 次到 1,000 次请求），能够将目标网页转换为结构化数据输出，支持实时网络搜索、全网爬取、深层网络归档库调用，并全面适配 MCP（Model Context Protocol）与各类主流开发 SDK，使开发者无需深陷反爬对抗，专注于智能体本身的业务逻辑构建。
### [19:02](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=1142s) 演讲总结与 BattleBots 展区邀请
演讲者在总结时指出，当今的 AI Agent 已经具备了极强的行动能力，底层模型本身非常优秀，但痛点在于“如今的网络环境已经学会了撒谎”。为了构建一个可扩展且值得信赖的可靠系统，必须为其注入经过验证的真实数据，而这正是 Bright Data 所扮演的核心角色。在进入问答环节之前，演讲者分享了一个轻松的花絮：Bright Data 是 BattleBots（博茨大战 / 机器人格斗联盟）的赞助商。
演讲者热情邀请在场观众前往 813 号展位体验操控真实的格斗机器人，打趣道大家可以“在不伤害任何人的情况下宣泄一下暴力情绪”。此外，Bright Data 当晚还将在 BattleBots 竞技场举办专属活动，若有剩余名额欢迎大家报名参与。
- **核心观点**：AI Agent 自身执行能力极强，但受制于网络虚假信息的干扰，必须依赖可信数据源保障决策准确性。
- **展位互动**：813 号展位提供 BattleBots 实体机器人操控体验，当晚在 BattleBots Arena 举办专场活动。
---
### [20:04](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=1204s) 问答环节：大规模数据验证与合规伦理机制
现场观众提出的第一个问题聚焦于数据质量评估：“网络上充斥着虚假新闻和伪造数据，对于使用者而言，数据的可靠性与实时性同等重要，Bright Data 是如何评估并确保抓取数据的真实可靠？”演讲者解释称，大规模数据采集最大的优势在于能够“在多个时间点、从多个维度交叉观察同一事物”。当系统检测到异常情况时，会自动触发警报机制，并立即切换不同的国家来源、IP 地址、浏览器代理或设备指纹进行多重交叉验证，以判断数据突变仅仅是瞬时抖动还是真实的全局变化。
```mermaid
flowchart TD
accTitle: Bright Data 数据异常验证与合规采集流程
subgraph S1["数据采集与监测"]
A["多维度实时数据抓取"] --> B{"检测到数据异常/价格突变?"}
end
subgraph S2["多维交叉验证"]
B -- "是" --> C["切换 IP / 国家地区"]
B -- "是" --> D["更换浏览器指纹与 Agent"]
C --> E["确认是否为局部限制或真实变更"]
D --> E
end
subgraph S3["伦理与合规控制"]
F["实时监测目标网站响应延迟"] --> G{"网站是否变慢?"}
G -- "是" --> H["主动降速 / 暂停采集"]
G -- "否" --> I["维持正常采集节奏"]
end
```
针对数据变动场景，演讲者举例说明，如果监控到某电商站点上某个特定数据点的价格发生了剧烈变动，系统有能力验证这一变化是仅针对特定国家用户的定向策略，还是面向所有用户的普遍调整。此外，演讲者强调了底层技术基础设施在避免被识别为 Bot 方面的关键作用，并特别指出了大规模采集中的伦理与合规边界：系统会实时监测目标网站的响应时间，一旦发现采集行为导致目标站点访问变慢，系统会主动退让（Back off）甚至完全暂停采集，以确保“绝对不干扰目标网站的真实正常用户”。
- **多维交叉验证**：利用全球分布式网络，从不同国家、IP 与指纹环境对比同一数据点，有效识别地域性差异与虚假信息。
- **负责任的数据采集（Ethical Scraping）**：根据目标服务器负载与响应时间动态调整并发速率，保障目标站点的稳定性与合规性。
---
### [21:43](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=1303s) 问答环节：全栈垂直整合的产品架构
第二位观众询问了 Bright Data 的核心产品形态：究竟是直接向客户交付抓取好的特定网站数据，还是重点提供轮换住宅代理（Rotating Residential Proxies）等底层工具供客户自行开发爬虫？演讲者回答道，Bright Data 构建了一整套自底向上的垂直整合技术栈，能够根据客户需求提供不同层级的接入方式。
```mermaid
flowchart TD
accTitle: Bright Data 垂直整合技术栈架构
subgraph Infrastructure["底层网络基础设施"]
A["全球代理网络\n(住宅代理 / 机房代理 / ISP)"]
end
subgraph Unlocking["网络解封层"]
B["自动化网站解封算法\n(绕过反爬机制 / 提取 HTML)"]
end
subgraph Processing["解析与交付层"]
C["数据解析与结构化引擎"]
D["交付模式\n(原始代理通道 / API / CSV / JSON)"]
end
A --> B
B --> C
C --> D
```
演讲者阐明，对于只需要网络通道的开发者，平台可以仅提供最基础的代理 IP，由客户自行编写并维护后续的所有逻辑；而对于只关注最终业务结果的企业，平台可以直接通过 API 或以 CSV/JSON 文件的形式交付清洗后的结构化数据，客户完全无需关心底层使用了何种代理或解封技术。这种自底向上的垂直整合能力是 Bright Data 的核心优势，业界诸多知名的搜索引擎公司正是依托其底层基础设施来实现对网络数据的实时访问。
- **代理层（Proxy Tier）**：覆盖全球多类型 IP（住宅、数据中心等），提供纯粹的网络传输管道。
- **解封层（Web Unlocking Tier）**：内置算法自动处理目标站点的反爬防御与验证机制，稳定获取完整 HTML 页面。
- **数据交付层（Data Tier）**：将网页内容自动解析为结构化字段，通过标准化 API 或文件格式无缝输出。
---
### [23:17](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=1397s) 问答环节：AI 驱动的 Scraper Studio 与代码自愈功能
针对观众关于大型主流网站（如房产平台 Zillow）爬虫支持及未知冷门网站采集的提问，演讲者表示，公司拥有庞大的预构建爬虫库，覆盖了市面上几乎所有主流网站。而当面对从未收录过的未知网站时，用户可以使用控制台内的 AI 工具——**Scraper Studio**。
用户只需在 Scraper Studio 中输入目标域名，系统便会自动访问该网页、抓取 HTML 并开启交互式对话。系统会自动分析页面结构并向用户展示拆解后的数据 Schema（例如将页面划分为价格、评论、商品描述等字段），用户只需通过几轮简单的自然语言确认，系统即可在“零代码开发门槛”的情况下自动生成专属爬虫代码，其运行效果与官方维护的爬虫完全一致。
更关键的是，针对网页结构频繁改版导致爬虫失效的行业难题，平台引入了**代码自愈（Self-Healing）**技术。当目标网站的前端架构发生变化时，背后的 AI 引擎会自动调整抓取规则与解析代码，用户侧无需进行任何手动维护，即可持续稳定地接收格式一致的结构化数据。
- **预置爬虫库**：开箱即用，全面覆盖包括 Zillow 在内的主流行业网站。
- **Scraper Studio**：AI 驱动的低代码/零代码开发环境，通过对话式交互自动推断 Schema 并生成爬虫脚本。
- **自愈机制（Self-Healing）**：网页前端改版时底层解析逻辑自动动态调整，确保对外数据格式和输出流的连续性与稳定性。
## Notable quotes
> [02:05](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=125s) 竞争对手必须自行构建爬虫、抓取网页并提取信息，必须通过自身的投资与创新来与谷歌竞争。 — 美国司法部反垄断诉讼法官
> [02:05](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=125s) 脱离了实时互联网访问的应用就像一本静态的书 — Ariel Shulman
> [04:48](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=288s) 在微观尺度下物理定律会失常，而网络数据抓取则会在大规模并发（生产环境）下彻底失效。 — Ariel Shulman
> [08:09](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=489s) 互联网学会了说谎 — Ariel Shulman
> [08:09](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=489s) AI 智能体缺乏这种常识直觉，它只会高速且大规模地犯错（Fail fast and at scale）。 — Ariel Shulman
> [14:33](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=873s) 全公司没有任何专职 QA（测试）人员，测试流程全部由自动化系统接管，团队坚信传统 QA 会大幅拉低研发迭代速度。 — Ariel Shulman
> [19:02](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=1142s) 当今的 AI Agent 已经具备了极强的行动能力，底层模型本身非常优秀，但痛点在于“如今的网络环境已经学会了撒谎”。 — Ariel Shulman
> [20:04](https://www.youtube.com/watch?v=dVZVCk8TZPU&t=1204s) 在大规模采集中的伦理与合规边界：系统会实时监测目标网站的响应时间，一旦发现采集行为导致目标站点访问变慢，系统会主动退让（Back off）甚至完全暂停采集，以确保“绝对不干扰目标网站的真实正常用户”。 — Ariel Shulman
## Mentioned
Bright Data, Ariel Shulman, Google, Gemini, Spark, Waymo, Tim Berners-Lee, Orson Welles, Yahoo!, Ask.com, MSN, Ben Affleck, Cloudflare, Meta, Elon Musk, BattleBots, Zillow, Scraper Studio, Model Context Protocol (MCP)
## Video description
Ariel Shulman - Chief Product Officer, Bright Data
For decades, the web was Google's domain, organizing the chaos for humans. Now AI agents are taking over - but they're only as good as what they can see, and they often get blocked or work off stale data. They then make confidently wrong decisions, faster than any human could. This session is about fixing that - giving agents real-time, reliable eyes on the world, so they stop automating mistakes and start delivering results you can actually trust.
[Source](https://www.youtube.com/watch?v=dVZVCk8TZPU)
Source quotes
[2:05] 美国司法部反垄断诉讼法官: 竞争对手必须自行构建爬虫、抓取网页并提取信息，必须通过自身的投资与创新来与谷歌竞争。
[2:05] Ariel Shulman: 脱离了实时互联网访问的应用就像一本静态的书
[4:48] Ariel Shulman: 在微观尺度下物理定律会失常，而网络数据抓取则会在大规模并发（生产环境）下彻底失效。
[8:09] Ariel Shulman: 互联网学会了说谎
[8:09] Ariel Shulman: AI 智能体缺乏这种常识直觉，它只会高速且大规模地犯错（Fail fast and at scale）。
[14:33] Ariel Shulman: 全公司没有任何专职 QA（测试）人员，测试流程全部由自动化系统接管，团队坚信传统 QA 会大幅拉低研发迭代速度。
[19:02] Ariel Shulman: 当今的 AI Agent 已经具备了极强的行动能力，底层模型本身非常优秀，但痛点在于“如今的网络环境已经学会了撒谎”。
[20:04] Ariel Shulman: 在大规模采集中的伦理与合规边界：系统会实时监测目标网站的响应时间，一旦发现采集行为导致目标站点访问变慢，系统会主动退让（Back off）甚至完全暂停采集，以确保“绝对不干扰目标网站的真实正常用户”。
Source quotes
For decades, the web was Google's domain, organizing the chaos for humans. Now AI agents are taking over - but they're only as good as what they can see, and they often get blocked or work off stale data.
[14:33] Ariel Shulman: Searching the web...
[19:02] Ariel Shulman: get some violence out of your system without hurting anyone
Original transcript
[0:04] So in 1938, there was no internet, there was no
television, there was only radio. And Orson Welles
[0:10] did a radio show where he basically fooled people
into thinking that Martians are invading the Earth
[0:15] and he caused mass hysteria. Now, like 90 years
later, it is AI agents that are being fooled.
[0:23] Um and that's what we're going to talk about here
today. Uh before jumping in, I'd like to just uh
[0:28] give you some historical perspective of how we
got to where we are today. So this guy, I don't
[0:34] know who knows who this guy is. This guy is Tim
Berners-Lee. He is the inventor of the internet.
[0:38] This guy single-handedly invented the URL, HTTP,
HTML, the first web browser, the first web server,
[0:46] and uh we should all thank him because without
him we would not be here today. He also did the
[0:51] right thing by open-sourcing the whole thing in
1993, which made the internet what it is today.
[0:57] Then in 1994 and on, the internet started
growing and people started to try to index
[1:04] what the internet was had. So this is the first
one, Yahoo, two students from Stanford University,
[1:11] and it's interesting to see that they had
23,000 sites categorized. They actually did
[1:17] that manually. So they had to look at site by site
to categorize that manually. As things progressed,
[1:22] we have more sites, for example, ask.com, which
just announced that they're ceasing operations.
[1:28] And then uh the era of portals, which now looks a
little bit cringey. Uh this this user interface,
[1:35] it's horrible. You had everything. You had a
search page, you had some information. Uh it's
[1:39] interesting to see that Nasdaq was 1,940. Right
now it's like 26,000, so we did good. And around
[1:47] 2000 or '98, everyone got MSN. If you remember,
every computer came with msn.com. That was that
[1:54] like the go-to homepage. Uh again, another attempt
at organizing information, and you can see here
[1:59] Ben Affleck smiling and happy, probably because he
was happy they didn't win his picture at the time.
[2:05] Uh, but the real jump forward was with Google.
So, who here has used in the last 24 hours
[2:14] YouTube or Android or Google Maps or Gmail or
anything like that? Everyone, right? Everyone.
[2:20] So, Google has a huge advantage in terms of
data. They have all the text in the world
[2:26] from all the websites that they index. They have
images from Google Photos. They have videos from
[2:30] YouTube. They have audio from YouTube as well and
e-commerce data and local data from Google Maps.
[2:36] And they have user behavior from sensors
inside Android phones, for example,
[2:41] and health data and knowledge and even physical
data from cars. Waymo is owned by Alphabet
[2:47] and things like that. So, Google has a
ton of information, which they are using
[2:52] for their models to train Gemini, which I think
everyone can agree has really improved over the
[2:57] last few years. And recently they announced that
they also be will be providing users with agents
[3:03] that will be able will take actions on behalf of
users. Uh, something called that they call Spark.
[3:09] So, a ton of data that Google has under their,
you know, in their hands. This is not something
[3:17] that went unnoticed. So, the DOJ, Department of
Justice, actually brought a case against Google.
[3:23] Kind of an antitrust case that's saying you
have too much information, that's unacceptable.
[3:27] And I'm not a lawyer, okay? It's really important
to stress that. I'm not a lawyer. But basically,
[3:33] I read the the opinion, which was given
in 2025, and the ans- the opinion says
[3:38] that Google needs to share some data only once,
but they keep all the secret sauce. For example,
[3:45] how do they do the page ranking and all sorts
of things like that. And that builders, and
[3:49] everyone in this room is a builder, still needs
to build stuff themselves. So, to quote the judge,
[3:56] competitors will have to build crawlers,
crawl web pages, extract information.
[4:01] They'll have to in you know, invest and innovate
to compete with Google. That was the outcome. So,
[4:07] why should you guys care? Why is this interesting
for people like you? Because you build stuff.
[4:14] And for products to be useful, they need to have
internet access. Without internet access, if you
[4:18] build some kind of a product, it's like a book.
It's not It's not dynamic. It's not interesting.
[4:23] So, product needs to have uh internet the ability
to access the internet, and the data that they
[4:28] collect from the internet must be reliable and
fresh and scalable. Otherwise, it's doesn't work.
[4:34] So, this is what we do in Bright Data. We provide
you with the ability to compete and deliver you
[4:41] uh what we call web data infrastructure
so that you get the data that you need.
[4:48] So, what is this web data infrastructure and
why do you need it anyway? The concept of
[4:53] getting in information from the internet is sounds
simple, but the problem is it doesn't scale. So,
[5:00] I like to make the analogy something like this.
In quantum mechanics, as you probably know,
[5:05] things go crazy when you go on a very small
scale, right? The laws of physics go crazy.
[5:11] In web scraping or web access, the behavior
changes at the large scale. In other words,
[5:17] things will break in production. If you have
something that's working for you as a pilot, maybe
[5:23] the few users or on your machine, when you go
live, it's going to go crazy. We know that, okay?
[5:28] It's going to get blocked, and we're going to talk
about those things and how we can help you. And
[5:32] what happens with web data infrastructure is it
fixes this. It allows you to get this consistent,
[5:37] reliable access to the internet. So, you know,
these days everyone is a builder. I'm sure that
[5:42] everyone here has a vibe coded something. So,
let's look Look this something like this. Some
[5:46] kind of a fictional company with hotel and the CEO
over the weekend sends the team a message, "Hey,
[5:53] I built a pricing agent that scrapes information
from Hilton, from all of the other hotels,
[5:58] and it works great. And let's ship it today." And
just like my product also said here, it's a bad
[6:04] idea. You should probably not do that. And I think
that the thing that this guy built will work. So,
[6:10] the interface will work. The user interface
will work. The logic will work. But, it will
[6:15] fail. It will fail in production for sure. That's
what's going to happen. And one of the, you know,
[6:23] one of my favorite movies in the world is called
Blade Runner. And Blade Runner is a movie about
[6:28] our bots or like intelligent bots, human or
not. And this is something that's very kind of,
[6:34] you know, it's kind of a interesting question.
And I apologize cuz I've been blocked, but
[6:40] basically the internet was built for
humans. And in the last few years,
[6:44] more and more automated systems are using
websites, more and more bots. Uh so, from now on
[6:52] agents are using the internet more than humans do.
In fact, we have CloudFlare's CEO, they see a lot
[6:56] of traffic. CloudFlare's CEO went on record saying
that they have seen that the traffic coming from
[7:01] agents has increased to the point that it exceeded
human traffic. And that, if you think about it,
[7:06] makes sense because if you go out and you want to
say buy a camera, then you would click on a few
[7:12] sites, and that's it. But, if you ask an agent,
"Which camera is good for me?" that agent could
[7:17] go to a hundred websites, okay? And click on all
sorts of things and do all sorts of things. So,
[7:25] the the old web fights back. Okay, the human web
fights back. Who here has seen a CAPTCHA in their
[7:32] life, right? Or tried to do something and got a
message that they are being throttled or slowed
[7:38] down. Those things are happening. Websites
are they don't really know what to do with
[7:43] automated traffic. A lot of the systems in place
are there to protect websites against malicious
[7:49] traffic. So against a bot that will try to open a
thousand accounts, okay? Or something like that.
[7:54] But if you if I have an agent and it's taking
an action, it's kind of kind of an extension
[7:59] of myself. It's not something that uh you
know, malicious. So, there's always been
[8:05] uh what we call throttling or blocking, but
the interesting things that we've been seeing
[8:09] lately are actually around fake data. Okay?
This information. So, what do I mean by that?
[8:17] For many, many years, websites have been
blocking scrapers or blocking automated access,
[8:22] as you saw with a CAPTCHA and things like
that. But in the last few months or years,
[8:27] we started seeing in Bright Data that websites
are doing something different. They are feeding
[8:34] wrong information. So, if you look at this
simple flowchart, someone goes to the website,
[8:38] the website try to analyze if this is a human or
or not. And if the website decides that this is
[8:44] not a human, it will feed them garbage, okay?
It looks correct, but it's wrong information.
[8:52] So, in other words, you know, the web has learned
to lie. And the and this is a problem, okay?
[9:00] This is a big problem and we're going to see
a few examples and uh and I will explain why.
[9:04] So, let's say that you have a website that
you built uh that collects information and
[9:10] allows your users to find tickets to different
concerts on the web, okay? That's your own site.
[9:16] When a human goes to the original ticket
site, let's say tickets.com, they're going
[9:20] to see that the website has tickets and there's
no problem. When your agent goes to that website,
[9:26] they may the agent might see no tickets
available. Okay? Which is fake. It's incorrect.
[9:32] You will report on your own site that there are
no tickets available. Your user will be frustrated
[9:37] because he there are no he will think that there
are he she that he will think there are that there
[9:40] are no tickets, but there are tickets. Another
example is if you have you are going back to the
[9:46] example we had before of a Wiz Hotel. You have
some kind of a agent that compares hotel prices.
[9:51] When humans go to the hotel site in this case,
let's call it xyzhotel.com or hilton.com,
[9:57] doesn't matter. They're going to see a price of
$199. So, as you know, hotels prefer that you
[10:03] book directly with them because they you that they
don't want to pay commission to someone else who
[10:08] they're extracted to their site. So, if your
agent from this Wiz Hotel goes to this website,
[10:13] the agent might see that the price is 399. And
that's again incorrect. So, if your user sees the
[10:20] price at 399 and goes to hilton.com or xyz.com and
sees 199, they will automatically lose confidence
[10:27] in your system and just get you know, it's a
bad system. I'm not going to use it anymore. So,
[10:33] the the the problem with with this is that
AI agents fail very differently from humans.
[10:42] Okay? Because if a human sees that the price is
399 and it's supposed to be something like $200,
[10:48] the human is going to kind of slow down
a little bit, okay? We have intuition.
[10:52] We don't want to disappoint our superiors. We we
think a little bit. But [snorts] AI agents don't
[10:58] operate like that. They can fail very fast and at
scale. So, it can be actually quite satisfying to
[11:06] see when it happens to other people, not to you,
but uh it they they can fail at scale. Um so, for
[11:12] example, that AI agent can see a price of $399 and
automatically send 5,000 email to 5,000 customers
[11:18] saying, "Hey, the price has been updated to $399."
Which is wrong. But from the agent perspective,
[11:24] it just saw a number. It's still another number.
It's okay. Now, you have to deal with the fallout
[11:29] of 5,000 people who are upset because they got
incorrect information. So, this is very dangerous
[11:35] of what this is what we call garbage in, garbage
out, and you should be very careful about that.
[11:41] Now, even when you have correct information, to
have a useful agent that's valuable for users,
[11:50] you have to keep the information fresh because
information on the internet changes all the time.
[11:55] So, imagine for example that you have a shopping
agent that compares prices of different products,
[12:00] and your agent has old information, and the info
the agent says the price of these headphones is
[12:06] $29.99. Then the users click
through to the actual retailer site,
[12:12] and then they're surprised to see it's actually
$44.99. Again, very bad user experience,
[12:17] and this is because prices change, products go
out of stock, your agents needs to continuously
[12:24] refresh this information, otherwise
you risk providing bad service. So,
[12:29] another example is this guy who wanted to go to
a restaurant, okay, and asked the the the chat,
[12:36] as we say, "Which restaurant should I go to?" Got
recommended this restaurant, and he got there and
[12:40] it was closed. So, you you all know the temporary
closed flag that we are we we know on Google Maps,
[12:46] for example. If you don't check these things in
real time, you will disappoint your users. So,
[12:51] stale data, you know, kills user
experience for these agentic applications.
[12:57] So, this is kind of a nice concept, but let's
see what the what the data looks like. We We
[13:02] write data as a data company, so let's look at
that. This is what we call the data decay issue.
[13:08] To give you some perspective, every day we scrape
something like 50 billion URLs. 50 billion URLs is
[13:14] about three times the advertised volume of Google
searches worldwide. So, we scrape a ton of data
[13:23] for about 20,000 customers. Um that's about 35
million URLs per minute, And this gives us the
[13:28] ability to see how things evolve over time and
to measure the decay of data. And this is what
[13:34] it looks like. I'll leave it on a You'll see it
in a second, but you can see different lines.
[13:40] At the top there is super fresh, reliable data.
At the bottom is really bad data and they're
[13:45] uh, where x-axis is times, date. So, this is the
interesting part. What we call time to staleness.
[13:53] If you are trying to get information off of social
media or if you want to get e-commerce information
[13:58] such as prices and stock availability, this
stuff changes really quickly. So, after a while,
[14:04] one day it's no longer true and in fact, it's even
worse because it's misleading. You can actually go
[14:10] it it's probably better not to to use that
than to rely on it. News, one or two days.
[14:15] Financial information, five to seven days.
Retail, meaning descriptions of products
[14:21] or reviews or something like that, changes this
thing change a little bit slower. And obviously,
[14:26] general web, which is like blogs, things like
that, they basically change very very slowly.
[14:33] So, the the solution to all of these problems
that I mentioned of having reliable access,
[14:39] not to get blocked, not to get lied to and
to have fresh information is what we call
[14:43] web data infrastructure. We want you to get
data that's reliable, fresh and scalable.
[14:50] Um, so just a few words about the Bright
Data because after all, we're here for you to
[14:54] to get to know us. As I said, we collect over
50 billion pages per day. We do that with a
[15:00] huge infrastructure of over 200 million IP
addresses in different countries around the world.
[15:07] And at any given point in time, there are more
than 250,000 browsers online that are actually
[15:13] clicking on different elements in pages, entering
text into, you know, text boxes, doing all sorts
[15:17] of things to show or to make sure that the website
sees that this is a normal user and not a bot.
[15:25] At the same time, we are a very strange
company. We are about 500 people.
[15:31] Uh we don't do any meetings. We only
have one meeting room in the company and
[15:37] even that is only allowed for use with external
vendors. So, everything is done ad hoc.
[15:42] We have zero people working in QA. There's no QA.
It's all all of it is automated. We believe that
[15:47] QA slows down slows down uh things significantly.
And every day we release about 1,000 Sorry,
[15:54] about 100 code releases of our uh of our, you
know, general SAS to keep it uh up to date. Um
[16:03] in fact, when you buy stuff online, if
you booked a ticket, if you bought it,
[16:09] you know, or if you bought something or
you went to some price comparison site,
[16:13] if you went to a chatbot and it said searching the
web {dot} {dot} {dot}, a lot of that goes through
[16:18] our infrastructure. We serve 75 of the large AI
from frontier models in chatbots that you all use.
[16:24] That all goes That all goes through our
system under the hood. So, as part of,
[16:32] you know, innovating and doing things quickly,
sometimes we go faster than the legal framework.
[16:38] Uh so, this got us uh in trouble a little bit, uh
which is always fun. Uh we got sued by Facebook
[16:45] and Mark Zuckerberg when he had a different
haircut. Um in fact, Facebook was a customer of
[16:53] other companies, but they also sued us saying that
we're not allowed to collect information that's
[16:59] public. And the same thing with Mr. Musk, both
of these lawsuits happened and I'm happy to say
[17:06] that we won both of these lawsuits and I'm not
talking about settlement. It's like clear-cut one.
[17:12] Um we invested millions of dollars in this, okay?
Extremely important, but the fact that we won
[17:19] has actually made it very, very clear that
it's totally okay to collect publicly available
[17:25] information. It made it clear what's okay,
what's not okay. And these verdicts and these
[17:30] opinions are actually used these days in many of
the cases that you hear read about in the news
[17:35] for, you know, AI companies being sued and
things like that with respect to what's
[17:38] allowed and what's not allowed when collecting
a publicly available web data. So, to wrap up,
[17:47] we have at the bottom here what we call the old
World Wide Web, which was built for normal people
[17:53] like us, okay, who see a website with designs and
things like that. Then you have the new internet
[17:59] or the new agent universe with different AI
agents. They can be the big ones that you know,
[18:04] or they can be your own, okay, when that agent
that you have built developed needs any kind
[18:09] of internet access. Normally, these new agents
will get blocked or get garbage data at scale.
[18:17] And this is the surprising part. You don't see it
when you're piloting and you're all excited about
[18:22] your shiny new agent. It's going to happen at the
worst possible time when you have some real users.
[18:27] The way we help here in BrightData is we replace
the old layer with our technology. I don't have
[18:35] much time left. I could talk all day about this,
but we have API endpoints that allow you to
[18:41] send one request per second or a thousand requests
per second to any site that you want. We will know
[18:46] how to get you the data in a structured format.
It can So, you can scrape websites, you can search
[18:52] the internet, you can crawl websites to see,
you know, basically discover the whole thing.
[18:57] We have a huge web archive. Obviously, everything
is accessible via MCP and SDKs and all of those
[19:02] things. So, again, I could go on about this
all day, but we we don't have time. So,
[19:09] I think to really summarize, we have now AI
agents that know how to act, very capable. The
[19:15] models are very good. They know how to act. The
web, as we know it, kind of learned how to lie.
[19:22] And to make sure that you have a reliable system
that's, you know, scalable and trusted, you need
[19:26] to give it trusted data. And this is where Bright
Data comes in. So, one last thing, just for fun.
[19:34] Bright Data is a sponsor of the BattleBots League.
I'm not sure if you guys are familiar with the
[19:38] BattleBots. So, first of all, you're all welcome
to come to our booth, 813, where you can actually
[19:44] play with the BattleBots and, you know, get some
get some violence out of your system without
[19:48] hurting anyone. Which is nice. And we also have an
event tonight that we're hosting in the BattleBots
[19:53] arena. So, if we do have space left, please
come in and we'll see if we can accommodate you.
[19:57] I see that I have 2 minutes left. So, if
there's any questions, I'm happy to take them.
[20:04] Thank you for the presentation. I'm
wondering how you evaluate the data
[20:08] that you scrape from the web, but you mentioned
that there's many fake news, fake data. For us,
[20:13] reliable data is very important. It's not only
real-time, but also reliable data. Mhm. Can you
[20:19] elaborate how you evaluate Bright Data data? Okay.
So, one of the advantages of doing things at scale
[20:25] is that you can see the same thing from
multiple directions and at multiple times.
[20:30] So, we have the ability to look at the other
information. If we detect some anomalies,
[20:33] then we would basically have kind of our own
flags and we will kind of double-check that
[20:38] maybe from another country, another IP
address, or using another browser agent,
[20:42] or another fingerprint, or something like that,
to see if it was a blip, or maybe something has
[20:47] changed over time. So, let's say that we
see a some kind of significant price change
[20:51] on a specific site where we collect information
and parse it into the different, you know,
[20:57] data points. If we see that something changed,
we have the methods to actually see if that was
[21:01] something that only occurs, for example, for
a specific country or for other, you know,
[21:07] on a on the larger scale. But, I think the
important thing is actually to have the
[21:12] technology and the infrastructure to not even
be detected as as a bot. And we have to do this
[21:18] in a very responsible way because we are, you
know, collecting a lot of information at scale.
[21:23] So, one of the things that we do, for example,
is we look at the response time of the website.
[21:27] If the website starts slowing down as we collect
information, we will back off or maybe even stop
[21:33] completely because we never want to get in
the way of the real users of that website.
[21:37] So, there's many, many elements to keep this,
you know, ethical, compliant, and responsible.
[21:43] Thank you for the presentation. Just a quick
question. I guess what's your primary product? Are
[21:47] you guys servicing customers by scraping specific
websites for them? Or do you guys also focus on,
[21:56] you know, giving people the ability to do the
scraping themselves with the rotating residential
[22:01] proxies? Okay, good question. So, the we built
a technology stack which started with proxies
[22:07] of different types and different countries. On top
of that, all sorts of algorithms that allow us to
[22:13] what we call unlock the website or to get
the content or the HTML of the website.
[22:19] Then you once you have that HTML, you can parse
it into the relevant pieces of information.
[22:24] We provide access to every one of those tiers
individually for customers as they choose. So, if
[22:29] you want proxies, well, you can give you proxies.
If you want structured information, you don't
[22:33] you don't even care how the was it, you know,
obtained, if it was with this proxy or that proxy,
[22:38] we can give you an API that's going to give you
that structured information. And one of the things
[22:43] that help us internally in Bright Data is that
we have kind of the whole vertical integration
[22:47] of the entire system from all the way bottom up.
Many of the search companies that you know, for
[22:52] example, are using our underlying infrastructure
to access the data. So, the answer to your
[22:59] question is, depending on what you want, we have
either very basic access to some IP address and
[23:04] then you are responsible for doing everything
because you just have a pipe, basically. Or we can
[23:09] give you all the way up to like a CSV file or JSON
that we did everything needed to to to generate.
[23:17] So, they just to repeat the question quickly. Yes,
we if we if we have a scrapers for major websites,
[23:23] absolutely. There's a big library for all the for
all the you know, big websites that we think of,
[23:28] Zillow included as one of the one of
those. But if you come in with some
[23:32] unknown websites that we have not seen before,
we have a a segment inside our control panel
[23:37] which is called a scraper studio. And that
is where you can actually put in the domain.
[23:42] We will go there our our system will go there,
get the HTML, start basically chatting with you,
[23:47] show you the schema of the data. So, for example,
this is a page but I've broken broken this up into
[23:53] prices, reviews, you know, description, ask you if
this is okay, and with a couple of iterations with
[23:59] no technical knowledge will the scraping code
which converts the website to structure data
[24:06] and allow you to then run it just like you would
run a normal kind of officially maintained bright
[24:11] data scraper. In addition to that, because things
change all the time, there's also a layer that
[24:16] we call self-healing so that if the website
changes, and each website change all the time,
[24:21] as far as you're concerned, you would not
know that. The scraping code will adjust
[24:24] automatically so that you would continue getting
your structure data in the same format. So,
[24:27] that's called scraper studio. It's obviously AI
powered and it does the scraping code automatic.
[24:33] Okay. Thank you everyone. We have to stop the
meeting now. A round of applause. Thank you.
