Anthropic Pilots Independent Research Access to Claude Usage Data

Anthropic Pilots Independent Research Access to Claude Usage Data Anthropic partnered with external research groups to conduct independent studies on real-world Claude usage data via its privacy-preserving tool, Anthropic Insights, and released aggregate datasets from the pilot. Three external research groups—Stanford's SALT Lab, Oxford's Human Information Processing Lab, and METR—analyzed roughly 250,000 Claude.ai or Claude Code conversations from April–May 2026. Researchers only accessed aggregated category outputs through Anthropic Insights, with raw conversations remaining unread and subject to an independent privacy audit by Imperial College London. Anthropic limited contractual review rights strictly to user privacy, usage policy bypasses, confidential information, and research accuracy, allowing researchers to publish critical or inconvenient findings. The pilot faced operational friction, including query sensitivity and resource-heavy workflows, while filtering out fewer than 5% of categories that described how safeguards were circumvented. Pilot Architecture and Partner Studies Anthropic partnered with Stanford University's SALT Lab, the University of Oxford's Human Information Processing Lab, and evaluation nonprofit METR. Each team formulated independent questions and used Anthropic Insights to examine roughly 250,000 Claude.ai or Claude Code conversations from April–May 2026. Stanford investigated human-AI collaboration breakdowns, Oxford examined emotional states during interactions, and METR evaluated productivity effects from coding agents across model generations. Aggregate data from the projects was released publicly on Hugging Face. Privacy and Governance Guardrails External researchers never received direct access to raw interaction logs. Instead, Claude analyzed queries across conversations and aggregated responses into thematic categories. Contractual review rights were restricted to user privacy, confidential information, analytical accuracy, and prevention of usage policy violations. Imperial College London conducted a third-party privacy audit on the data shared with external teams to verify that protections held before public release. Operational Challenges and Methodology Gaps Anthropic Insights is sensitive to prompt wording, meaning poorly phrased questions can aggregate conversations into misleading buckets. While internal teams iterate on questions over weeks, external partners attempted dry runs on the public WildChat dataset; however, WildChat's casual usage patterns differed significantly from Claude's real-world traffic, causing certain questions to fail on production data. Anthropic addressed this by issuing interpretation guidance and exploring earlier refinement processes for external queries. Policy Misuse and Future Expansion When outputs revealed acceptable use policy violations, Anthropic shared the aggregate trends with researchers, withholding only the specific methods used to circumvent safeguards. Fewer than 5% of clusters and conversations were affected across studies, with altered or removed categories disclosed to the researchers. Because the review process was resource-intensive and slower than standard AI lab research cadences, Anthropic is gathering feedback to determine whether and how to scale the program to additional researchers.

Anthropic 启动试点项目 支持外部学者独立研究 Claude 实际使用数据 Anthropic 宣布完成一项试点项目，通过其隐私保护分析工具 Anthropic Insights 向三家外部研究机构开放真实的 Claude 聚合使用数据。该计划旨在打破 AI 实验室对真实交互数据的垄断，在严格保护用户隐私的前提下赋能独立学术研究。 Anthropic 联合斯坦福大学 SALT 实验室、牛津大学 HIP 实验室以及非营利机构 METR，针对 2026 年 4 月至 5 月期间约 25 万次 Claude 对话展开独立分析。 研究人员仅能访问聚合数据而无法接触原始对话，外部合同规定研究团队享有独立发布权，Anthropic 仅在隐私、安全违规、保密信息和研究准确性方面保留审核权。 伦敦帝国理工学院对共享数据进行了第三方隐私审计，确认符合既定的隐私威胁模型。 各研究组输出的数据中涉及政策违规的类别不足 5%，Anthropic 过滤了涉及绕过安全防护机制的具体手段，仅保留违规尝试行为的聚合信息。 打破数据壁垒与独立合作模式 目前 AI 真实交互数据主要集中在少数机构内部，外部学者往往只能依赖可能偏向日常娱乐用途的公开数据集。Anthropic 与斯坦福大学、牛津大学以及评估前沿模型的 METR 合作，使研究人员能够围绕人机协作、用户情感反应以及代码智能体生产力等课题自主设计问题并展开研究。相关聚合数据集已在 Hugging Face 公开发布。 隐私保护机制与方法学挑战 试点依赖前身为 Clio 的 Anthropic Insights 工具，由 Claude 对对话进行自动化打标签并输出类别比例。由于无法直接查看原始对话以校验分类准确性，外部团队曾在公开数据集 WildChat 上预测试问题，但因语料特征差异导致部分分类在实际数据中产生偏差。此外，由于每轮共享前均需进行严格的法律与隐私审查，整个试点流程进展较慢且耗费资源，对未来的规模化推广构成了挑战。 滥用透明度与后续拓展规划 在分析输出中，出现违反使用政策行为的类别与对话比例均在 5% 以下。Anthropic 披露了大部分违规统计，但对涉及具体规避防护手段的内容进行了剔除或修改，并同步提交安全团队复核。Anthropic 正通过问卷调研外部学界需求，以评估未来如何在确保安全与隐私的前提下进一步扩大项目规模。

*Earlier this year, we ran a pilot giving external researchers access to aggregate, real-world Claude usage data. Three research groups designed their own studies for Anthropic Insights, our privacy-preserving analysis tool; we ran the data collection on their behalf, and they conducted their own independent analysis. In this post, we share high-level results from those studies and what we learned running this pilot. We’re also providing an [expression of interest form](https://docs.google.com/forms/d/e/1FAIpQLScujFA2mJXnLhrvE7wazD7drfHlfRmfnrjhMZRpeIFW_b6I7Q/viewform?usp=sharing&ouid=108982467493638405621) for researchers who may want to work with us in the future.* Ensuring the transition to transformative AI goes well requires understanding its impact on people and society. Right now, data on real-world interactions with AI is concentrated in a handful of labs. We think it would be good if more data was made widely available—to researchers, policymakers, and the general public. Researchers outside the labs have two options. They can draw on analyses the labs publish, which reflect real usage but often answer the lab’s questions, rather than their own. Or they can use public datasets, which they can study however they like, but skew toward more casual use, and may not reflect how most people actually use AI. Neither is sufficient for independent research on how AI is actually being used. This spring, we piloted a program in which three external research institutions designed and ran their own studies on Claude usage data through [Anthropic Insights](https://www.anthropic.com/research/clio) (formerly named ‘Clio’), the privacy-preserving tool our own teams use to analyze usage patterns across millions of Claude conversations. We hope to scale this program in the future, so we also conducted an additional privacy audit of all data shared with third-party researchers to verify that our privacy protections held (see [Appendix](https://www-cdn.anthropic.com/files/4zrzovbb/website/8a665c85eec3a63b4d86287b9255657016f50e29.pdf)). We believe this is the first time external researchers have run public independent studies on an AI company's own usage data. Below, we discuss what the external teams found, what we learned running the pilot, and what we are weighing as we decide how to expand the program more widely. We are also publicly releasing [the aggregate data from each project](https://huggingface.co/datasets/Anthropic/enabling-independent-research). We partnered with three research groups: the [Social and Language Technologies (SALT) Lab](https://cs.stanford.edu/~diyiy/group.html) at Stanford University, the [Human Information Processing Lab](https://humaninformationprocessing.com/) at the University of Oxford, and [METR](https://metr.org/), a non-profit organization that evaluates frontier AI models. Each group developed its own research questions and used Anthropic Insights to conduct privacy-preserving analysis of roughly 250,000 Claude.ai or Claude Code conversations from April-May 2026. We wanted our external partners to have as much independence as possible, so our contractual review rights were limited to user privacy, information that could help people violate our usage policies, Anthropic’s confidential information, and research accuracy. Anthropic otherwise had no say in the content of the findings and the researchers are free to publish their results even if they are inconvenient for Anthropic. Below are some early results. We're excited about the directions, and about what others will find now that the data is public. The **Social and Language Technologies Lab** studied how humans collaborate with AI. They looked at what types of work people bring to AI, what roles humans retain in completing that work, and where human-AI collaboration breaks down. They found: Read their full writeup [here](https://www.alphaxiv.org/abs/2608.human-ai-collaboration-at-scalev1). The **Human Information Processing Lab** is studying how people feel while using Claude and how that relates to Claude’s behavior. Their early results indicate: They are still completing their writeup. When it is public, we will add a link to it here. **METR** is estimating real-world productivity gains from coding agents and how these increases in productivity change across model generations. Their analysis of Claude Code conversations is still underway, but early results suggest: Sharing usage data is largely unprecedented in AI, so this pilot was as much an experiment in running such a program as it was a way to enable third-party research in a privacy-preserving way. Protecting our users’ privacy and the researchers’ independence were both paramount, and we achieved both. Anthropic Insights is designed for this—researchers never accessed raw conversations, only aggregated outputs after the same legal and privacy review as our internal work. However, all of this made the pilot slow for an AI lab’s normal research speed and resource intensive to run. Both factors present a challenge to effectively scaling it. For more details on how we ran this pilot, see the [Appendix](https://www-cdn.anthropic.com/files/4zrzovbb/website/8a665c85eec3a63b4d86287b9255657016f50e29.pdf). Below we discuss what we learned and how we addressed the challenges that arose. **It is valuable to pursue the same problem from different perspectives.** Some of our partners’ research questions overlapped with work being pursued internally. For example, METR’s proposal was similar to our economics research on “[Agentic coding and persistent returns to expertise](https://www.anthropic.com/research/claude-code-expertise).” We found this overlap valuable: it gave external researchers the chance to examine similar data and draw their own conclusions. Whether those align with ours is something we’ll follow as their study continues. We also connected METR with our Economics team and found that this connection improved both research teams’ work. **Research methods that work internally need to adapt for external partners.** When using Anthropic Insights, a researcher writes a question such as, “What type of guidance is this person asking for?” and Claude answers it for every conversation in the study. The answers are then aggregated into categories; researchers only see final categories and the percentage of conversations that fall under each one. Because we are relying on Claude’s judgments, the tool is sensitive to a question’s wording; a poorly phrased one can place conversations into categories that misrepresent them. Because no one can read the underlying conversations, these errors are hard to catch. Internally, we manage this by iterating on the questions many times over weeks. External partners couldn’t do that, since repeated privacy review before sharing each dataset would have made the study infeasible. Instead, we had them test their questions on WildChat, a public dataset of human-AI conversations where they could check the answers against the underlying conversations themselves. But WildChat skews toward casual and creative use, unlike Claude traffic, so some questions that performed well on WildChat produced misleading categories once applied to actual Claude conversations. We addressed this by providing guidance on how to interpret Anthropic Insight’s outputs (see [Appendix](https://www-cdn.anthropic.com/files/4zrzovbb/website/8a665c85eec3a63b4d86287b9255657016f50e29.pdf)). Going forward, we are exploring how external researchers can develop their questions and categories more effectively in advance. **Maintaining transparency about misuse without enabling it.** Some categories in our partners’ Anthropic Insights outputs surfaced violations of our Acceptable Use Policy or Terms of Service—for instance a category of people seeking guidance on a prohibited activity. We think the public should know about misuse of our platform, so we shared most of these violations. The exceptions were categories that described *how* users got around our safeguards rather than *what* they attempted. Less than 5% of categories and conversations were affected in each study, and in each case we told researchers which clusters we had altered or removed and why. As a standard practice, when Anthropic Insights surfaces such violations, we share the aggregated data with our Safeguards team for their review. This is also an important process for our work with external researchers moving forward. Understanding AI’s effects on society is too big a job for AI companies alone. Real oversight needs external researchers asking their own questions of real-world usage data and publishing what they find independently. This pilot was an experiment: could external researchers conduct independent studies on our platform without compromising our users’ privacy? The effort was more challenging than we expected, and we learned many lessons, but so far the answer seems to be yes. Our partners pursued research we would not have thought to design ourselves, and each told us something new about AI’s real-world impacts. This is a promising first step, but there is far more to do. The next step is for us to determine whether we can scale this program, both in what kinds of studies we can support given the constraints described above, and in how many we can run at once. We are starting slowly to ensure privacy, safety, and research quality. We want to gauge interest and understand what researchers would want to study. If you are a researcher and access to Anthropic Insights would let you pursue work you cannot do today, please fill out [this form](https://docs.google.com/forms/d/1PyyC-GU4Jra2WoXWcJjuFXZhM-lRxeBNnlh1tJv_2Jg/edit). The full appendix is [available here](https://www-cdn.anthropic.com/files/4zrzovbb/website/8a665c85eec3a63b4d86287b9255657016f50e29.pdf). It describes how we ran the program, including how we chose our three partners, the research primer we wrote to explain the program’s goals and what Anthropic Insights can do, and how each project moved from proposal to study design to analysis. It also includes details of our collaboration agreements, which explicitly say our partners are free to publish findings even when they are inconvenient for Anthropic. Furthermore, we cover the third-party privacy audit of this data, conducted by Imperial College London, and the privacy threat model we hold all released data to. We also include guidance on interpreting the data we are releasing from our partners’ Anthropic Insights research studies. Kunal Handa led the project, collaborated with the partners on their research proposals, drafted research guidance and the contract, ran partners’ Anthropic Insights studies, helped build the technical infrastructure supporting partners’ research, contributed to the internal review of the Anthropic Insights outputs, and wrote the blog post. Miranda Zhang coordinated the partnerships and communications, and contributed to all parts of the work. Gabriel Nicholas coordinated the third-party privacy audit, internal review of Anthropic Insights’ outputs, and contributed to all parts of the work. Miles McCain wrote the guidance on interpreting Anthropic Insights outputs, developed technical infrastructure to support partners’ research, contributed to the internal review of the Anthropic Insights outputs, and provided feedback on the blog post and privacy threat model. Ryan Heller contributed technical infrastructure to support partners’ research. Saffron Huang contributed to the internal review of the Anthropic Insights outputs and provided key feedback and discussion. Thomas Millar and Suzanne Wang contributed technical infrastructure to support partners’ research and to the internal review of the Anthropic Insights outputs. Shan Carter, Mo Julapalli, Matt Kearney, Sarah Pollack, and Judy Shen contributed to the internal review of the Anthropic Insights’ outputs. Matthew Jagielski contributed to the privacy threat model. Shaoyi Zhang contributed technical infrastructure to support partners’ research. Heather Whitney, Ankur Rathi, Aisling Keenan, and David Saunders provided legal and privacy guidance throughout the project. Jake Eaton and Sylvie Carr contributed to the framing and writing of the blog post. Jack Clark and Michael Stern provided valuable guidance, support, and discussion throughout the process. Deep Ganguli provided detailed guidance, organizational support, and feedback throughout all stages of the project. Additionally, we thank Miriam Chaum, Ishita Dasgupta, Esin Durmus, Adam Farina, Zoe Hitzig, Jerry Hong, Devin Kuokka, Hendson Lin, Maxim Massenkoff, Peter McCrory, Maryam Quasto, Nitarshan Rajkumar, Amie Rotherham, Divya Siddarth, Taylor Sorensen, Jerome Swannack, Alex Tamkin, Molly Villagra, Scott White, and Charles Yang for their helpful ideas, discussion, feedback, and support. For their partnership in this program, we thank Vishakh Padmakumar, Yijia Shao, Jennifer Wang, Diyi Yang, and Dora Zhao from the Social and Language Technologies Lab at Stanford, Tsvetomira Dumbalska, Hannah Rose Kirk, and Christopher Summerfield from the Human Information Processing Lab at Oxford, and Joel Becker (now at Anthropic), Daniel Paleka, and Parker Whitfill from METR. For conducting the third-party privacy audit, we thank Zexi Yao, Bozhidar Stevanoski, Peter Romov, Euodia Dodd, Xiaoxue Yang, and Nataša Krčo. ``` @online{handa2026enablingindependentresearch, author = {Kunal Handa and Miranda Zhang and Gabriel Nicholas and Miles McCain and Ryan Heller and Saffron Huang and Thomas Millar and Suzanne Wang and Shan Carter and Mo Julapalli and Matt Kearney and Sarah Pollack and Judy Shen and Matthew Jagielski and Shaoyi Zhang and Heather Whitney and Ankur Rathi and Aisling Keenan and David Saunders and Jake Eaton and Sylvie Carr and Jack Clark and Michael Stern and Deep Ganguli}, title = {Enabling independent research on how people use Claude}, date = {2026-08-26}, year = {2026}, url = {www.anthropic.com/research/enabling-independent-research}, } ```