Google Pilots World’s First Double-Blind Evaluations for Frontier AI Models

Google Pilots World’s First Double-Blind Evaluations for Frontier AI Models Google has announced what it describes as the world's first double-blind evaluation of a proprietary, frontier-class AI model. Conducted with external partners, the pilot runs a Gemini Flash Lite model against confidential benchmarks within a cryptographically secure, privacy-preserving environment to prevent benchmark contamination. External evaluations are confined to a cryptographic environment so models cannot access test prompts in advance to artificially inflate performance. The initiative tests Gemini Flash Lite in partnership with the Singapore AI Safety Institute, OpenMined, AVERI, and MLCommons. The technical and cryptographic safeguards build upon existing zero-logging protocols and contractual measures to verify evaluation integrity. Preventing Benchmark Contamination Evaluating AI models faces the challenge of benchmark contamination, where models gain prior exposure to test questions, analogous to a student seeing an exam in advance. Such visibility compromises evaluation integrity, artificially inflating scores and undermining the confidence that policymakers, researchers, and enterprises place in reported safety and capability metrics. Cryptographic Safeguards and Partnerships To counter blind spots beyond internal testing, Google partnered with specialized research labs, civil society, and national AI safety institutes. Testing Gemini Flash Lite inside a cryptographic box ensures test prompts remain confidential and unusable for training, introducing technical and cryptographic guarantees alongside contractual safeguards.

Google 试行全球首个前沿 AI 模型双盲评估 Google DeepMind 宣布开展全球首个专有前沿 AI 模型的双盲评估试点，借助密码学安全环境防止基准污染，确保评测结果的客观与真实。 基准污染（模型提前接触测试题）会导致评估分数虚高，影响对模型真实能力与安全性的衡量。 试点联合新加坡 AI 安全研究所、OpenMined、AVERI 与 MLCommons，在隐私保护计算环境中测试 Gemini Flash Lite。 在原有无日志协议和合同约定的基础上，通过密码学技术将外部测试题隔离，防止被模型提前用于优化。 应对基准污染与信任难题 随着前沿 AI 模型能力提升，行业面临评估题目被模型提前学习导致测试失真的风险。为了使评测数据能够真实反映模型水准，让政策制定者、研究人员和企业建立信任，测试必须确保模型在考核前对具体题目完全不可见。 引入密码学环境开展双盲测试 在此次试点中，Google 携手新加坡 AI 安全研究所、OpenMined、AVERI 和 MLCommons 等外部机构，将机密基准测试放置在密码学隔离的隐私计算环境中对 Gemini Flash Lite 进行测试。这一方案在既有的无日志流程与合同约束之上增加了技术与密码学防线，推动 AI 模型安全评估向更高完整性迈进。

August 27, 2026 Responsibility & Safety Building trust in proprietary model benchmarks using cryptographically secure environments Imagine a student is set to take a high-stakes exam. If they accidentally peek at the test questions in advance, achieving a perfect score is influenced by this knowledge, making it a meaningless accomplishment. To truly measure what they know, they must have no visibility of the test questions until it's time to take the exam. That is the exact challenge the industry faces when evaluating advanced AI models. If a model has already seen the test questions - a problem known as benchmark contamination - the results can only be trusted to an extent. Today, we’re introducing the **world’s first double-blind evaluation of a proprietary, frontier class AI model,** which keeps external evaluations confined to a cryptographic “box” where they can’t be used by models later to optimize performance ahead of testing. We're partnering with the Singapore AI Safety Institute, OpenMined, AVERI, and [MLCommons](https://mlcommons.org/2026/08/double-blind-reliability-evaluation/), to test a Gemini Flash Lite model against confidential benchmarks in a [privacy-preserving environment](https://cloud.google.com/blog/products/identity-security/verifiable-trust-in-the-ai-era-whats-new-in-confidential-computing), increasing evaluation integrity. At Google, we assess our AI systems using a broad spectrum of evaluations throughout model development and deployment, but we don’t rely on internal testing alone. To identify potential blindspots, we work with a diverse group of external partners, including specialized research labs, civil society and national AI Safety and Security Institutes (AISIs), using their unique expertise to stress-test our models. As AI models become more capable, ensuring the model has not seen the test questions or prompts in advance is critical, as this can skew the results. Policymakers, researchers, and enterprises need to trust that AI benchmarks accurately reflect a model's true capabilities and safety, but if models are able to “peek” at the evaluation questions in advance, it can artificially inflate scores and undermine this trust. Although zero-logging protocols and rigorous contractual safeguards have long kept external test prompts confidential, incorporating technical and cryptographic safeguards marks a major step forward in secure model evaluation.