Investing in Vals: Building the Trust and Evaluation Layer for AI

Investing in Vals: Building the Trust and Evaluation Layer for AI a16z has announced an investment in Vals, an AI evaluation platform designed to serve as an independent scorekeeper by benchmarking models on real-world workflows rather than academic exams. Traditional academic benchmarks are increasingly saturated, contaminated, or gamed, failing to reflect multi-step, real-world utility as AI agents take on autonomous, long-horizon tasks. Vals collaborates with domain experts to build benchmarks based on actual work—such as legal research, financial document analysis, and software development—evaluated by automated grading systems to expert standards. To counter benchmark obsolescence and 'benchmaxing,' Vals keeps private test sets secure and regularly replaces saturated tests, such as swapping CorpFin for an Excel Modelling Benchmark. Vals was co-founded by Stanford computer science graduates Rayan Krishnan and Langston Nashold, who previously collaborated on real-world measurement problems. Limitations of Academic AI Benchmarks Frontier models are increasingly scoring near-perfect marks on standard academic leaderboards. Because public datasets often saturate, leak into training data, or serve as explicit optimization targets, high leaderboard performance no longer guarantees capability in practical settings. As AI shifts from simple question-answering to autonomous multi-hour tasks across law, finance, and software engineering, suboptimal model choices risk substantial token expenses, lost time, and diminished customer satisfaction. Vals's Workflow-Based Evaluation Model Vals addresses the evaluation gap by designing tests centered on actual operational tasks rather than standardized tests like the bar exam. Working alongside field experts, Vals transforms professional workflows into benchmarks paired with automated grading stacks capable of delivering results within hours. The platform protects evaluation integrity through secure, private test sets with limited runs to prevent gaming and training data contamination. Adaptive Benchmarking and Independent Scorekeeping Vals treats benchmarks as perishable tools that must be retired once models reach saturation. For instance, the team replaced the CorpFin benchmark with a new Excel Modelling Benchmark after CorpFin stopped differentiating frontier systems. The investment thesis compares Vals to independent market auditors and rating agencies like Moody's, S&P, and UL, arguing that as sellers possess asymmetric incentives, an objective third-party evaluator is essential for the AI market.

a16z宣布投资AI独立评测平台Vals a16z宣布投资AI评测平台Vals。随着前沿大模型在传统公开学术基准上趋于饱和，企业亟需能够反映真实复杂工作流的独立评测体系，Vals致力于成为AI领域的第三方“独立记分员”。 传统公开基准测试存在饱和、数据泄露及刷榜（benchmaxing）问题，无法反映复杂现实任务表现。 Vals与领域专家合作，将法律研究、财务分析和编程等真实工作流转化为基准并实现自动化评分。 Vals采用私有测试集与防污染机制，能在获得模型访问权后数小时内产出基准结果，并动态淘汰已饱和的测试集。 Vals由斯坦福大学校友Rayan Krishnan和Langston Nashold联合创立。 传统AI基准测试面临失效 前沿大模型正逐渐攻克相同的学术测试，公开数据集面临饱和、训练数据泄露或被模型针对性优化的问题。模型在排行榜上表现优异，但在实际复杂的长周期工作流中可能表现不佳。随着模型从简单的回答问题演变为执行数小时甚至数天的自主任务，选错模型将带来巨大的Token成本与业务损失。 基于真实工作流的评估与基础设施 Vals针对用户实际需要模型完成的工作进行测试，而非虚构考试。团队与各领域专家合作，将实际工作流转化为严格基准，并构建自动化评分系统。例如在法律领域测试法律研究能力，在财务领域测试复杂文件分析，在编程领域测试构建可用应用。其评估架构兼具快速、可复现和安全性，私有测试集通过限制运行次数防止数据污染与刷榜，可在数小时内生成评估结果。 动态淘汰机制与独立记分员角色 Vals认为基准具有时效性，当所有模型得分接近满分时基准便需退役。例如，团队在今年5月使用新的Excel建模基准替代了区分度不足的CorpFin。a16z认为，正如金融市场催生了穆迪和标普，AI市场也需要独立的第三方评估机构。创始人Rayan Krishnan和Langston Nashold均毕业于斯坦福大学计算机系，在创立Vals前已长期合作研究现实度量问题。

[America](https://www.a16z.news/t/america) | [Tech](https://www.a16z.news/t/technology) | [Opinion](https://www.a16z.news/t/opinion) | [Culture](https://www.a16z.news/t/culture) | [Charts](https://www.a16z.news/t/charts) We all know AI is getting smarter day by day. For the majority of the users, there’s a more important question now: is it getting better at what I need it to do? For years, the industry has relied on academic benchmarks to track model progress. They gave us a common language for comparing models and, for a while this has been the most useful proxy for capability. However, frontier models are increasingly acing the same tests. Public datasets get saturated, leak into training corpora, or become targets that models are explicitly optimized against. A model can look brilliant on a leaderboard and still struggle when asked to perform the messy, multi-step work that actually matters in the real world. At the same time, the stakes are getting higher. Models are moving from “answering questions” to “doing the work”: analyzing financial documents, resolving legal cases, writing and debugging software, navigating complex enterprise workflows.The horizon of such tasks is getting longer too: agents are operating autonomously over hours and days. A bad choice of model for the task could cost not just thousands of dollars in tokens, but also time and customer satisfaction. *That’s where Vals comes in: building the trust layer between models and the people who rely on them.* Vals takes a fundamentally different approach to evaluation: **test models on the work people actually want them to do, not on contrived exams.** The team works with domain experts across different fields to turn real workflows into rigorous benchmarks, then builds automated grading systems that can evaluate the final work product to an expert standard. Instead of asking whether a legal model can pass the bar, Vals tests whether it can perform legal research; in finance, whether it can analyze complex documents; in coding, whether it can build a working application. The team works with domain experts to turn real workflows into rigorous benchmarks, then builds automated grading systems that can evaluate the final work product to an expert standard. Just as importantly, Vals has built the infrastructure to run these evaluations quickly, reproducibly, and securely at scale. Private test sets are kept private and have limited runs to protect against contamination and gaming, while Vals’ evaluation stack can produce benchmark results within hours of receiving model access. The result is a grading system that is both closer to real economical work and practical enough to keep pace with frontier model releases. Vals understands that benchmarks are perishable and adaptive. A good benchmark is supposed to become obsolete. If models have improved until every system scores nearly perfectly (so called “benchmaxing”), the benchmark has done its job, and it is time to build a harder one. Vals has been retiring saturated evaluations and continuously introducing new tasks as the frontier moves. In May, for example, the team swapped CorpFin for its new Excel Modelling Benchmark after CorpFin stopped providing enough differentiation between models. We believe this ability to continuously define the frontier is what’s needed by the industry to keep pace and make smart, informed decisions when it comes to AI adoption. Every market eventually needs an independent scorekeeper. When vendors grade their own homework, buyers need someone they can trust. Credit markets developed Moody’s and S&P; public markets rely on independent auditors; product manufacturers rely on UL to certify safety. These institutions emerged for the same reason: when sellers have more information than buyers - and every incentive to present themselves favorably - trusted third-party measurement makes the market work better. That’s where we’re at with AI which is evolving to be one of the biggest markets in the industry. The opportunity, and the responsibility, is enormous. Rayan Krishnan and Langston Nashold have been remarkably tenacious and thoughtful about this problem from the very beginning. Building AI’s independent scorekeeper requires an unusual combination: technical depth, skepticism, and the patience to keep rebuilding the test as the frontier moves. Rayan and Langston have all three. At Stanford, both co-founders studied computer science, had already been collaborating on real-world measurement problems years before starting Vals. That makes them uniquely suited not just to publish another leaderboard, but to build the trusted, adaptive evaluation layer the AI market will depend on. We’re thrilled to partner with Rayan, Langston, and the entire Vals team as they build the trust layer that will underpin the AI economy. *This newsletter is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. Furthermore, this content is not investment advice, nor is it intended for use by any investors or prospective investors in any a16z funds. This newsletter may link to other websites or contain other information obtained from third-party sources - a16z has not independently verified nor makes any representations about the current or enduring accuracy of such information. If this content includes third-party advertisements, a16z has not reviewed such advertisements and does not endorse any advertising content or related companies contained therein. Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z; visit [https://a16z.com/investment-list/](https://a16z.com/investment-list/) for a full list of investments. Other important information can be found at [a16z.com/disclosures](http://a16z.com/disclosures). You’re receiving this newsletter since you opted in earlier; if you would like to opt out of future newsletters you may unsubscribe immediately.*