Anthropic and Andon Labs Evaluate Autonomous Drone Surveillance in Project Pilot

Anthropic and Andon Labs Evaluate Autonomous Drone Surveillance in Project Pilot Anthropic and Andon Labs introduced Drone-Bench to evaluate frontier AI models on their ability to autonomously control commercial drones for indoor locate-and-follow surveillance tasks. Drone-Bench breaks down autonomous drone surveillance into five sub-tasks evaluated against a baseline established by human-AI engineering teams. Across 15 models tested, AI performed strongest on target detection and following, but struggled most with spatial reconstruction and localization. Claude Fable 5 surpassed the baseline on four of five sub-tasks and exhibited advanced reasoning, such as calculating camera tilt from floor grout lines. Compounding reconstruction errors prevented successful autonomous navigation between rooms during real-world flight tests, and average consistency across runs remains limited. Benchmark Setup and Methodology Project Pilot examines how frontier AI interacts with physical hardware by evaluating quad-rotor drone control in an indoor office environment. Working with Anthropic, Andon Labs created Drone-Bench, which decomposes the primary objective of locating and following a specific individual into five constituent sub-tasks. These tasks were replicated in software simulations alongside physical drone flights to allow faster, repeatable testing. The evaluation measures models against a baseline developed by human engineers working with coding agents, setting a benchmark for practical modern software engineering capability. Model Performance and Emergent Capabilities Evaluations across 15 models from three developers showed that newer generations consistently advanced further across all sub-tasks. Claude Fable 5 achieved the highest performance, exceeding the human-AI baseline on all tasks except 3D reconstruction. During evaluations, Fable 5 demonstrated autonomous pre-execution analysis, such as deducing camera tilt within four degrees using the vanishing point of floor grout lines and creating rough 2D top-down environment reconstructions to debug its code locally before submitting implementations. Hardware Limitations and Governance Concerns When deployed on physical drone hardware, Fable 5 failed to navigate autonomously between rooms because reconstruction errors cascaded into localization and navigation failures. Consistency also remains a challenge: across ten simulation runs, Fable 5 met the human baseline on average for only three of the five sub-tasks. Anthropic emphasizes that autonomous drone piloting is a dual-use capability carrying risks for physical security and privacy. The researchers warn that as models reliably surpass human-performance baselines, pressure may grow to remove human oversight, necessitating proactive governance.

Anthropic与Andon Labs发布Project Pilot：评估AI自主控制无人机执行追踪任务能力 Anthropic 与 Andon Labs 合作开展了名为 Project Pilot 的研究，并推出了新基准 Drone-Bench，旨在评估前沿 AI 模型自主控制四旋翼无人机在室内执行目标定位与跟踪等监控任务的能力与双重用途风险。 研究在室内办公环境中测试 AI 模型控制四轴无人机寻找并跟随指定人员的能力，任务被拆解为五个核心子任务并在软件中复现以进行基准评估。 基准以人类工程师与编程智能体协作制定的算法为达标基线，Andon Labs 测试了来自三家开发者的 15 款前沿模型。 Claude Fable 5 表现最佳，在除环境重建外的所有子任务中均突破基线，但因重建误差累积导致端到端真实飞行时无法自主在房间之间导航。 实验存在测试环境单一、速度较慢、未在户外人群中测试等局限性，但显示出硬件控制正在重演软件编程智能体的自主化发展轨迹。 研究背景与双重用途属性 继 Project Vend 与 Project Fetch 之后，Anthropic 进一步探索前沿模型与物理世界的交互。无人机作为商用现成硬件，兼具农业增产等民用价值与军事打击等潜在风险。本次实验选择的室内人员定位与追踪任务具有明确的政策与安全关联，既可用于搜救，也可能被滥用用于不受约束的监视。Anthropic 前沿红队参与此类研究，旨在提前把握 AI 自主操控机器人的能力演进节奏。 Drone-Bench 基准与评估方法 Andon Labs 咨询 Anthropic 后创建了 Drone-Bench 基准，将无人机定位与跟随任务拆解为五个必要子任务，并通过软件仿真实现快速、多次测试。评估基线设定为人类专业团队与编程智能体协同开发所能达到的性能标准，代表当前非全职机器人专家的现实应用水平。只要模型在子任务中达到或超过该基线，即被视作具备对应自主控制能力。 15款模型测试结果与表现分析 评估涵盖了 GPT 系列、Claude 系列与 Gemini 系列共 15 款模型。整体趋势表明越新的模型在各子任务中进展越大，各模型在目标检测与跟随上表现最好，而在重建与定位上表现最差。表现最优的 Claude Fable 5 在除环境重建外的所有任务中均超过基线，且展现出提交前在仿真中估算相机倾角或构建 2D 环境进行本地测试的自主分析能力；但在真实无人机端到端测试中，由于重建误差累积导致跨房间导航失败。 一致性不足与实验局限性 模型在输出的一致性上仍有明显短板。在 10 次仿真中，先进模型至少有一次能达到基线的任务有四项，但 Fable 5 平均达到基线的任务仅有三项。此外，实验局限在单一办公室户型、低飞行速度和有限人数的场景中，未涉及复杂的户外人群环境，因此不能直接等同于实战部署能力，但已为自主瞄准和跟踪能力的演变提供了明确信号。 硬件控制的治理与安全挑战 AI 控制物理硬件的发展历程正在复刻代码生成智能体的发展轨迹，正逐步从单步人工确认转向长周期自主执行。当模型越过可靠性与能力阈值后，行业往往面临减少人工监督以提高效率的压力。Anthropic 强调，由于机器人技术直接涉及人身物理安全与个人隐私，各方不能单纯以效率为导向，必须提前在模型对齐、行业规范与政府治理框架上达成共识。

*Anthropic and Andon Labs* Several of our research projects over the last year have looked at how frontier models interact with the physical world. In [Project](https://www.anthropic.com/research/project-vend-1) [Vend](https://www.anthropic.com/research/project-vend-2), AI models ran a small shop; [Project Fetch](https://www.anthropic.com/research/project-fetch-robot-dog) was an early look at robots as the intermediary between digital models and physical objects. As we recently noted in [Project Fetch: Phase two](https://www.anthropic.com/research/project-fetch-phase-two), we’re already seeing improvements in model capability such that their ability to use off-the-shelf robots is on track to approach the ease with which coding agents use software tools. Working again with our partners at [Andon Labs](https://andonlabs.com/), we developed a new series of demonstrations and evaluations that assess AI models’ ability to use a flying drone to autonomously perform a simple locate-and-follow task of the kind used in aerial surveillance, culminating in a new benchmark: Drone-Bench. We expect AI models to become broadly capable at many things that humans can do. Operating hardware, in particular robots, is one such capability. Being able to do this opens up a large surface over which AI could contribute to the economy, but likewise opens up a new area of risk. A key reason why Anthropic has a [Frontier Red Team](https://www.anthropic.com/research/team/frontier-red-team) is to measure capabilities like this, giving us situational awareness into how close we are to the world in which AI can autonomously pilot robots—with all the attendant benefits and risks. Aerial drones are especially important because they are readily available and frequently used by professionals and hobbyists. They have been used to increase crop yields in agriculture and target opposing forces in warfare. Like AI itself, drones are a dual-use technology; it is crucial to have better evidence about their intersection. By combining actual flight demonstrations and decomposing the constituent tasks into replicable evaluations, we can look back at the rapid progress of models so far, and project their capabilities in the near future. As is so often the case, our findings point toward a world of democratized opportunity and risk. Technology developers, civil society, and governments will need to converge on effective norms and governance frameworks in response. The core task we tested in Project Fetch—getting a robot dog to retrieve a beach ball—was neither especially practical nor especially concerning. In this project, we chose an objective with clearer utility and policy relevance: a simple locate-and-follow task used in aerial surveillance. Capabilities like automated person-detection and tracking can have legitimate purposes such as search and rescue, disaster response, and lawful public safety uses. But this is a class of capabilities that is also subject to abuse, either through overreach of a legitimate authority or by unaccountable private individuals or organizations. The work we report here thus more closely matches the “dual-use” nature of AI models. In these experiments, we ask the model to control a quad-rotor drone in an indoor office environment in order to locate and follow a person.¹ This requires a number of complex sub-tasks. The AI model needs to develop schema for controlling the aircraft, mapping and navigating the obstacle-laden indoor space, finding the target individual from a reference photo, and following them (plus reacquiring the target if they move out of frame). Individually, there are known algorithms for accomplishing all of these tasks. What is not trivial is for the AI model to understand the challenges, identify the preexisting resources it can use to solve them, adapt those off-the-shelf solutions to its current situation, and execute the mission in real time. As we will see, the difficulty—both individually and in chaining these tasks together—is sufficient to distinguish between models of varying intelligence and plot the trajectory of capability improvement. Drone-Bench is a benchmark created by Andon Labs (in consultation with Anthropic) to test if AI agents are capable of controlling a drone for surveillance tasks. Anthropic has not been given access to Drone-Bench; Andon Labs ran the evaluations we report here. First, Andon Labs took the main goal—find and follow a designated person in an office using the aerial drone—and decomposed it into five sub-tasks, all of which are necessary and, taken together, are likely to be sufficient for accomplishing the overall objective. These sub-tasks are: Next, each of these real-world tasks was reproduced in software so that we could run the models through them multiple times and far faster than needing to set up the physical demo for each instance (this is an improvement over Project Fetch, for example, which was an entirely physical experiment). It was also important to establish a meaningful baseline of performance. Human-only baselines increasingly don’t reflect the reality of contemporary software engineering, so Andon worked with coding agents to develop algorithms for each sub-task. Putting all of these algorithms crafted by human-AI teams together allowed them to demonstrate end-to-end success, as shown in the below video. A task is considered completed if the model meets or exceeds the baseline. Thus, if a model can complete all tasks, we can infer that it has the ability to autonomously control a drone to do at least as well on this surveillance task as the team at Andon Labs did. For more details, check out Andon Labs’ [post](https://andonlabs.com/evals/drone-bench) about Drone-Bench. It is worth underscoring that the evaluation’s baseline is neither the floor of unassisted human capability nor the ceiling of what is possible with concerted human-AI collaboration. Rather, it is indicative of what can be achieved in the present by AI experts (but not full-time roboticists) using a realistic suite of modern tools. The interesting question is if and when models operating essentially autonomously *reliably* pass this baseline of reasonable and realistic effort, as that is the point at which pressure to reduce human oversight may intensify—making deliberate, use case-specific judgments about the appropriate human role all the more important. Andon tested 15 models from three developers: GPT-4o, GPT-4o Nov, o1, o3, Claude Opus 4, Gemini 2.5 Pro, GPT-5, Gemini 3.1 Pro, Opus 4.5, GPT-5.2, Opus 4.7, GPT-5.5, Opus 4.8, Fable 5, and GPT-5.6 Sol. The overall trend we observe is that newer models get successively further on all sub-tasks. Of these tasks, models are most successful at detection and following, and least successful at reconstruction and localization. The best performing model was Claude Fable 5, which brings the frontier past the baseline on all tasks except reconstruction. When we then tested its ability to execute the entire demonstration end-to-end on the real drone, it performed noticeably better than the baseline at detecting and following. However, due to errors from reconstruction that compounded in localization and navigation, it was unable to autonomously navigate between rooms (as you can see in the first part of the below video). Clearly, Fable’s failure to accurately reconstruct the room is a huge stumbling block. But given models’ capabilities in the other phases, it really just amounts to the missing piece. Once it’s in place, end-to-end performance will suddenly be within reach. This is an advantage of decomposing the evaluation into constituent tasks: we are better positioned to avoid surprise. What would look like a discontinuous jump is revealed to be gradual progress in several necessary, but not sufficient, sub-tasks. The sub-task view also surfaces encouraging signs. A trend we're seeing when reading Fable 5's submissions is that the model is doing local analysis before submitting its implementation. In one submission, the model calculated the drone's camera extrinsics by analyzing a video from the simulation, estimating the camera tilt to within four degrees of the true value by using the grout lines on the floor to recover the scene's vanishing point. You can see its process below: In another run, Fable 5 built a 2D top-down reconstruction of *what it thought* the Follow task's environment looked like, so it could test and iterate on its implementation locally before burning a submission. The environment differs from the real environment (seen below), but it helped Fable catch some easy bugs! It’s important to understand how *consistently* models reach the reference level of performance, as well as *whether* they can reach it. Here there is obvious room for improvement. When we run 10 simulations, the models reach the human baseline in at least one simulation for four of five tasks. But even Fable 5, the current frontier model, reaches the human baseline *on average* for only three of the five tasks—and that level of consistency followed six months after the human baseline was exceeded as a one-off for the first time. Although the complexity of this experiment is greater than some of our previous work and the operating environment of an actual (or simulated) office is more challenging than a wide-open warehouse, the experiment has important limitations: the drones are moving at slow speeds, we only tested in one office floorplan with a limited number of people, and Andon did not test outdoors in large crowds, among many other factors that would have made this more realistic. We still think this pilot provides a real signal about the direction of model capabilities: this evaluation will provide meaningful information about the underlying performance and reliability of models for autonomous targeting and tracking, even though more realistic and diverse experiments would be needed to assess operational capability. This experiment highlights the potential of commercial-off-the-shelf (COTS) hardware and AI-tailored software to support useful, but possibly risky, tasks. It is important to take seriously the parallel between AI models’ use of software in agentic coding and AI models’ control of hardware. In the early days of agentic coding, humans approved nearly every tool call. But after only a few months, models are now much more trusted to execute long-horizon tasks with minimal intervention. More generally, at low levels of capability and reliability, keeping a human in the loop is an easy decision because it saves time and resources by augmenting model capabilities or preventing costly mistakes. Once models pass capability and reliability thresholds (such as the human-AI team baseline we used in this experiment), there will be real pressure to treat human oversight as a cost rather than a safeguard. That is exactly why these decisions must be made deliberately, particularly in domains like this one that implicate physical security and privacy and where efficiency alone should not be the governing consideration. As Anthropic has long argued, the requirements for investing in AI alignment, governance, and safety increase with the scale of capabilities. Robotics is no different than other domains in this regard, especially since it implicates physical security and individual privacy.