NVIDIA Releases AICR v1.0 for Stable and Verifiable GPU Cluster Configuration

NVIDIA Releases AICR v1.0 for Stable and Verifiable GPU Cluster Configuration NVIDIA has released AI Cluster Runtime (AICR) v1.0, introducing a stable compatibility contract and version-locked recipes to streamline GPU-accelerated Kubernetes configurations across diverse hardware, platforms, and deployment tools. AICR addresses silent deployment failures and version conflicts across kernels, GPU drivers, Kubernetes, and workload frameworks by providing version-pinned recipes. The v1.0 release establishes formal compatibility rules and merge-blocking baselines for the CLI, REST API, Go SDK, bundle layouts, and artifact schemas. The runtime is built on four decoupled capabilities: Snapshot, Recipe, Bundle, and Validation, accompanied by signed hardware testing evidence. Ecosystem integrations include a Pulumi Labs infrastructure-as-code provider and Mirantis k0rdent for multi-cluster management, alongside contributions from over 100 individuals. Challenges in GPU Cluster Configuration GPU-accelerated Kubernetes clusters require compatible alignments among dozens of components that follow independent release cycles, including host kernels, GPU drivers, runtimes, operators, and schedulers. Successful installation alone does not guarantee component health, required capabilities like gang scheduling, or acceptable performance. Historically, knowledge of functioning combinations was fragmented across disparate operational runbooks and scripts, making configurations difficult to discover and reproduce. Decoupled Lifecycle Architecture AICR operates through four independent capabilities: Snapshot records the observed cluster state; Recipe specifies the desired, version-locked configuration and constraints; Bundle renders the recipe into deployer-neutral artifacts for tools like Helm, Argo CD, Flux, or Helmfile; and Validation runs deployment, conformance, and performance checks against the active cluster. This workflow produces signed validation evidence from the hardware tested, allowing operators to verify bundles and cluster states explicitly. Public Stability Guarantees in v1.0 The v1.0 release introduces strict compatibility guarantees under a semantic versioning policy, requiring a major release for any future breaking changes to public interfaces. These baselines protect the aicr CLI, the aicrd REST API and OpenAPI contract, the generated bundle layout, artifact schemas, and the Go SDK located at github.com/NVIDIA/aicr/pkg/client/v1. The CLI and REST server share this Go client facade to ensure consistent operational behavior across integration entry points.

NVIDIA 发布 AICR v1.0：提供开放、稳定且可验证的 GPU 集群配置标准 针对 GPU 加速 Kubernetes 集群组件多、依赖复杂且升级易冲突的问题，NVIDIA 发布了 AI Cluster Runtime (AICR) v1.0。该版本通过锁定版本的配置配方与硬件签名验证，确立了 CLI、REST API、Go SDK 及工件架构的长期兼容性承诺。 AICR 提供版本锁定的 GPU 集群配置“配方”（recipes），可生成面向 Helm、Argo CD、Flux 或 Helmfile 的部署工件并附带硬件签名验证证据。 AICR 由四个独立核心能力构成：记录集群实际状态的 Snapshot、声明期望配置的 Recipe、渲染部署工件的 Bundle，以及执行合规与性能测试的 Validation。 v1.0 版本确立了公共 CLI、aicrd REST API、Go SDK (pkg/client/v1) 和工件架构的兼容性基准，后续任何破坏性改动均需发布新主版本。 AICR 已获 Pulumi Labs 和 Mirantis 等生态工具支持，项目贡献者超过 100 人，近半数来自 NVIDIA 外部。 GPU 集群配置的重现性挑战与配方机制 GPU 加速 Kubernetes 集群涉及内核、驱动、运行时、存储及工作负载框架等数十种独立周期的组件。仅成功安装并不代表组件健康或满足性能阈值。AICR 通过定义版本锁定的配方库解决该痛点，让运维人员可通过验证仪表板按服务、GPU 型号、操作系统和工作负载意图查找配置，并检查在目标硬件上经过测试与签名的验证证据。 AICR 的四大独立核心能力 AICR 包含 Snapshot、Recipe、Bundle 和 Validation 四项解耦能力。Snapshot 负责记录观测到的运行状态，Recipe 负责定义期望配置，Bundle 将其渲染为特定 CD 工具的部署工件，Validation 则对比实际状态并执行部署、一致性和性能检查。各环节解耦运行，即使切换 Helm、Argo CD 等部署工具，底层配置定义与验证逻辑仍保持统一。 v1.0 的稳定兼容性基线与生态协作 AICR v1.0 正式建立了公共接口的兼容性契约，涵盖 aicr CLI、aicrd REST API、Go 客户端以及生成的工件架构，合并前设有严格检查以杜绝破坏性变更。生态方面，Pulumi Labs 提供了 IaC 支持，Mirantis 将其集成于多集群管理，且社区贡献者可为未覆盖的硬件提交验证配方及签名凭证。

GPU-accelerated Kubernetes clusters depend on compatible versions across dozens of components, each on its own release cycle: host kernels, GPU drivers, container runtimes, networking, storage, operators, and workload frameworks. A configuration that works for one service, GPU generation, and Kubernetes release may silently fail for another, and tracing version conflicts after deployment is slow and error-prone. NVIDIA AI Cluster Runtime (AICR) addresses this with version-locked, validated recipes for GPU cluster configuration. Each recipe pins the component combinations that work together, renders deployment artifacts for Helm, Argo CD, Flux, or Helmfile, and carries signed validation evidence from the hardware it was tested on. The v1.0 release of AICR establishes a stable compatibility contract across its CLI, REST API, Go SDK, bundle layout, and artifact schemas so operators, integrators, and contributors can build on AICR’s public interfaces with confidence. Figure 1: AICR coverage across Kubernetes services, multiple generations of NVIDIA GPU accelerators, supported Linux operating systems, deployment frameworks, and workload platforms The validation dashboard lets operators find recipes by service, GPU, operating system, workload intent, and optional platform, then inspect each recipe’s status and any published evidence for the hardware configuration tested. Integrators can build against AICR’s public interfaces under the v1.x compatibility policy. Contributors can propose recipes for environments the maintainers cannot test, validate them on their own clusters, and submit signed evidence for maintainer review. That recipe model is also finding uses across the ecosystem. Pulumi Labs exposes AICR through an infrastructure-as-code provider, while Mirantis’s k0rdent integration packages it for multi-cluster management. Together, they demonstrate the value of defining GPU-accelerated Kubernetes configuration once and consuming it through different tools. Today, AICR has over 100 distinct contributors, with almost half from outside of NVIDIA! Why GPU cluster configuration needs a reproducible contract GPU-accelerated Kubernetes clusters depend on compatible versions and settings across host kernels, GPU drivers, container runtimes, Kubernetes, networking, storage, device plugins, operators, schedulers, and workload frameworks. These components follow different release cycles; upgrading one can break a previously working combination. A configuration validated for one service, GPU generation, fabric type, machine shape, and Kubernetes release may fail for another, and small version differences can be difficult to trace after deployment. Even if every component installs successfully, the cluster may not meet the recipe’s intended configuration. Installation doesn’t confirm that components are healthy, that required capabilities like gang scheduling or accelerator discovery work, or that measured results meet a recipe’s performance thresholds. Knowledge of which combinations work and how they were validated has lived in separate validation systems, deployment scripts, and operational runbooks. That makes it hard for teams to discover, reproduce, and update working configurations. Over the past six months, AICR has grown from a handful of recipes to a library spanning major Kubernetes services and the current NVIDIA accelerator portfolio, rendered as deployer-neutral bundles. We added live-cluster validation, signed evidence, public evidence aggregation, and supply-chain verification. For v1.0, we also added committed compatibility baselines and merge-blocking checks around the public integration surfaces. From observed state to a verifiable result AICR provides four core capabilities: Snapshot records observed cluster state, including Kubernetes, operating system, kernel, GPU, and topology information. Recipe describes the desired, version-locked component configuration and the constraints and validation phases that apply to it. Bundle renders the recipe into artifacts for the operator’s preferred deployment tooling. Validation compares the recipe with observed state and, where declared, runs deployment, conformance, and performance checks against the cluster. These four capabilities are deliberately independent. A snapshot records the observed state; it is not a desired configuration. A recipe describes the desired configuration; it does not reconcile a cluster. Common open source CD tools like Helm, Argo CD, Flux, or Helmfile apply or reconcile the bundle into the cluster. AICR can then validate the running cluster against the recipe, and record signed evidence of the result. These capabilities can be combined in multiple sequences. Snapshot data or explicit target criteria can produce a recipe. A recipe can produce a bundle for an existing deployer. The recipe and observed cluster state feed validation. An operator explicitly verifies bundles and evidence when they invoke the corresponding command. Figure 2: The NVIDIA AI Cluster Runtime lifecycle For example, an operator can select EKS, GB300, Ubuntu, training, and Kubeflow; resolve those criteria to a pinned recipe; render the recipe for Argo CD; deploy it through the existing GitOps workflow; and validate the running cluster against the same recipe. The intended configuration does not change if the operator instead renders it for Helm, Flux, or Helmfile. What’s new in AICR v1.0 AICR v1.0 defines compatibility rules for its public CLI, REST API, Go SDK, bundle layout, and artifact schemas. It also lets operators inspect the validation evidence published for each Supported recipe: what was tested, which checks passed, and who signed the results. AICR v1.0 sets compatibility rules for: aicr CLI’s public commands, flags, exit semantics, and structured output aicrd REST API and OpenAPI contract exported API of the github.com/NVIDIA/aicr/pkg/client/v1 package generated bundle layout and AICR artifact schemas Each public interface has a committed baseline checked before changes merge. The release policy also defines semantic breaking changes. After v1.0, removing or incompatibly changing a stable public interface requires a new major release. For Go integrators, pkg/client/v1 exposes the supported workflow without requiring imports from AICR’s internal packages. The CLI and REST server use the same facade, reducing the risk that one public entry point behaves differently from another. Try AICR and contribute Try a recipe for your environment, inspect its status and any published evidence, and run the dashboard’s aicr evidence verify command where evidence is available. Contributions are especially useful for hardware and cluster combinations outside current project coverage. You can: Contribute features, integrations, or documentation Propose a recipe for hardware, OS, or a service not yet represented in AICR Validate a recipe in your own cluster and submit signed evidence Report bugs, share feedback, or request features through GitHub Issues Start with the project repository , contributing guide , and issue tracker .