Alibaba and NVIDIA Support Qwen3.8-Flash-Next for Long-Context Agentic Coding

Alibaba and NVIDIA Support Qwen3.8-Flash-Next for Long-Context Agentic Coding Alibaba has released the weights for Qwen3.8-Flash-Next, a multimodal mixture-of-experts model previewing the upcoming Qwen4 architecture, accompanied by Day 0 functional support and optimization across NVIDIA hardware and software platforms. Qwen3.8-Flash-Next features a 125B-parameter main model supplemented by 51B N-gram embeddings, activating 6B parameters per token with up to 1M-token context via YaRN. The model pairs Gated DeltaNet with Qwen Sparse Attention across alternating layers to eliminate KV cache growth and accelerate 1M-token workloads. Running on an NVIDIA GB300 NVL72 rack-scale platform achieves over 16,000 tokens per second per GPU and over 200 tokens per second per user. Developers can fine-tune and serve the model using NVIDIA NeMo AutoModel, NeMo RL, SGLang, vLLM, and NVIDIA NIM. Hybrid Architecture for Long Contexts Qwen3.8-Flash-Next is built for context-heavy workloads such as agentic coding and document processing. To prevent attention compute and KV cache memory bottlenecks, three out of four layers use Gated DeltaNet to compress historical context into a fixed recurrent state, while the remaining layers use Qwen Sparse Attention to aggregate sequences into micro-blocks and retrieve relevant regions. In Alibaba's benchmarks, this sparse attention kernel yielded up to 7.6x speedups during prefill and achieved 8.6x the prefill throughput of Qwen3.7-Plus under a 1M-token context with a 90% prefix-cache hit rate. Hardware Deployment Across NVIDIA Systems The model is validated for inference on the NVIDIA GB300 NVL72, a rack-scale system containing 72 Blackwell Ultra GPUs linked by a 130 TB/s NVLink domain to eliminate expert routing bottlenecks. The setup delivers over 16K tokens per second per GPU and over 200 tokens per second per user. Workflows can also be prototyped on local hardware, including NVIDIA DGX Station, DGX Spark clusters, and workstations equipped with four NVIDIA RTX PRO 6000 Blackwell Max-Q GPUs, before scaling to GB300 NVL72 systems for production. Post-Training and Inference Ecosystem Developers can adapt the model using NVIDIA NeMo AutoModel for full SFT or LoRA fine-tuning directly on existing checkpoints, as well as NeMo RL for reinforcement learning recipes. For serving, NVIDIA provides support through engines such as SGLang, vLLM, TensorRT LLM, and TokenSpeed, with weights accessible on Hugging Face, ModelScope, QwenCloud, and deployable via model-free NVIDIA NIM containers.

阿里开源预览模型 Qwen3.8-Flash-Next，NVIDIA 提供 GB300 NVL72 首日推理与训练支持 阿里巴巴发布 Qwen3.8-Flash-Next 权重以供预览 Qwen4 架构，该混合专家模型专为长上下文与代理编程打造；NVIDIA 同时宣布对其提供全栈支持，涵盖机架级 GB300 NVL72 部署、多框架推理及 NeMo 后训练方案。 Qwen3.8-Flash-Next 包含 125B 主模型与 51B N-gram 嵌入，每 token 激活 6B 参数，原生支持 262,144 token 窗口且可通过 YaRN 拓展至 1M。 架构融合门控 DeltaNet（GDN）与稀疏注意力（QSA），大幅抑制 KV 缓存增长，在 1M 上下文下 prefill 速度相比全注意力提升最高达 7.6 倍。 部署于集成 72 块 Blackwell Ultra GPU 的 GB300 NVL72 时，单 GPU 吞吐量突破 16K tokens/s，单用户速率超 200 tokens/s。 长文本混合架构与性能表现 针对高吞吐与长文本场景中注意力计算和 KV 缓存瓶颈，该模型采用创新混合架构：四分之三的层采用 GDN 将历史上下文压缩为固定大小的循环状态以避免缓存膨胀，其余层采用 QSA 进行跨全上下文的精准检索。QSA 通过微块级重要性评估替代计算昂贵的逐 token 索引，Alibaba 基准测试显示其注意力内核在 1M token 负载下的 prefill 和 decoding 分别实现最高 7.6 倍和 4.9 倍加速；在 90% 前缀缓存命中率测试中，其 prefill 吞吐量达到 Qwen3.7-Plus 的 8.6 倍。 机架级至本地硬件适配 GB300 NVL72 凭借 72 块 GPU 组成的 130 TB/s NVLink 全互联域，排除了跨传统网络的专家通信瓶颈，单 GPU 吞吐量超过 16K tokens/s、单用户速度超过 200 tokens/s。除机架级环境外，开发者还可在配有 4 块 RTX PRO 6000 Blackwell Max-Q 的工作站、DGX Station 或 DGX Spark 集群等本地设备上进行原型设计与评估，再平滑扩展至生产环境。 后训练与多推理引擎支持 在训练与服务方面，开发者可借助 NVIDIA NeMo AutoModel 直接载入 Hugging Face 权重执行全量 SFT 或 LoRA 微调，并利用 NeMo RL 进行强化学习。推理端则提供了包括 SGLang、vLLM、TokenSpeed 等开源方案，或可通过 NGC 借助 model-free NVIDIA NIM 快速部署。

Alibaba released the model weights for Qwen3.8-Flash-Next as a preview of the upcoming Qwen4 architecture for developers to experiment with and evaluate. It’s a multimodal mixture-of-experts (MoE) model with a 125B-parameter main model supplemented by an additional 51B N-gram embeddings, with 6B parameters activated per token. It has a native 262,144-token context window, extensible to 1M tokens with YaRN. NVIDIA provides best-effort Day 0 functional support through SGLang, vLLM, and NVIDIA TensorRT LLM, validation across NVIDIA GB300 NVL72 for inference, and post-training recipes from NVIDIA NeMo AutoModel and NVIDIA NeMo RL. ## Architectural innovations for long-context inference Qwen3.8-Flash-Next is designed for high-volume, context-intensive applications such as agentic coding, document processing, and tool-driven workflows. As context grows, attention compute and KV cache memory become bottlenecks. The model addresses both with a hybrid architecture combining Gated DeltaNet (GDN) and Qwen Sparse Attention (QSA). Three out of every four layers use GDN to continuously compress historical context into a fixed-size recurrent state, eliminating KV cache growth as sequences lengthen. The remaining layer uses QSA for precise retrieval across the full context. Previous sparse-attention approaches rely on token-level indexers that become increasingly computationally expensive as context length grows. QSA aggregates the sequence into micro-blocks, estimates their importance at the block level, and selects only the most relevant regions. This cuts attention, compute, and indexing overhead within each layer, making the design well-suited to architectures alternating between GDN and QSA layers. Alibaba’s [published benchmarks](https://qwen.ai/blog?id=qwen3.8-flash-next) suggest that QSA can improve the efficiency of 1M-token workloads. Compared with full attention, its attention kernel delivered speedups of up to 7.6x during prefill and 4.9x during decoding. In a cache-heavy online serving test at a 1M-token context length and with a 90% prefix-cache hit rate, Qwen3.8-Flash-Next achieved 8.6x the prefill throughput of Qwen3.7-Plus. ## Running Qwen3.8-Flash-Next on NVIDIA GB300 NVL72 The GB300 NVL72 features a rack-scale architecture that integrates 72 NVIDIA Blackwell Ultra GPUs into a single platform. Its large, 72-GPU NVIDIA NVLink domain enables efficient all-to-all communication at 130 TB/s, eliminating bottlenecks that appear when expert traffic must cross traditional off-the-shelf networks. Running on NVIDIA GB300 NVL72 delivers over 16K tokens per second per GPU **and over 200 tokens per second per user,** enabling developers to experiment with agentic coding applications at high throughput and low latency. Beyond rack-scale deployment, Qwen3.8-Flash-Next also runs on local NVIDIA hardware, including NVIDIA DGX Station, NVIDIA DGX Spark clusters, and workstations with four NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition GPUs. Developers can prototype and evaluate agentic coding workflows on local hardware and scale the same model to GB300 NVL72 for production serving. ## Post-train Qwen3.8-Flash-Next and serve it with your preferred inference engine Developers can fine-tune the model for domain-specific use cases using [NVIDIA NeMo AutoModel](https://github.com/NVIDIA-NeMo/Automodel/tree/main/docs/model-coverage/llm/qwen/qwen3-8-flash-next.mdx), a PyTorch-native fine-tuning library with Day-0 Hugging Face checkpoint support. Train directly on existing checkpoints without model conversion, with support for full SFT or memory-efficient LoRA fine-tuning. Users can go a step to perform reinforcement learning using NVIDIA [NeMo RL recipes](https://github.com/NVIDIA-NeMo/RL/blob/qwen3-8-flash-next-support/docs/guides/models/qwen/qwen3-8-flash-next.md). NVIDIA supports multiple inference stacks to meet a variety of developer needs. [SGLang](https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.8-Flash-Next), [vLLM](https://recipes.vllm.ai/Qwen/Qwen3.8-Flash-Next), and [TokenSpeed](https://lightseek.org/tokenspeed/recipes/models#qwen3-8-flash-next) provide open-source inference recipes for developers requiring greater control over performance on the NVIDIA-accelerated platform. ## Get started with Qwen3.8-Flash-Next Try the model directly from [QwenCloud.](https://www.qwencloud.com/models/qwen3.8-flash?spm=a2ty_o06.30285417.0.0.1d73c921wJU38r&file=qwen3.8-flash) Download the model weights from [Hugging Face](https://huggingface.co/nvidia/Qwen3.8-Flash-Next-NVFP4) or [ModelScope](https://modelscope.cn/models/Qwen/Qwen3.8-Flash-Next) and [deploy](https://docs.nvidia.com/nim/vision-language-models/2.1.1/get-started/advanced/get-started-qwen3.8-flash-next.html) with a [model-free NVIDIA NIM](https://catalog.ngc.nvidia.com/orgs/nim/nvidia/containers/vllm-model-free-nim/-) from NVIDIA NGC.