Granite Switch 4.1 Combines Chat, Decisions, and Speech on a Single Endpoint

Granite Switch 4.1 Combines Chat, Decisions, and Speech on a Single Endpoint IBM has introduced Granite Switch 4.1, enabling 8B and 30B models to handle speech-to-text, natural-language generation, and general or specialized decisions from a single hosted endpoint. Granite-Switch-4.1-8B and -30B integrate a general Decision aLoRA, a safety-focused Guardian aLoRA, and the Granite-Speech-5.0-Turbo-CTC ASR model into one base model. The architecture occupies approximately the same GPU VRAM as the standalone base model and reuses KV cache across base generations and adapter-based decisions. Granite-Switch-4.1-30B finished a 5-step RAG workflow faster than Jev-27B, while a custom 3B model played Doom at 60–100 decisions per second compared to Jev's 10. Users can compose custom models via Granite Libraries, with an Adapter SDK planned for release in Mellea. Integrated Architecture and Efficiency Granite-Switch-4.1 models combine a base model with an embedded Granite-Speech-5.0-Turbo-CTC ASR encoder and cache-efficient Activated LoRAs (aLoRAs), including a general Decision aLoRA and a specialized Guardian aLoRA for safety tasks. The composed models use roughly the same GPU VRAM as the base model alone. Because aLoRAs reuse the base model's KV cache, the system can switch between chat, transcription, and decision-making at high speeds with the help of vLLM optimizations. Performance and Benchmarks The Guardian aLoRA outperforms models such as Jev 1.13, Microsoft-Decision-1, GPT-Luna-Decisions, and Clef-Flash on safety-related decision tasks. In a benchmark evaluation of a real-world, five-step RAG workflow, Granite-Switch-4.1-30B equipped with four specialized aLoRAs completed the pipeline faster than Jev-27B. Additionally, an experimental 3B parameter Granite Switch model navigated the game Doom at 60 to 100 decisions per second—surpassing Jev's reported 10 decisions per second—while retaining the ability to simultaneously narrate gameplay and answer questions. Customization and Future Tooling While Granite-Switch-4.1-8B and -30B come pre-configured, users can build custom models by selecting task-specific adapters from Granite Libraries. Adding specialized adapters alongside generalist decision models incurs minimal overhead due to shared KV cache. The team also plans to release an Adapter SDK for training in Mellea alongside supplementary tutorials.

IBM发布Granite Switch 4.1：单端点整合对话、决策与语音转录 IBM推出Granite Switch 4.1模块化模型系列，涵盖8B与30B版本。该架构将通用决策、安全审查以及语音识别组件整合至单一端点，仅占用相当于基础模型的显存，并支持快速任务切换。 Granite-Switch-4.1内置通用决策aLoRA、安全决策Guardian aLoRA以及OpenASR榜单领先的语音识别模型Granite-Speech-5.0-Turbo-CTC。 采用Activated LoRA技术并结合vLLM优化，各适配器可复用基础模型的KV缓存，实现多任务间低延迟切换且显存占用极低。 在5步RAG工作流实测中，30B模型完成速度超越Jev-27B；定制的3B模型在经典游戏《毁灭战士》中达到每秒60至100次决策。 用户可通过Granite Libraries自行组合适配器，官方计划推出Mellea适配器训练SDK及教程。 单端点多任务架构与显存优化 Granite-Switch-4.1-8B与30B将通用决策适配器、专注于安全决策的Guardian aLoRA以及语音转录模型嵌入至同一基础模型中。通过构建高效缓存的Activated LoRA（aLoRA），适配器可复用基础模型生成的KV缓存，并在单端点内实现对话、决策和语音任务的高速切换，整体显存开销基本等同于单独的基础模型。 基准对比与实际性能 测试显示Guardian aLoRA在安全决策任务上平均表现优于Jev 1.13、Microsoft-Decision-1、GPT-Luna-Decisions与Clef-Flash。在包含4个特化aLoRA的Granite-Switch-4.1-30B与采用单一通用LoRA的Jev-27B的5步RAG流程对比中，Granite速度更快。此外，3B参数的定制版本在运行《毁灭战士》时可实现每秒60至100次决策，且能在游玩过程中同步进行旁白和回答提问。 模块化生态与后续工具支持 用户可利用Granite Libraries中的特化适配器仓库，自由组合并拓展自定义的Granite Switch模型。团队计划推出用于在Mellea中训练适配器的Adapter SDK及相关教程，以支持社区为工作流构建专属的分类或特定语言任务适配器。

Introduced last spring, the Granite Switch project enables a modular approach to model development. Now Granite Switch 4.1 models, armed with a Decision adapter and the fastest ASR model on the OpenASR leaderboard, can move efficiently between natural-language responses, general and specialized decisions, and speech-to-text tasks—all from one hosted endpoint. Specifically, Granite-Switch-4.1-8B and -30B models are configured to include: A new Decision aLoRA for general-purpose decision-making (e.g. v1/systemone), A specialized Guardian aLoRA that, on average, outperforms Jev 1.13, Microsoft-Decision-1, GPT-Luna-Decisions, and Clef-Flash on safety-related decision tasks, and Granite-Speech-5.0-Turbo-CTC , the fastest ASR model on the OpenASR Leaderboard , embedded for use in transcription tasks. All three of these technologies (along with the underlying base model) are composed into one model, taking up approximately the same amount of VRAM on a GPU as the base model would by itself. That means more capabilities with fewer GPUs. And because the Switch adapters were built as cache-efficient Activated LoRAs , the KV cache created from a base model generation can be reused for any subsequent adapter-based decisions. Combined with our team's recent vLLM contributions to speed up low rank adapter inferencing, as well as a co-located ASR encoder, Granite Switch 4.1 models can switch between a chat request and a decision request and a spoken request, with blazing fast speeds. Check out our Speech-to-Decisions HF space where granite-switch-4.1-8b transcribes audio, generates summaries, screens for harmful content, and tests 15 other user-defined flags—all running on a single endpoint in real time. Granite Libraries: The OG Decision Models Decision models aren’t new. Our previous release of Granite Libraries features a repository of specialized “decision model” adapters and other specialized task adapters that can be composed into custom Granite Switch models. And while Jev and other closed decision models focus on generality, we’ve found that many of our specialized granitelib adapters can outperform generalists on specialized classification tasks. Compare granitelib adapters with generalist decision models when evaluated on specialized classification tasks. Because adapters share the base model's KV cache, adding specialists alongside a generalist decision adapter costs almost nothing. You get broad coverage from the generalist and higher accuracy from specialists on the tasks you care about. In a “Race” between Granite-Switch-4.1-30B (frozen base Granite + 4 specialized aLoRAs) and Jev-27B (frozen base Qwen model + 1 generic decision LoRA), Granite-Switch-4.1-30B was able to complete a real world, 5-step RAG workflow much faster than its competitor. Compose and Extend Your Own Granite Switch Model Granite-Switch-4.1-8B and -30B are released with “batteries included”. They are Granite Switch models that we compiled with a predefined set of helpful adapters. But you don’t need to use our version. Anyone can compose their own Granite Switch model , mixing and matching adapters from Granite Libraries to best meet their needs. Moving forward, we see tremendous opportunity for the community to compose and extend Granite Switch models with custom and third party adapters. We plan to release an Adapter SDK for training adapters in Mellea , along side other tutorials and Granite Switch support. These adapters could be specialized classification adapters to augment decision-based workflows or adapters for specialized language tasks. You could even use custom adapters to teach a 3B parameter version of Granite to play the classic game Doom. Similarly to Jev, our custom 3B parameter Granite Switch model made split second decisions to navigate the Doom game, but was able to do so at 60 - 100 decisions / second, compared to Jev’s reported 10 decisions / second. 1 Unlike Jev, this Granite Switch model can also narrate and respond to your questions while it is playing. We look forward to releasing more details and data on Granite Switch in the upcoming weeks, but until then, we encourage you to try out our latest Switch models and these other helpful resources for getting started. Resources for Getting Started Granite Switch 4.1 Model Cards Speech-to-Decisions Demo Granite Switch GitHub Repo Original Granite Switch and Granite Libraries Release Blog Create your own aLoRA v LoRA race on Colab . Footnotes 1 https://typesafe.ai/blog/introducing-system-one-models-and-jev