Google DeepMind Launches SL2T Model for Real-Time Sign Language Dictation on Pixel 11

Google DeepMind Launches SL2T Model for Real-Time Sign Language Dictation on Pixel 11 Google DeepMind has introduced sign-language-to-text (SL2T), a multilingual translation model powering new sign-to-text dictation features in Gboard and Live Transcribe on Pixel 11, debuting with American Sign Language to English. SL2T enables users to sign to their phones for tasks such as web searches, messaging, and querying Gemini, starting with ASL on Pixel 11. The model was trained on over 100,000 hours of data spanning more than 50 sign languages, with approximately 25% in ASL. To safeguard privacy, an on-device MediaPipe Holistic model converts video to landmark coordinates, allowing the raw video to be discarded immediately. SL2T achieved a zero-shot score of 70 BLEURT on the FLEURS-ASL benchmark and was developed in partnership with the AI Sign Language Advisory Committee. Consumer Product Integration SL2T marks the transition of sign language AI from research labs into consumer hardware. Available initially on Pixel 11 via Gboard and Live Transcribe at no extra cost, it allows Deaf and hard-of-hearing users to sign directly instead of typing. Testers reported that signing in ASL was faster and more natural than typing English. Google plans to expand the capability to additional devices and languages. Technical Architecture and Privacy Unlike spoken language transcription, which maps sequential sounds to words, sign language processing demands full machine translation of complex, non-linear physical movements across the hands, body, and face. SL2T bypasses traditional intermediate 'glosses' to translate coordinate sequences directly into text. For user privacy, the on-device MediaPipe Holistic framework tracks body landmark coordinates and transmits only these geometric points to servers, instantly discarding raw camera footage. Performance and Real-World Optimizations Trained on more than 100,000 hours of multi-language sign data, SL2T recorded a zero-shot score of 70 BLEURT on the FLEURS-ASL benchmark. In addition to benchmark performance, the developers optimized the system for everyday use by lowering streaming latency, curbing hallucinations on non-signing input, and accommodating left-handed signers as well as one-handed signing when holding a phone. Community Collaboration and Future Goals The project was shaped alongside the Deaf community from its inception, originating from Deaf Googler Sam Sepah and guided by the AI Sign Language Advisory Committee. A joint impact report accompanied the launch to outline capabilities and remaining limitations, such as handling rare signs or fast fingerspelling. Google plans to continue expanding into further sign languages, generation capabilities, and broader accessibility tools.

Google DeepMind 推出手语转文本模型 SL2T，首发落地 Pixel 11 Google DeepMind 发布了多语种手语转文本（SL2T）翻译模型，首次将手语 AI 引入消费级产品。该模型首发搭载于 Pixel 11 的 Gboard 和即时转录功能中，支持美式手语（ASL）至英语的输入与翻译。 SL2T 率先在 Pixel 11 上为 Gboard 和即时转录提供手语输入支持，初期支持 ASL 到英语，后续将拓展至更多设备和语言。 模型基于超过 50 种手语的 10 万小时以上数据训练，其中约四分之一为 ASL 数据，跨语言联合训练表现优于单语言模型。 采用设备端 MediaPipe Holistic 提取姿态关键点坐标传输至服务端，原始视频即刻丢弃以保护隐私，且绕过传统手语标注（glosses）实现端到端翻译。 在 FLEURS-ASL 基准测试中取得 70 BLEURT 的零样本评分，并针对流式延迟、左撇子手语及单手手语进行了优化。 产品应用与手语技术难点 全球约有 7000 万聋人和听障人士使用超过 200 种手语。SL2T 使聋人用户能够像健听用户使用语音听写一样，通过打手语进行搜索、发信息或与 Gemini 交互。与语音转录不同，手语是拥有独立语法和词汇的自然语言，需要真正的机器翻译；同时，手语依赖手部、面部、头部和躯干的连贯协同动作，需要高帧率的高难度计算机视觉感知。 技术架构与隐私保护机制 SL2T 摒弃了传统的中间层手语标注（glosses），解决了其无法捕捉非手动标记和空间结构的问题。为保护隐私，设备端利用 MediaPipe Holistic 将手语动作转换为人体姿态关键点几何坐标，仅将坐标发送至服务器进行文本翻译，原始摄像头画面会被立即丢弃。 基准表现与实际使用优化 在 FLEURS-ASL 测试中，SL2T 达到了创纪录的 70 BLEURT 零样本评分。针对实际应用，研发团队重点优化了流式延迟，防止非手语输入产生幻觉，并兼顾了 10% 的左撇子手势习惯以及单手握持手机时的单手手语识别。模型在生僻手势、快速指拼和无上下文语境的时态表达上仍偶有识别误差。 社区共建与未来规划 该项目从构思、数据收集到评估均由聋人深度参与。Google 成立了由全球聋人组织和专家组成的 AI 手语咨询委员会（AISLAC）以推动负责任的治理，并发布了联合影响报告。团队未来的目标是进一步支持更多手语类型、研发手语生成能力并推进前沿 AI 技术的融合。

August 12, 2026 Models Google DeepMind Sign Language Team Introducing sign-language-to-text (SL2T), our breakthrough model powering new sign language features for Deaf and hard of hearing users. AI's ability to process spoken languages has advanced rapidly over recent decades, enabling automatic translation, dictation, and conversational interfaces that feel effortless to hearing users. Yet this technological revolution has not reached the world’s more than 200 sign languages — and the estimated 70 million Deaf and hard of hearing people who use them. Today, we’re introducing a massively multilingual sign-language-to-text (SL2T) translation model that marks a breakthrough in quality and generality. With it, we are bringing sign language AI out of the lab and into consumer products for the first time: SL2T powers sign-to-text dictation in Gboard and Live Transcribe on Pixel 11 , starting with American Sign Language (ASL) to English. More devices are coming soon, and additional languages will follow. Similarly to how hearing users can use dictation to speak instead of typing, this feature enables Deaf users to sign to their phone anywhere they’d normally type. You can sign to search the web, draft messages or documents, and ask Gemini to solve queries or execute tasks. In Live Transcribe, you can sign responses in conversations instead of having to type back and forth. According to our testers, signing in ASL is faster, more natural, and more delightful than typing in English. Sign-to-text, powered by SL2T, enables users to sign to their phone anywhere they'd normally type. Why sign languages matter Sign languages are the primary languages of Deaf communities around the world and the cornerstone of Deaf cultural identity. There is great diversity among deaf people in terms of their level of proficiency in signing, speaking, reading, and writing, so it is important to support access in all modalities. Deaf people can benefit from sign language processing in the same way that hearing people benefit from spoken language processing, plus the technology opens new possibilities for bridging the communication gap between Deaf and hearing communities. Despite this opportunity for positive social impact, progress in sign language AI has been slow — both because building AI for sign languages presents complex challenges and because widespread misconceptions exist about how the languages themselves work. Compared to spoken language transcription, sign language translation presents two core challenges. First, transcribing speech is a matter of performing a sequential mapping from sound to text in the same language, whereas sign languages are independent, natural languages with their own distinct grammars and lexicons. As a result, they require true machine translation rather than a sequential process of sign-to-word transformations. Second, the model must learn to “see” and understand physical movement. Sign languages convey meaning through simultaneous movements of the hands, arms, torso, head, and face. Accurately tracking these at high frame rates is a difficult and computationally demanding computer vision task. Given this background, it is easy to understand why some early attempts at sign language technology, like sign language gloves, were fundamentally limited: sign languages aren't simply “English on the hands.” They require complex visual perception of fine-grained whole-body movements and full-fledged language translation. SL2T is designed to deliver both. SL2T sees sign language inputs as points on the signer's body and translates them into streaming text outputs. Example from the FLEURS-ASL benchmark. How SL2T works We built SL2T by combining a user-centric, culturally informed approach with massive data scaling. The model is trained on over 100,000 hours of data across more than 50 sign languages — with roughly a quarter of the data in ASL. Training jointly on diverse languages, dialects, and proficiency levels causes the model to learn shared underlying structures, outperforming single-language models in our experiments. To protect user privacy, SL2T sees sign language as a sequence of pose landmark locations rather than a raw camera feed. An on-device model ( MediaPipe Holistic ) tracks the location of points on the signer, and only these geometric coordinates are sent to the server for translation, allowing the original video to be discarded immediately. SL2T translates this coordinate sequence directly into text, bypassing intermediate annotations known as “glosses” that are widely used in prior work on sign language translation. Glosses fail to capture rich, non-linear aspects of sign languages such as non-manual markers and spatial constructions. Translating directly from landmarks removes artificial vocabulary limits and allows translation quality to scale directly with data. SL2T is the most capable sign language translation model to date according to key benchmarks like FLEURS-ASL (sd-test), which assesses ASL to English translation quality. SL2T achieves a remarkable zero-shot score of 70 BLEURT, which is significantly higher than any previously reported score. But optimizing academic benchmarks alone doesn’t guarantee usability in real-world applications, so we worked hard on practical issues like minimizing streaming latency, preventing hallucination on non-signing inputs, ensuring fairness for the 10% of signers who are left-handed, and improving performance for one-handed signing, which is used while holding a smartphone in the other hand. Original English SL2T's ASL → English output The Cook Islands do not have any cities but are composed of 15 different islands. The main ones are Rarotonga and Aitutaki. The Cook Islands have no cities and consist of 15 islands. The two main islands are Rarotonga and Aitutaki. The games kicked off at 10:00am with great weather and apart from mid morning drizzle which quickly cleared up, it was a perfect day for 7's rugby. Games start at 10 a.m. in great weather. There is a light rain in the morning that clears up. It's a perfect day for 7v7 rugby. In some federal countries, such as the United States and Canada, income tax is levied both at the federal level and at the local level, so the rates and brackets can vary from region to region. In some federal countries, like the US and Canada, income tax is collected at both the federal and local levels. This means that the rates and brackets vary depending on your region. This fully feathered, warm blooded bird of prey was believed to have walked upright on two legs with claws like the Velociraptor. This creature is warm-blooded, eats grey, and is covered in feathers. It is believed that it walks on two legs like a velociraptor. Maybe one day, your great grandchildren will be standing atop an alien world wondering about their ancient ancestors? Maybe one day your great-grandchildren will stand on an alien world and reflect on their ancestors. Examples from the FLEURS-ASL benchmark. SL2T accurately translates complex ASL into fluent English. Occasional errors remain in rare signs, rapid fingerspelling ("prey" → "grey"), passive constructions, classifier depictions (dropping "claws"), and tense without context ("kicked off" → "start"). Building with the community We believe in building with the Deaf community, not just for it. Deaf perspectives have shaped every stage of this project — from conceptualization by Sam Sepah, a Deaf Googler, to data collection with Deaf partners, evaluation in Deaf user studies, and impact assessment of the technology with Deaf experts. To guide responsible real-world deployment, we established the AI Sign Language Advisory Committee (AISLAC), bringing together many global Deaf organizations and subject-matter experts. Through this participatory governance model, the communities most impacted by our technology directly influence our development priorities. We co-authored a joint impact report for the release of SL2T 1.0 in Gboard and Live Transcribe, transparently detailing the technology's capabilities and current limitations — a collaborative approach we plan to continue for all major sign language releases. Looking ahead SL2T builds upon decades of foundational research across academia and industry, but bringing ASL input to users’ phones is only the beginning. Google’s mission is to organize the world's information and make it universally accessible and useful. Achieving universal accessibility means reaching full parity with spoken and written languages. Our team is working to expand this technology into additional sign languages, sign language generation, and frontier AI capabilities. We look forward to sharing our progress responsibly in order to make access through sign languages standard across the digital landscape. You can experience SL2T in Gboard and Live Transcribe first on Pixel 11 , with more devices coming soon — all at no additional cost. Acknowledgements This work was done jointly by teams from Google DeepMind and Android. The core team who developed the SL2T model is: Garrett Tanzer, Benoit Brard, Elizabeth Clark, Tim Dozat, Sebastian Ebert, Dan Garrette, Manfred Georg, Vicky Holgate, Shankar Kumar, Mohammad Saboorian, Miloš Stanojević, Megh Umekar, John Wieting, Andy Zhang, and Chris Dyer. The Android team who integrated the model into Gboard and Live Transcribe is: Ausmus Chang, Sai Aditya Chitturu, Dayle Chiu, Anna Chou, Ajay Dudani, Angana Ghosh, Alex Huang, Joanne Kim, Ed Lee, Thomas Lin, James Su, Yanchao Su, and Sharlene Yuan. We are grateful for additional support from Anelia Angelova, Abhishek Bapna, Sara Basson, Glenn Cameron, Scott Crowell, Trevor Cohn, Noah Fiedel, Zoubin Ghahramani, Raia Hadsell, Tom Hudson, Alexander Hauerslev Jensen, Kazuya Kawakami, Phoebe Kirk, Peike Li, Liam McCafferty, Caroline Pantofaru, Abhinav Parashar, Christopher Patnoe, Laura Rimell, Sagar Savla, Sam Sepah, Thad Starner, Dave Uthus, and Biao Zhang. We would also like to acknowledge the MediaPipe Holistic (Google AI Edge) team: Ivan Grishchenko, Artsiom Ablavatski, Valentin Bazarevsky, Esha Uboweja, George Sung, Jonathan Baccash, Gregory Karpiak, Sebastian Schmidt, Suril Shah, Raman Sarokin and Juhyun Lee. Many thanks also go to those who participated in early stage testing of our models.