Google has introduced SL2T (sign-language-to-text), a breakthrough translation model that marks a significant advancement in quality and generality for sign language AI. The technology powers sign-to-text dictation in Gboard and Live Transcribe on Android devices, starting with American Sign Language (ASL) to English translation.
Closing the Accessibility Gap
AI's ability to process spoken languages has advanced rapidly over recent decades, enabling automatic translation, dictation, and conversational interfaces that feel effortless to hearing users. Yet this technological revolution has not reached the world's more than 200 sign languages — and the estimated 70 million Deaf and hard of hearing people who use them.
SL2T brings sign language AI out of the lab and into consumer products for the first time. Similarly to how hearing users can use dictation to speak instead of typing, this feature enables Deaf users to sign to their phone anywhere they'd normally type — to search the web, draft messages or documents, and ask Gemini to solve queries or execute tasks.
Technical Breakthrough
Building AI for sign languages presents complex challenges. Sign languages are independent, natural languages with their own distinct grammars and lexicons, requiring true machine translation rather than sequential sign-to-word transformations. The model must also learn to "see" and understand physical movement — simultaneous movements of the hands, arms, torso, head, and face — a difficult and computationally demanding computer vision task.
SL2T addresses these challenges by combining a user-centric, culturally informed approach with massive data scaling. The model is trained on over 100,000 hours of data across more than 50 sign languages, with roughly a quarter of the data in ASL. Training jointly on diverse languages, dialects, and proficiency levels causes the model to learn shared underlying structures, outperforming single-language models.
Privacy-First Design
To protect user privacy, SL2T sees sign language as a sequence of pose landmark locations rather than a raw camera feed. An on-device model tracks the location of points on the signer, and only these geometric coordinates are sent to the server for translation, allowing the original video to be discarded immediately.
SL2T translates this coordinate sequence directly into text, bypassing intermediate annotations known as "glosses" that are widely used in prior work. Glosses fail to capture rich, non-linear aspects of sign languages such as non-manual markers and spatial constructions. Translating directly from landmarks removes artificial vocabulary limits and allows translation quality to scale directly with data.
Benchmark Performance
SL2T is the most capable sign language translation model to date according to key benchmarks like sd-test, which assesses ASL to English translation quality. The model achieves a remarkable zero-shot score of 70 BLEURT, significantly higher than any previously reported score. Beyond academic benchmarks, the team worked on practical issues like minimizing streaming latency, preventing hallucination on non-signing inputs, ensuring fairness for the 10% of signers who are left-handed, and improving performance for one-handed signing.
Community-Centered Development
Deaf perspectives have shaped every stage of this project — from conceptualization by Sam Sepah, a Deaf Googler, to data collection with Deaf partners, evaluation in Deaf user studies, and impact assessment with Deaf experts. Google established the AI Sign Language Advisory Committee (AISLAC), bringing together global Deaf organizations and subject-matter experts to guide responsible deployment.
SL2T builds upon decades of foundational research, but bringing ASL input to users' phones is only the beginning. The team is working to expand this technology into additional sign languages, sign language generation, and frontier AI capabilities.
You can experience SL2T in Gboard and Live Transcribe on Android devices at no additional cost, with more devices and languages coming soon.