editor.ijserin@gmail.com Peer-Reviewed · Open Access · Monthly Journal Submit Paper
International Journal of Scientific Engineering and Research ISSN 2347-3878  |  Peer Reviewed  |  Open Access  |  Monthly

Downloads: 1

India | Computer Science and Engineering | Volume 14 Issue 11, November 2026 | Pages: 1 - 12


Analysis of Vision Transformers for American Sign Language Recognition in Unconstrained Visual Conditions

Satwinder Kaur, Garima Joshi, Sumit Budhiraja, Sunil Agrawal

Abstract: Sign language is a spoken language for deaf people, but it becomes a tedious task for us to learn this language, thus creating a communication gap between them and us. The solution for this problem is automatic sign language recognition (SLR) systems that focus on translating hand gestures into their linguistic representations. But it is challenging for these systems to work in real-time due to noise in backgrounds, lighting effects, and having unbalanced datasets. The motivation of this research is to address these challenges with the help of Vision Transformer (ViT) architectures. We focus on static American Sign Language (ASL) benchmark datasets having different levels of complex backgrounds and human-induced noise. In this proposed work, stratified K-fold cross-validation and early stopping are used to reduce overfitting and tackle class imbalance, thus promoting model robustness. From experimental analysis, the ViT-based approach shows strong validation performance but notable generalization gaps when evaluated on inference datasets with a restricted variety or gesture classes that are similar in appearance. Although this is a limitation for deployment in real-time, findings reveal the importance of designing human-noise datasets that are deployment-oriented. This work describes the strengths and limitations of attention-based models and provides insights for designing a more robust framework for assistive communication applications.

Keywords: Sign Language Recognition, Vision Transformer, American Sign Language, Hand Gesture Recognition, Assistive Communication


View Article PDF


Rate This Article


Top