Final Project Abstract

American Sign Language Hand-Sign Classifictation

Throughout the coronavirus pandemic, communication has been an essentail component to getting through the tragedy of it all. As the world begins to reopen, face masks are still a necessary part of everyday wear to help protect our most vulnerable populations; however, these masks also cause issues for the Deaf and Hard of Hearing communities. These communities utilize mouth movements and facial expressions as an important component of American Sign Language. While ASL goes beyond facial expressions and signs, the exclusion of one of these factors leads to impediments in conversations, and makes communicating with the hearing community more of a challenge due to the inability of lip reading. As inclusivity increases online and in the real world, having a tool that provided instant translations of non-spoken languages like ASL would help in increasing accessibility of communication between Deaf and hearing individuals. This leads to my problem statement: how can I use a convolutional neural network to create a model that accurately predicts ASL hand-signs? To begin to solve this problem, I used the images in an ASL hand-sign dataset to create a CNN that was able to correctly classify images at about a 78% accuracy. Issues such as CPU constraints may have contributed to the ability to run a model that achieved higher accuracies. However, present research shows that the creation of accurate models using neural netoworks is achievable, with studies conducted by those from students at John Hopkins University.

Sources

American Sign Language Hand Gesture Recognition. Medium. https://towardsdatascience.com/american-sign-language-hand-gesture-recognition-f1c4468fb177

Chew, A., Mahmood, A., Silver, Z., Tamhane, A., & Huang, W.-L. (n.d.). ASL Hand Sign Classification and Prediction. Dias, R. (2019, December 18)

Gandhi, Rohith. “Improving the Performance of a Neural Network.” Medium, Towards Data Science, 17 May 2018, towardsdatascience.com/how-to-increase-the-accuracy-of-a-neural-network-9f5d1c6f407d.

Thakur, Ayush. ASL Sign Language Dataset. https://www.kaggle.com/ayuraj/asl-dataset