The idea comes from one of my friend who is having difficulty using zoom and such inequality may affect their work and study performance during these times of coronavirus pandemic, or even the post-pandemic era. My objective is to think of a way to lower the communication gap between the mute and sound community and aiming to build an Accessible environment for all of us.
The solution I propose here is an AI translator of Hong Kong Sign language. It can detect both single-hand and multi-hand sign gestures, based on the reference of Hong Kong Sign Language Browser. It is available for real-time and video translation for translating 25 words at 90% accuracy.
This project uses MediaPipe’s hand tracking detection for feature extraction and it is able to provide the landmark of precise key-point localization of 42 3D hand-knuckle coordinates inside the detected hand regions. By training the self-made training data set, which is a collection of 1500 hand landmark information of 25 words based on the reference of “Hong Kong Sign Language Browser”, with a LSTM model architecture built by tensorflow and Keras, the model is presented in the web application by taking either webcam video or uploaded video from the user and provide the hand sign translation result in the website. In the future, I hope this application can be made to use in daily live chat or online class in order to provide a new communication choice for the Hearing impaired community and the standard world.
Date & Time: 7 November 2020, Saturday 17:15-18:00
Speaker: Rush Wong
Rush is a university graduate majoring in political science and finishes the MicroMaster in A.I. and Programming Coding Bootcamp of Tecky Academy in August. With multi-discipline knowledge, he aspires to work in the field of machine learning.