Recognizing gestures of the Ukrainian dactylic alphabet
Loading...
Date
2023
Authors
Hlybovets, Andrii
Bikchentayev, Mykola
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
Sign language is a visual way of communicating used by people who are deaf or hard of hearing. It involves handshapes, facial expressions, and body movements to con-vey meaning. Sign language helps the deaf community interact with each other and the hearing world, allowing them to participate fully in society. According to the WHO (World Health Organization) over 5 % of the world’s population – or 430 mil-lion people — experience problems with hearing. More than 44,000 people with hearing impairments are registered with the Ukrainian Society of the Deaf, an all-Ukrainian public organization for the disabled. Therefore, it is extremely important to develop new software, available to the public, that would allow quickly and effec-tively learn and understand sign language. This work aims to review gesture recogni-tion techniques and develop a system for detecting and classifying gestures of the Ukrainian dactylic alphabet. Two main approaches to gesture recognition, glove-based and computer vision-based (CV), are explained, with the latter being preferred due to its flexibility and widespread usage. The text elaborates on deep learning-based approaches, particularly LSTM networks, and the advantages they offer in au-tomatically learning features from raw image data. The process of creating a dataset for training the gesture classification model is described, which involves recording videos of hand gestures and extracting keypoints using Google MediaPipe. The mo-del training phase is detailed, covering the architecture of the LSTM-based classifier, optimization algorithms, and loss functions. The resulting model achieves an accura-cy of 98.4% on the test dataset. A program for real-time gesture recognition is deve-loped using Python and relevant libraries. The program utilizes a webcam feed to de-tect and classify hand gestures, displaying the top three predicted letters of the Ukrainian dactylic alphabet. The scientific novelty of the obtained results: the paper presents a method that utilizes hand keypoints for recognizing hand gestures of the Ukrainian dactyl alphabet. Also, as part of the development of the gesture recognition system, a data set was collected, where each gesture corresponds to 50 videos of 65 frames. The practical significance of the results obtained: the model obtained as a result of the study can be used to interpret the gestures of the Ukrainian dactylic al-phabet. The dataset collected for training this model can be used in other works to train or validate similar models. The paper might be of use to the ones who are inte-rested in developing similar systems for gesture recognition.
Description
Keywords
Ukrainian sign language (USL), Ukrainian dactyl alphabet, gesture recognition, LSTM and Google MediaPipe, article
Citation
Hlybovets A. Recognizing gestures of the Ukrainian dactylic alphabet / Andrii Hlybovets, Mykola Bikchentayev // Problems of Control and Informatics. - 2023. - Vol. 68, No. 3. - P. 86-100. - https://doi.org/10.34229/1028-0979-2023-3-9