Recognizing gestures of the Ukrainian dactylic alphabet

dc.contributor.authorHlybovets, Andrii
dc.contributor.authorBikchentayev, Mykola
dc.date.accessioned2024-10-07T13:09:44Z
dc.date.available2024-10-07T13:09:44Z
dc.date.issued2023
dc.description.abstractSign language is a visual way of communicating used by people who are deaf or hard of hearing. It involves handshapes, facial expressions, and body movements to con-vey meaning. Sign language helps the deaf community interact with each other and the hearing world, allowing them to participate fully in society. According to the WHO (World Health Organization) over 5 % of the world’s population – or 430 mil-lion people — experience problems with hearing. More than 44,000 people with hearing impairments are registered with the Ukrainian Society of the Deaf, an all-Ukrainian public organization for the disabled. Therefore, it is extremely important to develop new software, available to the public, that would allow quickly and effec-tively learn and understand sign language. This work aims to review gesture recogni-tion techniques and develop a system for detecting and classifying gestures of the Ukrainian dactylic alphabet. Two main approaches to gesture recognition, glove-based and computer vision-based (CV), are explained, with the latter being preferred due to its flexibility and widespread usage. The text elaborates on deep learning-based approaches, particularly LSTM networks, and the advantages they offer in au-tomatically learning features from raw image data. The process of creating a dataset for training the gesture classification model is described, which involves recording videos of hand gestures and extracting keypoints using Google MediaPipe. The mo-del training phase is detailed, covering the architecture of the LSTM-based classifier, optimization algorithms, and loss functions. The resulting model achieves an accura-cy of 98.4% on the test dataset. A program for real-time gesture recognition is deve-loped using Python and relevant libraries. The program utilizes a webcam feed to de-tect and classify hand gestures, displaying the top three predicted letters of the Ukrainian dactylic alphabet. The scientific novelty of the obtained results: the paper presents a method that utilizes hand keypoints for recognizing hand gestures of the Ukrainian dactyl alphabet. Also, as part of the development of the gesture recognition system, a data set was collected, where each gesture corresponds to 50 videos of 65 frames. The practical significance of the results obtained: the model obtained as a result of the study can be used to interpret the gestures of the Ukrainian dactylic al-phabet. The dataset collected for training this model can be used in other works to train or validate similar models. The paper might be of use to the ones who are inte-rested in developing similar systems for gesture recognition.en_US
dc.identifier.citationHlybovets A. Recognizing gestures of the Ukrainian dactylic alphabet / Andrii Hlybovets, Mykola Bikchentayev // Problems of Control and Informatics. - 2023. - Vol. 68, No. 3. - P. 86-100. - https://doi.org/10.34229/1028-0979-2023-3-9en_US
dc.identifier.issn2786-6491
dc.identifier.urihttps://doi.org/10.34229/1028-0979-2023-3-9
dc.identifier.urihttps://ekmair.ukma.edu.ua/handle/123456789/31770
dc.language.isoenen_US
dc.relation.sourceProblems of Control and Informaticsen_US
dc.statusfirst publisheduk_UA
dc.subjectUkrainian sign language (USL)en_US
dc.subjectUkrainian dactyl alphabeten_US
dc.subjectgesture recognitionen_US
dc.subjectLSTM and Google MediaPipeen_US
dc.subjectarticleen_US
dc.titleRecognizing gestures of the Ukrainian dactylic alphabeten_US
dc.typeArticleuk_UA
Files
Original bundle
Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
Hlybovets_Bikchentayev_Recognizing_gestures_of_the_Ukrainian_dactylic_alphabet.pdf
Size:
1019.57 KB
Format:
Adobe Portable Document Format
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
1.71 KB
Format:
Item-specific license agreed upon to submission
Description: