Strengthening communication channels for people with hearing and speech disabilities in basic education settings through motion capture using sign languages
Article Sidebar
Main Article Content
The implementation of a deductive model aimed at vocabulary recognition based on Colombian Sign Language (CL) gestures is presented, which focuses on a solution to the lack of knowledge and accompaniment in its learning in people who are continuously related to this population with a speech and hearing disability. These hand gestures are used due to their high level of expressiveness and are the main source of communication for people with this type of disability.
Within the recognition of movement patterns and gestures in LSC, it is necessary to perceive and recognize the location, orientation, point of articulation and contact of the hands. Knowing about the technologies and research on gesture recognition algorithms, pattern analysis and neural networks that helped the correct selection of the implemented deductive model.
In such a way that the implementation of the image recognition model allowed to analyze frames and/or real images, analyzing important information and solving specific problems. These characteristics are focused within this research, managing to accompany the population in educational environments, by detecting objects in an image in real time using part of the artificial intelligence exposed in the deductive model.
Muñoz I, Ruiz M, Darbet C, Lago E, Fernández E. Comunidades sordas: ¿pacientes o ciudadanas?Deaf communities: patients or citizens? ScienceDirect [Internet]. 2011;25(1):72–8. Available from: https://doi.org/10.1016/j.gaceta.2010.09.020
Espectador E. Inclusión en la educación: estudiantes sordos en Colombia. In: Espectador E, editor. Maleducados. Bogota, Colombia; 2021.
González C, Yimes F. Sistema de reconocimiento gestual de lengua de señas Chilena mediante cámara digital [Internet]. PONTIFICIA UNIVERSIDAD CATÓLICA DE VALPARAÍSO; 2016. Available from: http://opac.pucv.cl/pucv_txt/txt-0500/UCC0990_01.pdf
Pichuho J, Constante P, Gordón A, Mendoza D. Interpretación de lenguaje de señas ecuatoriano empleando visión por computador. Ibérica Sist e Tecnol Inf [Internet]. 2019;(960–971):13. Available from: https://www.proquest.com/openview/84337a90042da817ad3453747249970a/1?pq-origsite=gscholar&cbl=1006393
Guerrero J. Algoritmos de procesamiento de imágenes y redes neuronales artificiales para el reconocimiento de la lengua de señs colombiana (LSC). Rev Colomb Tecnol Av [Internet]. 2016;2(28):1–8. Available from: https://www.unipamplona.edu.co/unipamplona/portalIG/home_40/recursos/05_v25_30/revista_28/20052017/01.pdf
Muñoz Y, Moreno L. Implementación de un algoritmo para a clasifiación automática de leguaje de señas colombiano en video usando aprendizaje profundo [Internet]. Universidad Catóica de Colombia; 2020. Available from: https://repository.ucatolica.edu.co/bitstream/10983/24980/1/Proyecto de grado.pdf
Alvarez P, Castro G. Modelo de red neuronal convolucional para el reconocimiento del alfabeto en lenguaje de señas colombiano [Internet]. Universidad del Sinú; 2019. Available from: http://repositorio.unisinucartagena.edu.co:8080/xmlui/handle/123456789/56
Barragán E, Lozano S. Identificación temprana de trastornos del lenguaje Early identification of language disorders. ScienceDirect [Internet]. 2011;22:227–32. Available from: https://www.sciencedirect.com/science/article/pii/S0716864011704175?via%3Dihub
Tzuta L. LabelImg [Internet]. Github. 2020. Available from: https://github.com/tzutalin/labelImg#labelimg
Nabende J, Tusubira J, Babirye C, Nsumba S, Omongo Christopher. A dataset of cassava whitefly count images. ScienceDirect [Internet]. 2022;41. Available from: https://www.sciencedirect.com/science/article/pii/S2352340922001238#cit_1
Everingham M, Eslami A, Gool L, Williams C, Winn J, Zisserman A. The PASCAL Visual Object Classes Challenge: A Retrospective. Int J Comput Vis Vol [Internet]. 2015;111:98–136. Available from: https://link.springer.com/article/10.1007/s11263-014-0733-5
Martinez E, Morillas F. Deep Learning Techniques for Spanish Sign Language Interpretation. Hindawi [Internet]. 2021;2021:10. Available from: https://doi.org/10.1155/2021/5532580
Brownlee J. How Do Convolutional Layers Work in Deep Learning Neural Networks? [Internet]. Machine Learning Mastery. 2019. Available from: https://machinelearningmastery.com/deep-learning-for-computer-vision/
Adeyanju I, Bello O, Adegboye M. Machine learning methods for sign language recognition: A critical review and analysis. Intell Syst with Appl [Internet]. 2021; Available from: https://doi.org/10.1016/j.iswa.2021.200056
Liang T, Bao H, Pan W, Pan F. Traffic Sign Detection via Improved Sparse R-CNN for Autonomous Vehicles. Hindawi [Internet]. 2022;2022:16. Available from: https://doi.org/10.1155/2022/3825532
Farfán O, Camargo J. Modelo computacional para reconocimiento de lenguaje de señas en un contexto colombiano. TecnoLógias [Internet]. 2020;23(48):197–232. Available from: https://revistas.itm.edu.co/index.php/tecnologicas/article/view/1585/1637
Tensorflow. TensorFlow JS [Internet]. npm. 2022. Available from: https://www.npmjs.com/package/@tensorflow/tfjs
Anirudh R. Convolutional Neural Network Tutorial (CNN) – Developing An Image Classifier In Python Using TensorFlow. edureka [Internet]. 2020;7. Available from: https://www.edureka.co/blog/convolutional-neural-network/
Lu, Jiaqi Liu, Ruiqing Yuejuan, Zhang Zhang, Xiaxiang Zheng, Longbo Zhang, Chao Zhang, Kaiming Li, Shuan Lu Y. Research on the development and application of a detection platform for colorectal cancer tumor sprouting pathological characteristics based on artificial intelligence. ScienceDirect [Internet]. 221AD;6. Available from: https://doi.org/10.1016/j.imed.2021.08.003
Cabrera J, Cervantes J, Lamont G, Ruiz J, Jalili L. Mexican sign language segmentation using color based neuronal networks to detect the individual skin color. Expert Syst Appl [Internet]. 2021; Available from: https://doi.org/10.1016/j.eswa.2021.115295
Wei X, Chen X, Lai C, Zhu Y, Yang H, Du Y. Automatic Liver Segmentation in CT Images with Enhanced GAN and Mask Region-Based CNN Architectures. Hindawi [Internet]. 2021;2021:11. Available from: https://doi.org/10.1155/2021/9956983
Guo X, Liu S. A Scatter Search Approach for Multiobjective Selective Disassembly Sequence Problem. Hindawi [Internet]. 2014;9. Available from: https://doi.org/10.1155/2014/756891
Programmer Click. SSD para detección de objetos en tiempo real [Internet]. 2020. Available from: https://programmerclick.com/article/91891967042/
Cloud I. IBM Cloud Object Storage [Internet]. IBM. 2022. Available from: https://www.ibm.com/co-es/cloud/object-storage
Vayghan L, Saied M, Toeroe M, Ferhat K. A Kubernetes controller for managing the availability of elastic microservice based stateful applications. J Syst Softw [Internet]. 2021; Available from: https://doi.org/10.1016/j.jss.2021.110924
Kubernetes. ¿Qué es Kubernetes? [Internet]. Documentación distribuida Kubernetes. 2022. Available from: https://kubernetes.io/es/docs/concepts/overview/what-is-kubernetes/
Accepted 2022-10-31
Published 2023-01-15
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Authors grant the journal and Universidad del Valle the economic rights over accepted manuscripts, but may make any reuse they deem appropriate for professional, educational, academic or scientific reasons, in accordance with the terms of the license granted by the journal to all its articles.
Articles will be published under the Creative Commons 4.0 BY-NC-SA licence (Attribution-NonCommercial-ShareAlike).