EfficientNet B0 Algorithm for SIBI to Text and Audio Translation System
DOI:
https://doi.org/10.33795/jartel.v15i3.6942Keywords:
Sign Language System, Deep learning, Convolutional Neural Network, Pre-trained Model, Hyperparameter TuningAbstract
The Indonesian Sign Language System (SIBI) serves as the fundamental mode of communication for the deaf and mute community in Indonesia. However, a significant communication gap exists as the general public often struggles to comprehend these signs. Leveraging advancements in Artificial Intelligence (AI), specifically Machine Learning, offers a solution to bridge this gap and improve accessibility. This research focuses on developing a SIBI sign language translation system that converts gestures into both text and audio formats, utilizing a Convolutional Neural Network (CNN) based on the EfficientNet B0 architecture. The model underwent rigorous training with varying epoch counts, ultimately identifying 35 epochs as the optimal duration to maximize performance while mitigating overfitting and underfitting risks. Experimental results from distance testing revealed an inverse relationship between accuracy and camera-to-hand distance: 74% at 5 cm, 64% at 25 cm, 52% at 45 cm, 41% at 55 cm, 28% at 65 cm, 26% at 75 cm, 16% at 85 cm, and 0% at 95 cm. These findings underscore distance as a critical factor. Overall, the application achieved an average accuracy of 76%, demonstrating its potential to facilitate effective communication and promote social inclusivity.
References
K. Ali, Z. A. Shaikh, A. A. Khan, and A. A. Laghari, “Multiclass skin cancer classification using EfficientNets – a first step towards preventing skin cancer,” Neuroscience Informatics, vol. 2, no. 4, Art. no. 100034, 2022, doi: 10.1016/j.neuri.2021.100034.
A. Ma’aruf and M. Hardjianto, “Application of the You Only Look Once Version 8 algorithm for Indonesian Sign Language alphabet,” in Seminar Nasional Mahasiswa Fakultas Teknologi Informasi, vol. 2, pp. 567–576, Sep. 2023.
M. Zikky, Z. F. Akbar, and S. Utomo, “Kamus Sistem Isyarat Bahasa Indonesia (KASIBI) dengan voice recognition sebagai pendukung belajar bahasa isyarat berbasis Android,” JST (Jurnal Sains Terapan), vol. 5, no. 2, pp. 121–130, 2019, doi: 10.32487/jst.v5i2.732.
D. Permana and J. Sutopo, “Aplikasi pengenalan abjad Sistem Isyarat Bahasa Indonesia (SIBI) dengan algoritma YOLOv5,” J. Simantec, vol. 11, no. 2, pp. 231–240, 2023, doi: 10.21107/simantec.v11i2.19783.
A. R. Ardiansyah, A. H. Nur’azizan, and R. Fernandis, “Implementasi deteksi bahasa isyarat tangan menggunakan OpenCV dan MediaPipe,” in Stain. (Seminar Nasional Teknol. Sains), vol. 3, no. 1, pp. 331–337, 2024.
P. A. Nugroho, I. Fenriana, and R. Arijanto, “Implementasi deep learning menggunakan convolutional neural network (CNN) pada ekspresi manusia,” ALGOR, vol. 2, no. 1, pp. 12–21, 2020.
I. W. Suartika E. P., A. Y. Wijaya, and R. Soelaiman, “Klasifikasi citra menggunakan convolutional neural network (CNN) pada Caltech 101,” Jurnal Teknik ITS, vol. 5, no. 1, 2016. A. Rohim, Y. A. Sari, and Tibyani, “Convolution neural network (CNN) untuk pengklasifikasian citra makanan tradisional,” JPTIIK (Jurnal Pengembangan Teknologi Informasi dan Ilmu Komputer), vol. 3, no. 7, pp. 7037–7042, 2019.
Q. Zheng, M. Yang, X. Tian, X. Wang, and D. Wang, “Rethinking the role of activation functions in deep convolutional neural networks for image classification,” Engineering Letters, vol. 28, no. 1, pp. 80–92, 2020.
T. M. Kadarina and M. H. I. Hajar, “Pengenalan bahasa pemrograman Python menggunakan aplikasi games untuk siswa/i di wilayah Kembangan Utara,” JAM: Jurnal Abdi Masyarakat (Univ. Mercu Buana), pp. 11–16, 2019.
G. I. E. Soen, Marlina, and Renny, “Implementasi cloud computing dengan Google Colaboratory pada aplikasi pengolah data Zoom participants,” JITU: Journal Informatic Technology and Communication, vol. 6, no. 1, pp. 24–30, Jun. 2022.
N. Wiranda, H. S. Purba, and R. A. Sukmawati, “Survei penggunaan TensorFlow pada machine learning untuk identifikasi ikan kawasan lahan basah,” IJEIS (Indonesian Journal of Electronics and Instrumentation Systems), vol. 10, no. 2, pp. 179–188, 2020, doi: 10.22146/ijeis.58315.
M. Tan and Q. V. Le, “EfficientNet: Rethinking model scaling for convolutional neural networks,” in Proc. 36th Int. Conf. Machine Learning (ICML), PMLR, vol. 97, pp. 6105–6114, 2019.
H. G. Ghifari, D. Darlis, and A. Hartaman, “Pendeteksi golongan darah manusia berbasis TensorFlow menggunakan ESP32-CAM,” ELKOMIKA, vol. 9, no. 2, pp. 359–373, Apr. 2021, doi: 10.26760/elkomika.v9i2.359.
D. Kothadiya, N. Pise, and M. Bedekar, “Different methods review for speech to text and text to speech conversion,” International Journal of Computer Applications, vol. 175, no. 20, pp. 9–12, 2020, doi: 10.5120/ijca2020920727.
