Code-mixed sentiment analysis of Prabowo-Gibran on social media X using mBERT model
DOI:
https://doi.org/10.33795/eltek.v24i2.10852Keywords:
Code-mixed, Imbalanced data, Opinion classification, mBERT, SMOTEAbstract
The sociopolitical dynamics following the 2024 Indonesian Presidential Election have significantly increased the intensity of public discourse on social media. Polarized opinions regarding various strategic policies are frequently expressed through Indonesian–English code-mixed text. Such texts are characterized by language switching, lexical variations, and semantic ambiguity, posing a major challenge for conventional Natural Language Processing (NLP) methods to accurately capture linguistic markers. This study evaluates Multilingual Bidirectional Encoder Representations from Transformers (mBERT) for opinion classification on Indonesian–English code-mixed text. mBERT was selected because it is a pre-trained model trained on monolingual corpora of 104 languages, granting it cross-lingual transfer capabilities without the need for explicit supervision or bilingual alignment. Additionally, to address the class imbalance issue in the dataset, this study applies the Synthetic Minority Over-sampling Technique (SMOTE). Experimental results demonstrate that mBERT achieved the best performance compared to other models, reaching an accuracy of 82.26% without SMOTE. Following the application of SMOTE, the accuracy increased to 83.00% (an increase of 0.74 percentage points). Although this overall numerical increase appears moderate, the value holds crucial significance: the model successfully mitigated the bias toward majority class predictions. This 0.74% increase represents a significant improvement in precisely recognizing minority classes, as evidenced by vastly more balanced class-wise evaluation metrics. In conclusion, mBERT is a highly promising approach for code-mixed opinion classification, while SMOTE plays an essential role in ensuring predictive fairness on imbalanced datasets.
References
M. H. Susanti and S. Khu, “The 2024 Indonesian presidential election in the accounts of millennials: A case study of Prabowo Subianto and Gibran Rakabuming Raka voters in Semarang, Central Java,” Social Sciences & Humanities Open, vol. 11, Art. no. 101629, 2025, doi: 10.1016/j.ssaho.2025.101629.
H. R. P. Sianturi, “Politics on a plate: Equivocal communication in Indonesian presidential nutrition policy,” Frontiers in Communication, vol. 10, Art. no. 1612652, 2025, doi: 10.3389/fcomm.2025.1612652.
A. A. Firdaus, A. Yudhana, I. Riadi, and Mahsun, “Indonesian presidential election sentiment: Dataset of response public before 2024,” Data in Brief, vol. 52, Art. no. 109993, 2024, doi: 10.1016/j.dib.2023.109993.
M. K. Nazir, M. Bilal, and S. C. Shongwe, “Sentiment analysis for code-mixed low-resource languages: A systematic review of approaches, techniques, applications, challenges, and future directions,” Social Network Analysis and Mining, vol. 16, Art. no. 47, 2026, doi: 10.1007/s13278-026-01588-2.
A. M. Barik, R. Mahendra, and M. Adriani, “Normalization of Indonesian-English code-mixed Twitter data,” in Proc. 5th Workshop on Noisy User-Generated Text (W-NUT), Hong Kong, 2019, pp. 417–424, doi: 10.18653/v1/D19-5554.
A. N. Rizal and S. Stymne, “Evaluating word embeddings for Indonesian–English code-mixed text based on synthetic data,” in Proc. 4th Workshop on Computational Approaches to Code Switching, Marseille, France: European Language Resources Association, 2020, pp. 26–35.
G. Manias, A. Mavrogiorgou, A. Kiourtis, C. Symvoulidis, and D. Kyriazis, “Multilingual text categorization and sentiment analysis: A comparative analysis of the utilization of multilingual approaches for classifying Twitter data,” Neural Computing and Applications, vol. 35, pp. 21415–21431, 2023, doi: 10.1007/s00521-023-08629-3.
A. Vaswani et al., “Attention is all you need,” in Advances in Neural Information Processing Systems, vol. 30, 2017, pp. 5998–6008.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional Transformers for language understanding,” in Proc. 2019 Conf. North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA, 2019, pp. 4171–4186, doi: 10.18653/v1/N19-1423.
T. Pires, E. Schlinger, and D. Garrette, “How multilingual is multilingual BERT?” in Proc. 57th Annu. Meeting of the Association for Computational Linguistics, Florence, Italy, 2019, pp. 4996–5001, doi: 10.18653/v1/P19-1493.
S. Khanuja, S. Dandapat, A. Srinivasan, S. Sitaram, and M. Choudhury, “GLUECoS: An evaluation benchmark for code-switched NLP,” in Proc. 58th Annu. Meeting of the Association for Computational Linguistics, 2020, pp. 3575–3585, doi: 10.18653/v1/2020.acl-main.329.
S. Santy, A. Srinivasan, and M. Choudhury, “BERTologiCoMix: How does code-mixing interact with multilingual BERT?” in Proc. 2nd Workshop on Domain Adaptation for NLP, Kyiv, Ukraine: Association for Computational Linguistics, 2021, pp. 111–121.
B. Wilie et al., “IndoNLU: Benchmark and resources for evaluating Indonesian natural language understanding,” in Proc. 1st Conf. Asia-Pacific Chapter of the Association for Computational Linguistics and 10th Int. Joint Conf. Natural Language Processing, Suzhou, China, 2020, pp. 843–857, doi: 10.18653/v1/2020.aacl-main.85.
F. Koto, A. Rahimi, J. H. Lau, and T. Baldwin, “IndoLEM and IndoBERT: A benchmark dataset and pre-trained language model for Indonesian NLP,” in Proc. 28th Int. Conf. Computational Linguistics, Barcelona, Spain, 2020, pp. 757–770, doi: 10.18653/v1/2020.coling-main.66.
F. Koto, J. H. Lau, and T. Baldwin, “IndoBERTweet: A pretrained language model for Indonesian Twitter with effective domain-specific vocabulary initialization,” in Proc. 2021 Conf. Empirical Methods in Natural Language Processing, 2021, pp. 10660–10668, doi: 10.18653/v1/2021.emnlp-main.833.
M. F. Adilazuarda, S. Cahyawijaya, G. I. Winata, P. Fung, and A. Purwarianti, “IndoRobusta: Towards robustness against diverse code-mixed Indonesian local languages,” in Proc. 1st Workshop on Scaling Up Multilingual Evaluation, 2022, pp. 25–34, doi: 10.18653/v1/2022.sumeval-1.5.
A. F. Hidayatullah, R. A. Apong, D. T. C. Lai, and A. Qazi, “Pre-trained language model for code-mixed text in Indonesian, Javanese, and English using Transformer,” Social Network Analysis and Mining, vol. 15, Art. no. 30, 2025, doi: 10.1007/s13278-025-01444-9.
E. Hashmi, S. Y. Yayilgan, and S. Shaikh, “Augmenting sentiment prediction capabilities for code-mixed tweets with multilingual Transformers,” Social Network Analysis and Mining, vol. 14, Art. no. 86, 2024, doi: 10.1007/s13278-024-01245-6.
C. Tho, Y. Heryadi, I. H. Kartowisastro, and W. Budiharto, “Code-mixed sentiment analysis Indonesian-English using Transformer model,” ICIC Express Letters, vol. 17, no. 11, pp. 1295–1302, 2023, doi: 10.24507/icicel.17.11.1295.
L. W. Astuti, Y. Sari, and Suprapto, “Code-mixed sentiment analysis using Transformer for Twitter social media data,” International Journal of Advanced Computer Science and Applications, vol. 14, no. 10, pp. 498–504, 2023, doi: 10.14569/IJACSA.2023.0141053.
N. H. Setiono and Y. Sari, “Exploring the impact of back-translation on BERT’s performance in sentiment analysis of code-mixed language data,” IJCCS (Indonesian Journal of Computing and Cybernetics Systems), vol. 19, no. 2, pp. 223–234, 2025, doi: 10.22146/ijccs.104757.
S. A. Nurbayani, D. S. Maylawati, and A. R. Atmadja, “Klasifikasi sentimen teks code-mixed Indonesia–Inggris non-formal pada X menggunakan model fine-tuned DistilBERT,” Explorer, vol. 6, no. 2, 2026, doi: 10.47065/explorer.v6i2.2856.
V. E. Sidauruk and W. Herowati, “IndoBERT-based sentiment analysis of political discourse on Platform X: The case of Prabowo-Gibran administration,” Journal of Applied Informatics and Computing, vol. 10, no. 1, pp. 673–683, 2026, doi: 10.30871/jaic.v10i1.11586.
D. A. Sulistyo and E. Setiadi, “Analisis sentimen kebijakan Makan Bergizi Gratis menggunakan IndoBERT dan machine learning,” Jurnal FASILKOM, vol. 15, no. 3, pp. 507–513, 2025, doi: 10.37859/jf.v15i3.10546.
N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE: Synthetic Minority Over-sampling Technique,” Journal of Artificial Intelligence Research, vol. 16, pp. 321–357, 2002, doi: 10.1613/jair.953.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Alifia Puspaningrum, Nurul Fauziah, Muhamad Mustamiin, Meyer Mega Eklesia Silaban
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
