Design and Hardening of an MQTT Broker Server on AWS EC2 for Secure IoT Communication

Authors

  • Deo Triyanuar Putra Politeknik Negeri Malang
  • Muhammad Syirajuddin Suja'i Politeknik Negeri Malang
  • Ahmad Wilda Yulianto Politeknik Negeri Malang

DOI:

https://doi.org/10.33795/jartel.v15i4.8336

Keywords:

AWS EC2, IoT Communication, MQTT Broker, Network Security, Server Hardening

Abstract

The rapid growth of Internet of Things (IoT) applications has increased the reliance on lightweight communication protocols such as Message Queuing Telemetry Transport (MQTT), while simultaneously raising security risks due to the exposure of broker servers to public networks. This study presents the design and security hardening of an MQTT Broker Server using EMQX deployed on Amazon Web Services (AWS) EC2 as a communication infrastructure for IoT devices. The research methodology includes system design, implementation of the MQTT Broker and Node-RED on AWS EC2, integration of IoT devices as MQTT clients, and the application of server hardening techniques. The hardening methods applied consist of operating system updates, root account restriction, firewall configuration using iptables and UFW, Secure Socket Layer (SSL/TLS) implementation through an Nginx reverse proxy, port access limitation, and system log monitoring. Security evaluation is conducted through penetration testing, including Information Gathering, Vulnerability Scanning, and simulated cyberattacks such as Denial of Service (DoS), Distributed Denial of Service (DDoS), Brute Force, and Remote Code Execution (Reverse Shell). The results show that before hardening, the server was highly vulnerable and could be taken down easily, while after hardening, all attack scenarios were successfully mitigated and system availability increased from 0% to 100%. These results demonstrate that server hardening significantly enhances the security and reliability of MQTT-based IoT communication on cloud infrastructure.

References

Calabretta, M., R. Pecori, M. Vecchio, and L. Veltri, “MQTT Auth: A token-based solution to endow MQTT with authentication and authorization capabilities,” *Journal of Communications Software and Systems*, vol. 14, no. 4, 2018, doi: 10.24138/jcomss.v14i4.604.

A. Lohachab, “ECC-based inter-device authentication and authorization scheme using MQTT for IoT networks,” *Journal of Information Security and Applications*, vol. 46, pp. 1–12, 2019, doi: 10.1016/j.jisa.2019.02.005.

I. Vaccari, M. Aiello, and M. Mongelli, “SlowITe: A novel denial of service attack affecting MQTT,” *Sensors*, vol. 20, no. 10, Art. no. 2932, 2020, doi: 10.3390/s20102932.

E. Elemam, A. M. Bahaa Eldin, N. H. Shaker, and M. Sobh, “Formal verification for a PMQTT protocol,” *Egyptian Informatics Journal*, vol. 21, no. 3, pp. 169–182, 2020, doi: 10.1016/j.eij.2020.01.001.

V. Seoane, C. Garcia Rubio, F. Almenares, and C. Campo, “Performance evaluation of CoAP and MQTT with security support for IoT environments,” *Computer Networks*, vol. 197, Art. no. 108338, 2021, doi: 10.1016/j.comnet.2021.108338.

D. Silva, L. Carvalho, J. Soares, and others, “A performance analysis of Internet of Things networking protocols: Evaluating MQTT, CoAP, OPC UA,” *Applied Sciences*, vol. 11, no. 11, Art. no. 4879, 2021, doi: 10.3390/app11114879.

Z. Chen, Y. Wang, H. Ning, and H. N. Dai, “A survey on security in IoT cloud computing,” *ACM Computing Surveys*, vol. 54, no. 8, 2021, doi: 10.1145/3447625.

J. Roldán Gómez, J. Carrillo Mondéjar, J. M. Castelo Gómez, and S. Ruiz Villafranca, “Security analysis of the MQTT-SN protocol for the Internet of Things,” *Applied Sciences*, vol. 12, no. 21, Art. no. 10991, 2022, doi: 10.3390/app122110991.

F. Dewanta and colleagues, “A study of secure communication scheme in MQTT: TLS vs AES in IoT networks,” *Jurnal Infotel*, vol. 14, no. 4, 2022.

V. Shilpa, A. Vidya, and S. Pattar, “MQTT-based secure transport layer communication for mutual authentication in IoT networks,” *Global Transitions Proceedings*, vol. 3, no. 12, 2022, doi: 10.1016/j.gltp.2022.04.015.

C. Surianarayanan, “Integration of the Internet of Things and cloud: Security challenges and solutions,” *International Journal of Cloud Applications and Computing*, vol. 13, no. 1, pp. 1–30, 2023, doi: 10.4018/IJCAC.325624.

K. Kosaka, Y. Noda, T. Yokotani, and K. Ishibashi, “Implementation and evaluation of the control mechanism among distributed MQTT brokers,” *IEEE Access*, vol. 11, pp. 134211–134216, 2023, doi: 10.1109/ACCESS.2023.3335273.

M. Kashyap, A. K. Dev, and V. Sharma, “Implementation and analysis of EMQX broker for MQTT protocol in the Internet of Things,” *e-Prime: Advances in Electrical Engineering, Electronics and Energy*, vol. 10, Art. no. 100846, 2024, doi: 10.1016/j.prime.2024.100846.

N. Saha, P. Paul, K. Ji, and R. Harik, “Performance evaluation framework of MQTT client libraries for IoT applications in manufacturing,” *Manufacturing Letters*, vol. 41, pp. 1237–1245, 2024, doi: 10.1016/j.mfglet.2024.09.150.

P. S. Bangare and K. P. Patil, “Enhancing MQTT security for Internet of Things: Lightweight two-way authorization and authentication with advanced security measures,” *Measurement: Sensors*, Art. no. 101212, 2024, doi: 10.1016/j.measen.2024.101212.

Downloads

Published

29-12-2025

How to Cite

Putra, D. T., Suja’i, M. S., & Yulianto, A. W. (2025). Design and Hardening of an MQTT Broker Server on AWS EC2 for Secure IoT Communication. JURNAL JARTEL: Jurnal Jaringan Telekomunikasi, 15(4), 428–435. https://doi.org/10.33795/jartel.v15i4.8336