Comparative Federated Algorithms for Solving Non- IID Data Challenges
DOI:
https://doi.org/10.48314/ramd.v1i2.56Keywords:
Federated learning, FedAvg, FedProx, MOON, Data heterogeneityAbstract
This study evaluates three Federated Learning (FL) algorithms—FedAvg, Federated Proximal (FedProx), and MOON—by assessing their performance in Independent and Identically Distributed (IID) and non-IID settings. We found that FedAvg performs best in IID scenarios, offering quick convergence and high accuracy. However, MOON stood out as the top performer in non-IID settings, thanks to its contrastive learning method, providing better stability and accuracy across heterogeneous data. FedProx improved over FedAvg in handling non-IID data but was less effective than MOON. Our findings suggest that for environments with IID data, FedAvg is ideal, while MOON is more suitable for non-IID cases. We also highlight the need for further research into personalized FL, regularization techniques, and multimodal data integration.
References
Abdulrahman, S., Tout, H., Ould-Slimane, H., Mourad, A., Talhi, C., & Guizani, M. (2021). A survey on federated learning: The journey from centralized to distributed on-site learning and beyond. IEEE internet of things journal, 8(7), 5476–5497. https://doi.org/10.1109/JIOT.2020.3030072
Huang, C., Huang, J., & Liu, X. (2022). Cross-silo federated learning: Challenges and opportunities. https://doi.org/10.48550/arXiv.2206.12949
Li, Q., Diao, Y., Chen, Q., & He, B. (2022). Federated learning on non-iid data silos: An experimental study. 2022 IEEE 38th international conference on data engineering (Icde) (pp. 965–978). IEEE. https://doi.org/10.1109/ICDE53745.2022.00077
Dashan Gao, Xin Yao, Q. Y. (2022). A survey on heterogeneous federated learning. https://doi.org/10.48550/arXiv.2210.04505
Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., & Smith, V. (2020). Federated optimization in heterogeneous networks. Proceedings of machine learning and systems (Vol. 2, pp. 429–450). MLSys. https://B2n.ir/qp5832
Che, L., Wang, J., Zhou, Y., & Ma, F. (2023). Multimodal federated learning: A survey. Sensors, 23(15), 6986. https://doi.org/10.3390/s23156986
McMahan, B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. y. (2017). Communication-efficient learning of deep networks from decentralized data. Proceedings of the 20th international conference on artificial intelligence and statistics (Vol. 54, pp. 1273–1282). PMLR. https://proceedings.mlr.press/v54/mcmahan17a.html
Hartmann, M., Danoy, G., & Bouvry, P. (2024). FedPref: Federated learning across heterogeneous multi-objective preferences. ACM transactions on modeling and performance evaluation of computing systems. Association for Computing Machinery. https://doi.org/10.1145/3708984
McMahan, H., Moore, E., & Ramage, D. (2016). Federated learning of deep networks using model averaging. https://b2n.ir/md1975
Sun, L., & Wu, J. (2023). A scalable and transferable federated learning system for classifying healthcare sensor data. IEEE journal of biomedical and health informatics, 27(2), 866–877. https://doi.org/10.1109/JBHI.2022.3171402
Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., & Chandra, V. (2018). Federated learning with non-IID data. http://arxiv.org/abs/1806.00582
Metrics
