Hierarchical Federated Learning for Heterogeneous Features and Distributed Data in IoT Networks

Considering the heterogeneous compositions of Internet of Things (IoT) networks, this paper proposes a hierarchical federated learning (HiFL) framework to exploit both heterogeneous features and distributed data generated therein. In specific, the HiFL framework is a hierarchical architecture that leverages both vertical and horizontal federated learning. Clients that having local data of the same feature are grouped into the same cluster, naturally forming a party; otherwise, they are in different clusters. Parallel training of multiple clusters is handled by cluster heads (i.e., sub-servers), which operate in a horizontal federated learning manner to make use of data samples of the same feature distributed in clients within the same party. Then, a server collects and aggregates the results of parties to update the global model for a decreased training loss. We implement the HiFL system in a parameter server architecture and assess its performance via experiments using the NSL-KDD data set. We first discuss the local results of different parties that only use part of the features based on the analysis of feature importance. Then, we compare our system with fully centralized learning (which violates data locality) and learning based on only local features. Numerical results demonstrate that the proposed HiFL model outperforms the models based on local features, and approaches to the centralized model.