Reinforcement Learning Based Control Domain Division in LEO Satellite Networks

The performance of SDN-based LEO satellite networks significantly depends on the control domain division approaches. The dynamical topology in LEO networks results in the time-varying delay for network management. Therefore, the control domain division is very challenging and should be dynamical. In this paper, we propose a control domain division approach based on reinforcement learning (RL), which aims at reducing the average management delay of controllers during the control period, and as well as minimizing the number of control domain handovers. Firstly, we analyze the factors that need to be paid attention to in controller deployment, such as delay, load and controller switching cost, and formulate the controller deployment. By relaxing the constraints, we prove that the problem is NP-hard. Secondly, we propose an approach that considers future motion trajectory based on reinforcement learning (FMT _ RL) to deploy controllers. Since the dimension of the neural network in reinforcement learning is fixed, we use the improved K-Means algorithm to select the set of mechanisms to determine the dimension of the neural network. Particularly, in order to obtain the node composition control domain, the observation including both future motion trajectory and history controller selection information are input into the neural network to calculate the control domain division results. The experimental results demonstrate that our approach significantly outperforms related approaches, with better stability and average control cost.