Multi-Objective Deep Reinforcement Learning for Adaptive Virtual Machine Allocation in Clouds

With the exponential growth of data and demands for computing capabilities, optimizing resource utilization has become increasingly critical in cloud data centers. Employing virtual machine (VM) allocation technology to maintain hosts within an appropriate workload range holds substantial promise for improving workload balance, energy efficiency, and quality of service (QoS). Existing multi-objective VM allocation strategies based on greedy heuristics and reinforcement learning with predefined fixed objective weights lack generalizability and quick adaptability in dynamic workload scenarios. In this paper, we present MOVMA, based on a novel Multi-Objective Reinforcement Learning (MORL) algorithm to coordinately optimize three objectives, i.e., energy consumption, load balancing in multidimensional resource utilization, and service level agreement (SLA) violations. MOVMA adopts our proposed Sliding Time Window-based Dynamic Weight (STWDW) method to adaptively calculate the weights instantly based on the current system condition, ensuring the actual impact of the parameters. Furthermore, it integrates our proposed Priority-based Selection and Adjustment (PSA) scheme and a Near on-policy Experience Replay (NER) strategy in model training to accelerate convergence and avoid catastrophic forgetting. The experiments conducted on a real-world dataset demonstrate the superior performance of our MOVMA against state-of-the-art multi-objective optimization algorithms.