Generative Diffusion-Based Black-Box Optimization for Robust Information Bottleneck Representation Learning

The Information Bottleneck (IB) provides a principled framework for representation learning in task-oriented communication, enabling joint feature extraction and transmission while preserving task-relevant information. However, IB-based optimization remains challenging due to the high dimensionality of data, the intractability of mutual information estimation, and the reliance of existing methods on Lagrangian multipliers, which require complex hyperparameter tuning and do not guarantee constraint satisfaction. To address these limitations, this paper investigates both the classical IB problem and the Robust Information Bottleneck (RIB) problem, which extends IB by incorporating robustness against uncertainty and perturbations, thereby aligning more closely with practical communication requirements. We propose two novel diffusion-based optimization frameworks. The first, Diffusion-based Information Bottleneck Optimization Model (DIBOM), addresses the classical IB problem by integrating conditional diffusion models with black-box optimization. It employs gradient-guided sampling and adaptive loss reweighting to achieve a stable balance between compression and predictive accuracy. The second, Diffusion-based Robust Information Bottleneck Multi-objective Bayesian Optimization (DRIB-MOBO) framework, extends the approach to the RIB problem. By formulating RIB as a multi-objective black-box optimization task, DRIB-MOBO leverages the generative capability of diffusion models within a Bayesian optimization loop to iteratively search, evaluate, and refine candidate solutions, ultimately constructing Pareto-optimal representations that jointly optimize accuracy, compressibility, and robustness. Extensive experiments demonstrate that the proposed frameworks effectively overcome the limitations of Lagrangian-based methods, achieving superior robustness and efficient multi-objective optimization under complex and uncertain environments.