Vanessa: Visual Connotation and Aesthetic Attributes Understanding Network for Multimodal Aspect-based Sentiment Analysis

Prevailing research concentrates on superficial features or descriptions of images, revealing a significant gap in the systematic exploration of their connotative and aesthetic attributes. Furthermore, the use of cross-modal relation detection modules to eliminate noise from comprehensive image representations leads to the omission of subtle contextual information. We present Vanessa, a visual connotation and aesthetic Attributes understanding network for multimodal aspect-based sentiment analysis. It incorporates a multi-aesthetic attributes aggregation (MA³) module that models intra- and inter-dependencies among bi-modal representations as well as emotion-laden aesthetic attributes. Moreover, we devise a self-supervised contrastive learning framework to explore the pairwise relevance between images and text via the Gaussian distribution of their CLIP scores. By dynamically clustering and merging multimodal tokens, Vanessa effectively captures both implicit and explicit sentimental cues. Extensive experiments on two widely adopted benchmarks verify Vanessa's effectiveness.