Video-Based Multimodal Personality Analysis

  • Ziqing Li
  • , Zhenyuan Yang
  • , Jiawen Zhang
  • , Lianggangxu Chen
  • , Gaoqi He

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

Personality analysis defines the task of computing human personality based on their behavior. Traditionally psychological personality research experiment depending on the subjective answers of participants has many inherent challenges. In this paper, we propose a multimodal personality analysis model based on audio features, facial features and environmental features to accurately analyze Big Five traits of people personality. In the feature extraction stage, MFCC features are extracted as the representation of audio data, for facial and environmental characteristics, ResNet and LSTM are used to extract features and integrate time information respectively. We then evaluate our model using the ChaLearn First Impressions V2 challenge dataset and finally reached an accuracy of 89.12%. We demonstrate that multimodal method is more effective for personality analysis, comparing with most of the methods using single modal. Finally, a representational personality analysis system is presented based on pyqt5. It has the function of taking photos or uploading videos to portray the character's personality.

Original languageEnglish
Title of host publicationThirteenth International Conference on Graphics and Image Processing, ICGIP 2021
EditorsLiang Xiao, Dan Xu
PublisherSPIE
ISBN (Electronic)9781510650428
DOIs
StatePublished - 2022
Event13th International Conference on Graphics and Image Processing, ICGIP 2021 - Kunming, China
Duration: 18 Aug 202120 Aug 2021

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
Volume12083
ISSN (Print)0277-786X
ISSN (Electronic)1996-756X

Conference

Conference13th International Conference on Graphics and Image Processing, ICGIP 2021
Country/TerritoryChina
CityKunming
Period18/08/2120/08/21

Keywords

  • Big five personality
  • Feature fusion
  • Multimodal
  • Representational personality analysis

Fingerprint

Dive into the research topics of 'Video-Based Multimodal Personality Analysis'. Together they form a unique fingerprint.

Cite this