TY - GEN
T1 - Facial Action Unit Recognition with Micro-Action-Aware Transformer
AU - Yuan, Yichen
AU - Cheng, Yifan
AU - Shao, Zhiwen
AU - Dang, Qianwen
AU - Chen, Rui
AU - Fu, Mingjian
AU - Jiang, Shengtian
AU - Li, Chunyu
AU - Ma, Lizhuang
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.
PY - 2025
Y1 - 2025
N2 - Facial action unit (AU) recognition is a challenging task, due to the subtlety of each AU and the correlations among AUs in global face. However, the learning of local-global features has not been thoroughly exploited in most of the existing methods. In this paper, we propose a novel micro-action-aware transformer to integrate local and global feature extractions, which effectively captures subtle AU details while maintaining the global relational modeling capacity of transformers. Besides, we jointly train facial AU recognition and facial landmark detection, in which the two correlated tasks contribute to each other and further facilitate the learning of local-global AU-related feature. Extensive experiments demonstrate that our approach achieves comparable performance to the state-of-the-art AU recognition methods on the challenging BP4D and GFT benchmarks, and works well for landmark detection. Particularly, our approach achieves average F1 score results of 63.3% and 55.8% on BP4D and GFT datasets, respectively.
AB - Facial action unit (AU) recognition is a challenging task, due to the subtlety of each AU and the correlations among AUs in global face. However, the learning of local-global features has not been thoroughly exploited in most of the existing methods. In this paper, we propose a novel micro-action-aware transformer to integrate local and global feature extractions, which effectively captures subtle AU details while maintaining the global relational modeling capacity of transformers. Besides, we jointly train facial AU recognition and facial landmark detection, in which the two correlated tasks contribute to each other and further facilitate the learning of local-global AU-related feature. Extensive experiments demonstrate that our approach achieves comparable performance to the state-of-the-art AU recognition methods on the challenging BP4D and GFT benchmarks, and works well for landmark detection. Particularly, our approach achieves average F1 score results of 63.3% and 55.8% on BP4D and GFT datasets, respectively.
KW - facial action unit recognition
KW - facial landmark detection
KW - Micro-action-aware transformer
UR - https://www.scopus.com/pages/publications/105005481942
U2 - 10.1007/978-981-96-5084-2_5
DO - 10.1007/978-981-96-5084-2_5
M3 - 会议稿件
AN - SCOPUS:105005481942
SN - 9789819650835
T3 - Communications in Computer and Information Science
SP - 71
EP - 84
BT - Emotional Intelligence - Second CSIG Conference, CEI 2024, Proceedings
A2 - Huang, Xiaohua
A2 - Mao, Qirong
PB - Springer Science and Business Media Deutschland GmbH
T2 - 2nd CSIG Conference on Emotional Intelligence, CEI 2024
Y2 - 6 December 2024 through 8 December 2024
ER -