Multimodal analysis of response-to-name in naturalistic play: bridging speech, gaze, and head pose data

Failure to respond to name (RTN) is a well-documented early indicator of autism spectrum disorder (ASD). This study aimed to enhance RTN detection and auxiliary judgment accuracy by integrating gaze patterns, head pose estimates, and RTN latency in a naturalistic play setting. A modified RTN protocol was used, where children interacted with an examiner in a table-based activity, and multimodal behavioral data were captured using a four-camera system. Automatic detection of name-calling, gaze behavior, and head pose was achieved through speech recognition and computer vision algorithms, validated against human-coded annotations. Significant differences in gaze patterns (e.g. Orient and Diverge), head pose variance (Variance_yaw), and RTN latency were observed between ASD, typically developing (TD), and developmentally delayed (DD) groups. Task Duration, a measure of RTN latency, emerged as a robust predictor of ASD, and multimodal models combining RTN latency and head pose estimates achieved the highest predictive accuracy (AUC = 0.98). These findings underscore the potential of multimodal data integration for ASD screening, offering a time-efficient, scalable approach applicable in telehealth and community-based environments. The study contributes to the growing literature by introducing a naturalistic, automated RTN protocol that complements traditional diagnostic tools, providing a nuanced understanding of early ASD markers in everyday activity settings. Future research should refine algorithms and validate this approach in larger, more diverse populations to further its applicability in real-world contexts.