Do Generative AI-Powered Pedagogical Agents Improve Learners’ Academic Performance Effectively? Evidence From Meta-Analysis

With the breakthrough advancements in generative artificial intelligence (GenAI) technology, GenAI-powered pedagogical agents (GenAI-PA) are emerging as a transformative paradigm shift in education. However, there is still debate about whether the use of GenAI-PA is beneficial for learners’ academic performance. Therefore, this study conducted a meta-analysis of 27 experimental and quasi-experimental studies from 2015 to 2025. The results showed that (1) GenAI-PA had a significant effect on learners’ academic performance (g = 0.401). (2) In a collectivist culture, GenAI-PA had a greater effect on learners’ academic performance. (3) The effect of GenAI-PA in teacher-directed learning was significantly better than in self-directed learning. (4) The dialogue modality of GenAI-PA moderated the effect on learners’ academic performance, with multimodal dialogue showing the highest pedagogical potential. (5) The predictive effect of GenAI-PA on learners’ academic performance was not influenced by grade level, gender, learning domain, learning duration, or the instructional role of GenAI-PA. Ultimately, recommendations for the design and application of GenAI-PA are discussed.