Predicting Chinese EFL Learners’ Human‐rated Writing Quality in Argumentative Writing Through Multidimensional Computational Indices of Lexical Complexity

The current study examined the factor structure and construct validity of lexical complexity and also unpacked its role in predicting perceived writing quality based on one data set. This study sampled 220 argumentative writings scored by expert raters from the International Corpus Network of Asian Learners of English (ICNALE) and used Coh-Metrix to compute the indices that tap into multiple constructs of lexical complexity. The results of Confirmatory Factor Analysis (CFA) confirmed the hypothesized factor structure and construct validity of lexical sophistication and lexical diversity. The results of Structural Equation Modeling (SEM) demonstrated that lexical sophistication and lexical diversity measured by computational indices significantly predicted human assessment of L2 writing quality at two language proficiency levels. On the other hand, human judgment on L2 writing quality was found more concerned with more perceivable text features such as linguistic errors and text length than the fine-grained features that tap into lexical complexity. Interpretations as to the small albeit significant variance explained by lexical complexity are provided in the discussion section. This study provides empirical evidence for understanding human scoring processes and implications for fine-tuning AES models to enhance their interpretability.