Assessing Intra- and Inter-Individual Reliabilities in Intensive Longitudinal Studies: A Two-Level Random Dynamic Model-Based Approach

Intensive longitudinal studies are becoming increasingly popular because of their potential for studying the individual dynamics of psychological processes. However, measures used in such studies are quite susceptible to measurement error due to the short lengths and therefore their psychometric properties, such as reliability, are of great concern. Most existing approaches for assessing reliability are not appropriate for the intensive longitudinal data (ILD) because of the conflation of inter- and intra-individual variations or the difficulty in handling interindividual differences. In addition, measurement models are always relegated or omitted in the ILD modeling approaches. Therefore, in this article, we introduce a two-level random dynamic measurement (2RDM) model for ILD, which takes into account measurement models for key variables of interest. Then we discuss how to derive the within-person and between-person reliabilities for items and scales in the context of the 2RDM model. A small simulation study is presented to illustrate the implementation of the 2RDM model and reliability estimation. An empirical study is then provided to demonstrate the application of the proposed approach for multidimensional scales, in which we calculated the within- and between-person reliabilities for both items and subscales of a short version of the Perceived Stress Scale and found large individual differences in the within-person reliabilities. We conclude by discussing the advantages and considerations of the proposed approach in practice.