Kernel estimators of the ROC curve are better than empirical

The receiver operating characteristic (ROC) is a curve used to summarise the performance of a binary decision rule. It can be expressed in terms of the underlying distributions functions of the diagnostic measurement that underlies the rule. Lloyd (1998) has proposed estimating the ROC curve from kernel smoothing of these distribution functions and has presented asymptotic formulas for the bias and standard deviation of the resulting curve estimator. This paper compares the asymptotic accuracy of the kernel-based estimator with the fully empirical estimator. It is shown that the empirical estimator is deficient compared to the kernel estimator and that this deficiency is unbounded as sample size increases. A simulation study using both unimodal and bimodal distributions indicates that the gains in accuracy are significant for realistic sample sizes. Kernel-based ROC estimators can now be recommended.