Hypothesis testing for quantitative trait locus effects in both location and scale in genetic backcross studies

Testing the existence of a quantitative trait locus (QTL) effect is an important task in QTL mapping studies. Most studies concentrate on the case where the phenotype distributions of different QTL groups follow normal distributions with the same unknown variance. In this paper we make a more general assumption that the phenotype distributions come from a location-scale distribution family. We derive the limiting distribution of the likelihood ratio test (LRT) for the existence of the QTL effect in both location and scale in genetic backcross studies. We further identify an explicit representation for this limiting distribution. As a complement, we study the limiting distribution of the LRT and its explicit representation for the existence of the QTL effect in the location only. The asymptotic properties of the LRTs under a local alternative are also investigated. Simulation studies are used to evaluate the asymptotic results, and a real-data example is included for illustration.