Table 1 Evaluation of the three main methods of using skill measures from LSAS

Test scores

▪ Sum scores (weighted, unweighted)

▪ CTT factor scores (Thurstone, Bartlett, EAP)

▪ IRT ability estimates (WLE, MLE, EAP, MAP)

▪ ME not (fully) controlled (–)

▪ Biased standard errors of the latent variable in regressions (–)

▪ Biased variance estimates (e.g., underestimation for EAP, overestimation for WLE) (–)

▪ Factor score indeterminacy (–)

▪ Sum scores: Very easy to compute (+)

▪ CTT and IRT test scores, if user-generated: Computation requires knowledge of psychometric models but is fairly easy (+)

▪ Very easy to use in analysis (+)

Sum scores:

▪ Immutable across sub-samples, analyses, and analysts (+)

CTT Factor scores/ IRT ability estimates:

▪ Immutable if estimates are included with LSAS data (+)

▪ Not immutable if estimates are user generated (–)

Structural equation modeling (SEM)

▪ Regular SEM

▪ IRT–SEM

▪ MESE

▪ ME controlled (+)

▪ Unbiased estimates of correlations, means, etc. of the latent variable (+)

▪ Measurement model sensitive to model (mis-)specification (–)

▪ Requires specialized statistical software (–)

▪ Requires additional psychometric expertise (–)

▪ Immutable if measurement model parameters are fixed (+)

▪ Not immutable with free measurement model parameters across sub-samples, analyses, and analysts (–)

Plausible Values (PV)

▪ ME controlled (+)

▪ Approximately unbiased estimates of correlations, means, etc. of the latent variable (+)

▪ User-generated PVs require statistical and programming and expertise (–)

▪ Using PVs in secondary analysis requires basic knowledge of multiple imputation methodology (–)

▪ Immutable if PVs are included with LSAS data (+)

▪ Not immutable if PVs are user generated (–)