Alexander Rudolph’s commentary correctly notes that achieving greater diversity in physics requires revamping admissions criteria: You only get what you select for. However, the recommendation against using Graduate Record Examination (GRE) scores draws heavily on a study by Casey Miller and coauthors.1 That study has three major problems:
• The study measured performance with a binary variable: completion. Eliminating gradations of performance obscures relationships that may be present in more granular data. A large meta-analysis examined student performance with fine-grained measures—for example, research productivity, faculty ratings of student work—and found significant predictive power in GRE scores.2
• The work by Miller and coauthors included program rank as an explanatory variable, despite its being strongly correlated with GRE scores. When two or more such variables are strongly correlated, a regression routine cannot easily determine which variable should get the larger coefficient; different coefficient choices could fit the data similarly well. Consequently, coefficient estimates will have large uncertainties.3 Thus the estimated coefficient of GRE score will almost certainly have a magnitude comparable to the estimate’s uncertainty.
• The predictive power of program ranking actually fits a model of efficient competition in admissions: The more desirable programs attract students who are more likely to succeed. Moreover, a student who is weak by one measure can gain admission by demonstrating strength in another measure. Such cases may camouflage correlations between student performance and other explanatory variables.4 Of course, there are other plausible explanations for the predictive power of program ranking, but nothing in the cited work enables readers to choose among explanations.
Admissions criteria are ultimately about values, and it is wholly appropriate to include diversity of backgrounds among them. However, if performance is also valued, then valid predictors of performance should also be included. The Miller study does not demonstrate that GRE scores lack predictive power, and it should not be cited uncritically.
