Cut-score Decision Making: Arbitrary And Worse

By Author: Jordon
Total Articles: 91
Comment this article

In order to assess the validity of the SAT-W 410 cut score, GSU asked AAC to study a very specific question: would fall 2006 students with an SAT-W score of 410 have had the same chance of passing College Writing as those with Wholesale Jewelry slightly higher scores of 420, 430, or 440 Examining the final College Writing grades of 1,867 students, AAC found that students who enrolled in College Writing with SAT-W scores of 410,420,430, or 440 "generally exhibit similar performance in ENWR 105". Based on these grade similarities, AAC concluded that the 410 cut score was as reasonable as 420, 430, or 440. Of course, this research also confirmed that the 410 cut score was arbitrary. AAC's examination of the SAT-W as a predictor confirmed this arbitrariness.

However, AAC's examination of whether SAT-W scores were a good predictor of success in College Writing revealed only a "low correlation" meaning that the SAT-W scores did not predict success in College Writing. Indeed, within the closely studied sample, the 420 score group had significantly higher grades in College Writing ...
... than did either the 410 or the 440 score group. Is there some special quality about students who receive 420 rather than 440 on the SAT-W section No, the test just doesn't work well in making these distinctions. In sum, although AAC found the cut score to be reasonable within a narrow range, it simultaneously uncovered clear evidence that the SAT-W scores would not reliably identify students who were underprepared for College Writing.

Moreover, placement and admissions decisions using SAT cut scores typically use only "highest" scores of those submitted, and do not discount for multiple test taking. Though we are unaware of any formal, educational research on Jewelry On Sale score impact for students who take the SAT more than once (sometimes several times), we believe that the practice of using a student's highest score favors multiple-time test takers, and that people who take the test more times are likely to be living in more affluent communities in the country. Some evidence of this theory was found in a press report about Baylor University administrators "paying admitted freshmen to retake the SAT and offering large financial rewards for those whose scores go up by certain levels".

At GSU, our 2007 review of 161 possibly misplaced students showed there were 58 single-time SAT takers who averaged a high SAT-E score of 6.9. There were 54 two-time takers who averaged 7.4. There were 20 three-time takers who averaged 7.6. Put another way, if GSU had used an SAT-E cut score of 7 for placement into College Writing for this group of 161 students, then 23 out of 58 single-time SAT test takers would have missed the cut. But only 5 out of 54 two-time test takers would have missed the cut, and only 4 out of 20 three-time test takers would have missed it. Although this sample is small, these numbers evidence that an aggressive test-taking strategy can be an important advantage in SAT score results. Clearly any measure that can reduce full reliance on these high-stakes tests using a valid measurement of actual writing ability would produce a more fair result.