Validating cut scores set by Angoff procedures with results from equating procedures
2015 (English)Conference paper, Presentation (Refereed)
In Sweden the cut scores for each new test form of national tests in mathematics are set before test administration. This demand has existed ever since the transition to the current criterion-referenced system in 1994. One argument given for this requirement is to make sure that teachers no longer score and interpret the test score in a relative manner. The cut scores are set with a judgemental Angoff procedure, without inclusion of item field test data and with no regular equating or linking procedure. Therefore, a relevant question is if it is naïve to assume that the cut scores are equivalent over years. In these studies the equivalence of the cut scores for two, different and separate, pairs of tests are investigated, by comparing cut scores set by Angoff procedures with the results from equating procedures. In both examples a non-equivalent group anchor test (NEAT) design was used. The cut scores was compared to equating procedures with linear and equipercentile methods. The results show that there are validity arguments supporting that the Angoff procedure is working. However, the equating procedures reveal several methodological and practical challenges.
Place, publisher, year, edition, pages
Research subject didactics of educational measurement
IdentifiersURN: urn:nbn:se:umu:diva-111086OAI: oai:DiVA.org:umu-111086DiVA: diva2:867121
Standard-setting: International state of research and practices in the Nordic countries, Oslo, 21-23 september, 2015