Change search
ReferencesLink to record
Permanent link

Direct link
Interrater reliability in large-scale assessments: can teachers score national tests reliably without external controls?
Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
2015 (English)In: Practical Assessment, Research & Evaluation, ISSN 1531-7714, Vol. 20, no 9Article in journal (Refereed) Published
Abstract [en]

In most large-scale assessment systems a set of rather expensive external quality controls are implemented in order to guarantee the quality of interrater reliability. This study empirically examines if teachers’ ratings of national tests in mathematics can be reliable without using monitoring, training, or other methods of external quality assurance. A sample of 99 booklets of students’ answers to a national test in mathematics was scored by five teachers independently. The interrater reliability was analyzed using consensus and consistency estimates, with the focus on the test as a whole, as well as on individual items. The results show that the estimates are acceptable and in many cases fairly high, irrespective of the reliability measure used. Some plausible explanations for lower interrater reliability in individual items are discussed, and some suggestions are made in the direction of further improving reliability without imposing any system of control.

Place, publisher, year, edition, pages
2015. Vol. 20, no 9
National Category
Educational Sciences
URN: urn:nbn:se:umu:diva-101511OAI: diva2:799871
Available from: 2015-03-31 Created: 2015-03-31 Last updated: 2015-06-04Bibliographically approved

Open Access in DiVA

fulltext(433 kB)313 downloads
File information
File name FULLTEXT01.pdfFile size 433 kBChecksum SHA-512
Type fulltextMimetype application/pdf

Other links


Search in DiVA

By author/editor
Lind Pantzare, Anna
By organisation
Departement of Educational Measurement
In the same journal
Practical Assessment, Research & Evaluation
Educational Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 313 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Total: 118 hits
ReferencesLink to record
Permanent link

Direct link