umu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Interrater reliability in large-scale assessments: can teachers score national tests reliably without external controls?
Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
2015 (English)In: Practical Assessment, Research & Evaluation, ISSN 1531-7714, E-ISSN 1531-7714, Vol. 20, no 9Article in journal (Refereed) Published
Abstract [en]

In most large-scale assessment systems a set of rather expensive external quality controls are implemented in order to guarantee the quality of interrater reliability. This study empirically examines if teachers’ ratings of national tests in mathematics can be reliable without using monitoring, training, or other methods of external quality assurance. A sample of 99 booklets of students’ answers to a national test in mathematics was scored by five teachers independently. The interrater reliability was analyzed using consensus and consistency estimates, with the focus on the test as a whole, as well as on individual items. The results show that the estimates are acceptable and in many cases fairly high, irrespective of the reliability measure used. Some plausible explanations for lower interrater reliability in individual items are discussed, and some suggestions are made in the direction of further improving reliability without imposing any system of control.

Place, publisher, year, edition, pages
2015. Vol. 20, no 9
National Category
Educational Sciences
Identifiers
URN: urn:nbn:se:umu:diva-101511OAI: oai:DiVA.org:umu-101511DiVA: diva2:799871
Available from: 2015-03-31 Created: 2015-03-31 Last updated: 2017-12-04Bibliographically approved

Open Access in DiVA

fulltext(433 kB)435 downloads
File information
File name FULLTEXT01.pdfFile size 433 kBChecksum SHA-512
758208da5e025491d126640961f33e8e3bc1a73f4a6a06a47c2962bba08e4ea6709efa5977603b76267afc33c1d6676161282fe28931faf064ad18eb7f46a71c
Type fulltextMimetype application/pdf

Other links

fulltext

Authority records BETA

Lind Pantzare, Anna

Search in DiVA

By author/editor
Lind Pantzare, Anna
By organisation
Departement of Educational Measurement
In the same journal
Practical Assessment, Research & Evaluation
Educational Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 435 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 425 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf