umu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Reliability of educational assessments: The case of classification accuracy
Umeå University, Faculty of Social Sciences, Department of Educational Measurement.
2004 (English)In: Scandinavian Journal of Educational Research, ISSN 0031-3831, E-ISSN 1470-1170, Vol. 48, no 4, 427-440 p.Article in journal (Refereed) Published
Abstract [en]

Reliability is a problem inherent in all educational assessments, but the amount of attention this particular problem should be given is related to the function and use of the assessment. In this article, classification accuracy is put forward as a conceptualization of reliability that is meaningful for a large number of educational assessments. Furthermore, a fairly transparent method of estimating the magnitude of classification accuracy is developed. The article examines how the classification accuracy of a Swedish national test in mathematics is affected by changes in the test that are motivated by other validity concerns. The inquiry is based on the results on 1201 students participating in a Swedish national test in mathematics. The results indicate that there can be a significant trade-off between reliability and other aspects of validity.

Place, publisher, year, edition, pages
Taylor & Francis, 2004. Vol. 48, no 4, 427-440 p.
Keyword [en]
educational assessment, validity, reliability, classification accuracy
National Category
Pedagogy
Identifiers
URN: urn:nbn:se:umu:diva-14171DOI: 10.1080/0031383042000245816OAI: oai:DiVA.org:umu-14171DiVA: diva2:153842
Available from: 2007-05-24 Created: 2007-05-24 Last updated: 2017-12-14Bibliographically approved
In thesis
1. Rätt mätt på prov: Om validering av bedömningar i skolan
Open this publication in new window or tab >>Rätt mätt på prov: Om validering av bedömningar i skolan
2004 (Swedish)Doctoral thesis, comprehensive summary (Other academic)
Alternative title[en]
Validation of educational assessments
Abstract [en]

The thesis focuses on validation as a process whereby the quality of educational assessments can be evaluated. On the basis of Messick’s (1989) validity concept, which takes consequences as well as inferences of assessment results into account, a model for validation is described. An argument made is that assessment purposes, epistemology, and curricular goals are necessary starting points for validation. Fur-thermore, it is argued that several of the criteria for good assessment proposed as alternatives to validity and reliability can be used together with Messick’s frame-work for validity and make it useful in practice. Four empirical studies, each pre-sented in a separate paper, supply examples of the application of this framework. In paper I it is claimed that reliability is a problem inherent in all educational assess-ments, and it is given a meaningful interpretation through the concept of classifica-tion accuracy. The article examines how the classification accuracy of a Swedish national test in mathematics is affected by changes in the test that are motivated by other validity concerns. The study is based on the results of 1,201 students partici-pating in a Swedish national test in mathematics. The results indicate that there can be a significant trade-off between reliability and other aspects of validity. Paper II presents a study of attainment differences in mathematics related to the grouping of students into different study programmes in the Swedish upper-secondary school. For a sample of 403 students, results from two consecutive national tests in mathe-matics were compared, one at the end of compulsory school and the other at the end of the first mathematics course in upper-secondary school. The results indicate that attainment is, on average, positively affected if students are grouped together with higher-achieving peers. This effect appears to be strongest for low-achievers. Paper III addresses questions concerning how perceptions of competence are formed in school mathematics. The study is based on a questionnaire distributed to a sample of Swedish upper secondary school students (n = 550). The results indicate that an inner feeling of understanding is most highly valued by the students as an indicator of competence, more than external cues like test results and teacher feed-back. Male and female students seem to value cues to perceived competence in similar ways. In paper IV an interview study with six mathematics teachers in upper-secondary school is presented, aiming at exploring the variation in what teachers at their particular school say about ability grouping. In summary, what these teachers express concerning the advantages and disadvantages of ability grouping varies a great deal. There are, however, some things they more or less agree on. They have a basically positive view of the way in which ability grouping has been practised at their school. They identify problems with ability grouping for low-ability groups, primarily related to the mixing of students who have trouble learning with those who primarily have motivational problems. The list of advantages and possibilities is definitely longer for groups of high-achieving students compared to groups of low-achievers.

Publisher
54 p.
Series
Akademiska avhandlingar vid Pedagogiska institutionen, Umeå universitet, ISSN 0281-6768 ; 71
Keyword
Education, Pedagogik, bedömning, validitet, läroplan, kunskapssyn
National Category
Pedagogy
Research subject
Education
Identifiers
urn:nbn:se:umu:diva-198 (URN)91-7305-609-X (ISBN)
Public defence
2004-03-12, Hörsal MA121, MIT-huset, Umeå universitet, Umeå, 10:00
Opponent
Supervisors
Available from: 2004-02-24 Created: 2004-02-24 Last updated: 2012-06-29Bibliographically approved

Open Access in DiVA

No full text

Other links

Publisher's full text

Search in DiVA

By author/editor
Nyström, Peter
By organisation
Department of Educational Measurement
In the same journal
Scandinavian Journal of Educational Research
Pedagogy

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 78 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf