umu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Reliabilitets- och validitetsstudier vid differentiell poängsättning av flervalsfrågor
Umeå University, Faculty of Social Sciences, Department of Education.
1973 (Swedish)Report (Other academic)Alternative title
Reliability and validity studies with differential scoring of multiple-choice questions (English)
Abstract [sv]

I föreliggande rapport redovisas tre delförsök där diffe-rentiel.l poängsättning av flervalsfrågors svarsalternativ har prövats och i reliabilitets- och validitetshänseende jämförts med konventionell dikotom poängsättning. Både empiriska och a priori-vikter. har tillämpats. Underlaget till de förra utgjordes dels avbiseriala korrelationer mellan respektive alternativ och totalpoäng och dels av medelvärden i totalpoäng för de som valt respektive svarsalternativ (guttmanvikter). A priori-vikterna erhölls från olika skattningsförfaranden. På prov utformade med ordnade svarsalternativ erhölls högre reliabilitet vid differen-tiell än vid konventionell poängsättning. De högsta reliabi litetBvärdena erhölls med de mera stringenta viktsystemen. Då jämförelserna gjordes med den reliabilitet som erhölls vid konventionell poängsättning på ett parallellt prov utformat utan ordnade svarsalternativ eliminerades denna positiva effekt utom för de stringenta viktsystemen. I validitetshänseende resulterade differentiell poängsättning genomgående i sämre resultat än konventionell poängsättning. I kommentarerna till de erhållna resultaten framhålls den betydelse provets utformning har på möjligheterna att erhålla positiva reliabilitetseffekter med differen-tiell poängsättning. Avslutningsvis redovisas begränsningar i undersökningen och förslag till fortsatta prövningar ges.

Abstract [en]

In the present report three studies will be accounted for, where differential scoring of the alternatives of multiple-choice questions has been tested and as for reliability and validity has been compared to conventional dichotomous scoring. Empirical as well as a priori weights have been used. The basis of the former ones consisted partly of biserial correlations between every alternative and the total score and partly of the mean of the total score for those who had chosen the actual alternative (Guttman weights). The a priori weights were received through different judgement procedures. Tests with ordered alternatives showed higher reliability with differential than with conventional scoring. The highest reliability values were received with the more stringent weight systems.When the comparisons were made with the reliability received with conventional scoring of a parallel test, constructed without ordered alternatives, this positive effect was eliminated except for the stringent weight systems. As for validity, differential scoring showed on the whole lower values than conventional scoring. When commenting on the received results the importance of the construction of the test to receive positive reliability effects with differential scoring is pointed out. Finally the restrictions of the study are accounted for and suggestions for future research are given.

Place, publisher, year, edition, pages
Umeå: Umeå universitet och Lärarhögskolan , 1973. , 61 p.
Series
Pedagogiska rapporter Umeå, ISSN 0348-9388 ; 32
National Category
Pedagogy
Identifiers
URN: urn:nbn:se:umu:diva-68284OAI: oai:DiVA.org:umu-68284DiVA: diva2:616169
Projects
digitalisering@umu
Available from: 2013-04-15 Created: 2013-04-15 Last updated: 2013-04-18Bibliographically approved
In thesis
1. Mätproblem i norm- och kriterierelaterade prov: några analyser och försök med tonvikt på reliabilitets- och diskriminationsmått
Open this publication in new window or tab >>Mätproblem i norm- och kriterierelaterade prov: några analyser och försök med tonvikt på reliabilitets- och diskriminationsmått
1973 (Swedish)Doctoral thesis, comprehensive summary (Other academic)
Alternative title[en]
Problems of measurement in norm- and criterionreferenced tests : some analyses and investigations with emphasis on reliability and discrimination measures
Abstract [en]

The present report is a summary discussion of a number of studies in connection with the following problemas a) analysis and empirical investigations of some test theoretical formulae and principles, b) differential scoring of multiple-choice questions and c) criterion-referenced tests: theoretical and empirical implications. In studies associated with the first problem is shown how some test theoretical formulae and principles can be derived from the information in the inter-item-covariancs matrix as well as how this information can be used to explain empirically found relations. In the latter respect the standard error of measurement and various biserial correlation techniques are dealt with. Studies carriëd out within the second field of problems account for reliability and validity effects with differential scoring of multiple-choice questions. A priori as well as empirical weight systems have been used. Systematically ordered alternatives gave higher reliability with differential scoring compared to conventional scoring. At the same time, however, the dimensionality of the test changed with negative validity effects as a consequence. The trends of the results were the same for the two types of weight systems. In connection with the third field of problems the development of criterion-referenced tests, their qualities and restrictions are discussed. Three aspects are quite closely penetrated, viz. the defining of objectives, the homogeneity and the different cutting-scores of the tests. Furthermore theoretical as well as empirical problems of measurement techniques when evaluating data from criterion-referenced measurement are dealt with. In the empirical study especially low and moderate correlations between norm- and criterion-referenced discrimination indices were received, as well as between different criterion-referenced discrimination indices. Moreover, the results indicated that increased vali- . dity defined in terms of the difference between pre- and post-tests does not necessarily imply increased reliability when placing subjects above and below a fixed cutting-score.

Place, publisher, year, edition, pages
Umeå: Umeå universitet, 1973. 26 p.
Series
Akademiska avhandlingar vid Pedagogiska institutionen, Umeå universitet, ISSN 0281-6768 ; 4
Series
Pedagogiska rapporter Umeå, 35
National Category
Pedagogy
Identifiers
urn:nbn:se:umu:diva-16620 (URN)
Projects
digitalisering@umu
Available from: 2007-10-07 Created: 2007-10-07 Last updated: 2013-04-18Bibliographically approved

Open Access in DiVA

Reliabilitets- och validitetsstudier vid differentiell poängsättning av flervalsfrågor(5485 kB)128 downloads
File information
File name FULLTEXT01.pdfFile size 5485 kBChecksum SHA-512
26887dd9cb794a5b44af3bf67c7737bcb95625937cd4d101563dd0ee25de2cc78edf4aa7befd00fabe9e1afab7c84e93d45b29919992190929f22ff3d14a5926
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Wedman, Ingemar
By organisation
Department of Education
Pedagogy

Search outside of DiVA

GoogleGoogle Scholar
Total: 128 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 96 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf