umu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Mätproblem i norm- och kriterierelaterade prov: några analyser och försök med tonvikt på reliabilitets- och diskriminationsmått
Umeå University, Faculty of Social Sciences, Department of Education.
1973 (Swedish)Doctoral thesis, comprehensive summary (Other academic)Alternative title
Problems of measurement in norm- and criterionreferenced tests : some analyses and investigations with emphasis on reliability and discrimination measures (English)
Abstract [en]

The present report is a summary discussion of a number of studies in connection with the following problemas a) analysis and empirical investigations of some test theoretical formulae and principles, b) differential scoring of multiple-choice questions and c) criterion-referenced tests: theoretical and empirical implications. In studies associated with the first problem is shown how some test theoretical formulae and principles can be derived from the information in the inter-item-covariancs matrix as well as how this information can be used to explain empirically found relations. In the latter respect the standard error of measurement and various biserial correlation techniques are dealt with. Studies carriëd out within the second field of problems account for reliability and validity effects with differential scoring of multiple-choice questions. A priori as well as empirical weight systems have been used. Systematically ordered alternatives gave higher reliability with differential scoring compared to conventional scoring. At the same time, however, the dimensionality of the test changed with negative validity effects as a consequence. The trends of the results were the same for the two types of weight systems. In connection with the third field of problems the development of criterion-referenced tests, their qualities and restrictions are discussed. Three aspects are quite closely penetrated, viz. the defining of objectives, the homogeneity and the different cutting-scores of the tests. Furthermore theoretical as well as empirical problems of measurement techniques when evaluating data from criterion-referenced measurement are dealt with. In the empirical study especially low and moderate correlations between norm- and criterion-referenced discrimination indices were received, as well as between different criterion-referenced discrimination indices. Moreover, the results indicated that increased vali- . dity defined in terms of the difference between pre- and post-tests does not necessarily imply increased reliability when placing subjects above and below a fixed cutting-score.

Place, publisher, year, edition, pages
Umeå: Umeå universitet , 1973. , 26 p.
Series
Akademiska avhandlingar vid Pedagogiska institutionen, Umeå universitet, ISSN 0281-6768 ; 4
Series
Pedagogiska rapporter Umeå, 35
National Category
Pedagogy
Identifiers
URN: urn:nbn:se:umu:diva-16620OAI: oai:DiVA.org:umu-16620DiVA: diva2:156293
Projects
digitalisering@umu
Available from: 2007-10-07 Created: 2007-10-07 Last updated: 2013-04-18Bibliographically approved
List of papers
1. Analysis of the Inter‐Item‐Covariance Matrix
Open this publication in new window or tab >>Analysis of the Inter‐Item‐Covariance Matrix
1972 (English)In: Scandinavian Journal of Educational Research, ISSN 0031-3831, E-ISSN 1470-1170, Vol. 16, no 1, 25-35 p.Article in journal (Refereed) Published
Place, publisher, year, edition, pages
Oslo: Universitetsforlaget, 1972
National Category
Mathematics Pedagogy
Identifiers
urn:nbn:se:umu:diva-68408 (URN)10.1080/0031383720160102 (DOI)
Available from: 2013-04-18 Created: 2013-04-18 Last updated: 2017-12-06Bibliographically approved
2. Mätningens standardfel och testlängd
Open this publication in new window or tab >>Mätningens standardfel och testlängd
1973 (Swedish)Report (Other academic)
Alternative title[en]
The standard error of measurement and the length of the test
Abstract [sv]

Med utgångspunkt i inter-item-kovariansmatrisen visas hurmätningens standardfel approximativt kan uttryckas i termerav uppgifternas svårighetsgrad. I anslutning till denhärledda relationen diskuteras också Lords (1959) empiriskt erhållna resultat SEM = .432√n.

Abstract [en]

With the inter-item-covariance matrix as a'basis it willhere be shown how the standard error of measurementapproximately can be expressed in terms of item difficulty.In connection with the derived relation Lord's (1959) empirically received result SEM = .432√n isdiscussed.

Place, publisher, year, edition, pages
Umeå: Umeå universitet och Lärarhögskolan, 1973. 7 p.
Series
Pedagogiska rapporter Umeå, ISSN 0348-9388 ; 31
National Category
Mathematics Pedagogy
Identifiers
urn:nbn:se:umu:diva-68285 (URN)
Projects
digitalisering@umu
Available from: 2013-04-15 Created: 2013-04-15 Last updated: 2013-04-18Bibliographically approved
3. Relationer mellan biseriala diskriminationsindex
Open this publication in new window or tab >>Relationer mellan biseriala diskriminationsindex
1971 (Swedish)Report (Other academic)
Abstract [sv]

I föreliggande undersökning har följande problem behandlats:

(1) Relationer mellan olika index för beräkning av biserial korrelation.

(2) Testlängdens inverkan på de index, som behandlats under (1).

Vid beräkning av diskriminationsindex talade resultaten för användandet av den koefficient, som uttrycker sambandet mellan respektive uppgift och den generella faktorn, som mäts av totaltestet, förutsatt att de antaganden som gäller för biserial korrelation kan anses vara uppfyllda.

Den överskattning man fick då okorrigerad koefficient beräknades relativt då ovan nämnda koefficient beräknades var mindre än .10 vid 20 uppgifter. Vid 40 uppgifter förelåg samma relation mellan okorrigerad koefficient och den koefficient, som uttrycker sambandet mellan respektive uppgift och summan av de återstående uppgifterna.

Abstract [en]

In the present study the following problems have been dealt with.

(1) Relations between different indices when estimating bi-serial correlation.

(2) The effect of the length of the test on the indices dealt with under (1).

When estimating the discrimination indices the results supported the use of the coefficient which expresses the correlation between every item and the general factor, measured by the total test, provided that the assumptions required for biserial correlation can be made.

The overestimation received when calculating the uncorrected coefficient was lower than .10 in a test of 20 items compared with using the coefficient mentioned above. With 40 items there was the same relation between the uncorrected coefficient and the coefficient expressing the correlation between every item and the remaining items.

Place, publisher, year, edition, pages
Umeå: Umeå universitet och Lärarhögskolan, 1971. 14 p.
Series
Pedagogiska rapporter Umeå, ISSN 0348-9388 ; 16
National Category
Pedagogy Mathematics
Identifiers
urn:nbn:se:umu:diva-68287 (URN)
Projects
digitalisering@umu
Available from: 2013-04-15 Created: 2013-04-15 Last updated: 2013-04-18Bibliographically approved
4. Reliabilitets- och validitetsstudier vid differentiell poängsättning av flervalsfrågor
Open this publication in new window or tab >>Reliabilitets- och validitetsstudier vid differentiell poängsättning av flervalsfrågor
1973 (Swedish)Report (Other academic)
Alternative title[en]
Reliability and validity studies with differential scoring of multiple-choice questions
Abstract [sv]

I föreliggande rapport redovisas tre delförsök där diffe-rentiel.l poängsättning av flervalsfrågors svarsalternativ har prövats och i reliabilitets- och validitetshänseende jämförts med konventionell dikotom poängsättning. Både empiriska och a priori-vikter. har tillämpats. Underlaget till de förra utgjordes dels avbiseriala korrelationer mellan respektive alternativ och totalpoäng och dels av medelvärden i totalpoäng för de som valt respektive svarsalternativ (guttmanvikter). A priori-vikterna erhölls från olika skattningsförfaranden. På prov utformade med ordnade svarsalternativ erhölls högre reliabilitet vid differen-tiell än vid konventionell poängsättning. De högsta reliabi litetBvärdena erhölls med de mera stringenta viktsystemen. Då jämförelserna gjordes med den reliabilitet som erhölls vid konventionell poängsättning på ett parallellt prov utformat utan ordnade svarsalternativ eliminerades denna positiva effekt utom för de stringenta viktsystemen. I validitetshänseende resulterade differentiell poängsättning genomgående i sämre resultat än konventionell poängsättning. I kommentarerna till de erhållna resultaten framhålls den betydelse provets utformning har på möjligheterna att erhålla positiva reliabilitetseffekter med differen-tiell poängsättning. Avslutningsvis redovisas begränsningar i undersökningen och förslag till fortsatta prövningar ges.

Abstract [en]

In the present report three studies will be accounted for, where differential scoring of the alternatives of multiple-choice questions has been tested and as for reliability and validity has been compared to conventional dichotomous scoring. Empirical as well as a priori weights have been used. The basis of the former ones consisted partly of biserial correlations between every alternative and the total score and partly of the mean of the total score for those who had chosen the actual alternative (Guttman weights). The a priori weights were received through different judgement procedures. Tests with ordered alternatives showed higher reliability with differential than with conventional scoring. The highest reliability values were received with the more stringent weight systems.When the comparisons were made with the reliability received with conventional scoring of a parallel test, constructed without ordered alternatives, this positive effect was eliminated except for the stringent weight systems. As for validity, differential scoring showed on the whole lower values than conventional scoring. When commenting on the received results the importance of the construction of the test to receive positive reliability effects with differential scoring is pointed out. Finally the restrictions of the study are accounted for and suggestions for future research are given.

Place, publisher, year, edition, pages
Umeå: Umeå universitet och Lärarhögskolan, 1973. 61 p.
Series
Pedagogiska rapporter Umeå, ISSN 0348-9388 ; 32
National Category
Pedagogy
Identifiers
urn:nbn:se:umu:diva-68284 (URN)
Projects
digitalisering@umu
Available from: 2013-04-15 Created: 2013-04-15 Last updated: 2013-04-18Bibliographically approved
5. Kriterierelaterade prov: bakgrund, egenskaper och begränsningar
Open this publication in new window or tab >>Kriterierelaterade prov: bakgrund, egenskaper och begränsningar
1973 (Swedish)Report (Other academic)
Alternative title[en]
Criterion-referenced tests : bakground, qualities and restrictions
Abstract [sv]

Rapporten inleds med en redogörelse för framväxten av och olika definitioner på kriterierelaterade prov. Därpå analyseras grundläggande problem vid kriterierelaterade mätningar. I det avsnittet behandlas målens inverkan på provens utformning, uppgifternas homogenitet och kravgränser för vad som skall betraktas som godkända kunskaper. I nästa avdelning tas konstruktion av provuppgifter upp. Olika användningsområden för kriterierelaterade prov behandlas därefter. Rapporten avslutas med några synpunkter på det fortsatta arbetet med kriterierelaterade prov.

Abstract [en]

The study begins with an account of the development of criterion-referenced tests and different definitions of them. After that fundamental problems of criterion-referenced measurements are analysed. In that part the effect of the objectives on the construction of the tests, the homogeneity of the items and the cutti;i3g'"SCof,es for-what should be regarded as satisfactory knowledge are dealt with. In the following part the construction of items is accounted for. Then different fields of application for criterion-referenced tests are discussed. The report ends with some views on the future work with criterion-referenced tests.

Place, publisher, year, edition, pages
Umeå: Umeå universitet och Lärarhögskolan, 1973. 25 p.
Series
Pedagogiska rapporter Umeå, ISSN 0348-9388 ; 33
National Category
Pedagogy
Identifiers
urn:nbn:se:umu:diva-68245 (URN)
Projects
digitalisering@umu
Available from: 2013-04-15 Created: 2013-04-15 Last updated: 2013-04-18Bibliographically approved
6. Reliabilitets-, validitets- och diskriminationsmått för kriterierelaterade prov
Open this publication in new window or tab >>Reliabilitets-, validitets- och diskriminationsmått för kriterierelaterade prov
1973 (Swedish)Report (Other academic)
Alternative title[en]
Reliability, validity and discrimination measures for criterion-referenced tests
Abstract [sv]

Denna undersökning har teoretiskt och empiriskt behandlat reliabilitets-, validitets- och diskriminationsrnått i anslutning till kriterierelaterade prov. Syftet med den empiriska studien var att undersöka relationer mellan olika diskriminationsrnått och vilka reliabilitets- och validi-tetseffekter som urval av uppgifter med dessa mått medförde. De olika diskriminationsmåtten var överlag måttligt interkorrelerade. Särskilt intressant att observera var de måttliga interkorrelationerna mellan de tre diskriminationsmåtten definierade som differensen i lösningaproportion mellan a) de 27 % bästa och de 27 % sämsta individerna utifrån eftertestningens resultat, b) resultaten på för-, och eftermätning och c) resultaten från en expertgrupp och en icke-expertgrupp. Reliabiliteten definierad som överensstämmelsen mellan resultaten vid två mättillfällen påverkades positivt oavsett vilket av de tre diskriminationsmåtten a), b) eller c) som användes för urval av uppgifter. Beträffande de beslutsorienterade reliabilitets-måtten var emellertid effekterna något olika för olika uppgiftsurval. Resultaten visade också att en ökning av reliabiliteten inte nödvändigtvis be-höver innebära en ökning av validiteten definierad som skillnaden mellan resultaten på en för- och eftermätning. Avslutningsvis behandlas begränsningar i undersökningen och förslag till-fortsatta prövningar ges.

Abstract [en]

This study has, theoretically and empirically, dealt with reliability, validity and discrimination measures in connection with criterion-referenced tests. The purpose of the empirical study was to investigate relations between different discrimination' measures and what effects on reliability and validity that selection of items gave with the help of these measures. The different discrimination measures were on the whole moderately intercorre-lated. It was especially interesting to observe the moderate intercorrelations between the three discrimination measures defined as the differences in item difficulty between a) the upper and lower 27 per cent of the subjects according to the results of the posttest, b) the results of the pre- and posttests and c) the results from an expert group as well as a non-expert one. The reliability defined as the agreement between the results of two posttests was positively influenced no matter which one of the three discrimination measures a), b) or c)that was used for the selection of the items. As for the decision-oriented reliability measures the effects were, however, somewhat different for the 'different selections of items. Moreover, the results showed that an increase in reliability does not necessarily imply an increase in validity defined "as the difference between the results of pre- and posttests. Finally restrictions of the study are dealt with and suggestions for further investigations are made.

Place, publisher, year, edition, pages
Umeå: Umeå universitet och Lärarhögskolan, 1973. 49 p.
Series
Pedagogiska rapporter Umeå, ISSN 0348-9388 ; 34
National Category
Pedagogy
Identifiers
urn:nbn:se:umu:diva-68313 (URN)
Projects
digitalisering@umu
Available from: 2013-04-16 Created: 2013-04-16 Last updated: 2013-04-18Bibliographically approved

Open Access in DiVA

Mätproblem i norm- och kriterierelaterade prov(3334 kB)357 downloads
File information
File name FULLTEXT02.pdfFile size 3334 kBChecksum SHA-512
dcbface1bf30a07d958b91b1d8489c1f6744f2d5f4ad8863b1561ad5f439cf9b0255d5217064f70424a93c916c3a2225973272e70d2908b19f84900692c935eb
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Wedman, Ingemar
By organisation
Department of Education
Pedagogy

Search outside of DiVA

GoogleGoogle Scholar
Total: 357 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 423 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf