umu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Computerized achievement tests: sequential and fixed length tests
Umeå University, Faculty of Social Sciences, Department of Statistics.
2003 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

The aim of this dissertation is to describe how a computerized achivement test can be constructed and used in practice. Throughout this dissertation the focus is on classifying the examinees into masters and non-masters depending on their ability. However, there has been no attempt to estimate their ability.

In paper I, a criterion-referenced computerized test with a fixed number of items is expressed as a statistical inference problem. The theory of optimal design is used to find the test that has the strongest power. A formal proof is provided showing that all items should have the same item characteristics, viz. high discrimination, low guessing and difficulty near the cutoff score, in order to give us the most powerful statistical test. An efficiency study shows how many times more non-optimal items are needed if we do not use optimal items in order to achieve the same power in the test.

In paper II, a computerized mastery sequential test is examined using sequential analysis. The focus is on examining the sequential probability ratio test and to minimize the number of items in a test, i.e. to minimize the average sample number function, abbreviated as the ASN function. Conditions under which the ASN function decreases are examined. Further, it is shown that the optimal values are the same for item discrimination and item guessing, but differ for item difficulty compared with tests with fixed number of items.

Paper III presents three simulation studies of sequential computerized mastery tests. Three cases are considered, viz. the examinees' responses are either identically distributed, not identically distributed, or not identically distributed together with estimation errors in the item characteristics. The simulations indicate that the observed results from the operating characteristic function differ significantly from the theoretical results. The mean number of items in a test, the distribution of test length and the variance depend on whether the true values of the item characteristics are known and whether they are iid or not.

In paper IV computerized tests with both pretested items with known item parameters, and try-out items with unknown item parameters are considered. The aim is to study how the item parameters for try-out items can be estimated in a computerized test. Although the unknown examinees' abilities may act as nuisance parameters, the asymptotic variance of the item parameter estimators can be calculated. Examples show that a more reliable variance estimator yields much larger estimates of the variance than commonly used variance estimators.

Place, publisher, year, edition, pages
Umeå: Statistiska institutionen, Umeå universitet , 2003. , 24 p.
Series
Statistical studies, ISSN 1100-8989 ; 29
Keyword [en]
Statistics, item respons theory, sequential analysis, statistical inference, power, optimal design, simulations
Keyword [sv]
Statistik
National Category
Probability Theory and Statistics
Research subject
Statistics
Identifiers
URN: urn:nbn:se:umu:diva-148ISBN: 91-7305-539-5 (print)OAI: oai:DiVA.org:umu-148DiVA: diva2:141202
Distributor:
Statistiska institutionen, 90187, Umeå
Public defence
2003-12-05, Hörsal, Norra beteendevetarhuset, Umeå universitet, Umeå, 10:00 (English)
Opponent
Supervisors
Available from: 2003-11-10 Created: 2003-11-10 Last updated: 2011-06-22Bibliographically approved
List of papers
1. An optimal design approach to criterion-referenced computerized testing
Open this publication in new window or tab >>An optimal design approach to criterion-referenced computerized testing
2003 (English)In: Journal of educational and behavioral statistics, ISSN 1076-9986, E-ISSN 1935-1054, Vol. 28, no 2, 97-110 p.Article in journal (Refereed) Published
Abstract [en]

A criterion-referenced computerized test is expressed as a statistical hypothesis problem. This admits that it can be studied by using the theory of optimal design. The power function of the statistical test is used as a criterion function when designing the test. A formal proof is provided showing that all items should have the same item characteristics, i.e. items that have high discrimination, low guessing and difficulty near the cut-off score give the most powerful statistical test. An efficiency study shows how many times more items are needed if nonoptimal items are used instead of optimal items in order to get the same power in the test.

Keyword
criterion-referenced tests, efficiency, power maximization
Identifiers
urn:nbn:se:umu:diva-2881 (URN)10.3102/10769986028002097 (DOI)
Available from: 2003-11-10 Created: 2003-11-10 Last updated: 2017-12-14Bibliographically approved
2. Optimal sequential mastery tests
Open this publication in new window or tab >>Optimal sequential mastery tests
(English)Manuscript (preprint) (Other academic)
National Category
Probability Theory and Statistics
Identifiers
urn:nbn:se:umu:diva-2882 (URN)
Available from: 2003-11-10 Created: 2003-11-10 Last updated: 2012-08-10Bibliographically approved
3. Sequential computerized mastery tests: three simulation studies
Open this publication in new window or tab >>Sequential computerized mastery tests: three simulation studies
2006 (English)In: International Journal of Testing, ISSN 1530-5058, E-ISSN 1532-7574, Vol. 6, no 1, 41-55 p.Article in journal (Refereed) Published
Abstract [en]

A simulation study of a sequential computerized mastery test is carried out with items modeled with the 3 parameter logistic item response theory model. The examinees' responses are either identically distributed, not identically distributed, or not identically distributed together with estimation errors in the item characteristics. The simulations indicated that the observed results from the operating characteristic function differ significantly from the theoretical results, which is probably due to the use of an approximation formula. The mean number of items in a test, the distribution of test length, and the variance depend highly on how well we know the true values of the item characteristics and whether they are identically distributed or not.

Place, publisher, year, edition, pages
Mahwah, NJ: Lawrence Erlbaum Associates, 2006
Keyword
item response theory, minimizing average sample number function
National Category
Computer and Information Science
Research subject
Statistics
Identifiers
urn:nbn:se:umu:diva-2883 (URN)10.1207/s15327574ijt0601_3 (DOI)
Available from: 2003-11-10 Created: 2003-11-10 Last updated: 2017-12-14Bibliographically approved
4. Estimation of try-out item parameters in computerized achievement tests
Open this publication in new window or tab >>Estimation of try-out item parameters in computerized achievement tests
(English)Manuscript (Other (popular science, discussion, etc.))
Identifiers
urn:nbn:se:umu:diva-2884 (URN)
Available from: 2003-11-10 Created: 2003-11-10 Last updated: 2011-03-10Bibliographically approved

Open Access in DiVA

fulltext(145 kB)1519 downloads
File information
File name FULLTEXT01.pdfFile size 145 kBChecksum SHA-1
34d5c15c01f0a78391892a60da36c8e34992cccab4aab0908e7be01d8104adc864564ba5
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Wiberg, Marie H.
By organisation
Department of Statistics
Probability Theory and Statistics

Search outside of DiVA

GoogleGoogle Scholar
Total: 1519 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 537 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf