Change search
ReferencesLink to record
Permanent link

Direct link
Optimal sequential mastery tests
Umeå University, Faculty of Social Sciences, Department of Statistics.
(English)Manuscript (preprint) (Other academic)
National Category
Probability Theory and Statistics
URN: urn:nbn:se:umu:diva-2882OAI: diva2:141199
Available from: 2003-11-10 Created: 2003-11-10 Last updated: 2012-08-10Bibliographically approved
In thesis
1. Computerized achievement tests: sequential and fixed length tests
Open this publication in new window or tab >>Computerized achievement tests: sequential and fixed length tests
2003 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

The aim of this dissertation is to describe how a computerized achivement test can be constructed and used in practice. Throughout this dissertation the focus is on classifying the examinees into masters and non-masters depending on their ability. However, there has been no attempt to estimate their ability.

In paper I, a criterion-referenced computerized test with a fixed number of items is expressed as a statistical inference problem. The theory of optimal design is used to find the test that has the strongest power. A formal proof is provided showing that all items should have the same item characteristics, viz. high discrimination, low guessing and difficulty near the cutoff score, in order to give us the most powerful statistical test. An efficiency study shows how many times more non-optimal items are needed if we do not use optimal items in order to achieve the same power in the test.

In paper II, a computerized mastery sequential test is examined using sequential analysis. The focus is on examining the sequential probability ratio test and to minimize the number of items in a test, i.e. to minimize the average sample number function, abbreviated as the ASN function. Conditions under which the ASN function decreases are examined. Further, it is shown that the optimal values are the same for item discrimination and item guessing, but differ for item difficulty compared with tests with fixed number of items.

Paper III presents three simulation studies of sequential computerized mastery tests. Three cases are considered, viz. the examinees' responses are either identically distributed, not identically distributed, or not identically distributed together with estimation errors in the item characteristics. The simulations indicate that the observed results from the operating characteristic function differ significantly from the theoretical results. The mean number of items in a test, the distribution of test length and the variance depend on whether the true values of the item characteristics are known and whether they are iid or not.

In paper IV computerized tests with both pretested items with known item parameters, and try-out items with unknown item parameters are considered. The aim is to study how the item parameters for try-out items can be estimated in a computerized test. Although the unknown examinees' abilities may act as nuisance parameters, the asymptotic variance of the item parameter estimators can be calculated. Examples show that a more reliable variance estimator yields much larger estimates of the variance than commonly used variance estimators.

Place, publisher, year, edition, pages
Umeå: Statistiska institutionen, Umeå universitet, 2003. 24 p.
Statistical studies, ISSN 1100-8989 ; 29
Statistics, item respons theory, sequential analysis, statistical inference, power, optimal design, simulations, Statistik
National Category
Probability Theory and Statistics
Research subject
urn:nbn:se:umu:diva-148 (URN)91-7305-539-5 (ISBN)
Statistiska institutionen, 90187, Umeå
Public defence
2003-12-05, Hörsal, Norra beteendevetarhuset, Umeå universitet, Umeå, 10:00 (English)
Available from: 2003-11-10 Created: 2003-11-10 Last updated: 2011-06-22Bibliographically approved

Open Access in DiVA

No full text

Search in DiVA

By author/editor
Wiberg, Marie
By organisation
Department of Statistics
Probability Theory and Statistics

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Total: 26 hits
ReferencesLink to record
Permanent link

Direct link