umu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Using plausible values in secondary analysis in large–scale assessments
Umeå University, Faculty of Social Sciences, Umeå School of Business and Economics (USBE), Statistics.ORCID iD: 0000-0001-7282-5384
Umeå University, Faculty of Social Sciences, Umeå School of Business and Economics (USBE), Statistics.ORCID iD: 0000-0001-5549-8262
2017 (English)In: Communications in Statistics - Theory and Methods, ISSN 0361-0926, E-ISSN 1532-415X, Vol. 46, no 22, p. 11341-11357Article in journal (Refereed) Published
Abstract [en]

Plausible values are typically used in large–scale assessment studies, in particular in the Trends in International Mathematics and Science Study and the Programme for International Student Assessment. Despite its large spread there are still some questions regarding the use of plausible values and how such use affects statistical analyses. The aim of this paper is to demonstrate the role of plausible values in large–scale assessment surveys when multilevel modelling is used. Different user strategies concerning plausible values for multilevel models as well as means and variances are examined. The results show that some commonly used user strategies give incorrect results while others give reasonable estimates but incorrect standard errors. These findings are important for anyone wishing to make secondary analyses of large–scale assessment data, especially those interested in using multilevel models to analyze the data.

Place, publisher, year, edition, pages
Taylor & Francis, 2017. Vol. 46, no 22, p. 11341-11357
Keywords [en]
Achievement, design study, multilevel modelling, simulation studies, testing
National Category
Probability Theory and Statistics
Identifiers
URN: urn:nbn:se:umu:diva-128615DOI: 10.1080/03610926.2016.1267764ISI: 000412555500031OAI: oai:DiVA.org:umu-128615DiVA, id: diva2:1054844
Note

Originally included in thesis in manuscript form 

Available from: 2016-12-09 Created: 2016-12-09 Last updated: 2018-06-09Bibliographically approved
In thesis
1. Statistical modeling in international large-scale assessments
Open this publication in new window or tab >>Statistical modeling in international large-scale assessments
2016 (English)Doctoral thesis, comprehensive summary (Other academic)
Alternative title[sv]
Statistisk modellering i internationella komparativa mätningar
Abstract [en]

This thesis contributes to the area of research based on large-scale educational assessments, focusing on the application of multilevel models. The role of sampling weights, plausible values (response variable imputed multiple times) and imputation methods are demonstrated by simulations and applications to TIMSS (Trends in International Mathematics and Science Study) and PISA (Programme for International Student Assessment) data.

The large-scale assessments use multistage sampling design, which means that the units such as schools, classrooms, or students at some or all stages are selected with unequal probabilities. In order to make valid estimates and inferences sampling weights should be used. Thus, in the first paper, we examine different approaches and give recommendations concerning handling sampling weights in multilevel models when analyzing large-scale assessments.

Due to limitations in time and the number of students, the complex surveys use matrix sampling of items. This means that a response variable, i.e. students' performance, contains a large amount of information that is missing by design. Therefore, in order to estimate students' proficiency, TIMSS and PISA use the plausible values approach, which results in a set of five plausible values – proficiencies, computed for each student. In the second paper, different user strategies concerning plausible values for multilevel models as well as means and variances are examined with both real and simulated data. Missing information that is present because of the matrix sampling design for instance like the one used in PISA, can be arranged into a non-monotone missing data pattern, where all variables are incomplete and highly positively correlated. In the third paper, we compare a few imputation methods: a single imputation from a conditional distribution (with and without weights) and multiple imputation, for data with a non-monotone missing pattern (with no complete variables) and high positive correlation between variables.

In several of the recent international large-scale assessments, students in Sweden demonstrate a decreasing performance. Some previous research has shown that changes in performance depend on students’ performance levels. In the fourth paper, we studied the relationship between student performance and the between-school variance and tried to identify factors associated with student performance in mathematics in PISA in low-, medium-, and high- performing schools in the Nordic countries.

Place, publisher, year, edition, pages
Umeå: Umeå universitet, 2016. p. 18
Series
Statistical studies, ISSN 1100-8989 ; 51
Keywords
multilevel model, plausible values, sampling weights, missing information, multiple imputation, non-monotone missing pattern, TIMSS, PISA
National Category
Probability Theory and Statistics
Research subject
Statistics; Education
Identifiers
urn:nbn:se:umu:diva-128618 (URN)978-91-7601-612-1 (ISBN)
Public defence
2017-01-12, Hörsal E, Humanisthuset, Umeå Universitet, Umeå, 10:00 (English)
Opponent
Supervisors
Available from: 2016-12-16 Created: 2016-12-09 Last updated: 2018-06-09Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full text

Authority records BETA

Laukaityte, IngaWiberg, Marie

Search in DiVA

By author/editor
Laukaityte, IngaWiberg, Marie
By organisation
Statistics
In the same journal
Communications in Statistics - Theory and Methods
Probability Theory and Statistics

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 742 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf