umu.sePublikationer
Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Statistical modeling in international large-scale assessments
Umeå universitet, Samhällsvetenskapliga fakulteten, Handelshögskolan vid Umeå universitet, Statistik.ORCID-id: 0000-0001-7282-5384
2016 (Engelska)Doktorsavhandling, sammanläggning (Övrigt vetenskapligt)Alternativ titel
Statistisk modellering i internationella komparativa mätningar (Svenska)
Abstract [en]

This thesis contributes to the area of research based on large-scale educational assessments, focusing on the application of multilevel models. The role of sampling weights, plausible values (response variable imputed multiple times) and imputation methods are demonstrated by simulations and applications to TIMSS (Trends in International Mathematics and Science Study) and PISA (Programme for International Student Assessment) data.

The large-scale assessments use multistage sampling design, which means that the units such as schools, classrooms, or students at some or all stages are selected with unequal probabilities. In order to make valid estimates and inferences sampling weights should be used. Thus, in the first paper, we examine different approaches and give recommendations concerning handling sampling weights in multilevel models when analyzing large-scale assessments.

Due to limitations in time and the number of students, the complex surveys use matrix sampling of items. This means that a response variable, i.e. students' performance, contains a large amount of information that is missing by design. Therefore, in order to estimate students' proficiency, TIMSS and PISA use the plausible values approach, which results in a set of five plausible values – proficiencies, computed for each student. In the second paper, different user strategies concerning plausible values for multilevel models as well as means and variances are examined with both real and simulated data. Missing information that is present because of the matrix sampling design for instance like the one used in PISA, can be arranged into a non-monotone missing data pattern, where all variables are incomplete and highly positively correlated. In the third paper, we compare a few imputation methods: a single imputation from a conditional distribution (with and without weights) and multiple imputation, for data with a non-monotone missing pattern (with no complete variables) and high positive correlation between variables.

In several of the recent international large-scale assessments, students in Sweden demonstrate a decreasing performance. Some previous research has shown that changes in performance depend on students’ performance levels. In the fourth paper, we studied the relationship between student performance and the between-school variance and tried to identify factors associated with student performance in mathematics in PISA in low-, medium-, and high- performing schools in the Nordic countries.

Ort, förlag, år, upplaga, sidor
Umeå: Umeå universitet , 2016. , s. 18
Serie
Statistical studies, ISSN 1100-8989 ; 51
Nyckelord [en]
multilevel model, plausible values, sampling weights, missing information, multiple imputation, non-monotone missing pattern, TIMSS, PISA
Nationell ämneskategori
Sannolikhetsteori och statistik
Forskningsämne
statistik; pedagogik
Identifikatorer
URN: urn:nbn:se:umu:diva-128618ISBN: 978-91-7601-612-1 (tryckt)OAI: oai:DiVA.org:umu-128618DiVA, id: diva2:1054880
Disputation
2017-01-12, Hörsal E, Humanisthuset, Umeå Universitet, Umeå, 10:00 (Engelska)
Opponent
Handledare
Tillgänglig från: 2016-12-16 Skapad: 2016-12-09 Senast uppdaterad: 2018-06-09Bibliografiskt granskad
Delarbeten
1. Importance of sampling weights in multilevel modeling of international large-scale assessment data
Öppna denna publikation i ny flik eller fönster >>Importance of sampling weights in multilevel modeling of international large-scale assessment data
2018 (Engelska)Ingår i: Communications in Statistics - Theory and Methods, ISSN 0361-0926, E-ISSN 1532-415X, Vol. 47, nr 20, s. 4991-5012Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

Multilevel modeling is an important tool for analyzing large-scale assessment data. However, the standard multilevel modeling will typically give biased results for such complex survey data. This bias can be eliminated by introducing design weights which must be used carefully as they can affect the results. The aim of this paper is to examine different approaches and to give recommendations concerning handling design weights in multilevel models when analyzing large-scale assessments such as TIMSS (The Trends in International Mathematics and Science Study). To achieve the goal of the paper, we examined real data from two countries and included a simulation study. The analyses in the empirical study showed that using no weights or only level 1 weights sometimes could lead to misleading conclusions. The simulation study only showed small differences in estimation of the weighted and unweighted models when informative design weights were used. The use of unscaled or not rescaled weights however caused significant differences in some parameter estimates.

Ort, förlag, år, upplaga, sidor
Philadelphia: Taylor & Francis, 2018
Nyckelord
informative weights, two-stage sampling, rescaling weights, simulation study
Nationell ämneskategori
Sannolikhetsteori och statistik
Forskningsämne
statistik; pedagogik
Identifikatorer
urn:nbn:se:umu:diva-128588 (URN)10.1080/03610926.2017.1383429 (DOI)000440044100006 ()
Forskningsfinansiär
Vetenskapsrådet, 2015-02160
Tillgänglig från: 2016-12-07 Skapad: 2016-12-07 Senast uppdaterad: 2018-09-04Bibliografiskt granskad
2. Using plausible values in secondary analysis in large–scale assessments
Öppna denna publikation i ny flik eller fönster >>Using plausible values in secondary analysis in large–scale assessments
2017 (Engelska)Ingår i: Communications in Statistics - Theory and Methods, ISSN 0361-0926, E-ISSN 1532-415X, Vol. 46, nr 22, s. 11341-11357Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

Plausible values are typically used in large–scale assessment studies, in particular in the Trends in International Mathematics and Science Study and the Programme for International Student Assessment. Despite its large spread there are still some questions regarding the use of plausible values and how such use affects statistical analyses. The aim of this paper is to demonstrate the role of plausible values in large–scale assessment surveys when multilevel modelling is used. Different user strategies concerning plausible values for multilevel models as well as means and variances are examined. The results show that some commonly used user strategies give incorrect results while others give reasonable estimates but incorrect standard errors. These findings are important for anyone wishing to make secondary analyses of large–scale assessment data, especially those interested in using multilevel models to analyze the data.

Ort, förlag, år, upplaga, sidor
Taylor & Francis, 2017
Nyckelord
Achievement, design study, multilevel modelling, simulation studies, testing
Nationell ämneskategori
Sannolikhetsteori och statistik
Identifikatorer
urn:nbn:se:umu:diva-128615 (URN)10.1080/03610926.2016.1267764 (DOI)000412555500031 ()
Anmärkning

Originally included in thesis in manuscript form 

Tillgänglig från: 2016-12-09 Skapad: 2016-12-09 Senast uppdaterad: 2018-06-09Bibliografiskt granskad
3. Single Imputation from a Conditional Distribution vs Multiple Imputation for Data with a Non-monotone Missing Pattern
Öppna denna publikation i ny flik eller fönster >>Single Imputation from a Conditional Distribution vs Multiple Imputation for Data with a Non-monotone Missing Pattern
(Engelska)Manuskript (preprint) (Övrigt vetenskapligt)
Abstract [en]

Missing information is common in real data studies. When missingness is large, it should not be ignored and, instead, a missing data imputation method should be considered. The choice of the imputation method depends on the type or the pattern of missing information, as well as the nature of data. For instance, observations in large-scale educational assessments are incomplete by missing some components and based on usually positively correlated results within students. In all types of analysis of such data, the correlation has to be considered in a reliable calculation of properties of estimates. The aim of this paper is to compare a single imputation from a conditional distribution (with or without weights) and multiple imputation for data with a non-monotone missing pattern and high positive correlation between variables. For this purpose, such estimates as mean and variance are compared. The simulation results showed that expectation and variance are estimated more reliably when the imputation from a conditional distribution (without and with weights) or a complete-data set are used, compared to multiple imputation.

Nyckelord
Item-nonresponse, plausible values, highly correlated measurements, large-scale assessments, planed missingness
Nationell ämneskategori
Sannolikhetsteori och statistik
Identifikatorer
urn:nbn:se:umu:diva-128616 (URN)
Tillgänglig från: 2016-12-09 Skapad: 2016-12-09 Senast uppdaterad: 2018-06-09
4. Low-, Medium-, and High-performing Schools in the Nordic Countries: student Performance at PISA Mathematics 2003-2012
Öppna denna publikation i ny flik eller fönster >>Low-, Medium-, and High-performing Schools in the Nordic Countries: student Performance at PISA Mathematics 2003-2012
(Engelska)Manuskript (preprint) (Övrigt vetenskapligt)
Abstract [en]

Decreasing performance among students in Sweden on international comparative studies and increasing segregation of schools, has led to a debate concerning strategies for improving student performance. The aim of this study is to analyse the between school variance and to identify factors associated with student performance in PISA in mathematics at different performance levels in the Nordic countries. In order to separate the effect of school-level variables, from the effect of student’s background factors and to take care of the multistage sampling design used in PISA, multilevel analysis was used. The results show that no evidence regarding the relationship between the average student performance in mathematics and the between-school variance was found which, is in contrast to previous studies conducted on science performance in PISA. Regarding school-level factors, our results overall have shown that few school-level factors (having a positive or a negative effect) seemed to be associated with performance. School-level factors associated with performance have mainly been identified among low- and medium-performing schools, and to a less extent among students at high-performing schools (only in Sweden and Denmark). This is a result which is in line with other studies showing the educational system’s incapacity to provide support for high-performing students and to enhance their learning.

Nyckelord
Performance levels, Between-school variance, Multilevel modelling, School-level factors, Large-scale comparative studies, School-effectiveness
Nationell ämneskategori
Sannolikhetsteori och statistik
Forskningsämne
pedagogik
Identifikatorer
urn:nbn:se:umu:diva-128617 (URN)
Projekt
What can we learn from large-scale assessments about promoting student success?
Forskningsfinansiär
Vetenskapsrådet, 2015-02160
Tillgänglig från: 2016-12-09 Skapad: 2016-12-09 Senast uppdaterad: 2018-09-06

Open Access i DiVA

omslag(132 kB)0 nedladdningar
Filinformation
Filnamn COVER01.pngFilstorlek 132 kBChecksumma SHA-512
22e2100360069298b82d228c23454c61bd3b29d50397eecf96c76534b4914d266bea8adf56fe4fd416c281e09a6b837fcf3192d89f5a4c0e0745d7ad6948283f
Typ coverMimetyp image/png
spikblad(98 kB)21 nedladdningar
Filinformation
Filnamn SPIKBLAD01.pdfFilstorlek 98 kBChecksumma SHA-512
f508db568d32708452c09b09af6c678f50e0e5bf27caa1daae428e2dfe645b6471684fc89cf45995bb742b2e70d0de64a3cc0f48839f623812bf37dfba6a957d
Typ spikbladMimetyp application/pdf
fulltext(535 kB)195 nedladdningar
Filinformation
Filnamn FULLTEXT01.pdfFilstorlek 535 kBChecksumma SHA-512
3894f191921fdacd19e53b30c05acaf6998c3b18b48c0042697dccb320e9561a37afaedee299e9918eee47399dcaf2758a436386a0e73e83f0b8c3f79efa89b2
Typ fulltextMimetyp application/pdf

Personposter BETA

Laukaityte, Inga

Sök vidare i DiVA

Av författaren/redaktören
Laukaityte, Inga
Av organisationen
Statistik
Sannolikhetsteori och statistik

Sök vidare utanför DiVA

GoogleGoogle Scholar
Totalt: 195 nedladdningar
Antalet nedladdningar är summan av nedladdningar för alla fulltexter. Det kan inkludera t.ex tidigare versioner som nu inte längre är tillgängliga.

isbn
urn-nbn

Altmetricpoäng

isbn
urn-nbn
Totalt: 1115 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf