Umeå universitets logga

umu.sePublikationer
Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Questionnaire-Taking Motivation: Using response times to assess motivation to optimize in the PISA student questionnaire
Umeå universitet, Samhällsvetenskapliga fakulteten, Institutionen för tillämpad utbildningsvetenskap, Beteendevetenskapliga mätningar (BVM).ORCID-id: 0000-0002-6998-3397
Umeå universitet, Samhällsvetenskapliga fakulteten, Institutionen för tillämpad utbildningsvetenskap, Beteendevetenskapliga mätningar (BVM).ORCID-id: 0000-0002-4630-6123
(Engelska)Manuskript (preprint) (Övrigt vetenskapligt)
Nationell ämneskategori
Övrig annan samhällsvetenskap
Forskningsämne
beteendevetenskapliga mätningar
Identifikatorer
URN: urn:nbn:se:umu:diva-206863OAI: oai:DiVA.org:umu-206863DiVA, id: diva2:1751760
Tillgänglig från: 2023-04-19 Skapad: 2023-04-19 Senast uppdaterad: 2023-04-19
Ingår i avhandling
1. Exploring and modeling response process data from PISA: inferences related to motivation and problem-solving
Öppna denna publikation i ny flik eller fönster >>Exploring and modeling response process data from PISA: inferences related to motivation and problem-solving
2023 (Engelska)Doktorsavhandling, sammanläggning (Övrigt vetenskapligt)
Alternativ titel[sv]
Modellering av responsprocessdata från PISA : inferenser relaterade till motivation och problemlösning
Abstract [en]

This thesis explores and models response process data from large-scale assessments, focusing on test-taking motivation, problem-solving strategies, and questionnaire response validity. It consists of four studies, all using data from PISA (Programme for International Student Assessment) data.

Study I processed and clustered log-file data to create a behavioral evaluation of students' effort applied to a PISA problem-solving item, and examined the relationship between students' behavioral effort, self-reported effort, and test performance. Results show that effort invested before leaving the task unsolved was positively related to performance, while effort invested before solving the tasks was not. Low effort before leaving the task unsolved was further related to lower self-reported effort. The findings suggest that test-taking motivation could only be validly measured from efforts exerted before giving up.

Study II used response process data to infer students' problem-solving strategies on a PISA problem-solving task, and investigated the efficiency of strategies and their relationship to PISA performance. A text classifier trained on data from a generative computational model was used to retrieve different strategies, reaching a classification accuracy of 0.72, which increased to 0.90 with item design changes. The most efficient strategies used information from the task environment to make plans. Test-takers classified as selecting actions randomly performed worse overall. The study concludes that computational modeling can inform score interpretation and item design.

Study III investigated the relationship between motivation to answer the PISA student questionnaire and test performance. Departing from the theory of satisficing in surveys a Bayesian finite mixture model was developed to assess questionnaire-taking motivation. Results showed that overall motivation was high, but decreased toward the end. The questionnaire-taking motivation was positively related to performance, suggesting that it could be a proxy for test-taking motivation, however, reading skills may affect the estimation.

Study IV examines the validity of composite scores assessing reading metacognition, using a Bayesian finite mixture model that jointly considers response times and sequential patterns in subitem responses. The results show that, the relatively high levels of satisficing (up to 30%) negatively biased composite scores. The study highlights the importance of considering response time data and subitem response patterns when the validity of scores from the student questionnaire.

In conclusion, response process data from international large-scale assessments can provide valuable insights into test-takers’ motivation, problem-solving strategies, and questionnaire validity.

Ort, förlag, år, upplaga, sidor
Umeå: Umeå University, 2023. s. 53
Serie
Academic dissertations at the department of Educational Measurement, ISSN 1652-9650 ; 15
Nyckelord
response processes, large-scale assessments, motivation, problem-solving, computational modeling, Bayesian modeling
Nationell ämneskategori
Övrig annan samhällsvetenskap
Forskningsämne
beteendevetenskapliga mätningar
Identifikatorer
urn:nbn:se:umu:diva-206866 (URN)978-91-8070-058-0 (ISBN)978-91-8070-057-3 (ISBN)
Disputation
2023-05-17, Aula Biologica, Umeå, 10:00 (Engelska)
Opponent
Handledare
Tillgänglig från: 2023-04-26 Skapad: 2023-04-19 Senast uppdaterad: 2023-04-19Bibliografiskt granskad

Open Access i DiVA

Fulltext saknas i DiVA

Person

Lundgren, ErikEklöf, Hanna

Sök vidare i DiVA

Av författaren/redaktören
Lundgren, ErikEklöf, Hanna
Av organisationen
Beteendevetenskapliga mätningar (BVM)
Övrig annan samhällsvetenskap

Sök vidare utanför DiVA

GoogleGoogle Scholar

urn-nbn

Altmetricpoäng

urn-nbn
Totalt: 71 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf