Umeå University's logo

This study aims to assess student motivation to provide valid responses to the PISA student questionnaire. This was done by modeling response times using a three-component finite mixture model, comprising two satisficing response styles (rapid and idle) and one optimizing response style. Each participant’s motivation was operationalized as their probability of providing an optimizing response to questionnaire items. Overall, the model offered a good fit to the data. Results indicate that most responders were motivated to optimize, with a slight decline toward the end. Further, results showed a positive effect of questionnaire-taking motivation on PISA performance, suggesting a positive relationship to test-taking motivation. In conclusion, response times can be valuable indicators for assessing survey response quality and may serve as a proxy for test-taking motivation.

Each time new PISA results are presented, they gain a lot of attention. However, there are many factors that lie behind the results, and they get less attention. In this study, we take a person-centered approach and focus on students’ motivation and beliefs, and how these predict students’ effort and performance on the PISA 2015 assessment of scientific literacy. Moreover, we use both subjective (self-report) and objective (time-based) measures of effort, which allows us to compare these different types of measures. Latent profile analysis was used to group students in profiles based on their instrumental motivation, enjoyment, interest, self-efficacy, and epistemic beliefs (all with regard to science). A solution with four profiles proved to be best. When comparing the effort and performance of these four profiles, we saw several significant differences, but many of these differences disappeared when we added gender and the PISA index of economic, social, and cultural status (ESCS) as control variables. The main difference between the profiles, after adding control variables, was that the students in the profile with most positive motivation and sophisticated epistemic beliefs performed best and put in the most effort. Students in the profile with unsophisticated epistemic beliefs and low intrinsic values (enjoyment and interest) were most likely to be classified as low-effort responders. We conclude that strong motivation and sophisticated epistemic beliefs are important for both the effort students put into the PISA assessment and their performance, but also that ESCS had an unexpectedly large impact on the results.

It is well established that math anxiety has a negative relationship with math performance (MP). A few theories have provided explanations for this relationship. One of them, the Attentional Control Theory (ACT), suggests that anxiety can negatively impact the attentional control system and increase one's attention to threat-related stimuli. Within the ACT framework, the math anxiety (MA)—working memory (WM) relationship is argued to be critical for math performance. The present meta-analyses provides insights into the mechanisms of the MA—MP relation and the mediating role of WM. Through database searches with pre-determined search strings, 1,346 unique articles were identified. After excluding non-relevant studies, data from 57 studies and 150 effect sizes were used for investigating the MA—MP correlation using a random-effects model. This resulted in a mean correlation of r = −0.168. The database search of WM as a mediator for the MA—MP relation revealed 15 effects sizes leading to a descriptive rather than a generalizable statistic, with a mean indirect effect size of −0.092. Overall, the results confirm the ACT theory, WM does play a significant role in the MA—MP relationship.

Student motivation and self-beliefs in relation to school are important constructs when it comes to student learning and achievement. Adaptive motivational dispositions can be valued not only as predictors of performance but also as outcomes of schooling. Thus, it is not surprising that ILSAs from the very start have attempted to assess aspects related to student motivation and self-beliefs in addition to scholastic proficiency and literacy.

In this chapter, the motivational frameworks used in different ILSAs, with emphasis on those used in PISA and TIMSS/PIRLS, are briefly described together with the characteristics of contemporary motivation theory in order to give a context to research that has been published on the topic.

The main part of this chapter is then devoted to a review of findings from studies using ILSA data to explore motivational aspects such as intrinsic and extrinsic motivation, self-concept, and self-efficacy. The ambition is not to cover all existing research across time and ILSAs but rather to provide a representative and informative overview of salient research questions and main findings in relation to student motivation and beliefs in ILSA studies. In particular, findings regarding the relevance of motivational constructs for student achievement are reviewed across studies and contexts.

On a more general level, an attempt is also made to situate the ILSA motivation research within the larger achievement motivation research field. It will be argued that provided that the measured motivational constructs are properly defined and operationalized, secondary analyses using ILSA data can contribute to motivation research, in particular regarding the generalizability and universality of theoretical assumption and corroboration of findings using large-scale data.

Collecting process data in computer-based assessments provides opportunities to describe examinee behaviour during a test-taking session. The number of actions taken by students while interacting with an item is in this context a variable that has been gaining attention. The present study aims to investigate how the number of actions performed on constructed-response items relates to self-reported effort, performance, and item cluster position in the test. The theory of planned behaviour was used as an interpretative framework. Data from two item clusters of the 2015 Swedish Programme for International Student Assessment (PISA) Science administration were utilised. Results showed that the number of actions was significantly related to performance on the items, self-reported test-taking effort, and cluster position. Latent variable models were examined separately for performance-level groups. Overall, the number of actions performed on constructed-response items as a behavioural indicator in testing situations may be useful in gauging test-taking engagement.

The present study used process data from a computer-based problem-solving task as indications of behavioural level of test-taking effort, and explored how behavioural item-level effort related to overall test performance and self-reported effort. Variables were extracted from raw process data and clustered. Four distinct clusters were obtained and characterised as high effort, medium effort, low effort, and planner. Regression modelling indicated that among students that failed to solve the task, level of effort invested before giving up positively predicted overall test performance. Among students that solved the task, level of effort was instead weakly negatively related to test performance. A low level of behavioural effort before giving up the task was also related to lower self-reported effort. Results suggest that effort invested before giving up provides information about test-takers’ motivation to spend effort on the test. We conclude that process data could augment existing methods of assessing test-taking effort.

A person-centered cluster analysis approach to the study of motivation in IEA’s Trends in International Mathematics and Science Study (TIMSS) mathematics has revealed interesting profiles of students across key motivational constructs. Between four and six different clusters were extracted from each sample analyzed. Unsurprisingly, some clusters had consistent motivation scores, but in almost every jurisdiction, there were clusters of students with inconsistent score distributions between the contributing motivational constructs. The clusters were systematically different on various external variables, such as mean mathematics achievement, gender composition, and the level of home resources available to students. The study also presents a novel way of looking at the relative importance of enjoyment of, confidence in, and value for mathematics, and the association of these motivation variables with achievement and other demographic characteristics at the cluster level. When motivation scores were mixed rather than consistent, there was a uniform achievement advantage enjoyed by the groups of students who had higher scores for confidence in mathematics over enjoyment of, or value for mathematics. This approach revealed that gender and socioeconomic background are not independent of cluster membership. Typically, clusters with high confidence values were comprised of more boys than girls, and students from better resourced homes. The findings can be linked to relevant literature on motivation in mathematics. Educational efforts to develop student motivation need to take into account differential student profiles and prioritize techniques that target skill and competence in mathematics.

This study aims at modeling the self-reported test-taking motivation items in PISA and TIMSS Advanced studies for Swedish students using an IRT approach. In the last two cycles of the assessments, six test-specific items were included in the Swedish student questionnaires to evaluate pupil’s effort, motivation and how they perceived the importance of the tests. Using a Multiple-Group Generalized Partial Credit model (MG-GPCM), we created an IRT motivation scale for each assessment. We also investigated measurement invariance for the two cycles of PISA (i.e., 2012 and 2015) and of TIMSS Advanced (i.e., 2008 and 2015). Results indicated that the proposed scales refer to unidimensional constructs and measure reliably students’ motivation (Cronbach’s alpha above 0.78). Differential item functioning across assessment cycles was restricted to two criteria (RMSD and DSF) and had more impact on the latent motivation scale for PISA than for TIMSS Advanced. Overall, the test-taking motivation items fit well the purpose of a diagnostic of test-taking motivation in these two surveys and the proposed scales highlighted the slight increase of pupils’ motivation across the assessment cycles.