Umeå University's logo

The same dataset can be analysed in different justifiable ways to answer the same research question, potentially challenging the robustness of empirical science. In this crowd initiative, we investigated the degree to which research findings in the social and behavioural sciences are contingent on analysts’ choices. We examined a stratified random sample of 100 studies published between 2009 and 2018, in which, for one claim per study, at least five reanalysts independently reanalysed the original data. The statistical appropriateness of the reanalyses was assessed in peer evaluations, and the robustness indicators were inspected along a range of research characteristics and study designs. We found that 34% of the independent reanalyses yielded the same result (within a tolerance region of ±0.05 Cohen’s d) as the original report; with a four times broader tolerance region, this indicator increased to 57%. Of the reanalyses conducted, 74% reached the same conclusion as the original investigation, 24% yielded no effects or inconclusive results and 2% reported the opposite effect. This exploratory study indicates that the common single-path analyses in social and behavioural research should not be simply assumed to be robust to alternative analyses. Therefore, we recommend the development and use of practices to explore and communicate this neglected source of uncertainty.

Students' diverse levels of knowledge and competence—shaped by individual interests and educational debts, including structural, systemic, and institutional barriers—create substantial cognitive heterogeneity in instructional settings. Adequately addressing this heterogeneity is challenging. Emerging studies applying artificial intelligence (AI) in education claim that advanced AI techniques like machine learning (ML) can mitigate educational debts by providing adaptive support. However, previous research offers limited clarity on how learning outcomes vary following AI-based adaptive instruction and which students improve their learning outcomes. To address these issues, this article quantitatively examines the extent to which ML-based adaptivity influences students' learning outcomes over time and identifies which students, considering their intersectional identities, benefit most from this adaptive support. Specifically, we illustrate a semester-long study conducted within an undergraduate organic chemistry course, where an ML model adaptively supported 266 students across four interventions on mechanistic reasoning. We identified five learning trajectories throughout these adaptive interventions. Our findings show that students with higher prior knowledge made greater progress than their peers. Additionally, men without an underrepresented minority (URM) status majoring in chemistry benefited more than URM women who are not chemistry majors. This indicates that the adaptive support maintained and partly exacerbated educational debts. Our study contributes to the literature by analyzing how ML-based adaptivity affects educational debts in undergraduate organic chemistry. In doing so, it adopts a theoretical framework—the enhanced educational debt framework—to assess when different aims of adaptive support are most appropriate in undergraduate education, informing an equity-centered design of adaptive support.

Recent research underscores the importance of inquiry learning for effective science education. Inquiry learning involves self-regulated learning (SRL), for example when students conduct investigations. Teachers face challenges in orchestrating and tracking student learning in such instruction; making it hard to adequately support students. Using AI methods such as machine learning (ML), the data that is generated when students interact in technology-enhanced classrooms can be used to track their learning and subsequently to inform teachers so that they can better support student learning. This study implemented digital workbooks in an inquiry-based physics unit, collecting cognitive, metacognitive, and affective data from 214 students. Using ML methods, an early warning system was developed to predict students’ learning outcomes. Explainable ML methods were used to unpack these predictions and analyses were conducted for potential biases. Results indicate that an integration of cognitive, metacognitive, and affective data can predict students’ productivity with an accuracy ranging from 60 to 100% as the unit progresses. Initially, affective and metacognitive variables dominate predictions, with cognitive variables becoming more significant later. Using only affective and metacognitive data, predictive accuracies ranged from 60 to 80% throughout. Bias was found to be highly dependent on the ML methods being used. The study highlights the potential of digital student workbooks to support SRL in inquiry-based science education, guiding future research and development to enhance instructional feedback and teacher insights into student engagement. Further, the study sheds new light on the data needed and the methodological challenges when using ML methods to investigate SRL processes in classrooms.

Energy is a central concept across the sciences and an important goal of science education is to support all students so that they develop a full understanding of the energy concept. However, given the abstract and complex nature of the energy concept, only a few students develop an understanding so that they can use energy ideas to make sense of phenomena. Research into energy learning progressions aims at developing models of learning about energy to guide instruction so that students can be best supported in developing competence and has provided a rich model of how students' understanding of energy develops over time. Being largely based on cross-section data, however, the extent to which this model can guide instruction is limited, especially concerning the continued learning of students about energy. To address this gap—the limited evidence regarding what supports students' continued learning about energy—it was investigated how holding non-normative ideas and the integratedness of students' energy knowledge affect students' continued learning about energy. Drawing on data from a 4-year longitudinal study covering Grades 6–9 on students' learning about energy, diagnostic classification models were used to characterize students' non-normative idea profiles and the integratedness of their knowledge and then related both to their continued learning. The results suggest no detrimental effects of holding non-normative ideas and strong positive effects of holding integrated knowledge for students' continued learning about energy. Implications for teaching and future research are discussed.

Machine learning (ML) has become commonplace in educational research and science education research, especially to support assessment efforts. Such applications of machine learning have shown their promise in replicating and scaling human-driven codes of students' work. Despite this promise, we and other scholars argue that machine learning has not yet achieved its transformational potential. We argue that this is because our field is currently lacking frameworks for supporting creative, principled, and critical endeavors to use machine learning in science education research. To offer considerations for science education researchers' use of ML, we present a framework, Distributing Epistemic Functions and Tasks (DEFT), that highlights the functions and tasks that pertain to generating knowledge that can be carried out by either trained researchers or machine learning algorithms. Such considerations are critical decisions that should occur alongside those about, for instance, the type of data or algorithm used. We apply this framework to two cases, one that exemplifies the cutting-edge use of machine learning in science education research and another that offers a wholly different means of using machine learning and human-driven inquiry together. We conclude with strategies for researchers to adopt machine learning and call for the field to rethink how we prepare science education researchers in an era of great advances in computational power and access to machine learning methods. 

Qualitative research methods have provided key insights in physics education research (PER) by drawing on non-numerical data such as text or video data. While different methods towards qualitative research exist, they share two essential steps: recognizing patterns in the data and interpreting these patterns. Although these methods have led to the development of rigorous theory, there are challenges: As such methods require a series of judgments by the analyst, they are difficult to validate and reproduce. Further, they are hard to scale so that they are unavailable to the analysis of large-scale data. In this way, important phenomena may remain inaccessible to qualitative analysis. Reacting to these challenges and leveraging the potential of emerging methods of artificial intelligence (AI) such as machine learning and natural language processing, sociologist Nelson has proposed the concept of computational grounded theory (CGT). CGT proceeds in a process of three consecutive steps: In the first step, one leverages the power of computational techniques, especially natural language processing and unsupervised machine learning techniques, for pattern detection in large datasets—those of a size and scope that may prohibit human-driven analysis from the outset. In the second step, one relies on the integrative and interpretative capabilities of human researchers to add quality and depth to the quantity and breadth of the first step. In the last step, one again uses computational techniques to test the extent to which the detected and refined patterns from the first and second step hold throughout the whole dataset under investigation. Interestingly, CGT does not aim at simply automating parts of the qualitative process by using AI, but rather aims at integrating AI into the human analyst’s workflow within a qualitative analysis. This leads to an analytical system that can do something that is quantitatively and qualitatively different from what a human or machine can do alone. In this way, CGT aims at addressing questions about validity, reproducibility, and scalability in qualitative research while preserving the theoretical sensitivity and unique inferencing capabilities of the human analyst. In this paper, we provide a primer on CGT, present how it can be used to investigate the physics problem-solving approaches of 𝑁=4⁢1⁢7 students based on textual data, and discuss CGT’s potentials and challenges in PER. In consequence, this paper can provide critical input to the discussion of how emerging AI technologies can provide new avenues in qualitative PER.

Students' motivation plays an important role in successful science learning. However, motivation is a complex construct. Theories of motivation suggests that students' motivation must be conceptualized as a motivational system with numerous components that interact in complex ways and influence metacognitive processes such as self-evaluation. This complexity is further increased because students' motivation and success in science learning influence each other as they develop over time. It is challenging to study the co-development of motivation and learning due to these complex interactions which can vary widely across individuals. Recently, person-centered approaches that capture students' motivational profiles, that is, the multiplicity of motivational factors as they co-occur in students, have been successfully used in educational psychology to better understand the complex interplay between the co-development of students' motivation and learning. We employed a person-centered approach to study how the motivational profiles, constructed from goal-orientation, self-efficacy, and engagement data of N = 401 middle school students developed over the course of a 10-week energy unit and how that development was related to students' learning. We identified four characteristic motivational profiles with varying temporal stability and found that students' learning over the course of the unit was best characterized by considering the type of students' motivational profiles and the transitions that occurred between them. We discuss implications for the design and implementation of interventions and future research into the complex interplay between motivation and learning. 

Aims. We investigate the suitability of α-viscosity in self-similar models for self-gravitating disks with a focus on active galactic nuclei (AGN) disks.

Methods. We use a self-similar approach to simplify the partial differential equations arising from the evolution equation, which are then solved using numerical standard procedures.

Results. We find a self-similar solution for the dynamical evolution of self-gravitating α-disks and derive the significant quantities. In the Keplerian part of the disk our model is consistent with standard stationary α-disk theory, and self-consistent throughout the self-gravitating regime. Positive accretion rates throughout the disk demand a high degree of self-gravitation. Combined with the temporal decline of the accretion rate and its low amount, the model prohibits the growth of large central masses.

Conclusions. α-viscosity cannot account for the evolution of the whole mass spectrum of super-massive black holes (SMBH) in AGN. However, considering the involved scales it seems suitable for modelling protoplanetary disks.