Umeå University's logo

BACKGROUND: Frailty is common after stroke and linked to poor outcomes, but many measures are clinician-rated, time-consuming, and not suited to patient-reported data. To address these issues, we developed and validated a frailty score from the Swedish Stroke Register (Riksstroke) three-month follow-up questionnaire.

METHODS: We analyzed responses from 19,470 stroke survivors to nine patient-reported items covering function, mood, fatigue, pain and general health, in the 2021-2022 Riksstroke questionnaire. Dimensionality was assessed with Mokken Scale Analysis and exploratory factor analysis. Item response theory (IRT) was used for score computation. Competing graded response IRT models (unidimensional, correlated-factor, bifactor) were compared, and measurement fairness was examined using differential item functioning (DIF) across age, sex, and education. Prognostic validity was tested with Kaplan-Meier curves and Cox regression for all-cause mortality.

RESULTS: From the Mokken Scale Analysis, all items met scalability criteria. Factor analysis suggested two correlated interpretable facets (Physical Functioning; Well-being/Mental Health). A bifactor IRT model provided the best fit to the data, comprising a general frailty dimension while addressing the strong correlation between the facets. DIF was minimal for sex and education, with modest age-related effects. Higher frailty scores were associated with increased mortality in adjusted Cox models and Kaplan-Meier curves. Tools for computing frailty scores are available at https://github.com/joakimwallmark/frailty-irt-scores.

CONCLUSIONS: A robust, fair, and prognostically meaningful frailty score can be derived from patient-reported items in Riksstroke. More broadly, the study demonstrates how routinely collected patient-reported outcome measures can be leveraged to build scalable frailty scores, offering efficient cost-effective tools for monitoring outcome and guiding quality improvement in stroke care.

In test equating, ensuring score comparability across different test forms is crucial but particularly challenging when test groups are non-equivalent and no anchor test is available. Local test equating aims to satisfy Lord’s equity requirement by conditioning equating transformations on individual-level information, typically using anchor test scores as proxies for latent ability. However, anchor tests are not always available in practice. This paper introduces two novel propensity score-based methods for local equating: stratification and inverse probability weighting (IPW). These methods use covariates to account for group differences, with propensity scores serving as proxies for latent ability differences between test groups. The stratification method partitions examinees into comparable groups based on similar propensity scores, while IPW assigns weights inversely proportional to the probability of group membership. We evaluate these methods through empirical analysis and simulation studies. Results indicate both methods can effectively adjust for group differences, with their relative performance depending on the strength of covariate-ability correlations. The study extends local equating methodology to cases where only covariate information is available, providing testing programs with new tools for ensuring fair score comparability.

Background: Compassion contributes to wellbeing and serves as a protective factor against mental health problems in young people. The Compassionate Engagement and Action Scale for Youth—Swedish version (CEASY-SE) is a self-report questionnaire that assesses compassion across six competencies, organized into three scales: Self-Compassion, Compassion for Others, and Compassion from Others. While previously validated in a Swedish school sample, its applicability in clinical psychiatric populations remains unexplored. This study aimed to evaluate the psychometric properties of CEASY-SE in a clinical psychiatric sample of Swedish youth aged 16–22.

Methods: A cross-sectional study was conducted, collecting self-reported data from young people (N = 355) receiving care in child and adolescent psychiatry and primary care. We assessed the CEASY-SE's factor structure, reliability, and validity (internal consistency, convergent, divergent, construct, and criterion-related validity). Sex and age differences were also analyzed, along with comparisons of total scores across diagnostic groups.

Results: Confirmatory factor analyses supported a two-factor model within each of the three scales. Internal consistency was good to excellent across all scales (α ranging from 0.75 to 0.92), except for the Self-Compassion Engagement subscale among males (α = 0.62; ω = 0.60). Convergent and divergent validity were satisfactory. Among the three CEASY-SE scales, Self-Compassion showed the strongest correlations with mental health outcomes. Sex differences primarily affected Compassion for Others, and a sex-by-age interaction was found for both Compassion for Others and Self-Compassion. A total score of 40 was associated with an 8.41-fold increase in the predicted probability of depression symptoms compared to a score of 100. The lowest scores were found among patients with eating disorders and depression.

Conclusions: The CEASY-SE demonstrates acceptable to excellent psychometric properties in a clinical psychiatric sample of youth. It is a promising tool for clinicians and researchers to assess and promote compassion in young people, with potential relevance for interventions aimed at improving mental health outcomes.

Modifications of current psychometric models for analyzing test data are proposed that produce an additive scale measure of information. This information measure is a one-dimensional space curve or curved surface manifold that is invariant across varying manifold indexing systems. The arc length along a curve manifold is used as it is an additive metric having a defined zero and a version of the bit as a unit. This property, referred to here as the scope of the test or an item, facilitates the evaluation of graphs and numerical summaries. The measurement power of the test is defined by the length of the manifold, and the performance or experiential level of a person by a position along the curve. In this study, we also use all information from the items including the information from the distractors. Test data from a large-scale college admissions test are used to illustrate the test information manifold perspective and to compare it with the well-known item response theory nominal model. It is illustrated that the use of information theory opens a vista of new ways of assessing item performance and inter-item dependency, as well as test takers' knowledge.

Test score equating is used to make scores from different test forms comparable, even when groups differ in ability. In practice, the non-equivalent group with anchor test (NEAT) design is commonly used. The overall aim was to compare the amount of bias under different conditions when using either chained equating or frequency estimation with five different criterion functions: the identity function, linear equating, equipercentile, chained equating and frequency estimation. We used real test data from a multiple-choice binary scored college admissions test to illustrate that the choice of criterion function matter. Further, we simulated data in line with the empirical data to examine difference in ability between groups, difference in item difficulty, difference in anchor test form and regular test form length, difference in correlations between anchor test form and regular test forms, and different sample size. The results indicate that how bias is defined heavily affects the conclusions we draw about which equating method is to be preferred in different scenarios. Practical implications of this in standardized tests are given together with recommendations on how to calculate bias when evaluating equating transformations.

Ensuring that test scores are fair and comparable across different test forms and different test groups is a significant statistical challenge in educational testing. Methods to achieve score comparability, a process known as test score equating, often rely on including common test items or assuming that test taker groups are similar in key characteristics. This study explores a novel approach that combines propensity scores, based on test takers' background covariates, with information from common items using kernel smoothing techniques for binary-scored test items. An empirical analysis using data from a high-stakes college admissions test evaluates the standard errors and differences in adjusted test scores. A simulation study examines the impact of factors such as the number of test takers, the number of common items, and the correlation between covariates and test scores on the method’s performance. The findings demonstrate that integrating propensity scores with common item information reduces standard errors and bias more effectively than using either source alone. This suggests that balancing the groups on the test-takers' covariates enhance the fairness and accuracy of test score comparisons across different groups. The proposed method highlights the benefits of considering all the collected data to improve score comparability.

Generalized Kernel Equating is a comprehensive guide for statisticians, psychometricians, and educational researchers aiming to master test score equating. This book introduces the Generalized Kernel Equating (GKE) framework, providing the necessary tools and methodologies for accurate and fair score comparisons.

The book presents test score equating as a statistical problem and covers all commonly used data collection designs. It details the five steps of the GKE framework: presmoothing, estimating score probabilities, continuization, equating transformation, and evaluating the equating transformation. Various presmoothing strategies are explored, including log-linear models, item response theory models, beta4 models, and discrete kernel estimators. The estimation of score probabilities when using IRT models is described and Gaussian kernel continuization is extended to other kernels such as uniform, logistic, epanechnikov and adaptive kernels. Several bandwidth selection methods are described. The kernel equating transformation and variants of it are defined, and both equating-specific and statistical measures for evaluating equating transformations are included. Real data examples, guiding readers through the GKE steps with detailed R code and explanations are provided. Readers are equipped with an advanced knowledge and practical skills for implementing test score equating methods.

This study aimed to empirically compare the effectiveness of Likert, Thurstonian, and Expected a Posteriori (EAP) scoring methods. A computational simulation of the two-parameter logistic model was employed under various conditions, including different sample sizes, number of items, scale levels, item extremeness, and varying discriminations. Effectiveness was assessed through correlation with the true score, root mean squared errors, bias, and the accurate recovery of effect sizes. The results indicated that Likert scores exhibit greater bias than EAP and Thurstonian scores for extreme scores, however, they strongly correlate with both the true score and EAP scores. Likert scores were slightly more effective in recovering mean differences between two groups, correlation estimates, and regression parameters. Overall, Likert scores should be avoided when ordering or thresholding individuals at the extremes of scales is the primary objective. However, they are preferable in situations where Thurstonian and EAP scores may fail to converge. The study also recommends that future research explore conditions involving more complex data-generating processes.

O objetivo deste artigo foi comparar empiricamente a eficácia dos métodos de escores de Likert, Thurstoniano e Esperado a Posteriori (EAP). Foi realizada uma simulação computacional de um modelo logístico de dois parâmetros em diversas condições (diferentes tamanhos amostrais, número de itens, níveis de escala, itens extremos e discriminações variadas). A eficácia foi medida pela correlação com o escore verdadeiro, pelo erro quadrático médio, viés e pela recuperação correta dos tamanhos de efeito. Os resultados mostraram que os escores de Likert são mais enviesados do que os escores EAP e Thurstoniano para escores extremos, embora apresentem correlação forte tanto com o escore verdadeiro quanto com os escores EAP. Os escores de Likert também se mostraram ligeiramente mais eficientes na recuperação de diferenças médias entre dois grupos, estimativas de correlação e parâmetros de regressão. De modo geral, os escores de Likert devem ser evitados quando a ordenação ou a definição de limites dos indivíduos nas extremidades das escalas for o principal interesse. No entanto, eles devem ser preferidos sempre que os escores Thurstoniano e EAP possam não convergir. Recomendamos também que estudos futuros testem condições com processos geradores de dados mais complexos.

El objetivo de este artículo fue comparar empíricamente la efectividad de los métodos de medición Likert, Thurstoniano y Esperado a Posteriori (EAP). Se realizó una simulación computacional de un modelo logístico de dos parámetros bajo diversas condiciones (diferentes tamaños de muestra, número de ítems, niveles de escala, ítems extremos y discriminaciones variadas). La efectividad se midió mediante la correlación con la puntuación verdadera, el error cuadrático medio, el sesgo y la correcta recuperación de los tamaños de efecto. Los resultados mostraron que las puntuaciones de Likert son más sesgadas que las puntuaciones EAP y Thurstonianas para puntuaciones extremas, aunque presentan una fuerte correlación tanto con la puntuación verdadera como con las puntuaciones EAP. Las puntuaciones de Likert también se mostraron ligeramente más eficientes en la recuperación de diferencias medias entre dos grupos, estimaciones de correlación y parámetros de regresión. En general, las puntuaciones de Likert deben evitarse cuando la ordenación o el establecimiento de límites de los individuos en los extremos de las escalas sea el principal interés. Sin embargo, deben preferirse siempre que las puntuaciones Thurstonianas y EAP puedan no converger. También recomendamos que estudios futuros prueben condiciones con procesos generadores de datos más complejos.

Observed-score test equating is a vital part of every testing program, aiming to make test scores across test administrations comparable. Central to this process is the equating function, typically estimated by composing distribution functions of the scores to be equated. An integral part of this estimation is presmoothing, where statistical models are fit to observed score frequencies to mitigate sampling variability. This study evaluates the impact of commonly used model fit indices on bivariate presmoothing model-selection accuracy in both item response theory (IRT) and non-IRT settings. It also introduces a new model-selection criterion that directly targets the equating function in contrast to existing methods. The study focuses on the framework of non-equivalent groups with anchor test design, estimating bivariate score distributions based on real and simulated data. Results show that the choice of presmoothing model and model fit criterion influences the equated scores. In non-IRT contexts, a combination of the proposed model-selection criterion and the Bayesian information criterion exhibited superior performance, balancing bias, and variance of the equated scores. For IRT models, high selection accuracy and minimal equating error were achieved across all scenarios.

In Item Response Theory (IRT), the conventional latent trait scale (θ) is inherently arbitrary, lacking a fixed unit or origin and often tied to specific population distributional assumptions (e.g., standard normal). This limits the direct comparability and interpretability of scores across different tests, populations, or model estimation methods. This paper introduces the “bit scale,” a novel metric transformation for unidimensional IRT scores derived from fundamental principles of information theory, specifically surprisal and entropy. Bit scores are anchored to the properties of the test items rather than the test-taker population. This item-based anchoring ensures the scale’s invariance to population assumptions and provides a consistent metric for comparing latent trait levels. We illustrate the utility of the bit scale through empirical examples: demonstrating consistent scoring when fitting models with different θ scale assumptions, and using anchor items to directly link scores from different test administrations. A simulation study confirms the desirable statistical properties (low bias, accurate standard errors) of Maximum Likelihood estimated bit scores and their robustness to extreme scores. The bit scale offers a theoretically grounded, interpretable, and comparable metric for reporting and analyzing IRT-based assessment results. Software implementations in R (bitscale) and Python (IRTorch) are available and practical implications are discussed.