Change search
ReferencesLink to record
Permanent link

Direct link
The Importance of Integrating Clinical Relevance and Statistical Significance in the Assessment of Quality of Care - Illustrated Using the Swedish Stroke Register
Umeå University, Faculty of Social Sciences, Umeå School of Business and Economics (USBE), Statistics.
Umeå University, Faculty of Social Sciences, Umeå School of Business and Economics (USBE), Statistics. Department of Applied Mathematics, Computer Science and Statistics, Ghent University, Ghent, Belgium.
Umeå University, Faculty of Medicine, Department of Public Health and Clinical Medicine.
Show others and affiliations
2016 (English)In: PLoS ONE, ISSN 1932-6203, E-ISSN 1932-6203, Vol. 11, no 4, e0153082Article in journal (Refereed) Published
Abstract [en]

BACKGROUND: When profiling hospital performance, quality inicators are commonly evaluated through hospital-specific adjusted means with confidence intervals. When identifying deviations from a norm, large hospitals can have statistically significant results even for clinically irrelevant deviations while important deviations in small hospitals can remain undiscovered. We have used data from the Swedish Stroke Register (Riksstroke) to illustrate the properties of a benchmarking method that integrates considerations of both clinical relevance and level of statistical significance.

METHODS: The performance measure used was case-mix adjusted risk of death or dependency in activities of daily living within 3 months after stroke. A hospital was labeled as having outlying performance if its case-mix adjusted risk exceeded a benchmark value with a specified statistical confidence level. The benchmark was expressed relative to the population risk and should reflect the clinically relevant deviation that is to be detected. A simulation study based on Riksstroke patient data from 2008-2009 was performed to investigate the effect of the choice of the statistical confidence level and benchmark value on the diagnostic properties of the method.

RESULTS: Simulations were based on 18,309 patients in 76 hospitals. The widely used setting, comparing 95% confidence intervals to the national average, resulted in low sensitivity (0.252) and high specificity (0.991). There were large variations in sensitivity and specificity for different requirements of statistical confidence. Lowering statistical confidence improved sensitivity with a relatively smaller loss of specificity. Variations due to different benchmark values were smaller, especially for sensitivity. This allows the choice of a clinically relevant benchmark to be driven by clinical factors without major concerns about sufficiently reliable evidence.

CONCLUSIONS: The study emphasizes the importance of combining clinical relevance and level of statistical confidence when profiling hospital performance. To guide the decision process a web-based tool that gives ROC-curves for different scenarios is provided.

Place, publisher, year, edition, pages
2016. Vol. 11, no 4, e0153082
National Category
Probability Theory and Statistics
URN: urn:nbn:se:umu:diva-119030DOI: 10.1371/journal.pone.0153082ISI: 000373608000075PubMedID: 27054326OAI: diva2:917939
Available from: 2016-04-08 Created: 2016-04-08 Last updated: 2016-09-28Bibliographically approved
In thesis
1. Statistical methods for register based studies with applications to stroke
Open this publication in new window or tab >>Statistical methods for register based studies with applications to stroke
2016 (English)Doctoral thesis, comprehensive summary (Other academic)
Alternative title[sv]
Statistiska metoder för registerbaserade studier med tillämpningar på stroke
Abstract [en]

This thesis adds to the area of register based research, with a particular focus on health care quality and (in)equality. Contributions are made to the areas of hospital performance benchmarking, mediation analysis, and regression when the outcome variable is limited, with applications related to Riksstroke (the Swedish stroke register).

An important part of quality assurance is to identify, follow up, and understand the mechanisms of inequalities in outcome and/or care between different population groups. The first paper of the thesis uses Riksstroke data to investigate socioeconomic differences in survival during different time periods after stroke. The second paper focuses on differences in performance between hospitals, illustrating the diagnostic properties of a method for benchmarking hospital performance and highlighting the importance of balancing clinical relevance and the statistical evidence level used.

Understanding the mechanisms behind observed differences is a complicated but important issue. In mediation analysis the goal is to investigate the causal mechanisms behind an effect by decomposing it into direct and indirect components. Estimation of direct and indirect effects relies on untestable assumptions and a mediation analysis should be accompanied by an analysis of how sensitive the results are to violations of these assumptions. The third paper proposes a sensitivity analysis method for mediation analysis based on binary probit regression. This is then applied to a mediation study based on Riksstroke data.

Data registration is not always complete and sometimes data on a variable are unavailable above or below some value. This is referred to as censoring or truncation, depending on the extent to which data are missing. The final two papers of the thesis are concerned with the estimation of linear regression models for limited outcome variables. The fourth paper presents a software implementation of three semi-parametric estimators of truncated linear regression models. The fifth paper extends the sensitivity analysis method proposed in the third paper to continuous outcomes and mediators, and situations where the outcome is truncated or censored.

Place, publisher, year, edition, pages
Umeå: Umeå universitet, 2016. 30 p.
Statistical studies, ISSN 1100-8989 ; 49
Registers, quality of care, socioeconomic status, hospital performance, stroke, mediation, sensitivity analysis, truncation, censoring
National Category
Probability Theory and Statistics
Research subject
urn:nbn:se:umu:diva-125953 (URN)978-91-7601-553-7 (ISBN)
Public defence
2016-10-21, Hörsal E, Humanisthuset, Umeå, 09:30 (English)
Available from: 2016-09-30 Created: 2016-09-23 Last updated: 2016-09-29Bibliographically approved

Open Access in DiVA

fulltext(576 kB)18 downloads
File information
File name FULLTEXT01.pdfFile size 576 kBChecksum SHA-512
Type fulltextMimetype application/pdf

Other links

Publisher's full textPubMed

Search in DiVA

By author/editor
Lindmark, AnitaGlader, Eva-LottaEriksson, Marie
By organisation
StatisticsDepartment of Public Health and Clinical Medicine
In the same journal
Probability Theory and Statistics

Search outside of DiVA

GoogleGoogle Scholar
Total: 18 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Altmetric score

Total: 74 hits
ReferencesLink to record
Permanent link

Direct link