Umeå University's logo

umu.sePublikasjoner
Endre søk
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
The reliability of replications: a study in computational reproductions
Organization and Program Planning, German Institute for Adult Education, Leibniz Center of Lifelong Learning, Bonn, Germany.
School of Politics and International Studies, University of Leeds, Leeds, United Kingdom.
Geschwister Scholl Institute, LMU Munich, Munich, Germany.
Department of Sociology, University of Maryland, College Park, United States.
Vise andre og tillknytning
2025 (engelsk)Inngår i: Royal Society Open Science, E-ISSN 2054-5703, Vol. 12, nr 3, artikkel-id 241038Artikkel i tidsskrift (Fagfellevurdert) Published
Abstract [en]

This study investigates researcher variability in computational reproduction, an activity for which it is least expected. Eighty-five independent teams attempted numerical replication of results from an original study of policy preferences and immigration. Reproduction teams were randomly grouped into a 'transparent group' receiving original study and code or 'opaque group' receiving only a method and results description and no code. The transparent group mostly verified original results (95.7% same sign and p-value cutoff), while the opaque group had less success (89.3%). Second-decimal place exact numerical reproductions were less common (76.9 and 48.1%). Qualitative investigation of the workflows revealed many causes of error, including mistakes and procedural variations. When curating mistakes, we still find that only the transparent group was reliably successful. Our findings imply a need for transparency, but also more. Institutional checks and less subjective difficulty for researchers 'doing reproduction' would help, implying a need for better training. We also urge increased awareness of complexity in the research process and in 'push button' replications.

sted, utgiver, år, opplag, sider
Royal Society, 2025. Vol. 12, nr 3, artikkel-id 241038
Emneord [en]
computational reproduction, reliability, replications, social and behavioural sciences
HSV kategori
Identifikatorer
URN: urn:nbn:se:umu:diva-237136DOI: 10.1098/rsos.241038Scopus ID: 2-s2.0-105001207223OAI: oai:DiVA.org:umu-237136DiVA, id: diva2:1952452
Forskningsfinansiär
German Research Foundation (DFG), 464546557Tilgjengelig fra: 2025-04-15 Laget: 2025-04-15 Sist oppdatert: 2025-05-16bibliografisk kontrollert

Open Access i DiVA

fulltext(1074 kB)64 nedlastinger
Filinformasjon
Fil FULLTEXT01.pdfFilstørrelse 1074 kBChecksum SHA-512
57eec97caa746571d8f7457fdc70cb7ff6fefe650e7c8b647c395769a74fd6b9bf580b1962a9bf132a77af2ee9a986da86170fd81edbd8f099eb137b3647c50a
Type fulltextMimetype application/pdf

Andre lenker

Forlagets fulltekstScopus

Person

Bohman, AndreaEger, Maureen A.Hjerm, Mikael

Søk i DiVA

Av forfatter/redaktør
Bohman, AndreaEger, Maureen A.Hjerm, Mikael
Av organisasjonen
I samme tidsskrift
Royal Society Open Science

Søk utenfor DiVA

GoogleGoogle Scholar
Totalt: 73 nedlastinger
Antall nedlastinger er summen av alle nedlastinger av alle fulltekster. Det kan for eksempel være tidligere versjoner som er ikke lenger tilgjengelige

doi
urn-nbn

Altmetric

doi
urn-nbn
Totalt: 723 treff
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf