Umeå University's logo

umu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
The information value of early career productivity in mathematics: a ROC analysis of prediction errors in bibliometricly informed decision making
Umeå University, Faculty of Social Sciences, Department of Sociology. (Inforsk)ORCID iD: 0000-0003-3623-2471
Umeå University, Faculty of Social Sciences, Department of Sociology. (Inforsk)
2016 (English)In: Scientometrics, ISSN 0138-9130, E-ISSN 1588-2861, Vol. 109, no 3, p. 2241-2262Article in journal (Refereed) Published
Abstract [en]

The aim of this study was to provide a framework to evaluate bibliometric indicators as decision support tools from a decision making perspective and to examine the information value of early career publication rate as a predictor of future productivity. We used ROC analysis to evaluate a bibliometric indicator as a tool for binary decision making. The dataset consisted of 451 early career researchers in the mathematical sub-field of number theory. We investigated the effect of three different definitions of top performance groups—top 10, top 25, and top 50 %; the consequences of using different thresholds in the prediction models; and the added prediction value of information on early career research collaboration and publications in prestige journals. We conclude that early career performance productivity has an information value in all tested decision scenarios, but future performance is more predictable if the definition of a high performance group is more exclusive. Estimated optimal decision thresholds using the Youden index indicated that the top 10 % decision scenario should use 7 articles, the top 25 % scenario should use 7 articles, and the top 50 % should use 5 articles to minimize prediction errors. A comparative analysis between the decision thresholds provided by the Youden index which take consequences into consideration and a method commonly used in evaluative bibliometrics which do not take consequences into consideration when determining decision thresholds, indicated that differences are trivial for the top 25 and the 50 % groups. However, a statistically significant difference between the methods was found for the top 10 % group. Information on early career collaboration and publication strategies did not add any prediction value to the bibliometric indicator publication rate in any of the models. The key contributions of this research is the focus on consequences in terms of prediction errors and the notion of transforming uncertainty into risk when we are choosing decision thresholds in bibliometricly informed decision making. The significance of our results are discussed from the point of view of a science policy and management.

Place, publisher, year, edition, pages
Springer Netherlands, 2016. Vol. 109, no 3, p. 2241-2262
Keywords [en]
Receiver operating characteristics, ROC, Performance, Bibliometric indicator, Prediction errors, Decision making, Productivity, Mathematics
National Category
Information Studies
Research subject
library and information science
Identifiers
URN: urn:nbn:se:umu:diva-128995DOI: 10.1007/s11192-016-2097-9ISI: 000389336100046Scopus ID: 2-s2.0-84981188303OAI: oai:DiVA.org:umu-128995DiVA, id: diva2:1058211
Conference
20th International Conference on Science and Technology Indicators (STI), Lugano, SWITZERLAND, SEP 02-09, 2015
Available from: 2016-12-20 Created: 2016-12-20 Last updated: 2023-07-04Bibliographically approved
In thesis
1. In search of future excellence: bibliometric indicators, gender differences, and predicting research performance in the early career
Open this publication in new window or tab >>In search of future excellence: bibliometric indicators, gender differences, and predicting research performance in the early career
2020 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

The governance of higher education institutions and science have endured significant changes during the last decades, emphasizing competitiveness, performance, and excellence. Embedded in this development is an increased use of bibliometric indicators as decision support tools in contexts of e.g., employment, appointment, and funding. These changes have gradually extended to the early career phase and the doctoral education.

The aim of this thesis is to make a contribution to an ongoing discussion about the predictability of research performance and the reasonability of using bibliometric indicators in the early career, with a focus on gender differences. The thesis revolves around three overarching research questions focusing the early career and the doctoral education: (1) the degree to which research performance, as operationalized with bibliometric indicators, is predictable; (2) the degree to which gender differences in early career performance can be explained by research performance during the doctoral education; and (3) to what degree factors such as collaboration and supervisor behaviour, might affect gender differences in research performance.

The main results suggests that research performance in the early career, as operationalized by bibliometric indicators, is predictable. Individuals who publish larger volumes, publish more in high prestige journals, and more excellent research early in their career, are more likely to attain excellence later on. The results also indicates that gender differences in performance can be observed as early asduring doctor education and that these differences partly explain the observed performance differences between males and females in the early career.

Finally, the results suggests that gender differences in performance during doctoral education can largely be explained by the doctoral student’s collaborative networks and supervisor behaviour. It is concluded that while research performance, as operationalized by bibliometric indicators, duringthe early career is predictable, there are gender differences in performance that have to be taken into consideration. If they are not, the use of these types of performance indicators in science policy and management might increase the gender gap in science.

Place, publisher, year, edition, pages
Umeå: Umeå universitet, 2020. p. 60
Series
Akademiska avhandlingar vid Sociologiska institutionen, Umeå universitet, ISSN 1104-2508 ; 84
Keywords
bibliometric indicator, gender, early career, doctoral education, excellence, decision support tool, prediction, research performance, doctoral student
National Category
Information Studies Sociology Educational Sciences
Research subject
library and information science
Identifiers
urn:nbn:se:umu:diva-167701 (URN)978-91-7855-208-5 (ISBN)978-91-7855-209-2 (ISBN)
Public defence
2020-02-28, Hörsal N360, Naturvetarhuset, Umeå, 10:00 (English)
Opponent
Supervisors
Available from: 2020-02-07 Created: 2020-02-01 Last updated: 2020-06-12Bibliographically approved

Open Access in DiVA

fulltext(654 kB)385 downloads
File information
File name FULLTEXT01.pdfFile size 654 kBChecksum SHA-512
0f5b8ebe3e43f9856ce50b695ce1f20e1aeb2d9b44496b02a9a4d2ab8c9734fb3d283682c0888b932e2d0508632452b9b14668904d2d35ee5c236fe4181b9247
Type fulltextMimetype application/pdf

Other links

Publisher's full textScopus

Authority records

Lindahl, JonasDanell, Rickard

Search in DiVA

By author/editor
Lindahl, JonasDanell, Rickard
By organisation
Department of Sociology
In the same journal
Scientometrics
Information Studies

Search outside of DiVA

GoogleGoogle Scholar
Total: 385 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 777 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf