umu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
The information value of early career productivity in mathematics: a ROC analysis of prediction errors in bibliometricly informed decision making
Umeå University, Faculty of Social Sciences, Department of Sociology. (Inforsk)
Umeå University, Faculty of Social Sciences, Department of Sociology. (Inforsk)
2016 (English)In: Scientometrics, ISSN 0138-9130, E-ISSN 1588-2861, Vol. 109, no 3, 2241-2262 p.Article in journal (Refereed) Published
Abstract [en]

The aim of this study was to provide a framework to evaluate bibliometric indicators as decision support tools from a decision making perspective and to examine the information value of early career publication rate as a predictor of future productivity. We used ROC analysis to evaluate a bibliometric indicator as a tool for binary decision making. The dataset consisted of 451 early career researchers in the mathematical sub-field of number theory. We investigated the effect of three different definitions of top performance groups—top 10, top 25, and top 50 %; the consequences of using different thresholds in the prediction models; and the added prediction value of information on early career research collaboration and publications in prestige journals. We conclude that early career performance productivity has an information value in all tested decision scenarios, but future performance is more predictable if the definition of a high performance group is more exclusive. Estimated optimal decision thresholds using the Youden index indicated that the top 10 % decision scenario should use 7 articles, the top 25 % scenario should use 7 articles, and the top 50 % should use 5 articles to minimize prediction errors. A comparative analysis between the decision thresholds provided by the Youden index which take consequences into consideration and a method commonly used in evaluative bibliometrics which do not take consequences into consideration when determining decision thresholds, indicated that differences are trivial for the top 25 and the 50 % groups. However, a statistically significant difference between the methods was found for the top 10 % group. Information on early career collaboration and publication strategies did not add any prediction value to the bibliometric indicator publication rate in any of the models. The key contributions of this research is the focus on consequences in terms of prediction errors and the notion of transforming uncertainty into risk when we are choosing decision thresholds in bibliometricly informed decision making. The significance of our results are discussed from the point of view of a science policy and management.

Place, publisher, year, edition, pages
Springer Netherlands, 2016. Vol. 109, no 3, 2241-2262 p.
Keyword [en]
Receiver operating characteristics, ROC, Performance, Bibliometric indicator, Prediction errors, Decision making, Productivity, Mathematics
National Category
Information Studies
Research subject
biblioteks- och informationsvetenskap
Identifiers
URN: urn:nbn:se:umu:diva-128995DOI: 10.1007/s11192-016-2097-9ISI: 000389336100046OAI: oai:DiVA.org:umu-128995DiVA: diva2:1058211
Conference
20th International Conference on Science and Technology Indicators (STI), Lugano, SWITZERLAND, SEP 02-09, 2015
Available from: 2016-12-20 Created: 2016-12-20 Last updated: 2017-01-10Bibliographically approved

Open Access in DiVA

fulltext(654 kB)28 downloads
File information
File name FULLTEXT01.pdfFile size 654 kBChecksum SHA-512
0f5b8ebe3e43f9856ce50b695ce1f20e1aeb2d9b44496b02a9a4d2ab8c9734fb3d283682c0888b932e2d0508632452b9b14668904d2d35ee5c236fe4181b9247
Type fulltextMimetype application/pdf

Other links

Publisher's full text

Search in DiVA

By author/editor
Lindahl, JonasDanell, Rickard
By organisation
Department of Sociology
In the same journal
Scientometrics
Information Studies

Search outside of DiVA

GoogleGoogle Scholar
Total: 28 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Altmetric score

Total: 40 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf