Umeå universitets logga

The aim of this study was to provide a framework to evaluate bibliometric indicators as decision support tools from a decision making perspective and to examine the information value of early career publication rate as a predictor of future productivity. We used ROC analysis to evaluate a bibliometric indicator as a tool for binary decision making. The dataset consisted of 451 early career researchers in the mathematical sub-field of number theory. We investigated the effect of three different definitions of top performance groups—top 10, top 25, and top 50 %; the consequences of using different thresholds in the prediction models; and the added prediction value of information on early career research collaboration and publications in prestige journals. We conclude that early career performance productivity has an information value in all tested decision scenarios, but future performance is more predictable if the definition of a high performance group is more exclusive. Estimated optimal decision thresholds using the Youden index indicated that the top 10 % decision scenario should use 7 articles, the top 25 % scenario should use 7 articles, and the top 50 % should use 5 articles to minimize prediction errors. A comparative analysis between the decision thresholds provided by the Youden index which take consequences into consideration and a method commonly used in evaluative bibliometrics which do not take consequences into consideration when determining decision thresholds, indicated that differences are trivial for the top 25 and the 50 % groups. However, a statistically significant difference between the methods was found for the top 10 % group. Information on early career collaboration and publication strategies did not add any prediction value to the bibliometric indicator publication rate in any of the models. The key contributions of this research is the focus on consequences in terms of prediction errors and the notion of transforming uncertainty into risk when we are choosing decision thresholds in bibliometricly informed decision making. The significance of our results are discussed from the point of view of a science policy and management.

The purpose of this study was to examine the relationship between publication rate, top journal publications and excellence during the first eight years of the career, and how well publication rate, top journal publications and highly cited publications during the first four years of the career can predict whether an author attain excellence in the fifth to the eighth year. The dataset consisted of publication track records of 406 early career mathematicians in the sub-field of number theory collected from the MathSciNet database. Logistic regression and dominance analysis was applied to the data. The major conclusions were (1) publication rate had a positive effect on excellence during the first eighth years of the career. However, those who publish many articles in top journals, which implicitly require a high publication count, had an even higher probability of attaining excellence. These results suggest that publishing in top journals is very important in the process of attaining excellence in the early career in addition to publishing many papers; and (2) a dominance analysis indicated that the number of top journal publications and highly cited publications during the first four years of the career were the most important predictors of who will attain excellence in the later career. The results are discussed in relation to indicator development and science policy.

Publishing in peer-reviewed journals as a part of the doctoral education is common practice in many countries. The publication output of doctoral students is increasingly used in selection processes for funding and employment in their early careers. Against the backdrop of this development, the aim of this study is to examine (1) how performance during the doctoral education affect the probability of attaining research excellence in the early career; and (2) if there is performance differences between males and females in the early career and to which degree these gender differences can be explained by performance differences during the doctoral education. The data consist of Swedish doctoral students employed at the faculty of science and technology and the faculty of medicine at a Swedish university. Our main conclusions are that (1) research performance during the doctoral education has a positive effect on attaining excellence in the early career; (2) there is an interaction between publication volume and excellence during doctoral education suggesting that a combination of quantity and quality in doctoral students’ performance is indicative of future excellence; (3) there are performance differences in the early career indicating that males have a higher probability of attaining excellence than females, and; (4) this difference is partly explained by performance differences during the doctoral education.