Research quality evaluation: comparing citation counts considering bibliometric database errors

详细信息    查看全文

• 作者：Fiorenzo Franceschini (1)
  Domenico Maisano (1)
  Luca Mastrogiacomo (1)
  
  1. Department of Management and Production Engineering (DIGEP) ; Politecnico di Torino ; Corso Duca degli Abruzzi 24 ; 10129聽 ; Turin ; Italy
  
• 关键词：Database quality ; Database error ; Citation count ; Omitted citations ; Pair ; wise comparison
• 刊名：Quality & Quantity
• 出版年：2015
• 出版时间：January 2015
• 年：2015
• 卷：49
• 期：1
• 页码：155-165
• 全文大小：415 KB
• 参考文献：1. Abt, H.A.: What fraction of literature references are incorrect? Publ. Astron. Soc. Pac. 104, 235鈥?36 (1992) CrossRef
  2. Adam, D.: Citation analysis: the counting house. Nature 415(6873), 726鈥?29 (2002) CrossRef
  3. Bornmann, L., Mutz, R., Neuhaus, C., Daniel, H.D.: Citation counts for research evaluation: standards of good practice for analyzing bibliometric data and presenting and interpreting results. Ethics Sci. Environ. Polit. 8(1), 93鈥?02 (2008) CrossRef
  4. Box, G.E.P., Hunter, W.G., Hunter, J.S.: Statistics for Experimenters. An Introduction to Design, Data Analysis, and Model Building. Wiley, New York (1978)
  5. Buchanan, R.A.: Accuracy of cited references: the role of citation databases. Coll. Res. Libr. 67(4), 292鈥?03 (2006) CrossRef
  6. Casser, D., Husson, F.: A 2-dimensional extension of the Bradley鈥揟erry model for paired comparisons. J. Stat. Plan. Inference 135(2), 245鈥?59 (2005) CrossRef
  7. Dalrymple, J., Edgeman, R.L., Finster, M., Guerrero-Cusumano, J., Hensler, D.A., Parr, W.C.: A white paper: quality at the crossroads of organizational excellence and the academy. Qual. Eng. 12(1), 97鈥?04 (1999) CrossRef
  8. Franceschini, F., Galetto, M., Maisano, D., Mastrogiacomo, L.: The success-index: an alternative approach to the h-index for evaluating an individual鈥檚 research output. Scientometrics 92(3), 621鈥?41 (2012) CrossRef
  9. Franceschini, F., Maisano, D.: Analysis of the Hirsch index鈥檚 operational properties. Eur. J. Oper. Res. 203(2), 494鈥?04 (2010) CrossRef
  10. Franceschini, F., Maisano, D., Mastrogiacomo, L.: A novel approach for estimating the omitted-citation rate of bibliometric databases. J. Am. Soc. Inf. Sci. Technol. 64(10), 2149鈥?156 (2013). doi:10.1002/asi.22898 CrossRef
  11. Guilera, G., Gomez-Benito, J., Hidalgo, M.D.: Citation analysis in research on differential item functioning. Qual. Quant. 44, 1249鈥?255 (2010) CrossRef
  12. Jacs贸, P.: Deflated, inflated and phantom citation counts. Online Inf. Rev. 30(3), 297鈥?09 (2006) CrossRef
  13. Jacs贸, P.: Grim Tales about the impact factor and the h-index in the Web of Science and the Journal Citation Reports databases: reflections on Vanclay鈥檚 criticism. Scientometrics 92(2), 325鈥?54 (2012) CrossRef
  14. Jeong, S., Lee, S., Kim, H.G.: Are you an invited speaker? A bibliometric analysis of elite groups for scholarly events in bioinformatics. J. Am. Soc. Inf. Sci. Technol. 60(6), 1118鈥?131 (2009) CrossRef
  15. Kim, W., Choi, B.J., Hong, E.K., Kim, S.K., Lee, D.: A taxonomy of dirty data. Data Min. Knowl. Discov. 7(1), 81鈥?9 (2003) CrossRef
  16. Li, J., Burnham, J.F., Lemley, T., Britton, R.M.: Citation analysis: comparison of Web of Science, Scopus, Scifinder, and Google Scholar. J. Electron. Resour. Med. Libr. 7(3), 196鈥?17 (2010) CrossRef
  17. MIUR: Decreto Ministeriale 7 giugno 2012 n. 76. http://attiministeriali.miur.it/anno-2012/giugno/dm-07062012.aspx (2012). Accessed 1 Oct 2013
  18. Montgomery, D.C.: Statistical Quality Control: A Modern Introduction, 6th edn. Wiley, New York (2009)
  19. Skellam, J.: The frequency distribution of the difference between two Poisson variates belonging to different populations. J. R. Stat. Soc. Ser. A (Gen.) 109(3), 296 (1946) CrossRef
  20. Sweetland, J.H.: Errors in bibliographic citations: a continuing problem. Libr. Q. 59(4), 291鈥?04 (1989) CrossRef
  
• 刊物类别：Humanities, Social Sciences and Law
• 刊物主题：Social Sciences
  Methodology of the Social Sciences
  Social Sciences
  
• 出版者：Springer Netherlands
• ISSN：1573-7845

文摘

When evaluating the research output of scientists, institutions or journals, different portfolios of publications are usually compared with each other. e.g., a typical problem is to select, between two scientists of interest, the one with the most cited portfolio. The total number of received citations is a very popular indicator, generally obtained by bibliometric databases. However, databases are not free from errors, which may affect the result of evaluations and comparisons; among these errors, one of the most significant is that of omitted citations. This paper presents a methodology for the pair-wise comparison of publication portfolios, which takes into account the database quality regarding omitted citations. In particular, it is defined a test for establishing if a citation count is (or not) significantly higher than one other. A statistical model for estimating the type-I error related to this test is also developed.