The Costs of Competition in Distributing Scarce Research Funds (2403.16934v1)
Abstract: Research funding systems are not isolated systems - they are embedded in a larger scientific system with an enormous influence on the system. This paper aims to analyze the allocation of competitive research funding from different perspectives: How reliable are decision processes for funding? What are the economic costs of competitive funding? How does competition for funds affect doing risky research? How do competitive funding environments affect scientists themselves, and which ethical issues must be considered? We attempt to identify gaps in our knowledge of research funding systems; we propose recommendations for policymakers and funding agencies, including empirical experiments of decision processes and the collection of data on these processes. With our recommendations we hope to contribute to developing improved ways of organizing research funding.
- OECD “Main science and technology indicators”, 2023 DOI: https://doi.org/10.1787/1cdcb031-en
- John P.A. Ioannidis “Fund people not projects” In Nature, 2011 DOI: https://doi.org/10.1038/477529a
- Philip Kitcher “Science, truth, and democracy” Oxford University Press, 2001
- Jamie Shaw “Peer review in funding-by-lottery: A systematic overview and expansion” In Research Evaluation 32.1, 2023, pp. 86–100 DOI: 10.1093/reseval/rvac022
- Ferric C Fang, Anthony Bowen and Arturo Casadevall “NIH peer review percentile scores are poorly predictive of grant productivity” Publisher: eLife Sciences Publications, Ltd In Elife 5, 2016, pp. e13323 DOI: 10.7554/eLife.13323
- Jamie Shaw “Bias, lotteries, and affirmative action in science funding policy” In The British Journal for the Philosophy of Science, 2024 DOI: https://doi.org/10.1086/730218
- “An efficient system to fund science: from proposal review to peer-to-peer distributions” In Scientometrics 110, 2017, pp. 521–528 DOI: 10.1007/s11192-016-2110-3
- B Latour “Science in action: How to follow scientists and engineers through society” Milton Keynes: Open University Press, 1987 URL: https://books.google.at/books?hl=de&lr=&id=sC4bk4DZXTQC&oi=fnd&pg=PA1&dq=cience+in+Action:+How+to+Follow+Scientists+and+Engineers+Through+Society&ots=WcmJFp77PB&sig=cb_CFMLrc3RtVw4-2WDfBcjSqIk
- Feng Shi, Jacob G. Foster and James A. Evans “Weaving the fabric of science: Dynamic network models of science’s unfolding structure” In Social Networks 43, 2015, pp. 73–85 DOI: 10.1016/j.socnet.2015.02.006
- Robert K Merton “The sociology of science: Theoretical and empirical investigations” University of Chicago press, 1973
- ““Excellence R Us”: university research and the fetishisation of excellence” In Palgrave Communications 3.1, 2017, pp. 1–13 DOI: https://doi.org/10.1057/palcomms.2016.105
- Tomas Hellström “Homing in on excellence: Dimensions of appraisal in Center of Excellence program evaluations” In Evaluation 17.2, 2011, pp. 117–131 DOI: 10.1177/1356389011400891
- “Micro-political practices in higher education: a challenge to excellence as a rationalising myth?” In Critical Studies in Education 61.2, 2020, pp. 195–211 DOI: 10.1080/17508487.2017.1381629
- Daniel Kahneman, Olivier Sibony and Cass R Sunstein “Noise: a flaw in human judgment” Hachette UK, 2021
- John PA Ioannidis and Zacharias Maniadis “In defense of quantitative metrics in researcher assessments” Publisher: Public Library of Science San Francisco, CA USA In Plos Biology 21.12, 2023, pp. e3002408 DOI: 10.1371/journal.pbio.3002408
- Peter Van den Besselaar and Ulf Sandström “Measuring researcher independence using bibliometric data: A proposal for a new performance indicator” Publisher: Public Library of Science San Francisco, CA USA In PloS one 14.3, 2019, pp. e0202712 DOI: 10.1371/journal.pone.0202712
- “Bibliometrics: the Leiden Manifesto for research metrics” Publisher: Nature Publishing Group UK London In Nature 520.7548, 2015, pp. 429–431 DOI: 10.1038/520429a
- Loet Leydesdorff, Paul Wouters and Lutz Bornmann “Professional and citizen bibliometrics: complementarities and ambivalences in the development and use of indicators—a state-of-the-art report” Publisher: Springer In Scientometrics 109, 2016, pp. 2129–2150 DOI: 10.1007/s11192-016-2150-8
- Ulf Sandström and Peter Van den Besselaar “Funding, evaluation, and the performance of national research systems” Publisher: Elsevier In Journal of Informetrics 12.1, 2018, pp. 365–384 DOI: 10.1016/j.joi.2018.01.007
- Bruno Broucker and Kurt De Wit “New public management in higher education” In The Palgrave international handbook of higher education policy and governance Springer, 2015, pp. 57–75
- Georg Krücken “Multiple competitions in higher education: a conceptual approach” Publisher: Taylor & Francis In Innovation 23.2, 2021, pp. 163–181 DOI: 10.1080/14479338.2019.1684652
- Christine Musselin “New forms of competition in higher education” Publisher: Oxford University Press In Socio-Economic Review 16.3, 2018, pp. 657–683 DOI: 10.1093/ser/mwy033
- “University research funding and publication performance—An international comparison” Publisher: Elsevier In Research policy 39.6, 2010, pp. 822–834 DOI: 10.1016/j.respol.2010.03.003
- “Is research funding always beneficial? A cross-disciplinary analysis of U.K. research 2014–20” Publisher: MIT Press Journals In Quantitative Science Studies 4.2, 2023, pp. 501–534 DOI: 10.1162/qss_a_00254
- Ian I Mitroff and Daryl E Chubin “Peer review at the NSF: A dialectical policy analysis” Publisher: Sage Publications Sage CA: Thousand Oaks, CA In Social Studies of Science 9.2, 1979, pp. 199–232 DOI: 10.1177/030631277900900203
- Jim McCullough “First comprehensive survey of NSF applicants focuses on their concerns about proposal review” Publisher: Sage Publications Sage CA: Thousand Oaks, CA In Science, Technology, & Human Values 14.1, 1989, pp. 78–88 DOI: 10.1177/016224398901400107
- Lutz Bornmann “Scientific peer review” Publisher: John Wiley & Sons, Inc. New York, NY, USA In Annual review of information science and technology 45.1, 2011, pp. 197–245 DOI: 10.1002/aris.2011.1440450112
- Lambros Roumbanis “The oracles of science: On grant peer review and competitive funding” Publisher: SAGE Publications Sage UK: London, England In Social Science Information 60.3, 2021, pp. 356–362 DOI: 10.1177/05390184211019241
- “Persistent nepotism in peer-review” Publisher: Akadémiai Kiadó, co-published with Springer Science+ Business Media BV … In Scientometrics 74.2, 2008, pp. 175–189 DOI: 10.1007/s11192-008-0211-3
- Gillian Breen “Nepotism and sexism in peer-review” Publisher: Nature Publishing Group UK London In Nature 389.6649, 1997, pp. 326–326 DOI: 10.1038/38594
- Von Bakanic, Clark McPhail and Rita J Simon “The manuscript review and decision-making process” Publisher: JSTOR In American Sociological Review, 1987, pp. 631–642 DOI: 10.2307/2095599
- Terttu Luukkonen “Conservatism and risk-taking in peer review: Emerging ERC practices” Publisher: Oxford University Press In Research evaluation 21.1, 2012, pp. 48–60 DOI: 10.1093/reseval/rvs001
- “Rescuing US biomedical research from its systemic flaws” Publisher: National Acad Sciences In Proceedings of the National Academy of Sciences 111.16, 2014, pp. 5773–5777 DOI: 10.1073/pnas.1404402111
- “Gender effects in the peer reviews of grant proposals: A comprehensive meta-analysis comparing traditional and multilevel approaches” Publisher: Sage Publications Sage CA: Los Angeles, CA In Review of Educational Research 79.3, 2009, pp. 1290–1326 DOI: 10.3102/0034654309334143
- G David L Travis and Harry M Collins “New light on old boys: Cognitive and institutional particularism in the peer review system” Publisher: Sage Publications Sage CA: Thousand Oaks, CA In Science, Technology, & Human Values 16.3, 1991, pp. 322–341 DOI: 10.1177/016224399101600303
- “Defining the role of cognitive distance in the peer review process with an explorative study of a grant scheme in infection biology” Publisher: Oxford University Press In Research Evaluation 24.3, 2015, pp. 271–281 DOI: 10.1093/reseval/rvv009
- Peter Van den Besselaar “Selection committee membership: Service or self-service” Publisher: Elsevier In Journal of Informetrics 6.4, 2012, pp. 580–585 DOI: 10.1016/j.joi.2012.05.003
- Peter A Abrams “The predictive ability of peer review of grant proposals: The case of ecology and the US National Science Foundation” Publisher: Sage Publications London In Social Studies of Science 21.1, 1991, pp. 111–132 DOI: 10.1177/030631291021001006
- “Low agreement among reviewers evaluating the same NIH grant applications” tex.eprint: https://www.pnas.org/doi/pdf/10.1073/pnas.1714379115 In Proceedings of the National Academy of Sciences 115.12, 2018, pp. 2952–2957 DOI: 10.1073/pnas.1714379115
- “Peering at peer review revealed high degree of chance associated with funding of grant applications” Publisher: Elsevier In Journal of clinical epidemiology 59.8, 2006, pp. 842–848 DOI: 10.1016/j.jclinepi.2005.12.007
- Rüdiger Mutz, Lutz Bornmann and Hans-Dieter Daniel “Heterogeneity of inter-rater reliabilities of grant peer reviews and its determinants: a general estimating equations approach” Publisher: Public Library of Science San Francisco, USA In PLoS One 7.10, 2012, pp. e48509 DOI: 10.1371/journal.pone.0048509
- Elena A Erosheva, Patrícia Martinková and Carole J Lee “When zero may not be zero: A cautionary note on the use of inter-rater reliability in evaluating grant peer review” Publisher: Oxford University Press In Journal of the Royal Statistical Society Series A: Statistics in Society 184.3, 2021, pp. 904–919 DOI: 10.1111/rssa.12681
- Lutz Bornmann, Rüdiger Mutz and Hans-Dieter Daniel “A reliability-generalization study of journal peer reviews: A multilevel meta-analysis of inter-rater reliability and its determinants” Publisher: Public Library of Science San Francisco, USA In PloS one 5.12, 2010, pp. e14331 DOI: 10.1371/journal.pone.0014331
- “Reviewer training for improving grant and journal peer review” Publisher: John Wiley & Sons, Ltd In Cochrane Database of Systematic Reviews, 2023 DOI: https://doi.org/10.1002/14651858.MR000056.pub2
- Stephen Cole, Jonathan R Cole and Gary A Simon “Chance and consensus in peer review” Publisher: American Association for the Advancement of Science In Science (New York, N.Y.) 214.4523, 1981, pp. 881–886 DOI: 10.1126/science.7302566
- Nicholas Graves, Adrian G Barnett and Philip Clarke “Funding grant proposals for scientific research: retrospective analysis of scores by members of grant review panel” Publisher: British Medical Journal Publishing Group In BMJ (Clinical research ed.) 343, 2011 DOI: 10.1136/bmj.d4797
- Michèle Lamont “How professors think: Inside the curious world of academic judgment” Harvard University Press, 2009
- Lambros Roumbanis “Academic judgments under uncertainty: A study of collective anchoring effects in Swedish Research Council panel groups” Publisher: SAGE Publications Sage UK: London, England In Social studies of science 47.1, 2017, pp. 95–116 DOI: 10.1177/0306312716659789
- “‘Are you siding with a personality or the grant proposal?’: observations on how peer review panels function” Publisher: Springer Science and Business Media LLC In Research Integrity and Peer Review 2.1, 2017 DOI: 10.1186/s41073-017-0043-x
- “Percentile ranking and citation impact of a large cohort of National Heart, Lung, and Blood Institute–funded cardiovascular R01 grants” Publisher: Am Heart Assoc In Circulation research 114.4, 2014, pp. 600–606 DOI: 10.1161/CIRCRESAHA.114.302656
- “Citation impact of NHLBI R01 grants funded through the American Recovery and Reinvestment Act as compared to R01 grants funded through a standard payline” Publisher: Am Heart Assoc In Circulation research 116.5, 2015, pp. 784–788 DOI: 10.1161/CIRCRESAHA.116.305894
- Peter Van den Besselaar and Ulf Sandström “Early career grants, performance, and careers: A study on predictive validity of grant decisions” In Journal of Informetrics 9.4, 2015, pp. 826–838 DOI: 10.1016/j.joi.2015.07.011
- “Association of percentile ranking with citation impact and productivity in a large cohort of de novo NIMH-funded R01 grants” Publisher: Nature Publishing Group In Molecular psychiatry 20.9, 2015, pp. 1030–1036 DOI: 10.1038/mp.2015.71
- “Prior publication productivity, grant percentile ranking, and topic-normalized citation impact of NHLBI cardiovascular R01 grants” Publisher: Am Heart Assoc In Circulation research 115.7, 2014, pp. 617–624 DOI: 10.1161/CIRCRESAHA.115.304766
- “The validation of peer review through research impact measures and the implications for funding strategies” Publisher: Public Library of Science San Francisco, USA In PLoS One 9.9, 2014, pp. e106474 DOI: 10.1371/journal.pone.0106474
- “Predicting productivity returns on investment: thirty years of peer review, grant funding, and publication of highly cited papers at the National Heart, Lung, and Blood Institute” Publisher: Am Heart Assoc In Circulation research 117.3, 2015, pp. 239–243 DOI: 10.1161/CIRCRESAHA.115.306830
- Martin Reinhart “Peer review of grant applications in biology and medicine. Reliability, fairness, and validity” Publisher: Akadémiai Kiadó, co-published with Springer Science+ Business Media BV … In Scientometrics 81.3, 2009, pp. 789–809 DOI: 10.1007/s11192-008-2220-7
- Lutz Bornmann, Loet Leydesdorff and Peter Van den Besselaar “A meta-evaluation of scientific research proposals: Different ways of comparing rejected to awarded applications” arXiv: 0911.3558 Publisher: Elsevier tex.arxivid: 0911.3558 In Journal of Informetrics 4.3, 2010, pp. 211–220 DOI: 10.1016/j.joi.2009.10.004
- Stephen A Gallo and Scott R Glisson “External tests of peer review validity via impact measures” Publisher: Frontiers Media SA In Frontiers in Research Metrics and Analytics 3, 2018, pp. 22 DOI: 10.3389/frma.2018.00022
- “Mining the archives: Analyses of CIHR research grant adjudications”, 2002 URL: https://www.researchgate.net/publication/251442420
- Karen B Schmaling and Stephen A Gallo “Gender differences in peer reviewed grant applications, awards, and amounts: a systematic review and meta-analysis” Publisher: Springer In Research Integrity and Peer Review 8.1, 2023, pp. 2 DOI: 10.1186/s41073-023-00127-3
- Shulamit Kahn, Stephen J Ceci and Wendy M Williams “Is there gender bias in grant success? An extended meta-analysis” In An Extended Meta-Analysis (march 31, 2023). Boston University Questrom School of Business Research Paper, 2023
- “Effective or predatory funding? Evaluating the hidden costs of grant applications” Publisher: John Wiley and Sons Inc In Immunology & Cell Biology 101.2, 2023, pp. 104–111 DOI: 10.1111/imcb.12592
- Adrian Barnett “Funding schemes that cost as much as they reward” In Median Watch, 2021 URL: https://medianwatch.netlify.app/post/hidden_funding_costs
- Gerald Schweiger “Can’t We Do Better? A cost-benefit analysis of proposal writing in a competitive funding environment” Publisher: Public Library of Science San Francisco, CA USA In Plos one 18.4, 2023, pp. e0282320 DOI: 10.1371/journal.pone.0282320
- “To apply or not to apply: A survey analysis of grant writing costs and benefits” Publisher: Public Library of Science San Francisco, CA USA In PloS one 10.3, 2015, pp. e0118494 DOI: 10.1371/journal.pone.0118494
- “On the time spent preparing grant proposals: an observational study of Australian researchers” Publisher: British Medical Journal Publishing Group In BMJ open 3.5, 2013, pp. e002800 DOI: 10.1136/bmjopen-2013-002800
- “The science of self-report: Implications for research and practice” Psychology Press, 1999
- European-University-Association “EFSI and horizon 2020: Efficiency and opportunity cost AN EUA REVIEW”, 2017 URL: https://eua.eu/downloads/publications/efsi%20and%20horizon%202020%20efficiency%20and%20opportunity%20cost%20-%20an%20eua%20review.pdf
- Sandra Schneider “Results of the 2018 FDP faculty workload survey. Technical report, the federal demonstration partnership”, 2020 URL: https://thefdp.org/wp-content/uploads/FDP-FWS-2018-Primary-Report.pdf
- Albert N. Link, Christopher A. Swann and Barry Bozeman “A time allocation study of university faculty” In Economics of Education Review 27.4, 2008, pp. 363–374 DOI: 10.1016/j.econedurev.2007.04.002
- “How much would each researcher receive if competitive government research funding were distributed equally among researchers?” Publisher: Public Library of Science In PLOS ONE 12.9, 2017, pp. e0183967 DOI: 10.1371/journal.pone.0183967
- Mozilla-Foundation “Announcing the 2017 mozilla fellows for science!”, 2017 URL: https://blog.mozilla.org/foundation-archive/mozilla-science/2017-science-fellows/
- Kyle Myers “The potential benefits of costly applications in grant contests” Publisher: Elsevier BV In SSRN Electronic Journal, 2022 DOI: 10.2139/ssrn.4154820
- Jian Wang, Reinhilde Veugelers and Paula Stephan “Bias against novelty in science: A cautionary tale for users of bibliometric indicators” Publisher: Elsevier In Research Policy 46.8, 2017, pp. 1416–1436 DOI: 10.1016/j.respol.2017.06.006
- Lingfei Wu, Dashun Wang and James A Evans “Large teams develop and small teams disrupt science and technology” Publisher: Nature Publishing Group UK London In Nature 566.7744, 2019, pp. 378–382 DOI: 10.1038/s41586-019-0941-9
- Kuhn Thomas “The structure of scientific revolutions” In International Encyclopedia of Unified Science 2.2, 1962
- “The science of science” Cambridge University Press, 2021
- Filippo Radicchi, Santo Fortunato and Claudio Castellano “Universality of citation distributions: Toward an objective measure of scientific impact” Publisher: National Acad Sciences In Proceedings of the National Academy of Sciences 105.45, 2008, pp. 17268–17272 DOI: 10.1073/pnas.0806977105
- Reinhilde Veugelers, Jian Wang and Paula Stephan “Do funding agencies select and enable risky research: Evidence from ERC using novelty as a proxy of risk taking”, 2022
- Marianne Lanoë “The evaluation of competitive research funding:. an application to French programs”, 2018
- “Looking across and looking beyond the knowledge frontier: Intellectual distance, novelty, and resource allocation in science” Publisher: INFORMS In Management science 62.10, 2016, pp. 2765–2783 DOI: 10.1287/mnsc.2015.2285
- Caroline S Wagner and Jeffrey Alexander “Evaluating transformative research programmes: A case study of the NSF Small Grants for Exploratory Research programme” Publisher: Oxford University Press In Research Evaluation 22.3, 2013, pp. 187–197 DOI: 10.1093/reseval/rvt006
- Kevin Gross and Carl T Bergstrom “Why ex post peer review encourages high-risk research while ex ante review discourages it” Publisher: National Acad Sciences In Proceedings of the National Academy of Sciences 118.51, 2021, pp. e2111615118 DOI: 10.1073/pnas.2111615118
- Pierre Azoulay, Joshua S Graff Zivin and Gustavo Manso “Incentives and creativity: evidence from the academic life sciences” Publisher: Wiley Online Library In The RAND Journal of Economics 42.3, 2011, pp. 527–554 DOI: 10.1111/j.1756-2171.2011.00140.x
- Jian Wang, You-Na Lee and John P Walsh “Funding model and creativity in science: Competitive versus block funding and status contingency effects” Publisher: Elsevier In Research Policy 47.6, 2018, pp. 1070–1083 DOI: 10.1016/j.respol.2018.03.014
- Hui Fang “Peer review and over-competitive research funding fostering mainstream opinion to monopoly” Publisher: Akadémiai Kiadó, co-published with Springer Science+ Business Media BV … In Scientometrics 87.2, 2011, pp. 293–301 DOI: 10.1007/s11192-010-0323-4
- Xuan Zhen Liu and Hui Fang “Peer review and over-competitive research funding fostering mainstream opinion to monopoly. Part II” Publisher: Akadémiai Kiadó, co-published with Springer Science+ Business Media BV … In Scientometrics 90.2, 2012, pp. 607–616 DOI: 10.1007/s11192-011-0526-3
- Michael J Joyner, Nigel Paneth and John PA Ioannidis “What happens when underperforming big ideas in research become entrenched?” Publisher: American Medical Association In JAMA : the journal of the American Medical Association 316.13, 2016, pp. 1355–1356 DOI: 10.1001/jama.2016.11076
- John PA Ioannidis, Iztok Hozo and Benjamin Djulbegovic “Federal funding and citation metrics of US biomedical researchers, 1996 to 2022” Publisher: American Medical Association In JAMA Network Open 5.12, 2022, pp. e2245590–e2245590 DOI: 10.1001/jamanetworkopen.2022.45590
- Charitini Stavropoulou, Melek Somai and John PA Ioannidis “Most UK scientists who publish extremely highly-cited papers do not secure funding from major public and charity funders: A descriptive analysis” Publisher: Public Library of Science San Francisco, CA USA In PLoS One 14.2, 2019, pp. e0211460 DOI: 10.1371/journal.pone.0211460
- Helen E Longino “Science as social knowledge: Values and objectivity in scientific inquiry” Princeton university press, 1990
- Bec Crew “Here’s how to deal with failure, say senior scientists” In Nature, 2019 URL: https://www.nature.com/nature-index/news/how-to-deal-with-failure-rejection-academic-research-say-senior-scientists
- Gemma Conroy “Here’s why so many young researchers want to quit – in five graphs” In Nature, 2020 URL: https://www.nature.com/nature-index/news/heres-why-so-many-young-researchers-want-to-quit-in-five-graphs
- “The impact of funding deadlines on personal workloads, stress and family relationships: a qualitative study of Australian researchers” Publisher: British Medical Journal Publishing Group In BMJ open 4.3, 2014, pp. e004462 DOI: 10.1136/bmjopen-2013-004462
- “Grieving academic grant rejections: Examining funding failure and experiences of loss” Publisher: SAGE Publications Sage UK: London, England In The Sociological Review, 2023, pp. 00380261231207196 DOI: 10.1177/00380261231207196
- Shannon Hall “A mental-health crisis is gripping science — toxic research culture is to blame” ISSN: 14764687 Number: 7962 Pages: 666–668 Publication title: Nature Volume: 617 Nature Research, 2023 DOI: 10.1038/d41586-023-01708-4
- Chris Woolston “Postdoc survey reveals disenchantment with working life” Publisher: Nature Publishing Group In Nature 587.7834, 2020, pp. 505–509 DOI: 10.1038/d41586-020-03191-7
- Elaine Howard Ecklund and Anne E. Lincoln “Scientists want more children” Publisher: Public Library of Science (PLoS) In PLoS ONE 6.8, 2011, pp. e22590 DOI: 10.1371/journal.pone.0022590
- Erin A. Cech and Mary Blair-Loy “The changing career trajectories of new parents in STEM” Publisher: Proceedings of the National Academy of Sciences In Proceedings of the National Academy of Sciences 116.10, 2019, pp. 4182–4187 DOI: 10.1073/pnas.1810862116
- “Gender differences in Australian research grant awards, applications, amounts, and workforce participation” Publisher: Center for Open Science, 2023 DOI: 10.31219/osf.io/cpvqk
- “Meta-research: justifying career disruption in funding applications, a survey of Australian researchers” Publisher: eLife Sciences Publications, Ltd In eLife 11, 2022 DOI: 10.7554/elife.76123
- “The tenure process and extending the tenure clock: The experience of faculty at one university” Publisher: Springer Science and Business Media LLC In Higher Education Policy 23.1, 2010, pp. 17–38 DOI: 10.1057/hep.2009.18
- “The perverse effects of competition on scientists’ work and relationships” Publisher: Springer Science and Business Media LLC In Science and Engineering Ethics 13.4, 2007, pp. 437–461 DOI: 10.1007/s11948-007-9042-5
- D.H. Osmond “Malices wonderland: Research funding and peer review” Publisher: Wiley In Journal of Neurobiology 14.2, 1983, pp. 95–112 DOI: 10.1002/neu.480140202
- Stijn Conix, Andreas De Block and Krist Vaesen “Grant writing and grant peer review as questionable research practices” Publisher: F1000 Research Ltd In F1000Research 10, 2021, pp. 1126 DOI: 10.12688/f1000research.73893.2
- Lesley A Schimanski and Juan Pablo Alperin “The evaluation of scholarship in academic promotion and tenure processes: Past, present, and future” Publisher: Faculty of 1000 Ltd In F1000Research 7, 2018 DOI: 10.12688/f1000research.16493.1
- “Prevalence of questionable research practices, research misconduct and their potential explanatory factors: A survey among academic researchers in The Netherlands” Publisher: Public Library of Science In PloS one 17.2, 2022, pp. e0263023 DOI: 10.1371/journal.pone.0263023
- Harold R Garner, Lauren J McIver and Michael B Waitzkin “Same work, twice the money?” Publisher: Nature Publishing Group UK London In Nature 493.7434, 2013, pp. 599–601 DOI: 10.1038/493599a
- “Questionable research practices in competitive grant funding: A survey” Publisher: Public Library of Science San Francisco, CA USA In Plos one 18.11, 2023, pp. e0293310 DOI: 10.1371/journal.pone.0293310
- Adrian Barnett, Philip Clarke and Nicholas Graves “Survey of NHMRC applicants” Publisher: OSF tex.copyright: Creative Commons Attribution 4.0 International, 2023 DOI: 10.17605/OSF.IO/9KMTG
- “Australia’s first criminal prosecution for research fraud: a case study from the University of Queensland”, 2017 URL: https://www.ccc.qld.gov.au/publications/australias-first-criminal-prosecution-research-fraud-case-study-university-queensland
- The Office of Research Integrity “Case summary: Brand, Toni M.”, 2022 URL: https://ori.hhs.gov/content/case-summary-brand-toni-m
- Juan Manuel Parrilla “ChatGPT use shows that the grant-application system is broken” Publisher: Springer Science and Business Media LLC In Nature 623.7986, 2023, pp. 443–443 DOI: 10.1038/d41586-023-03238-5
- Nick Huntington-Klein “The effect: An introduction to research design and causality” CRC Press, 2021
- James McKeen Cattell and Jaques Cattell “American men of science: A biographical directory” Bowker, 1910
- Julius H Comroe Jr and Robert D Dripps “Scientific basis for the support of biomedical science” Publisher: American Association for the Advancement of Science In Science (New York, N.Y.) 192.4235, 1976, pp. 105–111 DOI: 10.1126/science.769161
- Adrian G. Barnett “Funding by lottery: Political problems and research opportunities” Publisher: American Society for Microbiology In mBio 7.4, 2016 DOI: 10.1128/mbio.01369-16
- Susan Guthrie, Ioana Ghiga and Steven Wooding “What do we know about grant peer review in the health sciences?” Publisher: Faculty of 1000 Ltd In F1000Research 6, 2017 DOI: 10.12688/f1000research.11917.1
- “OpenSAFELY”, 2024 URL: https://www.opensafely.org/about/
- “Gender bias in funding evaluation: A randomized experiment” Publisher: MIT Press One Broadway, 12th Floor, Cambridge, Massachusetts 02142, USA … In Quantitative Science Studies 4.3, 2023, pp. 594–621 DOI: 10.1162/qss_a_00263
- “Overton: A bibliometric database of policy document citations” Publisher: MIT Press One Broadway, 12th Floor, Cambridge, Massachusetts 02142, USA … In Quantitative science studies 3.3, 2022, pp. 624–650 DOI: 10.1162/qss_a_00204
- “Changing directions: Steering science, technology and innovation towards the sustainable development goals” Publisher: STRINGS, SPRU, University of Sussex, 2022
- Richard Smith “Classical peer review: an empty gun” Publisher: Springer Science and Business Media LLC In Breast Cancer Research 12.S4, 2010 DOI: 10.1186/bcr2742
- Science “No pressure: NSF test finds eliminating deadlines halves number of grant proposals”, 2016 URL: https://www.science.org/content/article/no-pressure-nsf-test-finds-eliminating-deadlines-halves-number-grant-proposals#:$sim$:text=This%20week%2C%20at%20an%20NSF,favor%20of%20an%20anytime%20submission
- Ingvild Reymert “Bibliometrics in Academic Recruitment: A Screening Tool Rather than a Game Changer” In Minerva 59.1, 2021, pp. 53–78 DOI: 10.1007/s11024-020-09419-0
- Michael H MacRoberts and Barbara R MacRoberts “Problems of citation analysis: A study of uncited and seldom-cited influences” Publisher: Wiley Online Library In Journal of the American Society for Information Science and Technology 61.1, 2010, pp. 1–12 DOI: 10.1002/asi.21228
- John PA Ioannidis and Zacharias Maniadis “Quantitative research assessment: using metrics against gamed metrics” Publisher: Springer In Internal and Emergency Medicine, 2023, pp. 1–9 DOI: 10.1007/s11739-023-03447-w
- Charlie Mom, Torger Möller and Peter Van den Besselaar “Determinants of cognitive mobility” In Proceedings of ISSI 2023: 19th international conference of the International Society of Scientometrics and Informetrics, 2023, pp. 463–472
- CoARA “Coalition for advancing research assessment”, 2023 URL: https://coara.eu
- “Practicing responsible research assessment: Qualitative study of faculty hiring, promotion, and tenure assessments in the united states” Publisher: SocArXiv, 2023 DOI: 10.31235/osf.io/2d7ax
- Lutz Bornmann and Julian N Marewski “Heuristics as conceptual lens for understanding and studying the usage of bibliometrics in research evaluation” Publisher: Springer In Scientometrics 120.2, 2019, pp. 419–459 DOI: 10.1007/s11192-019-03018-x
- Peter Van den Besselaar and Ulf Sandström “Bibliometrically disciplined peer review: On using indicators in research evaluation” Publisher: Levy Library Press In Scholarly assessment reports 2.1, 2020 DOI: 10.29024/sar.16
- “How to increase value and reduce waste when research priorities are set” Publisher: Elsevier BV In The Lancet 383.9912, 2014, pp. 156–165 DOI: 10.1016/s0140-6736(13)62229-1
- “Rethink funding by putting the lottery first” Publisher: Nature Publishing Group UK London In Nature Human Behaviour, 2023, pp. 1–3 DOI: 10.1038/s41562-023-01649-y
- “Where next for partial randomisation of research funding? The feasibility of RCTs and alternatives” Publisher: The Wellcome Trust In Wellcome Open Research 8, 2023 DOI: 10.12688/wellcomeopenres.19565.1
- Steven De Peuter and Stijn Conix “The modified lottery: Formalizing the intrinsic randomness of research funding” Publisher: Taylor & Francis In Accountability in Research 29.5, 2022, pp. 324–345 DOI: 10.1080/08989621.2021.1927727
- “The acceptability of using a lottery to allocate research funding: a survey of applicants” Publisher: BioMed Central In Research integrity and peer review 5.1, 2020, pp. 1–7 DOI: 10.1186/s41073-019-0089-z
- The-british-academy “Promising’ results from first year of innovative grant awarding trial show greater diversity of awardees and institutions given funding”, 2023 URL: https://www.thebritishacademy.ac.uk/news/promising-results-from-first-year-of-innovative-grant-awarding-trial/
- “Using democracy to award research funding: an observational study” Publisher: Springer In Research Integrity and Peer Review 2, 2017, pp. 1–9 DOI: 10.1186/s41073-017-0040-0
- Kaare Aagaard, Alexander Kladakis and Mathias W Nielsen “Concentration or dispersal of research funding?” Publisher: MIT Press One Rogers Street, Cambridge, MA 02142-1209, USA journals-info … In Quantitative Science Studies 1.1, 2020, pp. 117–149 DOI: 10.1162/qss_a_00002
- “A systematic review and meta-analysis of discrepancies between logged and self-reported digital media use” Publisher: Nature Publishing Group UK London In Nature Human Behaviour 5.11, 2021, pp. 1535–1547 DOI: 10.1038/s41562-021-01117-5
- Athina Tatsioni, Effie Vavva and John PA Ioannidis “Sources of funding for Nobel Prize-winning work: Public or private?” Publisher: Wiley Online Library In The FASEB journal 24.5, 2010, pp. 1335–1339 DOI: 10.1096/fj.09-148239
- “Shifting the level of selection in science” Publisher: SAGE Publications Sage CA: Los Angeles, CA In Perspectives on Psychological Science, 2021, pp. 17456916231182568 DOI: 10.1177/17456916231182568
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.