Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Dataset Artefacts are the Hidden Drivers of the Declining Disruptiveness in Science (2402.14583v1)

Published 7 Feb 2024 in cs.DL and cs.SI

Abstract: Park et al. [1] reported a decline in the disruptiveness of scientific and technological knowledge over time. Their main finding is based on the computation of CD indices, a measure of disruption in citation networks [2], across almost 45 million papers and 3.9 million patents. Due to a factual plotting mistake, database entries with zero references were omitted in the CD index distributions, hiding a large number of outliers with a maximum CD index of one, while keeping them in the analysis [1]. Our reanalysis shows that the reported decline in disruptiveness can be attributed to a relative decline of these database entries with zero references. Notably, this was not caught by the robustness checks included in the manuscript. The regression adjustment fails to control for the hidden outliers as they correspond to a discontinuity in the CD index. Proper evaluation of the Monte-Carlo simulations reveals that, because of the preservation of the hidden outliers, even random citation behaviour replicates the observed decline in disruptiveness. Finally, while these papers and patents with supposedly zero references are the hidden drivers of the reported decline, their source documents predominantly do make references, exposing them as pure dataset artefacts.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (11)
  1. Papers and patents are becoming less disruptive over time. Nature 613, 138–144 (2023). [2] Funk, R. J. & Owen-Smith, J. A dynamic network measure of technological change. Management science 63, 791–817 (2017). [3] Waskom, M. Treat binwidth as approximate to avoid dropping outermost datapoints. (2023). https://github.com/mwaskom/seaborn/pull/3489. [4] Lin, Z., Yin, Y., Liu, L. & Wang, D. Sciscinet: A large-scale open data lake for the science of science research. Scientific Data 10, 315 (2023). [5] Uzzi, B., Mukherjee, S., Stringer, M. & Jones, B. Atypical combinations and scientific impact. Science 342, 468–472 (2013). [6] Hofmann, H., Wickham, H. & Kafadar, K. value plots: Boxplots for large data. Journal of Computational and Graphical Statistics 26, 469–477 (2017). [7] Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Funk, R. J. & Owen-Smith, J. A dynamic network measure of technological change. Management science 63, 791–817 (2017). [3] Waskom, M. Treat binwidth as approximate to avoid dropping outermost datapoints. (2023). https://github.com/mwaskom/seaborn/pull/3489. [4] Lin, Z., Yin, Y., Liu, L. & Wang, D. Sciscinet: A large-scale open data lake for the science of science research. Scientific Data 10, 315 (2023). [5] Uzzi, B., Mukherjee, S., Stringer, M. & Jones, B. Atypical combinations and scientific impact. Science 342, 468–472 (2013). [6] Hofmann, H., Wickham, H. & Kafadar, K. value plots: Boxplots for large data. Journal of Computational and Graphical Statistics 26, 469–477 (2017). [7] Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Waskom, M. Treat binwidth as approximate to avoid dropping outermost datapoints. (2023). https://github.com/mwaskom/seaborn/pull/3489. [4] Lin, Z., Yin, Y., Liu, L. & Wang, D. Sciscinet: A large-scale open data lake for the science of science research. Scientific Data 10, 315 (2023). [5] Uzzi, B., Mukherjee, S., Stringer, M. & Jones, B. Atypical combinations and scientific impact. Science 342, 468–472 (2013). [6] Hofmann, H., Wickham, H. & Kafadar, K. value plots: Boxplots for large data. Journal of Computational and Graphical Statistics 26, 469–477 (2017). [7] Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Lin, Z., Yin, Y., Liu, L. & Wang, D. Sciscinet: A large-scale open data lake for the science of science research. Scientific Data 10, 315 (2023). [5] Uzzi, B., Mukherjee, S., Stringer, M. & Jones, B. Atypical combinations and scientific impact. Science 342, 468–472 (2013). [6] Hofmann, H., Wickham, H. & Kafadar, K. value plots: Boxplots for large data. Journal of Computational and Graphical Statistics 26, 469–477 (2017). [7] Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Uzzi, B., Mukherjee, S., Stringer, M. & Jones, B. Atypical combinations and scientific impact. Science 342, 468–472 (2013). [6] Hofmann, H., Wickham, H. & Kafadar, K. value plots: Boxplots for large data. Journal of Computational and Graphical Statistics 26, 469–477 (2017). [7] Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Hofmann, H., Wickham, H. & Kafadar, K. value plots: Boxplots for large data. Journal of Computational and Graphical Statistics 26, 469–477 (2017). [7] Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023).
  2. A dynamic network measure of technological change. Management science 63, 791–817 (2017). [3] Waskom, M. Treat binwidth as approximate to avoid dropping outermost datapoints. (2023). https://github.com/mwaskom/seaborn/pull/3489. [4] Lin, Z., Yin, Y., Liu, L. & Wang, D. Sciscinet: A large-scale open data lake for the science of science research. Scientific Data 10, 315 (2023). [5] Uzzi, B., Mukherjee, S., Stringer, M. & Jones, B. Atypical combinations and scientific impact. Science 342, 468–472 (2013). [6] Hofmann, H., Wickham, H. & Kafadar, K. value plots: Boxplots for large data. Journal of Computational and Graphical Statistics 26, 469–477 (2017). [7] Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Waskom, M. Treat binwidth as approximate to avoid dropping outermost datapoints. (2023). https://github.com/mwaskom/seaborn/pull/3489. [4] Lin, Z., Yin, Y., Liu, L. & Wang, D. Sciscinet: A large-scale open data lake for the science of science research. Scientific Data 10, 315 (2023). [5] Uzzi, B., Mukherjee, S., Stringer, M. & Jones, B. Atypical combinations and scientific impact. Science 342, 468–472 (2013). [6] Hofmann, H., Wickham, H. & Kafadar, K. value plots: Boxplots for large data. Journal of Computational and Graphical Statistics 26, 469–477 (2017). [7] Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Lin, Z., Yin, Y., Liu, L. & Wang, D. Sciscinet: A large-scale open data lake for the science of science research. Scientific Data 10, 315 (2023). [5] Uzzi, B., Mukherjee, S., Stringer, M. & Jones, B. Atypical combinations and scientific impact. Science 342, 468–472 (2013). [6] Hofmann, H., Wickham, H. & Kafadar, K. value plots: Boxplots for large data. Journal of Computational and Graphical Statistics 26, 469–477 (2017). [7] Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Uzzi, B., Mukherjee, S., Stringer, M. & Jones, B. Atypical combinations and scientific impact. Science 342, 468–472 (2013). [6] Hofmann, H., Wickham, H. & Kafadar, K. value plots: Boxplots for large data. Journal of Computational and Graphical Statistics 26, 469–477 (2017). [7] Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Hofmann, H., Wickham, H. & Kafadar, K. value plots: Boxplots for large data. Journal of Computational and Graphical Statistics 26, 469–477 (2017). [7] Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023).
  3. Waskom, M. Treat binwidth as approximate to avoid dropping outermost datapoints. (2023). https://github.com/mwaskom/seaborn/pull/3489. [4] Lin, Z., Yin, Y., Liu, L. & Wang, D. Sciscinet: A large-scale open data lake for the science of science research. Scientific Data 10, 315 (2023). [5] Uzzi, B., Mukherjee, S., Stringer, M. & Jones, B. Atypical combinations and scientific impact. Science 342, 468–472 (2013). [6] Hofmann, H., Wickham, H. & Kafadar, K. value plots: Boxplots for large data. Journal of Computational and Graphical Statistics 26, 469–477 (2017). [7] Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Lin, Z., Yin, Y., Liu, L. & Wang, D. Sciscinet: A large-scale open data lake for the science of science research. Scientific Data 10, 315 (2023). [5] Uzzi, B., Mukherjee, S., Stringer, M. & Jones, B. Atypical combinations and scientific impact. Science 342, 468–472 (2013). [6] Hofmann, H., Wickham, H. & Kafadar, K. value plots: Boxplots for large data. Journal of Computational and Graphical Statistics 26, 469–477 (2017). [7] Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Uzzi, B., Mukherjee, S., Stringer, M. & Jones, B. Atypical combinations and scientific impact. Science 342, 468–472 (2013). [6] Hofmann, H., Wickham, H. & Kafadar, K. value plots: Boxplots for large data. Journal of Computational and Graphical Statistics 26, 469–477 (2017). [7] Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Hofmann, H., Wickham, H. & Kafadar, K. value plots: Boxplots for large data. Journal of Computational and Graphical Statistics 26, 469–477 (2017). [7] Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023).
  4. Sciscinet: A large-scale open data lake for the science of science research. Scientific Data 10, 315 (2023). [5] Uzzi, B., Mukherjee, S., Stringer, M. & Jones, B. Atypical combinations and scientific impact. Science 342, 468–472 (2013). [6] Hofmann, H., Wickham, H. & Kafadar, K. value plots: Boxplots for large data. Journal of Computational and Graphical Statistics 26, 469–477 (2017). [7] Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Uzzi, B., Mukherjee, S., Stringer, M. & Jones, B. Atypical combinations and scientific impact. Science 342, 468–472 (2013). [6] Hofmann, H., Wickham, H. & Kafadar, K. value plots: Boxplots for large data. Journal of Computational and Graphical Statistics 26, 469–477 (2017). [7] Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Hofmann, H., Wickham, H. & Kafadar, K. value plots: Boxplots for large data. Journal of Computational and Graphical Statistics 26, 469–477 (2017). [7] Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023).
  5. Atypical combinations and scientific impact. Science 342, 468–472 (2013). [6] Hofmann, H., Wickham, H. & Kafadar, K. value plots: Boxplots for large data. Journal of Computational and Graphical Statistics 26, 469–477 (2017). [7] Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Hofmann, H., Wickham, H. & Kafadar, K. value plots: Boxplots for large data. Journal of Computational and Graphical Statistics 26, 469–477 (2017). [7] Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023).
  6. value plots: Boxplots for large data. Journal of Computational and Graphical Statistics 26, 469–477 (2017). [7] Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023).
  7. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019). [8] Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Lin, Y., Frey, C. B. & Wu, L. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023).
  8. Remote collaboration fuses fewer breakthrough ideas. Nature 623, 987–991 (2023). [9] Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023).
  9. Tang, J. et al. Arnetminer: Extraction and mining of academic social networks, KDD ’08, 990–998 (Association for Computing Machinery, New York, NY, USA, 2008). [10] Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Ruan, X., Lyu, D., Gong, K., Cheng, Y. & Li, J. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023).
  10. Rethinking the disruption index as a measure of scientific and technological advances. Technological Forecasting and Social Change 172, 121071 (2021). [11] Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023). Macher, J. T., Rutzer, C. & Weder, R. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023).
  11. The illusive slump of disruptive patents. arXiv preprint arXiv:2306.10774 (2023).
Citations (3)
View on Semantic Scholar

Summary

  • The paper reveals that correcting omitted zero-reference entries in citation networks overturns the presumed decline in scientific disruptiveness.
  • Methodologically, reanalysis with adjusted histogram plots and regression models identified an additional 972,161 papers and 142,362 patents with maximum disruption scores.
  • The study implies that better metadata curation and robust analytical models are essential for accurately assessing innovation trends in science.

Dataset Artifacts and the Perceived Decline in Scientific Disruptiveness

The paper "Dataset Artefacts are the Hidden Drivers of the Declining Disruptiveness in Science" addresses the apparent reduction in scientific and technological disruptiveness reported by Park et al. through the lens of citation network analysis. This reassessment critically examines the methodologies underlying the computation of CD (Citation Disruption) indices and their dependence on dataset artifacts.

Overview of CD Index Discrepancies

The original paper by Park et al. quantified disruptiveness by computing CD indices across a massive dataset comprising 45 million papers and 3.9 million patents. The focus was on entries with a CD index of zero, signifying no references, which were inadvertently omitted in the histogram distributions due to a plotting software bug. This exclusion masked the concentration of outliers with a CD index equal to one. Upon reanalysis with corrected plotting settings, the authors identified an additional 972,161 papers and 142,362 patents with maximum CD values, significantly impacting the perceived declining trend.

Analytical Evaluation

The central claim of the analysis is that the observed decline in disruptiveness arises from the relative underrepresentation of these outliers rather than an intrinsic decline in innovative outputs. The robustness checks, including regression adjustments and Monte Carlo simulations employed by Park et al., failed to address these outliers appropriately. Notably, the regression models did not account for the discontinuous impact of zero-referenced entries, as evidenced by a marked improvement in model fit upon including a zero-references dummy variable, enhancing the adjusted R² significantly.

Further, the Monte Carlo simulations maintained a one-to-one correspondence between zero reference entries in original and rewired networks, preserving the illusion of decline. This highlights a systematic issue where the data sources inadvertently equate improved metadata quality over time with diminishing disruptiveness.

Data Source Verification

The authors substantiated their findings by examining metadata quality across diverse data sources such as Web of Science, PatentsView, and SciSciNet. They found a high proportion of zero-references among the identified outliers, which consistently decreases over time, thereby mirroring the apparent decline in disruptiveness.

Moreover, a detailed examination of a random sample of patents and papers revealed that a significant majority do list references in their original form, identifying them as artifacts rather than genuine contributions to the dataset.

Implications and Future Directions

The implications of this paper are twofold: methodologically, it stresses the importance of recognizing dataset artifacts in citation analyses to avoid misinterpreting trends as being indicative of shifts in scientific innovation. Practically, the paper underscores the need for improved metadata curation in academic databases.

Future research could benefit from enhancing the robustness of analysis methodologies to account for data quality issues comprehensively. This could involve regular audits and upgrades to citation databases and developing more sophisticated models that adapt to changes in data patterns over time. Additionally, broader spectrum validation of CD index methodologies across varying domains could provide insights into domain-specific attributes influencing disruptiveness measures.

In conclusion, a systematic reconsideration of dataset artifacts transforms our understanding of disruptiveness trends, emphasizing the role of data integrity in accurately capturing the dynamics of scientific innovation.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
Reddit Logo Streamline Icon: https://streamlinehq.com

Reddit

  1. Seaborn bug responsible for finding of declining disruptiveness in science (1 point, 1 comment)