Research Software Science Overview

• Research Software Science is an interdisciplinary field that applies the scientific method to examine and enhance the full lifecycle of research software.
• It utilizes empirical methodologies such as controlled experiments, data analysis, and reproducibility workflows to yield actionable insights.
• The field bridges technical, social, and cognitive aspects to support sustainable software development and advance metascience.

Research Software Science (RSS) is the application of the scientific method to understanding and improving how software is developed and used for research. It treats research software development and use as subjects of scientific study and regards research software as a core research artifact whose quality, reproducibility, and transparency directly affect scientific reliability, integrity, and discovery (Heroux, 2022, Eisinger et al., 16 Sep 2025). In the literature, RSS is described both as the empirical study of how research software is developed and used and as a distinct interdisciplinary domain spanning the creation, use, stewardship, and study of software in research settings (Eisinger et al., 16 Sep 2025, Katz et al., 2021).

1. Definitions, scope, and domain boundaries

RSS is defined as applying the scientific method to understanding and improving how software is developed and used for research, and as promoting scientific methodologies to explore and establish broadly applicable knowledge regarding research software (Heroux, 2022, Eisinger et al., 16 Sep 2025). It is concerned with sustainable, repeatable, and reproducible software improvements that positively impact research software toward improved scientific discovery, and it covers the full lifecycle and ecosystem of research software, including requirements elicitation and analysis, user-centered design, development, testing, deployment, use, team dynamics, and learning (Heroux, 2022).

The surrounding concept of research software is broad. It includes software used to generate, process, or analyze research results, and software designed and developed to support research activities across science and the humanities, including data collection, processing, analysis, visualization, modeling, and simulation (Gomez-Diaz et al., 2021, Bajraktari et al., 2024). Several papers emphasize that research software is not merely a static output. It is a living research object that must adapt to changes in dependencies, hardware, scientific methods, and use cases, and its sustainability depends on ongoing human effort to fix bugs, add features, and keep systems functional over time (Katz et al., 2021, Carver et al., 2021).

RSS is broader than a narrow account of software construction. It encompasses technical concerns such as modeling, simulation, data analysis, and domain methods; social concerns such as team roles, interactions, workflows, and tools; and cognitive concerns such as learning processes, problem framing, and the ways developers and users acquire insight (Eisinger et al., 16 Sep 2025, Heroux, 2022). This tripartite framing situates research software as both an engineering product and an epistemic instrument whose properties shape scientific practice (Bajraktari et al., 2024).

2. Relationship to metascience and to Research Software Engineering

RSS is closely aligned with metascience, but the classification is contested. Metascience is defined as the scientific study of science itself, with the explicit aim to describe, explain, evaluate, and ultimately improve scientific practices; it investigates how research is conducted, funded, and evaluated, and studies the process and products of scientific knowledge creation rather than the natural world directly (Eisinger et al., 16 Sep 2025). Within that frame, RSS overlaps with metascience through shared commitments to reproducibility, transparency, and empirical study of research processes, especially in computational settings (Eisinger et al., 16 Sep 2025).

The disagreement turns on scope. Under a broad definition of metascience as any empirical effort to improve science, RSS fits because it improves scientific practice in computational contexts and studies research processes around software (Eisinger et al., 16 Sep 2025). Under a narrow definition focused on fundamental epistemological frameworks and systemic structures of science itself, RSS may be better described as an interdisciplinary field that supports metascience rather than being metascience in its own right (Eisinger et al., 16 Sep 2025). The authors of the metascience analysis therefore conclude that RSS is best understood as a distinct interdisciplinary domain that aligns with, and in some definitions fits within, metascience (Eisinger et al., 16 Sep 2025).

RSS is also complementary to Research Software Engineering (RSE), but it is not identical to it. RSE emphasizes practical improvement, adaptation of software engineering techniques to research contexts, and tool or process selection. RSS makes those improvements the subject of scientific inquiry by formulating hypotheses, designing studies, collecting data on team interactions and outcomes, and publishing generalizable findings (Heroux, 2022). One formulation in the literature states the distinction succinctly: “A scientist builds in order to learn; an engineer learns in order to build” (Eisinger et al., 16 Sep 2025).

This distinction has institutional consequences. The RSS literature advocates an identifiable “research software scientist” role within RSE organizations to lead formal investigation into development and use practices, while RSEs remain central as practitioners who translate and adapt software engineering practice inside research workflows (Heroux, 2022). A plausible implication is that RSS provides the evidentiary and conceptual layer through which RSE practice becomes cumulative, comparable, and publishable.

3. Methodologies, conceptual frameworks, and working practices

RSS is explicitly empirical. It uses formal observation and experimentation, obtaining data to detect correlations, designing experiments to identify cause and effect, and publishing results creating a broad impact (Eisinger et al., 16 Sep 2025). The field emphasizes repeatable and reproducible processes designed to generate sharable knowledge rather than local anecdote or ad hoc improvement (Heroux, 2022).

Several frameworks organize this methodology. One is the technical-social-cognitive triad, which treats software quality, reproducibility, and transparency as outcomes of architectures and domain methods, team dynamics and workflows, and learning and problem-solving processes (Eisinger et al., 16 Sep 2025). Another is the “Research–Develop–Deploy” pipeline, in which deployment challenges such as usability, performance, or reproducibility issues generate research questions; findings from this research then inform development and lead to improved deployable capabilities (Heroux, 2022). A closely related organizational variant, “Research, Develop, Deploy (RDD),” has been implemented at Sandia National Laboratories as a triadic workflow linking knowledge artifacts, software artifacts, and operational services (Milewicz et al., 2020).

The literature repeatedly stresses that RSS does not rest on quantitative formulas for quality or reproducibility. The core papers explicitly provide no formal mathematical criteria and instead emphasize conceptual frameworks, empirical designs, and concrete practices (Eisinger et al., 16 Sep 2025, Heroux, 2022). Those practices include literate programming, robust code version control and sharing, controlled compute environments such as containers, persistent data sharing, comprehensive documentation, and adherence to Transparency and Openness Promotion guidelines for data and analytic methods (Eisinger et al., 16 Sep 2025).

Requirements engineering has become a particularly important RSS subarea. Research software is treated as a distinct application domain whose requirements are volatile, uncertain, and often derived from evolving research questions, hypothesized models, and experimental protocols rather than from stable stakeholder-facing specifications (Bajraktari et al., 2024). The proposed response is a lightweight, continuous approach centered on explicit stakeholder management, reproducibility as a first-class quality requirement, and methods that bridge research questions to software requirements through customized SRS templates, user stories and epics, goal-oriented requirements engineering, and a “research owner” role that maintains the research vision while prioritizing software work (Bajraktari et al., 2024).

4. Institutionalization, communities, and professional roles

A major theme in RSS is that software sustainability is inseparable from the sustainability of the people and institutions that produce software. Research Software Engineers face challenges distinct from those of traditional developers, including ill-defined and evolving requirements, dependence on short-term funding, and academic incentive systems that prioritize publications and grants over software outputs (Carver et al., 2021). Professional associations therefore function as a structural pillar of RSS by providing community, advocacy, and resources. The literature notes a global rise of RSE associations in at least eight countries or regions, with US-RSE emphasizing peer networks, advocacy, and materials to support the establishment and expansion of RSE groups (Carver et al., 2021).

National institutes are another major mechanism. The UK Software Sustainability Institute, established in 2010 with EPSRC funding, collaborated with more than 70 research groups, produced over 80 guides, built a fellowship cohort of 150 advocates, partnered with The Carpentries to build an instructor base of more than 350, and delivered training to 7,500 learners at more than 50 organizations (Katz et al., 2021). URSSI in the United States and AuSSI in Australia extend this institute model through community building, policy work, training, and FAIR-aligned agendas (Katz et al., 2021). These efforts are described as complementary mechanisms for advancing RSS by codifying best practices, changing incentives, and professionalizing maintenance work (Katz et al., 2021).

University and laboratory organizational models further show how RSS is embedded in practice. Manchester’s RSDS group, Illinois’ NCSA ISDA group, and Notre Dame’s CRC Software Development Group represent different institutional approaches to central RSE capacity, funding, and governance (Katz et al., 2019). Princeton’s central RSE group uses a partnership model in which RSEs work long-term with designated academic departments, institutes, centers, consortia, or PIs, with approximately 85% of effort devoted to partner-prioritized projects and 15% to group activities and professional development (Cosden, 2022). Sandia’s full-spectrum RDD department integrates software engineering research, embedded development, and DevOps or IT service management within a single unit (Milewicz et al., 2020).

A further institutional proposal comes from bioimage analysis, where the literature argues for embedding teams of RSEs within imaging and image analysis core facilities. In that model, user-driven requirements, direct usability feedback, sustained maintenance, interoperability work, and open-source stewardship are made part of the facility mission, alongside dependency management, test-driven development, continuous integration, and good documentation (Deschamps et al., 2023). This suggests that RSS matures most readily when software work is recognized as a first-class institutional function rather than as an informal by-product of individual projects.

5. Evaluation, FAIRness, and reproducibility infrastructures

Evaluation is central to RSS because it converts software from an invisible means into an assessable scholarly output. One influential framework is the CDUR procedure, a four-step protocol consisting of Citation, Dissemination, Use, and Research (Gomez-Diaz et al., 2021). Citation establishes software as a well-identified research output; Dissemination evaluates access and licensing; Use assesses usability, documentation, tests, versioning, and reproducibility; and Research places the software within its scientific context and measures impact, uptake, and related publications (Gomez-Diaz et al., 2021).

This evaluative perspective aligns with FAIR and Open Science. The literature argues that research software should be both archived for reproducibility and actively maintained for reusability, and that good scientific practice generally requires research software to be open source, with only rare exceptions kept closed (Hasselbring et al., 2019). Recommended mechanisms include DOIs via Zenodo, metadata through CFF and CodeMeta, long-term preservation through Software Heritage, containers for execution environments, and artifact evaluation processes that support repeatability, replicability, reproducibility, and reusability (Hasselbring et al., 2019).

RSS also develops practical reproducibility workflows. In Computational Science and Engineering, a publication-timed workflow ties software testing, result visualization, and periodic cross-linking of software, data, and publications to milestones in the publication process (Maric et al., 2022). Its pragmatic definition of research software quality is the ability to quickly find the publication, data, and software related to a published research idea, quickly reproduce results, understand or re-use a method, and extend it with new research ideas (Maric et al., 2022).

At ecosystem scale, FAIRness and metadata quality are increasingly studied through observatory infrastructures. The Software Observatory at OpenEBench aggregates metadata from multiple registries and repositories, producing a final integrated dataset of 45,334 unique software records and providing FAIRsoft-based dashboards and tool-level guidance (Pico et al., 7 Oct 2025). Related work on operationalizing research software for supply chain security argues that empirical RSS studies require explicit taxonomies for actor unit, supply chain role, research role, and distribution pathway in order to make definitions and sampling strategies comparable across studies (Kalu et al., 28 Jan 2026). Together, these efforts move RSS toward a more measurable and auditable empirical field.

6. Sustainability, technical debt, security, and open questions

Sustainability remains one of the field’s persistent concerns. Research software often grows organically from a thesis, a paper, or a grant deliverable, is maintained by graduate students and postdocs as “itinerant laborers,” and must adapt continuously to changing dependencies and platforms (Katz et al., 2021). This creates both organizational fragility and technical fragility, especially where maintenance is rarely explicitly funded and where turnover leads to software redevelopment rather than reuse (Carver et al., 2021).

Recent work has sharpened this concern through technical debt analysis. A mixed-methods study examining 28,680 self-admitted technical debt comments across nine research software projects identifies nine types of debt, including a new category, Scientific Debt, which captures threats to scientific validity arising from embedded assumptions, translation gaps, numerical approximations, missing edge cases, or outdated theory (Ernst et al., 20 Mar 2026). The same study reports project-level Scientific Debt prevalence such as 14.43% in CESM, 11.16% in Elmer, and 9.21% in Firedrake, and argues that research-specific debt is shaped by artifacts as boundary objects, science and organizational goals, people, and complexity (Ernst et al., 20 Mar 2026). This suggests that RSS must analyze not only software defects and maintainability costs, but also the ways code can compromise the validity of scientific claims.

Security has become another explicit RSS concern. A 2025 study of 3,248 high-quality, largely peer-reviewed research software repositories using the OpenSSF Scorecard finds a generally weak security posture with an average aggregate score of 3.50 out of 10, far below the recommended adequacy threshold of 7 (Hegewald et al., 5 Aug 2025). Important practices such as signed releases, restrictive token permissions, branch protection, code review, automated dependency updates, and security policies are rarely implemented, even though compromised research software can alter results, exfiltrate data, and undermine scientific integrity and reproducibility (Hegewald et al., 5 Aug 2025). The paper therefore recommends low-effort mitigations such as branch protection rules, least-privilege workflow tokens, and signed releases (Hegewald et al., 5 Aug 2025).

Open questions remain at the level of both classification and method. The metascience literature notes that RSS’s status depends on whether metascience is defined broadly or narrowly, and that without explicit attention to science’s systemic structures RSS may remain adjacent rather than fully meta (Eisinger et al., 16 Sep 2025). Other papers point to the rapid integration of artificial intelligence in software development, the need for better methods to identify cause and effect in team practices, the challenge of defining reproducibility operationally for research software, and the emergence of “metascience of software” studies as indications that the field is still consolidating its boundaries and methods (Eisinger et al., 16 Sep 2025, Heroux, 2022, Bajraktari et al., 2024). The common conclusion is stable: applying scientific rigor to research software is necessary if the tools of discovery are to meet the standards of the discoveries themselves (Eisinger et al., 16 Sep 2025).