BayesInsights: Modelling Software Delivery and Developer Experience with Bayesian Networks at Bloomberg

Abstract: As software in industry grows in size and complexity, so does the volume of engineering data that companies generate and use. Ideally, this data could be used for many purposes, including informing decisions on engineering priorities. However, without a structured representation of the links between different aspects of software development, companies can struggle to identify the root causes of deficiencies or anticipate the effects of changes. In this paper, we report on our experience at Bloomberg in developing a novel tool, dubbed BayesInsights, which provides an interactive interface for visualising causal dependencies across various aspects of the software engineering (SE) process using Bayesian Networks (BNs). We describe our journey from defining network structures using a combination of established literature, expert insight, and structure learning algorithms, to integrating BayesInsights into existing data analytics solutions, and conclude with a mixed-methods evaluation of performance benchmarking and survey responses from 24 senior practitioners at Bloomberg. Our results revealed 95.8% of participants found the tool useful for identifying software delivery challenges at the team and organisational levels, cementing its value as a proof of concept for modelling software delivery and developer experience. BayesInsights is currently in preview, with access granted to seven engineering teams and a wider deployment roadmap in place for the future.

• The paper introduces a Bayesian framework that integrates software delivery metrics and developer survey data into causal Bayesian Networks.
• It combines expert input with algorithmic structure learning to validate causal links and demonstrate real-time scenario analysis with median inference times below 40 ms.
• The study shows high practitioner approval, with over 95% finding the tool useful and 75% rating it easy to interpret for identifying delivery challenges.

Causal Modelling of Software Delivery and Developer Experience: An Analysis of BayesInsights

Introduction

"BayesInsights: Modelling Software Delivery and Developer Experience with Bayesian Networks at Bloomberg" (2603.29929) presents an integrated, data-driven approach to analyzing causal relationships among software engineering (SE) delivery metrics and developer experience (DevEx) using Bayesian Networks (BNs). The authors situate this work within the context of Bloomberg’s internal engineering practices, addressing the challenges of relating quantitative engineering outcomes (e.g., DORA metrics) and qualitative, human-centered aspects (e.g., satisfaction, time lost to obstacles). The BayesInsights tool operationalizes these relationships via interactive BN visualizations, offering practitioners actionable scenario analyses and root-cause exploration.

Network Structure Construction and Causal Modelling

The BayesInsights BNs are constructed using a hybrid methodology synthesizing evidence from established SE literature, expert surveys, and algorithmic structure learning. The process begins with the DORA model as the core DAG scaffold, augmenting it to account for internal Likert-scale survey data comprising over 2,000 responses to 20 items. Survey items are mapped to individual BN nodes, with causal directions hypothesized using Pearlian tiered reasoning and established BN idioms (cause-consequence, synthesis).

Expert input is central. A structured, directional survey elicits expert judgments on influence strength between 24 node pairs, using five-point scales transformed to weights for network pruning and refinement. Only relationships scoring above a 0.70 weighted threshold are included, with expert review introducing or removing links as necessary. The node–edge structure is further validated using score-based (Hill-Climbing, HC) and constraint-based (Peter-Clark, PC) structure learning. While algorithmic output supplements the expert model, only consistently supported and stability-checked (via bootstrap resampling) links are considered. Model selection is ultimately governed by BIC minimization, with the expert-augmented structure outperforming purely data-driven alternatives.

Conditional probability tables (CPTs) for the resulting DAGs are derived directly from the survey data, treating Likert responses as discrete states. Probabilities are estimated using marginal and joint frequency counts, with Bayesian-Dirichlet (BDeu) smoothing mitigating sparsity for low-frequency parent configurations.

The BayesInsights Tool: Implementation and Functionality

BayesInsights comprises a Django/Django Ninja backend (pgmpy for BN learning/inference) and a TypeScript/React Flow frontend for interactive visualization. Nodes (survey items/metrics) are represented by Chart.js bar plots of state distributions. Core functionality centers on evidence insertion ("what-if" analysis): users select observed/fixed outcomes for any node, and the network computes and visualizes posterior distributions for the entire BN, enabling forward and backward causal reasoning.

This approach provides a structural, rather than simple correlative, understanding of engineering trade-offs. For instance, the tool can isolate the ripple effect of improving CI/CD pipeline performance on deployment stability, or quantify how reducing time lost to obstacles alters the distribution of outcomes for team satisfaction and happiness.

Figure 1: The DevEx Bayesian Network in BayesInsights, showing learned causal dependencies among developer-centric factors and supporting evidence-based scenario analysis.

Evaluation: Performance and Practitioner Study

Performance evaluation reveals the tool is suitable for real-time, interactive analysis, with median inference response times below 40 ms for up to 50 concurrent users.

A qualitative study with 28 senior practitioners (24 responses; majority >10 years’ experience) underscores both the interpretability and utility of BayesInsights:

• 95.8% found it useful for identifying delivery challenges at both team and organizational levels.
• 75% deemed outputs easy to interpret; 83.3% could clearly visualize dynamics via "what-if" analysis.
• 79.2% would recommend the tool; 70.9% expressed trust in its outputs for DevEx and delivery analysis.
• 25% generated actionable performance improvements during short pilot sessions.

Suggestions for improvement centered on enhancing explainability (e.g., natural language summaries, results-first interface, confidence estimates for causal strengths) and better integration with team-level data, reinforcing the importance of transparency and usability in deploying causal ML tools in complex socio-technical contexts.

Related Work and Positioning

BNs have seen limited industrial uptake for SE delivery and DevEx, with past work mostly targeting software quality and effort estimation in restricted settings. The BayesInsights approach is notable for unifying pipeline metrics (DORA) and developer-focused factors (SPACE, survey constructs) under a single, causal, interactive framework. This contrasts with the more common use of propensity score matching or difference-in-differences methods in prior empirical SE work, which are often hampered by untested confounder assumptions and lack of actionable, user-facing outputs. By integrating human factors (focus, meaningful work), socio-technical environment variables (codebase understanding, technical debt), and process quality metrics, BayesInsights advances multi-level, cross-domain reasoning in industrial SE analytics.

Implications and Future Directions

The BayesInsights framework has both practical and theoretical implications:

• Operationalization of causal scenario analysis: Engineering teams can systematically predict, prioritize, and communicate the impact of process interventions or environment changes.
• Model interpretability and adoption: The hybrid model construction mitigates both naive data-only bias and domain-expert overfitting, providing a robust, auditable causal narrative for SE process management.
• Scalability to other SE domains: The demonstrated methodology can be adapted beyond Bloomberg (pending compatible survey and metric data), suggesting broader applicability in large-scale, data-rich SE organizations.

Future work should address scaling to heterogeneous teams with divergent toolchains, automating model re-fitting as processes evolve, integrating richer sources of sentiment or fine-grained behavioral telemetry, and moving toward explainable AI outputs for less technical user groups.

Conclusion

BayesInsights demonstrates the industrial viability of Bayesian causal modelling for joint analysis of software delivery and developer experience. Its hybrid approach to BN construction, rigorous practitioner-centered evaluation, and real-time scenario analysis bridge a longstanding gap between metric tracking and causal understanding in SE analytics. As SE organizations continue to seek actionable, interpretable, and scalable solutions for data-driven organizational improvement, tools built on these principles are poised to play a central role in both operational and strategic domains.