Create and explore imaginative 3D worlds with our new product Dreamfield (opens in a new tab)

Explainable product-costing design and optimization

Develop concrete designs and optimization techniques for explainable product costing in data pipelines that maintain detailed maps of monetary-cost propagation from upstream data entities to final data products while addressing the resulting storage and communication overhead.

Background

The paper’s CostApp algorithm computes exact manufacturing costs for final data products in large, cyclic data pipelines, but it does not provide a breakdown showing how individual upstream entities contribute to each product’s cost. The proposed direction is to maintain a map that tracks monetary-cost propagation through the pipeline, thereby attributing each product’s total cost to its constituent components and enabling fine-grained cost optimization.

Such explainability would introduce additional storage and communication overhead. The paper therefore leaves unresolved both the concrete system design and the optimization of methods for producing these cost breakdowns at scale. This is an explicit future-work problem rather than a result established by the paper.

References

This method yields a detailed breakdown of each data product, attributing its total cost to its constituent components and enabling explainability. This idea will incur additional storage and communication overhead. We leave the concrete design and optimization of solutions to this explainability as future work.

— Product Costing in Data Pipelines  (2610.21352 - Peng et al., 15 Oct 2026) in Appendix, Section “Extended Future Work,” paragraph “(1) Explainable product costing in data pipelines”