Disentangling epistemic and aleatoric uncertainty in Value Flows

Determine a principled method to disentangle epistemic uncertainty from aleatoric uncertainty within the Value Flows framework that estimates full return distributions via flow matching, providing separate, state-action–conditioned estimates of each uncertainty component from the learned flow-based return models.

Background

Value Flows models the full return distribution using flow-matching and derives an estimate of return variance to capture aleatoric uncertainty, which is then used to reweight the learning objective.

While the method leverages aleatoric uncertainty for prioritization, the authors explicitly note that separating epistemic uncertainty (model uncertainty) from aleatoric uncertainty in their current flow-based setup is unresolved.