Performance Metrics for Multi-Objective Optimisation Algorithms Under Noise

Abstract: This paper discusses the challenge when evaluating multi-objective optimisation algorithms under noise, and argues that decision maker preferences need to be taken into account. It demonstrates that commonly used performance metrics are problematic when there is noise, and proposes two alternative performance metrics that capture selection error of the decision maker due to mis-estimating a solution's true fitness values.