Iterative Supervised Learning for Regression with Constraints

Abstract: Regression in supervised learning often requires the enforcement of constraints to ensure that the trained models are consistent with the underlying structures of the input and output data. This paper presents an iterative procedure to perform regression under arbitrary constraints. It is achieved by alternating between a learning step and a constraint enforcement step, to which an affine extension function is incorporated. We show this leads to a contraction mapping under mild assumptions, from which the convergence is guaranteed analytically. The presented proof of convergence in regression with constraints is the unique contribution of this paper. Furthermore, numerical experiments illustrate improvements in the trained model in terms of the quality of regression, the satisfaction of constraints, and also the stability in training, when compared to other existing algorithms.