On the Correctness of the Generalized Isotonic Recursive Partitioning Algorithm (2401.04847v2)
Abstract: This paper presents an in-depth analysis of the generalized isotonic recursive partitioning (GIRP) algorithm for fitting isotonic models under separable convex losses, proposed by Luss and Rosset [J. Comput. Graph. Statist., 23 (2014), pp. 192--201] for differentiable losses and extended by Painsky and Rosset [IEEE Trans. Pattern Anal. Mach. Intell., 38 (2016), pp. 308-321] for nondifferentiable losses. The GIRP algorithm poseses an attractive feature that in each step of the algorithm, the intermediate solution satisfies the isotonicity constraint. The paper begins with an example showing that the GIRP algorithm as described in the literature may fail to produce an isotonic model, suggesting that the existence and uniqueness of the solution to the isotonic regression problem must be carefully addressed. It proceeds with showing that, among possibly many solutions, there indeed exists a solution that can be found by recursive binary partitioning of the set of observed data. A small modification of the GIRP algorithm suffices to obtain a correct solution and preserve the desired property that all the intermediate solutions are isotonic. This proposed modification includes a proper choice of intermediate solutions and a simplification of the partitioning step from ternary to binary.
- An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1124–1137.
- COSMO: A conic operator splitting method for large convex problems. In Proc. 18th Euro. Control Conf. (ECC), pp. 1951–1956.
- Generalized isotonic regression. J. Comput. Graph. Statist. 23(1), 192–210.
- Efficient regularized isotonic regression with application to gene–gene interaction search. Ann. Appl. Stat. 6(1), 253–283.
- Maxwell, W. L. and J. A. Muckstadt (1985). Establishing consistent and realistic reorder intervals in production-distribution systems. Oper. Res. 33(6), 1316–1341.
- Intervalrank: isotonic regression with listwise and pairwise constraints. In Proc. 3rd Int. Conf. Web Search and Data Mining, pp. 151–160. ACM.
- Consistent probabilistic outputs for protein function prediction. Genome Biol. 9(1), 1–19.
- Isotonic modeling with non-differentiable loss functions with application to lasso regularization. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 308–321.
- Picard, J.-C. (1976). Maximal closure of a graph and applications to combinatorial problems. Manag. Sci. 22(11), 1268–1272.
- Order Restricted Statistical Inference. Probability and Statistics. Wiley.
- The reduced monotonic regression method. J. Amer. Statist. Assoc. 92(437), 128–135.
- Least squares isotonic regression in two dimensions. J. Optim. Theory Appl. 117(3), 585–605.
- Contrast invariant SNR and isotonic regressions. Int. J. Comput. Vis. 127(8), 1144–1161.
- Query-level learning to rank using isotonic regression. In Proc. 46th Allerton Conf. Communication, Control, and Computing, pp. 1108–1115. IEEE.