Conditions for submodularity in minimax entropy feature selection

Determine the precise conditions under which the minimax entropy objective mapping a feature set ℱ to the entropy of its corresponding maximum entropy model S(P_ℱ) is submodular (i.e., exhibits diminishing reductions in entropy as features are added), thereby guaranteeing that the greedy algorithm for selecting features achieves a near‑optimal solution with the standard 1 − 1/e approximation bound.

Background

The paper proposes minimax entropy as a principle for selecting features that minimize the entropy of the maximum entropy model subject to matching empirical feature averages. Because a brute‑force search over feature sets is infeasible, the authors advocate a greedy algorithm, which is provably near‑optimal when the objective is submodular.

They note that for trees of pairwise correlations the greedy method finds the global optimum, but in general the problem’s submodularity is not established. Clarifying when the minimax entropy objective is submodular would provide theoretical guarantees for the greedy algorithm’s performance across broader model classes and data regimes.