Information-theoretic minimax and submodular optimization algorithms for multivariate Markov chains

Abstract: We study an information-theoretic minimax problem for finite multivariate Markov chains on $d$-dimensional product state spaces. Given a family $\mathcal B={P_1,\ldots,P_n}$ of $\pi$-stationary transition matrices and a class $\mathcal F = \mathcal{F}(\mathbf{S})$ of factorizable models induced by a partition $\mathbf S$ of the coordinate set $[d]$, we seek to minimize the worst-case information loss by analyzing $$\min_{Q\in\mathcal F}\max_{P\in\mathcal B} D_{\mathrm{KL}}^{\pi}(P|Q),$$ where $D_{\mathrm{KL}}^{\pi}(P|Q)$ is the $\pi$-weighted KL divergence from $Q$ to $P$. We recast the above minimax problem into concave maximization over the $n$-probability-simplex via strong duality and Pythagorean identities that we derive. This leads us to formulate an information-theoretic game and show that a mixed strategy Nash equilibrium always exists; and propose a projected subgradient algorithm to approximately solve the minimax problem with provable guarantee. By transforming the minimax problem into an orthant submodular function in $\mathbf{S}$, this motivates us to consider a max-min-max submodular optimization problem and investigate a two-layer subgradient-greedy procedure to approximately solve this generalization. Numerical experiments for Markov chains on the Curie-Weiss and Bernoulli-Laplace models illustrate the practicality of these proposed algorithms and reveals sparse optimal structures in these examples.

• The paper introduces an information-theoretic minimax formulation that recasts weighted KL divergence minimization as a concave maximization problem over the probability simplex.
• The proposed projected subgradient algorithm achieves an O(t⁻¹/²) convergence rate while efficiently updating weight vectors in high-dimensional stochastic systems.
• Submodular optimization techniques reveal sparse optimal structures and interpretable partitions, as validated by numerical experiments on benchmark models.

Information-Theoretic Minimax and Submodular Optimization Algorithms for Multivariate Markov Chains

This paper presents a comprehensive framework addressing minimax optimization and submodularity in the context of multivariate Markov chains. By leveraging information-theoretic concepts, the authors develop algorithms that efficiently tackle high-dimensional stochastic systems through approximation and optimization strategies. This exploration is relevant for fields like stochastic modeling, MCMC, and interacting particle systems. The work is structured to accommodate a robust analysis of transition matrices under a minimax criterion and reveals sparse optimal structures via submodular optimization techniques.

Information-Theoretic Minimax Problem

The starting point of the paper is the formulation of a minimax optimization problem for Markov chains on $d$-dimensional product state spaces. The problem involves minimizing the worst-case Kullback-Leibler (KL) divergence, weighted by a given stationary distribution $\pi$, between a family $\mathcal{B}$ of transition matrices and a class $\mathcal{F}$ of factorizable models. This can be expressed as:

$$\min_{Q \in \mathcal{F}} \max_{P \in \mathcal{B}} (P \| Q),$$

where $(P \| Q)$ denotes the $\pi$-weighted KL divergence. The authors recast this problem into a concave maximization problem over the $n$-probability-simplex.

By using strong duality and newly-derived Pythagorean identities, the authors establish the equivalence between the original minimax problem and a dual formulation. This critical transformation allows the minimax problem to be interpreted as an information-theoretic game with an existing mixed strategy Nash equilibrium.

Algorithms for Subgradient Optimization

The authors propose a projected subgradient algorithm to solve the minimax problem. This method iteratively updates the weights over the probability simplex and projects them back onto the feasible set, achieving convergence with a guaranteed $\mathcal{O}(t^{-1/2})$ rate, where $t$ is the number of iterations. The algorithm relies on explicit supergradients derived for objective values, which are essential for guiding efficient updates.

Figure 1: Curie-Weiss model.

To further address the complexity of multivariate Markov chains, the authors convert the minimax problem into an orthant submodular function. This approach gives rise to a two-layer subgradient-greedy algorithm to solve the generalized max-min-max submodular optimization problem. This layered strategy involves alternating between optimizing the weight vector via subgradient descent and updating partitions via greedy selection.

Figure 2: Curie-Weiss model.

Experiments and Numerical Evaluations

The framework is validated through numerical experiments on Curie-Weiss and Bernoulli-Laplace models. These experiments investigate the practical deployment of the proposed algorithms in scenarios characterized by complex multivariate transitions. The results highlight sparse optimal mixtures concentrating on a few extrema such as base and accelerated variants of Markov chains, demonstrating the efficacy of the algorithms in real-world applications.

Figure 3: Curie-Weiss model (d=8).

The experiments also exhibit the interpretability of partitions obtained through the submodular optimization process, emphasizing the potential of these methods to capture dominant dependencies while controlling information loss.

Figure 4: Curie-Weiss model.

Conclusion

This paper provides a sophisticated toolkit for tackling optimization problems in multivariate Markov chains through minimax and submodular approaches. The algorithms developed offer robust and interpretable solutions with applications in a variety of complex high-dimensional stochastic environments. Future work could extend these methods to wider classes of models and explore additional optimization paradigms that leverage similar theoretical foundations.

Through rigorous analysis and a suite of computational techniques, the authors set a foundation that promises enhanced understanding and management of stochastic systems characterized by multivariate interactions. The presented algorithms are anticipated to be adaptable for ongoing improvements in areas like machine learning and statistical data science, where similar structural challenges abound.

Figure 5: Curie-Weiss model.

Figure 6: Curie-Weiss model.