Entropy Rounding in Optimization

• Entropy rounding is a set of techniques in combinatorial optimization that leverages entropy measures to convert fractional solutions into integral ones while controlling errors.
• The method employs randomized procedures, including half-coloring and SDP-based random walks, to achieve bounded discrepancy and subgaussian tail bounds.
• Applications span bin packing, train delivery, and TSP rounding, with improvements in integrality gaps and approximation guarantees through entropy regularization.

The entropy method for rounding is a collection of techniques in combinatorial optimization and approximation algorithms leveraging entropy-based arguments and distributions to round fractional solutions of linear relaxations to integral solutions with provable guarantees. These methods capitalize on low-entropy structure or entropy-regularized objectives to enable efficient and controlled rounding, addressing longstanding challenges such as achieving small discrepancy or tight integrality gaps. The approach encompasses randomized, sampling, and message-passing variants, each grounded in rigorous probabilistic or information-theoretic foundations.

1. Foundations: Δ-Approximate Entropy and the Entropy Method

Let $A \in \mathbb{R}^{n \times m}$, and consider rounding a fractional solution $x \in [0,1]^m$ to an integral vector $y$ such that $A x \approx A y$ and the linear objective distortion $c \cdot y - c \cdot x$ is tightly controlled. The entropy method is based on the notion of Δ-approximate entropy of $A$ (Rothvoss, 2011). For Δ = $(\Delta_1, \ldots, \Delta_n)$:

$$H_\Delta(A) \equiv \min_{f_1, \ldots, f_n} H(f_1(\chi), \ldots, f_n(\chi))$$

where each $f_i$ satisfies $|A_i(\chi) - f_i(\chi)| \leq \Delta_i$ for all $x \in [0,1]^m$0, and $x \in [0,1]^m$1 denotes entropy when $x \in [0,1]^m$2 is uniform on $x \in [0,1]^m$3.

The key insight, originating from Beck's entropy method in discrepancy theory, is that if the Δ-entropy is sublinear in $x \in [0,1]^m$4, then a significant fraction of colorings (signings) of variables must fall into a small set of rounded outcomes. The pigeonhole principle can then be used to extract a low-discrepancy coloring and, via iterative half-coloring and bit-peeling, produce an integral solution retaining key properties of the original fractional solution (Rothvoss, 2011).

2. Entropy-Based Rounding Algorithms

Given a fractional vector $x \in [0,1]^m$5 and matrices $x \in [0,1]^m$6, the entropy method builds a randomized rounding procedure (Rothvoss, 2011):

• Iteratively, for each dyadic bit of $x \in [0,1]^m$7, identify coordinates with the current bit “on” and find a half-coloring $x \in [0,1]^m$8 such that $x \in [0,1]^m$9 and each half-coloring changes at least half of the variables in the subset.
• Flip the coloring sign randomly to maintain unbiasedness.
• Update the solution; after all phases, $y$0 is integral.

The process guarantees bounded discrepancy: $y$1 and subgaussian tail bounds (with high probability): $y$2 See [(Rothvoss, 2011), Theorem (Entropy Rounding)] for full statements.

To make the method constructive, Bansal's SDP-based framework is employed. Each rounding step solves a feasibility SDP encoding the Δ-bounds, and a random walk is performed in the constructed vector space, ensuring guaranteed freezing of variables and discrepancy control in expected polynomial time (Rothvoss, 2011).

3. Entropy and Max-Entropy Sampling in Polytope Rounding

In problems such as rounding half-integral solutions to the subtour-elimination LP for TSP, entropy-based methods are used not just via low entropy, but by directly sampling from max-entropy distributions with explicit marginals (Gupta et al., 2021). The maximum-entropy distribution on structures such as spanning trees of a graph (with given edge marginals $y$3) takes the form: $y$4 where the weights $y$5 are tuned so that $y$6. This distribution is Strongly Rayleigh, enjoying negative correlation and concentration properties critical to rounding analyses (Gupta et al., 2021).

Sampling from such distributions (via either combinatorial algorithms for spanning trees or matroid-intersection based methods) allows for randomized rounding with precise marginal guarantees. The algorithm interleaves matroid-intersection samplers (which exploit combinatorial structure) and max-entropy samplers (which exploit negative correlation) to achieve improved “evenness” (i.e., edges becoming “even-at-last” for Eulerian circuit construction), yielding better integrality gap bounds than prior deterministic or randomized rounding approaches (Gupta et al., 2021).

4. Entropy-Regularized Rounding in Message Passing

Entropy regularization is also used to transform linear relaxations into strongly convex smooth objectives, facilitating the design of efficient message passing and rounding algorithms (Lee et al., 2019). Given a graphical model with variables $y$7 and potentials $y$8, the entropy-regularized LP relaxation is:

$y$9

where $A x \approx A y$0 is the total Shannon entropy over pseudo-marginals. The solution is unique and the objective is smooth.

The message-passing algorithm (EMP—Editor’s term) involves cyclic (EMP-cyclic) or greedily scheduled (EMP-greedy) Bregman projections onto edge-to-vertex consistency constraints in the pairwise local polytope. Each projection (analogous to Sinkhorn scaling) strictly improves a global potential, and the sequence of projections converges rapidly: $A x \approx A y$1 iterations to drive marginalization errors below $A x \approx A y$2 (Lee et al., 2019).

After convergence, μ is almost feasible. A simple “argmax” rounding: $A x \approx A y$3 recovers the integral MAP assignment under suitable parameters on regularization and feasibility ($A x \approx A y$4), given LP tightness and solution uniqueness. Explicit polynomial iteration bounds and parameter schedules are derived (see Theorem statements in (Lee et al., 2019)).

5. Main Theoretical Guarantees and Error Bounds

Across the entropy rounding method variants, the central theoretical elements include:

• Bounded expected objective error: Where fractional and rounded solutions are close in cost, with deviations controlled by entropy and structure-dependent constants (Rothvoss, 2011, Lee et al., 2019, Gupta et al., 2021).
• Discrepancy and tail bounds: Rounding error in constraints is strongly concentrated, with subgaussian or exponential tails.
• Polynomial-time implementation: The algorithms (via SDP, random walk, message passing, or combinatorial samplers) are all implementable in polynomial time.
• Integrality gap improvements: Through sharper event coupling (e.g., “even-at-last” edges) and flow-based charging, entropy-based rounding has achieved integrality gaps below longstanding barriers (e.g., in TSP, below 1.5 for half-integral solutions (Gupta et al., 2021)).

The following table summarizes method classes and settings:

Method Class	Key Tool	Core Guarantee
Discrepancy-based	Δ-approx. entropy + half-coloring	$A x \approx A y$5 discrepancy
Max-Entropy Sampling	Strongly Rayleigh distributions	Exact marginals, negative correlation
Entropy-regularized	Strong convexity via entropy	Unique optimum, smooth convergence

6. Flagship Applications

Applications of entropy-based rounding include:

• Bin Packing with Rejection: By building a low-entropy cumulated-pattern matrix and exploiting random-walk entropy rounding, Rothvoß achieved a polynomial-time $A x \approx A y$6 algorithm, with all deficits in large and small items repaired using a bounded additive term (Rothvoss, 2011).
• Train Delivery: Items grouped by positional classes are rounded using the entropy method, controlling discrepancies via μ-weights and extra runs, yielding a running time polynomial in $A x \approx A y$7 and achieving a cost $A x \approx A y$8 (Rothvoss, 2011).
• TSP Half-Integral Rounding: Recent advances interleaving matroid-intersection and max-entropy samplers have improved the integrality gap to approximately 1.4983 for half-integral Held–Karp solutions (Gupta et al., 2021), strengthening earlier work which relied solely on max-entropy sampling.

7. Connections and Extensions

The entropy method for rounding interfaces with several adjacent fields:

• Discrepancy Theory: Many results trace to Beck’s entropy method and Spencer's work, connecting coloring and partitioning to entropy minimization.
• SDP and Random Walks: Bansal’s constructive approach situates entropy-method rounding as a general algorithmic paradigm, accessible in practical settings (Rothvoss, 2011).
• Smooth Message Passing: Entropy-regularized relaxations enable global smoothness and rapid convergence, with direct implications for probabilistic graphical models and MAP inference (Lee et al., 2019).

A plausible implication is that further interaction between entropy-minimization principles, negative correlation structures (Strongly Rayleigh measures), and combinatorial constructs (matroid intersection, Bregman projections) can yield sharper and more broadly applicable rounding methods.

• (Rothvoss, 2011) "The Entropy Rounding Method in Approximation Algorithms" (Rothvoß, 2011)
• (Lee et al., 2019) "Convergence Rates of Smooth Message Passing with Rounding in Entropy-Regularized MAP Inference" (2019)
• (Gupta et al., 2021) "Matroid-Based TSP Rounding for Half-Integral Solutions" (Gupta et al., 2021)