Gross Substitutes Reward Functions

• Gross substitutes reward functions are set functions defined by the property that increasing prices for some items does not lower the marginal value of others.
• They enable efficient algorithmic solutions in combinatorial auctions, contract theory, and resource allocation, ensuring convergence to equilibrium states.
• Advanced constructs like tree-concordance and Ultra properties extend their use while open challenges remain in universal representation and approximation limits.

Gross substitutes reward functions are a distinguished class of set functions arising in combinatorial optimization, economics, and mechanism design. They capture the property that the incremental value of adding a good or action to a bundle does not decrease when other goods become more expensive, formalizing the economic intuition that goods are substitutes in the agent’s “demand.” Mathematically, gross substitutes functions possess structural properties that enable efficient algorithmic solutions in combinatorial auctions, contract theory, multi-agent decision-making, and resource allocation. The tractability and robustness of optimization problems, such as welfare maximization and optimal contracting, often depend crucially on the presence of the gross substitutes property.

1. Mathematical Definition and Core Properties

A set function $f: 2^M \rightarrow \mathbb{R}$ is gross substitutes if for any price vectors $p \leq q \in \mathbb{R}^M$ and any set $S \in D(p)$ (where $$D(p) = \arg\max_{T \subseteq M} \{ f(T) - \sum_{j \in T} p_j \}$$), there exists $T \in D(q)$ such that $T \supseteq \{ j \in S : p_j = q_j \}$. This means that increasing the prices of some items does not induce the agent to drop items whose prices did not change.

Alternative characterizations for GS functions include:

• Marginal Monotonicity: For $S \subseteq T$ and $j \notin T$, $f(S \cup \{j\}) - f(S) \geq f(T \cup \{j\}) - f(T)$.
• Triplet Condition (Dobzinski et al., 2021):

$$\max\{ f(i,k|S) + f(j|S),\; f(j,k|S) + f(i|S) \} \geq f(i,j|S) + f(k|S),$$

where $f(A|B) = f(A \cup B) - f(B)$, for any $S$ and disjoint $i,j,k$.

Gross substitutes form a strict subclass of submodular functions (Dobzinski et al., 2021). Submodularity describes diminishing returns but does not guarantee the price-responsiveness and demand stability inherent in GS functions. GS sits within the hierarchy: $$\text{OXS} \subsetneq \text{GS} \subsetneq \text{Submodular} \subsetneq \text{XOS} \subsetneq \text{Subadditive}$$.

2. Construction and Structure of Gross Substitutes Functions

The construction problem asks whether every GS function can be represented as a positive linear combination of simple building blocks such as matroid rank functions. Shioura showed that many GS functions can be thus constructed, but it was an open problem whether this holds universally (Balkanski et al., 2018). The negative resolution demonstrates existence of GS functions (on a ground set of size five) that cannot be expressed as positive linear combinations of matroid rank functions. This is proved using a Farkas' lemma certificate: for some $v$ and vector $y$, $\langle v, y \rangle < 0$ while $\langle r, y \rangle \geq 0$ for all matroid rank functions $r$.

Necessary and sufficient conditions for the sum of two GS functions to remain GS are described via tree representations. Two GS functions are tree-concordant if their tree structures (encapsulating laminar families) are compatible; only in this case does $u+v$ remain GS. The “tree-concordant-sum” operation generalizes existing decomposition techniques and allows the aggregation of GS reward functions while maintaining substitutability.

For small ground sets ($n \leq 4$), every GS function is a convex combination of matroid rank functions, fully classified in (Balkanski et al., 2018). For larger ground sets, additional geometric and combinatorial operations (including the tree-concordant-sum) are necessary.

3. Algorithmic Implications and Tractability Frontiers

GS reward functions are pivotal for the tractable optimization of social welfare and contract design. With GS valuations, combinatorial auctions such as the Kelso–Crawford (ascending-price) auction converge to Walrasian equilibria (Duetting et al., 2021). The agent’s demand changes only polynomially many times as incentives (prices or contract parameters $\alpha$) are varied. The set of “critical values” $\alpha$—thresholds where demand sets change—is bounded by $O(n^2)$ in the context of additive costs, permitting polynomial-time optimal contract computation.

Contrastively, submodular functions may admit exponential numbers of critical values, and optimal contract computation becomes NP-hard (even for budget additive functions or coverage functions). The tractability “frontier” thus coincides with the GS property, not the broader class of submodular functions (Duetting et al., 2021, Dütting et al., 2023).

Recent advances (Feldman et al., 22 Jun 2025) clarify that Ultra functions (characterized by exchange inequalities) generalize GS and retain tractability; GS forms the intersection of Ultra and submodular classes. Efficient algorithms use exchange properties rather than relying exclusively on submodularity. Iterative exchange-based procedures correct deviations from optimality by swapping items, guided by Ultra conditions such as:

$$v(S) + v(S \cup \{i, j\}) \leq v(S \cup \{i\}) + v(S \cup \{j\})$$

for distinct $i, j \notin S$.

4. Approximation, Robustness, and Oracle Access

Welfare guarantees known for GS do not automatically extend to valuations only pointwise $\epsilon$-close to GS, unless further structural constraints (e.g., decreasing marginals, $\alpha$-submodularity) are imposed (Roughgarden et al., 2016). For example, “pointwise $\epsilon$-closeness” (i.e., $v(S) \leq \tilde{v}(S) \leq (1+\epsilon)v(S)$ for all $S$) fails to preserve the tractable welfare maximization property, even for arbitrarily small $\epsilon$. Approximating optimum welfare and simulating demand oracles using only value queries may require subexponential resources under mere pointwise closeness.

By strengthening the approximation model (requiring marginal-ε-closeness or using $\alpha$-submodularity), algorithm performance degrades gracefully in $\epsilon$: greedy algorithms and generalized auctions remain approximately optimal (e.g., greedy guarantees $v(S) \geq \frac{1-3\epsilon}{\alpha} v(S^*)$). When demand queries are allowed—and valuations are close to XOS—the configuration LP approach enables $(1-1/e-\epsilon)$-approximate welfare.

Symmetrization operations such as two-item max-symmetrization (“forcing” GS functions to respect symmetries) preserve the GS property and are crucial for approximation analysis (Dobzinski et al., 2021). The limits of approximability are nearly polylogarithmic in the number of items for budget additive valuations.

5. Applications in Mechanism Design, Learning, and Multi-Agent Systems

GS reward functions underpin the design of mechanisms guaranteeing the existence of Walrasian (competitive) equilibria in markets, combinatorial auctions, and contract theory (Duetting et al., 2021, Dütting et al., 2023). In principal-agent problems, GS reward structures enable robust, incentive-compatible contracts and allow for sensitivity analysis via enumeration of critical parameters.

In reinforcement learning, reward decompositions aligned with gross substitutes facilitate independently obtainable rewards: neural models parameterize additive decompositions such that each reward is maximized by distinct, non-interfering policies (Grimm et al., 2019). Empirically, such decompositions exhibit saturation (nearly all of the reward is assigned to one function per state) and non-overlapping visitation frequencies, reflecting substitute goods in economic theory.

Unified gross substitutes, generalizing GS to correspondences, are instrumental for monotone comparative statics in equilibrium analysis (Galichon et al., 2022). For supply correspondences $Q: P \rightrightarrows Q$, the unified GS condition imposes bracketed inequalities:

$$\begin{aligned} p_i \leq p'_i &\implies q_i \leq q^{\wedge}_i \quad \text{and} \quad q^{\vee}_i \leq q'_i \ p_i > p'_i &\implies q'_i \leq q^{\wedge}_i \quad \text{and} \quad q^{\vee}_i \leq q_i \end{aligned}$$

This structure guarantees that comparative statics (inverse mappings from outcomes to prices) preserve monotonicity and lattice structure, crucial for robust equilibrium analysis in matching markets and contract design.

6. Practical Reward Designs and Extensions

In high-dimensional control (e.g., robotics), robust, physically plausible behaviors can be achieved by substituting complex, hand-designed rewards with style rewards learned from demonstration data (Escontrela et al., 2022). Adversarial motion priors encode a style reward, combined additively with a task reward, supporting efficient optimization. This mirrors the substitutability of rewards: the agent can modulate between task achievement and style adherence, with the learned style acting as a clean substitute for explicit engineered reward terms.

Within Markov decision processes, the structuring of reward functions—including potential-based shaping and multiobjective formulations—influences sample complexity, learning efficiency, and policy safety (Dai, 2023). The maximum expected hitting cost ($\kappa(M)$)—a reward-sensitive parameter—refines complexity measures and regret bounds. Pareto-optimal reward decompositions rely on algebraic characterizations and cone optimization, and reward shaping can halve the hitting cost, accelerating learning.

7. Limitations and Open Problems

GS functions are not universally closed under natural operations (such as averaging or unconstrained addition), and construction from matroid rank functions is limited to small ground sets ($n \leq 4$) (Balkanski et al., 2018). For larger sets, more involved combinatorial structures must be considered (tree-concordance, laminar families). GS functions cannot approximate all submodular functions within a constant or even logarithmic factor; the lower bound is $\Omega\left(\frac{\log m}{\log \log m}\right)$ for $m$ items (Dobzinski et al., 2021). Furthermore, the tractability “frontier” has recently shifted from submodularity to the strictly broader Ultra class, prompting re-examination of algorithmic strategies for contract, mechanism, and reward design (Feldman et al., 22 Jun 2025).

Open problems include complete classification of GS functions over arbitrary ground sets, tight characterizations of tractability for broader classes (Ultra, monotone correspondences), and deeper integration of GS-based reward design frameworks into adaptive and robust multi-agent systems.