Multi-Objective Submodular Maximization with Differential Privacy

Abstract: In this paper, we study multi-objective submodular maximization (MOSM) subject to a cardinality constraint under differential privacy (DP). Specifically, we aim to select a set of at most $k \in \mathbb{Z}_{+}$ elements to maximize the minimum of $d > 1$ monotone submodular functions while satisfying $\varepsilon$-DP. Although extensive studies have been conducted on both differentially private single-objective submodular maximization on sensitive data and non-private MOSM, to the best of our knowledge, there has not yet been any prior work on MOSM with DP. We propose two novel algorithms: the first extends the classic greedy algorithm and the second employs a truncation technique, both of which are integrated with DP mechanisms for privacy protection and achieve approximation guarantees for MOSM. Finally, we conduct numerical experiments on two submodular maximization applications, namely maximum coverage and facility location, in multi-objective settings to validate the efficacy and efficiency of our proposed algorithms.

• The paper introduces DP-MultiGreedy and DP-Bicriteria algorithms that achieve multi-objective submodular maximization under differential privacy constraints.
• Both algorithms employ Laplace and exponential mechanisms to balance privacy budgets and approximation accuracy for complex set functions.
• Empirical results on maximum coverage and facility location illustrate that DP-Bicriteria scales effectively for larger objectives, advancing privacy-aware optimization.

Multi-Objective Submodular Maximization under Differential Privacy: A Technical Analysis

Problem Formulation and Motivation

The paper addresses the multi-objective submodular maximization (MOSM) problem subject to a cardinality constraint and a differential privacy (DP) requirement. Specifically, for $d > 1$ monotone submodular functions $f_1, \ldots, f_d$, the task is to select up to $k$ elements from a ground set $V$ to maximize $\min_{j \in [d]} f_j(S)$ while ensuring $\varepsilon$-DP for individual-level sensitive data (such as in clustering, facility location, or summarization). This setting generalizes both classic submodular optimization and standard DP-SM, motivating novel algorithmic approaches as prior DP work addressed only single-objective settings, whereas non-private MOSM does not guarantee privacy.

Algorithmic Contributions

Two algorithms are proposed: DP-MultiGreedy (a DP-aware greedy method) and DP-Bicriteria (a DP variant of the Saturate algorithm incorporating bicriteria relaxations). Both employ careful composition of Laplace and exponential mechanisms to enforce DP, and both yield explicit theoretical approximation guarantees.

DP-MultiGreedy

This method generalizes DP-Greedy to MOSM. It first allocates privacy budget among the $d$ objectives, running DP-Greedy independently for each to build $d$ partial solutions of size approximately $k/d$, uniting them to form an initial solution. The remaining budget is distributed among further greedy rounds that augment the set with elements maximizing the minimum objective (identifying the weakest objective via perturbed Laplace estimates, then using the exponential mechanism for selection).

Figure 1: Solution quality as a function of privacy budget $\varepsilon$ for maximum coverage, showing DP-MultiGreedy and DP-Bicriteria performance trends.

The approximation guarantee is quantified as:

$f_1, \ldots, f_d$0

with probability at least $f_1, \ldots, f_d$1, where $f_1, \ldots, f_d$2.

DP-Bicriteria

DP-Bicriteria adapts the Saturate approach to the differential privacy context via binary search over a candidate threshold $f_1, \ldots, f_d$3. For each $f_1, \ldots, f_d$4, a "truncated" single-objective submodular function $f_1, \ldots, f_d$5 is maximized under a relaxed budget ($f_1, \ldots, f_d$6 elements). DP-Greedy and the Laplace mechanism are then deployed to privately estimate $f_1, \ldots, f_d$7, informing the search. The process yields a bicriteria approximation of the form:

$f_1, \ldots, f_d$8

with a solution $f_1, \ldots, f_d$9, and overall $k$0-DP.

Figure 2: Solution quality as a function of the cardinality constraint $k$1 for maximum coverage; DP-MultiGreedy trails as $k$2 increases.

Theoretical Implications

The paper proves that both DP-MultiGreedy and DP-Bicriteria are $k$3-DP by explicit application of the DP composition theorem. The approximation bounds make explicit the dependence on the privacy parameter $k$4, solution size $k$5, and number of objectives $k$6. Notably, DP-MultiGreedy degrades in performance for larger $k$7 or when $k$8, due to both privacy budget fragmentation and combinatorial limitations.

DP-Bicriteria, via threshold truncation and solution size relaxation, offers superior performance when $k$9 is large, albeit with a bicriteria guarantee (the selected set may violate the cardinality bound but controls utility loss and privacy rigorously). Such bicriteria results are fundamentally unavoidable unless P = NP, as shown in non-private work.

Empirical Results

Comprehensive empirical validation is performed on canonical submodular maximization benchmarks: Maximum Coverage and Facility Location, across public datasets (DBLP, Flickr, FourSquare, Gowalla), for varying $V$0, $V$1, and $V$2. Results are compared with non-private baselines (MultiGreedy, Saturate, MWU, GeneralizedGreedy):

• DP-MultiGreedy achieves parity with non-private solutions for small $V$3 and moderate $V$4, but its utility deteriorates rapidly for larger $V$5 due to DP noise and inability to simultaneously optimize all objectives.
• DP-Bicriteria shows robust utility scaling with $V$6 and $V$7, outperforming DP-MultiGreedy for larger $V$8, at the cost of slightly larger solution sets.

For instance, in maximum coverage (Figure 1 and Figure 2), DP-MultiGreedy tracks the non-private baseline when $V$9 and $\min_{j \in [d]} f_j(S)$0, while DP-Bicriteria approaches optimality for $\min_{j \in [d]} f_j(S)$1 when $\min_{j \in [d]} f_j(S)$2. Increases in $\min_{j \in [d]} f_j(S)$3 (Figure 2) benefit both algorithms, but noise accumulates as $\min_{j \in [d]} f_j(S)$4 grows or $\min_{j \in [d]} f_j(S)$5 shrinks.

Figure 3: Solution quality as the number of objectives $\min_{j \in [d]} f_j(S)$6 increases in maximum coverage, highlighting the growing advantage of DP-Bicriteria as $\min_{j \in [d]} f_j(S)$7 increases.

In facility location, both algorithms require larger $\min_{j \in [d]} f_j(S)$8 to close the gap with non-private baselines, reflecting greater difficulty under more complex utility landscapes.

Practical and Theoretical Implications

This study definitively demonstrates that multi-objective submodular maximization can be achieved with provable DP, within explicit approximation factors that gracefully degrade with respect to privacy rigor and problem size. The proposed algorithms bridge a significant gap, providing practical methods for privacy-sensitive applications in federated learning, fair clustering, or private summarization, where multiple utility objectives and privacy must be reconciled.

Theoretically, the results clarify the inherent tradeoff curves between privacy loss ($\min_{j \in [d]} f_j(S)$9), solution quality, number of objectives, and the nature of bicriteria relaxations. The explicit finite-sample guarantees under DP consumption and the clear identification of where greedy and bicriteria approaches dominate are key advances.

Future Outlook

Important open questions remain, most prominently the extension of these DP MOSM algorithms to more general constraint structures (e.g., matroid, knapsack), as well as potential for continuous (non-greedy) DP mechanisms to improve approximation constants or mitigate noise accumulation. High-dimensional and streaming MOSM under DP, as well as tighter characterizations of lower bounds, are expected future directions.

Conclusion

The paper advances differentially private algorithmic design for multi-objective submodular maximization. DP-MultiGreedy and DP-Bicriteria establish new provable baselines, both theoretically and empirically, for privacy-preserving optimization of complex set functions with multiple stakeholders or fairness constraints. Their techniques can be foundational for future research on private combinatorial optimization, especially where multi-criteria and privacy guarantees must be balanced.