Cost of Structural Learning Under Censored Feedback: A Threshold-Bandit Approach

Abstract: In many multi-agent applications, tasks yield rewards only when executed by a coalition meeting an unknown size threshold; otherwise, feedback is fully censored. This censorship creates an identifiability problem: agents cannot distinguish stochastic failure from insufficient coordination. We formalize this setting as the Threshold-Activated Cooperative Multi-Armed Bandit (TAC-MAB) and analyze it under both centralized and decentralized coordination. We show that a centralized algorithm (C-TAC) achieves cumulative regret O(log T), decomposed into a structural-search term that captures the cost of resolving feasibility under censored feedback and a statistical-monitoring term for value estimation. We then introduce D-TAC, a decentralized event-triggered protocol in which agents synchronize only when their structural beliefs change. Empirically, D-TAC achieves a 23x reduction in communication relative to the centralized baseline while preserving feasibility alignment under conservative belief fusion. These results characterize the coordination cost of learning under censored feedback and show that near-centralized communication efficiency is achievable without continuous synchronization.

• The paper introduces TAC-MAB and C-TAC, showing that centralized threshold learning under censored feedback achieves gap-dependent O(log T) regret when feasible tasks satisfy p_k ≥ p_min > 0.
• The paper separates structural feasibility-search costs, which scale with threshold size and 1/p_min, from statistical reward estimation, while using exact knapsack planning as the oracle benchmark.
• The paper’s decentralized D-TAC protocol reduces communication by 23× in experiments—4,303 versus 100,000 messages—while achieving regret of 13,700 versus C-TAC’s 4,500, though it lacks a worst-case decentralization regret guarantee.

This paper formalizes a cooperative multi-agent learning problem in which task rewards are gated by unknown coalition-size thresholds, and feedback below threshold is fully censored. The authors, Ledford and Regli of the University of Maryland, introduce the Threshold-Activated Cooperative Multi-Armed Bandit (TAC-MAB), prove logarithmic regret for a centralized algorithm (C-TAC), and propose a decentralized event-triggered protocol (D-TAC) that empirically reduces communication by $23\times$ relative to the centralized baseline while preserving feasibility alignment.

Problem formulation

TAC-MAB considers $M$ homogeneous agents allocating themselves across $K$ stationary tasks over horizon $T$, with $K > M$ so that not all tasks can be attempted simultaneously. Each task $k$ has an unknown integer feasibility threshold $\tau_k$ (possibly exceeding $M$, making it structurally infeasible), success probability $p_k$, and known value $v_k$. When a coalition of size $M$0 acts on task $M$1, the team observes reward only if $M$2; otherwise the observation is deterministically zero. The key identifiability difficulty is that a zero outcome conflates censoring ($M$3) with stochastic failure ($M$4 under a feasible coalition). Importantly, censoring occurs at observation rather than generation: outcomes are realized regardless of coalition size but are simply uninformative below threshold.

The oracle benchmark solves a 0/1 knapsack with weights $M$5 and values $M$6, solvable exactly in $M$7 time via dynamic programming. Regret is measured against this ideal allocation, and decomposes into two coupled costs: threshold learning under censored feedback, and statistical estimation of $M$8 — both competing for the same scarce per-round resource of $M$9 agents. The model is positioned as a multi-agent specialization of censored semi-bandits (Shoemaker et al., 2019), differing in that the resource is a discrete coalition of distinct agents, thresholds are integer-valued, and observations are partitioned by agent assignment under decentralized execution.

Centralized baseline and regret guarantee

C-TAC maintains a per-task phase among search, monitor, and infeasible. Tasks begin in search at $K$0; after $K$1 consecutive failures the estimate advances by one (linear pruning), and the first nonzero observation freezes the threshold and transitions to monitoring, where only mean-reward estimates are updated on feasible executions. This update convention keeps $K$2 unbiased by censored feedback. Planning each round solves the exact knapsack over active tasks using UCB1-style indices (Liu, 2018).

The main theoretical result is a gap-dependent $K$3 regret bound under the identifiability condition $K$4 for all feasible tasks:

$K$5

The structural-search term is the paper's central contribution: it prices the cost of resolving feasibility under censored feedback, scaling linearly in the threshold magnitude and inversely in $K$6. The proof relies on a union bound showing that with failure budget $K$7, no feasible task is incorrectly pruned with probability at least $K$8. Two assumptions deserve emphasis: the identifiability condition is necessary (without it, sublinear regret is impossible since feasibility cannot be distinguished from stochastic failure), and the coordinator's knapsack solver is assumed exact, so regret is attributable purely to information limitations rather than computational approximation. Task values are also assumed known; extending to unknown values is left open.

Decentralized event-triggered coordination

D-TAC gives each agent a Virtual Coordinator — a local copy of the C-TAC planner operating on its belief state — following the "Public Agent" paradigm of Chakraborty et al. Because the planner is deterministic, identical beliefs yield identical plans without negotiation; agents share the planner, a rank-based assignment rule, fusion rules, and triggers prior to deployment. Belief states track interval bounds $K$9 around each threshold, with the planner weighting coalitions by the smallest empirically confirmed feasible size $T$0.

Communication is triggered by three events: Type I (a success refutes the synced lower bound or an infeasible label), Type II ($T$1 consecutive informative failures advance the lower bound), and a periodic heartbeat every $T$2 rounds bounding divergence in reward estimates. Fusion is conservative: lower bounds combine via max, upper bounds via min, phases follow the precedence monitor $T$3 infeasible $T$4 search, and reward estimates fuse as count-weighted means equivalent to pooling raw samples. Between syncs, agents persist with the last agreed plan ("sticky execution") to avoid mis-coordination.

The paper proves a communication bound (Proposition 1): total structural synchronization events are $T$5 network-wide, since synced lower bounds are monotone non-decreasing and upper bounds monotone non-increasing through a range of size $T$6. Total communication is therefore $T$7, with heartbeats dominating once consensus is reached. A notable concession: D-TAC carries no worst-case decentralization regret guarantee. Bounding the transient of mismatched plans, expected alignment time, and staleness effects is explicitly deferred to ongoing work; the claim that D-TAC approaches C-TAC's asymptotic performance is empirical, not proven.

Experimental evaluation

Experiments use $T$8, $T$9 agents, $K > M$0 tasks (including two infeasibility decoys, one full-team task, mid-coordination tasks, and low-threshold distractors), averaged over 40 runs. Four strategies are compared: Oracle Allocation, Independent UCB, C-TAC, and D-TAC. Notably, the authors omit comparison against decentralized cooperative-MAB baselines such as CoopUCB and DDUCB, citing inability to calibrate implementation choices against reference implementations — a limitation readers should weigh when interpreting the decentralized results.

Three findings stand out. First, independent UCB exhibits sustained linear regret under censored feedback, confirming the structural claim that independent exploration cannot resolve feasibility-gated tasks. Second, both C-TAC and D-TAC incur an early coordination cost that does not scale with the horizon, after which regret growth slows substantially. Third, the headline quantitative result: at $K > M$1, D-TAC achieves cumulative regret of 13,700 using 4,303 messages versus C-TAC's 4,500 regret at 100,000 messages — a $K > M$2 communication reduction within the same order of magnitude of regret. In a threshold sweep varying $K > M$3, independent UCB degrades rapidly while D-TAC approaches centralized performance gracefully, indicating that event-triggered coordination preserves feasibility learning as coordination requirements grow.

Limitations and open questions

The paper is explicit about scope constraints. The analysis assumes stationary thresholds, synchronous reliable communication, and zero-cost coordinator-to-agent messaging in the centralized idealization. Experiments use a fixed failure budget $K > M$4 rather than the theoretically dictated $K > M$5, and linear threshold search rather than binary search — reasonable for small teams but untested at scale. No worst-case regret bound exists for D-TAC under decentralization, and comparisons to existing decentralized MAMAB algorithms are deferred. Non-stationary coordination requirements, delayed or lossy communication, and adversarial failures are all left open, as is extension to unknown task values.

Conclusion

TAC-MAB isolates the structural cost of learning feasibility under censored feedback from ordinary statistical estimation, and quantifies it: feasibility resolution costs $K > M$6 probes per task, embedded in an overall $K > M$7 centralized regret bound. The D-TAC protocol demonstrates that near-centralized performance is achievable without continuous synchronization, relying on monotone conservative belief fusion to make structural disagreement self-limiting. The principal unresolved question is whether the empirical near-equivalence of D-TAC and C-TAC can be converted into a formal regret bound under intermittent communication — the problem the authors identify as their immediate next step.