FedPOB: Federated Prompt Optimization via Bandits

• The paper introduces FedPOB, a framework that adapts federated LinUCB and dueling bandit methods to optimize prompts for black-box LLMs while preserving local data.
• It leverages a shared prompt embedding space and event-triggered communication to aggregate linear-bandit statistics from multiple agents efficiently.
• Empirical results show that increasing agents boosts validation accuracy (e.g., from 0.663 to 0.712 on instruction tasks) while keeping query costs low.

Federated Prompt Optimization via Bandits (FedPOB) is a federated bandit framework for optimizing prompts for black-box LLMs when multiple agents collaborate without sharing raw data. It is formulated for settings in which prompt optimization must be simultaneously black-box compatible, sample-efficient because of query costs, and privacy-preserving in the federated-learning sense of keeping local histories and task data on-device. The framework contains two algorithms: FedPOB, a federated variant of LinUCB for scalar score feedback, and FedPOB-Pref, a preference-feedback extension based on federated dueling bandits with dynamic regularization (Lu et al., 29 Sep 2025).

1. Problem formulation and system model

FedPOB is defined over a task dataset

$\mathbb{D}=(\mathbb{X},\mathbb{Y}),$

where $\mathbb{X}=\{x_k\}$ are task inputs and $\mathbb{Y}=\{y_k\}$ are ground-truth outputs. For a prompt $p$, the paper defines a task score $s(p\mid \mathbb{D})$ through a task-specific metric $m$ applied to the black-box LLM output and the reference answer, and seeks

$$p^*=\arg\max_{p\in \mathbb{P}} s(p\mid \mathbb{D}).$$

The federated setting contains a set of $N>1$ agents $\mathbb{A}$. Each agent $a\in\mathbb{A}$ has its own prompt space $\mathbb{X}=\{x_k\}$0 and seeks

$\mathbb{X}=\{x_k\}$1

The paper assumes that all agents target the same underlying task $\mathbb{X}=\{x_k\}$2, although their local prompt pools may differ (Lu et al., 29 Sep 2025).

The framework distinguishes two feedback regimes. In score feedback, agent $\mathbb{X}=\{x_k\}$3 selects one prompt $\mathbb{X}=\{x_k\}$4 at round $\mathbb{X}=\{x_k\}$5 and receives a scalar score $\mathbb{X}=\{x_k\}$6 on a validation set $\mathbb{X}=\{x_k\}$7. In preference feedback, agent $\mathbb{X}=\{x_k\}$8 selects a pair $\mathbb{X}=\{x_k\}$9 and receives a binary preference signal. In both cases, raw prompt histories, raw local outcomes, and local data remain local. What is shared are only compact model summaries or bandit statistics.

Prompt representation is central. Each discrete prompt $\mathbb{Y}=\{y_k\}$0 is mapped to a feature vector

$\mathbb{Y}=\{y_k\}$1

using a pretrained embedding model. In the experiments, $\mathbb{Y}=\{y_k\}$2 and MPNet is used. Under score feedback, FedPOB assumes a linear reward model for each agent,

$\mathbb{Y}=\{y_k\}$3

with the implicit linear-bandit mean model

$\mathbb{Y}=\{y_k\}$4

This shared embedding space is what makes cross-agent transfer possible even when $\mathbb{Y}=\{y_k\}$5 overlaps only partially across agents.

2. FedPOB: federated LinUCB for score feedback

FedPOB adapts LinUCB to the federated prompt-optimization setting. Each agent maintains synchronized global sufficient statistics received from the server and unsent local increments. The synchronized quantities are a covariance-like matrix $\mathbb{Y}=\{y_k\}$6 and a reward-weighted feature sum $\mathbb{Y}=\{y_k\}$7; the unsent local updates are $\mathbb{Y}=\{y_k\}$8 and $\mathbb{Y}=\{y_k\}$9. The current local design matrix and ridge estimate are

$p$0

$p$1

where $p$2 is the regularization parameter (Lu et al., 29 Sep 2025).

At round $p$3, agent $p$4 selects a prompt by the UCB rule

$p$5

where $p$6 controls exploration. After querying the black-box LLM and observing $p$7, the local sufficient statistics are updated as

$p$8

$p$9

with $s(p\mid \mathbb{D})$0. These are precisely the statistics needed for regularized least-squares estimation, so the method does not require transmission of raw prompt-response trajectories.

Communication is event-triggered rather than round-synchronous. An agent triggers communication when newly accumulated information exceeds a threshold $s(p\mid \mathbb{D})$1, using the condition given in the algorithm,

$s(p\mid \mathbb{D})$2

If any agent requests communication, all agents upload

$s(p\mid \mathbb{D})$3

reset local unsent accumulators, and receive updated synchronized statistics from the server. The server aggregates by

$s(p\mid \mathbb{D})$4

$s(p\mid \mathbb{D})$5

This design makes the federated object of exchange a pair of linear-bandit sufficient statistics rather than prompt text, prompt trajectories, or raw scores. The paper’s practical interpretation is that prompt search becomes black-box compatible, query-efficient, and collaborative without exchanging raw local data. A plausible implication is that the embedding model $s(p\mid \mathbb{D})$6 functions as the principal inductive bias: if it captures prompt similarity well, linear transfer across agents becomes effective; if it does not, the federated gain is likely weaker.

3. FedPOB-Pref: preference feedback and federated dueling-bandit optimization

FedPOB-Pref addresses the case in which users cannot provide stable scalar scores but can express pairwise preference. The paper models preference using the Bradley–Terry–Luce model. If underlying prompt utilities are $s(p\mid \mathbb{D})$7 and $s(p\mid \mathbb{D})$8, then

$s(p\mid \mathbb{D})$9

where $m$0. Under the same linear utility assumption $m$1,

$m$2

This converts pairwise prompt comparison into a linear dueling-bandit problem over difference features (Lu et al., 29 Sep 2025).

Prompt-pair construction is asymmetric. The first prompt is pure exploitation,

$m$3

The second is chosen to produce an informative duel,

$m$4

with

$m$5

After querying both prompts, the observed preference is

$m$6

The local objective is a dynamically regularized pairwise logistic problem. For agent $m$7, the local loss up to time $m$8 is

$m$9

The local model is updated by solving

$$p^*=\arg\max_{p\in \mathbb{P}} s(p\mid \mathbb{D}).$$0

The paper interprets the three terms as local preference fitting, a linear correction compensating for local gradient drift, and a dynamic regularizer tethering the client to the previous global model. The uploaded pairwise covariance contribution is

$$p^*=\arg\max_{p\in \mathbb{P}} s(p\mid \mathbb{D}).$$1

Server aggregation in FedPOB-Pref uses both parameters and correction terms: $$p^*=\arg\max_{p\in \mathbb{P}} s(p\mid \mathbb{D}).$$2

$$p^*=\arg\max_{p\in \mathbb{P}} s(p\mid \mathbb{D}).$$3

The server then broadcasts $$p^*=\arg\max_{p\in \mathbb{P}} s(p\mid \mathbb{D}).$$4 back to all agents. Relative to plain gradient aggregation, this mechanism is explicitly designed to improve the communication/performance tradeoff under noisier preference feedback.

4. Theoretical basis and methodological lineage

The paper’s theoretical claims are partly inherited and partly explicit. For FedPOB, the main statement is that because the method is based on federated LinUCB, it inherits collaboration guarantees from federated linear bandits: increasing the number of agents effectively increases available information, reduces uncertainty in $$p^*=\arg\max_{p\in \mathbb{P}} s(p\mid \mathbb{D}).$$5, and improves prompt selection. The main text does not restate a complete prompt-specific regret theorem. For FedPOB-Pref, the paper gives an appendix-level justification of the local objective through an augmented-Lagrangian analysis of the consensus-constrained problem

$$p^*=\arg\max_{p\in \mathbb{P}} s(p\mid \mathbb{D}).$$6

At consensus, the first-order conditions reduce to

$$p^*=\arg\max_{p\in \mathbb{P}} s(p\mid \mathbb{D}).$$7

which the paper uses to motivate the necessity of the linear correction term. If one keeps only a quadratic penalty and drops the linear term, consensus would require

$$p^*=\arg\max_{p\in \mathbb{P}} s(p\mid \mathbb{D}).$$8

which is much stronger under client heterogeneity (Lu et al., 29 Sep 2025).

FedPOB sits within a broader prompt-learning and federated-bandit literature. Earlier federated prompt-tuning methods such as "PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models -- Federated Learning in Age of Foundation Model" (Guo et al., 2022) and "FedPrompt: Communication-Efficient and Privacy-Preserving Prompt Tuning in Federated Learning" (Zhao et al., 2022) optimize prompts by gradient descent while freezing a foundation backbone; neither uses exploration–exploitation or bandit feedback. Centralized bandit prompt-selection work such as "Efficient Prompt Optimization Through the Lens of Best Arm Identification" (Shi et al., 2024) treats prompt selection as fixed-budget best-arm identification, but does not address federation. Query-efficient federated black-box prompt learning in "FedOne: Query-Efficient Federated Learning for Black-box Discrete Prompt Learning" (Wang et al., 17 Jun 2025) analyzes costly LLM queries and argues that activating one client per round can be optimal for query efficiency, but it is not formulated as a bandit method. More general federated bandit templates, including communication-efficient nonlinear bandit optimization (Li et al., 2023) and asynchronous pure exploration in federated bandits (Wang et al., 2023), suggest adjacent design patterns—shared surrogate models, event-triggered synchronization, and fixed-confidence identification—that FedPOB specializes to prompt optimization. This suggests FedPOB is best understood as an overview: prompt-specific black-box optimization cast directly in federated bandit form.

5. Empirical evaluation

The experiments use MPNet embeddings and primarily GPT-3.5-turbo as the black-box LLM, with additional tests on GPT-4o-mini and Qwen3-235B-A22B-2507. Evaluation covers Instruction Induction and BIG-Bench Hard (BBH), yielding over 50 tasks in total after exclusions. Prompt pools are generated using the APE procedure from a small initial task description with five exemplars. To simulate heterogeneity, each agent’s prompt domain contains both shared prompts and unique prompts, so local candidate sets overlap only partially. In the score-feedback setting, the total budget is 50 validation queries; in the preference-feedback setting, the total budget is 100 validation queries, corresponding to 50 rounds of pairwise comparisons. Hyperparameters for FedPOB are

$$p^*=\arg\max_{p\in \mathbb{P}} s(p\mid \mathbb{D}).$$9

and FedPOB-Pref uses $N>1$0, learning rate $N>1$1, and the $N>1$2 schedule defined above (Lu et al., 29 Sep 2025).

For score feedback, the main empirical pattern is that performance improves with the number of participating agents. On the 22-task Instruction Induction subset reported in the main table, average validation accuracy rises from $N>1$3 with 1 agent to $N>1$4 with 3 agents and $N>1$5 with 10 agents; on the full 29-task Instruction Induction appendix results, the averages are $N>1$6, $N>1$7, and $N>1$8. On BBH, the averages are $N>1$9, $\mathbb{A}$0, and $\mathbb{A}$1, respectively.

Setting	Baseline / variant	Average score
Instruction Induction, 22 tasks	INSTINCT	0.669
Instruction Induction, 22 tasks	PromptBreeder	0.665
Instruction Induction, 22 tasks	FedOne (10 agents)	0.645
Instruction Induction, 22 tasks	FedPOB (1 / 3 / 10 agents)	0.663 / 0.701 / 0.712
Instruction Induction, 29 tasks	FedPOB (1 / 3 / 10 agents)	0.7637 / 0.7977 / 0.8068
BBH, 24 tasks	FedPOB (1 / 3 / 10 agents)	0.596 / 0.616 / 0.625

For preference feedback, FedPOB-Pref is reported against APOHF, Double TS, FLDB-GD, and FLDB-OGD. The clearest result is the joint communication/performance comparison: on Instruction Induction, FLDB-GD (10 agents) reaches $\mathbb{A}$2 with communication $\mathbb{A}$3, FLDB-OGD (10 agents) reaches $\mathbb{A}$4 with communication $\mathbb{A}$5, and FedPOB-Pref (10 agents) reaches $\mathbb{A}$6 with communication $\mathbb{A}$7. On BBH, the corresponding values are $\mathbb{A}$8 at $\mathbb{A}$9, $a\in\mathbb{A}$0 at $a\in\mathbb{A}$1, and $a\in\mathbb{A}$2 at $a\in\mathbb{A}$3.

Dataset	Method	Score / communication
Instruction Induction	FLDB-GD (10 agents)	0.8244 / 1500
Instruction Induction	FLDB-OGD (10 agents)	0.8123 / 50
Instruction Induction	FedPOB-Pref (10 agents)	0.8482 / 50
BBH	FLDB-GD (10 agents)	0.6457 / 1500
BBH	FLDB-OGD (10 agents)	0.6271 / 50
BBH	FedPOB-Pref (10 agents)	0.6583 / 50

The ablations reinforce a communication–performance tradeoff. Increasing the communication threshold $a\in\mathbb{A}$4 in FedPOB reduces communication rounds and degrades performance. Removing dynamic regularization in FedPOB-Pref reduces the method to a FedAvg-like scheme and harms stability, convergence speed, and robustness to client heterogeneity. The qualitative gains persist when GPT-3.5-turbo is replaced with GPT-4o-mini or Qwen, and when the prompt-pool generator changes from GPT-3.5-Turbo to GPT-4o-mini. At the same time, the averages are not uniformly better on every individual task, and the single-agent version of FedPOB is not always competitive with the best non-federated baseline. This suggests that the strongest empirical advantage of the framework lies in collaborative information sharing rather than in the single-agent bandit model alone.

6. Scope, limitations, and interpretations

FedPOB’s privacy guarantee is limited to data locality. Agents keep raw data, tested prompts, and local outcomes on-device, and share only sufficient statistics or model parameters, but the paper does not provide formal differential privacy guarantees for either FedPOB or FedPOB-Pref, does not defend against inference attacks on shared statistics, and assumes a trusted central server (Lu et al., 29 Sep 2025).

Several modeling assumptions bound the method’s scope. First, both algorithms rely on a linear reward assumption in the shared prompt embedding space,

$a\in\mathbb{A}$5

which may be restrictive for prompt quality surfaces that are highly nonlinear or interaction-dependent. Second, the framework assumes a shared feature representation $a\in\mathbb{A}$6 across all agents. Third, although prompt pools are heterogeneous, the formulation still assumes that agents are solving the same task $a\in\mathbb{A}$7; multi-task federation is outside the paper’s scope. Fourth, the environment is effectively stationary: prompt utilities are treated as stable over time even though deployed LLM APIs and user preferences may drift. Fifth, the preference experiments simulate federated preference feedback using BTL-generated binary comparisons from validation scores, which is a controlled but stylized substitute for real human preference signals.

These limitations place FedPOB at a particular point in the design space. It differs from gradient-based federated prompt tuning by treating prompt optimization as a sequential bandit problem over prompt embeddings rather than as direct prompt-parameter descent. It differs from centralized prompt-selection bandits by introducing federated aggregation of sufficient statistics and model updates. A plausible implication is that later extensions would need to relax at least one of its core assumptions—linear reward modeling, trusted-server aggregation, same-task federation, or stationarity—to address more general federated prompt optimization problems.