ProactiveBench: Benchmarking Proactivity

Updated 4 July 2026

ProactiveBench is a benchmark framework that defines proactivity as the ability to identify incomplete evidence and request targeted user interventions.
It repurposes seven datasets into multimodal and event-stream scenarios to test diverse proactive behaviors such as moving occlusions or enhancing image quality.
Empirical findings reveal that proactivity cannot be assumed from scale alone, emphasizing the need for specialized training to balance intervention timing and accuracy.

ProactiveBench is the name used in recent arXiv literature for benchmarks that operationalize proactivity as a first-class model capability rather than a by-product of instruction following. In its most specific and widely recognizable usage, it denotes a benchmark for multimodal LLMs (MLLMs) that tests whether a model can recognize when visual evidence is insufficient and request a simple user intervention that would make the query answerable, rather than hallucinating, abstaining, or guessing (Min et al., 19 Mar 2026). In a second usage, the same name refers to an event-stream benchmark and training corpus for proactive assistants in coding, writing, and daily life, where the model predicts whether and what help to offer from user activity, environmental events, and environment state (Lu et al., 2024). Across these usages, the common object is not reactive correctness alone, but initiative under uncertainty.

1. Conceptual basis

In the multimodal benchmark formulation, proactiveness is defined as the ability to either provide the correct answer or ask for help by suggesting an action that makes the query answerable. The key distinction is between a model that merely abstains and a model that proposes a concrete intervention such as moving an occluding object, rotating an object, improving image quality, waiting for a temporal occlusion to pass, or requesting more sketch detail (Min et al., 19 Mar 2026).

In the proactive-assistance formulation, the same term is attached to a broader anticipatory-agent setting. There, the proactive agent predicts a task from current context according to

$P_t = f_{\theta}(E_t, A_t, S_t),$

where $E_t$ is the environment event history, $A_t$ is the user activity history, and $S_t$ is the current environmental state; training is framed as maximizing user acceptance of suggestions (Lu et al., 2024). This suggests that, across benchmark lines, “proactiveness” denotes a family of capabilities involving recognition of insufficiency, latent need inference, and timely intervention, rather than a single metric or modality.

A recurrent misconception in this area is that proactivity is equivalent to aggressiveness in speaking or acting. The benchmarks explicitly reject that equivalence. In the visual setting, a model that selects arbitrary non-abstain actions is not thereby proactive; in the event-stream setting, a model that offers help when no help is needed incurs false alarms rather than credit (Min et al., 19 Mar 2026).

2. Benchmark construction in the multimodal visual setting

The benchmark titled “ProactiveBench: Benchmarking Proactiveness in Multimodal LLMs” is built by repurposing seven existing datasets into seven proactive scenarios and covers 19 proactive behaviors overall (Min et al., 19 Mar 2026). Its central design move is to begin from an intentionally ambiguous or incomplete first state and to ask whether the model will request the information needed to resolve that ambiguity.

The seven source datasets and scenarios are as follows. ROD is used for occluded objects, where the model asks to move an occluding object left or right. VSOD covers temporal occlusions in video and permits actions such as waiting or rewinding. MVP-N targets uninformative views, with proactive suggestions to rotate the object or change camera angle. ImageNet-C is converted into an image quality enhancement scenario, where the model asks to deblur, denoise, or otherwise improve image quality. QuickDraw becomes a coarse sketches scenario, in which the model asks for more strokes or more detail. ChangeIt targets temporal ambiguities, allowing requests for past or future frames where the target becomes visible. MS-COCO is used for camera movement or cropping, with actions such as moving the camera, zooming out, or changing viewpoint (Min et al., 19 Mar 2026).

Across these datasets, the benchmark contains more than 108k images grouped into about 18k samples. Because some ambiguous initial states remain answerable, the benchmark applies a filtering rule: a sample is removed if it is correctly predicted at the first turn at least 25% of the time across all evaluated MLLMs. This reduces the benchmark from 17,909 total samples to 7,557 final samples, and lowers average first-turn accuracy from 32.5% to 6.4%, thereby making proactive intervention materially necessary rather than optional (Min et al., 19 Mar 2026).

3. Task formulation, interaction protocol, and metrics

ProactiveBench evaluates models in two modes: multiple-choice question answering (MCQA) and open-ended generation (OEG) (Min et al., 19 Mar 2026). In the MCQA setting, the model receives the question, the current image or frame, and a discrete list of valid actions. The options include an abstain option, one or more proactive suggestions, several wrong category answers, and exactly one correct category. If the model chooses a proactive action, the environment transitions to a more informative state and the episode continues; if it abstains or chooses a wrong category, the episode terminates incorrectly; if it predicts the correct category, the episode terminates successfully.

This process is formalized as an MDP with state space $S$ , action set $A$ , policy $\pi$ , and reward $R$ . At time $t$ , the model observes

$s_t = \{I_t, A_t\}$

and samples an action

$E_t$ 0

The evaluation therefore tests not only recognition, but also sequential control over when to request more information and when to commit to an answer (Min et al., 19 Mar 2026).

The reported MCQA metrics are acc, defined as accuracy over the full multi-turn episode, and ps, the proactive suggestion rate, defined as the average number of human interventions requested by the model. In OEG, where the model answers freely without a predefined option set, evaluation is restricted to single-turn settings and uses an LLM-as-a-judge to determine whether the response semantically expresses a valid proactive suggestion and/or the correct category. The reported OEG metric is agg, an aggregate accuracy that counts an answer as correct if it contains either the correct category or a valid proactive suggestion (Min et al., 19 Mar 2026).

A central methodological point is that the benchmark distinguishes sharply between proactive suggestion, abstention, and correct direct answering. It is designed precisely to test whether a model can traverse those alternatives in a controlled manner.

4. Empirical findings and interpretive implications

The main empirical result is that current MLLMs are generally not proactive in the intended sense (Min et al., 19 Mar 2026). When given the reference frame directly, average accuracy is 79.8%, but on ProactiveBench the models underperform by more than 60 percentage points on average. In ROD, average accuracy is only 8.2% versus 98.3% on the reference frame. The benchmark therefore isolates a failure mode that is not basic recognition capacity, but failure to recognize the need for collaborative intervention.

The paper also finds that proactiveness does not correlate with model capacity. Smaller models sometimes outperform larger ones in MCQA accuracy or proactive-suggestion rate, but subsequent ablations show that much of this apparent proactivity reflects anti-abstention behavior rather than genuine task understanding. To test this, the valid proactive actions are replaced with random invalid proactive options from other datasets. Models such as Mistral, LLaVA-7B, and InternVL3-8B then lose most of their proactive-suggestion rate, indicating that their earlier behavior was not grounded proactiveness but a tendency to avoid abstention (Min et al., 19 Mar 2026).

Prompt engineering helps only weakly. Dataset-specific hints increase proactive behavior, and MCQA proactive-suggestion rate rises by about 1.9 on average, but average MCQA accuracy reaches only 25.8%. Moreover, in 16.0% of cases with hints, models blindly choose proactive suggestions and fail to answer even when the reference image becomes available. Likewise, conversation history and few-shot in-context learning introduce negative biases: they increase proactive suggestions but lower accuracy, often by pushing the model to repeat previous proactive moves until the exploration limit is reached (Min et al., 19 Mar 2026).

These findings yield two broader implications. First, proactiveness is not an automatic emergent property of scale. Second, naive encouragement to “be more proactive” can readily induce over-triggering or pattern imitation rather than calibrated intervention.

5. Learning proactiveness

Although pretrained MLLMs generally lack the target behavior, the benchmark literature also shows that proactiveness is at least partially learnable. In the multimodal ProactiveBench, two models—Mistral-7B and Qwen2.5-VL-3B—are fine-tuned with GRPO on data from QuickDraw and MS-COCO. The reward assigns $E_t$ 1 for the correct category, $E_t$ 2 for a valid proactive suggestion, and $E_t$ 3 otherwise. The best tradeoff occurs when $E_t$ 4, and the learned behavior generalizes to unseen datasets, including CIT (Min et al., 19 Mar 2026).

The event-stream ProactiveBench line reaches the same general conclusion by a different route. That benchmark contains 6,790 events for agent-model training, a 233 real-world event test set, and 1,760 labeled entries for a reward model. Fine-tuning on that corpus produces Qwen2-7B-Proactive with 66.47% F1-Score in proactively offering assistance, outperforming the reported open-source and closed-source baselines in that benchmark table (Lu et al., 2024).

The framework PRISM further treats proactive intervention as cost-sensitive selective decision-making on that event-stream benchmark. It uses calibrated estimates of need and acceptance, a dynamic threshold, and a selective Slow mode near the decision boundary. On the held-out test set of 233 clips, PRISM reports Recall 98.88%, Precision 77.05%, Accuracy 76.39%, False alarm rate 22.94%, and F1 86.61%, outperforming DeepSeek-R1 on precision, false alarms, and F1 (Fu et al., 2 Feb 2026). Taken together, these results indicate that benchmark-guided training can improve not only direct-answer accuracy but also restraint, timing, and burden control.

The term ProactiveBench is not fully canonical. Besides the multimodal visual benchmark (Min et al., 19 Mar 2026) and the event-stream proactive-assistance corpus (Lu et al., 2024), the label also appears in ProactiveVideoQA, where the paper states that ProactiveVideoQA is instantiated as ProactiveBench for proactive, multi-turn, time-aware video QA (Wang et al., 12 Jul 2025). This suggests that the name has begun to function as a broader label for benchmarks that evaluate initiative under partial observability, rather than a single fixed dataset.

This terminological overlap matters because neighboring works pursue related but non-identical objects. ProVoice-Bench evaluates proactive voice agents with tasks such as Proactive Intent Capture, Latent Topic Monitor, Context Fact Checking, and Environment Sound Sensing (Xu et al., 16 Apr 2026). $E_t$ 5-Bench targets proactive personal assistant agents in long-horizon workflows with hidden user intents, inter-task dependencies, and cross-session continuity (Zhang et al., 14 May 2026). PIRA-Bench studies GUI-based proactive intent recommendation from continuous screenshot trajectories (Chai et al., 9 Mar 2026). EgoPro-Bench addresses personalized proactive interaction in streaming egocentric video (Ran et al., 8 May 2026). ProAgentBench focuses on real working scenarios with 28,528 total events and a hierarchical When + How formulation (Tang et al., 4 Feb 2026). Pro $E_t$ 6Bench and EgoProactive address proactive procedural assistance and Out-of-Plan recovery from egocentric video (Kundu et al., 3 Jun 2026).

Within that expanding ecosystem, ProactiveBench occupies a foundational role because it sharpens a general research question into an executable evaluation target: whether a model can identify that its current evidence is inadequate, infer the right intervention, and ask for the specific help needed to proceed. A plausible implication is that benchmarking proactivity has shifted from measuring isolated prompt compliance to measuring calibrated collaboration—including when to ask, when to wait, and when to remain silent.