SCUBA in Research: Benchmark, Protocol & Heuristic

• SCUBA is a multi-domain acronym representing Salesforce CRM benchmarks, sidelink IoT protocols, and neutrality-aware search heuristics with distinct methodologies across fields.
• In the Salesforce domain, SCUBA evaluates computer-use agents on realistic CRM workflows using task-level resets and human-annotated, multi-variant templates, achieving up to 50% success with demonstration augmentation.
• In wireless systems and evolutionary computation, SCUBA implements a MAC protocol for IoT communications with latency–energy tradeoffs and leverages neutrality to enhance search evolvability in rugged fitness landscapes.

Searching arXiv for recent and relevant papers on “SCUBA” to ground the article in published work. SCUBA is an acronym with several distinct technical meanings in contemporary research. In recent arXiv usage it denotes the Salesforce Computer Use BenchmArk, a benchmark for evaluating computer-use agents on realistic CRM workflows inside the Salesforce platform (Dai et al., 30 Sep 2025). In wireless systems it denotes Sidelink Communications on Unlicensed BAnds, an in-device sidelink protocol for IoT and machine-type communication devices (Rajendran et al., 2020), together with a synchronization extension for out-of-sync devices (Chandrika et al., 2021). In evolutionary computation it denotes Scuba Search, a neutrality-aware local-search heuristic (0707.0643). The term also persists historically in astronomy through SCUBA-2, which is explicitly described as the successor to the original SCUBA instrument on the James Clerk Maxwell Telescope (Holland et al., 2013).

1. Principal technical referents

The acronym is not tied to a single research community. Its main technical referents in the supplied literature are summarized below.

Expansion	Domain	Representative paper
Salesforce Computer Use BenchmArk	Enterprise-agent evaluation in Salesforce CRM workflows	(Dai et al., 30 Sep 2025)
Sidelink Communications on Unlicensed BAnds	MAC protocol for D2D / sidelink IoT communication	(Rajendran et al., 2020)
Scuba Search	Local-search heuristic exploiting neutrality in fitness landscapes	(0707.0643)

This multiplicity matters because the surrounding literature is otherwise easy to conflate. A benchmark for browser-use agents, a sidelink protocol, and a neutrality-driven search heuristic share the same acronym but address unrelated problem classes. A further source of ambiguity is astronomical usage, where SCUBA survives mainly through the naming lineage of SCUBA-2 rather than through the expansion itself (Holland et al., 2013).

2. SCUBA as the Salesforce Computer Use BenchmArk

SCUBA stands for Salesforce Computer Use BenchmArk and is a benchmark for evaluating computer-use agents on realistic CRM workflows inside the Salesforce platform (Dai et al., 30 Sep 2025). It is motivated by the claim that existing benchmarks do not adequately capture the complexity of enterprise software automation. The benchmark description contrasts SCUBA with WebArena, which focuses on web navigation, OSWorld, which covers generic desktop tasks rather than Salesforce-specific business processes, and CRMArena, which emphasizes tool use / information retrieval but does not fully cover write actions that modify records and settings.

SCUBA is built in a Salesforce sandbox developer org. The environment design is central to its realism: it closely matches the real production UI and behavior while remaining safely usable for benchmarking. Resetting the environment is handled by snapshotting the initial org state via configuration files, comparing the modified org state after an agent run, and reverting only the changed configuration files. This makes resets feasible at the task level. The benchmark also supports parallel execution, allowing multiple browser sessions or OS desktops to run concurrently.

The benchmark contains 300 task instances derived from real Salesforce workflows. These are sourced from interviews with real Salesforce users, including solution engineers, org administrators, sales development representatives, business development representatives, and account executives. Each workflow is paired with a template, one Trailhead knowledge article, initialization scripts, and rule-based evaluators. Templates are instantiated into multiple queries by filling in different values and paraphrasing them for language diversity.

The tasks span three major personas: platform administrators, sales representatives, and service agents. Across those personas, SCUBA evaluates five enterprise-critical capabilities: UI navigation, data manipulation, workflow automation, information retrieval, and troubleshooting / QA. The construction pipeline is explicitly human-centered: task template creation from real workflows collected in interviews, task query creation by instantiating templates into 5 variants and paraphrasing them with ChatGPT, task initialization and evaluation using Salesforce APIs such as Metadata APIs and Tooling APIs, and human annotation to validate that tasks are solvable and that evaluators match human expectations.

The benchmark is therefore designed as a testbed for enterprise software interaction rather than generic browser or desktop control. A plausible implication is that SCUBA operationalizes “enterprise realism” through write-heavy workflows, persona specificity, and task-level environment reset, rather than through open-ended web navigation alone.

3. Formalization, evaluation, and reported results of the Salesforce benchmark

The paper models each task as a partially observable Markov decision process,

$(\Scal, \Acal, \Tcal, \Rcal, \Omega, \Ocal),$

where $\Scal$ is the state space, $\Acal$ the action space, $\Tcal(s'|s,a)$ the state transition probability, $\Rcal(s,a)$ the reward function, $\Ocal$ the observation space, and $\Omega(o'~|~s',a)$ the observation probability (Dai et al., 30 Sep 2025). The agent $\pi$ receives a task query $Q$ and history

$H_t=\{o_0, a_1, o_1, \ldots, a_t, o_t\},$

then samples the next action as

$\Scal$0

SCUBA’s evaluation protocol uses rule-based evaluators that produce Task Success, Milestone Score, Time, #Steps, #Tokens, and Costs. Task success is binary $\Scal$1. The milestone score is between 0 and 1 and captures rubric-based partial credit. The paper emphasizes milestone scoring because it reveals where an agent failed during execution, not just whether it succeeded. Task termination occurs when the agent says it is done, the max steps is reached, or the max time is reached; the default cap is 50 steps and 90 minutes. The benchmark also includes an asynchronous parallel inference infrastructure, and the paper reports that full evaluation can finish in less than 90 minutes for self-served models, with API models limited mainly by rate limits.

The experiments evaluate 9 agents in zero-shot and demonstration-augmented settings. In zero-shot, agents receive only the task query. In demonstration-augmented evaluation, they also receive human demonstration examples derived from trajectories. The paper emphasizes a large open-source versus closed-source gap, stating that open-source computer-use agents that have strong performance on OSWorld achieve less than 5% success on SCUBA in zero-shot (Dai et al., 30 Sep 2025).

Agent	Zero-shot success	Demonstration-augmented success
GPT-5 browser-use	51.33%	53.85%
o3/o4-mini browser-use	45.67%	50.00%
Gemini 2.5-Pro browser-use	31.00%	46.15%
Claude computer-use	27.00%	47.69%
Claude browser-use	—	46.15%
Agent S2 + GPT-5	39.00%	40.38%
UI-TARS 1.5-7B	2.67%	9.16%
OpenCUA 7B	0.67%	5.38%

The headline summary is that demonstration augmentation can raise performance to about 50% success while reducing time by 13% and cost by 16% overall (Dai et al., 30 Sep 2025). This suggests that on enterprise CRM workflows, trajectory-level human demonstrations improve not only final success rate but also execution efficiency. It also suggests that general computer-use capability does not transfer cleanly to Salesforce-specific business processes without stronger grounding, planning, or workflow-specific priors.

4. SCUBA as Sidelink Communications on Unlicensed BAnds

In wireless systems, SCUBA stands for Sidelink Communications on Unlicensed BAnds. It is presented as a medium access control (MAC) protocol for device-to-device (D2D) / sidelink communication designed specifically for IoT and machine-type communication (MTC) devices (Rajendran et al., 2020). Its central architectural idea is to make sidelink a secondary RAT that is time-division multiplexed with the primary cellular RAT while reusing the same radio hardware. The protocol is intended for low-cost cellular devices, including HD-FDD LTE-M / MTC devices, and is designed to operate over unlicensed spectrum such as 865–868 MHz in Europe and 902–928 MHz in the United States.

The protocol introduces SL-DRX and SL paging occasions (SL-POs), so that a UE listens for potential sidelink transmission only during configured intervals. This creates a direct latency-versus-battery-life tradeoff: larger SL-DRX cycles reduce listening and battery drain but increase sidelink latency. SCUBA also defines SL connected mode for interactive exchange after an initial acknowledgment, as well as two HARQ styles: a fixed MCS/TBS mode and a grant-based mode. For busy cellular conditions it introduces SAM-D and SAM-U, or Sidelink Availability Messages, to advertise dynamic availability or unavailability for sidelink.

The paper evaluates power, latency, and collision probability at the MAC timing level. It reports that power decreases as SL-DRX cycle increases, while latency increases nearly linearly with SL-DRX cycle. For SCUBA-LLM, where $\Scal$2, the UE listens on all available SCUBA opportunities; this gives near-immediate sidelink response but at substantially higher power cost. Using a 5 Wh battery model, the paper estimates 328.5 days for LTE-M standalone, 279 days for LTE-M + SCUBA with busy Poisson SCUBA traffic, and 328.3 days for LTE-M + SCUBA with very infrequent SCUBA traffic with 2-hour IAT. It also reports that SCUBA data collision probabilities are very small, on the order of $\Scal$3 (Rajendran et al., 2020).

A later paper extends the protocol to out-of-sync devices under the title "Synchronized SCUBA: D2D Communication for Out-of-Sync Devices" (Chandrika et al., 2021). That work addresses three scenarios: out-of-coverage (OOC), partial coverage (PC), and coverage-out-of-sync (COOS). It proposes two synchronization families: flexi-sync, based on SLSS, TimeREQ, and TimeRSP, and a sync beacon method with TX beacon and RX beacon variants. The paper states that synchronized SCUBA enables links even when devices are not already finely synchronized, extends the operating range by explicitly accounting for propagation delay $\Scal$4, and does so with negligible battery impact: legacy SCUBA battery life with LTE-M sync: 328.3 days, flexi-sync SCUBA battery life: 328.1 days, a less than 0.1% reduction (Chandrika et al., 2021).

In this usage, SCUBA is a protocol family rather than a benchmark. Its significance lies in combining single-radio hardware reuse, unlicensed-band sidelink, and explicit latency–energy control, with the synchronized extension removing the assumption that communicating devices already share fine timing.

5. SCUBA as Scuba Search in evolutionary computation

In evolutionary computation, Scuba Search, or SCUBA, is a local-search heuristic designed to exploit neutrality in fitness landscapes by explicitly favoring regions where the search can increase evolvability (0707.0643). The method is based on the observation that large flat regions, or neutral networks, can be traversed without immediate fitness gain, and that among equally fit solutions some have better future opportunities for improvement than others.

The algorithm alternates between two phases. In Conquest-of-the-Waters, the search remains within the neutral neighborhood and repeatedly chooses a neutral neighbor with maximal evolvability. If $\Scal$5 denotes the neutral neighborhood, the algorithm computes

$\Scal$6

and, when $\Scal$7, moves to a neutral neighbor $\Scal$8 such that $\Scal$9. In Invasion-of-the-Land, once no neutral neighbor has higher evolvability, the search moves to a fitter solution outside the neutral neighborhood. The paper defines a local maximum as

$\Acal$0

and a strict local maximum as

$\Acal$1

The empirical study uses NKq landscapes with $\Acal$2, a one-bit mutation neighborhood, $\Acal$3 runs per parameter setting, and, for Netcrawler, a maximum of 300 steps (0707.0643). SCUBA is compared against Hill Climbing (HC), Hill Climbing Two Steps (HC2), and Netcrawler (NC). The paper reports that SCUBA generally outperformed HC, HC2, and Netcrawler in terms of average fitness, especially when neutrality was high: the advantage was especially clear for $\Acal$4 and $\Acal$5, whereas for $\Acal$6 all methods were nearly equivalent except Netcrawler.

The paper also emphasizes cost. SCUBA is not the cheapest heuristic: HC has relatively low evaluation cost, HC2 is much more expensive, Netcrawler has the lowest fixed evaluation cost in the reported setup, and SCUBA is typically between HC and HC2. The key interpretation given is that SCUBA’s gain is not explained simply by more evaluations; rather, it comes from using neutral moves to alter the structure of the search trajectory. In this sense, SCUBA is a method for neutral search with purpose, not neutral drift.

6. Astronomical lineage and non-acronym uses

In astronomy, the supplied literature uses SCUBA primarily through SCUBA-2, which is described as a next-generation direct-detection camera for the James Clerk Maxwell Telescope (JCMT) and the successor to the original SCUBA instrument (Jenness et al., 2010). SCUBA-2 is a dual-wavelength, 10,000-pixel bolometer camera with 10,240 total detectors, or 5120 pixels per focal plane, operating at 450 $\Acal$7m and 850 $\Acal$8m (Holland et al., 2013). The instrument maps the sky 100–150 times faster than SCUBA, uses continuous scan-based observing rather than chopped staring, and is paired with a reduction stack centered on SMURF and ORAC-DR (Holland et al., 2013, Jenness et al., 2010). The observing strategy is explicitly time-domain: scan patterns such as “pong,” “lissajous,” and “daisy” are chosen so that the telescope does not revisit the same sky position at a fixed time interval, helping separate sky-fixed signal from time-varying noise (Jenness et al., 2010).

A second disambiguation concerns ordinary diving usage. Several underwater-robotics papers use scuba to refer to scuba divers or scuba diving, not to an acronym. Examples include visual diver face recognition for underwater human-robot interaction (Hong et al., 2020), underwater image transmission on commodity mobile devices for scuba diving communication (Tian et al., 15 Feb 2025), and robotic classification of diver swimming states for diver safety monitoring (Kutzke et al., 15 Oct 2025). These works are adjacent only lexically: they concern underwater perception, communication, and monitoring rather than benchmarks, sidelink protocols, or neutrality-aware optimization.

Taken together, the term SCUBA functions less as a single concept than as a recurring acronym and naming root across distinct technical domains. In current arXiv usage, its most elaborated standalone expansion is the Salesforce enterprise-agent benchmark (Dai et al., 30 Sep 2025); in communication systems it denotes a single-radio unlicensed sidelink protocol and its synchronization extension (Rajendran et al., 2020, Chandrika et al., 2021); in evolutionary computation it names a neutrality-aware search heuristic (0707.0643); and in astronomy it remains embedded in the instrument lineage culminating in SCUBA-2 [(Holland et al., 2013); (Jenness et al., 2010)].