---
title: 'SCUBA in Research: Benchmark, Protocol & Heuristic'
url: https://www.emergentmind.com/topics/scuba
type: topic
---

# SCUBA in Research: Benchmark, Protocol & Heuristic

Searching arXiv for recent and relevant papers on “SCUBA” to ground the article in published work.
SCUBA is an acronym with several distinct technical meanings in contemporary research. In recent arXiv usage it denotes the **Salesforce Computer Use BenchmArk**, a benchmark for evaluating computer-use agents on realistic CRM workflows inside the Salesforce platform [2509.26506]. In wireless systems it denotes **Sidelink Communications on Unlicensed BAnds**, an in-device sidelink protocol for IoT and machine-type communication devices [2012.04074], together with a synchronization extension for out-of-sync devices [2104.00809]. In evolutionary computation it denotes **Scuba Search**, a neutrality-aware local-search heuristic [0707.0643]. The term also persists historically in astronomy through **SCUBA-2**, which is explicitly described as the successor to the original SCUBA instrument on the James Clerk Maxwell Telescope [1301.3650].

## 1. Principal technical referents

The acronym is not tied to a single research community. Its main technical referents in the supplied literature are summarized below.

| Expansion | Domain | Representative paper |
|---|---|---|
| Salesforce Computer Use BenchmArk | Enterprise-agent evaluation in Salesforce CRM workflows | [2509.26506] |
| Sidelink Communications on Unlicensed BAnds | MAC protocol for D2D / sidelink IoT communication | [2012.04074] |
| Scuba Search | Local-search heuristic exploiting neutrality in fitness landscapes | [0707.0643] |

This multiplicity matters because the surrounding literature is otherwise easy to conflate. A benchmark for browser-use agents, a sidelink protocol, and a neutrality-driven search heuristic share the same acronym but address unrelated problem classes. A further source of ambiguity is astronomical usage, where SCUBA survives mainly through the naming lineage of SCUBA-2 rather than through the expansion itself [1301.3650].

## 2. SCUBA as the Salesforce Computer Use BenchmArk

SCUBA stands for **Salesforce Computer Use BenchmArk** and is a benchmark for evaluating **computer-use agents** on realistic **CRM workflows inside the Salesforce platform** [2509.26506]. It is motivated by the claim that existing benchmarks do not adequately capture the complexity of **enterprise software automation**. The benchmark description contrasts SCUBA with WebArena, which focuses on web navigation, OSWorld, which covers generic desktop tasks rather than Salesforce-specific business processes, and CRMArena, which emphasizes **tool use / information retrieval** but does not fully cover **write actions** that modify records and settings.

SCUBA is built in a **Salesforce sandbox developer org**. The environment design is central to its realism: it closely matches the real production UI and behavior while remaining safely usable for benchmarking. Resetting the environment is handled by snapshotting the initial org state via configuration files, comparing the modified org state after an agent run, and reverting only the changed configuration files. This makes resets feasible at the **task level**. The benchmark also supports **parallel execution**, allowing multiple browser sessions or OS desktops to run concurrently.

The benchmark contains **300 task instances** derived from real Salesforce workflows. These are sourced from interviews with real Salesforce users, including **solution engineers**, **org administrators**, **sales development representatives**, **business development representatives**, and **account executives**. Each workflow is paired with a **template**, one **Trailhead knowledge article**, initialization scripts, and rule-based evaluators. Templates are instantiated into multiple queries by filling in different values and paraphrasing them for language diversity.

The tasks span three major personas: **platform administrators**, **sales representatives**, and **service agents**. Across those personas, SCUBA evaluates five enterprise-critical capabilities: **UI navigation**, **data manipulation**, **workflow automation**, **information retrieval**, and **troubleshooting / QA**. The construction pipeline is explicitly human-centered: task template creation from real workflows collected in interviews, task query creation by instantiating templates into 5 variants and paraphrasing them with ChatGPT, task initialization and evaluation using Salesforce APIs such as Metadata APIs and Tooling APIs, and human annotation to validate that tasks are solvable and that evaluators match human expectations.

The benchmark is therefore designed as a testbed for enterprise software interaction rather than generic browser or desktop control. A plausible implication is that SCUBA operationalizes “enterprise realism” through write-heavy workflows, persona specificity, and task-level environment reset, rather than through open-ended web navigation alone.

## 3. Formalization, evaluation, and reported results of the Salesforce benchmark

The paper models each task as a **partially observable Markov decision process**,
\[
(\Scal, \Acal, \Tcal, \Rcal, \Omega, \Ocal),
\]
where \(\Scal\) is the state space, \(\Acal\) the action space, \(\Tcal(s'|s,a)\) the state transition probability, \(\Rcal(s,a)\) the reward function, \(\Ocal\) the observation space, and \(\Omega(o'~|~s',a)\) the observation probability [2509.26506]. The agent \(\pi\) receives a task query \(Q\) and history
\[
H_t=\{o_0, a_1, o_1, \ldots, a_t, o_t\},
\]
then samples the next action as
\[
a_{t+1} \sim \pi(a~|~H_t, Q).
\]

SCUBA’s evaluation protocol uses rule-based evaluators that produce **Task Success**, **Milestone Score**, **Time**, **\#Steps**, **\#Tokens**, and **Costs**. Task success is binary \(0/1\). The milestone score is between 0 and 1 and captures rubric-based partial credit. The paper emphasizes milestone scoring because it reveals where an agent failed during execution, not just whether it succeeded. Task termination occurs when the agent says it is done, the max steps is reached, or the max time is reached; the default cap is **50 steps** and **90 minutes**. The benchmark also includes an **asynchronous parallel inference infrastructure**, and the paper reports that full evaluation can finish in **less than 90 minutes** for self-served models, with API models limited mainly by rate limits.

The experiments evaluate **9 agents** in **zero-shot** and **demonstration-augmented** settings. In zero-shot, agents receive only the task query. In demonstration-augmented evaluation, they also receive human demonstration examples derived from trajectories. The paper emphasizes a large open-source versus closed-source gap, stating that open-source computer-use agents that have strong performance on OSWorld achieve **less than 5% success** on SCUBA in zero-shot [2509.26506].

| Agent | Zero-shot success | Demonstration-augmented success |
|---|---:|---:|
| GPT-5 browser-use | 51.33% | 53.85% |
| o3/o4-mini browser-use | 45.67% | 50.00% |
| Gemini 2.5-Pro browser-use | 31.00% | 46.15% |
| Claude computer-use | 27.00% | 47.69% |
| Claude browser-use | — | 46.15% |
| Agent S2 + GPT-5 | 39.00% | 40.38% |
| UI-TARS 1.5-7B | 2.67% | 9.16% |
| OpenCUA 7B | 0.67% | 5.38% |

The headline summary is that demonstration augmentation can raise performance to about **50% success** while reducing **time by 13%** and **cost by 16%** overall [2509.26506]. This suggests that on enterprise CRM workflows, trajectory-level human demonstrations improve not only final success rate but also execution efficiency. It also suggests that general computer-use capability does not transfer cleanly to Salesforce-specific business processes without stronger grounding, planning, or workflow-specific priors.

## 4. SCUBA as Sidelink Communications on Unlicensed BAnds

In wireless systems, SCUBA stands for **Sidelink Communications on Unlicensed BAnds**. It is presented as a **medium access control (MAC) protocol for device-to-device (D2D) / sidelink communication** designed specifically for **IoT and machine-type communication (MTC)** devices [2012.04074]. Its central architectural idea is to make sidelink a **secondary RAT** that is time-division multiplexed with the primary cellular RAT while reusing the same radio hardware. The protocol is intended for low-cost cellular devices, including **HD-FDD LTE-M / MTC devices**, and is designed to operate over unlicensed spectrum such as **865–868 MHz** in Europe and **902–928 MHz** in the United States.

The protocol introduces **SL-DRX** and **SL paging occasions (SL-POs)**, so that a UE listens for potential sidelink transmission only during configured intervals. This creates a direct latency-versus-battery-life tradeoff: larger SL-DRX cycles reduce listening and battery drain but increase sidelink latency. SCUBA also defines **SL connected mode** for interactive exchange after an initial acknowledgment, as well as two HARQ styles: a fixed MCS/TBS mode and a grant-based mode. For busy cellular conditions it introduces **SAM-D** and **SAM-U**, or Sidelink Availability Messages, to advertise dynamic availability or unavailability for sidelink.

The paper evaluates power, latency, and collision probability at the MAC timing level. It reports that **power decreases as SL-DRX cycle increases**, while **latency increases nearly linearly with SL-DRX cycle**. For **SCUBA-LLM**, where \(N=0\), the UE listens on all available SCUBA opportunities; this gives near-immediate sidelink response but at substantially higher power cost. Using a **5 Wh battery model**, the paper estimates **328.5 days** for LTE-M standalone, **279 days** for LTE-M + SCUBA with busy Poisson SCUBA traffic, and **328.3 days** for LTE-M + SCUBA with very infrequent SCUBA traffic with **2-hour IAT**. It also reports that SCUBA data collision probabilities are **very small**, on the order of **\(10^{-5}\)** [2012.04074].

A later paper extends the protocol to out-of-sync devices under the title **"Synchronized SCUBA: D2D Communication for Out-of-Sync Devices"** [2104.00809]. That work addresses three scenarios: **out-of-coverage (OOC)**, **partial coverage (PC)**, and **coverage-out-of-sync (COOS)**. It proposes two synchronization families: **flexi-sync**, based on **SLSS**, **TimeREQ**, and **TimeRSP**, and a **sync beacon method** with TX beacon and RX beacon variants. The paper states that synchronized SCUBA enables links even when devices are not already finely synchronized, extends the operating range by explicitly accounting for propagation delay \(t_d\), and does so with negligible battery impact: **legacy SCUBA battery life with LTE-M sync: 328.3 days**, **flexi-sync SCUBA battery life: 328.1 days**, a **less than 0.1% reduction** [2104.00809].

In this usage, SCUBA is a protocol family rather than a benchmark. Its significance lies in combining single-radio hardware reuse, unlicensed-band sidelink, and explicit latency–energy control, with the synchronized extension removing the assumption that communicating devices already share fine timing.

## 5. SCUBA as Scuba Search in evolutionary computation

In evolutionary computation, **Scuba Search**, or **SCUBA**, is a local-search heuristic designed to exploit **neutrality** in fitness landscapes by explicitly favoring regions where the search can increase **evolvability** [0707.0643]. The method is based on the observation that large flat regions, or **neutral networks**, can be traversed without immediate fitness gain, and that among equally fit solutions some have better future opportunities for improvement than others.

The algorithm alternates between two phases. In **Conquest-of-the-Waters**, the search remains within the neutral neighborhood and repeatedly chooses a neutral neighbor with maximal evolvability. If \(\mathcal{V}_n(s)\) denotes the neutral neighborhood, the algorithm computes
\[
M = \max \{ evol(s') \mid s' \in \mathcal{V}_n(s) - \{s\} \},
\]
and, when \(evol(s) < M\), moves to a neutral neighbor \(s'\) such that \(evol(s') = M\). In **Invasion-of-the-Land**, once no neutral neighbor has higher evolvability, the search moves to a fitter solution outside the neutral neighborhood. The paper defines a local maximum as
\[
\forall s' \in \mathcal{V}(s_{opt}),\ f(s') \le f(s_{opt}),
\]
and a strict local maximum as
\[
\forall s' \in \mathcal{V}(s_{opt}),\ f(s') < f(s_{opt}).
\]

The empirical study uses **NKq landscapes** with \(N=64\), a one-bit mutation neighborhood, **\(10^3\) runs per parameter setting**, and, for Netcrawler, a maximum of **300 steps** [0707.0643]. SCUBA is compared against **Hill Climbing (HC)**, **Hill Climbing Two Steps (HC2)**, and **Netcrawler (NC)**. The paper reports that SCUBA generally outperformed HC, HC2, and Netcrawler in terms of average fitness, especially when neutrality was high: the advantage was especially clear for **\(q=2\)** and **\(q=3\)**, whereas for **\(q=100\)** all methods were nearly equivalent except Netcrawler.

The paper also emphasizes cost. SCUBA is not the cheapest heuristic: HC has relatively low evaluation cost, HC2 is much more expensive, Netcrawler has the lowest fixed evaluation cost in the reported setup, and SCUBA is typically between HC and HC2. The key interpretation given is that SCUBA’s gain is not explained simply by more evaluations; rather, it comes from using neutral moves to alter the structure of the search trajectory. In this sense, SCUBA is a method for **neutral search with purpose**, not neutral drift.

## 6. Astronomical lineage and non-acronym uses

In astronomy, the supplied literature uses SCUBA primarily through **SCUBA-2**, which is described as a **next-generation direct-detection camera for the James Clerk Maxwell Telescope (JCMT)** and the successor to the original **SCUBA** instrument [1011.5876]. SCUBA-2 is a **dual-wavelength, 10,000-pixel bolometer camera** with **10,240 total detectors**, or **5120 pixels per focal plane**, operating at **450 \(\mu\)m** and **850 \(\mu\)m** [1301.3650]. The instrument maps the sky **100–150 times faster than SCUBA**, uses continuous scan-based observing rather than chopped staring, and is paired with a reduction stack centered on **SMURF** and **ORAC-DR** [1301.3650, 1011.5876]. The observing strategy is explicitly time-domain: scan patterns such as **“pong,” “lissajous,” and “daisy”** are chosen so that the telescope does not revisit the same sky position at a fixed time interval, helping separate sky-fixed signal from time-varying noise [1011.5876].

A second disambiguation concerns ordinary diving usage. Several underwater-robotics papers use **scuba** to refer to **scuba divers** or **scuba diving**, not to an acronym. Examples include visual diver face recognition for underwater human-robot interaction [2011.09556], underwater image transmission on commodity mobile devices for scuba diving communication [2502.10891], and robotic classification of diver swimming states for diver safety monitoring [2510.17863]. These works are adjacent only lexically: they concern underwater perception, communication, and monitoring rather than benchmarks, sidelink protocols, or neutrality-aware optimization.

Taken together, the term SCUBA functions less as a single concept than as a recurring acronym and naming root across distinct technical domains. In current arXiv usage, its most elaborated standalone expansion is the Salesforce enterprise-agent benchmark [2509.26506]; in communication systems it denotes a single-radio unlicensed sidelink protocol and its synchronization extension [2012.04074; 2104.00809]; in evolutionary computation it names a neutrality-aware search heuristic [0707.0643]; and in astronomy it remains embedded in the instrument lineage culminating in SCUBA-2 [1301.3650; 1011.5876].

Source: https://www.emergentmind.com/topics/scuba