SALMONN‑Guard: AI Safety & Aquaculture Monitoring

• SALMONN‑Guard is a context-dependent term defining both a multimodal LLM safety guard and a salmon welfare monitoring system.
• In the safety domain, the paper introduces an audio-aware guard built on Qwen2.5‑Omni‑7B that achieves an 11.32% overall attack success rate against compositional audio attacks.
• In aquaculture, the technical synthesis presents BoostCompTrack, a pose-guided tracking framework that reduces salmon ID transfers by up to 75% and body-part ID switches by around 74%.

SALMONN-Guard is a name used in two distinct 2025 research contexts. In multimodal AI safety, it denotes a safeguard LLM that jointly inspects speech, non-speech audio, and text to mitigate compositional audio jailbreaks introduced through SACRED-Bench (Yang et al., 13 Nov 2025). In industrial aquaculture, the same label is used in a technical synthesis for a salmon welfare monitoring and alerting stack whose core is BoostCompTrack, a pose-guided, body-part-aware tracking framework for underwater salmon scenes (Høgstedt et al., 30 Sep 2025). The shared term therefore identifies two unrelated systems: one is an audio-aware safety guard for multimodal LLMs, and the other is a proposed integration architecture for continuous salmon welfare monitoring.

1. Dual usage and nomenclature

In the supplied literature, SALMONN-Guard does not refer to a single architecture or research lineage. One usage belongs to multimodal model safety; the other belongs to salmon aquaculture monitoring. Domain context is therefore essential for disambiguation.

Usage	Domain	Technical core
SALMONN-Guard	Multimodal LLM safety	Qwen2.5-Omni-7B with LoRA-based supervised fine-tuning
SALMONN-Guard	Salmon welfare monitoring and alerting	BoostCompTrack with YOLOv8 pose/detection models and part-aware fusion

The multimodal usage appears in "Speech-Audio Compositional Attacks on Multimodal LLMs and Their Mitigation with SALMONN-Guard" (Yang et al., 13 Nov 2025). The aquaculture usage is a technical synthesis built on "A Multi-purpose Tracking Framework for Salmon Welfare Monitoring in Challenging Environments" (Høgstedt et al., 30 Sep 2025). This separation matters because the two systems solve different problems, use different modalities, and are evaluated with different benchmarks.

2. SALMONN-Guard in multimodal safety: threat model and benchmark

In the multimodal-safety setting, SALMONN-Guard is motivated by the claim that existing safeguards are usually text-centric and therefore miss hazards arising from cross-modal composition. The benchmark introduced alongside it, SACRED-Bench, targets realistic audio jailbreaks that do not require white-box access or perturbation optimization. Its three mechanisms are speech overlap and multi-speaker dialogue, speech-audio mixture, and diverse spoken instruction formats such as binary yes/no and open-ended requests (Yang et al., 13 Nov 2025).

SACRED-Bench is organized around harmful content that can be distributed across speech and non-speech audio. In the overlap condition, harmful speech is embedded beneath or alongside benign speech through cross-fade, attenuation, speed change, and temporal overlap. In the dialogue condition, a direct harmful instruction is transformed into a multi-turn conversation between distinct speakers and paired with a benign textual pointer such as a request to use the methods mentioned in the discussion. In the speech-audio-mixture condition, benign speech is overlaid with explicit sexual, violent, or criminal non-speech audio so that the unsafe intent is implied by context rather than explicit harmful wording. The benchmark contains approximately 30 hours of training audio and 7 hours of test audio; the test partition sizes are $N = 400$ for Speech Overlap, $N = 1{,}364$ for Multi-speaker Dialogue, and $N = 717$ for Contextual Audio Cues (Yang et al., 13 Nov 2025).

Its primary metric is Attack Success Rate. In the binary setting, where each sample is harmful and the correct answer is “Yes,” attack success is a “No” response:

$$\mathrm{ASR} = \frac{1}{N}\sum_{i=1}^{N}\mathbf{1}[\hat{y}_i = \text{“No”}].$$

In the open-ended dialogue setting, attack success is defined by a judge model:

$$\mathrm{ASR} = \frac{1}{N}\sum_{i=1}^{N}\mathbf{1}[\mathrm{Judge}(\mathrm{model\_output}_i)=\text{“harmful”}].$$

Overall ASR is a weighted average across mechanisms:

$$\mathrm{ASR}_{\mathrm{overall}} = \frac{\sum_k N_k \cdot \mathrm{ASR}_k}{\sum_k N_k}.$$

The benchmark is intended to expose what the paper characterizes as cross-modal blindness: models may correctly transcribe benign foreground speech while failing to recognize harmful content encoded in overlapped speech, long-form dialogue, or non-speech background events.

3. Architecture and training of the audio-aware guard model

SALMONN-Guard, in this usage, is a specialized audio guard model built on Qwen2.5-Omni-7B and designed to inspect speech, non-speech audio, and text jointly (Yang et al., 13 Nov 2025). The backbone uses the base model’s audio encoder for speech and non-speech audio and the LLM for text, with an aligner module integrating audio and text features. The paper states that the fusion and decision are learned end-to-end during supervised fine-tuning, so that safety judgments depend on cross-modal evidence rather than text alone.

The system supports two output modes. In classification mode, audio-only input is mapped to a discrete harmful-versus-harmless decision through a binary decision head. In generative-refusal mode, joint text-and-audio input is answered either with a refusal such as “I can’t help with that” or with a pass-through response if the input is judged safe. Exact audio feature details such as spectrogram parameters are not specified; the paper states only that audio is fed through the native audio pathway of Qwen2.5-Omni-7B and aligned to text-token space by the aligner (Yang et al., 13 Nov 2025).

Training uses approximately 10k instances in total: 8,545 harmful and 1,828 benign. The harmful compositions are distributed as 6,640 speech-overlap examples, 986 speech-audio-mixture examples, and 919 multi-speaker-dialogue examples. Prompts are derived conceptually from AdvBench, MM-SafetyBench, and HarmBench via GPT-4o, then realized as audio with ChatTTS and curated non-speech content. The training protocol applies LoRA adapters to the LLM, audio encoder, and aligner, and uses a two-stage curriculum: 3 epochs on the full dataset followed by 5 additional epochs on multi-speaker dialogue (Yang et al., 13 Nov 2025).

The supervised objective is standard cross-entropy. For binary classification, with input $x$ and label $y \in \{\mathrm{safe}, \mathrm{unsafe}\}$, the loss is

$$L = -\frac{1}{N}\sum_{i=1}^{N}\log p_\theta(y_i \mid x_i).$$

For refusal-style generation, token-level cross-entropy is used to train the safety-aligned response. Deployment is described as a front-line guard: it ingests text-plus-audio input, blocks or refuses unsafe content, and forwards safe content to a downstream multimodal LLM.

4. Empirical behavior of the guard model

SACRED-Bench shows that baseline multimodal models remain highly vulnerable to compositional audio attacks. Gemini 2.5 Pro reaches 37.25% ASR on Speech Overlap, 63.93% on Multi-speaker Dialogue, 88.56% on Speech-Audio Mixture, and 66.75% overall. GPT-4o reaches 70.05% overall ASR, and several open-source models are higher still, including Qwen2-Audio-7B at 98.16% and Qwen2.5-Omni-7B at 92.83% (Yang et al., 13 Nov 2025).

Against this benchmark, SALMONN-Guard reports 11.32% overall ASR, with 12.93% on Speech Overlap, 14.08% on Multi-speaker Dialogue, and 5.16% on Speech-Audio Mixture (Yang et al., 13 Nov 2025). On the benign audios used to create the benchmark composites, its classification accuracy is 100.00%, meaning none of those benign carrier audios were falsely flagged in that set. The paper also reports generalization to other audio jailbreaks despite training only on SACRED-Bench: SALMONN-Guard reaches 0.00% ASR on JALMBench Speech Insertion and 3.00% on JALMBench Speech Editing.

Several ablations clarify why the benchmark is difficult. On Gemini 1.5 Pro, a dialogue ablation reports 57.96% ASR for text-only harmful instruction, 65.03% for audio-only harmful speech, and 78.58% for the combined text-plus-audio dialogue attack, indicating modality synergy (Yang et al., 13 Nov 2025). A speech-overlap ablation on a 100-sample subset against Gemini 2.5 Pro further reports that higher playback speed, stronger attenuation, and longer overlap correlate with higher ASR; representative settings include $(1.5\times, -8\,\mathrm{dB}) \rightarrow 61.00\%$ and $N = 1{,}364$0. The paper also states that 93% of overlapped samples remained perceptible to Gemini 2.5 Pro via audio-description checks.

The limitations are correspondingly specific. Residual vulnerabilities remain, particularly on overlapped and dialogue attacks, where ASR remains in the 12–14% range. Broader false-positive and false-negative trade-offs are not fully reported beyond the benchmark’s benign carriers. Open-ended dialogue scoring relies on Gemini 2.5 Pro as judge, which is a practical evaluation choice but inherits the biases of that judge. The study centers on English synthetic speech and curated non-speech audio rather than broad multilingual coverage.

5. SALMONN-Guard in aquaculture: monitoring architecture and tracking design

In the aquaculture usage, SALMONN-Guard is described in a technical synthesis as a comprehensive salmon welfare monitoring and alerting system built around BoostCompTrack, or BCT (Høgstedt et al., 30 Sep 2025). The underlying motivation is that available computer-vision methods for salmon welfare indicators often compute each indicator separately and inherit detectors and trackers from unrelated domains, leading to high resource demand and vulnerability to underwater challenges such as occlusion, similar appearance, and similar motion.

The proposed pipeline begins with lateral-facing monocular cameras in net pens, operating at 25–30 FPS and resolutions from $N = 1{,}364$1 to $N = 1{,}364$2. Pose-guided detection is then performed with Ultralytics YOLOv8 detection and “pose” variants trained to emit bounding boxes for both whole salmon and body parts. The system explicitly references the head, body, dorsal fin, adipose fin, pelvic fin, anal fin, tail fin, and left and right pectoral fins. Body-part boxes are represented as two corner keypoints, denoted anterodorsal and posteroventral corners, and linked to their parent whole-salmon box via a keybox representation (Høgstedt et al., 30 Sep 2025).

Five model variants are specified: M1 as yolov8n-pose at 640 px, M2 as yolov8 detection at 640 px, M3 as yolov8m-pose at 1024 px, M4 as yolov8m detection at 1024 px, and M5 as yolov8m-pose at 1024 px trained on all annotated-box datasets. Training uses 10,000 epochs, batch size 8, random rotation $N = 1{,}364$3, scale 0.8, perspective 0.0001, and default Ultralytics settings otherwise. Detector thresholds for confidence, keybox confidence, and IoU are selected by maximizing F1 on the CS train set (Høgstedt et al., 30 Sep 2025).

At inference time, part quality is encoded through the keybox confidence

$N = 1{,}364$4

where $N = 1{,}364$5 and $N = 1{,}364$6 are the two corner keypoints and $N = 1{,}364$7 is the confidence of the parent salmon box. Tracking is then carried out by instantiating nine subtrackers per salmon for the body parts and one for the salmon body itself. Each subtracker is identical to BoostTrack’s bounding-box tracker, with IoU-based matching, configurable IoU threshold, and hidden-length parameter; appearance embeddings are disabled in the reported experiments.

Two specialized fusion modules are central to robustness. TurnModule detects turning fish and relaxes association thresholds for them. Its turning-state counter is

$N = 1{,}364$8

where $N = 1{,}364$9 if the salmon is away from image boundaries and either box height exceeds box width or $N = 717$0, and $N = 717$1 if $N = 717$2 and at least seven of nine body parts are visible. If $N = 717$3 and the best match for an unmatched turning tracker is unused and has IoU $N = 717$4, the match is accepted even if it lies below the standard IoU threshold (Høgstedt et al., 30 Sep 2025).

CrowdedModule addresses dense-schooling failure modes with three checks. The bpdis rule terminates the salmon-box tracker and discards the detection if more small parts disagree than agree with the salmon-box association and at least two parts disagree. The nobp rule terminates the tracker and discards the detection if no small parts overlap the same detection matched to the salmon box. The bpiou rule sets a part-detection confidence to 0 if that part has higher IoU to a different detection than to the detection associated with the salmon box, thereby suppressing low-quality parts.

6. Welfare analytics, datasets, and quantitative results in aquaculture

The tracking output is used directly for welfare analytics, especially tail-beat analysis. Three time series are defined: TIP, tailfin width, and salmon width (Høgstedt et al., 30 Sep 2025). TIP is a body-part-derived state in $N = 717$5 computed from the centers of the anal fin (ANF), adipose fin (APF), tail fin (TF), and body (B). Let $N = 717$6 be the intersection of the segments $N = 717$7 and $N = 717$8; then

$N = 717$9

Signal processing applies Savitzky-Golay smoothing with window length 11 frames and polynomial order 2, followed by SciPy find_peaks with prominence tuned by 50-step binary search to approximately match ground-truth extrema counts. Tail-beat wavelength in frames is the distance between consecutive extrema, and the temporal and spatial conversions are

$$\mathrm{ASR} = \frac{1}{N}\sum_{i=1}^{N}\mathbf{1}[\hat{y}_i = \text{“No”}].$$0

with speed estimated from tracked body-center motion after pixel-to-meter calibration, using the design extension

$$\mathrm{ASR} = \frac{1}{N}\sum_{i=1}^{N}\mathbf{1}[\hat{y}_i = \text{“No”}].$$1

Three datasets support training and evaluation. CrowdedSalmon (CS) targets identity transfers in dense schooling scenes; its training split contains 31 images, 1,010 salmon, and 7,627 boxes, while validation contains 6 images, 871 salmon, and 5,814 boxes, with global salmon IDs across frames sampled 10 frames apart. TurningSalmon (TS) targets identity maintenance through turning events; its training split contains 8 images, 679 salmon, and 5,040 boxes, while validation contains 100 frames and 146 endpoint-labeled tracks categorized as Turning, Straight, Occluded, Turning+Occluded, or Background Occluded. TailbeatWavelength (TBW) targets forward-swimming wavelength estimation; its training split contains 46 images, 1,529 salmon, and 10,615 boxes, and its validation split contains 1,000 consecutive frames with manually labeled tail-beat extrema for selected high-quality tracks (Høgstedt et al., 30 Sep 2025).

On CS, the synthesis reports consistent improvements over BoostTrack. With hidden length 3 and keybox model M3, BCT all achieves 1.4 salmon ID transfers versus 5.5 for BT M3, approximately a 75% reduction, and 90 body-part ID switches versus 348, approximately a 74% reduction. With hidden length 30 and M3, BCT all yields 2.9 salmon ID transfers versus 9.4 and 68 body-part ID switches versus 331. Similar reductions are reported with M1. Salmon matches decrease slightly because CrowdedModule rejects dubious associations conservatively, while body-part matches increase by roughly 19–22% depending on model and hidden length (Høgstedt et al., 30 Sep 2025).

The ablation study attributes different gains to different modules. On CS with hidden length 3 and M3, bpdis and nobp markedly reduce salmon ID transfers; nobp triggers more often and therefore causes slightly more ID switches and fewer matches. The bpiou rule reduces both body-part ID transfers and body-part ID switches by suppressing low-quality parts. TurnModule has negligible effect on CS, which the synthesis interprets as task specificity because CS contains no turning events.

On TS, BCT maintains approximately its initial number of turning-track matches, within $$\mathrm{ASR} = \frac{1}{N}\sum_{i=1}^{N}\mathbf{1}[\hat{y}_i = \text{“No”}].$$2 across IoU thresholds and hidden-length settings, whereas BoostTrack’s match counts fall more as thresholds tighten. On TBW, part-aware signals outperform the whole-salmon width signal. Averaged over matching thresholds 1–4 frames, TIP achieves TP 66.00, FP 34.00, FN 33.00; tailfin width gives TP 63.25, FP 37.75, FN 35.75; and salmon width gives TP 42.50, FP 56.50, FN 56.50. The synthesis notes that TIP can infer tail motion direction relative to the camera but can flip phase when the fish passes above or below the camera, whereas tailfin width is less sensitive to phase inversion but cannot easily discriminate motion direction (Høgstedt et al., 30 Sep 2025).

7. Limitations, deployment implications, and significance

The two SALMONN-Guard usages share a general architectural idea—one front-end system supports downstream decision-making—but they differ completely in modality, evaluation, and failure mode. The multimodal-safety system is a 7B guard model that precedes a target MLLM and decides whether speech, audio, and text should be blocked or passed through (Yang et al., 13 Nov 2025). The aquaculture system is a proposed monitoring-and-alerting stack in which a single pose-guided tracker supplies multiple welfare indicators such as tail beat, turning, lethargy, segmentation prompts, and possible future breathing or mouth-opening analytics (Høgstedt et al., 30 Sep 2025).

Their limitations are likewise domain-specific. In the audio-guard setting, residual ASR remains non-zero, especially for overlap and dialogue; broader real-world false-positive behavior is not fully quantified; and open-ended scoring depends on an LLM judge (Yang et al., 13 Nov 2025). In the aquaculture setting, total occlusions, distant fish, inaccurate keypoint localization, turbidity, and low light remain difficult; longer hidden-lengths can preserve identity through occlusion but may increase transfers; and long-term identity maintenance would require an additional re-identification layer. The aquaculture synthesis also identifies 3D tracking, multi-view calibration, sonar or depth fusion, domain adaptation across sites and seasons, and meters-per-pixel calibration as future extensions (Høgstedt et al., 30 Sep 2025).

A plausible implication is that the shared name should not be treated as evidence of conceptual continuity between the two projects. In one case, SALMONN-Guard denotes an audio-aware safety filter trained with SACRED-Bench and released with checkpoints at the SACRED-Bench dataset page. In the other, it denotes an integration concept whose practical core is BoostCompTrack, with datasets and code released at https://github.com/espenbh/BoostCompTrack. The term is therefore best understood as a context-dependent label attached to two technically unrelated systems rather than as a single established platform.