Semantic-Aware Ship Detection

• Semantic-Aware Ship Detection is an approach that embeds detailed semantic cues—such as contextual information, sea-land priors, and textual descriptors—directly into detection pipelines.
• It employs diverse methodologies including semantic enhancement in SAR detectors, scene-aware attention, multitask segmentation, and vision-language conditioning to boost accuracy.
• This integration improves ship localization and reduces false positives while paving the way for advanced remote sensing applications and future enhancements.

Searching arXiv for the cited SASD and related ship-detection papers to ground the article in current literature. Semantic-Aware Ship Detection (SASD) denotes ship-detection formulations that do not stop at object localization, but explicitly encode semantic information such as long-range contextual cues, sea-land priors, pixel-level target likelihood, fine-grained textual attributes, spatial relations, density, size categorization, and risk behavior understanding. In current remote-sensing literature, the term spans several non-identical paradigms: semantic enhancement inside SAR detectors, scene-aware attention suppression, multitask denoising-segmentation-detection, detector-guided visual question answering for SAR scenes, and text-conditioned ship localization without box-level annotations (Ke et al., 2022, Ke et al., 15 Jun 2025, Zhao et al., 2024, Wang et al., 2024, Li et al., 21 Aug 2025).

1. Conceptual scope and definitions

The term “semantic-aware” is used in multiple, partially overlapping senses. In one SAR formulation, SASD refers to explicitly enhancing and propagating semantic information across scales—especially into shallow maps—so the detector can distinguish ships from textured clutter and contextually confusing structures (Ke et al., 2022). In another, it denotes integrating a semantic prior, specifically sea versus land, into the detection network to reduce false positives on land and improve focus on sea regions where ships reside (Ke et al., 15 Jun 2025). A multitask formulation makes semantic awareness explicit through a rotated Gaussian-mask target segmentation branch whose pixel-level output encodes that ship centers have the highest probabilities and gradually decrease outward under a Gaussian distribution (Zhao et al., 2024). A vision-language formulation for SAR defines SASD as transcending pure object localization by coupling low-level detections with high-level semantics such as spatial relations, density patterns, size categorization, and risk behavior understanding (Wang et al., 2024). A text-driven remote-sensing formulation defines SASD as ship detection conditioned on fine-grained semantic descriptions of ships beyond coarse category labels (Li et al., 21 Aug 2025).

Taken together, these papers suggest that SASD is best understood as an umbrella formulation rather than a single architecture. The common denominator is not a specific backbone, loss, or supervision regime, but the insertion of semantic structure into ship detection. That structure may be encoded as enriched feature pyramids, scene priors, auxiliary segmentation masks, detailed text prompts, cross-modal similarity, or grounded dialogue. A plausible implication is that SASD marks a shift from the question “where are the ships?” toward formulations that also ask which ships are present, how they relate to their surroundings, and what scene-level interpretation follows from the detections.

2. Main technical paradigms

The literature currently exhibits several distinct SASD paradigms.

Representative variant	Modality	Distinguishing mechanism
Swin-T + FEFPN (Ke et al., 2022)	SAR	Long-range dependency modeling with Swin Transformer and gradual top-down semantic enhancement of shallow features
ULSM + LASM (Ke et al., 15 Jun 2025)	SAR	Unsupervised inshore/offshore classification, Otsu sea-land masks, and feature-level land attention suppression
MLDet (Zhao et al., 2024)	SAR	Joint rotated detection, speckle suppression, rotated Gaussian-mask segmentation, and WRBF fusion
YOLOv8n + Qwen2-VL-72B (Wang et al., 2024)	SAR	Detector-grounded multi-turn VQA without additional datasets or VLM fine-tuning
ViT-ShipSemVL + adaptive sliding windows (Li et al., 21 Aug 2025)	Optical remote sensing	Fine-tuned CLIP and text-conditioned window ranking without box-level annotations

Detector-centric SASD methods in SAR treat semantics as an internal property of the perception stack. They improve the semantic quality of feature maps, inject scene priors, or add auxiliary tasks so that the detector becomes less sensitive to speckle, clutter, wakes, docks, and small-target ambiguity. Vision-language SASD methods, by contrast, treat semantics as an explicit external conditioning signal, either by converting detections into grounded prompts for a large VLM or by learning a text-image similarity space that localizes ships matching detailed descriptions.

This division is not absolute. The SAR VQA framework still depends on a strong detector to produce grounded facts, while the annotation-light CLIP framework still performs localization through an image-search procedure. Nonetheless, the distinction clarifies the field: some SASD systems enrich detection with semantics, whereas others use semantics to define or interrogate the detection task itself.

3. Detector-internal semantics in SAR

A foundational detector-internal SASD design is the Swin Transformer plus Feature Enhancement Feature Pyramid Network. The detector uses a two-stage Faster R-CNN pipeline with a Swin-T backbone, hierarchical feature maps with channel sizes $\{96,192,384,768\}$, and block counts $\{2,2,6,2\}$, followed by FEFPN for multi-level semantic enhancement (Ke et al., 2022). Swin’s windowed and shifted-window self-attention is used to model long-range dependencies, with

$$Q = XW_Q,\quad K = XW_K,\quad V = XW_V,\quad A = \mathrm{softmax}\!\left(\frac{QK^T}{\sqrt{d_k}}\right)V.$$

The neck then propagates semantics downward through three top-down paths. Path 1 is FPN-like; Path 2 introduces a $0.5\times$ weighting before addition; Path 3 uses a $0.25\times$ weighting and residual connections from shallow layers of Path 1 to corresponding shallow layers in Path 3. The design rationale is explicitly semantic rather than geometric: because Swin is described as translational semi-invariant, FEFPN strengthens shallow semantic content without overemphasizing noisier spatial detail.

Scene-aware SASD introduces semantics as a prior over the scene layout itself. In the sea-land-guided method, ULSM first classifies SAR images into inshore and offshore via K-means with $K=2$, using ResNet-50 features, and then applies Otsu thresholding only to inshore scenes to generate sea-land masks (Ke et al., 15 Jun 2025). LASM uses these masks to suppress land responses after the backbone and before RPN and ROI heads. The paper gives the attention map as

$A = M \cdot (1-\omega) + (1-M)\cdot I,$

with $M=1$ for sea and $M=0$ for land, while also explaining that implementation should ensure that land pixels receive the suppression factor. This ambiguity is noteworthy because the method’s semantic claim depends on suppressing land rather than sea. The broader significance is clear: scene semantics can be introduced without additional supervision, by converting unsupervised scene parsing into feature-level attention control.

MLDet embeds semantics through multitask coupling. Its backbone is CSPDarknet with 23 residual blocks arranged in five stages, and the framework jointly optimizes rotated-object detection, speckle suppression, and rotated Gaussian-mask target segmentation (Zhao et al., 2024). The angle classification loss with aspect ratio weighting is

$$\mathcal{L}_{\theta}\left(\theta,\hat{\theta}\right) = \left|\sin\left(\alpha(\theta-\hat{\theta})\right)\right| \times \mathcal{L}_{\mathrm{smoothl1}}\left(\theta,\hat{\theta}\right), \quad \alpha = \begin{cases} 1, & (h/w)>r\ 2, & \text{otherwise} \end{cases}$$

with $\{2,2,6,2\}$0. The rotated Gaussian mask encodes a semantic prior over ship centers and supports an auxiliary segmentation loss, while WRBF fuses rotated boxes from detection and segmentation. Here, semantic awareness is not language-based; it is encoded in the geometry of auxiliary supervision, denoising-induced feature refinement, and inference-time fusion.

4. Vision-language conditioning and semantic reasoning

A SAR-specific vision-language SASD framework couples a YOLO-based object detection model with Qwen2-VL-72B to perform multi-turn visual question answering on ship location, density, size, and potential risks, without constructing a multimodal SAR-specific dataset or fine-tuning the VLM (Wang et al., 2024). Several YOLO variants were benchmarked on SSDD and HRSID—YOLOv6n, YOLOv7-tiny, YOLOv8n, YOLOv10n, and YOLO11n—and YOLOv8n was selected as the object detection model because it achieved the best overall mean Average Precision across both datasets. After inference and NMS, custom Python scripts convert bounding boxes into textual descriptors such as coordinates, size in pixels, relative location by quadrant, ship counts, and grouping information. The raw SAR image and structured text are then fed to Qwen2-VL. The resulting dialogue can be grounded in explicit formulas, including

$\{2,2,6,2\}$1

with risk flagged when $\{2,2,6,2\}$2. Because no pixel-to-meter calibration is provided, distances and sizes are evaluated in pixel units. The paper reports that box-guided prompts make Qwen2-VL responses specific, data-grounded, and aligned with image content, whereas prompts without boxes yield vague or mismatched answers.

A distinct vision-language SASD framework operates directly through text-conditioned localization. It fine-tunes CLIP with ViT-B/32 on ShipSem-VL, a dedicated remote-sensing vision-language dataset containing 2,611 image-text pairs from Google Earth across 23 types of ports and surrounding sea areas worldwide, and performs localization by a multi-scale adaptive sliding window search at test time (Li et al., 21 Aug 2025). No oriented or axis-aligned detection head is trained, because the dataset contains no box annotations. Localization emerges by ranking windows with high cosine similarity to the input text prompt. The training objective is CLIP-style text-image contrastive alignment,

$\{2,2,6,2\}$3

The evaluation is decomposed into Task A for ship detection in complex backgrounds, Task B for detection of specific ship types based on complex semantic descriptions, and Task C for detection when multiple visually similar targets are present.

These two VLM-based lines instantiate different roles for semantics. In the SAR VQA system, semantics are inferred after localization and used for grounded reasoning, dialogue, and scene explanation. In the ShipSem-VL system, semantics define the target to be localized in the first place. This suggests that SASD now includes both post-detection semantic interpretation and pre-detection semantic conditioning.

5. Datasets, evaluation, and empirical evidence

The empirical base of SASD is heterogeneous. SSDD appears in multiple studies: one uses 1,160 SAR images with a 928 train / 232 test split, while another adopts an 8:2 train:test split. HRSID is used both with the official 3,642 train / 1,962 test split and as 5,604 cropped images at 800×800. MLDet evaluates on SSDD+ and HRSID, with SSDD+ described as multisensor and multiresolution, containing 1,160 images and 2,456 ships. ShipSem-VL is optical rather than SAR, which is methodologically significant because it widens SASD beyond a single sensing modality.

Paper	Experimental setting	Selected reported results
(Wang et al., 2024)	YOLO benchmark on SSDD and HRSID for ODM selection	YOLOv8n: SSDD 98.6 / 73.1 and HRSID 91.3 / 67.5 for [email protected] / [email protected]:0.95
(Ke et al., 2022)	SSDD ablation and comparison	ResNet-50 + FPN 90.30% mAP; Swin-T + FEFPN 93.08% mAP
(Ke et al., 15 Jun 2025)	SSDD two-stage scene-aware detector	mAP 63.5%, mAP50 91.8%, mAPS 63.8%
(Zhao et al., 2024)	SSDD+ and HRSID multitask OBB detection	All modules: SSDD+ AP50 0.953, AP75 0.601; HRSID AP50 0.928, AP75 0.534
(Li et al., 21 Aug 2025)	Task A/B/C manual evaluation at thresholds 0.8, 0.7, 0.5	Precision improvements are consistent; Task B precision improves by +21.46 points at 0.8 and +30.17 points at 0.7

Several empirical patterns recur. First, semantic enhancement inside the detector can produce measurable gains: Swin-T + FEFPN exceeds Swin-T + FPN by $\{2,2,6,2\}$4 points mAP, while Swin-T + PAFPN drops to 88.60% mAP, indicating that semantic strengthening and spatial-detail emphasis are not interchangeable. Second, scene priors matter: the full K-means + Otsu pipeline outperforms Otsu-only in the scene-aware detector, with mAP rising from 62.2% to 63.5% and mAP50 from 88.4% to 91.8%. Third, multitask semantic guidance is additive: in MLDet, the full combination of DFF, target segmentation, and aspect-ratio-weighted angle loss substantially exceeds the baseline on both SSDD+ and HRSID, and WRBF further improves AP50 and AP75. Fourth, fine-grained text conditioning tends to improve precision more reliably than recall: in the ShipSem-VL framework, Task B recall decreases at higher thresholds even as precision rises sharply, reflecting stricter semantic matching. Fifth, the SAR VQA framework emphasizes qualitative improvement rather than a dedicated quantitative VQA benchmark; its main result is that injected detection boxes transform vague large-model responses into grounded scene analysis.

6. Limitations, misconceptions, and future directions

The current SASD literature reports several limitations. In SAR, speckle noise, low-contrast targets, nearshore clutter, dense moorings, and small ships remain difficult. NMS may drop true positives in dense clusters, and adaptive merging in text-conditioned sliding-window methods can collapse adjacent ships into a single ROI, especially at lower thresholds. Window-based localization without box-level supervision can limit boundary precision. Scene-aware suppression may reduce false alarms on land but can also affect ships close to shore, which is consistent with the lower mAPL reported by the sea-land-guided detector. In detector-guided VQA, distances and sizes are only in pixel units when calibration metadata are absent. In the ShipSem-VL framework, modality is optical, so SAR or multi-sensor generalization is not studied. Ethical and operational considerations are also explicit in the optical VLM paper, which notes privacy, dual-use implications, and compliance with local regulations.

Two misconceptions are contradicted by the literature. The first is that SASD is synonymous with vision-language prompting. In fact, Swin-T + FEFPN, ULSM + LASM, and MLDet are all SASD formulations that do not depend on VLM dialogue. The second is that semantic awareness necessarily requires SAR-specific multimodal dataset construction and model fine-tuning. The SAR VQA framework is designed precisely to avoid those costs, whereas the ShipSem-VL framework takes the opposite route and demonstrates that dedicated fine-grained vision-language data can improve semantic discrimination. This contrast indicates that SASD is defined by how semantics enters the pipeline, not by a single annotation or fine-tuning doctrine.

The future directions proposed across the papers are convergent. They include PEFT or LoRA on Qwen2-VL for SAR-specific cues, domain adaptation for small targets, SAR-specific prompt engineering, calibrated pixel-to-meter conversion when metadata are available, improved post-processing for dense scenes, self- or weakly supervised scaling, segmentation of ship instances, multi-sensor integration such as SAR + optical or AIS, temporal tracking, prompt-tuning, larger VLMs, cross-sensor domain adaptation, multi-scale attention, and contextual priors such as shipping lanes or harbor structures. One SAR VQA paper explicitly proposes a semantic feedback loop in which VLM outputs could suggest re-thresholding in crowded zones, request focused ROI re-scans, or adjust NMS for small targets. A plausible implication is that mature SASD systems will combine detector-side semantic priors, auxiliary pixel-level tasks, and language-grounded reasoning in a closed loop rather than treating perception and interpretation as separate stages.