---
title: Tongue Image Segmentation Model (TOM)
url: https://www.emergentmind.com/topics/tongue-image-segmentation-model-tom
type: topic
---

# Tongue Image Segmentation Model (TOM)

A Tongue Image Segmentation Model (often abbreviated as TOM) refers to any computational model designed for the automatic delineation of tongue regions within medical or clinical-grade images. TOMs support both foundational research and clinical applications in areas including speech production, disease diagnosis, and, most prominently, traditional Chinese medicine (TCM) tongue diagnosis. Modern TOMs span classic convolutional neural networks (CNNs), advanced transformer-based architectures leveraging prompt-based segmentation, multi-task pipelines incorporating classification, and domain-adaptive or zero-shot segmentation frameworks. Distinct imaging modalities—RGB photography, ultrasound, CT, and MRI—necessitate tailored TOM architectures, loss functions, and evaluation protocols.

## 1. Model Architectures and Core Methodologies

TOM design is modality- and task-dependent, but prevailing architectures derive primarily from fully convolutional networks, with the U-Net structure being foundational. For planar RGB tongue images, deep encoder–decoder networks perform pixelwise semantic segmentation, often integrating spatial priors or attention mechanisms.

Notable TOM frameworks and their architectural highlights include:

| Model/Approach         | Backbone/Key Blocks          | Salient Features/Strategies        |
|------------------------|-----------------------------|-------------------------------------|
| BGHNet [2003.06529]    | Lightweight DSS-nbt + HRFFM  | Context + boundary hybrid loss      |
| Ammonia-Net [2310.03472] | U-Net, ShuffleNetV2         | Multi-task with class correction    |
| TOM (Distillation) [2508.14932] | SAM, U-Net, DeepLabV3, TinyViT | Multi-teacher KD, diffusion augmentation |
| TongueSAM [2308.06444] | ViT-H/16 (SAM) + YOLOX      | Auto-prompt, zero-shot adaptation   |
| Memory-SAM [2510.15849]| DINOv3 + SAM2 (Hiera-L)     | Retrieval-to-prompt, training-free  |
| BowNet [1906.04232]    | Encoder–Decoder + dilation  | Parallel multi-scale features       |

Ultrasound and MRI tongue segmentation demand fine structure preservation with minimal annotated data. U-Net and its derivatives (e.g., Dense U-Net [1907.10210], wBowNet [1906.04232]) are routinely employed, with modifications such as batch normalization, dropout regularization, and domain-adaptive fine-tuning protocols [1906.04301]. TOMs for 4D-CT rely on volumetric nnU-Net architectures, exploiting 3D contextual cues and self-configurable preprocessing [2501.17897].

## 2. Data Modalities, Preprocessing, and Augmentation

*Editor's term: “Imaging-adaptive TOM”* refers to the fact that TOMs adjust design and preprocessing to the characteristics of the input modality.

**RGB Photographs**  
- Datasets typically include several hundred to several thousand images, spanning laboratory-controlled, web-harvested, and in-the-wild smartphone photos [2508.14932], [2308.06444].
- Preprocessing: resizing/cropping (often to 1024×1024 or 512×512), standard normalization to ImageNet statistics, and, when required, artifact suppression.
- Augmentation: color jitter, random rotations/scales, synthetic images via diffusion models [2508.14932].

**Ultrasound**  
- Frames downsampled to 128×128, with manual splines or binary contours as ground truth [1907.10210], [1906.04301], [1906.04232].
- Augmentation: horizontal flip, narrow-range rotations (±5–15°), zooms; “informed undersampling” for representativeness [1906.04301].
- Postprocessing: skeletonization and spline smoothing for tongue edge extraction.

**MRI and CT**  
- MRI: center-cropping, histogram equalization, manual contour annotation by consensus across annotators [2412.04893].
- 4D-CT: intensity clipping, z-axis resampling to isotropic spacing, z-score normalization of patches [2501.17897].

**Diffusion-based Augmentation** (for robust generalization in photometric conditions):  
- Forward and reverse denoising, mask-to-image synthesis with Stable Diffusion, and text-to-image synthesis to expand clinic-relevant appearance diversity [2508.14932].

## 3. Loss Functions, Evaluation Metrics, and Objective Design

Loss function selection reflects the severe class imbalance (contour pixels << background) and the need to jointly optimize for region and boundary precision.

**Common formulations:**
- **Binary Cross-Entropy (BCE):**  
  $$L_{\mathrm{BCE}} = -\frac{1}{N}\sum_{i=1}^N [y_i\log p_i + (1-y_i)\log(1-p_i)]$$  
  Used for binary contour extraction (MRI, ultrasound, boundary maps).

- **Categorical Cross-Entropy:**  
  For multi-class tasks (RGB: tongue body, tooth-marks, background), as in Ammonia-Net [2310.03472].

- **Dice Loss/Coefficient:**  
  $$\mathrm{Dice}(X,Y) = \frac{2|X\cap Y|}{|X| + |Y|}$$  
  Augments or replaces BCE to directly penalize false negatives/positives (segmentation overlap).

- **Compound, Weighted, and Hybrid Losses:**  
  - Map-level (F1), boundary-level (BF1), pixel (BCE), and patch-level (SSIM) losses in BGHNet [2003.06529].
  - Weighted BCE for severe class imbalance [2412.04893], [1907.10210].
  - Multi-task loss for segmentation + classification (Ammonia-Net):  
    $$L_{\text{total}} = L_{\text{seg}} + \lambda_B L_B + \lambda_Q L_Q$$

- **Knowledge Distillation Losses:**  
  Logits-based KL divergence, mask-level MSE, plus BCE between student and teacher(s) for parameter-efficient TOMs [2508.14932].

**Evaluation Metrics:**
- **Mean Intersection-over-Union (mIoU):** primary metric for region overlap (both binary and multi-class) [2508.14932], [2308.06444].
- **Mean Pixel Accuracy (mPA),** **Dice coefficient,** and **Boundary F1 (BF1):** important for evaluating boundary-specific improvements [2003.06529].
- **Mean Sum of Distances (MSD):** for contour-tracing benchmarks in ultrasound/MRI [2412.04893], [1907.10210], [1906.04232].

## 4. Innovations in Automated Prompting and Zero-/Few-Shot Segmentation

The advent of large prompt-driven segmentation models (SAM, MedSAM, Hiera-SAM2) has yielded a class of TOMs with novel mechanisms for automatic and robust tongue region localization.

**Automatic Prompt Generation and Zero-Shot Transfer:**
- TongueSAM [2308.06444] integrates a YOLOX object detector for box prompt generation, routed to a frozen SAM ViT-H backbone with only Mask Decoder fine-tuning. This enables “zero-shot” adaptation across data domains.  
  - mIoU: ~95.23% zero-shot on web-harvested data, significantly outperforming classic DeepLabv3+ [2308.06444].

- Memory-SAM [2510.15849] eliminates human-prompt dependency entirely by using DINOv3 ViT-L/16 feature retrieval and mask-constrained foreground/background matching; images are segmented without training or manual input, attaining mIoU = 0.9863 on mixed controlled/wild datasets.

**Knowledge Distillation for Model Compression:**
- TOM [2508.14932] introduces a multi-teacher distillation regime: SAM (LoRA-tuned), U-Net, and DeepLabV3 teacher models guide a parameter-efficient TinyViT-based student. The student TOM reduces parameter count by 96.6% compared to the teacher, retaining 95.22% mIoU.

## 5. Segmenting Specialized Structures and Multi-task Learning

TOMs address both holistic tongue boundary segmentation and the delineation of intra-tongue pathologies (e.g., tooth-marks in TCM). For example:

- **Ammonia-Net [2310.03472]:** Simultaneous segmentation of tongue body and tooth-marks. Embeds pixel-level outputs back into the classifier pipeline to boost 4-class TCM diagnosis accuracy (F1 = 76.85%; 2-class tooth-marked/non-tooth-marked, accuracy = 99.06%).

- **BGHNet [2003.06529]:** Achieves real-time inference speeds (≈30–40 fps) and high mIoU (0.9911) in clinical images using a lightweight design, with explicit architectural pathways and loss terms for precise boundary focus—critically relevant for TCM or any application sensitive to tongue region demarcations.

- **nnU-Net for 4D-CT [2501.17897]:** Deploys automatic preprocessing and a 3D U-Net “cascade” for organ motion analysis in swallowing studies, with tongue median Dice = 0.85, enabling dynamic tongue/bone/bolus tracking in volumetric data.

## 6. Failure Modes, Limitations, and Remediation Strategies

Failure modes recurrent in TOM development include:
- Boundary ambiguity due to contact with adjacent tissue or bolus (CT: palate, barium [2501.17897]).
- Motion blur and rapid movement artifacts (notably in functional MRI/CT or ultrasound) leading to “smearing” and minor under-segmentation.
- Interference from metal artifacts (e.g., dental work).
- In ultrasound/MRI, low contrast and speckle noise limit delineation; solutions are batch normalization, dropout, and careful contour post-processing (Dijkstra path smoothing, skeletonization) [2412.04893], [1907.10210].
- Supervised models overfit when training-data distribution is narrow; zero-shot and distillation-based TOMs generalize better but can still be sensitive to prompt misalignment (if auto-generated boxes overlap non-tongue regions) [2308.06444].

Suggested remedies, as identified within the literature:
- Expanding and diversifying annotated datasets, especially for rare or ambiguous presentations.
- Multi-stage segmentation (bones, cartilages as spatial priors).
- Interactive correction tools for efficient manual refinement.
- Incorporation of synthetic augmentation (diffusion modeling) to address domain shift [2508.14932].

## 7. Practical Deployment, Tool Availability, and Impact

TOM deployment spans from standalone offline executables to cloud-based batch segmentation services:

- **TOM (2025) Tool [2508.14932]:** Provides an online web UI and downloadable offline tool requiring no programming. Capable of processing bulk uploads of clinical tongue photographs, exporting both masks and patch-extracted tongue regions for downstream diagnosis.

- **Open Source Implementations:**  
  - CNN-based tracking tool in ultrasound: full pipeline and pre-trained weights publicly available [1907.10210].
  - Memory-SAM codebase at https://github.com/jw-chae/memory-sam [2510.15849].

- **Impact on Downstream Diagnosis:**  
  - Precise segmentation improves explainability and classification accuracy for TCM constitution typing; isolating the tongue surface from background artifacts enhances feature learning and clinician interpretability [2508.14932].

- **Real-Time and Clinical Relevance:**  
  Lightweight and high-speed TOMs (e.g., BGHNet) occupy a crucial niche in point-of-care settings, balancing region accuracy, edge fidelity, and computational cost to support clinical workflows [2003.06529].

Tongue Image Segmentation Models represent a technologically diverse, rapidly maturing field, with cross-fertilization between classical dense architectures, large prompt-based models, and diffusion-driven augmentation pipelines. Advances in data efficiency, generalization, and tool usability underpin their increasing role in both research and applied clinical diagnostics.

Source: https://www.emergentmind.com/topics/tongue-image-segmentation-model-tom