DeepRare System: Rare Event Modeling

• DeepRare System is an AI-driven framework that identifies and explains rare, unexpected phenomena across diverse domains.
• It combines unsupervised CNN-based saliency detection with modular, traceable reasoning for clear decision support.
• The system demonstrates robust performance in visual attention and clinical diagnosis, ensuring transparent and efficient anomaly discovery.

DeepRare System

The DeepRare System encompasses a series of methodologies and agentic AI architectures dedicated to the discovery and explanation of rare, surprising, or underrepresented phenomena across domains including computer vision, medical diagnostics, deep search, and recommendation. Originally developed for unsupervised visual saliency detection, the DeepRare approach has evolved to underpin agentic reasoning systems that couple deep neural feature processing with rare-event modeling and transparent, domain-aware decision support.

1. Foundational Principles and System Architectures

DeepRare systems are unified by the pursuit of identifying and interpreting rare or unusual data points. In vision, this entails constructing saliency maps which highlight surprising regions in images, while in medical or information domains, it denotes systems diagnosing rare diseases or unearthing infrequent items or answers.

The initial DeepRare architecture (DeepRare2019, DeepRare2021) devises an unsupervised visual attention model that merges the feature abstraction capacities of deep convolutional neural networks (CNNs) with rarity computation—a measure directly linked to the probability of occurrence within internal feature maps. Modern agentic variants (e.g., DeepRare for rare disease diagnosis) are structured as multi-tier modular systems with a central LLM-powered host, specialized agent servers, a persistent memory module, and real-time access to web-scale, curated knowledge or toolchains. This modular composition allows for dynamic workflow delegation, traceable decision steps, and seamless integration of external evidence.

2. Rare Feature Modeling and Reasoning

At the core of the visual DeepRare models, feature rarity is the principal notion. Given an image, features are extracted at multiple levels of abstraction (via pre-trained CNNs such as VGG16, VGG19, or MobileNetV2), after which each feature map undergoes a histogram-based analysis to estimate the frequency of activation values. Rarity is computed as:

$R(i) = -\log(p(i))$

where $p(i)$ is the empirical probability of bin $i$ in the feature map's histogram. Rare feature activations—by definition—receive high scores, and are backprojected into pixel space to construct rarity maps. These maps are subsequently fused, hierarchically and with scale-adaptive weighting, to produce a final saliency map.

In the agentic medical AI incarnation, the system accepts heterogeneous clinical input ($\mathcal{I} = \{\mathcal{P}, \mathcal{G}\}$: phenotypes and optional genotypes), and decomposes the diagnostic process via a sequence of analytic modules: phenotype extraction and normalization (to HPO), disease normalization (Orphanet/OMIM mapping), similarity search using HPO embeddings, variant analysis via tools such as Exomiser, and evidence retrieval from real-time and curated sources. Task orchestration and evidence synthesis are performed by an LLM-based host, with all intermediate results and reasoning chains captured for traceability.

3. Evaluation Metrics and Empirical Performance

DeepRare systems are evaluated using domain-appropriate metrics that emphasize both rare-event discovery efficacy and interpretability.

Visual Attention (DeepRare2019, DeepRare2021):

• Datasets: MIT1003 (real-world images), P³ (synthetic pop-out), O³ (odd-one-out)
• Metrics: Saliency AUC (Judd, Borji), NSS, CC, number of fixations, Global Saliency Index (GSI), Maximum Saliency Ratio (target, background), percentage of targets found.
• Results: Consistently ranked in the top-3 models on all benchmarks; uniquely robust across natural and synthetic, as well as low-level and high-level, stimuli. For instance, on P³, DeepRare found 87% of pop-out targets after ~16 fixations, outperforming classical and most DNN-based models.

Rare Disease Diagnosis (Agentic DeepRare):

• Datasets: 8 multi-national datasets spanning 2,919 diseases and 6,401 cases.
• Metrics: Recall@1, Recall@3, Recall@5 (diagnostic hit rates).
• Results: 57.18% avg. Recall@1 across all datasets, vs. 33.39% for second-best LLM system. Achieved 100% Recall@1 for 1,013 diseases. In multi-modal cases (HPO + gene), Recall@1 reached 70.60% compared to Exomiser’s 53.20%. Manual review documented 95.4% agreement with clinical expert validation on reasoning traceability.

Deep Search and Recommendation:

• Adoption of DeepRare-like reasoning in recommender and search systems (e.g., DeepRec, SimpleDeepSearcher) yields substantial improvements in recall and rare-item retrieval by enabling autonomous, multi-step, reasoning-driven exploration of item space, consistently outperforming RL-based and traditional RAG baselines with far less supervision.

4. Interpretability and Transparency

A defining feature across DeepRare system variants is transparent, stepwise explanation for each output. In visual applications, the system permits visualization of which feature levels contributed to each salient region, and adjustable thresholding yields granularity in highlighting rare versus common features.

In diagnosis, every hypothesis is accompanied by a linked reasoning chain that presents, in order:

• Phenotype-to-HPO mapping steps
• Relevant literature, guidelines, and prior cases with explicit citations
• Results from each analytic tool and their effect on current hypotheses
• A final synthesized rationale linking evidence to the diagnosis

These explanations are captured in structured output and, in the clinical system, further validated for correctness and provenance by expert review panels. This approach supports both direct clinical audit and broader research transparency.

5. Practical Implementation and Usage

Software corresponding to deep visual attention models is open-sourced at [https://github.com/numediart/VisualAttention-RareFamily], requiring only Python (Keras/TensorFlow or PyTorch) and a pre-trained backbone for immediate application. No further training is required. The modular design supports backbone replacement and grants full introspection into each stage.

The DeepRare clinical agentic system is delivered as a web application (http://raredx.cn/doctor), supporting both structured and unstructured input, AI-guided dynamic inquiry, real-time annotation, and report download. All stages are interactive and auditable, facilitating hospital integration and EHR compatibility.

In deep search and recommendation, practical deployment emphasizes small, highly curated SFT datasets (as in SimpleDeepSearcher) and agentic multi-turn architectures (as in DeepRec), reducing the need for costly RL or manual annotation, favoring modularity, explainability, and efficient scaling.

6. Applications and Broader Implications

DeepRare methods have demonstrated impact in:

• Visual attention prediction and explainable image analysis
• Detection of anomalies and rare objects in medical imaging and robotics
• Agentic clinical decision support for rare disease diagnosis, with traceable rationales aligning with auditing needs and regulatory requirements
• Deep, autonomous, and explainable item-space exploration in modern recommender and information-seeking systems, with heightened capability to discover underrepresented, long-tail, and novel items

The persistent focus on transparent reasoning, rare-event handling, and modular extensibility suggests wide potential for addressing rare phenomena in evolving and data-scarce domains, bridging discovery, interpretability, and practical scalability in agentic AI.

7. Summary Table

Property	Visual DeepRare (2019/2021)	Agentic DeepRare (Diagnosis, 2025)
Core principle	Rarity-based deep feature saliency	Agentic, modular, traceable rare reasoning
Model backbone	Pre-trained CNN (VGG16/19, MobileNetV2)	LLM host + 40+ expert module servers
Input modality	Images	Structured/unstructured phenotypes, genotypes
Key explanation feature	Layer- & rarity-wise heatmaps	Stepwise reasoning with evidence citations
Performance (selected)	Top-3 on all benchmarks; 87% pop-out	57.18% Recall@1 (avg.), 70.6% multi-modal R@1
Code/platform availability	https://github.com/numediart/VisualAttention-RareFamily	http://raredx.cn/doctor

DeepRare thus constitutes a unifying methodological and architectural theme for rare-event modeling in both vision and reasoning, with demonstrated advances in transparency, efficiency, and rare phenomenon discoverability.