Spider Dataset Overview

• Spider Dataset is a diverse collection integrating semantic parsing benchmarks, computational pathology images, optical reconstructions, and generative modeling for bioinspired design.
• It employs structured evaluation metrics such as exact matching accuracy, success rate, and component-wise F1 scores to challenge model generalization across domains.
• Its cross-disciplinary applications drive innovative research in AI, computational biology, and materials engineering through reproducible benchmarks and advanced visualization techniques.

The term “Spider Dataset” encompasses a diverse set of resources and methodologies within data science, computational biology, artificial intelligence, and mathematical theory, often unified by the visual or structural motif of a spider graph or the acronym “SPIDER.” Across disciplines, spider datasets are leveraged for visualization, semantic parsing, computational pathology, optical imaging, entity relation extraction, bioinspired materials design, and theoretical trace reconstruction. Below is a detailed survey of notable instances and designs, referencing key research developments and technical frameworks.

1. Spider Dataset for Semantic Parsing and Text-to-SQL

The seminal Spider dataset (Yu et al., 2018) serves as a benchmark for complex and cross-domain semantic parsing, particularly for text-to-SQL tasks. It comprises 10,181 human-labeled natural language questions and 5,693 unique SQL queries drawn from 200 multi-table databases spanning 138 domains. Unlike prior datasets limited to single schemas, Spider’s structure enforces strong generalization: train and test splits contain non-overlapping databases and SQL programs.

Key metrics include exact matching accuracy—computed by parsing SQL components as order-invariant sets for clauses such as SELECT, WHERE, GROUP BY, and ORDER BY—and bag-of-components F1 scores for detailed clause-wise evaluation. State-of-the-art models (e.g., SQLNet, TypeSQL) achieve only 12.4% exact matching on the database split, indicating the dataset’s high complexity and emphasizing the challenge of generalizing SQL generation to previously unseen schemas. This property has catalyzed research into schema linking, graph-based representations for structured queries, and cross-domain transfer learning.

Access and reproducibility are facilitated through public release with recommended splits and scripts (https://yale-lily.github.io/spider), fostering robust academic benchmarking.

2. Spider 2.0: Enterprise-Scale Text-to-SQL Evaluation Framework

Spider 2.0 (Lei et al., 12 Nov 2024) extends the original Spider paradigm to real-world enterprise settings, comprising 632 workflow problems sourced from large-scale business environments (Google Analytics, Salesforce, BigQuery, Snowflake, DuckDB, etc.). Databases in this benchmark frequently contain over 1,000 columns, nested records, and multi-schema structures. Tasks demand multi-turn reasoning, dialect adaptation, integration with project-level codebases, and database documentation search, far exceeding canonical text-to-SQL benchmarks.

Agentic evaluation metrics—Success Rate (SR) for interactive tasks and Execution Accuracy (EX) for one-shot tasks—highlight the steep challenge: LLM agents (o1-preview, Spider-Agent) succeed on only 21.3% of Spider 2.0 workflows, compared with 91.2% on Spider 1.0. The tasks involve iterative debugging, handling of advanced SQL functions (average 7 per query), and context lengths pushing the limits of LLM encoding capabilities.

Spider 2.0 exposes critical bottlenecks in schema linking, long-context attention, dialect handling, and interactive code generation. It is accessible for further development at https://spider2-sql.github.io.

3. Spider Dataset in Computational Pathology

The SPIDER pathology dataset (Nechaev et al., 4 Mar 2025) (Supervised Pathology Image-DEscription Repository) is the largest publicly available resource of patch-level histopathology images with multi-organ coverage. Constructed from annotated whole slide images (WSIs), it includes:

• Skin: 159,854 central patches (24 classes)
• Colorectal: 77,182 (14 classes)
• Thorax: 78,307 (14 classes)

Central 224×224 pixel patches are augmented by 24 surrounding context patches to provide 1120×1120 composite images, enabling spatially contextual classification. Labels are verified by expert pathologists through multi-stage annotation, similarity-based retrieval (using Hibou-L feature representations and Faiss indexing), and binary context-based verification.

Baseline models, utilizing the Hibou-L foundation vision transformer and an attention-based head, achieve state-of-the-art accuracies (e.g., 0.940 for skin, 0.914 for colorectal, 0.962 for thorax). The dataset supports rapid region identification, quantitative tissue metrics extraction, and the foundation for multimodal vision-language AI systems. Full resources are available for reproducibility at https://github.com/HistAI/SPIDER.

4. Spider Graphs for Visualization

The spider graph visualization method (Prakash, 2012) reconstructs Self-Organizing Maps (SOM) outputs into cobweb-like spider graphs to clarify inter-variable dependencies in high-dimensional unstructured datasets—a frequent scenario in Big Data analytics. The method proceeds by:

• Filtering SOM output to extract strength measures between variable pairs
• Constructing an n-sided polygon with vertices representing variables (positioned via polar coordinates $$(X_i, Y_i) = (R \cos(2\pi i / n), R \sin(2\pi i / n))$$)
• Drawing threads between vertices if inter-variable strength $S_{ij}$ exceeds a threshold $T$, with thread thickness/color indicating magnitude

This format augments traditional grid-based SOM visualizations by directly encoding multivariate relationships and is resilient to clutter in high-dimensional spaces.

5. Spider Datasets in Trace Reconstruction Theory

Spider graphs are formalized in the trace reconstruction literature (Sun et al., 2022) as trees with a root and n/d legs of length d. The trace reconstruction problem concerns recovering the spider’s binary-labeled structure from noisy observations post-deletion channel transmission, where each node is independently deleted with probability q.

For d large ($d \geq \log_{1/q}(n)$), reconstruction reduces to the classical string case. For d small ($d \leq \log_{1/q}(n)$), entire legs are frequently missing, posing significant combinatorial challenges. The mean-based reconstruction algorithm leverages generating functions and bivariate Littlewood polynomial analysis to distinguish candidates, achieving recovery with trace complexity

$$T = \exp\left(O\left((nq^d)^{1/3} \cdot d^{1/3} \cdot (\log n)^{2/3}\right)\right)$$

for all $q \in (0,1)$. These findings generalize classical trace reconstruction and inform robust data recovery strategies for graph-structured information in error-prone environments.

6. SPIDER Datasets in Optical and Dust Imaging

The SPIDER telescope imaging dataset (Pratley et al., 2019) models optical interferometric measurements as incomplete sets of Fourier visibilities. Sparse image reconstruction is performed via convex optimization (ADMM) and wavelet-based ℓ₁-regularization, with the measurement operator encoded via a Non-Uniform FFT (NUFFT) pipeline ($\Phi = WG F Z S$). Simulated datasets (e.g., M51 galaxy) illustrate the approach’s efficacy in reconstructing high-fidelity astronomical images from limited, noisy measurements inherent to the SPIDER instrument’s lenslet array.

Separately, the SPIDER dust emission dataset (Collaboration et al., 30 Jul 2024) analyzes polarized emission from interstellar dust in CMB studies. Using spatial and harmonic domain component separation (including SMICA), the dust spectral energy distribution is modeled as a modified blackbody with region-dependent spectral indices ($\beta_d = 1.45 \pm 0.05$ for E-modes, $1.47 \pm 0.06$ for B-modes), and spatial variations observed ($\Delta \beta_d$ up to 3.9σ between subregions). No significant line-of-sight decorrelation is detected, but joint Spider+Planck analysis reveals deviations from simple spectral models, informing future foreground removal in cosmological polarization experiments.

7. Generative Modeling for Spider Silk

The Spider Silkome dataset (Dubey et al., 11 Apr 2025) assembles MaSp repeat sequences from 1,098 spider species, with mechanical properties annotated for 446 species. This resource underpins a GPT-based generative modeling framework (SpiderGPT), distilled from ProtGPT2 and fine-tuned in two stages: general MaSp pattern learning (6,000 repeats), and mechanical property conditioning (592 repeats with toughness, Young’s modulus, etc.).

Both forward (properties-to-sequence) and reverse (sequence-to-properties) tasks are supported, facilitating property-driven sequence generation and predictive modeling. Validation comprises physicochemical analyses, motif coverage studies (poly-Ala, YGQGG), secondary structure assessments, and BLAST similarity search. The approach enables rational design of spider silk-inspired biomaterials with customizable mechanical attributes.

Table: Selected Spider Datasets Across Domains

Name / Domain	Scope / Use Case	Key Technical Features / Metrics
Spider (Text-to-SQL) (Yu et al., 2018)	Cross-domain semantic parsing, SQL synthesis	200 DBs, exact matching accuracy, F1 by SQL component
Spider 2.0 (Lei et al., 12 Nov 2024)	Enterprise workflows, agentic text-to-SQL	Multi-turn agent tasks, 1k+ column DBs, <21.3% SR, EX accuracy
SPIDER (Pathology) (Nechaev et al., 4 Mar 2025)	Patch-level multi-organ histopathology	3 organs, context labeling, Hibou-L SOTA baselines
Spider Silkome (Dubey et al., 11 Apr 2025)	Protein design, silk mechanical properties	6,000 MaSp repeats, GPT-based seq→property bidirectional modeling
Spider Graphs (Prakash, 2012)	SOM visualization, Big Data analytics	Polygon-based cobweb GUI, inter-variable strength encoding
Spider Graphs (Sun et al., 2022)	Trace reconstruction, error correction theory	Mean-based, polynomial analysis, exponential trace complexity
SPIDER (Optical) (Pratley et al., 2019)	Astronomical image reconstruction	NUFFT, ADMM/ℓ₁-sparse coding, simulated galaxy datasets
SPIDER (Dust) (Collaboration et al., 30 Jul 2024)	CMB dust emission foreground analysis	SED fit ($\beta_d$), power law spectrum, SMICA separation

8. Summary and Implications

Spider datasets, ranging from semantic parsing benchmarks and multi-organ pathology corpora to optical, biological, and theoretical graph structures, have driven methodological advances across fields. They challenge model generalization, foster reproducibility, and enable sophisticated visualization, data recovery, entity networking, and bioinspired design. Ongoing research builds on these foundations by expanding complexity (Spider 2.0), integrating multimodal data (SPIDER pathology), and refining models for material and molecular engineering (Spider Silkome). The continued expansion and refinement of spider datasets will likely remain central to the development of robust, generalizable, and interpretable artificial intelligence across scientific and industrial domains.