---
title: Extractor Architectures Overview
url: https://www.emergentmind.com/topics/extractor-architectures
type: topic
---

# Extractor Architectures Overview

“Extractor architectures” is a polysemous term used across machine learning, information extraction, security, coding theory, and fault-tolerant quantum computing. In the surveyed literature, it may denote a front-end feature former, a structured information extraction pipeline, a post-hoc concept module, a query mechanism over an internal world state, a reverse-engineering attack that extracts model structure, or a sparse transformation that extracts entropy. AXE treats extraction as query-conditioned pruning and grounding over the HTML DOM [2602.01838]; VIPTR treats the extractor as the central recognition module in scene text recognition [2401.10110]; Updater-Extractor separates state update from state query [2104.05500]; MACE and PACE treat extraction as concept recovery from CNN representations [2011.01472] [2108.13828]. This suggests that “extractor architecture” is best understood as a family resemblance term: across domains, the extractor is the subsystem that isolates, compresses, grounds, or reconstructs task-relevant structure from a larger substrate.

## 1. Domain-specific meanings and formal roles

In web and scientific information extraction, the extractor is a pipeline that converts noisy, semi-structured documents into structured outputs. In AXE, it is a two-stage structured information extraction pipeline wrapped around a lightweight LLM, with preprocessing, pruning, extraction, and Grounded XPath Resolution (GXR) over the original DOM [2602.01838]. In the QCL literature-extraction setting, JSG-IE is a three-stage JSON-Schema Guided Information Extraction Pipeline with pre-processing, schema-guided extraction, and a validation and correction loop [2605.09927]. In CTI analysis, EXTRACTOR converts unstructured prose into provenance graphs through normalization, resolution, summarization, and graph generation [2104.08618].

In recognition systems, the extractor is more often a learned feature-forming module. VIPTR is presented as a single-vision extractor that replaces the usual visual encoder plus sequence decoder in scene text recognition, producing a sequence-ready representation for CTC decoding [2401.10110]. In pathology MIL, the extractor is the frozen pre-trained patch encoder whose embeddings are aggregated for slide-level prediction, and its choice across pre-training dataset, backbone, and pre-training method is treated as a primary design variable [2408.01167]. HEE similarly redefines the speaker embedding extractor by replacing single pooled embeddings with multiple dense frame-level embeddings per segment [2211.04060].

Other usages are more specialized. In the transformer literature, the “Extractor” is a family of drop-in replacements for multi-head self-attention [2311.12678]. In coding theory, sparse extractor families are distributions over sparse functions that extract almost all entropy from high-min-entropy sources [1207.6260]. In FTQC, extractor architectures are modular qLDPC systems augmented so that logical Pauli operators can be measured fault-tolerantly in one logical cycle [2604.25094]. In security, architecture extraction denotes recovery of a deployed model’s structure from GPU profiles or EM leakage [2304.03388] [2401.13575]. Accordingly, the same word names very different formal objects, but in each case the architecture is organized around selective recovery of latent structure.

## 2. Front-end extractors for features, sequences, and multimodal signals

A major line of work treats the extractor as the front end that shapes raw sensory input into a representation already aligned with the downstream prediction axis. VIPTR is exemplary: it is a four-stage pyramid Vision Permutable extractor with three height-reduction steps, a two-layer CNN-based patch embedding, local attention blocks such as CSWin Self-Attention and MaSA, global attention blocks such as MHSA and OSRA, and a final output \(B \in \mathbb{R}^{1 \times \frac{W}{4} \times C_3}\) that can be sent directly to a CTC decoder. The extractor mixes local character-shape modeling and global inter-character context modeling, avoids an autoregressive decoder, and is reported to be insensitive to input length; SVIPTR-T has 5.1M parameters and 3.3 ms average inference time on V100, while VIPTRv1-L has 39.5M parameters, 16.9 ms inference time, 92.9 average English benchmark accuracy, and 72.6 Chinese scene accuracy [2401.10110].

Search-based extractor design appears in text recognition as well. TREFE defines a domain-specific search space with a 3D-structured spatial model and a Transformer-based sequential model, constraining the spatial path so that \(32=\prod_i s_i^h\) and \(4=\prod_i s_i^w\) for an input of size \(32 \times W\). Its supernet is trained progressively in blocks with an auxiliary neck, and the final sub-network is selected by natural gradient descent under an explicit latency constraint. The spatial search space alone is described as having about \(1.7\times 10^{17}\) possible architectures in the reported setting, and the searched models achieve strong accuracy with low latency, including 16.41 WER / 4.45 CER / 2.85 ms on IAM and 94.8 on IIIT5K with 2.62 ms latency [2209.13139]. For small-object detection, MRAE instead treats extraction as adaptive multiresolution fusion over \(C_1\), \(C_2\), and \(C_3\) from ResNet-101, using cosine-similarity-based feature interaction. The best variant, with template 2, reaches 5.0 AP on COCO small-object detection, compared with 2.3 for the plain baseline and 0.04 for the soft-attention attempt [2006.05941].

Speech and multimodal systems use comparable ideas with different substrates. HEE decomposes the extractor into a feature-map extractor and an enhancer implemented as five Conformer encoder blocks with residual connection, replacing global pooling with self-attention so that a 3.2 s window yields 40 embeddings every 80 ms; it reports at least 10% improvement on each evaluation set except one [2211.04060]. In cued-speech recognition, the extractor is not a CNN learned from scratch but a pre-trained MediaPipe-based landmark frontend producing 42 2D lip landmarks, 21 2D hand landmarks, and a fingertip stream, followed by PCA to 20 principal components per stream and a multistream Bi-GRU with CTC; the reported decoding accuracy at the phonetic level is 70.88% [2204.04965].

Extractor choice also determines transfer behavior when the downstream model is fixed. In WSI MIL, the paper evaluates extractors across pre-training dataset, backbone, and pre-training method, finding that ImageNet-21K generally improves over ImageNet-1K, modern and deeper backbones improve over standard ones, and DINO + ViT-S/16 is the strongest SSL choice in the reported study [2408.01167]. In cross-domain few-shot learning, FES makes the extractor collection itself the architecture: pretrained extractors are kept separate, fine-tuned independently on the support set, converted to cross-validated logits \(L_K^J[CV]\in\mathbb{R}^{N\times K\times J\times C}\), and combined by a linear stacking classifier \(W\in\mathbb{R}^{K\times J}\). Because FES works in logit space, it can utilize heterogeneous pretrained extractors out of the box and does not maintain a universal model that must be recomputed when the collection changes [2205.05831].

## 3. Structured information extraction, grounding, and graph construction

AXE is a representative architecture for low-cost, grounded web extraction. Its pipeline is explicitly organized as preprocessor \(\rightarrow\) AI extractor \(\rightarrow\) postprocessor. The preprocessor removes scripts and CSS, applies HTMLRAG lossless cleaning, and uses AutoChunker to create DOM-aware blocks. The AI extractor first runs a pruner that refines cleaned chunks into XPath-aligned mini-chunks and returns a JSON list of indices to keep, then re-merges the selected HTML into a compact pseudo-page for a Qwen3-0.6B extractor adapted with LoRA / rsLoRA. The postprocessing stage applies GXR, which normalizes the predicted string, iterates over DOM elements, computes token intersection and fuzzy semantic similarity with `SequenceMatcher`, and returns \(\{\text{text}, \text{xpath}, \text{sub\_index}, \text{score}\}\). On SWDE, AXE reports reducing average context length from 16,581.9 tokens to 350.6 tokens, a 97.9% reduction, and achieving 88.10% F1 in zero-shot mode, with large ablation drops when GXR or adaptor training is removed [2602.01838].

JSG-IE addresses a different substrate but a closely related problem: extraction of nested QCL architectures from scientific papers. Its three stages are pre-processing, schema-guided extraction, and a validation and correction loop. The central mechanism is a JSON Schema that specifies field names, types, required fields, ranges, array-length constraints, and logical dependencies, turning extraction into what the paper describes as a constraint-satisfaction problem. Reported average performance rises from 64.7% F1 with baseline prompting to 70.4% with JSG-IE, with the best score of 83.4% F1 from kimi-k2-thinking; the paper also reports that list-of-dictionary output organization outperforms dictionary-of-list, and that Markdown serialization outperforms raw PDF, LaTeX, and JSON in the reported comparison [2605.09927].

EXTRACTOR for CTI reports is structurally similar in its insistence on layered normalization before final extraction. It first normalizes prose through enhanced tokenization, homogenization with a system call synonym dictionary of 87 verbs and a CTI noun dictionary of over 1112 common noun phrases, and passive-to-active conversion. It then resolves ellipsis subjects, pronouns, and entity forms; filters productive sentences with a BERT classifier trained on 8,000 labeled threat sentences; removes word verbosity with a BiLSTM plus a System Entity Extractor; and finally builds provenance graphs with Semantic Role Labeling and graph-construction rules. The BERT classifier reports Precision \(=0.950\), Recall \(=0.957\), F1 \(=0.953\), SEE reports Precision \(=1\), Recall \(=0.98\), F1 \(=0.99\), and retraining SRL on cybersecurity text improves F1 from 0.84 to 0.93 [2104.08618]. This suggests that modern extractor architectures for unstructured documents often rely less on end-to-end generation than on staged constraint imposition, domain-specific normalization, and explicit output grounding.

## 4. Concept extraction, explanation, and state querying

A different branch of the literature uses “extractor” for modules that expose latent factors underlying an existing model’s behavior. MACE is inserted laterally between the final convolutional layer and the first dense layer of a pre-trained CNN and is decomposed into four modules: map generator, embedding generator, relevance estimator, and output generator. For class \(k\) and concept \(j\), the map generator computes \(c_{jk}=\mathrm{ReLU}(x * \theta^M_{jk})\); the embedding generator produces \(e_{jk}\in\mathbb{R}^Q\) and is trained with a triplet loss; the relevance estimator defines \(r_{jk}=(\theta^R_{jk})^T e_{jk}\), allowing both positive and negative concept relevance; and the output generator reconstructs the classifier’s intermediate representation with \(\mathcal{L}^D=\|z-\tilde z\|_2^2\) and a KL term \(\mathcal{L}^O\). In the reported user study, MACE receives 48.29% of votes, ahead of Excitation Backpropagation at 40.73%, GradCAM++ at 9.51%, GradCAM at 1.21%, and VisCNN at 0.24% [2011.01472].

PACE is also posthoc and architecture-agnostic, but its mechanism is a class-specific linear autoencoder plus global concept vectors. Given a feature map \(F\in\mathbb{R}^{H\times W\times D}\), PACE encodes it to \(E_k\in\mathbb{R}^{H\times W\times Q}\), matches each spatial embedding to class-specific global concepts \(\mathcal{C}_k^j\) by inverse Euclidean distance \(S_k^j[l,m]=1/\|E_k[l,m]-\mathcal{C}_k^j\|_2\), replaces strongly matched embeddings with concept vectors, decodes back to \(\hat F_k\), and reclassifies through the remainder of the black-box network. Concept relevance is defined by removal, \(r_k^j=p_k-p_k^j\), and the explainer is trained with a combined objective over classification preservation, relevance, diversity, and triplet losses. Reported agreement accuracies are 88.2% for VGG16/AWA2, 94.7% for VGG16/ImageNet-Birds, and 94.1% for VGG19/ImageNet-Birds, and the paper reports that over 72% of the extracted concepts are human interpretable [2108.13828].

Updater-Extractor moves the idea of extraction away from perception and toward persistent internal state. The architecture defines an Updater \(\hat f_\delta\) that maps \((w_{t-1},I_t)\) to \(w_t\) and an Extractor \(\hat f_\varepsilon\) that maps \((w_t,Q_t)\) to predictions, with both implemented as transformer decoders. Training uses stepwise updates with gradient stopping,
\[
w_t=\hat f_\delta(\mathrm{STOP\_GRADIENT}(w_{t-1}),I_t),
\]
and a “thorough querying” regime under which the paper proves that any stepwise-optimal model is also sequence-optimal. Empirically, the model is reported to generalize far beyond training horizons, including Game of Life rollouts up to 10,000 steps and Pathfinder accuracy around 0.73 [2104.05500]. Here the extractor is not a feature encoder but a readout mechanism over an explicitly maintained world-state representation.

## 5. Architecture extraction as reverse engineering and side-channel analysis

In security, extractor architectures become adversarial systems whose purpose is to recover the structure of a deployed model. EZClone formalizes architecture extraction as the first stage of a model extraction attack. One technique uses a malicious dynamically linked version of PyTorch to expose a detailed profiler trace and reconstruct PyTorch code directly; the lower-capability EZClone technique uses aggregate `nvprof` profiles containing GPU kernel statistics, API call statistics, and system signals, preprocesses them with normalization, missing-feature indicators, NaN filling, and optional Recursive Feature Elimination, and trains a supervised classifier \(\mathcal A\) over a candidate set \(C\). Over all 36 Torchvision vision architectures in the reported evaluation, Random Forest achieves 100/100 train/test accuracy with all 473 features, and several classifiers reach 100% test accuracy with only the top three non-memory GPU-kernel features [2304.03388].

The Jetson Nano work studies CNN architecture extraction as a profiled side-channel attack on an edge GPU using EM leakage. The threat model assumes physical access to the target device, access to an identical device for profiling, the ability to collect electromagnetic measurements, and knowledge that the target runs one of 15 considered CNN architectures. Measurements are taken with a PicoScope 3207B oscilloscope and a Riscure EM probe, using a GPIO trigger to mark inference; traces are preprocessed by windowed averaging of size 1,000 on the absolute value of the measurements, with no alignment required. A simple 1D CNN classifier over 15,900 traces reports 99% final test accuracy [2401.13575].

These two papers use the same word in a nearly inverted sense. The deployed model is no longer the extractor; it is the object being extracted. This suggests that “extractor architecture” can denote either a constructive subsystem for representation formation or an external reconnaissance system that infers architecture identity from observable side effects.

## 6. Specialized extractor families in transformers, hardware, coding theory, and FTQC

The paper “Interpretation of the Transformer and Improvement of the Extractor” defines the Extractor as a family of drop-in replacements for the multi-head self-attention sublayer. In this framing, self-attention uses dynamic weights generated from the current sequence, whereas the Extractor uses static weights plus dynamic element-wise multiplications. The improved variant iSHE scales the extraction output by \(1/\sqrt{i}\) at the \(i\)-th output position to stabilize variance across sequence lengths, without introducing additional trainable parameters. On the reported language-modeling benchmark built from the top 100 books in English children’s literature, the iSHE model is said to outperform self-attention variants with 1, 2, 4, 8, 16, 32, and 64 heads as well as SHE [2311.12678].

Hardware-oriented extractor design appears in the Random Feature Extractor (RFE) core for Extreme Learning Machines. The proposed D-ELM uses an analog ASIC first stage and a digital second stage, with a 128 \(\times\) 128 current mirror array, 128 input registers / DAC-like input generation, and 128 CCO outputs. Device mismatch provides random weights; input rotation virtually expands the hidden layer with expansion factor \(E=L/N\); and a cognizance check with \(\theta \approx 99.5\%\) filters incognizant neurons. The chip is implemented in \(0.35\,\mu\text{m}\) CMOS on a \(5\,\text{mm}\times 5\,\text{mm}\) die, reports total energy of \(5.97\,\mu\text{J/classify}\), and achieves better than 97% accuracy on MNIST [1512.07783].

In theoretical computer science, sparse extractor families are defined over functions with few input-output dependencies. A function is \(s\)-sparse if the number of dependency pairs \((i,j)\) is at most \(s\), and \(\ell\)-local if each output depends on at most \(\ell\) inputs. The paper gives a strong sparse extractor family \(H(x)=Mx\) based on a random sparse binary matrix and a weak sparse extractor family \(H(x,r)=Mx+Br\) using a sparse seed component \(B\), proves upper and lower bounds for strong extractor sparsity that are tight up to a constant factor over a broad regime, and shows that weak extractor families can achieve \(O(n\log n)\) sparsity in settings where strong extractors require more [1207.6260].

Extractor architectures in FTQC are again different. Here the extractor is part of a modular qLDPC architecture in which a memory block is augmented with an extractor system so that logical Pauli operators can be measured fault-tolerantly in one logical cycle. The INJEQT paper reports that over 90% of total program error arises from synthillation and proposes a 2-factory design that synthesizes \(R_z(\theta)\) states in an auxiliary code and injects them into extractor modules with only a constant number of inter-module measurements. The reported gains are up to \(22\times\) error reduction, \(13\times\) wall-clock improvement, and \(7.2\times\) space-time reduction [2604.25094]. Across these works, the word “extractor” ranges from an attention replacement, to an analog random-projection engine, to a family of sparse functions, to a module in a quantum-computing fabric.

## 7. Recurrent design principles, tradeoffs, and common misreadings

Several recurrent architectural principles appear across otherwise unrelated definitions. One is preservation of task geometry instead of premature flattening. VIPTR preserves width as the text-sequence axis while progressively reducing height [2401.10110]; HEE preserves frame-level resolution and distributes global context back to each frame through self-attention [2211.04060]; AXE preserves DOM structure and prunes by XPath-aligned chunks instead of chunking raw text [2602.01838]; JSG-IE finds that list-of-dictionary schema organization better matches the semantics of a single QCL layer than dictionary-of-list organization [2605.09927]. This suggests that many extractor architectures are effective precisely when they maintain the latent coordinate system of the source domain.

A second principle is modular separation of extraction subfunctions. AXE explicitly separates preprocessor, extractor, and grounding stages [2602.01838]; Updater-Extractor separates writing to world state from reading from it [2104.05500]; FES separates per-extractor fine-tuning from later logit-level stacking [2205.05831]; HEE separates local feature-map extraction from global enhancement [2211.04060]. A plausible implication is that extractor architectures often trade monolithic end-to-end modeling for staged specialization, allowing different inductive biases or constraints to be imposed at different points.

A third principle is explicit faithfulness or observability. GXR forces AXE outputs to be traceable to absolute XPath coordinates [2602.01838]; JSG-IE adds schema validation and a correction loop [2605.09927]; MACE reconstructs the classifier’s internal representation while assigning signed concept relevance [2011.01472]; PACE defines relevance by probability drop under concept removal and measures agreement with the black-box prediction [2108.13828]. Even the security papers fit this pattern in inverted form: EZClone and the Jetson Nano attack rely on the observability of architectural signatures in kernels, API calls, timing, or EM traces [2304.03388] [2401.13575].

A common misreading is to treat extractor architectures as synonymous with generic backbones or encoders. The surveyed papers do not support that simplification. Some extractors are front-end feature formers, but others are post-hoc explainers, query modules over persistent state, sparse randomized transformations, or reverse-engineering systems [2108.13828] [2104.05500] [1207.6260] [2304.03388]. Another common misreading is that extractor quality is primarily a function of scale. Several papers instead emphasize structure, constraints, or modularity: AXE couples a 0.6B model with aggressive DOM pruning and grounding [2602.01838]; VIPTR removes the decoder-centric STR paradigm [2401.10110]; TREFE searches under latency constraints rather than accuracy alone [2209.13139]; FES avoids a universal model altogether [2205.05831]. Taken together, these works portray extractor architectures not as a single architectural lineage, but as a recurrent design response to the same problem: how to isolate the information-bearing substructure of a complex signal while preserving the invariants required by the downstream task.

Source: https://www.emergentmind.com/topics/extractor-architectures