Hierarchical Adaptive Projector

• Hierarchical Adaptive Projector is an umbrella concept that integrates hierarchical structuring, adaptivity, and projection operations across diverse applications.
• It synthesizes methods from video multimodal models, Bayesian transfer learning, and geometric reconstruction to address specific task requirements.
• Empirical results show that adaptive weighting and multi-branch fusion enhance performance in benchmarks like video analysis, transfer studies, and CT imaging.

“Hierarchical Adaptive Projector” is not a standardized term with a single canonical definition in the cited literature. Instead, closely related mechanisms appear across several technical lineages: instruction-conditioned multi-projector fusion in video multimodal LLMs, hierarchical Bayesian transfer models followed by adaptive posterior projection, and geometric projector/backprojector constructions designed for hierarchical reconstruction. At the same time, the acronym HAP is heavily overloaded: in other papers it denotes Hierarchical Adaptive Pooling, Human structure-Aware Pre-training, or High Altitude Platform rather than any projector architecture. For that reason, the most precise treatment is to regard Hierarchical Adaptive Projector as an umbrella concept whose constituent properties—hierarchy, adaptivity, and projection/projector operations—are realized explicitly only in parts across different works (Zhao et al., 9 Jan 2025, Pal et al., 7 Jun 2026, Kulvait et al., 2021).

1. Terminological status and acronym ambiguity

The literature does not present a single method formally named “Hierarchical Adaptive Projector.” What it does contain are several nearby constructs that each realize part of that description. In video understanding, LLaVA-Octopus introduces an instruction-driven adaptive projector but explicitly organizes its three projectors as parallel branches, not a hierarchy. In transfer learning, ProjectionTL couples a source-guided hierarchical prior with an adaptive projection step. In cone-beam CT, the cutting voxel projector is described as particularly suited for hierarchical reconstruction approaches with nonuniform voxel grids and local refinement. These are adjacent, but not identical, uses of the term space surrounding a Hierarchical Adaptive Projector (Zhao et al., 9 Jan 2025, Pal et al., 7 Jun 2026, Kulvait et al., 2021).

Literature line	HAP-related meaning	Relation to “Hierarchical Adaptive Projector”
Video MLLMs	Instruction-conditioned adaptive projector fusion	Adaptive, but not hierarchical
Transfer learning	Hierarchical prior plus adaptive posterior projection	Closest conceptual match
Cone-beam CT	Projector suited for hierarchical reconstruction	Strong projector component
Other HAP usages	Pooling, pre-training, or high-altitude platforms	Terminologically unrelated

The acronym collision is substantial. In graph representation learning, HAP means Hierarchical Adaptive Pooling rather than projection (Liu et al., 2021). In human-centric vision, HAP denotes Human structure-Aware Pre-training with MIM (Yuan et al., 2023). In non-terrestrial networking, multiple papers use HAP for High Altitude Platform or HAP-LAP architectures, including satellite-to-HAP links, RIS-assisted HAP networks, UAV/HAP-assisted vehicular edge computing, and hierarchical HAP-LAP RSMA systems (Grieco et al., 2023, Tanash et al., 2024, Tanash et al., 2024, Traspadini et al., 2022, Kim et al., 2023). A common misconception is therefore that “HAP” names a single architecture family; the record instead shows a shared acronym attached to distinct technical objects.

2. Constituent properties of a HAP-style architecture

A useful synthesis is to treat “HAP-style” (Editor's term) as a design pattern defined by three ingredients. The first is hierarchy, meaning that adaptation occurs at more than one level. The clearest explicit instance is ProjectionTL, where transfer is “decouple[d] … at two levels”: source-level borrowing through a hierarchical prior and feature-level borrowing through posterior projection (Pal et al., 7 Jun 2026). The second is adaptivity, meaning that the projection or projector changes with task, prompt, source relevance, or local agreement. LLaVA-Octopus realizes this through instruction-conditioned projector weights, while ProjectionTL realizes it through source weights $\boldsymbol{\eta}$ and coordinate weights $w_j$ (Zhao et al., 9 Jan 2025, Pal et al., 7 Jun 2026). The third is projection/projector functionality itself: either a learned feature projector, an optimization-based projection step, or a geometric forward operator, as in the cutting voxel projector (Kulvait et al., 2021).

These three ingredients do not always co-occur. LLaVA-Octopus has adaptivity and projectors, but not strict hierarchy. ProjectionTL has hierarchy and adaptive projection, but its “projection” is a posterior-refinement optimization rather than a linear projector in the strict algebraic sense. The cutting voxel projector is a genuine projector/backprojector pair and is explicitly linked to hierarchical reconstruction, but it is not a complete adaptive hierarchy by itself. This suggests that Hierarchical Adaptive Projector is best understood as a compositional concept rather than the proper name of one established architecture.

Two equations capture the most central mechanisms. In LLaVA-Octopus, projector outputs are fused by

$F = w_1 \cdot F_1 + w_2 \cdot F_2 + w_3 \cdot F_3.$

In ProjectionTL, posterior draws are sparsified by the weighted projection

$$\boldsymbol{\beta}^\ast \;=\; \arg\min_{\boldsymbol{u}\in\mathbb{R}^p} \Big\{ \|\boldsymbol{X}\boldsymbol{\beta}-\boldsymbol{X}\boldsymbol{u}\|_2^2 \;+\; \lambda \sum_{j=1}^p \frac{|u_j|}{w_j} \Big\}.$$

The first is an adaptive fusion rule over specialized projectors; the second is an adaptive projection operator acting on posterior samples (Zhao et al., 9 Jan 2025, Pal et al., 7 Jun 2026).

3. Instruction-conditioned adaptive projectors in video multimodal models

LLaVA-Octopus is the clearest example of an adaptive projector in the contemporary multimodal literature, even though the paper explicitly states that it is “better described as a parallel multi-projector system with instruction-driven expert weighting, rather than a truly hierarchical projector stack” (Zhao et al., 9 Jan 2025). The model consists of four components: a visual encoder, a text encoder, an instruction-driven adaptive projector, and a LLM decoder. The instantiated backbones are SigLIP (so400m-patch14-384), BERT, and Qwen2.5-7B-Instruct. The adaptive projector contains three branches: an image-based projector implemented as MLP2x_GELU, a spatial-temporal projector implemented with the STC module from VideoLLaMA2, and a token-compress projector implemented with the LLaMA-VID projector.

The branches are functionally specialized. The image-based projector is intended to capture static scene details / appearance; the spatial-temporal projector captures inter-frame relationships / temporal dynamics; and the token-compress projector preserves temporal coherence over many frames, especially for long videos and tasks such as dynamic counting. The fusion controller is instruction-conditioned: BERT encodes the user instruction, the [CLS] token is passed through two MLPs, and the resulting three weights are applied to the full projector outputs through the fusion equation above. The weighting is therefore sample-level, projector-level, and sequence-wide rather than token-level or frame-level.

A practical constraint of this design is output-shape compatibility. LLaVA-Octopus modifies each projector so that all three produce 1568 visual tokens. The MLP projector is reduced from $14 \times 14 \times 8 + 8 = 1576$ to 1568 by removing separators between images; the STC projector is reparameterized with stride $(2,2,2)$ and $(1,2,2)$ and padding $(1,1,1)$ to also reach 1568; and the LLaMA-VID projector uses 128 frames, 6 context tokens + 6 content tokens per frame, and a separator every 4 frames so that $49 \times 32 = 1568$. This token alignment is essential because fusion is additive.

Empirically, the adaptive gate outperforms naive alternatives. On the fusion-strategy ablation, Projector Fusion Gate reaches 51.7 / 55.7 on MVBench / VideoMME, compared with 50.4 / 53.6 for Average, 51.2 / 54.8 for Concat, 50.1 / 52.9 for Random weights, and 50.9 / 53.4 for Random choose. On projector-combination ablations, the best single projector reaches 49.1, while all three together reach 51.7. The model also reports full-setting results such as 59.2 on EgoSchema, 57.5 on MLVU, 54.7 on VideoMME, and 66.9 on MVBench. The central significance is not hierarchy but instruction-conditioned adaptive projector fusion over specialized branches (Zhao et al., 9 Jan 2025).

4. Hierarchical adaptive projection in selective knowledge transfer

ProjectionTL is the closest direct realization of a Hierarchical Adaptive Projector in the strict compositional sense because it is explicitly organized into two transfer levels: source selection and feature selection (Pal et al., 7 Jun 2026). Its first stage builds a source-guided hierarchical prior. Let $\widehat{\boldsymbol\omega}_k$ be source coefficient summaries and $w_j$0 a target-only pilot estimate. Then the prior mean is assembled coordinatewise as

$w_j$1

or in matrix form $w_j$2. The mixing vector lies on the simplex,

$w_j$3

with $w_j$4, and each $w_j$5 is a cosine-similarity score,

$w_j$6

The $w_j$7th component is a dummy no-transfer source, allowing the model to fall back to a zero-centered prior when external sources are not useful.

Conditionally, the target coefficients follow

$w_j$8

Given target data, the conditional posterior is Gaussian: $w_j$9 with

$F = w_1 \cdot F_1 + w_2 \cdot F_2 + w_3 \cdot F_3.$0

This posterior is typically dense, which motivates the second stage: a posterior-projection step that acts at the feature level.

That projection uses coordinate-specific adaptive weights

$F = w_1 \cdot F_1 + w_2 \cdot F_2 + w_3 \cdot F_3.$1

so coordinates locally aligned with at least one source are penalized less heavily. The projected posterior is then summarized by a Median Probability Model, retaining coordinates that are nonzero in more than $F = w_1 \cdot F_1 + w_2 \cdot F_2 + w_3 \cdot F_3.$2 of projected draws. This gives ProjectionTL both global and local adaptivity: “where to borrow” is handled through $F = w_1 \cdot F_1 + w_2 \cdot F_2 + w_3 \cdot F_3.$3, while “what to borrow” is handled through $F = w_1 \cdot F_1 + w_2 \cdot F_2 + w_3 \cdot F_3.$4 and the projection step.

Theoretical analysis establishes posterior contraction at the sparse high-dimensional rate. With $F = w_1 \cdot F_1 + w_2 \cdot F_2 + w_3 \cdot F_3.$5, the paper proves

$F = w_1 \cdot F_1 + w_2 \cdot F_2 + w_3 \cdot F_3.$6

and gives an analogous predictive contraction result. Empirically, on ADNI data, ProjectionTL reports test MSE 3.9524 in a heterogeneous single-model-source study and 3.3196 in a multimodal block-missingness study, outperforming TRADER, TransGLM, CONCERT, and BR2 in the reported comparisons. Relative to a Hierarchical Adaptive Projector concept, this is the most explicit example of a method that is simultaneously hierarchical, adaptive, and projection-based (Pal et al., 7 Jun 2026).

5. Geometric projectors for hierarchical reconstruction

In cone-beam CT, the paper “Cutting Voxel Projector” provides a different but highly relevant meaning of projector: a forward and backward operator grounded in the volume of the portion of a voxel that is cut out by all rays reaching a detector pixel (Kulvait et al., 2021). This is not a learned feature projector but a geometric operator designed for algebraic reconstruction. Its importance for a HAP-like concept lies in three explicit properties: it is described as a near-exact projector and backprojector, it is “particularly suited for algebraic reconstruction techniques and hierarchical reconstruction approaches with nonuniform voxel grids,” and it “enables local refinement of voxels.”

The core forward model replaces standard ray-length approximations with an integral over the cut volume $F = w_1 \cdot F_1 + w_2 \cdot F_2 + w_3 \cdot F_3.$7. In its unit-sphere form,

$F = w_1 \cdot F_1 + w_2 \cdot F_2 + w_3 \cdot F_3.$8

and the pixel response aggregates these contributions as

$F = w_1 \cdot F_1 + w_2 \cdot F_2 + w_3 \cdot F_3.$9

Operationally, the implementation approximates each cut volume by a planar cut area times a vertical cut length,

$$\boldsymbol{\beta}^\ast \;=\; \arg\min_{\boldsymbol{u}\in\mathbb{R}^p} \Big\{ \|\boldsymbol{X}\boldsymbol{\beta}-\boldsymbol{X}\boldsymbol{u}\|_2^2 \;+\; \lambda \sum_{j=1}^p \frac{|u_j|}{w_j} \Big\}.$$0

This construction is voxel-centric rather than ray-centric, which is why it aligns naturally with local refinement and nonuniform discretization.

The paper reports that the projector/backprojector pair passes the discrete adjointness test

$$\boldsymbol{\beta}^\ast \;=\; \arg\min_{\boldsymbol{u}\in\mathbb{R}^p} \Big\{ \|\boldsymbol{X}\boldsymbol{\beta}-\boldsymbol{X}\boldsymbol{u}\|_2^2 \;+\; \lambda \sum_{j=1}^p \frac{|u_j|}{w_j} \Big\}.$$1

and benchmarks both a standard and a relaxed GPU implementation against Siddon ray tracing and TT footprint projection. In the first benchmark on an AMD Radeon VII, CVP relaxed reports projection/backprojection times of 12.3 s / 5.2 s, versus 101.1 s / 18.8 s for TT and 117.8 s / 244.2 s for Siddon8. In the second benchmark, CVP relaxed reports 7.5 s / 4.1 s, versus 27.6 s / 11.5 s for TT and 282.2 s / 684.5 s for Siddon8. The paper also states that the cutting voxel projector achieves higher accuracy than the TT projector, especially for large cone-beam angles.

The limitations are equally important. The work does not implement a full hierarchical voxel data structure, adaptive mesh refinement policy, or octree traversal. A plausible implication is that it should be regarded as a projector foundation for a Hierarchical Adaptive Projector in reconstruction, not as a complete adaptive hierarchy by itself. Even so, among projector-centric papers it is the strongest explicit link between projector design and hierarchical reconstruction (Kulvait et al., 2021).

6. Misconceptions, limits, and likely directions of convergence

Several misconceptions recur when the term Hierarchical Adaptive Projector is applied loosely. The first is that any adaptive projector fusion model is hierarchical. LLaVA-Octopus explicitly is not: it has parallel experts with a single global decision over three projector outputs, no stage-wise gating, no token-level adaptive selection, and no coarse-to-fine projector stack (Zhao et al., 9 Jan 2025). The second is that ProjectionTL uses “projection” in the sense of a linear orthogonal projector. It does not; its projection is a weighted $$\boldsymbol{\beta}^\ast \;=\; \arg\min_{\boldsymbol{u}\in\mathbb{R}^p} \Big\{ \|\boldsymbol{X}\boldsymbol{\beta}-\boldsymbol{X}\boldsymbol{u}\|_2^2 \;+\; \lambda \sum_{j=1}^p \frac{|u_j|}{w_j} \Big\}.$$2-penalized predictive projection applied to posterior draws (Pal et al., 7 Jun 2026). The third is that a high-fidelity projector suitable for hierarchical reconstruction is already a full adaptive framework. The cutting voxel projector is not: it lacks explicit parent-child hierarchy management, refinement rules, and multilevel solver logic (Kulvait et al., 2021).

A second class of misconception is purely terminological. The literature contains major, unrelated HAP expansions: Hierarchical Adaptive Pooling in graph learning, Human structure-Aware Pre-training in masked image modeling, and High Altitude Platform in multiple communications papers (Liu et al., 2021, Yuan et al., 2023, Grieco et al., 2023). Consequently, the phrase “HAP” by itself is not semantically stable across fields. Any encyclopedia treatment of Hierarchical Adaptive Projector therefore has to separate the conceptual family from the acronym.

The present record suggests, but does not yet standardize, a more integrated future architecture. A plausible implication is that a canonical Hierarchical Adaptive Projector would combine the multilevel control of ProjectionTL, the expert specialization and task conditioning of LLaVA-Octopus, and the operator consistency of projector/backprojector pairs such as the cutting voxel projector. The missing pieces are already visible in the cited limitations: explicit multi-level routing rather than single-shot global weighting, richer uncertainty treatment after projection, and true hierarchical data structures with refinement/coarsening rules (Zhao et al., 9 Jan 2025, Pal et al., 7 Jun 2026, Kulvait et al., 2021). Until such an overview is formalized, Hierarchical Adaptive Projector remains best understood as a technical umbrella concept rather than the settled name of a single method class.