Hierarchical Embedding Prompt Template

• Hierarchical embedding prompt templates are structured designs that distribute instructions across levels to control model representations for tasks such as zero-shot multimodal embedding.
• They utilize a two-level architecture where a global system prompt sets a compression objective and a query-specific prompt enhances discriminative performance.
• Empirical findings indicate improved accuracy and refined error quality in tasks like classification and retrieval, albeit with increased computational complexity.

Searching arXiv for the cited papers and closely related work on hierarchical prompting and embedding templates. A hierarchical embedding prompt template is a prompt organization scheme in which instructions are distributed across explicitly differentiated levels so that a model’s internal representation is shaped in a controlled, often task- or modality-aware way. In the most direct usage, the term denotes a two-level prompt architecture for multimodal LLMs in which a global system prompt establishes a shared compression objective and a query-specific representation prompt reinforces discriminative behavior on the query side, thereby repurposing a generative model as an embedder (Ju et al., 1 Aug 2025). Closely related literature broadens the idea into modular prompt decomposition, attribution-driven prompt editing, hierarchical prompt taxonomies, hierarchy-aware label prompting, and multi-level linguistic knowledge injection for vision-LLMs, indicating that the expression names a family of structured prompt designs rather than a single canonical artifact (Chen et al., 6 Jan 2026, Jeoung et al., 19 May 2025, Wang et al., 2022, Wang et al., 2023).

1. Terminological scope and research context

Across recent work, “hierarchical” refers to different but related kinds of structure. In some papers, hierarchy is a role distinction between a global controller and a local reinforcement cue; in others, it is a decomposition into semantic units, label-depth slots, or coarse-to-fine prompt levels. This suggests that the unifying property is not a fixed syntax, but the use of ordered levels with distinct functional responsibilities.

Paper	Hierarchical levels	Primary function
"From Generator to Embedder" (Ju et al., 1 Aug 2025)	System prompt + query representation prompt	Zero-shot multimodal embedding
"Learning from Prompt itself" (Chen et al., 6 Jan 2026)	Semantic units + optimization phases	Attribution-driven prompt refinement
"PromptPrism" (Jeoung et al., 19 May 2025)	Functional, semantic, syntactic levels	Prompt analysis and rewriting
"HPT" (Wang et al., 2022)	One prompt slot per hierarchy depth	Hierarchical text classification
"Learning Hierarchical Prompt" (Wang et al., 2023)	Global, high, low levels	Vision-language prompt tuning

The literature also uses “hierarchical embedding” in a non-prompt sense. "Representing Hierarchical Structure by Using Cone Embedding" introduces a metric-cone space $X := Z \times [0,1] \,/\, Z \times \{0\}$ with a one-dimensional height coordinate $s$, where smaller $s$ indicates a higher position in the hierarchy; this is a geometric embedding method rather than a prompt template (Takehara et al., 2021). The distinction is important because prompt-side hierarchy organizes instructions that shape hidden states, whereas cone embedding makes hierarchy an explicit coordinate of the representation space itself.

2. Canonical two-level architecture for multimodal embedding

The clearest instance of a hierarchical embedding prompt template appears in "From Generator to Embedder," which proposes a two-level instruction architecture for universal multimodal embeddings (Ju et al., 1 Aug 2025). The first level is a system-level prompt applied to both query and candidate inputs; the second is a representation prompt appended only to the query. The motivation is that ordinary prompting leaves the model in a generative regime, whereas the hierarchical template imposes a shared representation contract and then reinforces it asymmetrically on the query side.

The system prompt is defined explicitly as:

$\epsilon_{\text{sys}} \gg \epsilon_{\text{usr}}$0

The paper’s query and candidate templates are:

$\epsilon_{\text{sys}} \gg \epsilon_{\text{usr}}$1

$\epsilon_{\text{sys}} \gg \epsilon_{\text{usr}}$2

Several design choices are central. The system prompt is identical on both sides and is intentionally short and modality-general. The query side carries both a task instruction and a representation prompt, while the candidate side receives only the candidate content under the same global controller. The representation is extracted not from generated text but from the decoder-only MLLM’s hidden state by last-token pooling followed by L2 normalization, so the prompt is used to shape latent states rather than textual output. The paper further reports that system prompts receive much higher attention than user prompts, summarized as $\epsilon_{\text{sys}} \gg \epsilon_{\text{usr}}$, which is the main empirical justification for assigning the global embedding objective to the system level (Ju et al., 1 Aug 2025).

This two-level hierarchy is also asymmetric by design. The best practical zero-shot MMEB strategy applies the system prompt to both query and candidate and adds query reinforcement only; adding reinforcement to both query and candidate degrades performance. The reported prompt-strategy ablation gives System-QD + Q-Rein. at 43.3 average, whereas + QD-Rein. yields 39.1 average. On MMEB, the zero-shot prompted model reaches 43.3 average for Ours (Qwen2-VL-2B) versus 13.9 average for Qwen2-VL-2B base, and the hierarchical prompt improves a simple in-batch negative fine-tuning baseline by 4.8 points (Ju et al., 1 Aug 2025).

3. Prompt as a hierarchical semantic object

A second line of work treats hierarchical prompt templates as editable semantic structures rather than fixed two-level instructions. "Learning from Prompt itself" proposes Hierarchical Attribution Prompt Optimization (HAPO), which models a prompt as a hierarchical semantic object made of units that can be attributed, ranked, and edited (Chen et al., 6 Jan 2026). Instead of generating entirely new prompts from scratch, HAPO segments a prompt into semantic units, computes dynamic attribution scores, selects high-value edit targets with UCB, applies an edit operator, evaluates reward on a dev split, and then reinserts the best candidate into a meta-prompt. Its practical loop is summarized as segment → attribute → select → edit → reinsert into meta-prompt → regenerate.

HAPO’s template logic is explicitly modular. The prompt is segmented as

$S(p)=\{u_k\}_{k=1}^K=\Pi(\mathrm{RuleSplit}(p)),$

where RuleSplit(p) uses discourse markers, section headers, list items, and delimiters, while $\Pi$ is a frozen instruction parser that merges short fragments and splits run-on clauses. The resulting units can correspond to task objective, constraints, output format, examples, key notes, and multimodal annotations. The edit operator set is compact—Replace, Insert, Delete, Reorder, and Refine—and the framework is explicitly designed to mitigate prompt drift while preserving interpretability (Chen et al., 6 Jan 2026).

"PromptPrism" systematizes this view by defining a prompt taxonomy with three hierarchical levels: functional structure, semantic component, and syntactic pattern (Jeoung et al., 19 May 2025). At the functional level, prompts are role-organized discourse units such as System, User, Assistant, and Tools. At the semantic level, units are decomposed into purpose-bearing components such as Instruction, Contextual / Reference info, Output Constraints, User request / Question, and Response. At the syntactic level, the taxonomy records component indices, spans, delimiters, directive markers, and special tokens. The framework’s XML-like tags—such as <instruction:task>, <instruction:guideline:safety>, and <output_const:format>—turn a prompt into a nested labeled object suitable for analysis, controlled rewriting, and sensitivity experiments (Jeoung et al., 19 May 2025).

A related but more execution-oriented approach appears in "Towards Hierarchical Multi-Agent Workflows for Zero-Shot Prompt Optimization," which organizes prompt construction through a CEO–Manager–Worker pipeline with skip connections from the original query (Liu et al., 2024). Here the hierarchy lies in abstraction level: the CEO generates high-level guidance, the Manager expands it into detailed instructions, and the Worker produces the final response. Although this is not an embedding template in the narrow sense, it exemplifies the same structural principle: prompt quality is improved by allocating distinct semantic responsibilities to distinct levels (Liu et al., 2024).

4. Domain-specific realizations

Hierarchical prompt templates have been specialized to multiple application domains, often by aligning prompt levels with an external hierarchy such as label depth, ontology structure, or coarse-to-fine class relations.

In hierarchical text classification, "HPT: Hierarchy-aware Prompt Tuning for Hierarchical Text Classification" constructs a dynamic virtual template whose length equals hierarchy depth $L$:

$$\text{[CLS]} \; x \; \text{[SEP]} \; [V_1] \; [PRED] \; [V_2] \; [PRED] \; \cdots \; [V_L] \; [PRED] \; \text{[SEP]}.$$

Each hierarchy layer gets its own template slot and prediction token, and each slot predicts only labels from its own depth via layer-specific verbalizers. A graph attention network over label nodes and virtual depth nodes injects label-hierarchy structure into the soft prompt vectors, thereby recasting hierarchical text classification as a hierarchy-aware multi-label MLM problem (Wang et al., 2022).

In biomedical knowledge fusion, "HiPrompt" uses hierarchy-oriented prompts to rank candidate biomedical terms by exposing parent-child structure directly in the prompt (Lu et al., 2023). The prompt combines a Query, a candidate Choices list, and hierarchy context formatted as “Contexts: $\{t'_1$ isA $t'_{1,p}; \dots; t'_K$ isA $s$0”. This allows an LLM reranker to reason about specificity and ancestor relations rather than relying only on lexical overlap. The paper’s examples emphasize choosing the most specific valid term in a biomedical DAG rather than merely the closest surface form (Lu et al., 2023).

In vision-language prompt tuning, "Learning Hierarchical Prompt with Structured Linguistic Knowledge for Vision-LLMs" introduces a three-level hierarchy of global-level prompts, high-level prompts, and low-level prompts (Wang et al., 2023). Global prompts are category-agnostic learnable context vectors; high-level prompts are generated from layerwise last-token representations of LLM-generated descriptions; low-level prompts consist of entity and attribute tokens derived from structured graphs extracted from those descriptions. "HPT++" retains the same multi-level organization while improving multi-granularity knowledge generation, relation modeling, and consistency regularization (Wang et al., 2024).

In hierarchical image classification, "TransHP" learns one prompt token per coarse class, predicts the coarse class at an intermediate Transformer block, and injects the predicted ancestor prompt into intermediate features so that later layers focus on descendant discrimination (Wang et al., 2023). The method is explicitly described as the first to inject ancestor-class information as a tokenized hint benefiting descendant-class discrimination. A related but training-free hierarchy-aware use of language prompts appears in "Making Better Mistakes in CLIP-Based Zero-Shot Classification with Hierarchy-Aware Language Prompts," where leaf-peer, ancestor-peer, and path-based prompts are sent to an LLM to generate CLIP text classifiers that reduce mistake severity under a label tree (Liang et al., 4 Mar 2025).

Hierarchical prompt design has also been internalized into parameterized tuning mechanisms. "Teaching Prompts to Coordinate" proposes a hierarchical layer-grouped prompt tuning method for continual learning in which layers are partitioned into groups, a single task-specific root prompt generates shared implicit prompts for each group, and a position incentive embedding differentiates layers within a group (Jiang et al., 15 Nov 2025). This replaces fully independent layer-wise prompts with a rooted, group-wise, position-aware hierarchy intended to reduce catastrophic forgetting (Jiang et al., 15 Nov 2025).

5. Formal mechanisms and representational principles

The formal core of hierarchical embedding prompt templates is the assignment of distinct operations to distinct representational levels. In the multimodal embedding setting, the objective is expressed as keeping

$s$1

with the embedding extracted from the last input token and then L2-normalized (Ju et al., 1 Aug 2025). The prompt hierarchy functions as a control signal over the hidden state from which $s$2 is read out.

In HAPO, prompt structure is made explicit through semantic segmentation and optimization over units rather than tokens. Its attribution machinery includes the dynamic score

$s$3

the history-aware score

$s$4

and a UCB-based edit policy with reward

$s$5

These definitions formalize a prompt template as a mutable hierarchy of attributable units with explicit drift monitoring via $s$6 and $s$7 (Chen et al., 6 Jan 2026).

In hierarchical text classification, the template itself is depth-indexed. The input embedding sequence is written as

$s$8

with $s$9 the learnable template embedding for hierarchy layer $s$0 and $s$1 the special prediction token embedding. After encoding, each $s$2 becomes a layer-specific prediction state. The zero-bounded multi-label cross entropy

$s$3

aligns the prompt formulation with MLM while preserving multi-label hierarchical prediction (Wang et al., 2022).

In hierarchical vision-language prompt tuning, structure enters attention directly. HPT defines a relation matrix

$s$4

and modifies attention as

$s$5

HPT++ replaces additive biasing with multiplicative reweighting,

$s$6

and adds a consistency term

$s$7

which constrains the hierarchical text encoder to remain close to the pre-trained text encoder (Wang et al., 2023, Wang et al., 2024).

By contrast, cone embedding formalizes hierarchy geometrically rather than prompt-wise. Its metric-cone distance is

$s$8

and its hierarchy score uses the height coordinate: $s$9 This makes hierarchy intrinsic to the embedding geometry rather than an effect of prompt conditioning (Takehara et al., 2021).

6. Empirical profile, strengths, and limitations

The empirical record shows that hierarchical prompt templates are used for at least three distinct purposes: zero-shot embedding induction, structured prompt optimization, and hierarchy-aware transfer across downstream tasks. The reported outcomes are heterogeneous but consistently framed as improvements over flatter prompt designs or over hierarchy-agnostic baselines.

Paper	Setting	Reported outcome
"From Generator to Embedder" (Ju et al., 1 Aug 2025)	MMEB zero-shot	43.3 average vs 13.9 base; +4.8 fine-tuning lift
"Learning from Prompt itself" (Chen et al., 6 Jan 2026)	Multimodal prompt optimization	58.9 text-only, 61.5 original image, 65.7 enhanced features
"HPT" (Wang et al., 2022)	Hierarchical text classification	WOS 87.16/81.93; RCV1-V2 87.26/69.53; NYT 80.42/70.42
"Learning Hierarchical Prompt" (Wang et al., 2023)	Base-to-new VLM generalization	+2.04 base, +1.72 new, +1.68 harmonic mean
"HPT++" (Wang et al., 2024)	Cross-dataset VLM evaluation	68.02 average accuracy
"HAPrompts" (Liang et al., 4 Mar 2025)	CLIP zero-shot with label tree	78.49 Top1, 2.43 severity, 0.50 HD@1

Several strengths recur. Hierarchical templates can impose a global representation regime without parameter updates, as in zero-shot multimodal embedding (Ju et al., 1 Aug 2025). They can make prompt editing interpretable and drift-aware by localizing modifications to semantic units (Chen et al., 6 Jan 2026). They can align prompt slots with external structure such as label depth, biomedical DAGs, or entity-attribute graphs, thereby improving performance in low-resource, fine-grained, or structured prediction settings (Wang et al., 2022, Lu et al., 2023, Wang et al., 2023). They can also improve error quality rather than only top-1 accuracy, as in hierarchy-aware CLIP prompting for “making better mistakes” (Liang et al., 4 Mar 2025).

The limitations are equally consistent. Hierarchical prompting is not cost-free: multi-agent workflows require more than one inference pass and therefore incur extra runtime (Liu et al., 2024). Depth-aligned templates can grow with hierarchy depth and may approach sequence-length limits (Wang et al., 2022). Methods that depend on LLM-generated descriptions or hierarchy extraction inherit possible errors, omissions, hallucinations, and biases from the upstream LLM (Wang et al., 2023, Liang et al., 4 Mar 2025). Retrieval-rerank frameworks such as HiPrompt remain bounded by the quality of the retriever: if the correct term is absent from the candidate list, the LLM cannot recover it (Lu et al., 2023). In continual learning, overly flexible layer-wise prompts are identified as a source of forgetting, which motivates hierarchical coordination but also introduces design choices about grouping granularity and positional encoding (Jiang et al., 15 Nov 2025).

A common misconception is that a hierarchical embedding prompt template is necessarily a single textual scaffold. The surveyed work indicates otherwise. In some settings it is a literal two-level chat template; in others it is a semantic edit graph, a depth-indexed soft-prompt sequence, a multi-level prompt stack inside a Transformer, or a hierarchy-aware language-prompt generator for CLIP (Ju et al., 1 Aug 2025, Chen et al., 6 Jan 2026, Wang et al., 2022, Wang et al., 2023, Liang et al., 4 Mar 2025). This suggests that the term is best understood as a structural design principle: prompt components are organized so that coarse control, local specialization, and structural priors are separated but coordinated.