---
title: Slot-Based Historical Image Compression
url: https://www.emergentmind.com/topics/slot-based-historical-image-compression-shic
type: topic
---

# Slot-Based Historical Image Compression

Slot-Based Historical Image Compression (SHIC) is an architectural module for efficiently distilling high-dimensional, temporally extended visual histories into compact, expressive fixed-length representations. First introduced within the LongFly framework for long-horizon UAV vision-and-language navigation, SHIC transforms large-scale multi-view image sequences into a concise set of learnable slot embeddings. Through recurrent attention and gated updates, SHIC preserves crucial semantic and landmark information over extensive visual histories while maintaining strictly bounded memory and compute costs. Its integration into complex, spatiotemporally aware agents enables tractable reasoning over long observation windows without prohibitive resource demands [2512.22010].

## 1. Role within the LongFly Pipeline

SHIC operates as the initial sub-module in LongFly’s history-aware context modeling architecture for UAV navigation. The overall system comprises three major components:  
1. **SHIC:** Slot-based Historical Image Compression  
2. **STE:** Spatio-Temporal Trajectory Encoding  
3. **PGM:** Prompt-Guided Multimodal Integration  

At each decision step $t$, SHIC processes the accumulated visual history—specifically, all multi-view images $\{R_1,\ldots,R_{t-1}\}$—to produce a fixed-length matrix of slot embeddings $S_{t-1} \in \mathbb{R}^{K \times d}$. This set $S_{t-1}$, referred to as the compressed slot “memory,” feeds directly into the later multimodal PGM module, together with STE tokens and an instruction $L$. By abstracting the variable-length sequence of raw visual tokens to a fixed number of semantically rich slot representations, SHIC facilitates scalable long-horizon temporal reasoning while overcoming the bottlenecks associated with storing and computing over the entire observation history.

## 2. Inputs, Outputs, and Dataflow

At each time step $i = 1, \ldots, t-1$, the following inputs are provided to SHIC:
- Multi-view RGB images $R_i = \{R_i^{(1)}, \ldots, R_i^{(5)}\}$ from front, rear, left, right, and bottom UAV cameras.
- A pretrained CLIP-based visual encoder $F_v$ maps each image to spatial tokens:  
  $$ Z_i = F_v(R_i) = \{z_{i,1}, \ldots, z_{i,N_i}\}, \quad z_{i,j} \in \mathbb{R}^d $$
  Typical dimensionalities are $d=768$ or $d=1024$, and $N_i \approx 980$ tokens (from $14 \times 14$ grids across five views).

After iterative integration of all $t-1$ histories, the output is a set of $K$ slots:
$$ S_{t-1} = [s_{t-1,1}, \ldots, s_{t-1,K}], \quad s_{\cdot,\cdot} \in \mathbb{R}^d $$
Here, $K$ is a hyperparameter typically set in the range $8\leq K\leq64$, with $K=32$ as a canonical value. This output matrix substitutes for the vastly larger set $\{Z_1, \ldots, Z_{t-1}\}$ in downstream processing.

## 3. Core Mathematical Operations

SHIC compresses historical visual tokens via recurrent slot attention augmented by Gated Recurrent Unit (GRU) updates:

**(a) Initialization**  
At $i=1$, initialize slot matrix as learnable parameters:
$$ S_1 = \Phi, \quad \Phi \in \mathbb{R}^{K \times d} $$

**(b) Query-Key-Value (QKV) Projection**  
For each subsequent time step $i=2, \ldots, t-1$:
- For slots from previous step: $S_{i-1} \in \mathbb{R}^{K \times d}$
- For new tokens: $Z_i \in \mathbb{R}^{N_i \times d}$
Project into QKV space:
\[
\begin{aligned}
q_{i-1, k} &= W_q s_{i-1, k} \\
k_{i, j}   &= W_k z_{i, j} \\
v_{i, j}   &= W_v z_{i, j}
\end{aligned}
\]
with $W_q, W_k, W_v \in \mathbb{R}^{d \times d}$.

**(c) Slot-Token Attention**  
Compute slot-to-token attention via scaled dot-product:
\[
\alpha_{i,k,j} = \mathrm{softmax}_{j'}\left(\frac{q_{i-1,k}^\top k_{i,j}}{\sqrt{d}}\right)
\]
This yields an affinity matrix between all $K$ slots and $N_i$ tokens.

**(d) Token Aggregation and Slot Update**  
Update each slot with a weighted sum of token values:
\[
\hat{s}_{i,k} = \sum_{j=1}^{N_i} \alpha_{i,k,j} v_{i,j}
\]
Aggregate into matrix $\hat{S}_i \in \mathbb{R}^{K \times d}$.

**(e) Recurrent (GRU) Fusion**
Update slots recurrently:
\[
S_i = \mathrm{GRU}(S_{i-1}, \hat{S}_i)
\]
After all steps, output $S_{t-1}$ as the compressed visual memory.

## 4. Pseudocode and Training Approach

The SHIC compression process can be summarized as follows:

```
Algorithm: SHIC_Compress
Inputs: R₁,…,R_{t−1}, F_v, K, d, W_q, W_k, W_v ∈ ℝ^{d×d}, GRU cell, Φ ∈ ℝ^{K×d}
Output: S ← compressed slots ∈ ℝ^{K×d}

1. S ← Φ  # learnable init, shape K×d
2. for i in 1…(t−1) do
3.   Z_i ← F_v(R_i)  # Z_i: N_i×d token matrix
4.   if i>1 then
5.     Q ← S W_q⊤
6.     K_tokens ← Z_i W_k⊤
7.     V_tokens ← Z_i W_v⊤
8.     for each slot k=1..K do
9.       α_{k,·} ← softmax( (Q_k K_tokens⊤)/√d )
10.      \hat S_k ← α_{k,·} V_tokens
11.    S ← GRU( S, \hat S )  # update all K slots
12. end for
13. return S
```

There is no explicit reconstruction or auxiliary loss in SHIC; its parameters (QKV projections, GRU, slot initialization) are updated jointly with the entire LongFly model. Supervised losses—such as those arising in PGM for waypoint regression—implicitly drive the training of SHIC parameters via back-propagation [2512.22010].

## 5. Hyperparameters and Trade-Offs

The primary hyperparameters in SHIC are $K$ (number of slots) and $d$ (feature dimension):

- **K (slots):** Controls the memory capacity and compression fidelity. Smaller $K$ (e.g., 8) yields higher compression and lower compute, at the expense of semantic detail. Larger $K$ ($32-64$) increases memory and fidelity but raises computational cost.
- **d (token/slot dimension):** Determined by CLIP encoder, e.g. $d=768$.
- **$N_i$ (tokens per step):** Typically $\approx 1000$, from 5 views.
- **Softmax temperature $\frac{1}{\sqrt{d}}$:** Stabilizes gradient flow during slot-token attention.
- **Compression ratio:** For $t=100$, $N_i=1000$, $K=32$:
  \[
  \mathrm{Compression\,ratio} \approx \frac{t \cdot N_i}{K} \approx 3125\times
  \]
- **Computational complexity:**  
  - Attention: $\mathcal{O}(K \cdot N_i \cdot d)$ per step.
  - Memory: $\mathcal{O}(K \cdot d)$, compared to $\mathcal{O}(t \cdot N_i \cdot d)$ without compression.
- Returns from increasing $K$ diminish beyond $K \approx d$; excess slots often become redundant.

These trade-offs control real-time performance, with practical $K$ values ensuring that $K \cdot d \cdot t$ fits deployment constraints while maximizing informative context retention.

## 6. Architectural Dynamics and Information Flow

Conceptually, SHIC instantiates a fixed set of $K$ semantic “buckets,” each aggregating and abstracting distributed visual information across the trajectory:

- At each time step, five camera images generate a “cloud” of $N_i$ CLIP tokens.
- The $K$ slot nodes, initialized with learnable parameters, evolve recurrently over time as each attends to and integrates new observational tokens.
- Slot attention yields incremental slot-wise updates, temporally propagating and refining landmark and contextual information through recurrent GRU fusion.
- At the conclusion of the historical window, $S_{t-1}$ comprises a checkpointed set of embeddings characterizing the salient visual context for integration with trajectory and instruction data in downstream navigation planning.

This recurrent, slot-centric abstraction scheme enables generalization to arbitrarily long observation histories, eliminating the need for computationally intensive global attention over all tokens.

## 7. Compression Efficacy and Limitations

SHIC achieves substantial compression of spatiotemporal visual history, with empirical compression ratios exceeding 3,000× under typical navigation horizons and tokenization schemes. The architecture explicitly bounds memory and per-step compute irrespective of visual history length.

A plausible implication is that, while increasing $K$ improves representational fidelity, task performance improvements saturate rapidly and surplus slots add redundancy rather than new information. Moreover, the absence of explicit reconstruction loss may limit the preservation of certain fine-grained details. Nonetheless, in applied contexts such as vision-and-language navigation, SHIC demonstrates sufficient semantic precision, outperforming alternative memory architectures in both success rate and path-length-weighted success metrics [2512.22010].

Source: https://www.emergentmind.com/topics/slot-based-historical-image-compression-shic