Event-Image Knowledge Distillation
- Event-image knowledge distillation is a framework that uses a teacher–student paradigm to transfer cross-modal supervision from images to event representations.
- It leverages methods like probabilistic ranking, feature regression, and relational graph-based losses to enhance performance in tasks such as multimodal summarization, depth estimation, and object detection.
- The approach supports modular training designs where auxiliary components used during training are removed at inference, ensuring efficient, reliable performance across various benchmarks.
An event-image knowledge distillation framework is a teacher–student paradigm in which supervision extracted from an image-domain, frame-domain, or vision-LLM is transferred to a student that must model events. In the cited literature, “event” has two distinct technical meanings. In multimodal summarization, it denotes semantic event structure in news, where a gold summary serves as an event-level description and images are ranked by their relevance to that description. In neuromorphic vision, it denotes asynchronous event-camera streams, represented as spikes, voxel grids, or event tensors, which are distilled from aligned grayscale or RGB imagery. Across these settings, the defining property is training-time cross-modal supervision and test-time reliance on the student event representation or on a unified multimodal model rather than on the teacher modality itself (Zhang et al., 2021, Zhang et al., 2024, Li et al., 2023, Bao et al., 23 Jun 2026, Zhang et al., 1 Oct 2025, Wang et al., 2021).
1. Scope and representative formulations
Within the supplied literature, the term covers a family of frameworks rather than a single architecture. The family includes event-centric image selection in multimodal summarization, event-camera depth estimation, event-based object detection, open-vocabulary event detection, and cross-modal transfer to unlabeled event data. What remains constant is the use of a strong image-side teacher to shape an event-side or event-aware student.
| Instantiation | Teacher and student | Transferred signal |
|---|---|---|
| "UniMS: A Unified Framework for Multimodal Summarization with Knowledge Distillation" (Zhang et al., 2021) | CLIP ViT-B-32 teacher, BART-based multimodal encoder–decoder student | Distribution over candidate images conditioned on gold summary |
| "A Novel Spike Transformer Network for Depth Estimation from Event Cameras via Cross-modality Knowledge Distillation" (Zhang et al., 2024) | Frozen DINOv2 teacher, spike-driven transformer student | Feature-map matching plus supervised depth |
| "Object-centric Cross-modal Feature Distillation for Event-based Object Detection" (Li et al., 2023) | Grayscale/RGB detector teacher, event detector student | Slot features, attention-masked pixel features, inter-object affinities |
| "M2C-EvDet: Multi-Domain Multi-Order Cross-Modal Knowledge Distillation for Event-based Object Detection" (Bao et al., 23 Jun 2026) | RGB detector teacher, event detector student | Frequency-decoupled features and low-/high-order relations |
| "Adaptive Event Stream Slicing for Open-Vocabulary Event-Based Object Detection via Vision-Language Knowledge Distillation" (Zhang et al., 1 Oct 2025) | Frozen CLIP image/text encoders, SNN+CNN event detector student | Region-level contrastive distillation with spatial attention |
| "EvDistill: Asynchronous Events to End-task Learning via Bidirectional Reconstruction-guided Cross-modal Knowledge Distillation" (Wang et al., 2021) | Image teacher plus event/APS students | Bidirectionally bridged prediction, distribution, graph, and mutual distillation |
The summarization formulation is notable because it makes the event–image alignment problem explicit. Given a multimodal document , UniMS outputs an abstractive summary and an image subset ; in the MSMO news setting, the images are event-related, and the CLIP-based distillation signal is the image–summary similarity distribution over the document image set. In the event-camera literature, the event stream is the student modality and the image or frame modality is the teacher, with the teacher used only during training in SDT, the object-centric detector, MC-EvDet, the open-vocabulary detector, and EvDistill (Zhang et al., 2021, Zhang et al., 2024, Li et al., 2023, Bao et al., 23 Jun 2026, Zhang et al., 1 Oct 2025, Wang et al., 2021).
2. Canonical teacher–student formulations
The cleanest probabilistic formulation appears in UniMS. For each candidate image , CLIP computes an image–summary similarity
which is softened into a teacher distribution
The student image-selection head produces scores and a corresponding distribution
Distillation minimizes
so the student learns CLIP’s relative ranking of event-relevant images without requiring captions (Zhang et al., 2021).
In event-camera depth estimation, the formalism shifts from distribution matching to feature regression. SDT uses a frozen DINOv2 teacher on RGB images and a spike-driven transformer student on event spikes. The feature/domain term is described through 0 and perceptual loss, while the task loss is the scale-invariant depth loss
1
The total training objective is described conceptually as
2
with the teacher fixed and only the spiking network updated (Zhang et al., 2024).
Detection-oriented frameworks replace global matching with structured feature supervision. M3C-EvDet defines
4
where 5 and 6 distill low- and high-frequency components, and 7 and 8 distill low-order and high-order relations (Bao et al., 23 Jun 2026). The open-vocabulary event detector instead uses a cross-modal InfoNCE-style region loss,
9
in which spatial attention 0 reweights the event-region embedding before contrastive alignment with CLIP image embeddings (Zhang et al., 1 Oct 2025).
EvDistill is the most expansive formulation. Its total loss combines supervised source-domain training, bidirectional modality reconstruction, distribution adaptation, affinity graph distillation, and mutual distillation: 1 This is the only cited framework explicitly designed for unlabeled and unpaired event data, using bidirectional reconstruction to craft synthetic cross-modal pairs for distillation (Wang et al., 2021).
3. Alignment mechanisms and the knowledge being transferred
The frameworks differ less in teacher choice than in what they regard as the transferable object. UniMS transfers a distribution over images, not raw embeddings. The gold abstractive summary 2 is treated as an event-level proxy; CLIP’s similarity to each image acts as a pseudo-labeling mechanism for event–image matching, and the student learns image relevance scores from a multimodal encoder in which text tokens and image patches attend to one another (Zhang et al., 2021).
A second class of frameworks transfers dense feature structure. SDT directly regresses event features toward DINOv2 RGB features after spatial-resolution matching by upsampling the RGB input by a factor of 1.75. The open-vocabulary detector also performs feature-space transfer, but at the level of proposal regions and with explicit spatial saliency: for a feature tensor 3, it defines
4
then averages event and image attention maps to obtain 5, which is used to weight event features during contrastive distillation. This makes the student focus on spatial regions that are salient in both modalities rather than on arbitrary event edges (Zhang et al., 2024, Zhang et al., 1 Oct 2025).
A third class transfers object-centric or relational structure. The object-centric detector applies slot attention to FPN feature maps, producing object-centric slot features, attention maps over pixels, direct slot-feature distillation, attention-guided pixel-feature distillation, and affinity-matrix distillation based on cosine similarities between slots. M6C-EvDet generalizes this relational view further: AF7D8 performs multi-level DWT decomposition into low- and high-frequency components, while MORD distills both self-attention-based pairwise relations and hypergraph-based high-order relations. EvDistill likewise avoids naive feature matching by using an affinity graph loss over local neighborhoods and a modality discriminator to align source and target distributions (Li et al., 2023, Bao et al., 23 Jun 2026, Wang et al., 2021).
This suggests a useful taxonomy. Some frameworks distill rankings over candidate images, some distill features or regions, and others distill relations or graphs. A common misconception is that event-image KD is synonymous with temperature-scaled logit matching. The cited work shows the opposite: the central supervisory objects are frequently distributions over images, feature maps, slot embeddings, affinities, wavelet subbands, hypergraph relations, or rationale-conditioned internal states rather than class logits alone (Zhang et al., 2021, Zhang et al., 2024, Li et al., 2023, Bao et al., 23 Jun 2026, Wang et al., 2021).
4. Architectural patterns
One architectural pattern is the single unified multimodal encoder. UniMS extends BART’s encoder to accept both text embeddings and projected image patches, concatenates them into one sequence, and lets cross-modal interaction occur entirely inside the shared Transformer encoder. The same encoder hidden states support extractive sentence scoring, image selection, and visual-guided abstractive decoding. Its decoder inserts separate cross-attention blocks to visual hidden states 9 and textual hidden states 0, making summary generation explicitly conditioned first on images and then on text (Zhang et al., 2021).
A second pattern is the event-native student paired with a frozen image teacher. SDT exemplifies the event-native side: it uses a spiking patch embedding, multi-step leaky integrate-and-fire neurons, spiking self-attention without softmax, binary 1, 2, and 3, and a fusion depth estimation head that upsamples and sums multi-stage features. The network is intentionally shallow at 4 blocks because the authors argue that knowledge distillation compensates for capacity (Zhang et al., 2024).
A third pattern is the use of training-time modules that disappear at inference. The object-centric detector adds slot attention, coarse and fine alignment networks, and auxiliary heads during training, but keeps only the event detector at test time. M5C-EvDet inserts AF6D7 and MORD as plug-in modules over multi-scale PAFPN features and notes that hypergraph computation exists only during training, so inference overhead is unchanged. The open-vocabulary detector adds an SNN-based adaptive slicer whose spikes determine event segmentation boundaries, then hands the resulting feature maps to a Mask R-CNN-style detector with CLIP text embeddings as classifier weights (Li et al., 2023, Bao et al., 23 Jun 2026, Zhang et al., 1 Oct 2025).
A fourth pattern is explicit modality bridging. EvDistill introduces Bidirectional Modality Reconstruction with generators 8 and 9, discriminators, cycle consistency, and Dynamic Semantic Consistency. BMR is trained jointly with the task networks and KD losses, then discarded at inference. The dense-prediction literature in the supplied material proposes a related transferable pattern: complementary boundary and context distillation isolates explicit boundary supervision and self-relation supervision as training-only branches, and the accompanying discussion presents this formulation as directly adaptable to event–image KD for dense tasks while keeping inference cost unchanged (Wang et al., 2021, Zhang et al., 2024).
5. Reported empirical behavior
| Paper | Reported result | Diagnostic ablation |
|---|---|---|
| UniMS (Zhang et al., 2021) | Full UniMS: R-1 42.94, R-2 20.50, R-L 40.96, IP 69.38, 0 29.72 | KD-based image reference slightly exceeds caption-based ROUGE-ranking |
| SDT (Zhang et al., 2024) | DENSE: Abs Rel 0.80, Sq Rel 8.32, SI log 0.46, 1 0.53, Power 12.43 mJ | No KD: Abs Rel 2.89, Sq Rel 72.25, 2 0.39 |
| M3C-EvDet (Bao et al., 23 Jun 2026) | DSEC-Detection: baseline 24.7 mAP, full model 28.3 mAP, 45.9 mAP50 | AF4D5 alone 27.4 mAP; MORD alone 27.2 mAP |
| Object-centric detector (Li et al., 2023) | CARLA: student 53.8 mAP, distilled 56.4, teacher 59.6; DSEC: student 27.4, distilled 30.8 | RGB teacher harms distillation: distilled student 51.1 on CARLA |
| Open-vocabulary detector (Zhang et al., 1 Oct 2025) | DSEC: Car 0.545, Pedestrian 0.312, Large vehicle 0.408; Gen1: 0.657 mAP50, 0.383 mAP50:95 | LIC + SSF + KD + Frame gives best Gen1 ablation |
| EvDistill (Wang et al., 2021) | DDD17 events 58.02 MIoU; MVSEC events 55.09 MIoU, APS 68.85; N-Caltech 0.902 accuracy | AG KD and BMR both improve over prediction-only variants |
The summarization case shows a particularly clean evaluation of caption-free supervision. UniMS reports that caption-based ROUGE-ranking gives R-L 40.90, IP 69.25, and 6 29.56, while the KD-based system gives R-L 40.96, IP 69.38, and 7 29.72. The full model also outperforms variants without the visual-guided decoder or without 8, indicating synergy among image distillation, extractive event salience, and abstractive generation (Zhang et al., 2021).
The event-camera depth results indicate that KD is not marginal. On DENSE, the proposed SDT + distillation reduces Abs Rel from 1.57 in Spike-T to 0.80 and lowers power from 41.77 mJ to 12.43 mJ; on DSEC, the proposed method reaches Abs Rel 1.000, Sq Rel 0.999, RMS log 0.105, SI log 0.212, and 9 0.387. The ablation with a fusion head but no KD is especially strong evidence: Abs Rel degrades to 2.89 and Sq Rel to 72.25, implying that the DINOv2 teacher is the dominant contributor to accurate depth structure in the low-data spiking regime (Zhang et al., 2024).
For event-based detection, the strongest evidence for structured distillation comes from complementary ablations. M0C-EvDet improves a D-FINE event-only baseline from 24.7 to 28.3 mAP and exceeds FGD at 27.3 mAP; low-frequency KD only gives 26.6, high-frequency KD only 26.7, AF1D2 27.4, LORD + HORD 27.2, and the full AF3D4 + MORD 28.3, which indicates additive benefit from frequency-domain and relational supervision. The object-centric detector reports that on CARLA the student rises from 53.8 to 56.4 mAP, reducing the teacher–student gap from 5.8 to 3.2 mAP, and on DSEC it rises from 27.4 to 30.8 mAP. The open-vocabulary detector reports 0.408 mAP on the novel “Large vehicle” category in DSEC after training only on base classes, and its Gen1 ablation shows progressive gains from LIC, SSF, KD, and then real frame supervision (Bao et al., 23 Jun 2026, Li et al., 2023, Zhang et al., 1 Oct 2025).
EvDistill demonstrates that paired labels in the event domain are not strictly necessary. On DDD17, it reports 58.02 MIoU on events without using pseudo-labels on events; on MVSEC, 55.09 MIoU on events and 68.85 on APS; on N-Caltech101, 0.902 accuracy without event labels, surpassing supervised event baselines listed in the paper. Its ablations further show that affinity graph KD outperforms FitNet and AT in the reported setting, and that removing BMR weakens performance (Wang et al., 2021).
6. Limitations, misconceptions, and directions of development
Most event-camera formulations still require paired RGB–event data during training, together with reasonable temporal and spatial alignment. SDT assumes synchronized RGB, depth, and event sequences; M5C-EvDet relies on RGB–Event datasets with frame-level alignment and transferred annotations; the object-centric detector requires aligned grayscale images and event voxel grids; and the open-vocabulary detector depends on corresponding frames or reconstructed frames for CLIP supervision. EvDistill is the explicit exception in the supplied material: it is designed for unlabeled and unpaired event data and uses bidirectional reconstruction to bridge the modalities (Zhang et al., 2024, Bao et al., 23 Jun 2026, Li et al., 2023, Zhang et al., 1 Oct 2025, Wang et al., 2021).
Teacher quality is also not a scalar notion. A stronger teacher in its own modality is not always a better distillation source. The object-centric detector shows that a grayscale teacher yields a distilled CARLA student at 56.4 mAP, whereas an RGB teacher, although stronger as a detector, reduces the distilled student to 51.1 mAP. UniMS similarly notes that CLIP-based KD is appealing not because it is dramatically stronger than caption-based ranking on MSMO, but because it removes the requirement for image captions. These cases indicate that modality proximity and supervision form matter as much as nominal teacher accuracy (Li et al., 2023, Zhang et al., 2021).
Another recurring issue is supervision granularity. UniMS distills summary-level event–image alignment, not sentence-level or event-node-level alignment, and explicitly notes that mapping individual events may require more fine-grained teacher queries. SDT reports that temperature and encoder-layer placement matter, with 6 performing best and middle layers favoring image selection while earlier layers can benefit generation. M7C-EvDet adds training-time complexity through K-Means-based hyperedge construction, and the open-vocabulary detector adds several slicing losses—8, 9, and 0—whose optimization burden is not present in standard detector training (Zhang et al., 2021, Zhang et al., 2024, Bao et al., 23 Jun 2026, Zhang et al., 1 Oct 2025).
The supplied literature also points toward broader design space. The dense-prediction work on complementary boundary and context distillation isolates explicit object-level semantic boundaries and implicit pixel-level self-relations as separate training-only knowledge channels, and the discussion explicitly presents this as a transferable pattern for event–image KD in dense tasks (Zhang et al., 2024). A plausible implication is that event-image KD need not remain confined to feature and relation transfer alone: rationale-oriented distillation, in which a teacher generates and then distills explanation-conditioned representations for event pairs, could provide a higher-level supervisory layer for multimodal event reasoning, although that extension is suggested rather than demonstrated in the supplied material (Nath et al., 2024).