Blink Data Generation Methods

• Blink data generation is a family of methods that convert raw visual or sensor signals into structured supervision signals for applications like face detection and attention localization.
• It employs diverse techniques including event-based temporal analysis, automated ROI annotation in GUI agents, and blink-aware conditioning in face synthesis pipelines.
• Challenges include inconsistent annotation regimes, temporal ambiguities, and the need for precise synchronization across heterogeneous data sources.

Blink data generation denotes a family of procedures for constructing blink-related supervision, representations, or conditioning signals rather than a single standardized method. In the literature, it includes manually averaged event-camera blink templates for face detection, automated region-of-interest annotations for a GUI agent’s “Blink” stage, blink-conditioned latent features for talking-face synthesis, continuous physiological blink waveforms and spectrograms, and multisensor blink datasets built from frame, sequence, or event annotations (Lenz et al., 2018, Zhang et al., 19 Sep 2025, Zhang et al., 26 Jan 2026, Cho, 2021, Nakano et al., 2020, Daza et al., 2023, Alves et al., 3 Jun 2026). In one important non-ocular usage, “Blink” denotes a fast visual attention phase rather than an eyelid event, so the phrase can refer either to eye-blink data or to attention-localization supervision, depending on the paper (Zhang et al., 19 Sep 2025).

1. Conceptual scope and representational regimes

The common structure across blink data generation methods is the conversion of raw observations into compact supervision tailored to a downstream task. In event-based vision, the raw signal is the asynchronous event tuple

$ev=(x,y,t,p),$

with pixel coordinates $(x,y)$, timestamp $t$, and polarity $p\in\{\mathrm{ON},\mathrm{OFF}\}$; the generated blink data are temporal ON/OFF activity traces or local templates rather than geometric eyelid labels (Lenz et al., 2018). In GUI-agent work, the raw input is a screenshot plus task context, and the generated blink data are structured ROI annotations represented as selected UI elements with bounding boxes and dynamic or static captions (Zhang et al., 19 Sep 2025). In talking-face synthesis, blink information is produced as an eye-condition feature derived from OpenFace action units and eye landmarks rather than as an explicit event sequence (Zhang et al., 26 Jan 2026).

Other regimes move further toward physiological or behavioral abstraction. One line derives a continuous eye-openness waveform from facial landmarks and transforms it into a blink spectrogram in the time-frequency domain (Cho, 2021). Another converts viewer eye-tracker signals into framewise blink-rate targets by aggregating the percentage of people blinking at each video frame (Nakano et al., 2020). Large supervised datasets extend this spectrum with frame-level open/closed labels, sequence-level blink/no-blink clips, multispectral eye crops, and species-specific half-blink versus full-blink annotations (Daza et al., 2023, Alves et al., 3 Jun 2026).

2. Event-based temporal template generation

In “Event-based Face Detection and Tracking in the Blink of an Eye,” a blink is defined through event-based temporal dynamics rather than eyelid geometry. The method treats a blink as a stereotyped local burst pattern with low activity before blink, an ON-dominated closing phase, a quiet closed phase, and an OFF-dominated reopening phase. The physiological blink duration cited from the literature is typically $100\text{--}150\,\mathrm{ms}$, while the detector uses a $250\,\mathrm{ms}$ temporal window to capture the full local signature (Lenz et al., 2018).

The blink model is generated from manually annotated examples. The paper states that robust models can be built “by manually labelling fewer than 50 blinks,” and the model shown was generated from 20 blinks from 4 subjects. Annotation marks the “very centre of the eye,” after which events within a spatio-temporal window of one tile size and $250\,\mathrm{ms}$ are used to build the trace. Local activity is accumulated separately for ON and OFF polarities with an exponential decay update and temporal resolution

$R_t = 100\,\mu s.$

The resulting continuous traces are averaged and smoothed to form a canonical blink model

$B(t)=B_{ON}(t)\cup B_{OFF}(t).$

Because the ON/OFF ratio varies with illumination, separate indoor and outdoor blink models are learned (Lenz et al., 2018).

At runtime, the sensor resolution $304\times 240$ is partitioned into overlapping $(x,y)$0 tiles, using one $(x,y)$1 grid plus a second $(x,y)$2 grid shifted by half a tile width and height. Detection uses sparse cross-correlation over the most recent $(x,y)$3:

$(x,y)$4

where

$(x,y)$5

A true blink is declared only when two local candidates satisfy temporal and spatial coherence, with typical parameters $(x,y)$6, $(x,y)$7, and $(x,y)$8. The paper reports that blinks are “usually detected with a ratio of 60%,” which is sufficient because blink detections act as tracker initialization and periodic reset events rather than exhaustive event capture (Lenz et al., 2018).

A related event-camera paper, “Real-Time Face & Eye Tracking and Blink Detection using Event Cameras,” contributes a different data-generation perspective. Its Neuromorphic HELEN dataset is generated from static HELEN face images by applying random 6-DoF homographic camera motion, propagating landmarks, deriving face and eye boxes, and simulating events with ESIM. Positive and negative contrast thresholds are sampled from approximately $(x,y)$9, and the refractory period is $t$0. The paper is explicit that the released synthetic dataset is for face and eye detection rather than blink classification, but its pipeline provides machinery for producing event sequences with accurate spatial annotations (Ryan et al., 2020). This suggests a distinction within event-based blink data generation: one approach learns a temporal blink template from a few real annotated blinks, whereas another builds synthetic event corpora from rendered or transformed imagery and leaves explicit blink animation as a missing component.

3. Automated ROI pseudo-annotation for GUI-agent “Blink”

In BTL-UI, “Blink” denotes a rapid visual attention phase analogous to saccadic eye movements. The full policy is formalized as

$t$1

where $t$2 is the current screenshot, $t$3 the task instruction, $t$4 the interaction history, $t$5 the Blink output containing visual attention regions, $t$6 the Think-stage reasoning trace, and $t$7 the Link-stage executable action. Blink data generation in this setting is therefore the creation of structured ROI labels for attention grounding rather than the generation of eye-blink signals (Zhang et al., 19 Sep 2025).

The pipeline is explicitly two-stage. First, OmniParser extracts a comprehensive screenshot-derived element list

$t$8

with each element represented as

$t$9

Second, Qwen2.5-VL evaluates “visual salience and contextual relevance” conditioned on the instruction $p\in\{\mathrm{ON},\mathrm{OFF}\}$0 and history $p\in\{\mathrm{ON},\mathrm{OFF}\}$1, filters the parsed elements, and produces the final ROI set $p\in\{\mathrm{ON},\mathrm{OFF}\}$2. The output format is XML-like: $(x,y)$04 where captions are one of [dynamic, static], and a null case $(x,y)$05 is allowed when no relevant ROI is present. The system prompt constrains the output to $p\in\{\mathrm{ON},\mathrm{OFF}\}$3 selected elements, and an ablation on the number of ROIs shows AndroidControl-High step success rate rising from $p\in\{\mathrm{ON},\mathrm{OFF}\}$4 at $p\in\{\mathrm{ON},\mathrm{OFF}\}$5 to $p\in\{\mathrm{ON},\mathrm{OFF}\}$6 at $p\in\{\mathrm{ON},\mathrm{OFF}\}$7, with the final choice $p\in\{\mathrm{ON},\mathrm{OFF}\}$8 (Zhang et al., 19 Sep 2025).

The generated annotations are used both as supervised targets and as reinforcement-learning references. In the Blink reward, predicted ROIs $p\in\{\mathrm{ON},\mathrm{OFF}\}$9 are matched against generated blink targets $100\text{--}150\,\mathrm{ms}$0 using a Hungarian matcher $100\text{--}150\,\mathrm{ms}$1 with IoU threshold $100\text{--}150\,\mathrm{ms}$2, though $100\text{--}150\,\mathrm{ms}$3 itself is not specified. The paper reports that, under SFT, adding Blink data improves AndroidControl-High step success rate from $100\text{--}150\,\mathrm{ms}$4 to $100\text{--}150\,\mathrm{ms}$5, and under RFT the full setup with Blink data and BTL reward improves success rate from $100\text{--}150\,\mathrm{ms}$6 to $100\text{--}150\,\mathrm{ms}$7. At the same time, several key internals remain unspecified, including the exact analyzer prompt, the operationalization of “visual salience,” the prioritization function $100\text{--}150\,\mathrm{ms}$8, and post-processing for overlapping or duplicate elements (Zhang et al., 19 Sep 2025).

4. Blink-aware conditioning in face synthesis and augmentation

In “Audio-Driven Talking Face Generation with Blink Embedding and Hash Grid Landmarks Encoding,” blink data generation is not a standalone blink simulator but a process of constructing blink-aware conditioning signals from video. Blink information comes from OpenFace-extracted eye-related action units, blink intensity scores on a 0 to 5 scale stored in CSV files, and six landmarks per eye region extracted from 68-point facial landmarks. These signals are transformed by a mapping network comprising convolutional and fully connected layers, then used by an Eyes Movement prediction module to predict next-frame eye state. The central motion equation is

$100\text{--}150\,\mathrm{ms}$9

where $250\,\mathrm{ms}$0 denotes audio features, $250\,\mathrm{ms}$1 denotes video encoded using blink encoding, and $250\,\mathrm{ms}$2 denotes predicted facial landmarks (Zhang et al., 26 Jan 2026).

The paper is explicit that audio is not a meaningful blink driver and that blinking is handled by a dedicated eye branch. The resulting blink features are injected into the dynamic landmark radiance field together with NeRF rendering conditions. Training is identity-specific, using 3–5 minute videos at 25 fps and $250\,\mathrm{ms}$3 resolution. Although no explicit blink-specific loss is written, the ablation on the eye movement generator shows perceptual benefit: LPIPS improves from $250\,\mathrm{ms}$4 to $250\,\mathrm{ms}$5, and Sync improves from $250\,\mathrm{ms}$6 to $250\,\mathrm{ms}$7; with tri-hash added, LPIPS reaches $250\,\mathrm{ms}$8 and Sync reaches $250\,\mathrm{ms}$9. The paper does not define an explicit formula for the blink embedding or the eye-state prediction loss, so the conditioning pathway is clear while the exact optimization target remains underspecified (Zhang et al., 26 Jan 2026).

A different construction appears in “EyeBAG: Accurate Control of Eye Blink and Gaze Based on Data Augmentation Leveraging Style Mixing.” Here the objective is to create pseudo-paired blink training data from FFHQ random seed images. The authors manually collect 1120 closed-eye face images and 614 uncertain-eye face images, then use StyleGAN2 style mixing with the average latent vector $250\,\mathrm{ms}$0 injected into layers 6 and 7 to create corresponding open-eye versions. These paired images are cropped into $250\,\mathrm{ms}$1 eye-centered views, and left-eye crops are horizontally flipped to match right-eye orientation. Blink state is controlled by a scalar score $250\,\mathrm{ms}$2, with $250\,\mathrm{ms}$3 for closed eye, $250\,\mathrm{ms}$4 for uncertain eye, and $250\,\mathrm{ms}$5 for open-eye reconstruction. The loss is

$250\,\mathrm{ms}$6

$250\,\mathrm{ms}$7

and

$250\,\mathrm{ms}$8

The reported weights are $250\,\mathrm{ms}$9, $R_t = 100\,\mu s.$0, $R_t = 100\,\mu s.$1, and $R_t = 100\,\mu s.$2, with Adam learning rates $R_t = 100\,\mu s.$3 for the generator and $R_t = 100\,\mu s.$4 for the discriminator (Kim et al., 2023).

Taken together, these two papers define two distinct blink-data paradigms for face synthesis. One derives blink-aware conditioning from measured eye movement and landmarks, while the other constructs pseudo-paired supervision by latent-space style mixing. This suggests that blink data generation in generative face pipelines can target either latent control signals or directly editable training pairs, depending on whether the downstream objective is personalized video synthesis or controllable image editing (Zhang et al., 26 Jan 2026, Kim et al., 2023).

5. Physiological and viewer-derived blink representations

“Rethinking Eye-blink” models blinking as a continuous physiological signal rather than a count of discrete events. Each frame of ordinary webcam video is processed with facial landmark detection, six landmarks around each eye are used to compute an eye aspect ratio, and the resulting scalar signal is detrended by subtracting a 1-second moving average:

$R_t = 100\,\mu s.$5

A short-time Gaussian sliding window is then applied, and Lomb–Scargle power spectra are stacked to form a blink spectrogram. The blink band is set to $R_t = 100\,\mu s.$6, corresponding to 2 to 25 blinks per minute, with a 61-second window and 1-second step. For 260-second usable segments, the resulting spectrogram has 200 time units; in Study II, each spectrogram is $R_t = 100\,\mu s.$7. Blink Entropy is defined as

$R_t = 100\,\mu s.$8

where $R_t = 100\,\mu s.$9 is spectrogram amplitude at time $B(t)=B_{ON}(t)\cup B_{OFF}(t).$0, frequency $B(t)=B_{ON}(t)\cup B_{OFF}(t).$1. The paper reports that standard blink rate and blink duration show no significant task effect, whereas Blink Entropy does: $B(t)=B_{ON}(t)\cup B_{OFF}(t).$2 (Cho, 2021).

The downstream learner is a 2D LSTM with one layer, 16 hidden units, a fully connected layer, softmax output, and cross-entropy loss. On person-independent 18-fold leave-one-subject-out evaluation, the spectrogram plus 2D LSTM outperforms hand-engineered feature baselines, achieving $B(t)=B_{ON}(t)\cup B_{OFF}(t).$3 for event-based labels, $B(t)=B_{ON}(t)\cup B_{OFF}(t).$4 for subjective binary labels, and $B(t)=B_{ON}(t)\cup B_{OFF}(t).$5 for objective binary labels. The paper’s data-generation significance lies in its representational hierarchy: video-derived geometry, continuous blink timeseries, and time-frequency blink spectrogram (Cho, 2021).

A different label-construction pipeline appears in “Estimating Blink Probability for Highlight Detection in Figure Skating Videos.” Here blink data are obtained from human viewers measured with a Tobii Pro Spectrum eye tracker at 120 Hz. Blink onset for each participant is extracted by detecting “a combination of rapid increase in pupil diameter followed by a rapid decrease within 0.5 s,” and the target for each video frame is the percentage of viewers blinking at that frame. Video features consist of 3-second windows of OpenPose-derived 18-joint $B(t)=B_{ON}(t)\cup B_{OFF}(t).$6 coordinates, yielding $B(t)=B_{ON}(t)\cup B_{OFF}(t).$7 input arrays; 112,039 samples are created. A three-layer 1D-CNN with kernel size 8 and filter counts 64, 128, and 64 is trained with RMSE to predict the blink rate of the last frame in each window. The paper reports significant positive correlation between estimated and actual blink-rate time series in 45 of 48 clips ($B(t)=B_{ON}(t)\cup B_{OFF}(t).$8) and in $B(t)=B_{ON}(t)\cup B_{OFF}(t).$9 of jump events (Nakano et al., 2020).

These two lines illustrate a basic split within physiological blink data generation. One constructs subject-centered continuous signals whose discriminative content lies in temporal organization and entropy. The other constructs population-level framewise blink-probability labels from synchronized viewer measurements. This suggests that blink data can be generated either as an intrinsic behavioral trace or as an externally aggregated attention proxy, depending on whether the downstream task is workload estimation or event salience modeling (Cho, 2021, Nakano et al., 2020).

6. Datasets, annotation regimes, and recurrent difficulties

Large blink datasets increasingly combine frame labels, sequence labels, multiple sensors, and auxiliary signals. The mEBAL2 database contains 21,100 events from 180 students or sessions, captured with one RGB and two NIR cameras at 30 Hz and $304\times 240$0, plus NeuroSky EEG information. Each event contains 19 frames, leading to

$304\times 240$1

The dataset provides 21,000 open-eye frames and 21,000 closed-eye frames at frame level, plus 10,500 blinks and 10,500 no-blinks at sequence level. Blink candidates are first proposed automatically from EEG blink information and then manually checked. The benchmark uses leave-one-out cross-validation, and the strongest sequence model, OE-ConvLSTM, reaches $304\times 240$2 accuracy on sequence-level blink detection, while full-spectrum training with RGB and NIR improves RGB-only inference from $304\times 240$3 to $304\times 240$4 accuracy (Daza et al., 2023).

Species-specific blink annotation introduces further granularity. In “Horse Eye Blink Detection and Classification for Equine Affective State Assessment,” the classes are none, half-blink, and full-blink. Half-blink is assigned “when the eye was at least halfway closed,” while full-blink requires “the eye to be fully closed with no eyeball visible.” The dataset uses videos of 6 horses from different breeds recorded at $304\times 240$5, with a public subset of 12 videos for testing. For YOLOv12 frame-based training, 13,206 manually annotated frames are used, with $304\times 240$6 open, $304\times 240$7 half-blink, and $304\times 240$8 full-blink. The paper reports macro-F1 $304\times 240$9 for binary blink detection and $(x,y)$00 for three-class blink classification at video level, but half-blink remains the hardest class: human agreement is $(x,y)$01 with Krippendorff’s $(x,y)$02, and frame-level YOLO half-blink F1 is $(x,y)$03 (Alves et al., 3 Jun 2026).

Across these datasets, recurrent difficulties are consistent. Blink classes are often imbalanced, event boundaries are uncertain, subtle partial closures are hard to distinguish from non-blink eye tension, and temporal smoothing or clip-level modeling materially changes performance. The event-based literature adds related ambiguities: the exact neighborhood thresholds of noise filters, the correlation threshold, and ON/OFF weighting constants are sometimes left unspecified, even when the main blink-modeling recipe is compact and reproducible (Lenz et al., 2018). A plausible implication is that blink data generation is not limited by the availability of raw video or event streams; it is equally constrained by annotation granularity, class definitions, synchronization fidelity, and the treatment of ambiguous intermediate states.