REINA: Regularized Entropy Information Adaptation

• The paper introduces a novel framework that trains a Read/Write controller using information gain from audio inputs to optimize token emission timing.
• It leverages conditional entropy differences and surrogate log-probabilities to decide whether to read more input or write the next token.
• Extensions like REINA-SAN and REINA-TAN add temporal grounding, reducing read loops and improving the streaming latency–quality trade-off.

Searching arXiv for REINA and closely related papers to ground the article.

Regularized Entropy INformation Adaptation (REINA) is an information-theoretic policy-learning framework for simultaneous speech translation (SimulST) built around a simple operational principle: wait for more input only if doing so yields information about the next output token. In its named formulation, REINA trains a Read/Write controller on top of a non-streaming speech-to-text translation model by estimating the information gain obtained from additional audio and regularizing the resulting policy scores with monotonicity and norm penalties. The method is designed to shift the latency–quality Pareto frontier, and later extensions, notably REINA-SAN and REINA-TAN, retain the same information-based core while adding temporal grounding through supervised alignment or explicit timestep encoding (Hirschkind et al., 7 Aug 2025, Liu et al., 10 Apr 2026).

1. Information-theoretic formulation

REINA is defined in the SimulST setting, where source speech arrives incrementally as acoustic frames $x_{1:T}$ and the system emits target tokens $y_{1:N}$. At each decision point, the controller chooses between waiting for more source audio and emitting the next target token. The timing of token emission is represented by $g(i)$, the number of source frames or chunks read before emitting the $i$-th target token; in time-based reporting, $\tau(i)=g(i)\cdot \Delta$, where $\Delta$ is the chunk duration (Hirschkind et al., 7 Aug 2025).

Its central quantity is an information-gain score for the next ground-truth token. With $a_t$ denoting the audio prefix available at decision time, $a_T$ the full audio, and $S_n$ the emitted target prefix, REINA defines

$F(a,S,n,t)=I(S_{n+1};a_T,S_n)-I(S_{n+1};a_t,S_n).$

Using conditional entropy and log-likelihood identities, this is operationalized by the surrogate

$y_{1:N}$0

Large positive values indicate that future audio materially improves confidence in the next token, so waiting is justified; small values indicate that writing immediately is preferable (Hirschkind et al., 7 Aug 2025).

The conceptual decision rule is thresholded: $y_{1:N}$1 Because $y_{1:N}$2 depends on full audio and the ground-truth future token, it is unavailable at inference time. REINA therefore trains a lightweight policy network $y_{1:N}$3 whose scalar output correlates with the information-gain signal and can be thresholded online (Hirschkind et al., 7 Aug 2025).

A frequent misconception is that REINA is an RL reward-shaping method in the usual sense. In the named SimulST formulation, it is instead a supervised policy-learning procedure driven by label-conditioned log-probability differences from an offline teacher model, rather than direct reward optimization or explicit exploration bonuses (Hirschkind et al., 7 Aug 2025).

2. Loss construction and policy training

The original REINA loss is built from a covariance-style objective. Let $y_{1:N}$4, where $y_{1:N}$5 and $y_{1:N}$6. After batch normalization, REINA minimizes

$y_{1:N}$7

This objective makes $y_{1:N}$8 large when waiting is informative and small when it is not (Hirschkind et al., 7 Aug 2025).

Two auxiliary regularizers stabilize the policy. The monotonicity term

$y_{1:N}$9

penalizes score drops larger than a margin, with $g(i)$0 in the reported setup. The norm term

$g(i)$1

prevents score explosion. The full objective is

$g(i)$2

with $g(i)$3 in the original paper (Hirschkind et al., 7 Aug 2025).

The training pipeline is three-stage. First, a non-streaming S2TT teacher is trained with joint ASR, NMT, and S2TT losses. Second, the same model is fine-tuned on randomly truncated audio; this step is described as crucial because REINA depends on calibrated partial-audio next-token probabilities. Third, the teacher is frozen and only the policy network is trained with $g(i)$4 (Hirschkind et al., 7 Aug 2025).

The later Whisper-based study preserves the covariance-maximization principle but emphasizes that the policy is trained to preserve the ordinal ranking of information-gain values rather than regress their absolute magnitudes. It also states explicitly that the paper uses L2 and monotonicity constraints but does not add explicit policy entropy regularization (Liu et al., 10 Apr 2026).

3. Architectures and inference mechanisms

Two concrete instantiations appear in the supplied literature.

System	Backbone	Policy head
REINA	Whisper Medium acoustic encoder + Transformer decoder + T5 MT encoder at training	2-layer Transformer encoder, $g(i)$5, 4 heads
Whisper-based REINA	Frozen Whisper Large V3	3-layer Transformer module, hidden dim 1280, feedforward dim 7680

In the original REINA system, the acoustic encoder is Whisper Medium, the text decoder is a 16-layer Transformer decoder with $g(i)$6, 8 heads, and FFN$g(i)$7, and an auxiliary T5 encoder is used only during training for an MT objective. The total trainable parameter count is reported as approximately $g(i)$8M at training and $g(i)$9M at inference. The policy network is small, about $i$0M parameters, and is trained after the S2TT model is frozen (Hirschkind et al., 7 Aug 2025).

Inference is threshold-based. Given current partial input and hypothesis prefix, REINA computes a score $i$1 and applies

$i$2

The reported implementation uses streaming beam search with beam size $i$3, patience $i$4, and audio chunk size $i$5 s. At end of audio, the controller is disabled and ordinary beam search continues to EOS (Hirschkind et al., 7 Aug 2025).

The Whisper-based implementation keeps the backbone frozen and feeds decoder hidden states into the policy module. It also uses streaming beam search with beam size $i$6 and chunk stride $i$7 ms, but its threshold notation is $i$8, with

$i$9

This version is explicitly positioned as a lightweight policy head controlling scheduling without changing backbone parameters (Liu et al., 10 Apr 2026).

4. Temporal-awareness variants

The principal limitation identified in later work is that information-based policies often lack temporal context. In practice, this produces two characteristic failure modes: bias toward reading too much before writing and “read loops,” in which the controller repeatedly chooses Read until the utterance ends (Liu et al., 10 Apr 2026).

Variant	Added mechanism	Reported effect
REINA-SAN	Supervised alignment network	More robustness against read loops
REINA-TAN	Timestep-augmented network	Slightly superior Pareto frontier for streaming efficiency
REINA-ALL	SAN + TAN	Underperforms the individual variants

REINA-SAN adds weak monotonic supervision from alignments. WhisperX supplies audio–text force alignment, Qwen3-32B produces monotonic text chunk alignments, and each target token receives an ideal emission time $\tau(i)=g(i)\cdot \Delta$0. The target emission probability is

$\tau(i)=g(i)\cdot \Delta$1

and the alignment loss is a BCE term added to the REINA objective: $\tau(i)=g(i)\cdot \Delta$2 with $\tau(i)=g(i)\cdot \Delta$3 in the reported experiments (Liu et al., 10 Apr 2026).

REINA-TAN instead injects an explicit clock signal. With elapsed audio time $\tau(i)=g(i)\cdot \Delta$4, it forms a sinusoidal time embedding

$\tau(i)=g(i)\cdot \Delta$5

and adds it to decoder features: $\tau(i)=g(i)\cdot \Delta$6 This provides a continuous temporal signal and increases write propensity as time grows (Liu et al., 10 Apr 2026).

An important empirical point is that REINA-ALL, which combines SAN and TAN, underperforms the individual variants. The paper attributes this to conflicting inductive biases between alignment-forced emission schedules and time-conditioned policy dynamics (Liu et al., 10 Apr 2026).

5. Evaluation metrics and empirical behavior

REINA is evaluated with latency, quality, and streaming-efficiency metrics. The original work reports BLEU, AL, and LAAL; the later extension also reports XComet-XL and read-loop percentage. The original paper introduces Normalized Streaming Efficiency (NoSE),

$\tau(i)=g(i)\cdot \Delta$7

which normalizes the area under the latency–BLEU curve by the non-streaming BLEU baseline. Values closer to $\tau(i)=g(i)\cdot \Delta$8 indicate that streaming quality stays close to offline quality across the latency interval (Hirschkind et al., 7 Aug 2025).

In the original paper, REINA is reported to improve the latency/quality trade-off by as much as $\tau(i)=g(i)\cdot \Delta$9 compared to prior approaches, normalized against non-streaming baseline BLEU scores. On MUST-C, reported NoSE values include $\Delta$0 for en$\Delta$1de, $\Delta$2 for en$\Delta$3fr, and $\Delta$4 for en$\Delta$5es in the MUST-C-only setting; the same section reports REINA surpassing DiG-SST and DiSeg on the stated bounds and pushing the Pareto frontier especially at low latencies. The paper also states that training on only open source or synthetically generated data yields state-of-the-art streaming results for models of comparable size (Hirschkind et al., 7 Aug 2025).

The Whisper-based extension reports that both REINA-TAN and REINA-SAN significantly outperform the baseline and resolve stability issues. Example FLEURS operating points include De$\Delta$6En at LAAL $\Delta$7 s with BLEU $\Delta$8 for REINA-TAN versus LAAL $\Delta$9 s and BLEU $a_t$0 for baseline REINA, and EuroparlST De$a_t$1En at LAAL $a_t$2 s with BLEU $a_t$3 versus $a_t$4 s and $a_t$5. The paper states that both methods improve the Pareto frontier of streaming efficiency as measured by NoSE scores up to $a_t$6 over existing competitive baselines. It also quantifies loop reduction: at FLEURS around $a_t$7 BLEU, REINA-TAN has $a_t$8 read loops versus REINA $a_t$9, while REINA-SAN/ALL show none in that regime (Liu et al., 10 Apr 2026).

These results clarify another common misunderstanding: the monotonicity component is not primarily a global quality booster. The original ablations indicate that its main effect is stronger low-latency behavior, while the later work shows that explicit temporal grounding is what most directly addresses conservative reading and read-loop pathologies (Hirschkind et al., 7 Aug 2025, Liu et al., 10 Apr 2026).

6. Broader theoretical lineage

The integrated syntheses accompanying REINA place it within a broader family of entropy- and information-regularized learning procedures. In entropy-regularized MDP theory, conditional entropy regularization yields soft Bellman equations of the form

$a_T$0

and optimal policies satisfy $a_T$1. The same framework is used to formalize TRPO, mirror descent, and dual averaging in entropy-regularized average-reward RL, and the exact TRPO variant is stated to converge to the optimal policy for the entropy-regularized problem (Neu et al., 2017).

A closer information-theoretic analogue appears in mutual-information-regularized RL. There, optimizing a state-independent reference marginal $a_T$2 turns a KL penalty into

$a_T$3

with actor-critic updates based on $a_T$4. The resulting MIRACLE algorithm learns $a_T$5 rather than fixing it uniformly, and the optimal policy takes the form $a_T$6 (Leibfried et al., 2019).

Other supplied syntheses extend the same entropy/information pattern beyond sequential decision-making. In neural image compression, the regularizer

$a_T$7

uses $a_T$8 as a structural regularizer for rate–distortion training and is reported to impose no inference overhead (Zhang et al., 2024). In convexified IB optimization, REINA denotes

$a_T$9

paired with symbolic continuation to stabilize the solution path across $S_n$0 (Alpay, 14 May 2025). In a separate RL formulation, the advanced policy

$S_n$1

defines a continuous path from policy gradient to soft Q-learning under KL regularization (Lee, 2020). Local entropic smoothening provides yet another adaptation, with a smoothed loss

$S_n$2

used as an entropic alternative to initialization (Musso, 2021).

This suggests a broader reading of REINA as a recurring design pattern rather than a single domain-specific implementation: entropy or information terms are not treated merely as static penalties, but as adaptive control signals, learned marginals, or continuation variables that shape optimization and inference.

7. Limitations and open problems

REINA depends on teacher calibration. The original SimulST formulation assumes that the offline S2TT model produces reliable full- versus partial-audio next-token log-probabilities, and the truncated-audio fine-tuning stage is reported to be essential; skipping it substantially degrades NoSE in the ablations (Hirschkind et al., 7 Aug 2025).

Threshold selection remains external to the training objective. Both REINA and its later variants generate operating points by sweeping a decision threshold, so deployment still requires empirical tuning to meet latency targets. NoSE itself also depends on chosen latency bounds, and the later paper notes that conclusions can vary with those choices (Hirschkind et al., 7 Aug 2025, Liu et al., 10 Apr 2026).

The temporal-awareness extensions introduce their own assumptions. REINA-SAN relies on WhisperX and LLM-derived chunk alignments; misalignments can teach suboptimal emission schedules. REINA-TAN reduces read loops and improves the Pareto frontier, but the broader multilingual evidence reported so far is restricted to fr/de/es$S_n$3en benchmarks. The Whisper-based study also notes computational overhead from computing both partial and full-context log-probabilities during training and from streaming state management (Liu et al., 10 Apr 2026).

Several directions are explicitly identified for future work: improved information-gain estimators based on mutual-information bounds or uncertainty-aware surrogates, on-device pruning or distillation of the policy head, broader multilingual generalization, hybridization with wait-k, monotonic attention, or divergence-guided policies, more robust temporal encodings, and dynamic threshold schedules or confidence calibration to avoid early error cascades (Liu et al., 10 Apr 2026). In the broader syntheses, analogous open problems include automatic adaptation of $S_n$4 or $S_n$5, richer priors or state distributions in mutual-information RL, and scalable Hessian-based continuation for large IB models (Leibfried et al., 2019, Alpay, 14 May 2025).