Edit-Based Flow Matching (EdiTPP) Overview

• Edit-Based Flow Matching (EdiTPP) is a generative modeling framework that employs local edit operations (insertion, deletion, substitution) in a continuous-time Markov chain for sequence synthesis.
• It leverages a transformer-based parameterization and a flow-matching objective derived from Bregman divergence to reduce edit counts and computational runtime compared to diffusion models.
• EdiTPP demonstrates state-of-the-art performance in both unconditional and conditional (forecasting) temporal point process synthesis across synthetic and real-world benchmarks.

Edit-Based Flow Matching (EdiTPP) is a non-autoregressive generative modeling framework for discrete sequences—most prominently, temporal point processes (TPPs)—characterized by the use of local, flexible edit operations (insertion, deletion, substitution) embedded within a continuous-time Markov chain (CTMC) over the combinatorial state space. EdiTPP enables parallel, globally coherent, and variable-length sequence synthesis, achieving significant reductions in the number of edit operations and computational runtime compared to prior diffusion-style models. The method's foundation is a flow-matching objective rooted in Bregman divergence, with dynamics parameterized by transformer architectures and analytically tractable reference processes. EdiTPP attains state-of-the-art empirical performance in both unconditional and conditional (forecasting) sequence generation tasks across synthetic and real-world benchmarks (Lüdke et al., 7 Oct 2025, Havasi et al., 10 Jun 2025).

1. Problem Formulation and Prior Approaches

EdiTPP addresses the challenge of modeling TPPs—random finite, ordered sets of events $x=\{t^{(1)}<\cdots<t^{(n)}\}\subset[0,T]$—without the limitations of autoregressive or fixed-schedule generation. Classical neural TPPs parameterize the joint density sequentially:

$$p(x)=\prod_{i=1}^n p\bigl(t^{(i)}\mid t^{(1:i-1)}\bigr)\cdot\exp\Bigl(-\int_0^T\lambda^*(s)\,ds\Bigr),$$

with sampling cost $O(n)$ and error compounding. Early non-autoregressive models, such as AddThin and PSDiff, employ Markov chains that iteratively delete events and superimpose Poisson noise, but are restricted to insertion and deletion operations with rates set by fixed schedules. These approaches lack efficient mechanisms for local sequence modifications and suffer from high edit counts and computational inefficiency (Lüdke et al., 7 Oct 2025).

2. Continuous-Time Edit Flow Process and State Space

EdiTPP generalizes prior diffusion models by expanding the atomic operations to include substitution, alongside insertion and deletion, over a state space

$\Xcal=\bigcup_{n\ge0} \{(0, t^{(1)}, \ldots, t^{(n)}, T) : 0<t^{(1)}<\cdots<t^{(n)}<T\},$

where sequences are padded by boundary events $0$ and $T$. The atomic edits are:

• Insertion: $\ins(x,i,j)$ places a new event, spatially discretized into $B_{\ins}$ bins and randomly dequantized.
• Deletion: $\del(x,i)$ removes the $i$-th non-boundary event.
• Substitution: $$p(x)=\prod_{i=1}^n p\bigl(t^{(i)}\mid t^{(1:i-1)}\bigr)\cdot\exp\Bigl(-\int_0^T\lambda^*(s)\,ds\Bigr),$$0 moves event $$p(x)=\prod_{i=1}^n p\bigl(t^{(i)}\mid t^{(1:i-1)}\bigr)\cdot\exp\Bigl(-\int_0^T\lambda^*(s)\,ds\Bigr),$$1 within a window of width $$p(x)=\prod_{i=1}^n p\bigl(t^{(i)}\mid t^{(1:i-1)}\bigr)\cdot\exp\Bigl(-\int_0^T\lambda^*(s)\,ds\Bigr),$$2 across $$p(x)=\prod_{i=1}^n p\bigl(t^{(i)}\mid t^{(1:i-1)}\bigr)\cdot\exp\Bigl(-\int_0^T\lambda^*(s)\,ds\Bigr),$$3 bins, followed by a re-sort.

The CTMC over this space is governed by position-specific rates $$p(x)=\prod_{i=1}^n p\bigl(t^{(i)}\mid t^{(1:i-1)}\bigr)\cdot\exp\Bigl(-\int_0^T\lambda^*(s)\,ds\Bigr),$$4, $$p(x)=\prod_{i=1}^n p\bigl(t^{(i)}\mid t^{(1:i-1)}\bigr)\cdot\exp\Bigl(-\int_0^T\lambda^*(s)\,ds\Bigr),$$5, $$p(x)=\prod_{i=1}^n p\bigl(t^{(i)}\mid t^{(1:i-1)}\bigr)\cdot\exp\Bigl(-\int_0^T\lambda^*(s)\,ds\Bigr),$$6, and corresponding categorical distributions for bin selection ($$p(x)=\prod_{i=1}^n p\bigl(t^{(i)}\mid t^{(1:i-1)}\bigr)\cdot\exp\Bigl(-\int_0^T\lambda^*(s)\,ds\Bigr),$$7, $$p(x)=\prod_{i=1}^n p\bigl(t^{(i)}\mid t^{(1:i-1)}\bigr)\cdot\exp\Bigl(-\int_0^T\lambda^*(s)\,ds\Bigr),$$8). The master equation describes evolution in flow-time $$p(x)=\prod_{i=1}^n p\bigl(t^{(i)}\mid t^{(1:i-1)}\bigr)\cdot\exp\Bigl(-\int_0^T\lambda^*(s)\,ds\Bigr),$$9:

$O(n)$0

where $O(n)$1 is nonzero only for a single edit difference (Lüdke et al., 7 Oct 2025, Havasi et al., 10 Jun 2025).

3. Flow-Matching Objective and Auxiliary Alignment

EdiTPP’s training objective uses a flow-matching loss, derived from Bregman divergence, to match the model generator $O(n)$2 to a reference process defined via explicit sequence alignments. For each data pair $O(n)$3, an alignment $O(n)$4 (e.g., Needleman–Wunsch) encodes the minimal set of edits as aligned mixture paths. The reference edit rates during the morph from $O(n)$5 to $O(n)$6 are known analytically, enabling an explicit loss:

$O(n)$7

where $O(n)$8. This framework supports both unconditional and conditional (forecasting) generation via simple sequence masking (Lüdke et al., 7 Oct 2025, Havasi et al., 10 Jun 2025).

4. Neural Parameterization and Inference Algorithms

The edit rates $O(n)$9 are parameterized by a transformer network (Llama-style), processing event position embeddings, sinusoidal encodings of both flow time $\Xcal=\bigcup_{n\ge0} \{(0, t^{(1)}, \ldots, t^{(n)}, T) : 0<t^{(1)}<\cdots<t^{(n)}<T\},$0 and sequence length $\Xcal=\bigcup_{n\ge0} \{(0, t^{(1)}, \ldots, t^{(n)}, T) : 0<t^{(1)}<\cdots<t^{(n)}<T\},$1, and event time embeddings via MLP($\Xcal=\bigcup_{n\ge0} \{(0, t^{(1)}, \ldots, t^{(n)}, T) : 0<t^{(1)}<\cdots<t^{(n)}<T\},$2). From the transformer’s hidden states, rates and categorical distributions for each edit type are computed:

$\Xcal=\bigcup_{n\ge0} \{(0, t^{(1)}, \ldots, t^{(n)}, T) : 0<t^{(1)}<\cdots<t^{(n)}<T\},$3

with analogous expressions for deletion and substitution. Hyperparameters in benchmark experiments include $\Xcal=\bigcup_{n\ge0} \{(0, t^{(1)}, \ldots, t^{(n)}, T) : 0<t^{(1)}<\cdots<t^{(n)}<T\},$4, 2 transformer layers, 4 heads, $\Xcal=\bigcup_{n\ge0} \{(0, t^{(1)}, \ldots, t^{(n)}, T) : 0<t^{(1)}<\cdots<t^{(n)}<T\},$5, and $\Xcal=\bigcup_{n\ge0} \{(0, t^{(1)}, \ldots, t^{(n)}, T) : 0<t^{(1)}<\cdots<t^{(n)}<T\},$6 (Lüdke et al., 7 Oct 2025).

Sampling and inference leverage an Euler–Maruyama integration of the CTMC: all edit proposals across positions are computed in parallel per step, with edits realized according to their rates. For conditional forecasting, a mask conditions the beginning of the sequence, fixing initial events along the mixture path and editing the remainder freely (Lüdke et al., 7 Oct 2025).

5. Empirical Evaluation and Baseline Comparisons

EdiTPP has been evaluated extensively on six synthetic benchmarks (e.g., Hawkes processes, Poisson, Renewal, Self-Correcting) and seven real-world TPP datasets (PUBG, Reddit, NYC Taxi, Twitter, Yelp). Metrics include MMD over event times, Wasserstein distances on event counts and inter-event times for unconditional generation, and Wasserstein-2/MRE/first-event-error for conditional (forecasting) setups.

EdiTPP outperforms baselines such as IFTPP (autoregressive), AddThin, and PSDiff:

• Edit efficiency: ≈193 edits/sample on average, vs. ≈233 for PSDiff (17% reduction), with substitutions enabling local moves replacing single delete+insert pairs.
• Runtime (batch 1024, NVIDIA H100, R/C dataset): EdiTPP ≈1.5 ms; PSDiff ≈3.9 ms; AddThin ≈17.7 ms.
• Statistical performance: Best or equal best on almost all relevant metrics across unconditional and conditional settings (Lüdke et al., 7 Oct 2025).

The method also demonstrates flexible runtime/quality trade-offs by varying the number of sampler steps $\Xcal=\bigcup_{n\ge0} \{(0, t^{(1)}, \ldots, t^{(n)}, T) : 0<t^{(1)}<\cdots<t^{(n)}<T\},$7: more steps improve quality with diminishing returns.

6. Theoretical and Practical Significance

Incorporating substitution into edit-based flows increases modeling flexibility, providing direct local sequence modifications (e.g., event relocation) absent in insert+delete-only models. The CTMC flow-matching paradigm learns continuous-time edit rates using Bregman divergence without extensive marginalization, further reducing computational overhead. EdiTPP's unconditional training regime supports both unconditional and conditional tasks, requiring only adjustments to input masking rather than separate objectives or tuning (Lüdke et al., 7 Oct 2025).

Extensions to the framework include richer conditioning (e.g., marked point processes), learned bin widths, and the use of deterministic reverse-time ODE solvers for sampling.

7. Relationship to Prior and Related Work

EdiTPP generalizes the Edit Flow framework introduced for generic discrete sequences (Havasi et al., 10 Jun 2025) to the temporal domain, maintaining the principles of CTMC-based transition modeling, auxiliary alignment processes, and flow-matching objectives. The core ideas are applicable beyond TPPs to variable-length discrete data such as text and program code, as demonstrated in other domains. The alignment process, involving auxiliary sequence pairs with blanks and element-wise mixture paths, enables tractable handling of variable-length outputs without explicit length models. Empirical studies validate that alignment quality is not critical, as models implicitly learn to prefer minimal-edit couplings through loss minimization (Havasi et al., 10 Jun 2025).

Table: Comparison of Key EdiTPP Components Versus Related Diffusion-Style Models

Model	Atomic Edits	CTMC Parametrization	Empirical Edit Count per Sample
AddThin	Insert, Delete	Fixed schedule	High
PSDiff	Insert, Delete	Fixed schedule	≈233
EdiTPP	Insert, Delete, Sub	Learned via Transformer	≈193

EdiTPP's inclusion of substitution and transformer-based continuous-time rates yields both edit efficiency and flexibility improvements over prior work (Lüdke et al., 7 Oct 2025).