HawkesNest Benchmark: Synthetic STPP Evaluation

• HawkesNest is a synthetic benchmarking framework for spatiotemporal point processes that provides controlled, adjustable latent structures to diagnose model performance.
• The framework varies four complexity axes—space–time entanglement, background heterogeneity, cross-type interaction, and domain topology—while keeping global simulation parameters fixed.
• Empirical results reveal that likelihood improvements can mask failures in capturing latent structure, emphasizing the need for diagnostic evaluations beyond aggregate scores.

Searching arXiv for HawkesNest and closely related benchmark papers. I’ll look up the dedicated HawkesNest benchmark paper and its Seahorse companion on arXiv. HawkesNest is a synthetic benchmarking framework for spatiotemporal point process (STPP) models built on a controlled multivariate Hawkes-process simulator. Its defining feature is that the latent generative structure is known, adjustable, and interpretable, so model behavior can be examined under controlled changes in structural difficulty rather than only on opaque real-world corpora. The standalone benchmark defines four complexity axes—space–time entanglement, background heterogeneity, cross-type interaction, and domain topology—and varies them while holding global rate, stability, and simulation budget fixed. In the related SEAHORSE framework, HawkesNest serves as the synthetic stress-test suite used for diagnostic benchmarking under known ground-truth intensity (Aalaila et al., 15 Jun 2026, Aalaila et al., 1 Jul 2026).

1. Diagnostic rationale and benchmark philosophy

HawkesNest was introduced to address a specific limitation of standard STPP evaluation: real datasets such as crime, earthquakes, traffic, or epidemics are valuable for external validity, but they are poor for diagnosis because their latent mechanisms are unknown. In that setting, improvements in likelihood or forecasting can reflect dataset-specific artifacts rather than recovery of genuine spatiotemporal structure, and it is difficult to distinguish failures caused by inductive-bias mismatch, limited capacity, optimization pathologies, or intrinsic data complexity. HawkesNest replaces this ambiguity with a generator-aligned design in which complexity is specified in the simulator rather than inferred after the fact (Aalaila et al., 15 Jun 2026).

This generator-aligned design is intended to support mechanism-level attribution. When a model degrades, the failure can be associated with a specific structural assumption: separable versus non-separable space–time triggering, homogeneous versus heterogeneous background rate, type independence versus cross-type excitation, or Euclidean versus constrained domain geometry. Within SEAHORSE, this role is made explicit: HawkesNest is described as a controlled synthetic event suite with known ground-truth intensity, used to probe inductive bias rather than only leaderboard performance (Aalaila et al., 1 Jul 2026).

A recurrent misconception is that HawkesNest is meant to replace real-world benchmarks. The benchmark paper states the opposite in substance: synthetic control is used for diagnostic validity, whereas real datasets remain necessary for real-world relevance. The complexity coordinates are therefore not presented as universal scores of “real-world complexity,” but as controlled axes within a fixed generative family (Aalaila et al., 15 Jun 2026).

2. Hawkes-process backbone and simulation model

At its core, HawkesNest uses a spatiotemporal Hawkes process. For a history $\mathcal H_t$, the conditional intensity is written as

$$\lambda(\mathbf{s},t \mid \mathcal H_t) = \mu(\mathbf{s},t) + \sum_{j:t_j<t} \phi(\mathbf{s}-\mathbf{s}_j,\; t-t_j),$$

where $\mu(\mathbf{s},t)$ is the exogenous background rate and $\phi$ is the endogenous triggering kernel. In the multitype setting, each type $m$ has its own intensity $\lambda_m(\mathbf s,t)$, and past events of type $n$ contribute through cross-triggering kernels $\phi_{mn}$. The branching structure is summarized by a branching matrix $A$, with stability enforced by $\rho(A)<1$ through conservative rescaling (Aalaila et al., 15 Jun 2026).

The simulator uses Ogata-style thinning with a constant envelope $$\lambda(\mathbf{s},t \mid \mathcal H_t) = \mu(\mathbf{s},t) + \sum_{j:t_j<t} \phi(\mathbf{s}-\mathbf{s}_j,\; t-t_j),$$0: interarrival proposals are drawn from $$\lambda(\mathbf{s},t \mid \mathcal H_t) = \mu(\mathbf{s},t) + \sum_{j:t_j<t} \phi(\mathbf{s}-\mathbf{s}_j,\; t-t_j),$$1, a candidate mark and location are sampled, and the proposal is accepted with probability proportional to the local intensity. This makes the generator modular and permits controlled variation of latent structure without altering the overall simulation stack (Aalaila et al., 15 Jun 2026).

The SEAHORSE companion paper provides a simpler Hawkes form as contextual reference,

$$\lambda(\mathbf{s},t \mid \mathcal H_t) = \mu(\mathbf{s},t) + \sum_{j:t_j<t} \phi(\mathbf{s}-\mathbf{s}_j,\; t-t_j),$$2

with $$\lambda(\mathbf{s},t \mid \mathcal H_t) = \mu(\mathbf{s},t) + \sum_{j:t_j<t} \phi(\mathbf{s}-\mathbf{s}_j,\; t-t_j),$$3 as background rate, $$\lambda(\mathbf{s},t \mid \mathcal H_t) = \mu(\mathbf{s},t) + \sum_{j:t_j<t} \phi(\mathbf{s}-\mathbf{s}_j,\; t-t_j),$$4 as excitation strength, $$\lambda(\mathbf{s},t \mid \mathcal H_t) = \mu(\mathbf{s},t) + \sum_{j:t_j<t} \phi(\mathbf{s}-\mathbf{s}_j,\; t-t_j),$$5 as temporal decay, $$\lambda(\mathbf{s},t \mid \mathcal H_t) = \mu(\mathbf{s},t) + \sum_{j:t_j<t} \phi(\mathbf{s}-\mathbf{s}_j,\; t-t_j),$$6 as spatial interaction scale, and $$\lambda(\mathbf{s},t \mid \mathcal H_t) = \mu(\mathbf{s},t) + \sum_{j:t_j<t} \phi(\mathbf{s}-\mathbf{s}_j,\; t-t_j),$$7 as spatial kernel. This contextualization matters because HawkesNest’s synthetic difficulty is grounded in known self- and cross-excitation structure rather than in arbitrary procedural data generation (Aalaila et al., 1 Jul 2026).

3. Complexity axes and deterministic indices

HawkesNest defines four structural axes of STPP complexity. Each axis corresponds to a component of the latent Hawkes data-generating process and is associated with a deterministic index in $$\lambda(\mathbf{s},t \mid \mathcal H_t) = \mu(\mathbf{s},t) + \sum_{j:t_j<t} \phi(\mathbf{s}-\mathbf{s}_j,\; t-t_j),$$8 that increases with the intended difficulty (Aalaila et al., 15 Jun 2026).

Axis	Latent modification	Index summary
Space–time entanglement	Deforms a separable kernel into a non-separable one	Normalized Gaussian-reference mutual information
Background heterogeneity	Replaces homogeneous background with structured field of unit mean	Variance of normalized background intensity
Cross-type interaction	Strengthens and structures off-diagonal excitation in $$\lambda(\mathbf{s},t \mid \mathcal H_t) = \mu(\mathbf{s},t) + \sum_{j:t_j<t} \phi(\mathbf{s}-\mathbf{s}_j,\; t-t_j),$$9	Based on matrix norm and modularity
Domain topology	Replaces flat Euclidean accessibility with graph-constrained accessibility	Euclidean vs graph-geodesic distance gap

The entanglement axis starts from a separable baseline

$\mu(\mathbf{s},t)$0

and applies a non-separable deformation

$\mu(\mathbf{s},t)$1

where $\mu(\mathbf{s},t)$2 controls coupling strength and $\mu(\mathbf{s},t)$3 preserves total kernel mass. The associated index is tied to a Gaussian reference model: $\mu(\mathbf{s},t)$4 The key construction is that branching mass and temporal scale remain fixed while the joint space–time shape becomes increasingly non-separable (Aalaila et al., 15 Jun 2026).

For background heterogeneity, the simulator defines

$\mu(\mathbf{s},t)$5

with $\mu(\mathbf{s},t)$6 normalized to unit mean, so $\mu(\mathbf{s},t)$7. The corresponding index is

$\mu(\mathbf{s},t)$8

Thus heterogeneity changes where background events occur, not their expected aggregate count (Aalaila et al., 15 Jun 2026).

Cross-type interaction is implemented through

$\mu(\mathbf{s},t)$9

where $\phi$0 is shared self-excitation and $\phi$1 is a block-modular template with stronger within-block than between-block interaction. Off-diagonal terms encode cross-excitation and directed cascades, and the matrix is globally rescaled to satisfy stability. Domain topology, by contrast, modifies spatial accessibility using a random geometric graph over the unit square with connection radius

$\phi$2

after which the largest connected component is retained. As $\phi$3 increases, graph distances lengthen relative to Euclidean distances and accessibility becomes more constrained (Aalaila et al., 15 Jun 2026).

Two structural validation properties are central. First, monotonicity: in one-dimensional sweeps, each intended index increases monotonically with its own control parameter. Second, near-orthogonality: in a full $\phi$4 factorial sweep with five replicates per cell, totaling $\phi$5 runs, variance decomposition shows that main effects explain at least $\phi$6 of the variance for each intended index, while higher-order interactions are negligible or near zero (Aalaila et al., 15 Jun 2026).

4. Benchmark construction, suites, and reporting semantics

HawkesNest is implemented in two usage modes: a custom DGP mode, in which users specify domain, background, kernel, marks or adjacency, rate or stability controls, and seeds directly; and a recipe mode, in which validated presets are invoked. The released suites include an EntanglementSuite producing levels $\phi$7–$\phi$8 and a HeterogeneitySuite producing levels $\phi$9–$m$0 (Aalaila et al., 15 Jun 2026).

The main benchmark regimes reported in the paper are threefold. The first is a joint heterogeneity–entanglement sweep $m$1–$m$2, using

$m$3

with topology and interaction held fixed at $m$4 and $m$5. The second is an isolated entanglement sweep $m$6–$m$7. The third is the full four-dimensional validation sweep. Appendix settings for the full validation sweep include $m$8, $m$9, $\lambda_m(\mathbf s,t)$0, a moving-Gaussian background field, entanglement option rt, and seed offset 1234 (Aalaila et al., 15 Jun 2026).

Outputs include CSV and JSONL event files, metadata, intensity grids, complexity-index summaries, and diagnostic figures. A CSV row has the schema

$\lambda_m(\mathbf s,t)$1

whereas JSONL stores one entire sequence per line (Aalaila et al., 15 Jun 2026).

Within SEAHORSE, HawkesNest inherits a stricter evaluation protocol designed to make STPP comparisons auditable. The protocol uses fixed temporal $\lambda_m(\mathbf s,t)$2 train/validation/test splits, requires that any normalization or coordinate transform be estimated from the training split only, and reports all benchmark NLLs in a common raw-coordinate space via Jacobian correction. The motivation is that STPP likelihoods depend on coordinate system and reference measure; without raw-space reporting, benchmark comparisons can be distorted by preprocessing rather than modeling ability (Aalaila et al., 1 Jul 2026).

SEAHORSE also frames heterogeneous STPP models through an encode–evolve–decode interface, so HawkesNest can test structurally different model families under a common executable contract while preserving their native inductive biases. This suggests that HawkesNest is not merely a dataset generator but a diagnostic layer coupled to benchmark semantics (Aalaila et al., 1 Jul 2026).

5. Empirical findings and diagnostic use

The benchmark paper first evaluates Hawkes-family baselines, deliberately choosing models structurally aligned with the generator. Even under that favorable alignment, performance degrades as controlled complexity increases. In the joint $\lambda_m(\mathbf s,t)$3–$\lambda_m(\mathbf s,t)$4 sweep, per-event test log-likelihood drops monotonically from $\lambda_m(\mathbf s,t)$5 to $\lambda_m(\mathbf s,t)$6 for Cox-Hawkes and from $\lambda_m(\mathbf s,t)$7 to $\lambda_m(\mathbf s,t)$8 for Hawkes. The point is methodological: controlled heterogeneity and entanglement are sufficient to stress even models drawn from the same broad family as the simulator (Aalaila et al., 15 Jun 2026).

For AutoSTPP under isolated entanglement, the paper reports that test NLL worsens at the highest entanglement level but is not strictly monotone at intermediate levels. A more structural diagnostic—correlation with the ground-truth intensity—degrades clearly: $\lambda_m(\mathbf s,t)$9 from $n$0 to $n$1. This is one of the benchmark’s central lessons: likelihood alone can mask failure to recover latent structure, whereas access to the true intensity exposes degradation more directly. Training-budget analysis further shows that increasing optimization budget narrows low-entanglement gaps but does not eliminate the separation at the hardest level, suggesting that the remaining deficit reflects inductive bias or optimization limitations rather than insufficient training time alone (Aalaila et al., 15 Jun 2026).

The SEAHORSE companion broadens these findings across neural STPP families. On the HawkesNest entanglement sweep $n$2–$n$3, NSMPP attains the best likelihood at every level; continuous-time and flow-based neural variants form the next tier; AutoSTPP and DeepSTPP follow; sample-based generative models such as SMASH and DSTPP trail under NLL; and simpler temporal baselines such as RMTPP and THP are far behind. The paper interprets NSMPP’s advantage as alignment between its additive learned-kernel structure and the Hawkes-like generator. At the same time, post-NLL diagnostics show that AutoSTPP and DeepSTPP retain the strongest correlation with the ground-truth intensity surface, while NSMPP, despite best NLL, has substantially lower intensity correlation. Teacher-forced temporal CRPS and autoregressive rollout further separate predictive accuracy, latent-structure recovery, and rollout coherence into distinct diagnostic dimensions (Aalaila et al., 1 Jul 2026).

Taken together, these findings establish HawkesNest as a stress test of model assumptions rather than a single-score benchmark. A plausible implication is that any STPP evaluation regime relying only on held-out NLL risks conflating density fit with structural recovery.

6. Scope, limitations, and relation to the broader Hawkes literature

The benchmark paper is explicit that HawkesNest’s indices are meaningful within a fixed generative family, not across arbitrary datasets or point processes. They are coordinates for controlled sweeps rather than universal scalar rankings of STPP difficulty. Additional stated limitations are that current experiments cover only a limited set of model families, evaluation emphasizes likelihood and intensity-recovery diagnostics, operational task metrics are not the main focus, and the current scenarios remain synthetic and mechanism-specific (Aalaila et al., 15 Jun 2026).

Future directions proposed in the benchmark paper include richer background fields, including PDE- or SDE-driven heterogeneity, contextual covariates such as weather, mobility, land use, and remote sensing, mixed-pillar settings, user-contributed generators, and extension toward a “living” unit-test layer for STPP components before deployment on noisy or proprietary data (Aalaila et al., 15 Jun 2026).

The benchmark’s broader significance is tied to the range of empirical domains in which Hawkes processes are already used. In social-media analysis, multivariate Hawkes models have been used to quantify how URL categories trigger one another and to characterize the influence signature of pathogenic social media accounts (Alvari et al., 2019). In network event modeling, the Group Network Hawkes Process introduces latent group structure on a fixed observed network to capture heterogeneous baseline activity, self-excitation, and neighbor influence (Fang et al., 2020). In humanitarian monitoring, Bayesian spatiotemporal discrete-time Hawkes models have been used to estimate sub-national conflict risk over time and space with explicit uncertainty quantification (Browning et al., 2024). This application breadth suggests that benchmarked failure modes—separability assumptions, background homogeneity assumptions, mark or type independence assumptions, and domain-geometry assumptions—are not merely synthetic curiosities but correspond to modeling decisions that recur across substantive STPP problems.

In that sense, HawkesNest occupies a specific methodological niche. It does not attempt to be a universal model of spatiotemporal event data. Instead, it makes latent generative structure explicit enough that model failure becomes interpretable: performance drops can be attributed to entanglement, heterogeneity, interaction structure, or topology, rather than left as unexplained benchmark variance (Aalaila et al., 15 Jun 2026).