PEGASUS-X: Dual-Use Framework

• PEGASUS-X is a dual-use term referring to a transformer model for long-input summarization and an invariant-generation engine for formal verification.
• The summarization model adapts PEGASUS/T5 with PagedAttention, demonstrating varied performance under scarce data conditions on radiological report summarization.
• The Pegasus–KeYmaera X integration automatically generates and verifies invariants for hybrid systems using sound ODE proof tactics and a modular methodology.

Searching arXiv for the cited PEGASUS-X papers and related context. arXiv search: PEGASUS-X summarization 16384 tokens PagedAttention arXiv search: ([2509.15419](/papers/2509.15419)) PEGASUS-X is used in the cited arXiv literature in two distinct senses. In medical NLP, it denotes a transformer-based encoder–decoder member of the PEGASUS/T5 family, evaluated for abstractive summarisation of radiological reports under scarce-data conditions, with pegasus-x-base and pegasus-x-large checkpoints (Benzoni et al., 18 Sep 2025). In deductive verification of hybrid systems, “PEGASUS-X” also refers to the Pegasus–KeYmaera X integration, in which an automatic continuous-invariant generator is coupled to KeYmaera X so that generated candidates are checked by sound ODE proof tactics rather than trusted heuristics (Sogokon et al., 2020).

1. Terminological scope and disambiguation

The shared label conceals two unrelated technical objects. One is a neural abstractive summarisation model family whose pre-training extends the original PEGASUS objective to support inputs up to 16 384 tokens via PagedAttention, and whose behaviour was studied during fine-tuning on the Open-I Indiana chest X-ray dataset (Benzoni et al., 18 Sep 2025). The other is a formal-methods framework in which Pegasus serves as a standalone invariant-generation engine and KeYmaera X serves as the proof-checking environment for differential-dynamic logic obligations (Sogokon et al., 2020).

A common misconception is to treat PEGASUS-X as a single framework spanning both machine learning and theorem proving. The cited record does not support that reading. Instead, the same name is attached to a long-input summarisation architecture in one line of work and to a Pegasus–KeYmaera X integration in another. This suggests that disambiguation is essential in technical communication, especially when cross-referencing arXiv literature across domains.

Usage	Technical setting	Source
PEGASUS-X	Transformer-based encoder–decoder summarisation model family	(Benzoni et al., 18 Sep 2025)
Pegasus–KeYmaera X (“PEGASUS-X”)	Automatic continuous-invariant generation integrated with KeYmaera X	(Sogokon et al., 2020)

2. PEGASUS-X as a summarisation model family

In the radiology summarisation study, PEGASUS-X reuses the transformer-based encoder–decoder design of PEGASUS/T5 in two sizes: pegasus-x-base and pegasus-x-large (Benzoni et al., 18 Sep 2025). The consolidated summary states that, where exact values were not given in the paper, the reported layer counts and related dimensions are the typical settings inherited from PEGASUS/T5.

Checkpoint	Parameters	Typical inherited settings
pegasus-x-base	~257 M	nominally 12 encoder layers + 12 decoder layers, hidden dimension $H\approx 768$, 12 self-attention heads
pegasus-x-large	~512 M	24 encoder + 24 decoder layers, $H\approx 1024$, 16 heads

Both variants use SentencePiece subword tokenization and a standard feed-forward size of approximately $4\times H$ (Benzoni et al., 18 Sep 2025). Their pre-training objective is the original PEGASUS “gap sentence generation” objective over massive web/news corpora, extended by Phang et al. to support longer inputs. In the study’s downstream task, the models were adapted to map Findings to Impressions in radiology reports, where Findings are longer text with median approximately 35 tokens and Impressions are short summaries with median approximately 6 tokens.

The dataset was the Open-I Indiana chest X-ray collection. The reported pipeline began with 3 996 raw reports, retained 85% of them, and yielded approximately 3 350 total examples; three malformed pairs were removed (Benzoni et al., 18 Sep 2025). The train/validation/test partition was 80%/10%/10%, corresponding to approximately 2 680 / 335 / 335 examples. This data regime is central to the paper’s interpretation, because the work explicitly studies adaptation of highly expressive summarisation models when the available supervision is limited.

3. Fine-tuning protocol and observed training dynamics

The fine-tuning configuration was held constant across the reported experiments except for checkpoint size, batch size, and training subset size (Benzoni et al., 18 Sep 2025). Training ran for 300 epochs with AdamW, Huggingface default settings, $\texttt{weight\_decay}=0.01$, mixed precision (fp16), and a constant learning rate of $2\times 10^{-5}$. Generation used $\texttt{beam\_size}=4$, $\texttt{max\_length}=64$, and $\texttt{predict\_with\_generate=True}$. Batch sizes were train $=26$, eval $=2$ for pegasus-x-base and train $H\approx 1024$0, eval $H\approx 1024$1 for pegasus-x-large. Data subsets of 10%, 50%, and 100% of the training set were evaluated, corresponding to approximately 268, 1 340, and 2 680 examples. Token truncation and padding length were set to 1.33 times the 98th percentile of word-tokenized lengths. Validation metrics were computed every epoch, and model states were saved every 5 epochs.

For pegasus-x-base, the validation trajectory was comparatively well-behaved. Metrics such as ROUGE-1 rose almost monotonically, and the model reached 95% of final ROUGE-1 by approximately 134 epochs with 10% data, approximately 60 epochs with 50% data, and approximately 29 epochs with 100% data (Benzoni et al., 18 Sep 2025). The study reports no pronounced “peak/drop/recovery” pattern for PEGASUS-X-base, but it does identify a clear earlier plateau with more data.

For pegasus-x-large, the picture was different. The validation curve was jagged and lower, which the authors interpret as indicating overfitting and poor generalization; outputs became incoherent or were mixed with pretraining artifacts, including mid-career footballer names (Benzoni et al., 18 Sep 2025). The paper relates this behaviour to the overfitting limb of epoch-wise double-descent, even though the clearest three-phase early peak, forgetting, and recovery pattern was observed in PEGASUS rather than PEGASUS-X.

The study formalizes generalization error over epochs as

$H\approx 1024$2

where $H\approx 1024$3 is the epoch index. A “peak” at $H\approx 1024$4 is a local minimum of $H\approx 1024$5, a “drop” at $H\approx 1024$6 a local maximum, and subsequent “recovery” beyond $H\approx 1024$7. Within that framing, PEGASUS-X-large’s erratic degradation is treated as analogous to an overfitting phase rather than as evidence of stable late recovery.

4. Evaluation metrics, reported results, and practical implications

The evaluation suite included ROUGE-N, ROUGE-L, BLEU, METEOR, and BERTScore-R (Benzoni et al., 18 Sep 2025). ROUGE-N is described as recall of overlapping $H\approx 1024$8-grams; ROUGE-L as LCS recall; BLEU as precision with brevity penalty; METEOR as using WordNet synonyms and $H\approx 1024$9; and BERTScore-R as a semantic similarity measure computed from contextual embeddings.

At 100% training data, the best validation ROUGE-1 for pegasus-x-base was 0.6505 at epoch 170, whereas pegasus-x-large reached 0.5687 at epoch 61 (Benzoni et al., 18 Sep 2025). For reference, PEGASUS-large and PEGASUS-xsum peaked at 0.6027 and 0.6323. On the reported evidence, the larger PEGASUS-X checkpoint was not the stronger model in this scarce-data setting.

The paper attributes the underperformance of pegasus-x-large to two factors. First, its capacity of approximately 512 M parameters is presented as greatly exceeding the data variance available from 2 680 examples, leading to rapid memorization and overfitting. Second, its pre-training is described as being geared toward summarizing very long documents, whereas radiology summaries in this task are extremely concise (Benzoni et al., 18 Sep 2025). A plausible implication is that checkpoint scale and pre-training regime must be matched not only to task family but also to target-summary length and supervision volume.

The recommendations are correspondingly cautious. The authors advise against relying solely on early stopping at the first validation peak, because the full curve may contain later improvements. They also propose curriculum learning, elastic weight consolidation, progressive unfreezing of layers, dropout or weight decay, and time-series overfitting alarms, and they explicitly suggest exploring Parameter-Efficient Fine-Tuning methods such as LoRA and QLoRA to reduce the number of trainable parameters and stabilize training (Benzoni et al., 18 Sep 2025). Future directions include clinician-in-the-loop evaluations on test splits, expansion to multiple anatomies and modalities such as MIMIC-RRS, domain-adaptive pretraining on biomedical or clinical text before task fine-tuning, and automatic metrics better aligned with clinical correctness, including RadGraph, COMET, BLEURT, and human–machine hybrid validation.

5. Pegasus–KeYmaera X as an invariant-generation architecture

In formal verification, Pegasus is implemented in the Wolfram Language (Mathematica) as a standalone invariant-generation engine whose internal modules may call symbolic, numeric, and heuristic routines, and which exposes a simple API over a TCP link or the built-in Mathematica link (Sogokon et al., 2020). KeYmaera X’s tactic layer, Bellerophon, calls Pegasus when it reaches an unproved continuous-evolution proof obligation.

The architecture comprises three main components. The classifier reads the input proof obligation in differential-dynamic logic, analyses the ODEs $4\times H$0, the evolution domain constraint $4\times H$1, and the initial and post sets, and selects an invariant-generation strategy. The generator implements a palette of generation methods together with a combinator such as differential-saturation or differential-divide-and-conquer. The proof-hint extractor packages each generated invariant together with a hint identifying its origin, for example a Darboux polynomial of degree 2 or an exponential-type barrier certificate with $4\times H$2, so that KeYmaera X can choose the corresponding ODE rule such as dI, dC, or dG (Sogokon et al., 2020).

Integration is mediated by the Mathematica interface. The Bellerophon tactic invgen marshals the current proof goal

$4\times H$3

and sends $4\times H$4 to Pegasus. Pegasus returns candidate invariants $4\times H$5 together with proof hints. KeYmaera X then invokes its sound, axiom-based ODE tactics to discharge

$4\times H$6

for each candidate invariant $4\times H$7. If one succeeds and also $4\times H$8 and $4\times H$9, the proof closes. The central architectural point is that invariant synthesis and proof checking are separated: Pegasus may use heterogeneous search procedures, but acceptance depends on proof reconstruction within KeYmaera X.

6. Invariant-generation methods and soundness guarantees

Pegasus implements several classes of continuous invariant generation methods (Sogokon et al., 2020). Qualitative abstraction discretizes $\texttt{weight\_decay}=0.01$0 by sign conditions on polynomials $\texttt{weight\_decay}=0.01$1, yielding cells

$\texttt{weight\_decay}=0.01$2

It then builds an exact abstraction graph by checking, for each pair $\texttt{weight\_decay}=0.01$3,

$\texttt{weight\_decay}=0.01$4

and extracts reachable sets as unions of cells, each of which is a semi-algebraic invariant.

Polynomial first integrals are sought via a template

$\texttt{weight\_decay}=0.01$5

subject to the Lie-derivative constraint

$\texttt{weight\_decay}=0.01$6

This yields a linear system in the coefficients $\texttt{weight\_decay}=0.01$7. A nontrivial solution gives a first integral $\texttt{weight\_decay}=0.01$8, and level sets $\texttt{weight\_decay}=0.01$9 or bounds $2\times 10^{-5}$0 can be used as invariants.

Darboux polynomials satisfy

$2\times 10^{-5}$1

for some cofactor $2\times 10^{-5}$2, from which $2\times 10^{-5}$3 is invariant because $2\times 10^{-5}$4 on the zero set. Rational first integrals are then obtained by combining Darboux polynomials $2\times 10^{-5}$5 with cofactors $2\times 10^{-5}$6 and solving $2\times 10^{-5}$7 over $2\times 10^{-5}$8 so that

$2\times 10^{-5}$9

satisfies $\texttt{beam\_size}=4$0. Barrier certificates are handled in both convex and exponential forms. The convex case uses a polynomial template $\texttt{beam\_size}=4$1 with initial-set containment, unsafe-set separation, and the differential side condition

$\texttt{beam\_size}=4$2

The exponential barrier variant uses the Kong–Jha inequality $\texttt{beam\_size}=4$3.

The soundness story is entirely proof-theoretic. The safety rule in differential-dynamic logic is given as

$\texttt{beam\_size}=4$4

For a candidate $\texttt{beam\_size}=4$5, inductiveness is reduced to the semi-algebraic obligation

$\texttt{beam\_size}=4$6

Initial containment and safe containment are discharged by propositional or real-arithmetic entailments, while inductiveness is established by specialized tactics implementing the dI, dC, and dG rules. The paper’s stated conclusion is that the framework is fully automatic, sound, and extensible, and that no unchecked heuristics can subvert soundness because acceptance always depends on the sound core of KeYmaera X.

7. Configuration, benchmarks, and representative case studies

Invocation from KeYmaera X is exposed through the Bellerophon tactic invgen, for example: $\texttt{max\_length}=64$7 A GUI pane is also described as listing Pegasus options such as maximum polynomial degree, per-method timeouts, toggle heuristics including grid sampling and locally transverse forms, and the choice of combination strategy, saturation versus divide-and-conquer (Sogokon et al., 2020).

The experimental evaluation used 150 continuous-safety benchmarks: 71 two-dimensional, 30 three-dimensional, 35 higher-dimensional, and 14 product systems, covering linear, affine, multi-affine, homogeneous, and polynomial ODEs with bounded or unbounded, open or closed, atomic or semi-algebraic specifications (Sogokon et al., 2020). Under a 20-second cutoff, Differential Saturation solved 138/150; qualitative abstraction alone solved 110/150; first integrals 127/150; Darboux 120/150; and barrier methods 105/150. In 16 cases, Differential Saturation solved problems that no single method solved. Median Differential Saturation total time was approximately 0.2 seconds, compared with a best-other median of approximately 1.5 seconds.

Representative examples include the bouncing ball and thermostat. For the bouncing ball with ODE mode $\texttt{beam\_size}=4$7, safety condition $\texttt{beam\_size}=4$8, and initial condition $\texttt{beam\_size}=4$9, Pegasus returns the first integral $\texttt{max\_length}=64$0 and the invariant $\texttt{max\_length}=64$1; a proof script using invgen(strategy:"firstIntegrals", maxDeg:2); dI; closes the proof (Sogokon et al., 2020). For the thermostat, with two modes $\texttt{max\_length}=64$2 and $\texttt{max\_length}=64$3 and safety objective $\texttt{max\_length}=64$4, Pegasus generates the barrier certificate

$\texttt{max\_length}=64$5

and the two ODE proof obligations are solved by the exponential-barrier tactic dC with $\texttt{max\_length}=64$6.

Taken together, the two arXiv uses of PEGASUS-X illustrate a purely nominal overlap rather than a methodological one. In one case, PEGASUS-X is a long-input abstractive summarisation architecture whose larger checkpoint can underperform under scarce supervision; in the other, it denotes a sound integration of automatic invariant generation with theorem proving. The former highlights model-capacity risks in specialised-domain fine-tuning, while the latter highlights proof reconstruction as the mechanism that converts heterogeneous search into certified safety results.