3GPP Change Requests Overview

• 3GPP Change Requests (CRs) are structured documents that capture specification revisions for correcting, clarifying, and enhancing cellular network standards.
• The dataset spans Releases 5–19 and includes both pre- and post-revision texts with expert rationales, providing a comprehensive view of specification evolution.
• The analysis leverages CR-eval, a framework that employs LLMs to detect weaknesses, outline revisions, and validate changes via diff analysis.

Searching arXiv for the specified paper and closely related cellular-specification refinement work. 3GPP Change Requests (CRs) are approved records of specification revision that document how cellular-network standards are corrected, clarified, or extended. In the reported study, they are treated not merely as archival edits but as a large empirical substrate for specification-refinement research: 205,374 raw CR files were crawled from the 3GPP FTP server, parsed through “Track Changes,” and converted into 189,904 structured CRs spanning Releases 5 through 19 and covering TS 21–55 series, supplemented by TR 21.900 and other catalogues such as 33.855, 33.512, and 33.117. Because these records contain both pre-revision and post-revision text, as well as expert rationales and consequences, they expose how weaknesses in 3GPP standards are identified and repaired in practice, including security-relevant weaknesses that affect reliability and security in cellular networks (Dong et al., 6 Jul 2025).

1. Corpus scope and provenance

The reported CR corpus was assembled by crawling all approved CRs from the 3GPP FTP server, specifically from the 3GPP information databases, across Releases 5 through 19. The source coverage includes TS 21–55 series and is supplemented by TR 21.900, which concerns CR drafting procedures, as well as other catalogues including 33.855, 33.512, and 33.117. Raw CR files were available in Word formats (doc and docx) and were parsed via “Track Changes” into structured records. This process yielded 189,904 fully parsed CRs after discarding malformed cases from an initial raw set of 205,374 files (Dong et al., 6 Jul 2025).

The release distribution is described as roughly uniform from Rel-5 to Rel-19, with peaks at Rel-8 and Rel-15. The reported interpretation associates Rel-8 with LTE introduction and Rel-15 with 5G introduction. This makes the corpus useful for studying both longitudinal specification evolution and periods of concentrated architectural change. A plausible implication is that CR activity is especially informative around major generational transitions, because those transitions coincide with both new functionality and intensified specification repair.

The same corpus also supports security-focused filtering. Among the 189,904 structured CRs, 4,869 CRs were auto-annotated “High-Risk” via LLaMA-3.1-70B; cross-referencing with Chen et al. (2022) yielded 529 true positives; and these were manually distilled into a 200-case high-quality security subset (Dong et al., 6 Jul 2025).

2. Record structure, metadata, and category system

Each structured CR contains a rich set of metadata fields that collectively define the revision event. The extracted fields are CR ID, Title, Date, Release version, specification identifier, Category, Submitter, Priority if present, affected section or sections, Original statements (orig), Revised statements (rev), expert rationales in the form of Reason for change ($R_n$) and Consequences if not revised ($R_c$), and a Change summary (Dong et al., 6 Jul 2025).

This field structure is important because it makes CRs suitable for both documentary analysis and machine learning. The orig and rev fields expose the textual delta; $R_n$ and $R_c$ encode expert justification; and category labels provide a coarse ontology of change type. In the reported dataset, the categories are distributed as follows:

Category	Meaning	Approximate share
A	consistency	~12%
B	feature addition	~18%
C	functional mod	~25%
D	editorial	~30%
F	correction	~15%

The paper further interprets these categories as mapping to design-issue types. Category F (correction) often fixes design flaws, with repeated keystream leak given as an example. Category D (editorial) can clarify underspecified behaviors, exemplified by ABBA handling. Category C (functional) can add new features or restrictions, with emergency detach given as an example (Dong et al., 6 Jul 2025).

A common misconception is that editorial CRs are purely cosmetic. The reported taxonomy directly complicates that view: category D is said to clarify underspecified behaviors, and underspecification can be security-relevant. This suggests that the formal category name alone is not a reliable proxy for the substantive importance of a CR.

Two concrete examples illustrate the documentary role of CRs. CR C1-172658, titled “MO Detach without Integrity Protection,” identifies a DoS risk when UEs send unprotected Detach. CR C1-193185, titled “ABBA under-specification,” specifies how the UE should treat non-zero or over-length ABBA fields (Dong et al., 6 Jul 2025).

3. CRs as evidence of weakness discovery and specification repair

Within the reported study, CRs are used as direct evidence of how weaknesses in cellular specifications are recognized and refined. This use is grounded in the presence of paired original and revised statements together with expert rationales. In that sense, a CR is not only a change log; it is also a compact record of defect identification, diagnosis, and repair proposal (Dong et al., 6 Jul 2025).

The security-focused subset was created through a multi-stage pipeline. First, 4,869 CRs were auto-annotated as “High-Risk” using LLaMA-3.1-70B. Second, these were cross-referenced with prior literature, specifically Chen et al. (2022), yielding 529 true positives. Third, manual filtering produced a 200-case benchmark chosen to ensure orig/rev completeness and to avoid empty orig contexts. The final benchmark spans 74 distinct TS/TR documents and covers Releases 5–17 (Dong et al., 6 Jul 2025).

The resulting benchmark highlights a central property of CRs: many involve long and dispersed context. The paper reports that many benchmark cases exceed 5,000 tokens, stressing long-context capabilities. It also notes failure cases involving subtle weaknesses such as cross-clause interactions across TS 24.008 and TS 33.501, which still evade even GPT-o3-mini in single trial. This suggests that a substantial fraction of CR reasoning depends on context assembly across specification boundaries rather than on isolated clause reading alone (Dong et al., 6 Jul 2025).

The examples reported in the paper reinforce that point. “MO Detach without Integrity Protection” concerns a DoS risk associated with unprotected Detach. “ABBA under-specification” concerns how the UE should handle non-zero or over-length ABBA fields. These are different kinds of refinement events: one is framed explicitly as a risk, while the other is framed as underspecification. Yet both are represented as CRs, indicating that the CR mechanism accommodates both overt vulnerability repair and normative clarification.

4. Formalization in CR-eval

The paper introduces CR-eval, described as a principled evaluation framework that formalizes specification refinement into three LLM-tractable tasks on the 200-case security benchmark (Dong et al., 6 Jul 2025).

The first task, Fill-CR (orig), gives the original specification clause and asks the model to “play expert reviewer” by uncovering hidden weaknesses and drafting a CR, including reason, summary, and consequences. The second task, Outline Revision (orig), provides the specification text together with an identified weakness and asks for a high-level plan for the textual revision. The third task, Diff Analysis (rev), provides original and revised clauses in diff form and asks the model to verify that the revision addresses the identified weakness (Dong et al., 6 Jul 2025).

Generation quality is evaluated using pass@k, in Humaneval style, with $n=10$ samples per test case and $c$ denoting the number of correct outputs:

$$\mathrm{pass}@k \;=\; 1 \;-\; \frac{\binom{n-c}{k}}{\binom{n}{k}} .$$

The paper also notes that, where applicable, one could report classification-style metrics for a binary detection sub-task:

$$\mathrm{Precision}=\frac{TP}{TP+FP},\quad \mathrm{Recall}=\frac{TP}{TP+FN},\quad F_1=2\;\frac{\mathrm{Precision}\times\mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}} .$$

Judgment is automated through GPT-4o as an “LLM-as-a-Judge,” which scores each output on a 5-point Likert scale from $-2$ to $+2$, with scores of at least $R_c$0 mapped to “correct” (Dong et al., 6 Jul 2025).

The task design makes explicit that CRs support multiple forms of technical reasoning. Fill-CR emphasizes weakness discovery and rationale generation; Outline Revision emphasizes repair planning; Diff Analysis emphasizes post hoc validation of the revision. A plausible implication is that these tasks correspond to distinct points in an editorial workflow rather than to a single monolithic automation problem.

5. Benchmarking LLMs on CR tasks

The reported benchmark evaluates 16 state-of-the-art LLMs divided into three groups. The medium-sized open models are GLM-4-9B, Mistral-7B-v0.3, InternLM-2.5-7B, Qwen-2.5-7B, Qwen-2.5-14B, LLaMA-3.1-8B, and CRitic-LLaMA-3.1-8B. The advanced or closed models are LLaMA-3.1-70B, Qwen-2.5-32B, Qwen-2.5-72B, DeepSeek-V3, GPT-3.5-turbo, GPT-4o-mini, GPT-4o, and Claude-3.5-Sonnet. The reasoning models are DeepSeek-R1, GPT-o3-mini, and Gemini-2.0-flash-thinking (Dong et al., 6 Jul 2025).

Performance varies strongly across tasks. For Outline Revision, the top models achieve more than 97 out of 200 under pass@5. For Diff Analysis, CRitic-8B reaches 148/200, while the best reasoning model, Gemini-2.0, is reported at approximately 169/200. Fill-CR is identified as the hardest task: CRitic-8B reaches 57.8/200, GPT-o3-mini approximately 127.9/200, and the base LLaMA-8B 18.1/200 (Dong et al., 6 Jul 2025).

These results support two technical conclusions stated in the paper. First, top models can discover security-related weaknesses in over 127 out of 200 test cases within five trials. Second, specialization can allow an 8B model to match or surpass advanced LLMs such as GPT-4o and DeepSeek-R1 on some tasks. At the same time, the benchmark does not indicate full automation. The paper explicitly states that evaluations on 30 cellular attacks identify open challenges for achieving full automation (Dong et al., 6 Jul 2025).

The error profile is also informative. Even advanced reasoning models can miss subtle weaknesses in a single trial, especially when the weakness depends on cross-clause interactions. This indicates that raw model scale or reasoning specialization does not remove the need for better context construction and verification.

6. Specialization through CRitic-LLaMA

To close the gap between open 8B models and stronger proprietary or larger models, the paper presents a three-stage fine-tuning recipe for an 8B model called “CRitic-LLaMA” (Dong et al., 6 Jul 2025).

Stage 1 is Domain-Adaptive Continual Training (DACT) on the TSpec-LLM corpus together with filtered Wikipedia and ArXiv data. The loss is

$R_c$1

Stage 2 is Task-Specialized Tuning (TST) on all CR-converted instances, specifically 185,035 non-sec and 4,669 sec instances. The loss is

$R_c$2

Stage 3 is Security-Centric Tuning (SCT) via LoRA adapters with $R_c$3 and $R_c$4 on 4,669 security-related CRs. The loss is

$R_c$5

The hyperparameters across the three stages are AdamW with $R_c$6 and $R_c$7, epochs $R_c$8, cosine scheduler, warmup ratios $R_c$9, batch sizes $R_n$0, learning rates $R_n$1, and weight decay $R_n$2 (Dong et al., 6 Jul 2025).

The reported behavioral analysis links this specialization pipeline to domain-specific output characteristics. CRitic-8B shows increased use of security-specific tokens such as “_interception” and “_confidentiality” compared with base LLaMA-8B. In the Fill-CR task, CRitic-8B correctly proposed adding MAC-S into keystream for re-sync, whereas base models often missed the cryptographic nuance. This is a concrete instance in which specialization improved sensitivity to security-relevant technical detail (Dong et al., 6 Jul 2025).

7. Limitations, misconceptions, and future directions

The paper identifies three unresolved issues. The first is Calibration and Hallucination: LLMs generate plausible but sometimes incorrect CRs, and blind acceptance yields false positives. The second is Ambiguous or Cross-Clause Context: many weaknesses span multiple sections, and extracting complete context remains challenging. The third is Dynamic Specification Evolution: new releases, including Rel-18+, are not yet in CR-eval, creating domain drift over time (Dong et al., 6 Jul 2025).

These limitations bear directly on how CR automation should be interpreted. A common misconception is that a high-performing model on text generation can therefore replace the specification-review process end to end. The reported results do not support that conclusion. They support automation of specification refinement, but they also state that achieving full automation requires better calibration, broader context gathering, and human-in-the-loop orchestration (Dong et al., 6 Jul 2025).

The recommendations given in the paper are correspondingly procedural rather than purely model-centric. Under Tooling, the paper recommends integrating LLM outputs with lightweight formal checkers or rule-based filters to triage proposals before human review. Under Dataset Enhancements, it recommends continuously incorporating new CRs such as Rel-18+ and academic-reported attacks, and enriching rationales with multi-perspective chain-of-thoughts. Under Methodology, it recommends LLM-in-the-loop workflows in which experts iteratively refine LLM suggestions and exploration of reinforcement-learning from real editorial feedback. Under Calibration Mechanisms, it recommends verifier agents, such as a secondary LLM or symbolic checks, to flag uncertain or low-confidence CR suggestions (Dong et al., 6 Jul 2025).

Taken together, these recommendations position 3GPP CRs as a technically rich supervision source for automated specification-refinement pipelines, while also delimiting the boundary between assistance and autonomy. The empirical record extracted from CRs shows that refinement is not a single operation but a compound process involving weakness discovery, rationale articulation, textual revision, and post-revision verification.