COLA-GEC: Bidirectional Grammatical Framework

• COLA-GEC is a bidirectional framework that integrates CoLA-style acceptability judgment with grammatical error correction via mutual transfer.
• It repurposes GEC datasets by converting ungrammatical sentences into binary labels and injecting acceptability signals as dynamic loss weights.
• The framework yields multilingual gains in ACC, MCC, and F0.5, demonstrating robust cross-lingual transfer and enhanced grammatical competence.

Searching arXiv for the specified topic and paper to ground the article in current research. COLA-GEC is a bidirectional framework for coupling grammatical acceptability judgment and grammatical error correction within a shared grammatical modeling pipeline. Introduced in “COLA-GEC: A Bidirectional Framework for Enhancing Grammatical Acceptability and Error Correction” (Yang et al., 16 Jul 2025), it formalizes two complementary transfers: grammatical error correction (GEC) corpora are repurposed to augment CoLA-style acceptability training, and acceptability predictions are injected into GEC optimization through a dynamic loss. The framework is positioned on the observation that GEC and CoLA depend on overlapping grammatical competence but have usually been trained on disjoint supervision regimes. In COLA-GEC, this separation is replaced by a mutual-transfer design intended to strengthen both binary acceptability classification and sequence-to-sequence correction across multiple languages (Yang et al., 16 Jul 2025).

1. Conceptual basis and task formulation

COLA-GEC links two established NLP tasks. Grammatical acceptability judgment, denoted CoLA in the paper’s terminology, predicts whether a single sentence is grammatically acceptable; it is typically treated as binary classification. Grammatical error correction maps an ungrammatical sentence to a corrected sentence, usually with neural sequence-to-sequence or sequence-to-edit models (Yang et al., 16 Jul 2025).

The framework’s central claim is that both tasks encode the same underlying grammatical contrasts in different forms. CoLA datasets explicitly label sentences as acceptable or unacceptable, often through minimal pairs or textbook-derived examples. GEC datasets, by contrast, encode acceptability implicitly through aligned errorful/correct sentence pairs. COLA-GEC makes this relationship operational: errorful GEC inputs are labeled as unacceptable and corrected targets as acceptable for acceptability-model training, while a pretrained acceptability model supplies grammaticality-sensitive feedback during GEC training (Yang et al., 16 Jul 2025).

This design is explicitly bidirectional rather than multi-task in the strict sense. The paper reports separate training of acceptability and GEC models, with the acceptability model acting as a fixed critic during GEC optimization. There is no reported joint optimization or gradient flow from GEC back into the acceptability model (Yang et al., 16 Jul 2025). A plausible implication is that the framework prioritizes modularity and ease of integration over tighter end-to-end coupling.

2. Bidirectional transfer architecture

The architecture comprises two transfer directions.

First, acceptability models are improved with GEC-derived supervision. This component is termed G-CoLA. Existing CoLA-style resources are augmented by converting GEC pairs into binary acceptability instances: corrected sentences receive label 1 and original errorful sentences receive label 0. Only training and development splits of GEC corpora are used for this construction, specifically to avoid test leakage (Yang et al., 16 Jul 2025).

Second, acceptability signals are integrated into GEC training. This component is termed G-CoLA GEC. A BART-based sequence-to-sequence GEC model produces candidate corrections during training, and a pretrained G-CoLA model scores those candidates with logits for “unacceptable” and “acceptable.” The resulting signal is then used to dynamically weight the GEC cross-entropy objective via a Label Confidence Weighted Learning mechanism (Yang et al., 16 Jul 2025).

The paper also reports an inference-time extension through dual-critic decoding. For English and Chinese, the trained G-CoLA GEC model is combined with GEC-di, a dual-critic decoding framework. For German, because no prior dual-critic setup was available, the authors train a German language-model critic and a target-side error-detection critic, then adapt dual-critic decoding accordingly (Yang et al., 16 Jul 2025).

The overall flow is therefore sequential: construct GEC-augmented acceptability data, train G-CoLA, use G-CoLA as a fixed critic to guide GEC via dynamic loss, and optionally combine the resulting GEC model with critic-guided decoding at inference (Yang et al., 16 Jul 2025).

3. Data construction and multilingual coverage

The framework draws on both acceptability corpora and GEC corpora, with 12 languages represented for acceptability and three languages used for GEC model training and evaluation (Yang et al., 16 Jul 2025). The paper emphasizes that Chinese, English, German, and Arabic are explicitly augmented with GEC-derived acceptability examples, while the multilingual setup extends benefits to related languages through cross-lingual transfer.

For acceptability, the base resources include English CoLA, Chinese CoLAC, textbook-derived examples for German and Arabic, and MELA for multilingual coverage. Additional multilingual resources include JCoLA for Japanese, NoCoLA for Norwegian, and DaLAJ for Swedish, among others standardized to binary labels (Yang et al., 16 Jul 2025). The aggregate train/dev/test sizes reported in the paper are as follows.

Language	Train/Dev/Test
English	36,551 / 2,400 / 516
Chinese	16,772 / 2,928 / 931
Russian	7,869 / 1,483 / 2,341
Italian	7,801 / 946 / 975
German	19,621 / 2,803 / 402
French	500 / 521 / 521
Spanish	500 / 321 / 322
Japanese	7,419 / 1,378 / 694
Arabic	14,330 / 2,413 / 313
Icelandic	500 / 1,194 / 1,194
Norwegian	8,030 / 1,000 / 1,000
Swedish	7,682 / 890 / 888

The language-specific augmentation procedures are described in some detail. For Chinese, the framework starts from CoLAC, described as containing 7,495 sentences with crowd-sourced binary labels, and adds pairs from MuCGEC, FCGEC, and NaSGEC. The resulting corpus uses 6,072 CoLAC sentences and 10,700 GEC-derived sentences for training and development (Yang et al., 16 Jul 2025). For English, the framework begins with CoLA and adds filtered pairs from W{paper_content}I+LOCNESS, FCE, and NUCLE, yielding 8,551 CoLA sentences plus 28,000 GEC-derived sentences for training and development (Yang et al., 16 Jul 2025). German and Arabic are augmented from textbook-derived examples with Falko+MERLIN and QALB 2014/2015 respectively (Yang et al., 16 Jul 2025).

For GEC training corpora, Chinese uses Lang8 and HSK for pretraining, then FCGEC, MuCGEC-dev, and NaSGEC-Exam splits for fine-tuning; English uses Clang8 for pretraining, then W{paper_content}I+LOCNESS, FCE, NUCLE, and BEA-19-dev for fine-tuning; German uses Falko for pretraining and MERLIN for fine-tuning (Yang et al., 16 Jul 2025).

The labeling rule for GEC-to-CoLA conversion is fixed and simple: corrected = 1, errorful original = 0 (Yang et al., 16 Jul 2025). The paper does not report additional balancing or noise-filtering heuristics beyond split construction and, for English, filtering for high-quality pairs.

4. Model families and optimization mechanics

The acceptability side of COLA-GEC uses two model classes. Monolingual models are RoBERTa-based encoders with a binary classification head, trained separately for English, Chinese, and German, and named G-Cola-EN, G-Cola-CN, and G-Cola-DE. The multilingual variant, G-Cola-Multi, is XLM-R-based and jointly fine-tuned over 12 languages (Yang et al., 16 Jul 2025).

On the GEC side, the framework uses BART-based sequence-to-sequence models. Chinese employs BART-large-Chinese in a two-stage procedure, with pretraining on Lang8 and fine-tuning on HSK, MuCGEC-dev, FCGEC train/dev, and NaSGEC-Exam train/dev. English uses BART-large in three stages: pretraining on Clang8, fine-tuning on W{paper_content}I+LOCNESS, and then further fine-tuning on FCE+NUCLE+BEA-19-dev. German uses a BART-German backbone with Falko pretraining and MERLIN fine-tuning (Yang et al., 16 Jul 2025).

The acceptability model outputs two logits, denoted $Logits_0$ for “unacceptable” and $Logits_1$ for “acceptable.” These are converted to a scalar acceptability signal by

$$COLA\_score = \frac{1}{1 + e^{-(Logits\_0-Logits\_1)}}$$

as reported in the paper (Yang et al., 16 Jul 2025). Under this formulation, a larger score corresponds to greater predicted unacceptability of the candidate correction.

This signal enters the GEC loss through a dynamic weighting term:

$$Dynamic\_loss = -\sum_{t=1}^{T} Y_t \log(\hat{Y}_t) * \sqrt{Acc * COLA\_score}$$

where $T$ is the sequence length, $Y_t$ is the token-level ground truth, $\hat{Y}_t$ is the predicted probability at step $t$, and $Acc$ is the CoLA model’s accuracy (Yang et al., 16 Jul 2025). The paper characterizes this as a Label Confidence Weighted Learning mechanism. Because the weight grows with $COLA\_score$, the training penalty is amplified when the acceptability model judges the correction to be less grammatical.

The paper does not specify training hyperparameters such as optimizer, batch size, learning rate, warmup schedule, dropout, tokenizer details, or compute budget (Yang et al., 16 Jul 2025). It also does not report calibration procedures or uncertainty-aware alternatives for the acceptability signal.

5. Empirical performance

The acceptability results show consistent gains over vanilla CoLA baselines in Chinese, English, German, and Arabic. The reported improvements are from ACC 81.30 to 85.43 and MCC 56.61 to 57.20 for Chinese; from ACC 85.51 to 88.10 and MCC 63.49 to 65.60 for English; from ACC 62.90 to 66.42 and MCC 37.74 to 41.81 for German; and from ACC 60.67 to 64.90 and MCC 36.30 to 42.24 for Arabic (Yang et al., 16 Jul 2025).

Against MELA, described in the paper as the state of the art, G-CoLA surpasses MCC on all four reported languages, including English 65.60 versus 60.64, Chinese 57.20 versus 54.94, German 41.81 versus 26.72, and Arabic 42.24 versus 14.12 (Yang et al., 16 Jul 2025). The multilingual G-CoLA-Multi model also improves over vanilla multilingual baselines, which the paper interprets as evidence of positive cross-lingual transfer.

For GEC, evaluation uses ChERRANT for Chinese and ERRANT for English and German, with Precision, Recall, and $Logits_1$0 as the main metrics (Yang et al., 16 Jul 2025). The strongest reported results are associated with the combination of G-CoLA-guided training and dual-critic decoding.

Language / Test set	System	Reported P / R / F0.5
Chinese / MuCGEC	G-CoLA GECdi-CN	57.12 / 33.9 / 49.91
Chinese / FCGEC	G-CoLA GECdi-CN	67.11 / — / 58.6
English / CoNLL-14	G-CoLA GECdi-EN	79.55 / 49.04 / 70.84
English / BEA-19	G-CoLA GECdi-EN	77.33 / 61.67 / 74.03
German / Falko-MERLIN	G-CoLA GECdi-DE	71.93 / 62.47 / 70.80

The Chinese MuCGEC score of 49.91 $Logits_1$1 improves over GECdi’s 48.61. On CoNLL-14, the English system reaches 70.84 $Logits_1$2, improving over GECdi’s 69.6. On Falko-MERLIN, the German system substantially exceeds WikiGerman’s 29.73 $Logits_1$3, though it remains below MultimodelGEC’s 76.30 (Yang et al., 16 Jul 2025).

The ablation study isolates the effect of acceptability guidance. On CoNLL-14, base GEC improves from 54.08 to 59.17 $Logits_1$4 when augmented with G-CoLA; GECdi improves from 70.11 to 70.84 when G-CoLA is added. On BEA-19, the corresponding gains are 61.12 to 62.97 and 73.55 to 74.03. For Chinese, MuCGEC improves from 39.75 to 42.93 and FCGEC from 42.75 to 46.93 when moving from base GEC to +G-CoLA, with smaller additional gains on top of GECdi (Yang et al., 16 Jul 2025). These results support the paper’s claim that the GEC-augmented acceptability critic is stronger than a vanilla CoLA critic.

6. Error profile, limitations, and open technical issues

A central limitation identified in the paper concerns punctuation and uncommon error categories. Performance on BEA-19 is reported to be markedly lower than on CoNLL-14, and the paper attributes this to differences in error-type composition. BEA-19 contains more punctuation errors, 16.73% versus 7.60%, and more “OTHER” uncommon errors, 15.69% versus 1.50%, totaling 32.4% versus 9.1%. Orthographic errors are also more frequent in BEA-19, 8.03% versus 3.81% (Yang et al., 16 Jul 2025).

The paper reports controlled analyses on BEA-19 dev. The baseline G-CoLA GECdi-EN obtains $Logits_1$5. Removing punctuation errors raises this to 74.40; removing “OTHER” uncommon errors gives 73.96; removing both yields 75.92. Targeted fine-tuning on these errors produces only a modest gain to 73.11 (Yang et al., 16 Jul 2025). The authors therefore argue that punctuation and rare phenomena likely require specialized modeling.

This limitation is theoretically important because acceptability classifiers trained mainly on lexical-syntactic contrasts may not fully capture punctuation-sensitive clause structure, discourse segmentation, or stylistic conventions. The paper explicitly notes that punctuation often interacts with sentence-level structure and pragmatics in ways not fully addressed by token-level cross-entropy or standard acceptability supervision (Yang et al., 16 Jul 2025). This suggests that the main benefit of COLA-GEC lies in grammaticality-sensitive correction of canonical error patterns rather than in fully resolving structurally or pragmatically marked phenomena.

Several methodological caveats are also noted. The dynamic weighting depends on the quality and calibration of the acceptability model; if acceptability judgments are noisy or biased, GEC training could be misdirected (Yang et al., 16 Jul 2025). No per-example uncertainty estimates or calibration mechanisms are introduced beyond the square-root LCWL factor. The paper also does not report sensitivity analyses for alternative weighting forms, alternative functions of the logits difference, or variation in the scale and composition of GEC-to-CoLA augmentation (Yang et al., 16 Jul 2025).

7. Significance, applicability, and reproducibility

COLA-GEC is significant within grammatical modeling because it reframes CoLA and GEC as mutually informative tasks rather than parallel benchmarks. Its contribution is not merely that one task regularizes the other, but that the supervision format of each task can be transformed into a signal useful for the other: aligned errorful/correct pairs become binary acceptability supervision, and acceptability judgments become dynamic penalties over GEC outputs (Yang et al., 16 Jul 2025).

The framework is also multilingual in orientation. Monolingual RoBERTa-based acceptability models are trained for English, Chinese, and German, while the multilingual XLM-R model is trained jointly over 12 languages (Yang et al., 16 Jul 2025). The paper’s evidence suggests that explicit GEC augmentation in some languages can benefit other languages through cross-lingual transfer. This suggests a practical path for extending grammatical acceptability modeling in settings where direct CoLA resources are scarce but GEC corpora exist in the same or related languages.

From an application standpoint, the paper outlines a concrete pipeline: build or acquire CoLA-style data, augment it with GEC training and development pairs, train G-CoLA, train a BART-based GEC model using the dynamic acceptability-weighted loss, and optionally integrate dual-critic decoding at inference (Yang et al., 16 Jul 2025). For languages without an existing dual-critic framework, the German setup indicates that a language-model critic and a target-side GED critic can be constructed and combined with the acceptability model (Yang et al., 16 Jul 2025).

Reproducibility is partial rather than complete. The code repository is reported as https://github.com/YXY2gdufs/colagec-project (Yang et al., 16 Jul 2025), and the paper provides dataset compositions and split sizes for the principal languages. However, training hyperparameters, tokenization details, and compute specifications are not given. A plausible implication is that the high-level framework is reproducible, while exact numerical replication may require repository-specific defaults or additional experimentation.

Within the 2025 paper’s scope, COLA-GEC is presented as a modular, empirically effective synthesis of acceptability judgment and grammatical correction. Its strongest reported contribution is the demonstration that grammatical acceptability signals can serve both as additional supervision for classification and as a training-time critic for correction, yielding gains in ACC, MCC, and $Logits_1$6 across several multilingual settings while leaving punctuation and rare-error handling as unresolved problems (Yang et al., 16 Jul 2025).