Zero-shot S-CFG: Cross-Modal Grammar Mapping

• Zero-shot S-CFG is a framework that utilizes synchronous context-free grammars to generate aligned utterance–program pairs for tasks like semantic parsing and TTS.
• It employs condition separation and selective guidance techniques to balance content fidelity, naturalness, and speaker similarity during cross-modal synthesis.
• Empirical results show competitive accuracy with supervised models, reducing dependency on annotated data while enhancing performance in zero-shot scenarios.

Zero-shot Synchronous Context-Free Grammar (S-CFG) is a paradigm exploiting synchronous context-free grammar-based generative strategies for tasks requiring cross-modal or cross-system output mapping, particularly in domains where fully supervised data is scarce or unavailable. In recent years, the Zero-shot S-CFG framework has played a central role in both semantic parsing—with automatic generation of utterance–program pairs—and speech synthesis—where classifier-free guidance (CFG) is extended to support zero-shot scenarios using condition separation and selective guidance schemes. These advances are critically evaluated in text-to-speech (TTS) and controlled semantic parsing, with particular attention to balancing the competing desiderata of fidelity, naturalness, and coverage.

1. Formal Definition of Synchronous CFG and Its Zero-Shot Instantiation

A synchronous context-free grammar (S-CFG) is defined as the 5-tuple:

$G = (N, \Sigma, \Gamma, R, S)$

where $N = N_u \cup N_p$ is a set of nonterminals partitioned into utterance-side ($N_u$) and program-side ($N_p$) categories. $\Sigma$ and $\Gamma$ denote the natural language and logic/program terminal sets, respectively. $R$ contains synchronous productions of the form $A \rightarrow \langle \alpha,\ \beta\rangle$, synchronously expanding an utterance fragment and its aligned meaning representation or program. $S \in N_u$ designates the utterance start symbol.

For zero-shot learning, one exhaustively enumerates all derivations to depth $D$, collects canonical utterance–program pairs, and then synthesizes linguistically-diverse paraphrases to expand the dataset coverage without manual annotation. The S-CFG guarantees strong compositional alignment between modalities and supports effective generalization to previously unobserved forms (Yin et al., 2021).

2. CFG Strategies for Zero-shot Text-to-Speech: Condition Separation and Selective Guidance

In zero-shot TTS, context vectors are typically partitioned as $N = N_u \cup N_p$0 (speaker-identity condition) and $N = N_u \cup N_p$1 (text content condition). CFG-based strategies for zero-shot TTS generate two (or more) separate model derivatives at each denoising step. The classical formulation extends the standard CFG by extrapolating along the conditioned–unconditioned direction:

$N = N_u \cup N_p$2

Condition separation further allows assignment of distinct guidance weights:

$N = N_u \cup N_p$3

This mechanism enables direct trade-offs between preserving text content and cloning speaker characteristics. Mega-TTS 3 and related variants alternatively extrapolate based on conditional subtraction steps, anchoring the joint and marginal conditionals (Zheng et al., 24 Sep 2025).

3. Selective CFG Scheduling in Zero-shot Speech Synthesis

Performance in zero-shot TTS is highly dependent on the schedule with which CFG strategies are applied through the iterative denoising process. Selective guidance switches from standard CFG (operating on all conditions) during early steps to a mode that focuses on a subset of conditions (e.g., prioritizing $N = N_u \cup N_p$4 over $N = N_u \cup N_p$5) in later steps. Empirically, this is achieved by a threshold $N = N_u \cup N_p$6 (e.g., $N = N_u \cup N_p$7) on normalized denoising steps; standard CFG is used for $N = N_u \cup N_p$8, and a selective form (e.g., emphasizing speaker) for $N = N_u \cup N_p$9: $N_u$6 Delayed or selective CFG increases speaker similarity (SIM) while containing WER growth, notably with models leveraging robust text encoders such as CosyVoice 2, although model- and language-dependency is observed. For instance, CosyVoice 2 exhibits consistent SIM gains for both English and Mandarin, contrasting with the more variable behavior found in F5-TTS (Zheng et al., 24 Sep 2025).

4. Data Generation, Paraphrasing, and Coverage Optimization in Zero-shot S-CFG

In semantic parsing, the zero-shot S-CFG pipeline is structured as follows:

1. Exhaustive Grammar Enumeration: All utterance–program pairs up to depth $N_u$0 are systematically generated.
2. LM-based Canonical Data Selection: A pretrained GPT-2 ranks canonical utterances; top-$N_u$1 per template per depth are selected.
3. Paraphrase Generation and Filtering: BART-large, finetuned on paraphrase corpora, generates diverse linguistic forms. Paraphrases accepted into training are filtered by a neural parser for semantic equivalence.
4. Iterative Data Augmentation: Model–paraphrase–re-filter cycles are conducted to incrementally expand both linguistic and logical coverage.

The degree of match between real user data and synthesized data is quantified by language gap (LM perplexity) and logical gap (program coverage fraction). Empirical ablation demonstrates that idiomatic and compositional extensions to the S-CFG grammar decrease language gap and increase coverage. For example, supplementing with superlative productions reduces held-out LM perplexity (PPL↓ by 2–3) and pushes logical coverage above 90% (Yin et al., 2021).

5. Empirical Results and Performance Characteristics

Zero-shot S-CFG approaches consistently yield competitive accuracy relative to supervised counterparts:

• Semantic parsing: On Scholar, zero-shot S-CFG achieves 75.5% accuracy versus 79.7% for fully supervised models.
• Speech synthesis: Selective CFG raises speaker SIM from 0.675 to 0.682 (F5-TTS, $N_u$2) with only a marginal WER increase (0.020→0.022); in CosyVoice 2, text adherence is maintained with a SIM gain of 0.010.
• Spontaneous style TTS: LLaMA-based codec LMs with multi-stage CFG guidance produced top-1 mean opinion score (MOS) rankings in naturalness (3.80), and strong performance in quality and similarity in the CoVoC 2024 challenge (Zhou et al., 2024).

The table below summarizes key metrics from TTS selective CFG experiments:

Model	Language	SIM Gain	WER Δ	Notes
F5-TTS	English	+0.007	+0.002	def_text schedule, $N_u$3
F5-TTS	Mandarin	—	+0.002	No SIM gain
CosyVoice 2	English	+0.010	0	Consistent across languages
CosyVoice 2	Mandarin	+0.010	0

6. Model- and Modality-Specific Observations

• Transferability: Techniques adapted from CFG for image generation (e.g., weight schedules, perpendicular gradients) degrade performance in TTS contexts, highlighting the importance of modality-appropriate guidance design (Zheng et al., 24 Sep 2025).
• Robustness to Representation: Large LLM-based text encoders (e.g., CosyVoice 2) exhibit greater resilience to language-dependent effects and require less per-language tuning of hyperparameters versus compact models.
• Practical Tuning: The threshold $N_u$4 for selective guidance and weights $N_u$5 for CFG mixing are empirically set; small changes may impact stability or fidelity, especially in lower-capacity encoders.

7. Prospects and Recommendations

Zero-shot S-CFG approaches achieve strong performance in both semantic parsing and voice synthesis without requiring labeled data for new compositions or speakers. Selective and separated guidance are emerging as effective, inference-time-tunable levers for balancing content fidelity and style adherence, especially as systems scale. However, system effectiveness is contingent on modality, representation richness, and language, suggesting the continued necessity of architecture- and corpus-specific validation. Future TTS systems can incorporate selectable CFG strategies as simple inference-time switches, potentially broadening the scope of zero-shot controllability in learned generative models (Zheng et al., 24 Sep 2025, Yin et al., 2021, Zhou et al., 2024).