Sufficiency of DeBERTa-base for Japanese G2P under larger data scales

Determine whether the DeBERTa-base backbone remains sufficiently rich for Japanese grapheme-to-phoneme conversion as the scale of training data increases, while preserving a favorable balance between performance and computational efficiency.

Background

The paper compares Japanese G2P models using DeBERTa-tiny, DeBERTa-base, and DeBERTa-large backbones. Reducing the backbone from DeBERTa-base to DeBERTa-tiny degrades all evaluation metrics, whereas scaling to DeBERTa-large produces only a marginal improvement in sentence phoneme error rate and slightly worsens accuracy and target-word phoneme error rate.

Based on these results, the authors conjecture that DeBERTa-base already supplies sufficiently rich contextual representations at the current training-data scale and offers an advantageous trade-off between accuracy and computational cost. The unresolved aspect is whether this sufficiency and trade-off continue to hold beyond the current data scale.