What is the Best Way for ChatGPT to Translate Poetry?

Abstract: Machine translation (MT) has historically faced significant challenges when applied to literary works, particularly in the domain of poetry translation. The advent of LLMs such as ChatGPT holds potential for innovation in this field. This study examines ChatGPT's capabilities in English-Chinese poetry translation tasks, utilizing targeted prompts and small sample scenarios to ascertain optimal performance. Despite promising outcomes, our analysis reveals persistent issues in the translations generated by ChatGPT that warrant attention. To address these shortcomings, we propose an Explanation-Assisted Poetry Machine Translation (EAPMT) method, which leverages monolingual poetry explanation as a guiding information for the translation process. Furthermore, we refine existing evaluation criteria to better suit the nuances of modern poetry translation. We engaged a panel of professional poets for assessments, complemented evaluations by using GPT-4. The results from both human and machine evaluations demonstrate that our EAPMT method outperforms traditional translation methods of ChatGPT and the existing online systems. This paper validates the efficacy of our method and contributes a novel perspective to machine-assisted literary translation.

• The paper introduces Explanation-Assisted Poetry Machine Translation (EAPMT) to enhance ChatGPT's ability to translate poetry.
• It utilizes a two-step process combining detailed explanations with translation to preserve poetic structure, emotions, and cultural context.
• Evaluations on the ModePoem dataset using metrics like SacreBLEU and BERTScore show significant improvements, particularly with GPT-4.

Summary of "What is the Best Way for ChatGPT to Translate Poetry?"

The paper investigates the potential of ChatGPT for the challenging task of translating English poetry into modern Chinese poetry. It acknowledges the inherent difficulties of machine translation (MT) when applied to poetry, a domain laden with nuanced linguistic and cultural subtleties. The study introduces an innovative Explanation-Assisted Poetry Machine Translation (EAPMT) method designed to overcome existing inadequacies in ChatGPT's translation capabilities.

Introduction to the Research Challenge

Translating poetry involves more than substituting words between two languages; it requires the preservation of the poem's essence, emotions, and aesthetic structure. Modern poetry notably deviates from classical forms, emphasizing irregular rhythmic structures and diverse genres. Consequently, traditional MT approaches, which focus on structured forms and rhythmic adherence, prove insufficient for the fluid nature of modern poetry.

Explanation-Assisted Poetry Machine Translation

Framework and Methodology

EAPMT builds upon the premise that understanding the monolingual context and nuances can guide better translations. The two-step process involves generating an explanation of the source poem, which is then used to inform the translation process. This explanation comprises literal content, thematic elements, and cultural or historical contexts critical for conveying nuanced poetic meaning accurately.

Figure 1: Comparison between the framework of the traditional translation method and the proposed Explanation-Assisted Poetry Machine Translation (EAPMT).

Evaluation Metrics and Dataset Construction

The ModePoem dataset, consisting of 400 bilingual poems, serves as the foundation for evaluating ChatGPT's translation capabilities. The dataset emphasizes high-standard professional translations, ensuring quality benchmark comparisons. Both human assessments and machine metrics like SacreBLEU, BERTScore, and COMET were employed to quantify translation performance against refined evaluation criteria tailored for modern poetry.

Detailed Findings and Analytical Insights

Optimal Prompt Design

Successful translation using LLMs hinges on effective prompt design. The study explored various prompts, both human-crafted and model-generated, to identify those maximizing translation fidelity. Results indicated that prompts designed with specificity in handling poetic structures significantly enhanced translation performance, particularly for GPT-4.

Performance Gains with EAPMT

Analysis of GPT-3.5 and GPT-4 showed marked improvements with the EAPMT approach. For instance, improvements in "Accuracy" and "Line-breaking" were noted, reflecting better adherence to poetic constructs and aesthetic delivery. Human judges, all seasoned poets, confirmed the superiority of EAPMT translations, further supported by machine evaluation metrics.

Limitations and Poeticity Evaluation

Despite advancements, challenges remain in achieving complete poetic fidelity, especially concerning intricate emotional and cultural layers inherent to poetry. An experiment using GPT models to assess poeticity affirmed the difficulty LLMs face in completely capturing poetic essence, highlighting ongoing areas for methodological refinement.

Conclusion and Future Directions

The paper posits EAPMT as a significant enhancement over conventional translation methodologies for poetry, especially evident in complex domains like modern Chinese poetry. It opens pathways for further refinements, such as integrating expert-led corrections in the explanation phase to further boost translation accuracy. Future explorations may extend EAPMT's application to diverse poetic forms across broader linguistic landscapes.

The proposed method assumes significant importance given the nascent yet growing role of AI in creative fields, illustrating how AI can augment human creativity rather than replace it, by facilitating nuanced literary translations inaccessible by existing automated systems alone.