The Multilingual Amazon Reviews Corpus (2010.02573v1)

Abstract: We present the Multilingual Amazon Reviews Corpus (MARC), a large-scale collection of Amazon reviews for multilingual text classification. The corpus contains reviews in English, Japanese, German, French, Spanish, and Chinese, which were collected between 2015 and 2019. Each record in the dataset contains the review text, the review title, the star rating, an anonymized reviewer ID, an anonymized product ID, and the coarse-grained product category (e.g., 'books', 'appliances', etc.) The corpus is balanced across the 5 possible star ratings, so each rating constitutes 20% of the reviews in each language. For each language, there are 200,000, 5,000, and 5,000 reviews in the training, development, and test sets, respectively. We report baseline results for supervised text classification and zero-shot cross-lingual transfer learning by fine-tuning a multilingual BERT model on reviews data. We propose the use of mean absolute error (MAE) instead of classification accuracy for this task, since MAE accounts for the ordinal nature of the ratings.

• The paper presents the Multilingual Amazon Reviews Corpus (MARC), a new large-scale dataset of Amazon reviews in six languages with defined splits to address the need for robust resources in multilingual text classification research.
• Using a fine-tuned multilingual BERT model and mean absolute error (MAE), researchers established baseline classification performance across languages and demonstrated promising zero-shot transfer learning capabilities with the dataset.
• MARC provides an accessible resource via AWS Open Datasets, addressing deficiencies in existing corpora and supporting future work in improving multilingual NLP models and their practical applications in various industries.

Analyzing the Multilingual Amazon Reviews Corpus for Enhanced Multilingual Text Classification

The paper "The Multilingual Amazon Reviews Corpus" presents the development and application of a new dataset (MARC) designed explicitly for multilingual text classification tasks. This corpus comprises Amazon reviews across six languages: English, Japanese, German, French, Spanish, and Chinese, collected between 2015 and 2019. This balanced dataset addresses the need for large-scale multilingual corpora to support research in multilingual NLP and includes well-defined training, development, and test splits to facilitate reproducible research.

Corpus Composition and Characteristics

The paper outlines the rigorous data preparation process, which includes language detection to ensure reviews are correctly associated with the intended language and steps to anonymize product and reviewer IDs. Language detection achieved an impressively low misclassification rate across the six languages tested. Reviews that did not meet set inclusion criteria, including overly short lengths or uncommon vocabulary, were excluded from the dataset to maintain data quality. The dataset provides 200,000 training reviews per language and is designed with a class balance across the five possible star ratings, reducing potential class imbalance issues.

The MARC dataset also accounts for variation in product category distribution across languages, which is significant, particularly in Chinese reviews that heavily favor book categories. This characteristic might influence the overall effectiveness of models trained with the corpus when applied to specific tasks.

Methodology and Baseline Results

To evaluate the dataset's applicability, the researchers conducted experiments using a fine-tuned multilingual BERT (mBERT) model, measuring performance using mean absolute error (MAE) as the primary metric due to the ordinal nature of star ratings. Supervised tasks showed varying degrees of accuracy across languages, with average MAE values indicating practical baseline performance.

Furthermore, in zero-shot transfer learning scenarios, results displayed promising cross-lingual capabilities, suggesting the dataset's effectiveness in training models that can generalize from one language to another without direct supervision in the target language. The paper emphasizes the utility of the target language development set for selecting model checkpoints, avoiding variability associated with relying on source language data alone.

Discussion and Future Implications

The creation of MARC marks an essential step in providing robust resources for multilingual text classification research. The dataset addresses critical deficiencies in existing corpora, such as size limitations, unclear language categorization, and accessibility issues. Given its extensive scale and accessibility via AWS Open Datasets, MARC is poised to become a critical asset in the development and evaluation of multilingual NLP models.

With comprehensive baseline results supplied by the paper, future research can build upon these established metrics, potentially improving text classification accuracy through refined techniques or advanced models. This corpus paves the way for practical applications in industries dealing with vast quantities of multilingual text, such as e-commerce platforms, automated customer feedback analysis, and multilingual social media monitoring.

The paper does not sensationalize the corpus's potential impact but leaves open questions regarding enhancing zero-shot performance or adapting models trained on MARC for emerging NLP architectures. As NLP continues evolving, MARC's contributions could spur breakthroughs in understanding cross-lingual nuances and tailoring models for robust multilingual environments. Future work might focus on expanding language options, integrating more granular data insights, or refining existing models to leverage the corpus more effectively in scenarios requiring precise sentiment and opinion mining across languages.