Pacifism Score: Quantifying Peace in Media

• Pacifism Score (P-Score) is a continuous index that quantifies peace levels in news media using NLP and machine learning.
• It employs advanced preprocessing and L₂-regularized logistic regression to convert word frequencies into calibrated scores from 0 to 100.
• Derived from diverse peace indices and extensive corpora, the P-Score offers interpretable insights while facing language and preprocessing challenges.

The Pacifism Score (P-Score) is a quantitative, continuous index ranging from 0 to 100 that measures the degree of peace signaled by the language in news media from a given country. It is derived using NLP and machine learning on large corpora of news articles to operationalize linguistic characteristics associated with peace or conflict. The P-Score provides a data-driven alternative to traditional peace indices by leveraging word usage patterns to differentiate between lower-peace and higher-peace polities and to interpolate values for those at intermediate-peace status. The methodology underpinning the P-Score was first formalized by Liebovitch et al. (2023) (Liebovitch et al., 2023).

1. Basis: Peace Indices, Corpus Construction, and Preprocessing

The ground truth for peace levels is established using an ensemble of established indices: Global Peace Index (GPI), Positive Peace Index (PPI), World Happiness Index (WHI), Fragile States Index (FSI), and Human Development Index (HDI). Each is averaged over a ten-year window (2010–2019) and rescaled linearly to [0,100] across 18 countries.

The textual corpus comprises 724,000 English-language news articles drawn from these countries. Preprocessing includes tokenization, stop-word removal, named-entity removal (covering people, locations, and companies), and optionally stemming or lemmatization. The vocabulary $V$ is formed by taking the union of the top 300 words from each country, yielding $N=767$ unique terms.

For each document $d$, the term-frequency vector $n_j(d)$ is constructed, where $n_j(d)$ is the raw count of term $j$ in $d$, $j=1,\dots,N$. Feature vectors are normalized as follows:

$$x_j(d) = \frac{n_j(d)}{\sum_{k=1}^N n_k(d)}, \quad \sum_{j=1}^N x_j(d) = 1$$

Alternatively, tf–idf features may be computed:

$$\mathrm{idf}_j = \log \left(\frac{|D|+1}{\mathrm{df}_j+1}\right) + 1, \quad \mathrm{tfidf}_j(d) = x_j(d) \cdot \mathrm{idf}_j, \quad x(d) = \frac{\mathrm{tfidf}(d)}{\|\mathrm{tfidf}(d)\|_2}$$

where $N=767$0 is the total number of documents and $N=767$1 is the number of documents containing term $N=767$2.

2. Labeling Regime: Extreme-Groups Approach

To create ground-truth labels, countries are ranked and partitioned into thirds for each index after rescaling. The “extreme-groups” approach defines:

• Class 0 (lower-peace): Country is in the bottom third on at least 3 out of 5 indices.
• Class 1 (higher-peace): Country is in the top third on at least 3 out of 5 indices.
• Class 2 (intermediate-peace): All others.

Model training and validation use only classes 0 and 1, discarding intermediate (class 2) countries at this phase.

Class	Index Ranking Criteria	Used in Model Training
Lower-peace (0)	Bottom third on ≥3 indices	Yes
Higher-peace (1)	Top third on ≥3 indices	Yes
Intermediate (2)	None of the above	No

3. Model Specification: L₂-Regularized Logistic Regression

The classification model is L₂-regularized binary logistic regression (e.g., scikit-learn’s LogisticRegression(penalty='l2', C=1.0, solver='lbfgs')). Given a set of training pairs $N=767$3, $N=767$4, the model learns weights $N=767$5 and bias $N=767$6 by minimizing:

$N=767$7

where

$N=767$8

and $N=767$9. The minimization is performed via Newton–Raphson (lbfgs) or any suitable convex solver.

4. Training and Validation Procedure

Two main training-validation regimes are considered:

• Random split: 80% of class-labeled documents for training, 20% for testing.
• Leave-one-country-out cross-validation: For each fold, train on all documents from nine of the ten “extreme” countries (class 0 and 1), test on the held-out country.

Weights $d$0 and bias $d$1 are fit to the selected documents by optimizing the regularized loss.

Model performance is assessed using standard classification metrics (accuracy, precision, recall, F₁ score):

• Logistic regression (leave-one-country-out): accuracy = 1.000, precision = 1.000, recall = 1.000, F₁ = 1.000
• Random forest (100 trees): accuracy ≈ 0.960, precision ≈ 0.965, recall ≈ 0.960, F₁ ≈ 0.963

5. From Model Outputs to the Pacifism Score (P-Score)

To map the model’s probabilistic output to a continuous peace index, for any input vector $d$2 (document- or country-level):

$d$3

The continuous P-Score is then defined as:

$d$4

A value of P-Score $d$5 corresponds to $d$6 (“certainly low peace”), while P-Score $d$7 corresponds to $d$8 (“certainly high peace”). The resulting P-Score is thus interpretable as a calibrated probability scaled to [0,100]. No post-hoc calibration was performed, as the logistic regression’s outputs were well-calibrated.

6. Application, Generalization, and Interpretation

After training on class 0 and 1 (“extreme”) countries, the resulting model is used to compute P-Scores for all countries, including those labeled as class 2 (intermediate-peace). For these, P-Scores tend to fall between the lower-peace and higher-peace ranges, even though the model was not explicitly trained or scored with their data. This provides a mechanism for continuous quantification of peace strictly from language data.

The framework is designed for flexibility: once weights $d$9 and bias $n_j(d)$0 are learned, any set of news articles (from a single country, time period, or other grouping) can be summarized by its normalized word-frequency or tf–idf vector and mapped into a P-Score without further model adjustments.

7. Significance and Limitations

The P-Score operationalizes the connection between language and social peacefulness, providing a data-driven, quantitative index derived from textual features rather than solely relying on traditional, multidimensional peace indices. This approach demonstrates that NLP and machine learning can produce interpretable, calibrated societal metrics grounded in observable communication (Liebovitch et al., 2023).

A notable limitation is the reliance on English-language news and standard preprocessing: differences in media structure or press freedom, as well as omission of named entities, may affect generalizability. The selection of vocabulary and the extreme-groups labeling strategy may limit immediate application beyond the set of countries and temporal window studied in Liebovitch et al. (2023). A plausible implication is that further research is required to evaluate cross-linguistic and cross-domain robustness, as well as the sensitivity to feature engineering and index selection.

Despite these considerations, the Pacifism Score establishes a reproducible and interpretable methodology for measuring peace-related linguistic signals in large-scale text corpora.