---
title: Macro-F1 Score Overview
url: https://www.emergentmind.com/topics/macro-f1-score
type: topic
---

# Macro-F1 Score Overview

The macro-F1 score is a class-decomposable metric for evaluation of classification systems, defined as the unweighted arithmetic mean of the per-class F₁ scores. Each class contributes equally, regardless of its prevalence, making macro-F1 particularly relevant for imbalanced datasets or fairness-sensitive tasks. It is employed across multi-class and multi-label settings as a primary indicator of system balance, penalizing neglect of rare or underrepresented classes just as severely as failure on dominant ones [2008.05756, 2205.09460, 2112.13756, 2404.16958, 2104.05700]. Multiple disciplines in machine learning—including medical coding, activity recognition, relation classification, and machine translation—have adopted macro-F1 as a standard or supplementary metric, often alongside micro-averaged alternatives to provide a fuller picture of model behavior.

## 1. Formal Definition and Mathematical Properties

Let $C$ denote the number of classes. For each class $c$, let $TP_c$, $FP_c$, and $FN_c$ be true positive, false positive, and false negative counts, respectively, when class $c$ is treated as the positive label. The per-class precision and recall are defined as:
\[
   P_c = \frac{TP_c}{TP_c + FP_c}, \qquad R_c = \frac{TP_c}{TP_c + FN_c}.
\]
The per-class F₁ score is the harmonic mean:
\[
   F1_c = \frac{2 P_c R_c}{P_c + R_c}.
\]
Macro-F₁ aggregates by averaging over all classes:
\[
   \mathrm{Macro\text{-}F1} = \frac{1}{C} \sum_{c=1}^C F1_c.
\]
In multi-label settings, this averaging can be over labels for each instance, or across the entire batch [1402.1892].

Alternative but non-equivalent forms exist, notably the harmonic mean of the macro-precision and macro-recall (sometimes called $\mathbb{F}_1$), but the class-decomposable mean of F1_c (sometimes labeled $\mathcal{F}_1$) is the most widely adopted for macro-F1 reporting [1911.03347, 2404.16958]. The difference between these variants can be up to 0.5 in extreme cases and is non-negligible even in moderate class imbalance [1911.03347].

## 2. Rationale and Comparative Role

Macro-F₁'s core rationale is its invariance to class frequency: it forces a model to uniformly balance precision and recall across all classes. In contrast:
- **Micro-F₁** aggregates all $TP$, $FP$, $FN$ globally before computing the score, thus emphasizing overall accuracy and favoring dominant classes [2008.05756, 2205.09460].
- **Weighted-F₁** weights per-class F1_c by class support (number of true instances), creating a compromise between macro and micro [2008.05756].
- **AUC** (area under the ROC curve) generally focuses on ranking and does not decompose into class-wise F₁s [2205.09460].

Scenarios in which macro-F₁ is preferred include those requiring equal attention to every class or robust evaluation under long-tail (rare) class distributions [2205.09460, 2104.05700, 2402.09445]. Micro-F₁, in contrast, may be more appropriate when performance on the majority class dominates real-world utility.

## 3. Applications and Experimental Usage

Macro-F₁ has seen widespread adoption in diverse fields:
- **Automated disease code assignment**: Macro-F₁ is used to evaluate models assigning ICD-10 codes, which present a pronounced class imbalance. In one study, macro-F₁ values of 0.83 (fastText), 0.84 (LSTM), and 0.88 (RoBERTa) were reported, with improvements attributed to deeper contextual representations and better capture of rare-code distinctions [2112.13756].
- **Fitness activity recognition**: Macro-F₁ measures balanced detection across multiple activity classes with strong class imbalance (e.g., arm opener is rare). Fusion of modalities and contrastive learning improved macro-F₁ from 81.49% (IMU baseline) to 84.71% (IMU+contrastive) and 89.57% (sensor fusion) [2402.09445].
- **Machine translation evaluation**: Macro-F₁ over word types emphasizes strict adequacy by giving rare words the same impact as frequent function words, escaping Zipf’s law bias seen in micro-F₁, BLEU, or chrF. Macro-F₁ aligns better with human semantic adequacy judgments and downstream cross-lingual IR task success [2104.05700].
- **Relation classification** and other NLP tasks: Macro-F₁ exposes long-tail performance weaknesses and enables fairer comparison of models under highly skewed label distributions [2205.09460].

## 4. Behavior Under Class Imbalance and Limitations

Macro-F₁'s equal weighting means that poor performance on rare classes can precipitously reduce the overall score, making it sensitive to a single misclassification in a minority class. This property is beneficial when rare outcomes are critical but can introduce instability if such classes are small or, more problematically, have noisy or inconsistent labels, as seen in ICD-10 coding or fitness activity detection [2112.13756, 2402.09445]. Systems optimized solely for macro-F₁ may sacrifice utility on frequent classes and can be penalized if evaluation data contain annotation inconsistencies [2112.13756].

Macro-F₁ is not invariant to prevalence shifts: artificially altering class distribution in the test set will alter macro-F₁, even if model performance per class remains unchanged [2404.16958]. This contrasts with micro-F₁, which is strictly proportional to the proportion of correctly classified examples.

## 5. Thresholding, Optimization, and Reporting Practices

For probabilistic classifiers, threshold selection to maximize per-class F₁ is nontrivial. In multi-label settings, the optimal threshold for each label is half its maximal attainable F₁ if probabilities are well calibrated [1402.1892]. However, with uninformative classifiers or in rare class regimes, this can yield pathological all-positive predictions for rare classes, which may not be desirable in practice.

Macro-F₁ must be reported alongside class support statistics and, ideally, per-class breakdowns to ensure transparency. When macro-F₁ diverges significantly from weighted or micro-F₁, this indicates issues with model coverage of rare classes [2008.05756, 2205.09460]. In evaluation settings, it is essential to explicitly specify the precise macro-F₁ formula used to avoid ambiguity [2404.16958, 1911.03347].

## 6. Variants, Controversies, and Best Practices

The literature documents several aggregation ambiguities:
- The standard, class-decomposable macro-F₁ (mean of per-class F₁’s, $\mathcal{F}_1$).
- The less common harmonic mean of macro-precision and macro-recall ($\mathbb{F}_1$), which is not class-decomposable and can yield inflated scores in certain error distributions [1911.03347, 2404.16958].

Best practice recommendations include unambiguously stating the metric and its formula, always motivating its choice based on deployment needs and class importance, and supplementing macro-F₁ with micro-F₁ and either per-class or weighted breakdowns [2404.16958, 2008.05756]. Researchers should be wary of tuning solely for macro-F₁ without monitoring shifts in overall accuracy or performance on dominant classes.

## 7. Extensions and Intermediate Weighting Schemes

Recent work explores intermediate weighting strategies between macro and micro, such as "dodrans" (weighting by $n_i^{3/4}$) and entropy-based schemes that balance sensitivity to rare classes against dominance by large ones [2205.09460]. These alternatives offer nuanced evaluation lenses and can better reflect application-specific preferences. Nonetheless, macro-F₁ remains the canonical choice in settings demanding maximal class parity, especially when rare or tail phenomena are of intrinsic or regulatory concern.

---

**Key References:**

- “Metrics for Multi-Class Classification: an Overview” [2008.05756]  
- “Macro F1 and Macro F1” [1911.03347]  
- “A Closer Look at Classification Evaluation Metrics and a Critical Reflection of Common Evaluation Practice” [2404.16958]  
- “Secondary Use of Clinical Problem List Entries for Neural Network-Based Disease Code Assignment” [2112.13756]  
- “Why only Micro-F1? Class Weighting of Measures for Relation Classification” [2205.09460]  
- “Macro-Average: Rare Types Are Important Too” [2104.05700]  
- “iMove: Exploring Bio-impedance Sensing for Fitness Activity Recognition” [2402.09445]  
- “Thresholding Classifiers to Maximize F1 Score” [1402.1892]

Source: https://www.emergentmind.com/topics/macro-f1-score