---
title: 'BanglaMultiHate: Multi-Task Hate Speech Benchmark'
url: https://www.emergentmind.com/topics/banglamultihate
type: topic
---

# BanglaMultiHate: Multi-Task Hate Speech Benchmark

Searching arXiv for BanglaMultiHate and closely related Bangla hate-speech resources to ground the article.
{"query": "\"BanglaMultiHate\" OR \"LLM-Based Multi-Task Bangla Hate Speech Detection: Type, Severity, and Target\"","max_results":5}
BanglaMultiHate is a manually annotated Bangla hate-speech benchmark designed as a **multi-task** resource rather than a simple hate/non-hate corpus. Each comment is labeled along three coordinated dimensions—**type of hate**, **severity of hate**, and **target of hate**—so that a system can model what kind of hate is expressed, how intense it is, and who or what it is directed at [2510.01995]. The dataset was introduced as **the first multi-task Bangla hate-speech dataset** and as **one of the largest manually annotated Bangla hate-speech corpora**, with **50,746** filtered comments drawn from Bangla YouTube news discourse [2510.01995].

## 1. Definition and conceptual scope

BanglaMultiHate was created to address a limitation in earlier Bangla hate-speech research: most prior resources emphasized binary hate/offense detection, or at most one additional label dimension, which constrained both moderation research and fine-grained linguistic analysis [2510.01995]. Its organizing idea is that Bangla hate speech is not adequately captured by a single yes/no label, because moderation decisions often depend on three separate but related judgments: whether the text contains hate, what subtype it instantiates, and what entity is targeted.

The dataset is therefore best understood as a **multi-output classification benchmark**. It is not a conventional within-task multi-label resource in which one comment receives an arbitrary subset of hate labels. Instead, each comment receives **one label per task dimension**: one hate-type label, one severity label, and one target label [2510.01995]. This distinction is important because BanglaMultiHate differs structurally from Bangla multi-label corpora such as BOISHOMMO, where comments can receive several simultaneous hate categories, and from BanTH, where hateful transliterated Bangla comments receive one or more target-group labels [2504.08408][2410.13281].

A further defining property is its **hierarchical annotation logic**. If the **type** label is `None`, then **severity** is automatically `Little to None` and **target** is automatically `None` [2510.01995]. This creates an explicit dependency between tasks and makes the benchmark closer to operational moderation logic than a collection of independent flat labels.

## 2. Corpus construction and topical coverage

The corpus was collected from **YouTube** via the **YouTube API**, primarily from **Somoy TV**, a major Bangla news channel [2510.01995]. The raw crawl yielded **approximately 55,000 comments** associated with Bangla news videos spanning **19 categories** and **120 sub-topics**. The 19 top-level categories are **Business, Celebrities, Disaster, Entertainment, Fashion, Geopolitics, Health, History, International, Lifestyle, Literature, Miscellaneous, National, Opinion, Politics, Religion, Science, Sports,** and **Technology** [2510.01995].

The dataset was filtered before annotation. The reported filtering operations removed comments containing **only emojis**, removed **URLs**, removed **duplicate entries**, and excluded **Banglish comments**, meaning Bangla written in the English alphabet [2510.01995]. After filtering, the final corpus size became **50,746** comments [2510.01995].

This construction yields a specific domain profile. BanglaMultiHate is broad in **topic coverage**, but narrow in **platform and genre**, because it centers on Bangla YouTube news comments rather than general social media, private messaging, or cross-platform discourse. The exclusion of Banglish also makes it complementary to transliterated resources such as BanTH rather than a substitute for them [2410.13281]. Most comments are short: the paper notes that the most common length bucket is **\(\leq 20\) words**, which is consistent with YouTube-comment discourse [2510.01995].

## 3. Annotation framework and reliability

Annotation was performed by **35 native Bangla-speaking undergraduate students**, including **both male and female annotators**, under expert supervision [2510.01995]. Each comment was **independently annotated by three annotators**. The workflow combined explicit guidelines, periodic quality checks on randomly selected samples, feedback sessions, majority agreement, and **consensus meetings** for unresolved cases [2510.01995].

The three task dimensions are as follows. **Type of Hate** has six classes: `Abusive`, `Sexism`, `Religious Hate`, `Political Hate`, `Profane`, and `None`. **Severity of Hate** has three classes: `Little to None`, `Mild`, and `Severe`. **Target of Hate** has five classes: `Individual`, `Organization`, `Community`, `Society`, and `None` [2510.01995].

The paper reports **Fleiss’ Kappa** for each task: **0.71** for type, **0.84** for severity, and **0.79** for target [2510.01995]. Using the interpretation cited in the paper, type and target show **substantial agreement**, while severity reaches **almost perfect agreement**. The relative ordering is informative. Type is the hardest annotation dimension, plausibly because six-way hate-type distinctions require finer semantic judgments than deciding whether a comment is mild or severe, while target requires inference about the social object of hostility.

The annotation design also clarifies a common misconception. BanglaMultiHate is “multi-task,” but not “multi-label” in the same sense as BOISHOMMO or BanTH. A comment does not receive several hate-type labels simultaneously; instead it receives one type label, one severity label, and one target label [2510.01995][2504.08408][2410.13281].

## 4. Label space and statistical profile

BanglaMultiHate is strongly imbalanced across all three tasks. The imbalance is especially visible in the dominance of `None` and in the rarity of `Sexism` and `Religious Hate` within the type task [2510.01995].

| Dimension | Labels | Total counts |
|---|---|---|
| Type | None, Abusive, Political Hate, Profane, Religious Hate, Sexism | 28,603; 11,637; 6,021; 3,382; 933; 170 |
| Severity | Little to None, Mild, Severe | 33,643; 9,763; 7,340 |
| Target | None, Individual, Organization, Community, Society | 30,347; 7,972; 5,582; 3,732; 3,113 |

The dataset split is **70% train**, **10% development**, and **20% test**, with exact sizes **35,522 / 5,024 / 10,200** produced by stratified sampling [2510.01995]. Within the type task, the train split contains **8,212 Abusive**, **4,227 Political Hate**, **2,331 Profane**, **676 Religious Hate**, **122 Sexism**, and **19,954 None** labels [2510.01995]. For severity, the train split contains **23,489 Little to None**, **6,853 Mild**, and **5,180 Severe** labels. For target, it contains **21,190 None**, **5,646 Individual**, **3,846 Organization**, **2,635 Community**, and **2,205 Society** labels [2510.01995].

The corpus also includes cross-task interaction analyses. The paper reports that **Abusive** is the most common hateful type and peaks at **mild** severity, **Profane** is concentrated in **severe** cases, and **Individuals** and **organizations** are the primary targets overall, with **Abusive** especially concentrated on **individuals** [2510.01995]. These interactions make BanglaMultiHate more than a flat label inventory; they enable study of how hate subtype, intensity, and target co-vary in Bangla news-comment discourse.

## 5. Benchmarking, shared-task use, and empirical findings

The benchmark study compares **majority** and **random** baselines, a classical **SVM** with **TF-IDF 1–5-grams** and **\(C=1\)**, a monolingual **BanglaBERT**, and instruction-tuned LLMs in both zero-shot and LoRA-fine-tuned form [2510.01995]. The LLMs are **Llama-3.2-3B-Instruct** and **Qwen3-4B-Instruct-2507**. The LoRA configuration uses **Adam**, **FP16**, learning rate **\(2 \times 10^{-4}\)**, **\(\alpha = 16\)**, rank **\(r = 64\)**, maximum sequence length **512**, batch size **8**, and **3 epochs** [2510.01995]. BanglaBERT is fine-tuned for **3 epochs**, with **10 runs using different random seeds**, and the best development model is selected [2510.01995].

Across all three tasks, **BanglaBERT is the strongest model** in the paper. It achieves **0.712** micro-F1 for hate type, **0.722** micro-F1 for severity, and **0.715** micro-F1 for target [2510.01995]. The SVM remains competitive, reaching **0.609**, **0.672**, and **0.629** micro-F1 on type, severity, and target respectively [2510.01995]. Zero-shot LLM performance is much weaker: for example, zero-shot Llama-3.2-3B-Instruct obtains **0.275** on type and **0.340** on target, while zero-shot Qwen3 obtains **0.520** on type and **0.434** on target [2510.01995]. LoRA substantially improves both models, with Llama-3.2-3B-Instruct rising to **0.620**, **0.685**, and **0.610** micro-F1 on type, severity, and target, but still trailing BanglaBERT [2510.01995].

The paper’s principal empirical conclusion is that **culturally and linguistically grounded pretraining remains critical** for Bangla hate-speech analysis [2510.01995]. In other words, adaptation helps general LLMs, but does not erase the advantage of a Bangla-specific encoder.

BanglaMultiHate also became the basis of the **BLP-2025 Task 1: Bangla Multi-task Hate Speech Identification** shared-task ecosystem. A system paper by the Retriv team explicitly cites BanglaMultiHate as “the first multi-task Bangla hate speech dataset jointly modeling type, severity, and target” and reports official blind-test shared-task scores of **72.75%** micro-F1 for hate-type classification, **72.69%** micro-F1 for target-group identification, and **72.62%** weighted micro-F1 for joint detection [2511.07304]. Another shared-task paper, by Gradient Masters, reports **73.23%** micro-F1 on type classification and **73.28%** on target classification, with leaderboard positions of **6th** and **3rd** respectively [2511.18324]. Shared-task system papers further indicate that the original **5,024-instance development portion** was operationalized as a **2,512** public Dev set plus a **2,512** blind Dev Test split in competition settings [2511.07304][2511.18324].

## 6. Relation to earlier Bangla hate-speech resources

BanglaMultiHate occupies a distinct position within Bangla hate-speech research because earlier datasets generally emphasized different task formulations. The **30,000-comment** dataset of “Hate Speech detection in the Bengali language” is a broad **binary** benchmark with hate vs not-hate labels and seven source-topic categories that function only as metadata, not as hate subclasses [2012.09686]. **HS-BAN** is also binary, although it is comparatively strong in annotation criteria and agreement reporting, with **50,314** comments and **Fleiss’ kappa = 0.658** [2112.01902]. **BD-SHS** introduces a hierarchical design with **hate/non-hate**, **target**, and **type**, but its type inventory is narrower and its structure differs from BanglaMultiHate’s explicit type–severity–target triad [2206.00372].

At the other end of the design spectrum are corpora that are multi-label within a single task. **BOISHOMMO** contains **2,499** Facebook comments with overlapping labels such as **Race, Behaviour, Physical, Class, Religion, Disability, Nationality/Ethnicity, Gender, Sexual Orientation,** and **Political Statement** [2504.08408]. **BanTH** contains **37,350** transliterated Bangla YouTube comments, with binary hate labeling followed by one-or-more target-group labels such as **Political, Religious, Gender, Personal Offense, Abusive/Violence, Origin,** and **Body Shaming** [2410.13281]. BanglaMultiHate differs from both by remaining in **native Bangla script**, excluding Banglish, and using **one label per dimension** rather than within-dimension multi-label assignment [2510.01995].

This positioning makes BanglaMultiHate especially useful when the research objective is **structured moderation reasoning** rather than only hate detection or target attribution. It is broader than a binary detector, but more constrained and operationally cleaner than a free-form multi-label ontology.

## 7. Limitations, interpretation, and significance

Several limitations are explicit in the dataset paper. BanglaMultiHate is drawn from a **single platform genre**—YouTube news comments, primarily from Somoy TV—so cross-domain generalization is not guaranteed [2510.01995]. It excludes **Banglish/transliterated Bangla**, which strengthens native-script consistency but limits direct use for mixed-script moderation [2510.01995]. The class imbalance is severe, most notably **Sexism = 170** and **Religious Hate = 933** within the full type inventory, while `None` dominates all three tasks [2510.01995]. The paper also notes that the dataset does not yet include **reasoning annotations**, and that hateful material poses obvious annotator-exposure and subjectivity risks [2510.01995].

These constraints shape how the benchmark should be interpreted. High micro-F1 can coexist with weak minority-class behavior; this is one reason the paper itself notes the need for richer reporting such as macro-F1 and per-class analysis in future evaluation [2510.01995]. A plausible implication is that the rarest labels—especially `Sexism`, `Religious Hate`, and `Society`—will remain bottlenecks unless future versions expand those classes or use targeted collection.

Despite these limitations, BanglaMultiHate marks a substantial shift in Bangla hate-speech benchmarking. It formalizes hate analysis as a coordinated prediction problem over **type**, **severity**, and **target**; it provides a large manually annotated native-script corpus; and it demonstrates that, in Bangla, **monolingual culturally grounded pretraining still outperforms generic zero-shot or lightly adapted LLMs** [2510.01995]. In the Bangla hate-speech literature, its importance lies less in novelty of model architecture than in establishing a benchmark where the structure of hateful expression is itself the object of study.

Source: https://www.emergentmind.com/topics/banglamultihate