HebID: Hebrew Identity Detection Dataset

• HebID is a publicly released multilabel corpus that captures affirmative social identity expressions in Hebrew political discourse using 12 fine-grained labels.
• The corpus anchors its labels in national survey data, enabling comparative analysis between elite speech and public self-reports.
• It benchmarks encoder and seq2seq LLM models, with Hebrew-tuned models outperforming multilingual alternatives in macro F1 performance.

Searching arXiv for the specified HebID paper and closely related context. HebID is a corpus, benchmark, and analytical framework for detecting social identities in Hebrew-language political text. It was introduced as the first publicly released multilabel corpus of Hebrew-language political text and the first multilabel Hebrew corpus for social identity detection, using 5,536 manually annotated sentences drawn from Israeli politicians’ Facebook posts collected between December 2018 and April 2021. Its label space is grounded in a 12-wave national panel survey of identity salience, and its empirical program combines annotation, model comparison, and downstream analysis of political discourse across Facebook posts, Knesset speeches, and survey self-reports (Mor-Lan et al., 21 Aug 2025).

1. Motivation and problem formulation

HebID was designed to address three shortcomings in prior NLP resources for identity detection: English-centricity, coarse granularity, and single-label design. Existing corpora were described as predominantly English-focused, often restricted to broad classes such as party membership or ethnicity, and typically structured so that each example receives only one label even when political language invokes multiple identities simultaneously (Mor-Lan et al., 21 Aug 2025).

Within this framework, social identities are treated as politically consequential linguistic categories through which actors mobilize supporters, frame debates, and shape public discourse. HebID therefore defines the task as multilabel detection of identity expressions in Hebrew political text rather than as single-category classification. Its stated goals were to develop a publicly released multilabel corpus, define twelve fine-grained identities that exceed a 5% salience threshold in survey data, benchmark encoder-only and seq2seq LLM models, test cross-genre generalization from social media to parliamentary speech, and compare elite discourse with citizens’ identity priorities (Mor-Lan et al., 21 Aug 2025).

A central design choice is that the target labels are not derived solely from textual regularities. They are anchored in citizens’ self-reported identity salience from national survey data. This makes HebID simultaneously a computational resource and a survey-grounded operationalization of politically salient identity language.

2. Corpus construction and annotation design

The source material consists of Facebook posts by Israeli politicians, including members of Knesset, party accounts, and candidates, collected from December 2018 through April 2021. The raw corpus contains 64,174 posts, segmented into 375,718 sentences. From this material, 5,536 Hebrew sentences were annotated and split into 70% train, 15% validation, and 15% test (Mor-Lan et al., 21 Aug 2025).

The annotation schema was derived from a 12-wave national panel survey of Jewish citizens of Israel with $N = 1{,}769$. In that survey, respondents chose up to three identities from a predefined list of 28. The twelve most salient identities, each selected by more than 5% of respondents, became the task labels:

Rightist, Leftist, Conservative, Liberal, Democrat, Capitalist, Socially-oriented, Zionist, Security-oriented, Honest (anti-corruption), Ultra-Orthodox, and Palestinians and Arab Citizens of Israel (Mor-Lan et al., 21 Aug 2025).

The codebook emphasized positive expressions only. Praise of welfare-state policies could be labeled “Socially-oriented,” support for free markets could be labeled “Capitalist,” and affirmative references to IDF service or national symbols could be labeled “Zionist.” Negative or critical mentions, such as opposition to liberalism, were explicitly not labeled as that identity (Mor-Lan et al., 21 Aug 2025). This restriction is methodologically important because the task is not generic topic detection; it is detection of affirmative identity invocation.

Annotation quality was assessed on a 304-sentence overlap labeled by two expert annotators. The reported mean percent agreement was 96.1%, and the mean Cohen’s $\kappa$ was 0.77; disagreements were adjudicated jointly (Mor-Lan et al., 21 Aug 2025). The label distribution is nontrivial: per-identity positive counts range from 129 for “Ultra-Orthodox” to 703 for “Rightist,” with a mean of 413. Of the 5,536 sentences, 37.5% have no positive labels, 41.1% have exactly one label, and 21.4% have two or more labels (Mor-Lan et al., 21 Aug 2025). These figures confirm that multilabel structure is not peripheral but intrinsic to the corpus.

3. Modeling framework and representation choices

HebID evaluates three model families. The first is multilabel encoder models: mBERT (110M parameters), AlephBERT (110M), HeRo (125M), DictaBERT-base (184M), and DictaBERT-large (340M). These models were fine-tuned for up to 10 epochs with learning rates in $$\{1\mathrm{e}{-5}, 3\mathrm{e}{-5}, 5\mathrm{e}{-5}\}$$ and losses in $\{$default BCE, positive-weight BCE, focal loss$\}$, jointly predicting all twelve labels through a single multilabel head (Mor-Lan et al., 21 Aug 2025).

The second family is single-label encoder models. Here the best multilabel encoder backbone, DictaBERT-large, was reused to fine-tune twelve separate binary classifiers, one per identity, under the same hyperparameter grid (Mor-Lan et al., 21 Aug 2025). This setup allows comparison between joint multilabel learning and decomposed one-vs-rest prediction.

The third family is seq2seq LLM decoders: Gemma-2B, Gemma-9B, Qwen3-8B, and Hebrew-tuned DictaLM 2.0 with 7B parameters. These were fine-tuned with QLoRA using 4-bit quantization, LoRA rank $= 256$, and $\alpha = 512$, for up to 5 epochs with learning rates in $\{1\mathrm{e}{-5}, 1\mathrm{e}{-4}\}$. The task was formulated generatively: given an input sentence, the model outputs a comma-separated list of Hebrew identity labels (Mor-Lan et al., 21 Aug 2025).

The reported explanation for the strongest results centers on Hebrew-specific adaptation. DictaLM 2.0 continues pre-training of the Mistral-7B base on 100B Hebrew tokens and adapts tokenization to Hebrew script, which the paper argues enables better capture of culturally specific political vocabulary and subtle linguistic cues than multilingual alternatives (Mor-Lan et al., 21 Aug 2025). In the reported experiments, this configuration yielded the best macro-averaged $F_1$ and the best per-label performance on 8 of 12 identities.

4. Evaluation protocol and empirical performance

HebID reports Precision, Recall, and $F_1$ both per label and macro-averaged. The macro-$\kappa$0 is defined as

$\kappa$1

with $\kappa$2 identities (Mor-Lan et al., 21 Aug 2025).

On the test set, DictaLM 2.0 achieved macro-Precision $\kappa$3, macro-Recall $\kappa$4, and macro-$\kappa$5. The other reported results were macro-$\kappa$6 for Gemma-9B, $\kappa$7 for Gemma-2B, $\kappa$8 for Qwen-8B, $\kappa$9 for the best multilabel encoder (DictaBERT-large), and $$\{1\mathrm{e}{-5}, 3\mathrm{e}{-5}, 5\mathrm{e}{-5}\}$$0 for the combined single-label DictaBERT-large system (Mor-Lan et al., 21 Aug 2025). The gap between the best seq2seq Hebrew-tuned model and the best encoder baseline is therefore substantial within the reported benchmark.

Per-label $$\{1\mathrm{e}{-5}, 3\mathrm{e}{-5}, 5\mathrm{e}{-5}\}$$1 for DictaLM 2.0 ranged from 0.653 for “Ultra-Orthodox” to 0.852 for “Palestinians and Arab Citizens,” indicating that the best model did not perform well only on majority classes but also remained competitive on rarer identities (Mor-Lan et al., 21 Aug 2025). The project also tested cross-genre generalization by applying the fine-tuned DictaLM 2.0 to 500 randomly sampled, precision-oriented sentences from Knesset plenary speeches. The resulting macro-$$\{1\mathrm{e}{-5}, 3\mathrm{e}{-5}, 5\mathrm{e}{-5}\}$$2 was 0.72, with Precision $$\{1\mathrm{e}{-5}, 3\mathrm{e}{-5}, 5\mathrm{e}{-5}\}$$3 and Recall $$\{1\mathrm{e}{-5}, 3\mathrm{e}{-5}, 5\mathrm{e}{-5}\}$$4, described as on par with the Facebook results (Mor-Lan et al., 21 Aug 2025).

These results situate HebID as both a corpus benchmark and a transfer study. The evidence reported in the paper indicates that the learned representation of identity language is not narrowly tied to a single platform register.

5. Findings from large-scale application

HebID applies its best classifier to large political corpora and compares the resulting identity distributions with panel-survey self-reports. In identity popularity, the top three identities in elite texts were Socially-oriented, Rightist, and Democrat. Spearman rank correlations of identity shares were $$\{1\mathrm{e}{-5}, 3\mathrm{e}{-5}, 5\mathrm{e}{-5}\}$$5 for Facebook versus Knesset, $$\{1\mathrm{e}{-5}, 3\mathrm{e}{-5}, 5\mathrm{e}{-5}\}$$6 for Facebook versus Survey, and $$\{1\mathrm{e}{-5}, 3\mathrm{e}{-5}, 5\mathrm{e}{-5}\}$$7 for Knesset versus Survey. “Honest” and “Zionist” were more prominent in survey self-identification, whereas “Socially-oriented” was more frequent in parliamentary speeches (Mor-Lan et al., 21 Aug 2025).

Temporal analysis showed that weekly average identity mentions per sentence in Facebook data fluctuated between 0.38 and 0.87, with clear peaks around three of four election dates. Rightist, Leftist, and Democrat identities spiked before elections. The paper also reports divergent timing between elite discourse and public self-reports: for example, “Socially-oriented” declined among the public but rose in politicians’ posts after the March 2020 election, while “Honest” and “Democrat” rose in survey responses after the election but fell in elite discourse (Mor-Lan et al., 21 Aug 2025).

Identity co-occurrence was analyzed through factor analysis on speaker and respondent identity profiles. The primary dimension was a left-versus-right structure. The reported left bundle comprised Leftist, Democrat, Honest, Liberal, and Palestinian; the right bundle comprised Rightist, Conservative, Zionist, Security-oriented, Capitalist, and Ultra-Orthodox. Additional sub-bundles were also observed, including Conservative+Rightist and Zionist+Security-oriented. Mean absolute correlation of identities was 0.235 for Facebook, 0.215 for Knesset, and 0.159 for Survey, indicating weaker coupling in self-reports than in elite texts (Mor-Lan et al., 21 Aug 2025).

Gender differences were likewise systematic. Women expressed more identities per sentence on average than men, with the largest gap in Knesset data at 0.07. Men skewed toward Rightist, Security-oriented, Capitalist, and Ultra-Orthodox identities, while women skewed toward Socially-oriented across all sources. The paper also notes mismatches between textual and survey gender patterns, such as “Honest” leaning female in survey responses but slightly male in text (Mor-Lan et al., 21 Aug 2025).

Taken together, these analyses position HebID not merely as a classification benchmark but as an instrument for comparative study of elite rhetoric, institutional speech, and public identity priorities.

6. Scope, limitations, and reuse

HebID’s main contribution is identified as filling a resource gap: a fine-grained, multilabel Hebrew corpus for political identity detection grounded in survey evidence and expert definitions. The reported results indicate that Hebrew-tuned seq2seq LLMs outperform both multilingual LLMs and encoder-only baselines on macro-$$\{1\mathrm{e}{-5}, 3\mathrm{e}{-5}, 5\mathrm{e}{-5}\}$$8, and that classifiers trained on Facebook text generalize to parliamentary speech with macro-$$\{1\mathrm{e}{-5}, 3\mathrm{e}{-5}, 5\mathrm{e}{-5}\}$$9 (Mor-Lan et al., 21 Aug 2025).

The project’s limitations are explicit. Its temporal and platform scope is restricted to December 2018 through April 2021 and to Facebook and Knesset data. The survey sampling excluded non-Jewish citizens, so identities salient to Palestinians or other minorities are described as under-represented. The annotation schema excludes negative mentions and identities below the 5% survey-salience threshold. The authors also note that model biases inherited from training data and pre-training corpora may affect performance on dialectal or hostile discourse, and that cross-genre validation beyond Knesset speeches remains untested (Mor-Lan et al., 21 Aug 2025).

These constraints are important for interpretation. A common misreading would be to treat HebID as an exhaustive inventory of identity language in Israeli politics; the paper does not support that conclusion. Another would be to equate identity mention with ideological endorsement in all cases; in fact, the codebook labels only positive expressions, excluding critical invocation of an identity. A plausible implication is that HebID is best understood as a resource for affirmative identity signaling rather than for all identity reference.

The paper recommends extending the approach to additional non-English political contexts by replicating its survey-grounded label selection and multilabel annotation design, broadening platform coverage to media such as Twitter and news websites, widening demographic sampling, refining annotation with emerging identities such as environmentalist and feminist, and investigating debiasing and fairness techniques (Mor-Lan et al., 21 Aug 2025). Code, data under cc-by-4.0, and model checkpoints are publicly available at https://github.com/guymorlan/hebid/, making HebID a reusable foundation for further computational and social-scientific work on identity discourse in Hebrew and related settings.