BGM-HAN: Hierarchical Model for Admissions

• BGM-HAN is a hierarchical neural architecture that employs Byte-Pair Encoding, multi-head self-attention, and gated residual networks to process semi-structured profiles.
• It decomposes applicant profiles into academic records, leadership entries, and personal essays, enabling token-to-sentence-to-field analysis for robust admission predictions.
• Empirical results demonstrate that BGM-HAN outperforms traditional baselines with marked improvements in precision, recall, F1 score, and accuracy on a university admissions dataset.

BGM-HAN, short for Byte-Pair Encoded, Gated Multi-head Hierarchical Attention Network, is a hierarchical neural architecture for decision assessment on semi-structured profiles, introduced for high-stakes domains and instantiated on university admissions (Liu et al., 23 Jul 2025). It is designed to model profiles that combine structured numeric or categorical information, semi-structured records, and free text, and it learns a binary mapping from a full profile to an admission outcome. In its reported formulation, BGM-HAN decomposes each applicant profile into academic records, leadership records, and Personal Insight Question responses, and processes them through a token-to-sentence-to-field hierarchy that combines BPE tokenization, multi-head self-attention, gated residual networks, and mean pooling (Liu et al., 23 Jul 2025).

1. Problem formulation and data model

BGM-HAN was proposed for high-stakes decision assessment over semi-structured profiles, with university admissions used as the representative task. Each applicant profile $p_i \in \mathcal{P}$ is decomposed into four principal components: $f_{\text{GCEA}}$ for GCE A-Level results, $f_{\text{GCEO}}$ for GCE O-Level results, $f_{\text{Leadership}}$ for leadership records, and $f_{\text{PIQ}}$ for free-text Personal Insight Question responses. The task is to learn a mapping

$\mathcal{D}: \mathcal{P} \rightarrow \{0,1\}$

that predicts admission outcomes, where $1$ denotes an offer and $0$ denotes a rejection (Liu et al., 23 Jul 2025).

The model targets data that are neither purely tabular nor purely unstructured. Academic records contain structured fields such as subject grades, exam types, and University Admission Scores. Leadership records are semi-structured, with entries such as role title, years of participation, activity category, and level of commitment. PIQ responses are multi-sentence essays. The motivation given for this architecture is threefold: improving predictive accuracy, improving consistency in settings affected by human heuristics and cognitive bias, and improving interpretability in workflows where explanations matter (Liu et al., 23 Jul 2025).

The admissions dataset used in the reported experiments contains 3,083 anonymized profiles from a single admissions cycle of a major engineering university. Each profile includes academic records, leadership experience entries, five PIQ essays, and a binary admission label. The train/validation/test split is 90\% / 5\% / 5\%, using stratified sampling to preserve class proportions (Liu et al., 23 Jul 2025).

2. Input representation and Byte-Pair Encoding

A central design choice in BGM-HAN is the use of Byte-Pair Encoding (BPE) as the tokenizer across all textual fields. The paper specifies a standard subword-merging procedure: initialize the vocabulary with unique characters, repeatedly count frequencies $f(a,b)$ of consecutive symbol pairs, merge the most frequent pair

$$(a^*, b^*) = \operatorname*{arg\,max}_{(a,b)} f(a,b),$$

and continue until the desired vocabulary size is reached. The final BPE vocabulary size is $f_{\text{GCEA}}$0 (Liu et al., 23 Jul 2025).

This subword representation is used for academic fields, leadership records, and PIQ essays alike. The stated rationale is robustness to out-of-vocabulary and rare tokens, including unusual school names, award names, acronyms, and similar profile-specific terms. The model therefore avoids the brittleness of word-level tokenization while avoiding the long sequence lengths of pure character-level encoding (Liu et al., 23 Jul 2025).

For each text field $f_{\text{GCEA}}$1, the paper defines a hierarchical embedding tensor by splitting the field into sentences $f_{\text{GCEA}}$2, applying BPE tokenization within each sentence, mapping tokens to embeddings, and then truncating or padding to a fixed shape. The resulting representation is

$f_{\text{GCEA}}$3

with maximum sentences per field $f_{\text{GCEA}}$4, maximum tokens per sentence $f_{\text{GCEA}}$5, and embedding dimension $f_{\text{GCEA}}$6 (Liu et al., 23 Jul 2025).

Missing text fields are handled by inserting NaN or zero-padding tokens, preserving a uniform tensor shape across applicants. This keeps the hierarchy intact without relying on manual imputation. In effect, BGM-HAN treats each field as a bounded document, each document as a sequence of sentences, and each sentence as a sequence of subword tokens (Liu et al., 23 Jul 2025).

3. Hierarchical architecture and core components

BGM-HAN follows a three-level hierarchy: token or word level within each sentence, sentence level within each field, and field level across the profile. At each level, the processing pattern is the same: layer normalization, multi-head self-attention, a gated residual network, and mean pooling. The output of one level becomes the input sequence for the next (Liu et al., 23 Jul 2025).

At the attention stage, the model uses multi-head self-attention rather than the single attention vector used in the classic HAN formulation. For an input matrix $f_{\text{GCEA}}$7, head $f_{\text{GCEA}}$8 is defined through learned projections

$f_{\text{GCEA}}$9

with attention

$f_{\text{GCEO}}$0

The heads are concatenated and projected: $f_{\text{GCEO}}$1 The final configuration uses $f_{\text{GCEO}}$2 attention heads and hidden dimension $f_{\text{GCEO}}$3 after hyperparameter search (Liu et al., 23 Jul 2025).

The second architectural addition is the Gated Residual Network (GRN): $f_{\text{GCEO}}$4 where $f_{\text{GCEO}}$5 is a learnable gating vector and

$f_{\text{GCEO}}$6

The gate modulates how much transformed information is added through the residual path. The paper positions this component as a mechanism for improving training stability and controlling nonlinear transformations (Liu et al., 23 Jul 2025).

The full profile encoding proceeds hierarchically. Token embeddings within a sentence are processed by MHA and GRN, then mean-pooled into a sentence embedding. Sentence embeddings within a field are processed the same way and mean-pooled into a field embedding. Field embeddings across the profile are again processed by MHA and GRN and mean-pooled into a final profile vector $f_{\text{GCEO}}$7. A classifier head then produces

$f_{\text{GCEO}}$8

which is interpreted as the probability of admission (Liu et al., 23 Jul 2025).

4. Objective function, optimization, and training protocol

The prediction layer is trained with class-weighted binary cross-entropy, motivated by class imbalance in admissions: $f_{\text{GCEO}}$9 with class weight

$f_{\text{Leadership}}$0

An $f_{\text{Leadership}}$1 regularization term is added: $f_{\text{Leadership}}$2 The paper explicitly notes that there is no fairness-specific loss term, such as demographic parity or equalized-odds regularization (Liu et al., 23 Jul 2025).

Training uses AdamW with learning rate $f_{\text{Leadership}}$3, batch size 32, dropout 0.6, and gradient clipping with max norm 1.0. Learning-rate scheduling is handled by ReduceLROnPlateau with decay factor $f_{\text{Leadership}}$4 and minimum learning rate $f_{\text{Leadership}}$5. Early stopping uses patience $f_{\text{Leadership}}$6 epochs, and hyperparameter search considers up to 50 epochs per trial (Liu et al., 23 Jul 2025).

The comparison baselines span several methodological families. Traditional machine-learning baselines include TF-IDF + Logistic Regression and XGBoost over concatenated BERT embeddings. Neural baselines include MLP, BiLSTM-Indv, BiLSTM-Concat, and a standard HAN. The paper also evaluates GPT-4o (zero-shot) and GPT-4o-RA (5-shot retrieval-augmented). For BERT-based baseline features, the model is bert-base-uncased with maximum sequence length 512. Neural baselines are trained with Adam, learning rate $f_{\text{Leadership}}$7, maximum 100 epochs, and early stopping with patience 10 (Liu et al., 23 Jul 2025).

5. Empirical performance and ablation results

On the test set, BGM-HAN achieves the best reported performance among all baselines, with precision 0.8622, recall 0.8405, F1 0.8453, and accuracy 0.8506. The strongest non-BGM-HAN baseline is BiLSTM-Concat, with precision 0.8291, recall 0.8178, F1 0.8176, and accuracy 0.8276. The standard HAN baseline reaches precision 0.7716, recall 0.7707, F1 0.7711, and accuracy 0.7759. XGBoost reaches F1 0.7878 and accuracy 0.7931, while TF-IDF + Logistic Regression reaches F1 0.6488 and accuracy 0.6839 (Liu et al., 23 Jul 2025).

The comparison with LLMs is notable. GPT-4o (0-shot) records precision 0.5579, recall 0.5114, F1 0.4111, and accuracy 0.5600. GPT-4o-RA (5-shot) improves substantially to precision 0.7347, recall 0.7365, F1 0.7352, and accuracy 0.7371, but still remains below HAN, BiLSTM-Concat, and BGM-HAN. In this setting, the reported results favor a domain-tailored hierarchical model over prompt-based general-purpose LLM inference (Liu et al., 23 Jul 2025).

The paper also reports an ablation study relative to the base HAN. Starting from HAN: F1 = 0.7711, accuracy = 0.7759, adding BPE yields +1.8 percentage points F1, adding multi-head attention yields +5.2 points F1, and adding gated residual connections yields +2.6 points F1. The paper identifies multi-head attention as the largest single contributor. Collectively, the final BGM-HAN configuration yields a total improvement of 7.4 F1 points and 9.6 accuracy points over the base HAN (Liu et al., 23 Jul 2025).

These results support the specific architectural claim of the paper: that semi-structured decision data benefit from a hierarchy that preserves field boundaries and internal document structure, rather than collapsing all inputs into a single flat representation. The reported gains over HAN indicate that the improvements do not arise from hierarchy alone, but from the combination of BPE, multi-head attention, and gated residual processing (Liu et al., 23 Jul 2025).

6. Interpretability, fairness claims, scope, and terminological distinction

The paper frames BGM-HAN as both a performance model and an interpretable decision-support architecture. Interpretability is attributed to three properties: explicit field-level modeling, inspectable attention weights at the word, sentence, and field levels, and component-level ablations that expose the effect of each architectural modification. The intended explanation style is hierarchical—for example, identifying whether the model relied more on PIQ responses, leadership records, or academic indicators in a given decision (Liu et al., 23 Jul 2025).

Its fairness claims are more limited than the title’s emphasis on “accurate and fair decision assessment” might suggest. The paper explicitly states that there is no formal fairness constraint or sensitive-attribute debiasing term in the loss, and it does not report subgroup fairness metrics such as demographic parity or equalized odds. Fairness is instead treated qualitatively through consistency, auditability, and the reduction of dependence on inconsistent human heuristics. The paper therefore positions BGM-HAN as a framework for structured and inspectable decision support rather than as a formally constrained algorithmic-fairness method (Liu et al., 23 Jul 2025).

The reported limitations follow directly from the experimental setting. The dataset is proprietary, comes from a single institution and a single admissions cycle, and is therefore not an empirical test of cross-institution generalization. Structured numeric features such as UAS are represented textually rather than with specialized numeric encoders. The architecture is also computationally heavier than simpler baselines because it combines hierarchical processing with multi-head attention and a hidden size of 1024. At the same time, the paper proposes broader applicability to hiring, lending, healthcare, and procurement, all of which involve mixtures of structured records and narrative documents (Liu et al., 23 Jul 2025).

The acronym also requires disambiguation. The named model BGM-HAN refers specifically to the admissions-oriented Byte-Pair Encoded, Gated Multi-head Hierarchical Attention Network. It is unrelated to other uses of BGM in the arXiv literature, including background music systems and retrieval or generation frameworks (Khan et al., 2024, Li et al., 2021, Li et al., 2024), the BGM diagnosis model for multiprocessor fault diagnosis (Lin et al., 2021), and BGM-HAN-type terminology in braid-group cryptography (0708.2571). The official code repository reported for the admissions model is https://github.com/junhua/bgm-han (Liu et al., 23 Jul 2025).