Beyond Tokens: Concept-Level Training Objectives for LLMs

Abstract: The next-token prediction (NTP) objective has been foundational in the development of modern LLMs, driving advances in fluency and generalization. However, NTP operates at the \textit{token} level, treating deviations from a single reference continuation as errors even when alternative continuations are equally plausible or semantically equivalent (e.g., mom'' vs.mother''). As a result, token-level loss can penalize valid abstractions, paraphrases, or conceptually correct reasoning paths, biasing models toward surface form rather than underlying meaning. This mismatch between the training signal and semantic correctness motivates learning objectives that operate over higher-level representations. We propose a shift from token-level to concept-level prediction, where concepts group multiple surface forms of the same idea (e.g., mom,''mommy,'' ``mother'' $\rightarrow$ \textit{MOTHER}). We introduce various methods for integrating conceptual supervision into LLM training and show that concept-aware models achieve lower perplexity, improved robustness under domain shift, and stronger performance than NTP-based models on diverse NLP benchmarks. This suggests \textit{concept-level supervision} as an improved training signal that better aligns LLMs with human semantic abstractions.

• The paper demonstrates that re-framing training objectives at the concept level improves semantic understanding by rewarding valid paraphrases and abstractions instead of rigid token matches.
• The methodology clusters tokens into concept groups using both context-independent (WordNet) and context-sensitive (LLM prompting) techniques to augment training data.
• Results show improved perplexity and robustness, with NCP models outperforming NTP baselines on downstream tasks and cross-domain evaluations.

Concept-Level Training Objectives in LLMs: A Technical Analysis of “Beyond Tokens: Concept-Level Training Objectives for LLMs”

Motivation and Limitations of Next-Token Prediction

Prevailing LLMs are overwhelmingly trained using the next-token prediction (NTP) objective. While this has driven major advances in fluency and generalization, it fundamentally anchors supervision at the token level: errors occur whenever the model diverges from the reference sequence, regardless of whether the alternative is a semantically valid paraphrase, synonym, or conceptual abstraction. Thus, NTP naturally biases models toward rigid surface-form matching rather than semantic equivalence. This is particularly limiting for downstream reasoning, abstraction, and robust generalization, where capturing conceptual structure, not just sequence likelihood, is essential.

To address this intrinsic limitation, the authors propose a paradigm shift: move from a discrete token-level target space to a concept-level target space, rewarding semantic equivalence over string match. The goal is to build models directly aligned with human abstraction—where multiple surface forms (e.g., “mom,” “mother,” “mommy”) collapse into a unified concept (MOTHER) and broader conceptual generalizations become available.

Methodology: Formulation of the Concept-Level Objective

The authors’ key idea is to cluster tokens into concepts—collections of surface forms (synonyms and hypernyms) that are contextually or context-independently equivalent. Concepts are extracted at two levels:

• Synonym Level: Tokens are collapsed into surface-level synonym sets, sourced either context-independently (from WordNet) or contextually (via LLM-based sentence-aware prompting).
• Hypernym Level: Tokens are grouped under more abstract hypernyms.

They introduce a next-concept prediction (NCP) objective: models are trained to assign high probability mass to any token belonging to the correct concept cluster, not just the canonical reference target.

Data Pipeline:

The procedure is outlined as follows:

Figure 1: Pipeline for extracting context-dependent and -independent synonyms/hypernyms and using them for concept-level training or as augmentation; all models are fine-tuned on downstream benchmarks.

The authors collected post-Llama-3-release data from YouTube, arXiv, and NYT—avoiding contamination. For every noun token in these datasets, they built lists of synonymous or hypernymic alternatives, via context-free (WordNet) and context-aware (LLM-prompted) mechanisms.

Training Variants:

• NTP Baselines: Standard next-token modeling, both vanilla and with repeated (upsampled) instances for parity with NCP augmentation.
• NCP Data Augmentation: Multiple copies of each training example are created by substituting the target noun with every synonym/hypernym in the cluster, increasing exposure to concept variation.
• NCP Loss Modification: The loss is redefined so the target is the set of conceptually equivalent completions, and success is credited for any correct member of that set.

Empirical Evaluation: Robustness and Generalization

Post-Training Perplexity:

All model variants are evaluated—without concept signal injection at test—on NTP perplexity across all domains. The best-performing NCP variant (“NHP Context-Aware Data Augmentation”) achieves the lowest perplexity scores in every held-out domain, outperforming token-level NTP.

Cross-Domain Transfer:

Domain robustness is measured as the ratio of evaluation perplexity when trained on out-of-domain data to in-domain. NCP models exhibit consistently better cross-domain resilience compared to NTP—indicating stronger generalization beyond surface fit.

Qualitative Observation:

Whereas NTP models narrowly distribute score across string-variants (“articles,” “Article,” spacing/capitalization), NCP models’ probability mass is spread over true semantic alternatives—indicating reduced lexical fixation and improved abstraction.

Downstream Fine-Tuning: Transfer to Semantic Tasks

After post-training, all models are fine-tuned—with consistent LoRA hyperparameters—for seven benchmark tasks: SNLI, GLUE, Empathetic Dialogues, Hate Speech, Spam, Fake News, and Logical Fallacy detection. In all settings, NCP-trained models achieve superior performance compared to NTP-trained controls, often by noticeable margins. Notably, the NHP Context-Free Data Augmentation variant is either best or comparable on four out of seven tasks. The performance improvement persists even in domain-mismatched training, attesting to the robustness of concept-level supervision.

Critical Analysis: Implications for Model Training and AI Research

Practical Impact:

Concept-level objectives mitigate overstability to superficial form, thus increasing paraphrase robustness and decreasing the risk of brittle degeneration in OOD or creative contexts. The concrete empirical gains in both perplexity and fine-tuned task accuracy—especially in the presence of distributional shift—make this approach practical for real-world deployments demanding interpretability, transfer, and robust generalization.

Theoretical Insight:

This work substantiates the hypothesis that, for LLMs, “surface likelihood” is only a proxy for semantic understanding. Decoupling these objectives—by supplementing or revising standard NTP with a concept-level loss—induces representations more closely aligned with human cognition, capable of abstracting away from the specific token inventory. This supports a shift toward richer semantic targets in future LLM pretraining regimes.

Limitations:

• The current approach is noun-centric and relies on adjectives/verbs extensions in future work.
• Concept extraction is imperfect: context-free methods ignore polysemy; context-aware models inherit LLM biases.
• Evaluation is currently confined to standard classification tasks; stronger benefits may surface with more challenging generative or reasoning-oriented benchmarks.

Potential Extensions:

• Hierarchical and multi-lingual concept groupings.
• Integration with multi-modal or reasoning objectives.
• Large-scale concept-informed pretraining, potentially surpassing NTP as the primary signal.

Conclusion

This paper rigorously demonstrates that concept-level supervision offers demonstrable advantages over token-level NTP in LLM training. By elevating the granularity of the objective to match the semantic abstraction in human language use, models gain both quantitative improvements in perplexity/task accuracy and qualitative improvements in robustness and flexibility. This paradigm furnishes a path beyond the current proxy of next-token likelihood, toward models that are both more linguistically plausible and more capable in downstream human-centric tasks.