---
title: 'HCNR: Restoring Honesty in Fine-Tuned LLMs'
url: https://www.emergentmind.com/topics/honesty-critical-neurons-restoration-hcnr
type: topic
---

# HCNR: Restoring Honesty in Fine-Tuned LLMs

Honesty-Critical Neurons Restoration (HCNR) is a parameter-efficient approach for recovering the honesty trait in large language models (LLMs) that have undergone supervised fine-tuning (SFT). HCNR leverages the empirical insight that, post-SFT, the model’s internal representations still encode knowledge about what the model does or does not know; however, this knowledge is no longer faithfully expressed in outputs. By surgically identifying and repairing only the neurons critical for honesty expression—and aligning them with downstream task-oriented neurons via Hessian-guided compensation—HCNR restores honesty with minimal impact on downstream task accuracy, substantially reducing data and computational requirements compared to prior methods [2511.12991].

## 1. Theoretical Motivation

Empirical analysis demonstrates that SFT, while effective for domain specialization, often leads to a collapse in “honesty”—the model’s willingness to admit uncertainty or refuse to answer unanswerable queries. Importantly, linear probes reveal that the internal knowledge-boundary signals in pre-trained LLMs remain largely intact even after SFT; it is the capacity to express this awareness that is degraded. This suggests that SFT masks, rather than destroys, the model’s self-knowledge, reframing post-SFT dishonesty as an “expression” rather than a “knowledge” problem. Consequently, honesty restoration does not require global parameter retraining but can be accomplished by addressing a small subset of “expression-governing” neurons [2511.12991].

## 2. Identification of Honesty-Critical Neurons

HCNR targets neurons that meet three criteria: (a) high influence on honesty loss, (b) minimal effect on the domain task, and (c) substantial perturbation during SFT. The selection procedure is two-fold:

- **Fisher-based intra-layer importance:** For weight tensor \(W_j\) in layer \(j\) and neuron \(k\):

  $$
  s^{\text{hon}}_{j,k} = \mathbb{E}_{(x,y)\sim D^{\text{hon}}} \left[ \left( \partial_{W_{j,k}} \mathcal{L}_{\mathrm{hon}}(x,y) \right)^2 \right ], \qquad s^{\text{task}}_{j,k} = \mathbb{E}_{(x,y)\sim D^{\text{task}}} \left[ \left( \partial_{W_{j,k}} \mathcal{L}_{\mathrm{task}}(x,y) \right)^2 \right ]
  $$

  The priority score is defined as:

  $$
  r_{j,k} = s^{\text{hon}}_{j,k} \times \log\left( \frac{s^{\text{hon}}_{j,k}}{s^{\text{task}}_{j,k} + \epsilon} \right)
  $$

  Top-ranked neurons per layer (top \(R_{IW}\) fraction) are candidates.

- **Cross-layer perturbation:** For candidates, the SFT-induced shift per layer is:

  $$
  d_j = \frac{\| (W_j^{\text{sft}} - W_j^{\text{orig}} ) \odot M_j \|_2 }{ \| W_j^{\text{orig}} \odot M_j \|_2 }
  $$

  Layers are ranked by \(d_j\), and the top \(R_{CW}\) fraction are designated “honesty-critical”. The final honesty-critical set comprises the top neurons within these layers:

  $$
  A^{\rm hc} = \{ (j,k) \mid j \in A^{\rm layer},\; k \in A_j^{\rm neuron} \}
  $$

## 3. Hessian-Guided Compensation Mechanism

Directly resetting honesty-critical neurons to their pre-trained values can misalign these neurons with task-adapted circuitry. HCNR addresses this by optimizing a small, neuron-specific compensation using second-order (Hessian) information from an honesty calibration loss:

- **Discrepancy loss:** For activations \(X_{\rm hon}\) on an honesty dataset,

  $$
  d_{\text{hon}} = \| W^{\rm hc} X_{\rm hon} - W^{\rm orig} X_{\rm hon} \|_2^2
  $$

- **Layerwise Hessian computation:** Compute (or approximate) the Hessian with respect to pre-trained weights:

  $$
  H_j = \nabla^2_{W^{\rm orig}_j} d_{\rm hon} \big|_{W^{\rm orig}_j}
  $$

  Given SFT perturbation \(\delta w_{j, k} = W^{\rm sft}_{j,k} - W^{\rm orig}_{j,k}\), optimal compensation is

  $$
  c_{j, k} = [H_j^{-1} \delta w_j]_k
  $$

  The restored weight is set as \(W^{\rm hc}_{j,k} = W^{\rm orig}_{j,k} + c_{j,k}\).

This mechanism ensures restored neurons are locally consistent with the Hessian geometry of the fine-tuned network, harmonizing honesty-related and task-related functionalities.

## 4. HCNR Algorithmic Workflow

The HCNR procedure comprises two algorithmic stages:

1. **Neuron Recognition:**
   - Compute intra-layer honesty and task Fisher scores (\(s^{\rm hon}, s^{\rm task}\)), form priority scores (\(r_{j,k}\)), and select top candidates (\(A_j^{\rm neuron}\)).
   - Compute per-layer SFT perturbation (\(d_j\)), select top honesty-critical layers (\(A^{\rm layer}\)).
   - Aggregate honesty-critical neuron indices (\(A^{\rm hc}\)), initialize honesty-restored weights by setting \((j, k) \in A^{\rm hc}\) to pre-trained values.

2. **Hessian-Guided Compensation:**
   - Compute layerwise Hessians \(H_j\) on \(D^{\rm hon}\).
   - For each \((j, k) \in A^{\rm hc}\), derive and apply the scalar compensation \(c_{j, k}\).

The result is a set of model weights \(W^{\rm hc}\) with restored honesty expression and preserved domain task capacity.

## 5. Experimental Validation and Comparative Analysis

HCNR has been empirically validated across four QA tasks and five LLM families. Key findings include:

- **Honesty recovery:** On average, HCNR restores 33.25% of the honesty lost to SFT.
- **Pareto optimality:** The approach lies strictly above baseline task-honesty trade-off curves (evaluated with KUQ/SelfAware benchmarks).
- **Efficiency:** For Llama-3.1-8B-Instruct on HotpotQA and MedMCQA, baselines (RAIT, DPO, ORPO) require 4,000–9,000 “IDK” samples and 100% of parameters, while HCNR uses only 256 honest + 128 task samples and modifies ≈20% of weights. Wall-clock improvements range from 2.2× to 10×.
- **Ablation analysis:** Randomization or omission of importance scoring (Stage 1) reduces accuracy; skipping Hessian compensation (Stage 2) reduces honesty recovery by ~40%. Combining both ablations eliminates recovery.
- **Data sensitivity:** Performance saturates at 128 samples for honesty and task calibration, confirming minimal data demands.

## 6. Context, Significance, and Implications

HCNR introduces a novel paradigm based on mechanistic understanding: post-SFT dishonesty primarily affects expression, not internal knowledge estimation. By restricting intervention to a sparse, mechanistically identified subset of neurons and leveraging Hessian-guided compensation for alignment, HCNR achieves a favorable blend of restoration effectiveness and efficiency. A plausible implication is that the approach generalizes to other fine-tuning-induced behavioral degradations where internal model knowledge is preserved but suppressed in output. This suggests new directions for model repair that prioritize targeted, data- and compute-light interventions over global retraining [2511.12991].

Source: https://www.emergentmind.com/topics/honesty-critical-neurons-restoration-hcnr