---
title: 'FLAMe-24B: Foundational Autorater Model'
url: https://www.emergentmind.com/topics/flame-24b
type: topic
---

# FLAMe-24B: Foundational Autorater Model

FLAMe-24B refers to a large-scale foundational autorater model adapted from the PaLM-2-24B Transformer, specialized for robust, generalizable automatic evaluation of large language model (LLM) outputs via fine-tuning on extensive, permissively licensed human judgment datasets. The FLAMe-24B family is designed for LLM-as-a-Judge (LaaJ) applications, outperforming proprietary systems on multiple benchmarks, while demonstrating significantly reduced evaluation biases and efficiency gains with novel fine-tuning strategies [2407.10817].

## 1. Model Architecture

FLAMe-24B is implemented as a decoder-only Transformer, derived by fine-tuning the PaLM-2-24B base (Anil et al., 2023), with the following core architectural traits:

- **Layers:** 64 Transformer blocks  
- **Hidden dimension (“model width”):** 6144  
- **Feed-forward (MLP) intermediate size:** 16,384  
- **Attention heads per layer:** 48  
- **Rotary position embeddings:** used in attention mechanisms  
- **Pre-layer normalization:** LayerNorm precedes both self-attention and MLP sub-layers  
- **Parameter count:** approximately $24 \times 10^9$  

The canonical single Transformer block sequence is:
```
Input → LayerNorm → MultiHeadSelfAttn + Residual →
       LayerNorm → FeedForward + Residual → Output
```
Layer normalization (with per-feature centering and scaling) is:
$$
\mathrm{LayerNorm}(x) = \frac{x - \mu(x)}{\sqrt{\sigma^2(x) + \varepsilon}} \odot \gamma + \beta
$$
where $\mu(x)$, $\sigma^2(x)$: per-feature mean/variance, $\gamma$, $\beta$: learned parameters, and $\varepsilon = 10^{-5}$ for numerical stability.

## 2. Pretraining and Fine-tuning Regimes

FLAMe-24B is produced with a two-stage supervised finetuning pipeline following a full PaLM-2-24B initialization:

**Stage 1: General Autorater Fine-tuning (FLAMe)**
- **Data:** 102 quality assessment tasks, 5.3M human judgments, spanning pairwise, pointwise, classification, and open-ended quality explanations.
- **Mixture weights:** “examples-proportional,” truncated to $2^{16}$ samples/task.
- **Optimization:** Adam, learning rate $1 \times 10^{-4}$, dropout 0.05.
- **Schedule:** 30,000 steps, batch size 32 on 256 Cloud TPU v5 chips.
- **Loss:** Token-level cross-entropy for text-to-text tasks:
  $$
  \mathcal{L}_{\mathrm{CE}} = -\sum_{t=1}^T \log P_\theta(y_t \mid x, y_{<t})
  $$

**Stage 2: Reward-Model–Specific Finetuning (FLAMe-RM)**
- **Initialization:** FLAMe Stage 1 checkpoint
- **Data:** Four pairwise preference datasets (HelpSteer, PRM800K, CommitPack, HH-RLHF Harmlessness), mixed equally.
- **Schedule:** 50 finetuning steps, batch size 8 on 128 Cloud TPU v5 chips.
- **Loss:** Pairwise logistic loss for human-labeled preferences:
  $$
  \mathcal{L}_{\mathrm{RM}} = -[ p \log \sigma(s_\theta(A) - s_\theta(B)) + (1-p) \log (1 - \sigma(s_\theta(A) - s_\theta(B))) ]
  $$
  where $\sigma(z) = 1/(1 + e^{-z})$.

## 3. Tail-Patch Fine-tuning Strategy

FLAMe-24B incorporates a “tail-patch” strategy to efficiently reweight mixture components for target evaluation distributions (e.g., RewardBench):

- **Ablation:** For each of 102 training tasks, fine-tune individually for 3000 steps and measure per-category RewardBench effect.
- **Effect rating:** Tasks are labeled as Helpful (+2), Somewhat helpful (+1), No effect (0), or Harmful (−1), and grouped into: one "Generally helpful" bundle, five "Category-specific" bundles, and an "Others" bundle.
- **Weighting:** Assign fixed per-bundle weights ($w_{\mathrm{general}}=100$K, $w_{\mathrm{specific}}=30$K, $w_{\mathrm{others}}=3$K); the top two tasks in three weakest categories get $200$K.
- **Aggregate:** Sum bundle weights/task across bundles.
- **Finetune:** On this reweighted mixture for only 5000 steps.

This yields the FLAMe-Opt-RM variant, requiring $25\times$ fewer datapoints than standard FLAMe training to reach near-equivalent RewardBench accuracy (87.0% vs. 86.0%).

## 4. RewardBench and Generalization Performance

FLAMe-RM-24B demonstrates leading performance on the RewardBench suite of 23 pairwise tasks, across Chat, Chat Hard, Reasoning, and Safety categories. Key statistics:

| Model             | Overall accuracy (%) |
|-------------------|---------------------|
| FLAMe-RM-24B      |       87.8          |
| GPT-4-0125        |       85.9          |
| GPT-4o            |       84.7          |

For $N \approx 10,000$ test cases, the approximate $95\%$ confidence interval for $p = 0.878$ is
$$
0.878 \pm 1.96 \sqrt{\frac{0.878 (1-0.878)}{10,000}} \approx 87.8\% \pm 1.2\%
$$
A two-proportion $z$-test confirms the 1.9pp difference over GPT-4 is highly significant ($z\approx5.7$, $p<0.001$).

## 5. Autorater Bias Evaluation (CoBBLEr)

FLAMe-24B and its RM/Opt variants are evaluated on the CoBBLEr benchmark, which assesses six bias categories. Below are the absolute bias scores (lower is better):

| Autorater           | Avg. | Order | Compassion | Length | Egocentric | Bandwagon | Attention |
|---------------------|------|-------|------------|--------|------------|-----------|-----------|
| GPT-4               | 0.31 | 0.23  | 0.79       | 0.06   | 0.78       | 0.00      | 0.00      |
| **FLAMe-24B**       | 0.13 | 0.08  | 0.09       | 0.03   | 0.38       | 0.18      | 0.00      |
| FLAMe-RM-24B        | 0.13 | 0.11  | 0.08       | 0.02   | 0.40       | 0.17      | 0.00      |
| FLAMe-Opt-RM-24B    | 0.15 | 0.15  | 0.14       | 0.00   | 0.41       | 0.17      | 0.00      |

The overall bias score of FLAMe-24B (0.13) is less than half that of GPT-4 (0.31), indicating superior resistance to biases associated with ordering, input length, egocentrism, and model name cues.

## 6. Comparative Results on Held-Out Autorater Benchmarks

FLAMe-24B achieves consistently strong results across eleven held-out LLM evaluation benchmarks, often outperforming leading closed models (e.g., GPT-4-0125) despite training exclusively on permissively licensed public data.

\[
\begin{table}[h]
  \centering
  \small
  \begin{tabular}{lrrrr}
    \hline
    \textbf{Benchmark}   & \textbf{Llama-3} & \textbf{GPT-3.5} & \textbf{GPT-4-0125} & \textbf{FLAMe-24B} \\
    \hline
    RewardBench         & 76.1 & 64.5 & 85.9 & \mathbf{86.0} \\
    LLM-AggreFact       & 76.1 & 70.0 & 80.6 & \mathbf{81.1} \\
    SummFeedback        & 50.8 & 15.6 & 46.5 & \mathbf{48.0} \\
    AlpacaFarm          & 53.9 & 55.5 & 49.6 & \mathbf{58.2} \\
    RankGen             & 65.6 & 58.2 & 62.5 & \mathbf{62.1} \\
    CoPoet              & 53.6 & 49.0 & 56.9 & \mathbf{53.6} \\
    ContrSearch         & 53.1 & 57.5 & 55.8 & \mathbf{69.9} \\
    HHH                 & 91.9 & 85.5 & 94.6 &   91.4           \\
    Dipper              & 42.8 & 45.0 & 45.0 &   48.3           \\
    LitTrans            & 60.5 & 54.3 & 67.6 &   67.2           \\
    LFQAEval            & 71.1 & 69.9 & 77.0 &   74.2           \\
    \hline
  \end{tabular}
  \caption{Held‐out benchmark accuracies (\%) comparing FLAMe-24B to leading autoraters; precision/recall ≈ accuracy for these balanced pairwise tasks.}
\end{table}
\]

In eight of eleven held-out tasks, FLAMe-24B exceeds GPT-4-0125, demonstrating robust cross-domain generalization for LLM evaluation and model comparison.

## References

- Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluation, arXiv:2407.10817

All data, terminology, and results are reported verbatim following [2407.10817].

Source: https://www.emergentmind.com/topics/flame-24b