Create and explore imaginative 3D worlds with our new product Dreamfield (opens in a new tab)

Generalization of the Safety-SFT Composition Curve Across Model Scales

Determine whether the non-monotonic relationship between the fraction of general data in the safety-SFT mix and over-refusal, including the observed 40–50% knee point, persists at other model scales.

Background

The paper evaluates the fraction of general data in the safety-SFT mixture only for Manacá-1B-Instruct-2, using a controlled scan from 10% to 60% general data in 10-percentage-point increments. The results suggest a non-monotonic over-refusal curve and identify a 40–50% range as a promising knee point balancing safety and helpfulness. However, each composition point uses only 16 safety items and 34 over-refusal items, with a single training round, so the authors characterize the finding as exploratory. Whether the same curve and knee point apply to models of different sizes remains unresolved.

References

it is unknown whether the same shape (or the same knee point) holds at another model scale.