Evaluate safety-oriented behavior under depth compression

Investigate hallucination, refusal, and safety behavior in XMerge-compressed language models using instruction-tuned or safety-tuned variants, which were not evaluated for the base decoder-only models studied here.

Background

The paper evaluates XMerge primarily through downstream task performance, perplexity, and a limited calibration probe on one Llama-3-8B backbone. Although XMerge exhibits the smallest calibration degradation among the evaluated operators in that probe, the authors explicitly distinguish this result from a general safety guarantee.

The unresolved issue is whether depth compression with XMerge preserves safety-relevant behaviors, including hallucination control, refusal behavior, and other safety properties. The paper does not assess these properties because the experiments use base models rather than instruction- or safety-tuned variants, leaving their evaluation for future work.