Complex Instruction Following (CIF)
- Complex Instruction Following (CIF) is a framework where models must adhere to multiple, interacting constraints spanning content, style, and context.
- It emphasizes structured constraint composition and dependency analysis, with benchmarks evaluating performance across language, visual, and code tasks.
- Evaluation protocols focus on joint adherence, revealing gaps between local compliance and overall task success while guiding future methodological improvements.
Searching arXiv for recent CIF papers and the cited benchmark/method papers. Complex instruction following (CIF) denotes the ability of a language or multimodal model to execute instructions that carry multiple, interacting constraints on content, structure, style, sequencing, and context, often across multiple turns and modalities. In visually grounded dialogue, CIF further requires sustained contextual understanding, visual entity tracking, reasoning over object states and spatial relations, and suppression of hallucinations (Han et al., 21 Aug 2025). In language-only settings, recent work emphasizes implicit reasoning, intricate logical relations, and multi-constraint dependencies as central sources of difficulty (Yang et al., 4 Feb 2026). The acronym itself is not uniform across the literature: in "CIF-Bench" it expands to Chinese Instruction-Following, although that benchmark explicitly operationalizes complex instruction following in Chinese (Li et al., 2024).
1. Definition and scope
Across recent work, CIF is framed less as a single task than as a family of adherence problems in which a model must satisfy multiple requirements simultaneously. In general LLM settings, these requirements span content, situation, style, format, examples, numerical limits, multilinguality, citation, and safety or privacy-related constraints; TRACE, for example, treats complexity as both the number of constraints and their breadth across 26 fine-grained dimensions, with up to 15 constraints per instruction (Zhang et al., 2024). In this formulation, failure is typically not gross irrelevance but partial compliance: the model answers the task while missing one or more instruction-side requirements.
The scope widens further in modality-specific work. In task-oriented dialogue, CIF is strict adherence to natural-language process instructions encoded as multi-level IF–THEN condition–action logic over multiple turns (Ghazarian et al., 20 Nov 2025). In text-to-speech, CIF becomes the ability to interpret and satisfy multi-dimensional, under-specified, and temporally dynamic natural-language instructions governing prosody, timbre, emotion, and role-based vocal style (Huang et al., 19 Jun 2025). In embodied settings such as CrafText, CIF is embedded in a goal-based POMDP in which textual instructions must be grounded in visual observations, stochastic transitions, and long-horizon action chains (Volovikova et al., 17 May 2025).
A recurring implication is that CIF is not exhausted by output correctness in the narrow sense. Code-generation work makes this especially explicit: MultiCodeIF defines CIF as compliance with layered, fine-grained programming instructions spanning both functional and non-functional constraints, together with iterative refinement under feedback (Duan et al., 1 Jul 2025). Video and image-grounded papers make a parallel point: correct perception alone is insufficient if the model violates explicit output schemas, style conditions, or structural ordering (Xu et al., 3 Jun 2026).
2. Constraint structure and compositional difficulty
A central contribution of recent CIF research is the shift from isolated constraints to structured compositions of constraints. TRACE organizes instructions into five major constraint types—Content, Situation, Style, Format, and Example—and 26 dimensions, with average constraint counts of 4.36 in training and 4.89 in evaluation (Zhang et al., 2024). ComplexBench sharpens the compositional view by defining four constraint types, 19 dimensions, and four composition types—Single, And, Chain, and Selection—with nesting as a first-class source of difficulty (Wen et al., 2024). In that benchmark, complexity does not come only from “more constraints,” but from structural operators such as sequential dependence and conditional branch selection.
Other domains instantiate the same idea with domain-specific taxonomies. MultiCodeIF defines 9 categories and 27 constraint types for code, including Interface Specification, Environment, Data Structure, Algorithm, Coding Style, Code Quality, Scenario, Code Context, and Exemplar (Duan et al., 1 Jul 2025). VCIFBench uses 40 constraint types across content, format, style, and structure for video understanding (Xu et al., 3 Jun 2026). EIFBENCH goes even further toward workflow-level composition: it averages 74.01 constraints and 8.24 instructions per instance, explicitly modeling serial chains, parallel segments, conditional branches, and nested hierarchies (Zou et al., 10 Jun 2025).
Several papers therefore treat composition as the decisive source of difficulty. ComplexBench argues that prior benchmarks model different types of constraints while neglecting the composition of constraints, and EIFBENCH reports that sensitivity to the number of constraints is stronger than sensitivity to the number of instructions (Wen et al., 2024, Zou et al., 10 Jun 2025). This suggests that CIF difficulty is driven not only by constraint count, but by dependency structure, branch selection, and cross-constraint interference.
3. Benchmarks and evaluation protocols
Recent CIF benchmarks differ primarily in modality, composition style, and scoring protocol.
| Benchmark | Domain | Salient properties |
|---|---|---|
| MMDR-Bench (Han et al., 21 Aug 2025) | Multi-turn image dialogue | 300 scenarios, average 5–7 turns, six evaluation dimensions |
| CIF-Bench (Li et al., 2024) | Chinese instruction following | 150 tasks, 15,000 input–output pairs, public/private split |
| ComplexBench (Wen et al., 2024) | Chinese composed instructions | 1,150 instructions, 5,306 scoring questions, dependency-aware scoring |
| TRACE (Zhang et al., 2024) | Text CIF training/eval | 119,345 training examples, 1,042 evaluation examples |
| EIFBENCH (Zou et al., 10 Jun 2025) | Multi-instruction workflows | 1,000 instances, average 74.01 constraints |
| MultiCodeIF (Duan et al., 1 Jul 2025) | Code generation | 2,021 tasks, hierarchical constraints, multi-turn repair |
| InstructTTSEval (Huang et al., 19 Jun 2025) | Instruction-following TTS | 6,000 items across APS, DSD, and RP |
| VCIFBench (Xu et al., 3 Jun 2026) | Video understanding | 306 satisfiable instructions, 540 DPO pairs, 30 conflict cases |
| TOD-ProcBench (Ghazarian et al., 20 Nov 2025) | Task-oriented dialogue | 55 instruction documents, three CIF tasks |
| CrafText (Volovikova et al., 17 May 2025) | Embodied multimodal world | 496 implemented goals, approximately 3,924 instructions |
Evaluation has become increasingly explicit about the distinction between local and joint adherence. MMDR-Bench scores each turn on a 1–5 scale across visual entity tracking, dialogue consistency, reasoning depth, instruction adherence, error suppression, and response fluency (Han et al., 21 Aug 2025). ComplexBench converts each annotated constraint or composition into a yes/no scoring question and aggregates them with dependency structure through the Decomposed Requirements Following Ratio:
This scoring rule encodes the fact that in Chain or Selection structures, downstream success may depend on upstream branch choice or prior subtask completion (Wen et al., 2024).
Workflow-scale and multimodal benchmarks make the same joint-success issue explicit in different notation. EIFBENCH reports Instruction-Level Accuracy and Constraint-Level Accuracy, plus a full-workflow satisfaction indicator that equals 1 only when every constraint in every instruction is satisfied (Zou et al., 10 Jun 2025). VCIFBench separates instruction-level pass from per-constraint pass using
which makes compounding failure probabilities directly measurable (Xu et al., 3 Jun 2026).
Two further evaluation tendencies are notable. First, many benchmarks now mix rule-based checking with LLM judging. This is explicit in ComplexBench, VCIFBench, CIF-Bench, and InstructTTSEval (Wen et al., 2024, Xu et al., 3 Jun 2026, Li et al., 2024, Huang et al., 19 Jun 2025). Second, several benchmarks deliberately stress generalization rather than contamination-prone surface matching: CIF-Bench uses a public/private split and diversified paraphrases, VCIFBench includes a conflict subset, and CrafText separates paraphrased from new-object generalization (Li et al., 2024, Xu et al., 3 Jun 2026, Volovikova et al., 17 May 2025).
4. Methodological approaches
One major line of work treats CIF as an inference-time systems problem. CoLVLM Agent augments a base LVLM with a modular loop over dialogue memory, visual perception, reasoning/planning, and execution, formalized as
Its key claim is that stronger multi-turn, visually grounded CIF can be obtained without extensive retraining, through iterative memory–perception–planning–execution and optional self-correction (Han et al., 21 Aug 2025).
A second line makes latent instruction structure explicit and verifiable. ImpRIF maps instructions to directed acyclic explicit reasoning graphs whose nodes are conditional, mathematical, or knowledge operations, and uses graph-aligned chain-of-thought plus GRPO with verification-derived rewards (Yang et al., 4 Feb 2026). Light-IF diagnoses “lazy reasoning” as a major source of CIF failure and trains preview-and-self-check behavior through a small cold-start dataset, Entropy-SFT, and token-wise entropy-adaptive RL with dense rule-based rewards (Wang et al., 5 Aug 2025). Both approaches treat successful CIF as a process property: the model should reason in a way that mirrors constraint dependencies rather than only produce an apparently compliant final answer.
A third line is data-centric. CRAB uses constraint back-translation: instead of asking a strong model to generate high-quality constrained responses from scratch, it infers constraints already satisfied by existing high-quality responses and turns them into new CIF training triples (Qi et al., 2024). DeMoRecon generates fine-grained instruction variants by decomposing a prompt into sub-instructions, modifying exactly one sub-component, and minimally reconstructing the prompt, thereby targeting sensitivity to subtle changes in constraint semantics (Yang et al., 2024). UltraIF decomposes real-world instructions into simplified queries, constraints, and evaluation questions, then trains an UltraComposer to synthesize more complex prompts and filter outputs through the associated evaluation questions (An et al., 6 Feb 2025).
Preference optimization has also diversified beyond standard output-side DPO. IOPO trains on matched versus mismatched input–output groups so that the model learns not only which response is better, but which instruction demands which response (Zhang et al., 2024). RPO reverses the constraints that a candidate response violates so that the chosen response becomes perfect under a reversed instruction, reducing noise in preference pairs and enlarging the preference gap (Huang et al., 28 May 2025). MuSC combines coarse-grained constraint-aware preference construction with fine-grained token-aware preference optimization (Huang et al., 17 Feb 2025). MISO, by contrast, modifies training-time attention rather than the preference objective: it restructures a single instruction into multiple sub-contexts and learns a mixture-of-contexts attention scheme while preserving standard decoder-only inference at test time (Lu et al., 17 May 2025).
5. Empirical findings and recurrent failure modes
A striking empirical regularity is the gap between local adherence and joint success. In VCIFBench, the top proprietary models reach approximately 52% instruction pass rate with approximately 86% constraint pass rate, showing that “one-constraint-away” failures dominate even when per-constraint behavior is strong (Xu et al., 3 Jun 2026). EIFBENCH reports the same phenomenon at workflow scale: models achieve much higher Constraint-Level Accuracy than Instruction-Level Accuracy, and the full-satisfaction indicator is near zero in dyadic dialogue for most systems (Zou et al., 10 Jun 2025). MultiCodeIF shows an analogous drop in code generation, where Hard Satisfaction Rate falls from 54.5% in single-level tasks to 18.8% in multi-level scenarios (Duan et al., 1 Jul 2025).
Multi-turn memory, contextual coherence, and error suppression remain persistent bottlenecks. On MMDR-Bench, CoLVLM Agent reaches an average human score of 4.03, ahead of GPT-4o at 3.92 and Gemini 1.5 Pro at 3.85, while also reducing context loss from 15.0% to 5.0% and visual hallucination from 12.0% to 4.0% relative to GPT-4o (Han et al., 21 Aug 2025). Yet the same paper notes that ambiguous references, highly occluded scenes, complex counterfactuals, and longer-horizon dialogues can still induce errors.
Paraphrase robustness and linguistic transfer are also unresolved. CIF-Bench reports that the best model on the private split scores only 52.9%, with 25 of 28 models dropping from public to private data and diversified instructions reducing score variance across models (Li et al., 2024). In task-oriented dialogue, TOD-ProcBench finds that models can retrieve relevant instruction statements much more reliably than they can predict the correct next action or detect violations under instruction-format and multilingual variation (Ghazarian et al., 20 Nov 2025). In embodied CIF, CrafText shows consistent degradation under paraphrased instructions and new object combinations, indicating that grounding and policy adaptation are only partially robust (Volovikova et al., 17 May 2025).
Role abstraction and temporally dynamic expression are especially difficult outside text. In InstructTTSEval, Role-Play is the hardest task and has the lowest human–LLM agreement, while systems broadly struggle with non-verbal vocal events, extreme emotions, and multi-stage transitions (Huang et al., 19 Jun 2025). This is conceptually similar to nested Selection or long Chain failures in text benchmarks: the difficulty lies in coordinating multiple constraints over time rather than satisfying a single local condition.
6. Limitations, controversies, and open directions
The literature is converging on a more precise definition of CIF, but its evaluation stack remains uneven. Many papers rely on LLM-as-judge because CIF often includes open-ended constraints that rule-based metrics cannot cover. ComplexBench improves agreement by combining rules with judging; VCIFBench reports 88.89% agreement and in a human study; InstructTTSEval reports lower agreement for the more subjective Role-Play task and explicitly notes the possibility that Gemini may over-score Gemini outputs (Wen et al., 2024, Xu et al., 3 Jun 2026, Huang et al., 19 Jun 2025). The controversy is therefore not whether LLM-based evaluation is useful, but how much subjectivity and bias remain after calibration.
Reproducibility is another recurring limitation. CoLVLM Agent does not specify prompt templates, retrieval scoring details, dataset splits, or release status for MMDR-Bench (Han et al., 21 Aug 2025). ImpRIF provides formalization and training details, but also notes unresolved issues around checker idiosyncrasies, evaluator bias, and scaling to larger models (Yang et al., 4 Feb 2026). TOD-ProcBench, CrafText, and CIF-Bench each expose related gaps: limited domain coverage, English-only instructions in multilingual evaluation, incomplete dialect or register coverage, or absence of interactive clarification behavior (Ghazarian et al., 20 Nov 2025, Volovikova et al., 17 May 2025, Li et al., 2024).
Future directions are correspondingly concrete. Several papers call for stronger formal criteria for retrieval, adherence checks, plan validation, or conflict detection; broader tool integration; and more efficient training or inference through caching, parallelization, or distillation (Han et al., 21 Aug 2025, Yang et al., 4 Feb 2026). Multimodal work emphasizes richer temporal control, non-verbal event synthesis, longer-video evaluation, and multilingual extension (Huang et al., 19 Jun 2025, Xu et al., 3 Jun 2026). Embodied and dialogue settings point toward interactive CIF: instructions that are clarified, negotiated, revised, or grounded through external tools rather than treated as fixed strings (Volovikova et al., 17 May 2025, Ghazarian et al., 20 Nov 2025).
A broader implication is that CIF research is moving away from a view of instruction following as single-shot response quality. The more recent benchmarks and methods instead treat CIF as strict joint satisfaction under composition, uncertainty, and verification. Under that view, progress depends not only on larger models, but on better representations of constraint structure, stronger process supervision, and evaluation protocols that do not confuse partial compliance with full execution.