GenCtrl -- A Formal Controllability Toolkit for Generative Models

Abstract: As generative models become ubiquitous, there is a critical need for fine-grained control over the generation process. Yet, while controlled generation methods from prompting to fine-tuning proliferate, a fundamental question remains unanswered: are these models truly controllable in the first place? In this work, we provide a theoretical framework to formally answer this question. Framing human-model interaction as a control process, we propose a novel algorithm to estimate the controllable sets of models in a dialogue setting. Notably, we provide formal guarantees on the estimation error as a function of sample complexity: we derive probably-approximately correct bounds for controllable set estimates that are distribution-free, employ no assumptions except for output boundedness, and work for any black-box nonlinear control system (i.e., any generative model). We empirically demonstrate the theoretical framework on different tasks in controlling dialogue processes, for both LLMs and text-to-image generation. Our results show that model controllability is surprisingly fragile and highly dependent on the experimental setting. This highlights the need for rigorous controllability analysis, shifting the focus from simply attempting control to first understanding its fundamental limits.

• The paper introduces a formal framework leveraging control theory to define and quantify controllability in generative models.
• It employs Monte Carlo sampling and PAC bounds to estimate reachable and controllable sets, ensuring statistical guarantees.
• Empirical validation on LLMs and T2IMs reveals model-dependent limits and crucial design considerations for safe controller deployment.

Formal Controllability Analysis of Generative Models: The GenCtrl Toolkit

Motivation and Theoretical Foundation

The GenCtrl framework introduces a rigorous control-theoretic lens to the study of generative models, addressing foundational gaps in prior approaches to controlled generation. Contemporary practice in controllable text/image generation typically assumes the feasibility of steering models to arbitrary outputs via design choices such as prompting, fine-tuning, or representation-level interventions. However, such assumptions implicitly ignore the operational boundaries imposed by the discrete nature of input/output spaces and the nonlinear, opaque dynamics characterizing large-scale models. GenCtrl recasts dialogue with generative models (LLMs, T2IMs) as a nonlinear discrete-time control process. Central definitions of reachability and controllability are properly formalized in output space (not just state), evaluating not mere existence of control mechanisms but the structural possibility of achieving desired attribute values under arbitrary interventions.

A crucial contribution is GenCtrl’s deployment of Monte Carlo algorithms and Probably Approximately Correct (PAC) bounds for estimating reachable and controllable sets. Importantly, these are distribution-free, agnostic to internal model dynamics, and hinge solely on output boundedness. This lays the theoretical foundation for quantifying the fundamental limits of control in generative models—in both discrete and continuous settings—moving beyond empirical controller design to statistically guaranteed feasibility analysis.

Monte Carlo Algorithms and Confidence Guarantees

GenCtrl’s algorithmic core consists of two complementary routines:

1. Reachable Set Estimation: For a given initial state, the framework samples input trajectories, evaluates measurement values via a deterministic or quantized readout map, and aggregates the empirical output set. Coarse-grained quantization permits tractable handling of continuous attributes, mitigating the discrete bottleneck inherent to string-based dialog (see Figure 1).
2. Controllable Set Estimation: By sampling multiple initial states, computing their individual reachable sets, and intersecting these sets, GenCtrl estimates the $\alpha$-controllable region—where a proportion $1-\alpha$ of initial states can be reliably driven to a given output set. Sample complexity is automatically determined by user-specified precision, confidence, and quantization bandwidth, with closed-form PAC bounds guaranteeing probabilistic correctness (Figure 2, Figure 3).

These formal guarantees underpin statistically robust hypothesis testing and safety analysis: target outputs not present in the controllable set estimate are unreachable with prescribed confidence, and all estimates are valid across arbitrary initial state and intervention distributions.

Empirical Analysis: Controllability in Practice

GenCtrl’s framework is empirically validated across a suite of LLMs (SmolLM3-3B, Qwen3-4B, Gemma3-4B) and T2IMs, spanning tasks of formality control, object counting, string length specification, and image attribute manipulation.

LLM Formality Task

GenCtrl reveals that full controllability in LLMs is not guaranteed—even for seemingly trivial output attributes like text formality. In zero-shot settings, all tested models exhibit strong bias and limited reachability. In contrast, five-shot in-context learning markedly improves controllability for larger models, with Qwen3-4B and Gemma3-4B achieving $\mathrm{cvg}=1.0$ at acceptable $\mathrm{MAE}=0.09$ (Figure 4).

Figure 4: Evolution of controllable sets in a five-turn formality control task for multiple LLMs, with yellow regions denoting the $\alpha$-controllable set across initial states.

Figure 5: Larger Qwen models display increased controllability and calibration for text formality, with correlation metrics plateauing for sizes above 8B.

Model scaling is decisive: controllable set size as well as correlation and monotonicity metrics (Spearman $\rho$, Pearson $R$, MAE) improve sharply with parameter count up to saturation at 8–14B (Figure 5).

Structural Attribute Control

For attributes such as string length and average word length, even models with high faithfulness (accuracy, F1) to input requests may not be fully controllable, as reachable sets exhibit sensitivity to both initial states and prompt semantics. Gemma3-4B, for example, achieves high controllability for string length but struggles on average word length tasks with broad spread in calibration metrics (Figure 6, Figure 7).

Figure 6: Output distributions for average word length reveals nontrivial sensitivity to initial state and model tendency towards default responses.

Figure 7: String length controllability and calibration saturate at larger Qwen3 sizes.

Image Generation Tasks

T2IM controllability is highly dependent on task semantics and measurement definition. Control over discrete attributes such as object count is achievable only to limited accuracy, with strong model-dependent variance (Figure 8). More challenging attributes like object placement or image saturation yield poor controllability and low calibration ($\rho, R < 0.1$; Figure 9, Figure 10), even for state-of-the-art diffusion models.

Figure 8: T2IM control on object number task shows limited faithfulness, with best results from FLUX-s (MAE = 3.52).

Figure 9: T2IM fails to place objects at specific image locations, evidencing a strong central bias and sensitivity to object identity.

Figure 10: Attempts to control image saturation display low correlation with model outputs, highlighting response drift.

Implications and Limitations

GenCtrl demonstrates that operational controllability in generative models is fragile, non-universal, and strongly architecture- and task-dependent. This contradicts implicit assumptions underpinning much of contemporary controller design. The framework mandates a paradigm shift: before deploying steering mechanisms, practitioners must rigorously verify the existence and boundaries of controllability for their particular task, prompt, and model distribution. The toolkit is broadly applicable to controller benchmarking, deployment safety, safety region estimation under adversarial input, and compliance auditing.

Framework guarantees are local to specified input and initial state distributions; they do not transfer seamlessly across domains or tasks, and scalability to high-dimensional joint attribute analysis remains an open problem due to quantization-induced sample complexity explosion.

Conclusion

GenCtrl advances the state of the art by providing a formal, model-agnostic toolkit for controllability analysis in generative models. Its core strengths include black-box applicability, explicit PAC guarantees, and empirical findings exposing the nontrivial limits of control in modern AI systems. The package paves the way for safer, more reliable controller design, rigorous safety compliance verification, and foundational research into operational boundaries of generative model intervention. Future directions span controllability-under-training, adversarial robustness estimation, and automated controller synthesis informed by GenCtrl’s principled analyses.