Ground-Compose-Reinforce: Compositional RL Schema

• Ground-Compose-Reinforce is a compositional RL paradigm that separates atomic grounding, task composition, and reinforcement to improve sample efficiency and generalization.
• The framework applies distinct operations—pretraining on invariant perceptual features, recomposing complex task specifications, and using RL objectives—with applications in 3D navigation, semantic parsing, and formal-language tasks.
• Empirical findings show that modular architectures and curriculum learning reduce training episodes dramatically while enhancing zero-shot performance on unseen task combinations.

Searching arXiv for the named framework and closely related papers to ground the article in current literature. Tool call: arxiv_search({"^^^^^^^^6query6 OR ti:\6^^^^^^^^"Ground-Compose-Reinforce\" OR ti:\6^^^^^^^^"Compositional Learning of Visually-Grounded Concepts Using Reinforcement\"^^^^^^^^6 OR ti:\6^^^^^^^^"Human-like compositional learning of visually-grounded concepts using synthetic environments\"^^^^^^^^6 OR ti:\6^^^^^^^^"Compositional Instruction Following with LLMs and Reinforcement Learning\"","max_results":^^^^6all:\6query6^^^^,"sort_by":"relevance"}) Ground-Compose-Reinforce denotes a recurrent three-stage schema in recent reinforcement-learning and vision-language research: an agent first grounds atomic concepts or symbols in perceptual inputs, then composes those grounded units into more complex task specifications, and finally reinforces behavior through an RL objective defined over the resulting task structure. The phrase appears explicitly in work on visually grounded navigation, compositional instruction following, and formal-language tasking, but the concrete implementation varies substantially across papers: grounding may mean invariant concept acquisition in a synthetic ^^^^6 OR ti:\6^^^^D room, semantic parsing into Boolean task expressions, learned labelling of atomic propositions for Reward Machines, or explicit evidence localization for intermediate reasoning steps (&&&^^^^6query6^^^^&&&, &&&^^^^6all:\6^^^^&&&, &&&^^^^6 OR ti:\6^^^^&&&, &&&^^^^6 OR ti:\6^^^^&&&, &&&^^^^6 OR ti:\6^^^^&&&).

^^^^6all:\6^^^^. Conceptual schema and scope

Across the literature, the three verbs refer to distinct operations. In the ^^^^6 OR ti:\6^^^^D navigation work on visually grounded concepts, Ground denotes pretraining on single-attribute RL tasks so that the agent learns color- or shape-invariance; Compose denotes fine-tuning on conjunctions such as color-shape targets; and Reinforce denotes the fact that the RL objective alone, together with the curriculum from simpler to more complex environments, is used to induce this behavior (&&&^^^^6query6^^^^&&&). In the later synthetic-environment extension, the same pattern is applied to richer linguistic concept classes, including determiners and prepositions, again with curriculum learning as the main mechanism for improving efficiency (&&&^^^^6all:\6^^^^&&&).

A different instantiation appears in CERLLA, where Ground is semantic parsing of a natural-language instruction into a Boolean task expression, Compose is closed-form assembly of a task policy from primitive world-value functions, and Reinforce is used both to learn the primitives and to refine the parser context through rollout-based success feedback (&&&^^^^6 OR ti:\6^^^^&&&). In the formal-language framework that explicitly adopts the name Ground-Compose-Reinforce, Ground is a learned labelling function from states to atomic propositions, Compose is value composition under Reward Machine semantics, and Reinforce is PPO on an augmented MDP with potential-based reward shaping (&&&^^^^6 OR ti:\6^^^^&&&).

This distribution of meanings suggests that Ground-Compose-Reinforce is best understood not as a single algorithm but as a family of compositional design principles. The commonality lies in the separation of atomic grounding from downstream task composition, followed by RL over the composed task representation.

Paper	Ground / Compose / Reinforce instantiation	Representative claim
(&&&^^^^6query6^^^^&&&)	Single-attribute concept pretraining / color-shape recomposition / A^^^^6 OR ti:\6^^^^C navigation	Concept pretraining reduced episodes for zero-shot compositional learning by about ^^^^6 OR ti:\6query6^^^^ times
(&&&^^^^6all:\6^^^^&&&)	Grounding determiners and prepositions / held-out instruction recombination / actor-critic with curriculum	Curriculum reduced required training episodes by ^^^^6all:\6 OR ti:\6^^^^% in determiner environments and enabled prepositional learning
(&&&^^^^6 OR ti:\6^^^^&&&)	LLM semantic parsing / min-max-Boolean value composition / DQN plus RL-driven in-context refinement	Reached the oracle upper-bound success rate of 9^^^^6 OR ti:\6^^^^% on ^^^^6all:\66 OR ti:\6^^^^ tasks
(&&&^^^^6 OR ti:\6^^^^&&&)	Learned proposition labelling / Reward Machine composition / PPO with reward shaping	End-to-end baselines failed to generalize to unseen compositions

^^^^6 OR ti:\6^^^^. Visually grounded compositional learning in synthetic ^^^^6 OR ti:\6^^^^D navigation

The ^^^^6 OR ti:\6query6 OR ti:\6 OR ti:\6^^^^ navigation study provides one of the clearest early formulations of the pattern. The environment is a first-person synthetic ^^^^6 OR ti:\6^^^^D room in Unity in which an agent must navigate to a single target object among four spawned at fixed positions. Static landmarks include a door, window, shelf, and human figure; the objects are drawn from five shapes—capsule, cube, cylinder, prism, sphere—and five colors—red, green, blue, yellow, black. Each object is identified by a PRESERVED_PLACEHOLDER_^^^^6query6^^^^ pair, and instructions may specify a conjunction such as “red cube,” a three-word combination such as “green sphere prism,” or a single attribute such as “blue” (&&&^^^^6query6^^^^&&&).

The agent architecture is a standard multimodal recurrent actor-critic. A PRESERVED_PLACEHOLDER_^^^^6all:\6^^^^ RGB frame is encoded by three convolutional layers into 6^^^^6 OR ti:\6^^^^ feature maps of size PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6^^^^, flattened into a ^^^^6 OR ti:\6all:\6 OR ti:\66^^^^-dimensional vector. Language is represented either by two one-hot vectors for color and shape, each linearly mapped to ^^^^6all:\6 OR ti:\6^^^^8 dimensions and concatenated, or by a frozen pretrained text encoder such as CLIP or BERT followed by a ^^^^6all:\6 OR ti:\6^^^^8-dimensional projection. The ^^^^6 OR ti:\6all:\6 OR ti:\66^^^^-dimensional visual embedding and ^^^^6all:\6 OR ti:\6^^^^8-dimensional language embedding are concatenated, passed through a ^^^^6 OR ti:\6 OR ti:\66^^^^-dimensional mixing layer, and then through an LSTM whose hidden state feeds an actor head over PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6^^^^ and a critic head PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6^^^^ (&&&^^^^6query6^^^^&&&).

Training uses synchronous A^^^^6 OR ti:\6^^^^C with RMSProp at learning rate PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6^^^^. The reward function gives $+10$ for reaching the correct target, $-3$ for colliding with non-target objects, $-1$ for hitting walls, and $-10$ for exceeding ^^^^6 OR ti:\6query6query6^^^^ steps. The performance criterion is an average episodic return of at least 9 over ^^^^6all:\6query6query6^^^^ consecutive episodes. Concept learning is not enforced by auxiliary contrastive or classification losses; rather, it is operationalized by RL pretraining in the single-attribute PRESERVED_PLACEHOLDER_^^^^6all:\6query6^^^^ environment under

PRESERVED_PLACEHOLDER_^^^^6all:\6all:\6^^^^

followed by compositional training in PRESERVED_PLACEHOLDER_^^^^6all:\6 OR ti:\6^^^^ under

PRESERVED_PLACEHOLDER_^^^^6all:\6 OR ti:\6^^^^

The policy objective is

PRESERVED_PLACEHOLDER_^^^^6all:\6 OR ti:\6^^^^

The key empirical result is that agents trained naively on ^^^^6 OR ti:\6query6^^^^ of the ^^^^6 OR ti:\6 OR ti:\6^^^^ possible color-shape pairs nevertheless learned to decompose and recompose those combinations, achieving zero-shot success on the ^^^^6 OR ti:\6^^^^ held-out pairs. The one-hot agent required about 67K episodes to meet the training criterion and about 9^^^^6 OR ti:\6^^^^K to succeed zero-shot on held-out combinations. When pretrained on single-attribute concepts for ^^^^6all:\66^^^^8K episodes and then fine-tuned on PRESERVED_PLACEHOLDER_^^^^6all:\6 OR ti:\6^^^^, the same system required only ^^^^6query6^^^^.6K episodes to learn the ^^^^6 OR ti:\6query6^^^^ training pairs and ^^^^6 OR ti:\6^^^^.^^^^6 OR ti:\6^^^^K episodes to reach zero-shot performance on held-out combinations. Only agents trained on both concept and compositional learning solved a more complex out-of-distribution environment in zero-shot fashion, and only text encoders pretrained on image-text datasets, such as CLIP, reduced the number of episodes needed for compositional learning while also generalizing to five unseen colors—orange, cyan, pink, purple, white—with mean return about 6.9 (&&&^^^^6query6^^^^&&&).

^^^^6 OR ti:\6^^^^. Determiners, prepositions, and curriculum as a compositional scaffold

The ^^^^6 OR ti:\6query6 OR ti:\6 OR ti:\6^^^^ extension generalizes the synthetic-environment program from color-shape conjunctions to richer linguistic concept classes, specifically determiners and prepositions. Navigation-from-instruction is formalized as an MDP PRESERVED_PLACEHOLDER_^^^^6all:\66^^^^ in which the state PRESERVED_PLACEHOLDER_^^^^6all:\6^^^^7 contains the raw RGB image PRESERVED_PLACEHOLDER_^^^^6all:\6^^^^8 and a discrete instruction PRESERVED_PLACEHOLDER_^^^^6all:\6^^^^9 of length PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6query6^^^^ words, with PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6all:\6^^^^ in determiner environments, PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6 OR ti:\6^^^^ in preposition environments, and PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6 OR ti:\6^^^^ in combined determiner-plus-preposition environments. The action space remains PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6 OR ti:\6^^^^, transitions are deterministic physics-based rollouts in Unity, and the reward is PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6 OR ti:\6^^^^ for reaching the correct target, PRESERVED_PLACEHOLDER_^^^^6 OR ti:\66^^^^ for reaching a wrong target, PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6^^^^7 for colliding with walls, and PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6^^^^8 if no target is reached by PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6^^^^9 steps, with PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6query6^^^^ typically set to PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6all:\6^^^^ (&&&^^^^6all:\6^^^^&&&).

The instruction vocabulary is partitioned into determiners, prepositions, colors, and shapes. Determiners are PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6 OR ti:\6^^^^, and prepositions are PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6 OR ti:\6^^^^. Each word slot is one-hot encoded, mapped through a shared linear embedding to ^^^^6all:\6 OR ti:\6^^^^8 dimensions, and concatenated into a ^^^^6all:\6 OR ti:\6^^^^8-dimensional language representation. Visual processing again uses three convolutional layers producing a ^^^^6 OR ti:\6all:\6 OR ti:\66^^^^-dimensional vector, followed by linear fusion to ^^^^6 OR ti:\6 OR ti:\66^^^^ dimensions and an LSTM. Actor and critic heads are defined by

PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6 OR ti:\6^^^^

with hidden size PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6 OR ti:\6^^^^ (&&&^^^^6all:\6^^^^&&&).

The central intervention is curriculum learning. For determiners, tasks are ordered as ^^^^6 OR ti:\6^^^^ simple determiners, then ^^^^6 OR ti:\6^^^^ mid-level determiners, then all 8. For prepositions, the sequence is PRESERVED_PLACEHOLDER_^^^^6 OR ti:\66^^^^. Progression requires at least 8^^^^6query6^^^^% success—described as average PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6^^^^7 reward—over ^^^^6all:\6query6query6query6^^^^ consecutive episodes. This scaffold materially changes sample efficiency. Training all eight determiners from scratch required ^^^^6query6^^^^.87M episodes; the PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6^^^^8 curriculum required PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6^^^^9, and the PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6query6^^^^ curriculum required PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6all:\6^^^^. Prepositions were substantially harder: naive training on 8P failed to converge by ^^^^6 OR ti:\6^^^^.^^^^6 OR ti:\6^^^^M episodes, whereas PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6 OR ti:\6^^^^ required PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6 OR ti:\6^^^^ and PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6 OR ti:\6^^^^ required PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6 OR ti:\6^^^^, reported as about ^^^^6 OR ti:\6^^^^8% fewer episodes versus naive extrapolation (&&&^^^^6all:\6^^^^&&&).

The generalization results are correspondingly stratified. Held-out determiner instructions yielded about 77–79% success on test, close to training performance. Held-out preposition instructions reached about 76% success, compared with ^^^^6 OR ti:\6 OR ti:\6^^^^% for non-converged agents. In the combined PRESERVED_PLACEHOLDER_^^^^6 OR ti:\66^^^^ setting with ^^^^6all:\66query6^^^^,^^^^6query6query6query6^^^^ instructions, pretraining on prepositions with PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6^^^^7 and then fine-tuning for only ^^^^6query6^^^^.^^^^6all:\6 episodes produced ^^^^6 OR ti:\6 OR ti:\6^^^^% success in a zero-shot PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6^^^^8 test set, whereas removing curriculum or pretraining abolished convergence in preposition environments and prevented generalization to PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6^^^^9 (&&&^^^^6all:\6^^^^&&&). A common misconception is that compositional generalization in RL is uniform across linguistic operators; these results indicate that relational concepts such as prepositions are substantially more difficult than determiner concepts under otherwise similar architectures and objectives.

^^^^6 OR ti:\6^^^^. Semantic parsing and value-function composition

CERLLA demonstrates a more explicitly symbolic variant of Ground-Compose-Reinforce. Instructions are parsed by a frozen LLM, such as GPT-^^^^6 OR ti:\6^^^^, into Boolean expressions over primitive symbols using operators PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6query6^^^^, PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6all:\6^^^^, and PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6 OR ti:\6^^^^. At each episode, the parser is given a short system instruction, up to PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6 OR ti:\6^^^^ retrieved in-context examples from a set PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6 OR ti:\6^^^^ via BM^^^^6 OR ti:\6 OR ti:\6^^^^, and the new instruction PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6 OR ti:\6^^^^, then returns PRESERVED_PLACEHOLDER_^^^^6 OR ti:\66^^^^ candidate parses. There is no gradient-based parser loss in the implemented system; parser improvement occurs by augmenting PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6^^^^7 with successful examples identified through rollout-based evaluation (&&&^^^^6 OR ti:\6^^^^&&&).

Composition is performed by a bank of primitive world-value functions PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6^^^^8, one per primitive attribute, pretrained by DQN under an augmented reward that strongly penalizes reaching the wrong goal. Closed-form composition then yields a task PRESERVED_PLACEHOLDER_^^^^6 OR ti:\6^^^^9-function:

$+10$^^^^6query6^^^^

$+10$^^^^6all:\6^^^^

Given a Boolean formula $+10$^^^^6 OR ti:\6^^^^, the same operator tree is applied recursively to obtain $+10$^^^^6 OR ti:\6^^^^, from which the policy is induced by softmax with temperature $+10$^^^^6 OR ti:\6^^^^ or by greedy action selection (&&&^^^^6 OR ti:\6^^^^&&&).

Reinforcement enters twice. First, each primitive $+10$^^^^6 OR ti:\6^^^^ is learned with the temporal-difference objective

$+10$6

Second, candidate parses are filtered by policy performance: each candidate $+10$7 induces a policy $+10$8 that is evaluated over $+10$9 rollouts, producing empirical success rate $-3$^^^^6query6^^^^. If $-3$^^^^6all:\6^^^^, the pair $-3$^^^^6 OR ti:\6^^^^ is added to or replaces an entry in the in-context example set $-3$^^^^6 OR ti:\6^^^^ (&&&^^^^6 OR ti:\6^^^^&&&).

The reported sample-complexity profile is highly asymmetric between primitive learning and downstream adaptation. CERLLA learns $-3$^^^^6 OR ti:\6^^^^ primitives in about ^^^^6all:\6^^^^9M environment steps, then accumulates about ^^^^6all:\66 OR ti:\6^^^^ in-context examples in another ^^^^6query6^^^^.6M steps, for about ^^^^6all:\6^^^^9.6M total steps to reach 9^^^^6 OR ti:\6^^^^% average success on all ^^^^6all:\66 OR ti:\6^^^^ tasks. The best non-compositional baseline, a single Q-network over language and vision, is trained for ^^^^6 OR ti:\6all:\6^^^^M steps but reaches only about 8^^^^6query6^^^^% success. On held-out splits of 8^^^^6all:\6^^^^ training and 8^^^^6all:\6^^^^ test tasks, CERLLA reaches about 9^^^^6query6^^^^% on both train and test in about ^^^^6all:\6^^^^M steps, whereas the non-compositional baseline maintains a substantial generalization gap even after ^^^^6 OR ti:\6all:\6^^^^M steps (&&&^^^^6 OR ti:\6^^^^&&&). Here, “compose” is not latent recombination inside a recurrent policy but explicit algebra over reusable value functions.

^^^^6 OR ti:\6^^^^. Formal-language tasking with Reward Machines

The most formal version of Ground-Compose-Reinforce is the neurosymbolic framework based on Reward Machines. It adopts Reward Machines as the task-specification language, with atomic propositions $-3$^^^^6 OR ti:\6^^^^ and propositional conditions generated by

$-3$6

An RM is $-3$7, where $-3$8 and $-3$9 define state transitions and instantaneous rewards over truth assignments to the atomic propositions. Given a learned labelling function $-1$^^^^6query6^^^^, the induced reward and automaton transition are

$-1$^^^^6all:\6^^^^

Grounding is thus the problem of learning $-1$^^^^6 OR ti:\6^^^^ from labelled state data (&&&^^^^6 OR ti:\6^^^^&&&).

The grounding module is dataset-driven. In GeoGrid, the labeler is a two-layer convolutional network with Conv$-1$^^^^6 OR ti:\6^^^^ReLU−1-1^^^^6 OR ti:\6^^^^Conv−1-1^^^^6 OR ti:\6^^^^ReLU−1-16flatten−1-17MLP−1-18^^^^6 OR ti:\6^^^^-way logits. In DrawerWorld, it is a two-layer MLP $-1$9. The objective is binary cross-entropy per proposition:

$-10$^^^^6query6^^^^

Composition occurs at the value-function level. Primitive value functions for eventual satisfaction of literals, $-10$^^^^6all:\6^^^^ and $-10$^^^^6 OR ti:\6^^^^, are learned once; DNF decomposition then uses max over clauses and min over literals, and a high-level value iteration over the RM graph approximates $-10$^^^^6 OR ti:\6^^^^ for arbitrary Reward Machines (&&&^^^^6 OR ti:\6^^^^&&&).

Reinforcement is performed on the augmented MDP with state $-10$^^^^6 OR ti:\6^^^^ under PPO and shaped reward

$-10$^^^^6 OR ti:\6^^^^

The data regime is deliberately small relative to downstream task complexity: GeoGrid uses ^^^^6 OR ti:\6^^^^,^^^^6query6query6query6^^^^ random-policy episodes of length ^^^^6all:\6query6query6^^^^ labelled automatically, and DrawerWorld uses ^^^^6 OR ti:\6 OR ti:\6query6^^^^ manually controlled episodes performing generic interactions. Downstream tasks include temporally extended sequences, loops, logical ordering, and safety constraints, some of which require behaviors never seen together in the grounding data. In these experiments, the framework is reported as the only method to consistently solve all tasks in both domains, with end-to-end baselines failing to generalize to unseen compositions; ablations further show that the compositional reward shaping is critical in sparse settings such as DrawerWorld (&&&^^^^6 OR ti:\6^^^^&&&).

6. Antecedents, extensions, and interpretive boundaries

A control-theoretic antecedent appears in compositional RL for discrete-time stochastic control systems. There, the same triad can be recognized as Ground through implicit finite abstraction of each subsystem, Compose through assume-guarantee synthesis over a network of subsystems and conjunctions of scLTL specifications, and Reinforce through minimax-Q learning on the product of each abstraction with an automaton-derived reward structure. The framework provides an abstraction error bound

$-10$6

and a compositional lower bound on global satisfaction probability under decentralized policies. Sparse automaton rewards are mitigated by potential-based shaping, and multi-level discretization is used as a warm start to accelerate tabular minimax-Q learning (&&&^^^^6 OR ti:\6^^^^7&&&). Although the paper does not use the term Ground-Compose-Reinforce as a formal label, it supplies a closely related structural template.

A later extension broadens the phrase into grounded visual reasoning rather than navigation or formal tasking. H-GRPO uses a pretrained vision encoder and an LLM decoder to generate grounded reasoning triplets $-10$7, where each sub-question and sub-answer is paired with a localized evidence bounding box. The RL objective uses a composite reward

$-10$8

with permutation invariance enforced by Hungarian matching over predicted and reference triplets. The training dataset contains ^^^^6all:\6query6^^^^K automatically synthesized grounded chains from Visual7W, Visual-CoT, A-OKVQA, and ERQA, plus a gold-standard seed set of ^^^^6 OR ti:\6query6^^^^ human-verified examples. Reported gains include improvements from ^^^^6 OR ti:\6 OR ti:\6^^^^.^^^^6 OR ti:\6^^^^% to 7^^^^6query6^^^^.^^^^6 OR ti:\6^^^^% on RoboSpatial and from ^^^^6 OR ti:\66^^^^.^^^^6query6^^^^% to ^^^^6 OR ti:\6 OR ti:\6^^^^.^^^^6 OR ti:\6^^^^% on MMStar for Qwen^^^^6 OR ti:\6^^^^.^^^^6 OR ti:\6^^^^-VL under H-GRPO, along with an interpretability score of ^^^^6 OR ti:\6^^^^.7^^^^6 OR ti:\6^^^^ on a ^^^^6all:\6^^^^–^^^^6 OR ti:\6^^^^ scale (&&&^^^^6 OR ti:\6^^^^&&&). This does not redefine the earlier RL-tasking frameworks, but it shows that the same ground-compose-reinforce motif can be transferred to evidence-grounded reasoning chains.

Two interpretive boundaries recur across the literature. First, Ground-Compose-Reinforce is not synonymous with generic language-conditioned RL. The grounding object may be perceptual invariances, logical forms, atomic propositions, or evidence regions, and the composition operator may be recurrent recombination, curriculum-mediated recomposition, Boolean min/max algebra, Reward Machine semantics, or permutation-invariant matching. Second, zero-shot or few-shot compositionality does not appear as an automatic by-product of large-scale training alone. In these studies, gains are tied to structural scaffolds: single-attribute pretraining, curriculum ordering, primitive value reuse, formal language semantics, or intermediate evidence supervision (&&&^^^^6query6^^^^&&&, &&&^^^^6all:\6^^^^&&&, &&&^^^^6 OR ti:\6^^^^&&&, &&&^^^^6 OR ti:\6^^^^&&&, &&&^^^^6 OR ti:\6^^^^&&&).

Taken together, the literature positions Ground-Compose-Reinforce as a technical program for separating atomic grounding from higher-order task composition while preserving RL as the mechanism for behavior acquisition. The most conservative reading is that this separation improves sample efficiency, strengthens out-of-distribution generalization to unseen combinations, and makes the task structure more explicit than monolithic end-to-end baselines under the experimental settings reported in these papers.