---
title: 'CogniQ-H: Soft Hierarchical AutoDP Framework'
url: https://www.emergentmind.com/topics/cogniq-h
type: topic
---

# CogniQ-H: Soft Hierarchical AutoDP Framework

Searching arXiv for the specified paper to verify metadata and grounding.
CogniQ-H is a framework for automated data preparation (AutoDP) that formulates operator selection as a soft hierarchical reinforcement learning problem, with action choice implemented through Bayesian inference rather than a rigid macro-to-micro commitment structure [2507.13710]. In AutoDP, the objective is to construct a sequence of data-transformation operators—such as imputation, encoding, scaling, feature engineering, and feature selection—that, when applied to a raw dataset \(D_{\rm raw}\), maximizes downstream model performance. The framework is motivated by the combinatorial scale of pipeline search and by the observation that conventional flat reinforcement learning does not exploit the semantic structure of preprocessing decisions. CogniQ-H combines a high-level strategic prior generated by a Large Language Model (LLM), a supervised Learning-to-Rank (LTR) score for immediate operator utility, and a reinforcement learning \(Q\)-function for long-term value estimation into a single synergistic policy [2507.13710].

## 1. Problem setting and search-space structure

AutoDP is presented as a sequential decision problem over preprocessing operators. The operator library has \(|\mathcal O|=26\) operators, and for a pipeline length \(T=8\), the search space contains \(\mathcal O(26^8)\approx2.1\times10^{11}\) possible pipelines [2507.13710]. This scale makes exhaustive search infeasible and renders naive exploration highly inefficient.

The paper contrasts two ways of structuring this decision space. In a flat RL formulation, each operator is treated as an independent atomic action at every step. The stated consequence is unguided exploration and poor sample efficiency. In a hierarchical RL formulation, decisions are decomposed into high-level macro-stages, such as Imputation or Encoding, and low-level micro-operators, such as MeanImputer or OneHotEncoder. This decomposition is described as semantically aligned with AutoDP and, in principle, reduces the search complexity to \(\mathcal O(6^5\times4^3)\approx5\times10^5\) [2507.13710].

A central distinction in CogniQ-H is between hard and soft hierarchy. A hard hierarchy irrevocably commits to a macro-stage before choosing a micro-operator. The paper argues that such commitment can prune optimal pipelines prematurely and produce brittle decisions. A plausible implication is that AutoDP differs from classical hierarchical control settings in which stage-level commitment is less costly, because data preprocessing pipelines often benefit from interleaving and revisiting semantically distinct transformations.

## 2. Soft hierarchical formulation as Bayesian inference

CogniQ-H replaces a rigid two-step hierarchy with a single soft hierarchy in which each micro-action is selected by Bayesian inference [2507.13710]. The state space \(\mathcal S\) consists of dataset characteristics, including meta-features capturing missingness and categorical versus numerical counts. The action space is \(\mathcal A=\mathcal O\), namely the full set of micro-operators.

The Bayesian formulation has two components. The prior \(p(a_t\mid s_t)\) represents high-level strategic guidance from an LLM. The likelihood \(p(O=1\mid s_t,a_t)\) represents evidence of immediate and long-term utility through the LTR model and the agent’s own \(Q\)-function. By Bayes’ rule,

\[
p(a_t\mid s_t,O=1)\;\propto\;p(a_t\mid s_t)\;\times\;p(O=1\mid s_t,a_t)\,.
\]

The LLM strategic prior is defined as

\[
p(a_t\mid s_t)\;=\;\sum_{m\in\mathcal M}P_{\rm LLM}(m\mid s_t)\,\mathbf1[a_t\in\mathcal A_m],
\]

where \(\mathcal M\) is the set of \(K\) macro-stages and \(\mathbf1[\cdot]\) is the indicator. The operator-quality likelihood is given by

\[
p(O=1\mid s_t,a_t)\;\propto\;\exp\bigl(\alpha\,Q(s_t,a_t)\;+\;\gamma\,r_{\rm LTR}(a_t\mid s_t)\bigr),
\]

with \(Q(s_t,a_t)\) denoting the learned RL value estimate, \(r_{\rm LTR}(a_t\mid s_t)\) the supervised immediate score, and \(\alpha,\gamma>0\) weighting hyperparameters.

Substitution yields the unnormalized logit

\[
\mathrm{logit}(a_t)\;=\;\alpha\,Q(s_t,a_t)\;+\;\gamma\,r_{\rm LTR}(a_t\mid s_t)\;+\;\beta\,\log P_{\rm LLM}(a_t\mid s_t),
\]

and the final synergistic policy

\[
\pi(a_t\mid s_t)\;=\;\mathrm{softmax}\bigl(\alpha Q(s_t,a_t)\;+\;\beta\log P_{\rm LLM}(a_t\mid s_t)\;+\;\gamma r_{\rm LTR}(a_t\mid s_t)\bigr).
\]

Equivalently,

\[
\pi(a_t\mid s_t)\;\propto\;P_{\rm LLM}(a_t\mid s_t)^{\beta}\;\exp\!\bigl(\alpha\,Q(s_t,a_t)\;+\;\gamma\,r_{\rm LTR}(a_t\mid s_t)\bigr).
\]

This formulation makes the hierarchy soft in the precise sense that macro-level guidance remains probabilistic rather than binding. A common misconception in hierarchical decision systems is that stronger top-down commitment necessarily improves efficiency; the CogniQ-H formulation explicitly rejects that premise by preserving non-zero support over the full action set while still biasing exploration strategically.

## 3. Constituent models and signal fusion

The framework integrates three distinct sources of decision signal [2507.13710]. The LLM planner is prompted with state meta-features \(s_t\) and outputs a distribution \(P_{\rm LLM}(m\mid s_t)\) over macro-stages; the paper gives Llama3.3 as an example. This macro-stage distribution is then marginalized to micro-operators through the prior definition above.

The LTR component is a LightGBM ranker trained offline on tuples \((s,a,\text{performance})\). At runtime it produces \(r_{\rm LTR}(a_t\mid s_t)\), described as a dense and fine-grained score for immediate operator utility. The RL component is a standard Q-Learning or DQN agent that updates \(Q(s,a)\) from environment rollouts and captures long-term pipeline reward.

These three signals are fused directly in the policy. The parameter \(\beta\) controls adherence to the LLM prior, while \(\alpha\) and \(\gamma\) trade off long-term value estimation and immediate supervised scoring. This suggests that CogniQ-H is not merely an ensemble of independent heuristics; rather, it is a unified decision rule in which strategic priors, local operator preferences, and delayed reward estimates contribute at the logit level. A plausible implication is that the design is especially suited to preprocessing settings where sparse task-level rewards and semantically meaningful priors coexist.

## 4. Learning procedure and execution workflow

The paper provides high-level pseudocode for episodic training and deployment [2507.13710]. The system is initialized with a randomly weighted \(Q\)-network, a replay buffer \(\mathcal B\), an offline-trained LTR model, an LLM-planner API, and hyperparameters \(\alpha,\beta,\gamma\). At the start of each episode, the state \(s\) is initialized from dataset meta-features.

At each time step \(t=0,\ldots,T-1\), the procedure first queries the LLM planner to obtain \(\{P_{\rm LLM}(m\mid s)\}\). For every operator \(a\in\mathcal O\), it computes the prior term \(p_{\rm prior}[a]\), the ranking score \(r_{\rm ltr}[a]\), the predicted value \(q_{\rm val}[a]\), and the fused logit
\[
\alpha\cdot q_{\rm val}[a] + \gamma\cdot r_{\rm ltr}[a] + \beta\cdot \log(p_{\rm prior}[a]+\epsilon).
\]
A softmax over these logits defines \(\pi(\cdot\mid s)\), from which an action is sampled and executed.

The transition \((s,a,r_{\rm env},s')\) is inserted into the replay buffer, after which a minibatch is sampled to perform the Q-learning update on the \(Q\)-network. The state is then replaced by \(s'\), and the process continues until the episode ends. This execution pattern preserves the standard RL loop while altering the action-selection mechanism through prior- and likelihood-based modulation.

The workflow is significant because it leaves the action space at the micro-operator level throughout. The hierarchy enters through probabilistic structuring of the policy rather than through a separate controller that enforces stage-level commitment. This suggests a design choice aimed at retaining recoverability from early strategic errors.

## 5. Experimental design and empirical results

The experimental evaluation uses 18 diverse real-world tabular datasets from OpenML, described as the same source used by DiffPrep [2507.13710]. Baselines include flat RL methods (Q-Learning and DQN with \(\epsilon\)-greedy exploration), AI-Pipe RL methods (CtxPipe and HAI-AI), a hard-HRL method (Options-DQN with rigid macro-to-micro selection), zero-shot LLM planners (Llama3.3 and Qwen3), and non-RL AutoML systems (AutoSklearn with Bayesian optimization and TPOT with evolutionary search).

Implementation details reported in the paper include a 3-layer MLP for the \(Q\)-network, replay buffer size \(5\,000\), Llama3.3 with a structured prompt on meta-features, a LightGBM ranker trained offline on state-action-performance logs, an operator library of 26 common preprocessing functions, and Logistic Regression as the downstream model. The evaluation metrics are accuracy of Logistic Regression trained on transformed data, average rank across datasets, convergence speed measured as the number of pipelines evaluated to reach performance thresholds, and pipeline length and running time.

The reported quantitative results place CogniQ-H at an average accuracy of \(0.834\), compared with \(0.806\) for CtxPipe, \(0.818\) for AutoSklearn, \(0.791\) for Q-Learning, and \(0.753\) for DQN. On the avila dataset, CogniQ-H achieves \(0.866\) versus \(0.759\) for CtxPipe, described as a \(+14.1\%\) absolute gain and approximately \(13.9\%\) relative improvement. The average rank is \(2.06\), compared with \(2.83\) for AutoSklearn and \(3.78\) for CtxPipe. Learning curves are reported to show near-optimal accuracy in roughly \(1/2.8\) the number of explored pipelines relative to flat Q-Learning or hard HRL, corresponding to \(2.8\times\) faster convergence.

The framework also produces shorter pipelines. The reported average pipeline length is \(2.72\), compared with \(6.00\) for CtxPipe or hard HRL. Overall runtime is described as competitive with other baselines while yielding higher accuracy. These findings support the paper’s claim that probabilistic hierarchical guidance can improve both search efficiency and final pipeline quality without enforcing a rigid stage structure.

## 6. Claimed contributions, guarantees, and extensions

CogniQ-H is presented as the first formalization of end-to-end AutoDP as a soft hierarchical RL problem and as the first framework to implement a soft hierarchical paradigm for robust, end-to-end automated data preparation [2507.13710]. Its central methodological contribution is the Bayesian fusion of LLM strategic priors, supervised LTR signals, and RL value estimates.

The paper also states several theoretical guarantees. It attributes pruning of nonsensical actions to KL regularization in § 3.4, argues that dense reward shaping from LTR accelerates convergence, and states that non-zero probability for all actions ensures escape from hard-hierarchy traps. Taken together, these claims articulate a specific position in the design space of hierarchical RL for AutoDP: semantic structure should influence exploration, but should not eliminate action recoverability.

Potential extensions identified in the paper include dynamic pipeline lengths or conditional termination criteria, online meta-learning of the hyperparameters \(\alpha,\beta,\gamma\), application to other hierarchical decision problems such as robotics, code generation pipelines, and multi-stage ETL in data engineering, and integration of richer LLM feedback such as chain-of-thought rationales to refine priors. These are framed as future directions rather than established results. A plausible implication is that the CogniQ-H formulation is intended as a general policy-composition template for domains where symbolic or language-derived strategic abstractions can be combined with learned value functions and supervised dense signals.

Source: https://www.emergentmind.com/topics/cogniq-h