---
title: Model-Agnostic Ideation Framework
url: https://www.emergentmind.com/topics/model-agnostic-ideation-framework
type: topic
---

# Model-Agnostic Ideation Framework

A model-agnostic ideation framework refers to a formalized approach for generating, exploring, or selecting novel ideas or hypotheses that is not bound to a specific learning, reasoning, or generative model architecture. Instead, it provides unified methodologies, workflow abstractions, or modular architectures that can be instantiated atop various model classes—be they probabilistic, neural, cognitive, or agentic—in fields such as scientific discovery, product innovation, AI model interpretation, and group creativity support.

## 1. Principles and Formalisms Underpinning Model-Agnostic Ideation

At the core, a model-agnostic ideation framework is characterized by mechanisms that separate the generative process (how ideas are produced, refined, or selected) from the particulars of the model implementation. In the seminal work "Exploratory Model Building" [1302.6789], ideation is based on constructing hypothetical situation-descriptions defined as $\langle T, R \rangle$, where $T$ is a set of domain attributes and $R$ a set of causal or probabilistic dependencies, drawn from diverse domain databases. The process enables "controlled imagination" through selection and recombination of context-invariant relationships, supporting hypothesis generation unconstrained by any one probability model.

This abstraction manifests across domains; for interpretability in machine learning, the SIPA (Sampling, Intervention, Prediction, Aggregation) framework [1904.03959] offers a four-stage pipeline that unifies existing model-agnostic explanation methods—regardless of the nature of the learned predictor—by focusing on how data is perturbed, predictions are made, and effects are aggregated. Similar model-agnostic design is central to scientific ideation systems [2503.19257], creativity-support tools, and cognitive models for AGI [2301.13556], where dynamic operational memory and operational models enable broad applicability and adaptability.

The shared attribute is that ideation steps (such as sampling, transformation, modular decomposition, or evaluation) are formulated to be independent of internal model specifics, focusing instead on workflow, search, or manipulation in a space of ideas or explanatory constructs.

## 2. Modular Architectures and Representative Pipelines

Model-agnostic ideation frameworks typically employ modular, multi-stage pipelines, where each component can interface with various underlying models:

- **Exploratory Model Building** [1302.6789] decomposes ideation as:
    - Attribute and dependency selection from collective domain knowledge
    - Hypothetical context assembly ($\langle T, R \rangle$)
    - Hypothesis ranking by "interestingness," using probabilistic merit functions

- **Agentic Multi-Agent Systems**: In Agent Ideate [2507.01717], modular agents—such as Patent Analyst, Keyword Extractor, Business Idea Generator, and Validator—perform subtasks in parallel or sequence, connected via standardized protocols (e.g., JSON), allowing agents to be backed by any LLM or reasoning tool.

- **Latent Space Exploration**: The latent-space ideation framework [2507.13874] routes ideas through
    1. A semantic encoder mapping input text to latent representations,
    2. A latent explorer (for interpolation, extrapolation, or noise perturbation),
    3. A cross-modal projector for token embedding alignment,
    4. A decoder LLM, and
    5. An evaluator LLM for originality and relevance scoring.

    This design, illustrated in Figure 1 of [2507.13874], is modular: each stage’s implementation may be swapped out or tuned to suit the task.

- **Human-in-the-Loop and Feedback Integration**: Frameworks such as IRIS [2504.16728] integrate human agency at multiple junctures, employing test-time compute expansion strategies (e.g., Monte Carlo Tree Search) while keeping human feedback central for validation and refinement.

## 3. Evaluation, Novelty, and Interestingness Criteria

A distinguishing feature is the formalization of what qualifies as an "interesting" or "novel" idea.

- In probabilistic frameworks [1302.6789], "interestingness" of a context $S$ is given by the merit function $F(S) = P[d = d^* | E(S)]$, with $d^*$ as the desired outcome and $E(S)$ as the set of constraints. During search (e.g., via A*), hypothetical scenarios are ranked by this measure, not mere observed likelihood.

- In SCI-IDEA [2503.19257], idea novelty is operationalized using cosine similarity:
    $$
    \text{Novelty}(c_i) = 1 - \max(\text{Cosine\_similarity}(c_i, c_j)),\quad \forall c_j \in \mathcal{C}
    $$
  Surprise is computed as $-\log p(c_i|\mathcal{C})$, where $p$ is model-estimated probability.

- In agentic frameworks for patent-based ideation [2507.01717], outputs are evaluated across multiple criteria including technical validity, innovativeness, specificity, and market need, typically by an LLM-as-a-judge protocol or by human raters. Aggregation may follow a scoring formula such as $E = \sum_{i=1}^{6} w_i \cdot c_i$ with $w_i$ as weights and $c_i$ as criterion scores.

## 4. Adaptability and Domain-Agnostic Implementation

Adaptability is central to model-agnostic ideation frameworks:

- **Plug-and-Play Modules**: Systems like Agent Ideate [2507.01717] and Acceleron [2403.04382] separate agent logic from LLM internals, allowing rapid substitution of back-end models (e.g., swapping GPT-4, Llama-3, or custom domain models without pipeline changes).

- **Generalized Workflow Templates**: The Hourglass Ideation Framework [2503.00946] abstracts the full ideation cycle across preparation (scope, material structuring), divergence (generation, refinement), and convergence (evaluation, selection) with iterative human and LLM interplay. This structure was shown to map onto 61 distinct studies across individual and group ideation, regardless of the underlying AI system.

- **Flexible Data and Task Formats**: Model-agnostic frameworks are applicable to images, text, tabular data, and more [2006.03896]. In video captioning, the MAMS framework [2501.18269] selects modules and attention masks dynamically based on video content, improving performance across SwinBERT, mPLUG-2, and UniVL, further demonstrating architectural independence.

## 5. Applications Across Domains

Model-agnostic ideation frameworks have been deployed across a spectrum of domains:

- **Scientific Hypothesis Generation**: Systems such as IRIS [2504.16728] and SCI-IDEA [2503.19257] use LLM-powered agents, semantic embeddings, and human-in-the-loop processes to generate, refine, and evaluate research hypotheses, impacting ML, NLP, chemistry, and physics.

- **Product and Business Innovation**: Agent Ideate [2507.01717] automates the interpretation of patent data for novel business idea extraction, outperforming standalone LLMs by leveraging multi-agent orchestration, external search, and modular validation tools.

- **Explainable AI and Interpretability**: Exemplar synthetization [2006.03896] uses generative models (VAEs, GANs) and evolutionary search to produce exemplars across data types for model explanation without internal access, while frameworks like MACE [2205.15540] provide model-agnostic counterfactuals even for non-differentiable models.

- **Creativity and Group Ideation**: Brainwriting frameworks [2402.14978] and LLM-assisted hourglass pipelines [2503.00946] harness LLMs for both idea expansion and evaluation, with feedback loops supporting divergence and convergence phases in group and educational settings.

- **Adaptive Testing**: The MAAT framework [2101.05986] formulates question selection for computerized adaptive tests so as to maximize both information gain (quality) and knowledge concept coverage (diversity), regardless of the test-taker model.

## 6. Emerging Challenges and Future Directions

While model-agnostic ideation frameworks provide significant flexibility and generality, several open challenges and avenues have been identified:

- **Evaluation Bottlenecks**: Frameworks that incorporate an LLM-as-a-judge [2507.01717] or use proprietary LLMs for idea scoring [2507.13874] may face limitations related to evaluation objectivity, scalability, or domain mismatch. Future work includes developing lightweight, domain-specific, or human-in-the-loop evaluators.

- **Exploration Strategy Enhancement**: Latent space ideation pipelines [2507.13874] have so far used conservative exploration strategies. More ambitious approaches such as swarm-based search or adaptive exploration tuned by user feedback are indicated as promising paths.

- **Collaborative and Multimodal Ideation**: Much current research targets individual, text-based workflows [2503.00946], leaving room for richer, multimodal, or synchronous group-centric designs.

- **Ethical Considerations**: Frameworks such as SCI-IDEA [2503.19257] emphasize the need for attribution transparency, misuse mitigation, and balanced integration of human and AI creativity, especially as LLM-powered ideation systems become more prevalent.

- **Unified Taxonomies and Process Models**: The emergence of the Hourglass Ideation Framework [2503.00946] and similar structural templates suggests a trend towards formal process models that can index, compare, or integrate systems and studies, facilitating knowledge transfer and methodological rigor.

## 7. Representative Models and Formulas

A selection of representative mathematical and algorithmic formulations from the literature:

- **Exploratory merit function** ([1302.6789]):
    $$
    F(S) = P[d = d^* | E(S)]
    $$
- **Latent space interpolation/perturbation** ([2507.13874]):
    $$
    e_\text{new} = \lambda e_i + (1-\lambda) e_j
    $$
    $$
    e_\text{new} = e_i + \epsilon, \quad \epsilon \sim \mathcal{N}(0, \sigma^2 I)
    $$
- **SIPA workflow stages** ([1904.03959]):

    | Stage          | Description                                                                              |
    |----------------|------------------------------------------------------------------------------------------|
    | Sampling       | Select data points or features to perturb                                                |
    | Intervention   | Modify data (e.g., permute, set to fixed value, add noise)                              |
    | Prediction     | Compute model output on intervened data                                                  |
    | Aggregation    | Summarize results (e.g., mean/variance, fit surrogate, compare losses)                   |

- **MAAT importance weighted knowledge coverage** ([2101.05986]):
    $$
    \operatorname{IWKC}(Q_T) = \frac{ \sum_k w_k \cdot \operatorname{IncCov}(k, Q_T) }{ \sum_k w_k }
    $$
    where $\operatorname{IncCov}(k, Q_T) = \frac{ \operatorname{cnt}(k, Q_T) }{ \operatorname{cnt}(k, Q_T) + 1 }$

- **SCI-IDEA novelty and surprise** ([2503.19257]):
    $$
    \text{Novelty}(c_i) = 1 - \max_{c_j \in \mathcal{C}} \text{Cosine\_similarity}(c_i, c_j)
    $$
    $$
    \text{Surprise}(c_i) = -\log p(c_i | \mathcal{C})
    $$

- **IRIS MCTS Upper Confidence Bound** ([2504.16728]):
    $$
    \operatorname{UCT}(n) = \frac{ Q(n) }{ N(n) } + c \sqrt{ \frac{ \ln( N(n_p) ) }{ N(n) } }
    $$

## Summary Table: Notable Model-Agnostic Ideation Frameworks

| Framework                    | Domain                    | Key Mechanisms/Features                                             |
|------------------------------|---------------------------|---------------------------------------------------------------------|
| Exploratory Model Building    | Hypothesis generation     | Probabilistic recombination; context assembly; merit-based search   |
| SIPA                         | Interpretability (ML)     | Unified perturbation–prediction–aggregation stages                  |
| SCI-IDEA                     | Scientific ideation       | Prompting, novelty/surprise metrics, embedding-based Aha detection  |
| IRIS                         | Scientific ideation       | Multi-agent, MCTS, human-in-the-loop, query-based retrieval         |
| Agent Ideate                 | Product innovation        | Multi-agent, LLM+tool, modular, LLM-as-a-Judge evaluation          |
| MAMS                         | Video captioning          | Module/token selection, adaptive attention, architecture-agnostic   |
| MAAT                         | Adaptive testing          | Active learning-inspired, quality/diversity/importance modules      |
| LLM Hourglass Framework      | General ideation          | Preparation–divergence–convergence pipeline, model/task-agnostic    |

Model-agnostic ideation frameworks are transforming creative AI, hypothesis generation, product and scientific innovation, and model interpretability by enabling systematic, modular, and adaptive processes untethered from the constraints of any single model architecture. These frameworks facilitate not only creativity and flexibility in computational systems but also support rigorous evaluation, reproducibility, and cross-domain innovation.

Source: https://www.emergentmind.com/topics/model-agnostic-ideation-framework