AlphaContext: An Evolutionary Tree-based Psychometric Context Generator for Creativity Assessment

Abstract: Creativity has become a core competence in the era of LLMs and human-AI collaboration, underpinning innovation in real-world problem solving. Crucially, the systematic improvement of creativity necessitates scientifically valid assessment instruments. Psychometric research recognizes context-based assessment as an effective way to measure creative thinking. However, high-quality expert-designed contexts remain scarce. Existing LLM-based generators often struggle with insufficient assessment cues, weak narrative coherence, limited stylistic diversity, and poor support for creative thinking. To address these challenges, we propose AlphaContext, an evolutionary tree-based psychometric context generator for creativity assessment. First, the HyperTree Outline Planner formalizes expert-designed outlining as a rule-guided hypertree and performs top-down hierarchical planning. The MCTS-based Context Generator fills the outline via MCTS to balance global structure and local quality. Then, the Evolutionary Context Optimizer evolves contexts with MAP-Elites by repeatedly updating niche elites to jointly improve diversity and quality. Finally, the Assessment-Guided Evolution Refiner simulates virtual participants with diverse styles and recycles weak contexts for further evolution. Experiments show that AlphaContext yields an average improvement of 8% over competitive methods across 6 quality metrics.

• The paper introduces a novel modular pipeline that integrates hypertree-based planning with MCTS-driven generative search to automate creativity assessment context generation.
• It employs MAP-Elites evolutionary optimization to refine context diversity and quality, significantly improving coherence, engagement, and psychometric alignment.
• Empirical results demonstrate AlphaContext's superior performance over traditional baselines, reducing expert labor while achieving robust creativity assessment metrics.

AlphaContext: An Evolutionary Tree-based Psychometric Context Generator for Creativity Assessment

Motivation and Foundations

Systematic creativity assessment is a pivotal concern in psychometrics and educational AI, especially with the proliferation of LLMs and human–AI interaction frameworks. Context-based paradigms have emerged as effective instruments for assessing creative thinking, drawing from established theories such as Sternberg's triarchic intelligence and models advocating future-oriented scenario stimuli. However, the bottleneck persists in obtaining high-quality, expert-designed creativity assessment contexts—primarily due to labor-intensive human craftsmanship. Generic LLM-generated narratives, previously explored for related tasks, inadequately meet psychometric requirements, lacking implicit assessment cues, structural coherence, stylistic diversity, and validity.

AlphaContext addresses these gaps with an architecture that combines hypertree-based outline planning, MCTS-driven generative search, quality-diversity refinement via MAP-Elites, and automated psychometric validation using simulated participant responses. The objective is to automate creation of assessment-ready contexts that elicit measurable and open-ended creative thinking in human and machine subjects, ensuring both diversity and quality at scale.

Figure 1: Workflow illustrating expert-driven context design and participant completion of open-ended tasks, foundational to creativity assessment.

Architecture and Methodology

AlphaContext is a modular pipeline comprising four interdependent stages:

1. HyperTree Outline Planner: Formalizes context structure as a hierarchical hypertree, enabling expert-level outline expansion through rule-guided hyperedges. Outline construction involves iterative selection and expansion of hyperchains, leveraging contextual grammar and LLM-driven pruning and node selection strategies for search efficiency and semantic alignment.
2. MCTS-based Context Generator: Models text generation as a sentence-level, outline-constrained tree search. Each section of the context corresponds to a separate search tree, with LLM policies proposing next sentences and dedicated evaluators scoring for cue alignment, vividness, and coherence. The MCTS loop—selection, expansion, evaluation (including lightweight look-ahead for low-confidence nodes), and backpropagation—identifies the highest-quality narrative path to initiate subsequent evolutionary refinement.

   Figure 2: AlphaContext pipeline: context query, hypertree planning, MCTS generative search, MAP-Elites diversity optimization, and assessment-guided refinement.

3. Evolutionary Context Optimizer: Implements MAP-Elites evolutionary search in a behavior space defined by proximity scope, knowledge density, and viewpoint diversity. The optimizer maintains a 3D grid archive, each cell representing a stylistic niche with an elite context. Mutation operations (insertion, deletion, replacement) controllably shift style, and elite updates ensure the archive continually improves in both diversity and quality across the behavior spectrum.
4. Assessment-Guided Evolution Refiner: Simulates participant responses with a role-conditioned, temperature-controlled LLM, generating response profiles (talkative, normal, quiet) and scoring for creativity. Only contexts surpassing an expert threshold for assessment effectiveness are retained; weaker candidates recycle for further mutation and optimization, ensuring constructs reliably elicit creativity.

Empirical Evaluation

Dataset and Experimental Setup

AlphaContext is evaluated on the CreaTE dataset—a compact, expert-curated archive of 203 title–theme pairs spanning diverse domains, specifically crafted for creativity assessment. Comparative baselines included general LLMs (DeepSeek-V3.1, Qwen3-235B-A22B, Llama3-70B-Instruct, GPT-5.1, Gemini-3.0-Pro-Preview), specialized long-form generators (LongWriter variants), and structured frameworks (DOC-v2, CRITICS, SS-GEN).

Evaluation metrics reflect seven psychometric dimensions: Coherence, Relevance, Engagement, Significance, Concreteness, Uncertainty, and Diverse Verbs (for action diversity). The assessment protocol utilizes pairwise comparisons with DeepSeek-V3.1 outputs, in line with arena-hard-auto methodology, and semantic similarity metrics (ROUGE-1, ROUGE-L, BERTScore) against expert contexts.

Results and Analysis

AlphaContext achieves dominant performance across all metrics, with average improvements of 8% over baselines. Notable increases occur in coherence (81.28%), engagement (79.93%), significance (71.06%), and uncertainty (80.3%)—all critical for eliciting open-ended, multi-perspective creative thinking. Ablation studies reveal marked performance drops when omitting modules, especially the HyperTree Planner (relevance falls to 70.20%) and the MCTS-based Generator (coherence drops to 74.38%), validating the necessity of structured planning and sentence-level tree search. MAP-Elites refinement is critical for uncertainty and overall diversity, confirming its dual role in expanding stylistic coverage and filtering for quality.

AlphaContext's outputs exhibit superior alignment with expert-designed contexts across all similarity metrics: ROUGE-1 (30.41%), ROUGE-L (25.48%), and BERTScore (81.88%).

Figure 3: Example input format from the CreaTE dataset, emphasizing psychometric grounding via title–theme pairing.

Preference evaluations show strong consistency between automated LLM judging and human assessments (Cohen's $\kappa$ > 0.8). AlphaContext contexts are preferred in >60% of cases when compared to GPT-5.1 and >73% compared to Gemini-3.0-Pro-Preview. Real-world studies with secondary students demonstrate unimodal, normal creativity score distributions and significant positive Pearson correlation ($r=0.3770$) with established benchmarks (AUT), supporting criterion validity for elicited creativity.

Figure 4: Human and LLM preference evaluation results for AlphaContext vs. GPT-5.1.

Figure 5: Human and LLM preference evaluation results for AlphaContext vs. Gemini-3.0-Pro-Preview.

AlphaContext also achieves the highest measurement-level alignment with expert rankings (Spearman $r_s=0.84$), outperforming the Gemini baseline ($r_s=0.58$) and generalized LLM outputs.

Figure 6: Measurement-level alignment—scatter plot comparing AlphaContext and expert-induced score ranks.

Computational cost analysis shows AlphaContext requires more tokens (12.89k per context) and longer inference time (∼6 minutes per context) than zero-shot baseline LLMs, but it substantially reduces expert labor (weeks for manual design) and achieves higher psychometric validity.

Figure 7: Cue diversity comparison—AlphaContext embeds broader, more diverse assessment cues than SS-GEN under identical thematic input.

Implications and Future Directions

AlphaContext demonstrates scalable automation of psychometric creativity context generation, maximizing both diversity and validity. Practical implications include standardizing creativity assessment datasets, supporting reliable benchmarking of human and machine creative cognition, and facilitating deployment in intelligent education systems. Theoretically, AlphaContext operationalizes hierarchical planning and quality-diversity optimization in natural language—methods likely extensible to other assessment-driven generative tasks and meta-cognitive LLM benchmarking.

Extensions could include broadening domain and population coverage, cross-linguistic generalization, fine-tuning lightweight generators with AlphaContext outputs for enhanced efficiency, and real-time integration into collaborative human–AI problem solving environments. Limitations in generation efficiency, dependency on expert-crafted inputs, and delineation of psychometric cue control suggest further research in adaptive outline expansion, sampled prompt diversity, and evaluation framework robustness.

Conclusion

AlphaContext introduces a modular architecture for automated generation of psychometric creativity assessment contexts, integrating hypertree-based outline planning, MCTS-driven generative search, MAP-Elites quality-diversity optimization, and simulated assessment-guided refinement. Empirical results substantiate its superiority over strong LLM baselines and structured generation frameworks on multiple quality metrics, semantic alignment, preference, and validity measures. While requiring greater computational resources, AlphaContext effectively reduces reliance on scarce expert labor and facilitates robust, standardized assessment of creative thinking.

Figure 8: Example of an assessment-ready context generated by AlphaContext, demonstrating multi-cue integration and narrative coherence.

AlphaContext provides a scalable foundation for creativity assessment, setting a new standard for psychometrically informed, automated context construction in AI-driven educational and cognitive evaluation applications.