- The paper introduces a hierarchical taxonomy constructed from 7,310 manually validated rules across 83 AI IDE projects.
- It demonstrates that additive rule evolution improves artifact compliance by an average of 22.99%, highlighting practical efficacy.
- The findings reveal a disconnect between developer priorities and rule prevalence, urging refined rule curation and integration with static analysis tools.
Rule Taxonomy and Evolution in AI IDEs: An Empirical Analysis
Introduction
This study presents a large-scale, mixed-methods empirical investigation of rules in AI-powered integrated development environments (AI IDEs). The research addresses an outstanding gap regarding the taxonomy, evolution, and practical impact of AI IDE rules—textual artifacts enabling developers to impose project-specific behavioral constraints and architectural guidance on LLM-based coding agents. The work integrates extensive mining of open-source projects with a practitioner survey, aiming to systematize the classification of rules, analyze their longitudinal evolution, and quantify their influence on artifact compliance in real-world development.
Rule Taxonomy Construction and Analysis
A dataset of 83 OSS projects developed with AI IDEs (Cursor, Windsurf, Trae, Kiro, Qoder) was curated, yielding 7,310 segmented and manually validated rules. Through inductive open coding, a hierarchical taxonomy was established: 5 primary and 25 secondary categories spanning Architecture & Design, Code Implementation, Development Workflow & Project Management, Quality Assurance, and AI Collaboration Specifications. This taxonomy encapsulates a wide spectrum of constraints, from explicit technical stack selections to abstract LLM behavior directives.
Contrasting the prevalence of rules in repository mining with developer-perceived importance (N=99 survey), the analysis highlights a systematic disconnect. Developers prioritize high-level architectural and LLM-contextual constraints (mean importance values for "System Architecture" and "AI Context Management" >4.1/5), yet the majority of rules in real-world projects are concentrated in formatting, code style, and workflow categories. This divergence reflects an operational focus on correcting immediate friction over the specification of durable, system-level principles, corroborated by the predominance of incremental and AI-assisted rule creation strategies.
Rule Evolution: Patterns, Drivers, and Impact
Longitudinal analysis of 1,540 rule evolution events demonstrates that rules in AI IDEs are frequently updated, particularly in the context of business logic and system architecture (change rates >30%). Most rule evolution consists of additive operations—appending new rules rather than modifying or removing existing ones. Deleted operations are disproportionately found in performance and security categories, indicating the practical difficulty and adverse effects of enforcing non-functional requirements purely via LLM prompting.
A cross analysis of mined commit data and survey reports exposes a stark perception vs. reality gap. While commit histories evidence that rule evolution is primarily driven by context expansion and enrichment (∼55%), developers overwhelmingly report "correction of AI errors" (77.8%) and prompt refinement as their main triggers for evolution. Corrective actions are usually operationalized as negative constraints through additive patches, seldom via holistic prompt refactoring.
The compliance of project artifacts with rules was rigorously evaluated via a structured LLM-as-a-judge protocol. Artifact compliance rates exhibit a statistically significant mean increase of 22.99% post rule update, rising from 49.14% to 72.13%. The effect size is large (Wilcoxon, r=0.71). However, this benefit is uneven: rules governing dependency management and concrete testing practices yield large compliance gains (up to +64.8%), while abstract documentation or system design rules exhibit minimal or negligible impact.
Implications for Practice and Theory
Practical Implications:
- Natural language rules in AI IDEs are optimal for enforcing explicit, low-level technical constraints but are substantially less effective for high-level architectural or NFR guidance. The latter remain better served by architectural knowledge extraction and formal models.
- Incrementally accretive (additive) rule management rapidly leads to context bloat, rule staleness, and contradictions. Regular curation, refactoring, and automated conflict detection are essential for sustainable rule hygiene.
- Static analysis tools and traditional project linters should remain the mechanism of choice for code style and NFRs, with AI IDEs focusing rule context on aspects that benefit most from LLM interpretive capabilities.
Theoretical and Tooling Implications:
- The identified taxonomy provides a foundation for further empirical studies and motivates research into automated architectural rule extraction from code artifacts, commit histories, and dependency graphs.
- Observed gaps between perceived importance and empirical prevalence, and between corrective vs. constructive evolution, suggest new research questions on cognitive and organizational dimensions of agentic programming. Models of developer-AI interaction need to account for temporal context drift and prompt staleness.
- LLM-based compliance evaluation demonstrates feasibility for scalable rule impact assessment but reveals inherent limits in current LLM instruction adherence, especially for rules with broad or ambiguous scope.
- Advanced AI IDEs must go beyond static prompt injection, toward intent decomposition and multi-layered context management.
Future Directions
Future work should address automated conflict detection and rule refactoring assistance, scalable extraction of architectural rules from codebases, and evaluation in large-scale, multi-team, or legacy enterprise systems for broader generalizability. Integration of dynamic artifact analysis, interaction trace mining, and continuous feedback into rule evolution pipelines will be critical to next-generation agentic IDEs.
Conclusion
This study offers the first comprehensive taxonomy and evolutionary analysis of AI IDE rules, empirically quantifies their effect on codebase compliance, and reveals both the power and limits of prompt-based agent alignment in modern software engineering. Its findings motivate the need for improved rule curation mechanisms, better-integrated static analysis, and research into automatic extraction and decomposition of developer intent.
Reference: "Rule Taxonomy and Evolution in AI IDEs: A Mining and Survey Study" (2606.12231)
