Can LLMs Produce Better Object-Oriented Designs than Human-Involved Development?

Abstract: Background: LLMs are increasingly used for code generation. However, their ability to generate multi-class projects that require object-oriented design (OOD) remains unclear, especially relative to projects developed with human involvement. Aims: The primary objective of this study is to compare OOD quality in projects from three authorship conditions: PreAI (human-involved projects produced before widespread LLM use), PostAI (human-involved projects produced after widespread LLM use), and PureAI (projects generated end-to-end by contemporary LLMs). Method: We conducted a comparative case study on a postgraduate Java assignment. Two offerings of the same assignment were selected as the PreAI and PostAI datasets. PureAI projects were generated using three contemporary LLMs. We analyzed OOD quality using project-level OOD metrics, code smell density, and domain modeling. Results: Relative to human-involved projects, PureAI projects show lower code smell density and generally appear simpler in terms of total size, complexity, and coupling. However, this is consistent with oversimplification, as it is associated with missing abstractions and weaker responsibility separation. PostAI is closer to PureAI than PreAI on many OOD measures and also shows tendencies toward oversimplification. Conclusions: Our findings indicate that appropriate human guidance on object-oriented decomposition and responsibility assignment remains important when LLMs are used for object-oriented design.

• The paper demonstrates that LLM-generated designs achieve lower overall complexity and coupling metrics compared to human-involved methods.
• The study shows PureAI outputs concentrate complexity in fewer classes and underrepresent key domain abstractions.
• Enhanced prompt specificity improves domain modeling and balance in complexity, yet human oversight remains crucial.

Comparative Assessment of LLM-Generated vs. Human-Involved Object-Oriented Designs

Introduction and Motivation

LLMs have become pervasive within the software engineering landscape, advancing well beyond single-function code generation and now presenting capabilities for full-scale project synthesis from natural language. Despite this proliferation, unresolved questions remain regarding the structural, object-oriented quality of such generated systems relative to those produced by human practitioners. Of particular interest is the balance between maintainability—as captured through object-oriented design (OOD) metrics, code smells, and domain model fidelity—and issues of oversimplification or abstraction loss.

The paper "Can LLMs Produce Better Object-Oriented Designs than Human-Involved Development?" (2605.19901) presents a rigorous, empirical case study contrasting OOD quality across three authorship regimes: PreAI (human-involved, pre-LLM ubiquity), PostAI (human-involved, post-LLM ubiquity), and PureAI (fully LLM-generated using contemporary flagship models with varying prompt specificity).

Experimental Protocol and Methodology

The study leverages a postgraduate-level Java assignment implementing a Kalah board game, drawing upon two historic cohorts for human-involved codebases (PreAI: 2021, PostAI: 2024) and generating PureAI projects via OpenAI (gpt-5.4) and Google Gemini (gemini-2.5-pro, gemini-3.1-pro-preview), using three prompt strategies: None, Broad, and Specific OOD guidance.

A multi-faceted evaluation suite is employed:

• OOD Metrics: Classical Chidamber-Kemerer and related metrics including WMC, CBO, LCOM, DIT, class count, and LOC, both aggregated and normalized.
• Code Smell Densities: Method-level and class-level smells, normalized by KLOC, detected via DesigniteJava and PMD.
• Domain Modeling: Manual assessment of explicit representation and runtime conformance of six domain entities (Board, Game, Player, Pit, House, Store).
• Statistical Analysis: Mann–Whitney U-test, Cliff’s delta for continuous metrics; chi-square and Fisher’s exact test, odds ratios for binary metrics; Benjamini-Hochberg correction for multiple hypothesis testing.

An iterative generate-and-repair protocol is adopted to ensure only functionally valid, test-passing projects enter the analysis, isolating design quality independent of outright functional correctness.

Figure 1: Overview of the experimental workflow from dataset curation through metric extraction and comparative statistical analysis.

Key Empirical Results

The comparative analysis yields several notable findings:

OOD Metrics and Code Smell Density

Figure 2: Cliff’s delta effect sizes for OOD metrics and code smell density, contrasting PreAI, PostAI, and PureAI.

• Structural Simplicity in PureAI: LLM-generated projects demonstrate consistently lower total complexity, size, and coupling metrics compared to human-involved baselines.
• Concentrated Burden: Despite lower totals, PureAI projects have higher per-class averages and max-percentage values, suggesting that complexity and responsibility are centralized in few classes.
• Reduced Cohesion: PureAI generally yields lower cohesion (higher LCOM), especially in worst-case classes.
• Shallower Inheritance: Decreased DIT values reflect flatter inheritance hierarchies.
• Code Smell Profiles: PureAI outputs are characterized by lower code smell density, both method- and class-level, though this is not true for all LLM models examined. Conversely, PostAI has higher code smell density than PreAI.

Domain Concept Representation and Conformance

Figure 3: Effect sizes (Cliff’s delta, odds ratio) for domain concept representation and runtime conformance across PreAI, PostAI, and PureAI.

• Loss of Abstraction in PureAI: Fewer domain concepts (notably Pit, House, Store) are explicitly modeled as classes in PureAI outputs compared to human-involved designs.
• Runtime Fidelity: When domain concepts are represented, PureAI tends to achieve more faithful object instantiation during execution.
• Impact on Human Development: PostAI projects evidence a shift towards the oversimplified structural tendencies seen in PureAI, reducing domain concept representation relative to PreAI, without corresponding improvements in code smell density.

Prompt Specificity Effects within LLM Generations

Figure 4: Cliff’s delta for OOD metrics and code smell density contrasting PureAI prompt variants.

Figure 5: Effect sizes for concept representation and runtime conformance between PureAI prompt variants.

• Increasing OOD Guidance: Greater OOD specificity in prompts induces more classes and improves concept representation, diffusing complexity and coupling more evenly.
• Diminishing Returns: Certain models (notably gpt-5.4) become insensitive to further prompt specificity beyond a moderate level; others (Gemini) respond incrementally.
• Code Smell Mitigation: More explicit object-oriented guidance reduces method-level code smell densities but has negligible effect on class-level smells.
• Runtime Conformance Stability: High runtime conformance is sustained across prompt variants, conditional on concept representation.

Theoretical and Practical Implications

Oversimplification versus Maintainability

These results indicate that while LLMs tend to avoid code smells and generate functionally correct yet structurally simple designs, such simplicity often manifests as oversimplification—missing abstractions and poor separation of concerns. The implication is a trade-off: maintainability metrics (as captured by code smell tools) may be superficially improved, while deeper structural qualities (e.g., fidelity to domain model, future extensibility) deteriorate.

LLM Influence on Human Development Practices

The convergence of PostAI human-involved codebases toward PureAI-like simplification implies that the presence and influence of LLM tools may indirectly encourage or reinforce suboptimal OOD patterns, absent targeted guidance or review protocols. Notably, PostAI does not inherit the low code smell density characteristic of PureAI, and instead accumulates additional method-level smells—potentially via manual LLM output adaptations—suggesting increased maintenance overhead downstream.

Guidance for AI-Integrated Software Engineering

Prompt engineering emerges as a partial but insufficient remedy: OOD-guidance specificity can improve class count and domain mapping, but cannot fully eliminate the abstraction gap relative to human-led decomposition. Systematic and explicit design intent, provided by experienced practitioners and enforced via review or architectural constraints, remains crucial when integrating LLMs into an object-oriented development workflow.

Limitations and Future Directions

The generalizability of observed phenomena is inherently sample-bound: a single Java assignment, moderate codebase size, and a population of postgraduate students. While these constraints ensure methodological precision, extension to larger, more complex, and multi-domain projects is essential for validating these trends at scale.

Further work is warranted to examine the interaction between explicit architectural annotations (e.g., class diagrams, design patterns) injected as input to LLMs and downstream OOD quality. Additionally, exploring field and method granularity in domain modeling, as well as compositional effects in hybrid human-AI workflows, presents fertile avenues for empirical inquiry.

Conclusion

This comprehensive comparative study establishes that while state-of-the-art LLMs can synthesize functionally correct and code-smell-averse object-oriented programs, the apparent gains in maintainability often mask losses in essential OOD qualities, notably abstraction and responsibility separation. Human-influenced design authority—particularly in high-level decomposition—remains indispensable within AI-augmented software development pipelines. Prompt specificity aids but does not replace principled engineering discipline, and as LLMs continue to influence software engineering practice, vigilant assessment and critical integration strategies are required to maintain both code hygiene and architectural robustness.