Automated Feature Transplantation

• Automated Feature Transplantation is a set of methodologies that extract, adapt, and integrate features—ranging from code fragments to neural modules—into host systems while ensuring functionality through dependency analysis and rigorous validation.
• It spans diverse domains including software engineering, deep learning, and tabular data transformation, drastically reducing manual engineering and boosting integration efficiency.
• Advancements such as LLM-based code synthesis, modular neural transplanting, and gradient-ascent optimized feature transformation enable significant speedups, high pass-rates, and near-zero integration errors.

Automated Feature Transplantation refers to a class of methodologies and software frameworks that enable the extraction, adaptation, and seamless integration (“transplantation”) of features, either as software behaviors, code structures, model components, or learned representations, into existing digital artifacts—most notably codebases, neural networks, or data transformation pipelines—via automated or highly automated workflows. The objective is to minimize manual engineering effort, improve integration quality, and support scalable or reusable feature engineering. Automated feature transplantation encompasses a spectrum of techniques in software systems engineering, deep learning, and feature transformation for tabular data.

1. Core Principles and Problem Formalization

Automated feature transplantation centers on systematically extracting and integrating "features"—defined as self-contained software behaviors, code fragments (“organs”), transformation sequences, or neural submodules—into a host artifact, ensuring functional integrity, dependency resolution, and minimal interference with existing components. This process is formalized as a mapping $\mathcal{T}$:

$$\mathrm{Transplant}:\bigl(\mathcal{O},\,H,\,(E_d,\,I_h)\bigr)\;\longrightarrow\;H'$$

where $\mathcal{O}$ is the extracted feature (organ), $H$ the host, $(E_d, I_h)$ the entry and insertion points, and $H'$ the host with the transplanted feature, passing all relevant validation criteria (Souza et al., 2023).

In the context of software projects, the transplantation pipeline parses a natural-language feature request, analyzes the project’s dependency structure, generates or extracts feature code, applies modifications, and validates the resulting system for structural consistency and correctness (Vsevolodovna, 2024). In deep learning, transplantation extends to neural modules or feature vectors; for example, category encoders, adapters, or attention projections are inserted into generic or target networks with the goal of expanding function, transferring knowledge, or transferring alignment (Zhang et al., 2019, Dong et al., 2024, Kowsar et al., 8 Nov 2025).

2. Automated Transplantation Methodologies

2.1 Software Code Feature Integration

Feature-Factory formalizes end-to-end automation of software feature integration using generative AI. Its stages:

• Feature Request Parsing: Natural-language feature prompt $F$ triggers intent extraction and subtask decomposition by an LLM.
• Project Parsing/Dependency Resolution: The project $P$ is parsed into a dependency graph $G=(V,E)$; each node $v$ is a file/module, with edge $(v\to u)$ denoting dependency.
• Vector-Database Indexing: Each file $p_i$ is embedded via LLM-provided semantic vectors and indexed, enabling rapid retrieval for code synthesis.
• Feature Mapping: A mapping $M(F,G) = \{(v,\omega)\}$ links tasks $\omega$ to affected project components $v$.
• Code Generation/Integration: For each $(v,\omega)$, a task-specific prompt triggers LLM-generated code $C_i$, applied by transformation function $\mathcal{T}$.
• Validation: The system checks that the new project $P'$’s dependency graph remains valid; failures trigger automatic repair/rollback (Vsevolodovna, 2024).

Experimental results show integration times of $\sim$45 seconds (small projects) and 3 minutes (medium), $>90\%$ reduction in developer effort, and near-zero code consistency errors.

2.2 Code Organ and Multi-File Feature Transplantation

Foundry automates software product line engineering through software transplantation. Features (“organs”) are extracted via inter- and intra-file program slicing, adapted via genetic improvement to meet host and integration constraints, then merged into the host codebase with duplication avoidance by clone detection. Each transplanted organ is required to pass regression, regression$^{++}$, and acceptance test suites, with strict pass-rate thresholds (Souza et al., 2023). Empirical results show a 4.8$\times$ speedup over manual feature migration by SPL experts.

Empirical code organ transplantation further demonstrates the feasibility of extracting “organs” from git commit history (“add” commits) and integrating them into host systems. Key findings:

• $\sim$60% of “add” commits represent practical features
• $\sim$70% of these organs are “easy-to-transplant” (self-contained)
• Unit test pass rates: 80–97% across Java, Python, and C
• Partial automation exists for mining, with the remainder (dependency analysis, insertion, test oracle generation) requiring further research (Wang et al., 2018)

2.3 Neural Network and Representation Transplantation

Network Transplanting employs modular architectures with category modules, task modules, and adapters. The transplantation of a new category into an existing generic network involves:

• Extracting the relevant category encoder $f_t$
• Training a small adapter $h$ to map its output into the input space of the target task module $g_s$
• Holding $f_t$ and $g_s$ fixed, training $h$ with back-distillation to match both outputs and local input-output Jacobians

This procedure achieves negligible catastrophic forgetting, and empirical results demonstrate that the transplanted module can match or exceed fine-tuning approaches, even with zero new labels (Zhang et al., 2019).

Concept transplantation for alignment (ConTrans) generalizes feature transplantation to internal model representations. It refines “concept directions” (e.g., emotion, honesty, toxicity) as principal directions in a source model’s residual space, reformulates them to target models via affine alignment, and injects them additively into the residual streams of the target. This mechanism, which introduces no additional trainable parameters or gradient-based fine-tuning, consistently transfers alignment properties across model sizes and families, surpassing even instruction-tuned models on truthfulness in some settings (Dong et al., 2024).

Neuron transplantation for model fusion combines ensembles of models by concatenating all non-output neurons, pruning the lowest-importance neurons to recover the original parameter count, and fine-tuning the resulting network. This procedure dominates or matches competing fusion strategies (e.g., OT-fusion) in both computational efficiency and recovered accuracy, consistently outperforming individual ensemble members after minimal fine-tuning (Öz et al., 7 Feb 2025).

3. Automated Feature Transformation in Data Pipelines

Automated feature transplantation extends to ML feature construction and transformation:

3.1. Transformer-Based Automated Feature Transformation

GPT-FT models the search for optimal feature transformation sequences as an embedding optimization problem. Its key steps:

• Record Collection: RL-based collector produces pairs $(\gamma_i, v_i)$, mapping postfix transformation sequences to downstream performance.
• Embedding Construction: A parameter-efficient decoder-only GPT represents each $\gamma_i$ via an embedding $E_i$; heads estimate both sequence reconstruction and performance.
• Gradient-Ascent Search: Embeddings are optimized by ascending the estimated performance gradient, producing new candidate feature transformations.
• Autoregressive Sequence Reconstruction: Decoded embeddings yield new transformation sequences applied to the data.

GPT-FT achieves superior or equivalent performance to baselines on 15 public tabular tasks, with significant reductions in model size and inference time (Gao et al., 28 Aug 2025).

3.2. Evolutionary LLM-Driven Feature Transformation

ELLM-FT integrates RL and evolutionary algorithms with LLM-based few-shot generation:

• Multi-population RL Database: RL agents operate on feature–operator sequences, building diverse populations.
• Evolutionary Maintenance: Culling strategies refine both individuals and populations.
• LLM Prompting: Few-shot “sequential prompting” with performance-ranked examples guides the LLM to propose strong new transformation sequences.
• Evaluation and Curation: Each candidate is validated and curated based on downstream utility.

ELLM-FT demonstrates higher efficiency and performance than non-LLM evolutionary baselines, with task-agnostic robustness across both classification and regression datasets (Gong et al., 2024).

3.3. Cross-domain Tabular Feature Transplantation

LATTLE implements attention-weight transplantation from an LLM trained on source tabular data into a target task transformer (gFTT):

• Stage 1: Fine-tune an LLM (DistilGPT2) on source tabular data, taking serialized table rows as input.
• Stage 2: Extract and freeze the key/value projection matrices from the final LLM attention layer, transplanting these into the lowest gFTT layer of a new tabular model for the target domain.
• Stage 3: Fine-tune the resulting gFTT on target-domain data.

LATTLE performs robustly on cross-domain tabular classification, outperforming strong baselines (XGBoost, TabNet, FT-Transformer), transfer frameworks (TransTab, CM2), and even larger LLMs (Tabula-8B), without requiring feature overlap or prompt engineering (Kowsar et al., 8 Nov 2025).

4. Formal Models, Validation, and Benchmarking

Transplantation frameworks employ rigorous formal models to structure the search, integration, and validation steps:

• Graph-based Dependency Analysis: Formalizes source code dependencies (files, modules, imports) for identifying safe insertion points, ensuring structural integrity post-integration (Vsevolodovna, 2024).
• Slice-based Feature Boundaries: Program slicing and dependence graphs enable over-approximate extraction and conservative transplantation of software features (Souza et al., 2023).
• Jacobian Distillation Loss: In neural transplantation, the back-distillation objective aligns both output and local Jacobian to guarantee faithful functional integration (Zhang et al., 2019).
• Joint Losses for Sequence/Performance: Transformer-based feature builders use joint reconstruction and downstream estimation objectives to efficiently optimize the transformation search space (Gao et al., 28 Aug 2025).

Empirical evaluations are performed on public benchmarks, internal codebases, and, for LATTLE, on source-target table pairs disjoint in content and feature identity. Metrics include pass-rates ($R_x$), unit test coverage, classification metrics (F1, AUC), efficiency statistics, and integration times.

5. Computational and Practical Considerations

Automated feature transplantation methods are evaluated not only on correctness, but also on computational efficiency, generality, and extensibility:

• Efficiency: Feature-Factory and Foundry show order-of-magnitude speed-ups over manual workflows; neuron and attention transplantation reduce memory and inference loads versus model ensembling (Vsevolodovna, 2024, Öz et al., 7 Feb 2025, Souza et al., 2023, Kowsar et al., 8 Nov 2025).
• Generality: LLM-driven methods (GPT-FT, ELLM-FT, LATTLE) are agnostic to downstream model and can handle a variety of task domains and data modalities, given appropriate representation and prompting schemes (Gao et al., 28 Aug 2025, Gong et al., 2024).
• Extensibility: ConTrans and Network Transplanting demonstrate successful transplantation across model sizes and families, with minimal additional resource needs (Dong et al., 2024, Zhang et al., 2019).
• Limitations: Feature granularity, dependency complexity, test oracle generation, and model alignment constraints represent prevailing challenges. For code transplantation, multi-file, pointer-heavy, and semantically entangled features increase analysis complexity; for neural and semantic transplantation, bottlenecks appear in cases of highly heterogeneous model architectures or when attempting to combine multiple concept vectors simultaneously (Souza et al., 2023, Wang et al., 2018, Dong et al., 2024).

6. Emerging Directions and Open Problems

Current frameworks approach full automation, but open challenges remain:

• Source and Host Compatibility: Improved static/dynamic analysis (e.g., cross-language similarity, semantic clone detection) is required for identifying robust feature boundaries and transplant locations (Wang et al., 2018, Souza et al., 2023).
• Data-Free and Few-Shot Settings: Better support for situations where no (or minimal) target data is available for validation or adaptation, as in “zero-shot” network transplantation or concept migration (Zhang et al., 2019, Dong et al., 2024).
• Richer Transformation Grammars: Extending LLM-based frameworks beyond postfix mathematical expressions and incorporating domain-specific transformations (Gao et al., 28 Aug 2025, Gong et al., 2024).
• Integration with Feature Models: For SPL, richer feature-model synchronization is needed, including automatic updates as new features are transplanted (Souza et al., 2023).
• Automated Validation Oracles: Addressing the oracle problem for arbitrary software features, ML pipelines, or novel neural behaviors remains a key research area (Wang et al., 2018).

7. Summary Table: Principal Automated Feature Transplantation Frameworks

Framework	Domain	Feature Granularity	Core Methods
Feature-Factory (Vsevolodovna, 2024)	Software (code)	Project/module/file/function	LLM-guided code synthesis, dependency graph, vector DB, validation
Foundry (Souza et al., 2023)	SPL Re-engineering	Multi-file feature/“organ”	Program slicing, genetic improvement, clone-aware merging
GPT-FT (Gao et al., 28 Aug 2025)	ML feature engineering	Transformation sequence	RL record collection, GPT embedding, gradient ascent, autoregressive recon.
ELLM-FT (Gong et al., 2024)	ML tabular features	Sequence of transformations	Multi-population evolutionary search, LLM few-shot prompt generation
Network Transplanting (Zhang et al., 2019)	Deep models	Neural module/adapter	Modular assembly, adapter + back-distillation, fixed base modules
ConTrans (Dong et al., 2024)	LLM alignment	Internal concept vector	Concept mean-diff, affine mapping, residual injection
LATTLE (Kowsar et al., 8 Nov 2025)	Tabular transfer	Attention key/value matrices	Attention projection transfer, LLM fine-tuning on source, gFTT target model
Neuron Transplantation (Öz et al., 7 Feb 2025)	Model fusion	Subnetwork/neuron	Concatenation, magnitude pruning, fine-tuning

Each method operationalizes feature transplantation through a sequence of automated or semi-automated machine- and representation-level analysis, with rigorous pipeline structures, formal optimization objectives, and empirical benchmarking, reflecting the maturation and diversification of the field across software engineering and machine learning.