Context-Aware Kernel Evolution (CAKE)

• Context-Aware Kernel Evolution (CAKE) is a framework that dynamically adapts machine learning kernels based on contextual data, enhancing flexibility and performance.
• It integrates symbolic, neural, and LLM-based genetic methods to update kernel representations in real-time for Bayesian optimization, multi-task learning, and system scheduling.
• Empirical results demonstrate significant improvements in data efficiency, convergence speed, and resource utilization across applications like photonic chip design and adaptive system scheduling.

Context-Aware Kernel Evolution (CAKE) encompasses a set of methodologies for adaptive kernel design in machine learning, optimization, and systems applications. The shared principle is the dynamic, data-driven evolution of kernel representations—learned or constructed—guided by the context provided by observations, computational constraints, or application-specific dynamics. CAKE frameworks integrate symbolic, neural, genetic, or analytic operators for kernel modification, often leveraging context from recent data, environmental statistics, or task-specific requirements to optimize performance. This approach contrasts sharply with the traditional static or hand-designed kernel paradigm, enabling improved adaptation, data efficiency, and robustness across Bayesian optimization, multi-task learning, computer vision, and systems scheduling.

1. Theoretical Foundations and General Frameworks

Context-Aware Kernel Evolution treats kernel design and refinement as a search or learning problem conditioned on evolving context. Central to CAKE frameworks is the encoding of kernels—whether as symbolic expressions, parametrized neural operators, or combinatorial constructs—whose structure and parameters are updated as new information becomes available.

In Bayesian optimization, for instance, CAKE formalizes kernel design as a symbolic search within a grammar over a set of atomic base kernels (e.g., squared exponential (SE), linear (LIN), periodic (PER), rational quadratic (RQ), Matérn 3/2 and 5/2). Composite kernels are generated through recursive application of binary operators such as addition and multiplication, yielding a grammar closure:

• $\mathbb{K}_0 = \mathfrak{B}$,
• $$\mathbb{K}_i = \mathbb{K}_{i-1} \cup \{ T_j(k_1, k_2)\ |\ k_1, k_2 \in \mathbb{K}_{i-1},\ T_j \in \{\!+, \times\}\ \}$$, where each composite kernel is uniquely serialized (e.g., “(SE×PER)+(LIN×RQ)”) (Suwandi et al., 22 Sep 2025).

In multi-task scenarios, compositional kernels with sparsity-inducing priors, such as the horseshoe, admit an evolutionary mechanism by proposing local edits (birth, death, split, merge) on the kernel structure; the proposal acceptance is governed by a Metropolis–Hastings ratio comparing marginal likelihood and historical priors (Shin et al., 2021).

In deep context-aware kernel networks for visual tasks, the kernel function is iteratively updated based on both content and multi-order spatial context, formalized via recurrent or feed-forward architectures, and solved as the fixed point of a joint optimization over kernel similarity matrices and contextual adjacency operators (Jiu et al., 2019, Jiu et al., 27 Dec 2024).

2. LLM-Based and Genetic Kernel Evolution in Bayesian Optimization

Recent CAKE instantiations integrate LLMs as genetic operators within the kernel search loop. The LLM receives as input candidate GP kernel specifications, observed data, and few-shot kernel evolution examples, then proposes offspring kernels via:

• Crossover: Merging two parent kernels with $+$ or $\times$, leveraging the LLM’s implicit understanding of kernel properties and prior data-encoded context.
• Mutation: Modifying a single parent by substituting one base kernel with another from the atomic pool.

After proposing new kernels, each is re-fit to observed data, and their Bayesian information criterion (BIC) is computed. Population truncation retains the most promising candidates. The BIC-acquisition kernel ranking method (BAKER) weighs normalized BIC and expected improvement (EI) to select the next kernel for querying: $$w_k = \frac{\exp(-\mathrm{BIC}(k))}{\sum_{k'} \exp(-\mathrm{BIC}(k'))},\quad k^* = \arg\max_{k} w_k \cdot \alpha(x_{t,k}),$$ where $\alpha(x; D, k)$ is the EI acquisition function (Suwandi et al., 22 Sep 2025).

Empirical evaluation over hyperparameter optimization, controller tuning, and photonic chip design tasks shows accelerated convergence, improved data efficiency, and higher solution quality compared to fixed-kernel, adaptive-heuristic, deep-GP, GP ensemble, and compositional kernel search baselines. Ablation confirms that both LLM-driven evolution and BAKER are essential for maximal performance (Suwandi et al., 22 Sep 2025).

3. CAKE in Automated Systems and Program Scheduling

In computer systems and scheduling, CAKE principles inform dynamic optimization for kernel placement, clustering, and adaptive reconfiguration. In the K-PACT framework for wideband spectrum sensing on reconfigurable hardware architectures, CAKE is instantiated as follows:

• Kernels (compute primitives) are clustered by temporal independence, using trace-driven activity intervals extracted from sample workloads.
• Greedy clustering populates PE-local instruction memories with kernels that do not co-occur, minimizing runtime reconfiguration (“soft” or “no-switch”) and heavy “hard-switch” overhead.
• The planner optimizes a cost function incorporating switching latency, scheduling delay, and dataflow, subject to memory and workflow constraints.
• Clusters and placements are adjusted as environmental hypotheses shift, achieving adaptive context-sensitive evolution (Suluhan et al., 25 Jul 2025).

Quantitative results demonstrate over two orders of magnitude reduction in off-chip binary fetches, hundred-fold reductions in switching time, and more than 130$\times$ speedup in per-subband execution under realistic workloads. These results exemplify the impact of context-driven kernel evolution and resource adaptation in latency-critical applications (Suluhan et al., 25 Jul 2025).

4. Context-Aware Kernel Evolution in Structured Learning and Vision

Deep context-aware kernel networks implement CAKE via feed-forward architectures that jointly optimize Gram matrices and context-adjacency operators. For image classification and annotation, context is learned as adjacency or attention operators capturing spatial or semantic relationships between image patches or regions.

A typical objective is: $$\min_{K, \{\Theta_c\}} -\mathrm{Tr}(K S^\top) - \alpha \sum_c \mathrm{Tr}(K \Theta_c K \Theta_c^\top) + \frac{\beta}{2} \|K\|_F^2,$$ where $S$ is the content similarity kernel and $\Theta_c$ encode directional or classwise context (Jiu et al., 2019).

Fixed-point iteration results in multi-layer feature maps,

$$K^{(t+1)}=S+\gamma\sum_{c=1}^C\Theta_cK^{(t)}\Theta_c^\top,$$

mirrored by explicit mappings at each layer, and facilitating stationary, layerwise, or classwise context modeling. End-to-end training is achieved via alternating minimization between feature maps and SVM classifiers, with PSD constraints and backpropagation of context parameters.

Multi-order context-aware kernel networks further extend these principles by incorporating higher-order neighborhoods (up to third order) and attention-based aggregation, producing explicit discriminative embeddings suitable for multi-label classification. Empirical results on Corel5K and NUS-WIDE benchmarks show state-of-the-art performance, highlighting the significance of evolving multi-scale contextual features (Jiu et al., 27 Dec 2024).

5. Evolutionary and Retrieval-Augmented CAKE for Kernel Code Optimization

Context-Aware Kernel Evolution is also applied in program and systems kernel code generation. The Evolution of Kernels (EoK) framework combines LLM-driven mutation/crossover with historical mining of optimization ideas, guiding exploration by actionable thoughts extracted from kernel library commit histories. Retrieval-Augmented Generation (RAG) injects relevant architectural and code documentation proportional to the query context and historical effectiveness, maximizing domain fitness.

Fitness is directly linked to performance metrics (e.g., throughput or inverse latency), and selection employs softmax with a Boltzmann temperature for balance between exploration and exploitation. Empirical studies on RISC-V show that EoK achieves a median 1.27$\times$ speedup across 80 kernel design problems, outperforming human experts and previous LLM-based automated approaches by a significant margin (Chen et al., 14 Sep 2025).

6. Multi-Task and Online CAKE: Transfer and Adaptation

CAKE frameworks support multi-task learning and online adaptation by evolving kernel structures across tasks or users. In mobile health, compositional GP kernels with hierarchical or nonparametric priors enable transfer of kernel evolutions, cluster-level priors, or global kernel pools across users or tasks, facilitating shared structure learning and improved sample efficiency (Shin et al., 2021).

Online evolution proposes local edits to kernel structures with acceptance based on marginal likelihood and prior, supporting continual adaptation. Joint or hierarchical priors over kernel grammars—though detailed formulations may be incomplete—capture the evolution of functional representations in a temporally and contextually coherent fashion.

7. Significance, Empirical Impact, and Future Directions

Context-Aware Kernel Evolution addresses the fundamental limitations of fixed or heuristically-selected kernels in surrogate modeling, code generation, and task-adaptive learning. By coupling kernel construction to the immediate context—whether measured data, workload characteristics, or environmental state—CAKE frameworks accelerate convergence, raise sample efficiency, and deliver robust, interpretable models.

Empirical evidence confirms the value of context-driven kernel evolution across diverse domains, with LLM-augmented search and adaptation emerging as a unifying mechanism for scalable, expressive, and high-performing kernel discovery (Suwandi et al., 22 Sep 2025, Chen et al., 14 Sep 2025, Suluhan et al., 25 Jul 2025, Jiu et al., 27 Dec 2024, Jiu et al., 2019, Shin et al., 2021). Future work may extend these frameworks to broader classes of learning problems, richer context modalities, and new forms of neural-symbolic kernel synthesis.