---
title: 'FHE-Agent: Automated FHE Configuration'
url: https://www.emergentmind.com/topics/fhe-agent
type: topic
---

# FHE-Agent: Automated FHE Configuration

Fully Homomorphic Encryption (FHE) provides a cryptographic mechanism for computation on encrypted data without decryption, enabling secure outsourced inference in privacy-preserving machine learning as a service (MLaaS). However, configuring practical FHE inference using state-of-the-art schemes (notably CKKS) presents steep barriers due to the high-dimensional, coupled parameter space and the need for expert-level cryptographic reasoning. "FHE-Agent" denotes a class of LLM-driven agentic frameworks that automate this expert configuration process. By coupling deterministic FHE toolchains with a hierarchy of LLM agents, FHE-Agent systems systematically search and optimize FHE configuration, eliminating the guess-and-check loop and enabling practical deployment for deep neural networks, even where heuristic- and prompt-based approaches fail [2511.18653].

## 1. Motivation and Problem Landscape

Deployment of CKKS-based FHE, central for approximate arithmetic over encrypted vectors, is stymied by complex configuration requirements. Each setup must jointly determine ring dimension ($N$), modulus chain ($Q = [q_0,\ldots,q_L]$), packing layout, scale schedule, and bootstrapping placement to balance depth support, security (typically ≥128 bits), throughput, and runtime. Existing compilers (e.g., Orion, CHET, FHeliPe) provide sophisticated static signals, but selecting feasible, efficient parameterizations has remained a manual process highly sensitive to network depth, layer type, and packing strategy. One-shot LLM or heuristic searches routinely over-provision resources (resulting in excessive latency) or fail to yield feasible configurations, especially for nontrivial models such as AlexNet. FHE-Agent frameworks address this by decomposing configuration into globally and locally guided repair workflows, pruning infeasible regimes early, and reserving encrypted computation for credible candidates [2511.18653].

## 2. Overall Architecture and Agents

FHE-Agent consists of a deterministic, backend-agnostic FHE tool suite and a multi-agent LLM controller.

- **Tool Suite Components:**
    - *StaticAnalyzer:* Evaluates multiplicative depth versus modulus chain length, ensuring compliance with security level constraints ($\text{SecBits}(\mathcal{C})$ derived from $(\log N, \log Q)$).
    - *LayerProfiler:* Conducts slot-accurate, cleartext simulations (CLEAR_ONLY) to collect, for each layer $i$, shape ($s_i$), performance primitives ($p_i$), and numeric diagnostics ($n_i$, including approximation error and noise margins).
    - *BootstrapScheduler:* Annotates where bootstrapping is required and marks "depth-critical" layers incapable of tolerating added multiplicative depth without blowing noise budgets.
    - *CostModel:* Models layerwise runtime as $cost(\mathcal{C}) = \sum_i (\alpha\,\mathrm{mul}_i + \beta\,\mathrm{rot}_i + \gamma\,\mathrm{boot}_i + \delta\,\mathrm{mem\_cost}_i)$, with coefficients initialized and calibrated by microbenchmarks and encrypted test runs.
    - *EncryptedEvaluator:* Supports $run\_trial(\mathcal{C}, mode)$ across four evaluation fidelities: STATIC_ONLY, CLEAR_ONLY, FHE_LIGHT, and FHE_FULL.

- **LLM Controller (Agent Hierarchy):**
    - *InitAgent/RegimeAgent:* Given plaintext computation graph and global constraints, proposes a family of global regimes $\mathcal{C}_k$ via variations in ($\log N$, $\log Q$), packing, and scale schedule.
    - *GlobalTradeoffAgent:* Suggests global patches (chain tailoring, bootstrap insertion, scale schedule adjustments) to a selected baseline.
    - *LayerwiseAgent:* Identifies bottleneck layers (using a score function: $w_1 r_i + w_2 (1-u_i) + w_3 \mathrm{normalized}(\mathrm{rot}_i) + w_4 z_i$; $u_i$: slot utilization, $z_i$: low noise flag) and proposes local packing or degree adjustments.
    - *PatchGateAgent:* Screens candidate patches under STATIC_ONLY and CLEAR_ONLY, admitting only those passing security and precision thresholds for further encrypted evaluation.

## 3. Hierarchical Search and Configuration Decomposition

FHE-Agent models configuration as a two-level object:
- **Global Regime:** $(\log N,\ \log Q,\ \sigma,\ \text{scale schedule},\ \text{boot plan})$
- **Per-Layer Local Parameters:** $(\text{packing}_i, \text{embedding method}, \text{BSGS gap}, \text{activation degree})$

Ring dimension is governed by the required total multiplicative depth $D$, with $\log N \geq \lceil \log_2(2D+1) \rceil$. Modulus chain length is set to accommodate the largest scale plus noise headroom, with $\sum_\ell \log q_\ell \geq \text{bits}(\text{scale}_\text{max}) + \text{headroom}_\text{noise}$. Packing layouts mediate the trade-off between parallelism and the cost of rotations and key switching.

The search proceeds from global exploration (limit $O(10)$ candidate regimes via STATIC_ONLY/CLEAR_ONLY pruning) to localized, layerwise patching (up to $O(5)$ per iteration, respecting bootstrapping and depth masks).

## 4. Multi-Fidelity Optimization Workflow

FHE-Agent introduces a structured three-phase search:

- **Phase A (Structure Search):**
    - Evaluate $O(10)$ candidate regimes using STATIC_ONLY and CLEAR_ONLY; prune those failing depth, security, or MAE/precision criteria.
    - Rank survivals via proxy latency (CostModel); retain 1–2 as baselines.

- **Phase B (Calibration):**
    - Perform FHE_LIGHT runs on validation subsets. Use results to regress $(\alpha, \beta, \gamma, \delta)$ for CostModel calibration.
    - Select the best-performing regime ($\mathcal{C}_\text{base}$).

- **Phase C (Admitted Refinement):**
    - Iteratively generate and evaluate agent-suggested patches (GlobalTradeoffAgent → LayerwiseAgent), subject to gating by STATIC_ONLY and CLEAR_ONLY.
    - Permit only the top-ranked patch to proceed to FHE_LIGHT per iteration, maintaining an encrypted evaluation budget (typically $<10$ light runs).
    - Commit to FHE_FULL only for final verification.

This multi-fidelity pruning ensures that almost all infeasible or suboptimal designs are eliminated at minimal cost, reserving expensive encrypted inference for credible candidates.

## 5. Integration with Orion and CKKS Toolchains

FHE-Agent operates as an orchestration framework above the Orion compiler and Lattigo CKKS libraries. Its tool suite invokes Orion’s internal passes and Lattigo’s backend primitives for both static-depth/security validation and cleartext slot-accurate simulations, as well as full encrypted trials. The LLM controller generates only high-level, semantically validated configuration "patches," which are rendered as modifications against Orion’s JSON configuration files. The toolchain's shared semantic model (Orion IR and Lattigo's CKKS) eliminates simulation/execution discrepancies.

## 6. Empirical Results and Comparative Performance

Under a strict 128-bit security threshold, FHE-Agent demonstrates substantial improvements over both one-shot LLM and heuristic-based approaches. The results below highlight key metrics across standard benchmarks:

| Model   | Precision (Naive LLM / Agent) | MAE (Naive / Agent) | FHE time/sec (Naive / Agent) | SecBits |
|---------|-------------------------------|---------------------|------------------------------|---------|
| MLP     | 17.37 / 24.82                 | 5.9e-6 / ≈0         | 1.31 / 0.91                  | ≥128    |
| LeNet   | 23.07 / 22.42                 | 1.13e-7 / 1.79e-7   | 9.08 / 3.19                  | ≥128    |
| LoLA    | 19.54 / 21.16                 | 1.31e-6 / ≈0        | 2.10 / 0.79                  | ≥128    |
| AlexNet | no feasible / 21.81           | —       / ≈0        | —     / 262.5                | ≥128    |

FHE-Agent not only improves precision (by 5–7 bits for MLPs and CNNs) and reduces inference latency by factors of 2–4×, but crucially, automatically discovers a feasible 128-bit secure configuration for AlexNet—a model where baseline one-shot and heuristic methods fail entirely [2511.18653].

## 7. Design Principles Enabling Efficient High-Dimensional Search

The success of FHE-Agent is attributable to the following factors:

- **Hierarchical Decomposition:** Global regime selection and local repair prevent combinatorial explosion, isolating search to $O(10)$ global and $O(5)$ local candidates per iteration.
- **Multi-Fidelity Pruning:** Early elimination via STATIC_ONLY and CLEAR_ONLY ensures near-zero cost discarding of infeasible configurations.
- **Discrete, Validated patch directions:** Only semantically approved parameter changes (e.g., chain shortening, packing modification, activation degree reduction) are permitted, minimizing the risk of invalid states.
- **Enforced Constraints:** All modifications are gated by static analysis and bootstrapping masks, ensuring depth and security invariants are never violated.

In aggregate, these design features enable FHE-Agent to navigate the nonlinear, tightly coupled CKKS configuration space automatically, reliably emulating (and frequently surpassing) human expert configurations while efficiently using the expensive encrypted evaluation budget [2511.18653].

Source: https://www.emergentmind.com/topics/fhe-agent