---
title: Edge-Efficient Recipe & Cooking State Generator
url: https://www.emergentmind.com/topics/edge-efficient-recipe-and-cooking-state-guided-generator
type: topic
---

# Edge-Efficient Recipe & Cooking State Generator

The Edge-Efficient Recipe and Cooking State Guided Generator is a generative model architecture designed for real-time synthesis of realistic cooked food images from raw inputs on edge devices. The approach facilitates user-preferred target states—such as specific doneness or texture—rather than static presets, leveraging recipe and cooking-state conditioning to remain computationally feasible for embedded deployment. It incorporates a novel culinary similarity metric as both a perceptual loss and a runtime progress signal, achieving significant improvements in fidelity and perceptual similarity over prior methods, with tightly integrated hardware-aware design and a domain-specific dataset annotated by culinary experts [2511.16965].

## 1. Model Architecture

The generative framework consists of two principal networks: a U-Net–based generator and an EfficientNet-B1–based Siamese similarity network. The generator, denoted \( G_e \), comprises 8.7M parameters and inputs a raw food image \( I_{\text{raw}} \in \mathbb{R}^{3 \times 224 \times 224} \), a categorical recipe label \( c \) (one of 30 classes), and a discrete cooking state index \( d_s \in \{ \text{cs1}, \text{cs2}, \text{cs3} \} \). It outputs a synthesized cooked-state image:

\[
\hat I_{d_s} = G_e(I_{\text{raw}}, c, d_s) \in \mathbb{R}^{3 \times 224 \times 224}.
\]

Conditioning on recipe and cooking state is implemented via FiLM (Feature-wise Linear Modulation) injected into every encoder/decoder block. The context encoding pipeline assigns each (recipe, state) pair a unique index \( p_i \), computes a 32-dimensional sinusoidal positional embedding \(\mathrm{SPE}(p_i)\), projects to a context vector \( E_p \) via MLP\(_\phi\), and maps to per-layer FiLM parameters \([\gamma_l, \beta_l]\) using an MLP with parameters \(\phi_l\):

\[
[\gamma_l, \beta_l] = \mathrm{MLP}_{\phi_l}(E_p), \quad (\gamma_l, \beta_l \in \mathbb{R}^{F_l}).
\]
\[
z^{(l)}_{\mathrm{mod}} = \gamma_l \odot z^{(l)} + \beta_l.
\]

The U-Net backbone has base channel width 32 and scaling factors (1, 2, 4, 8) across four down/up-sampling stages; each stage houses two ResNet blocks with eight-group normalization. The final output is mapped to RGB via a 1×1 convolution.

For adversarial training, a 70×70 PatchGAN discriminator processes concatenated raw/cooked images using a depth-doubling cascade up to 512 channels.

## 2. Culinary Image Similarity Metric

Temporal consistency and culinary plausibility are achieved via the Culinary Image Similarity (CIS) metric. An EfficientNet-B1–based Siamese network \( f_{\mathrm{sim}}: \mathbb{R}^{3\times224\times224} \to \mathbb{R}^{128} \) outputs L2-normalized embeddings. The cosine similarity

\[
F_{\mathrm{cul}}(I_i, I_j) = \cos(f_{\mathrm{sim}}(I_i), f_{\mathrm{sim}}(I_j)) \in [0,1]
\]

provides a scalar indication of progression along the cooking trajectory.

Training uses session-timestamped frame pairs \( (I_i, I_j) \) labeled

\[
s_{i,j} = 1 - \frac{| t_i - t_j |}{T},
\]

where \( t_i \) and \( t_j \) are frame times and \( T \) is the total session duration. Supervision employs mean squared error:

\[
\mathcal{L}_{\mathrm{sim}} = \mathbb{E}_{i,j} \left[ ( F_{\mathrm{cul}}(I_i, I_j) - s_{i,j} )^2 \right].
\]

This embedding forms the basis for generator training and runtime monitoring, enabling inference-time comparison between live and target images to detect doneness peaks.

## 3. Generator Training Objectives

The generator loss is a weighted sum of adversarial, perceptual, and culinary similarity terms. Given ground-truth cooked image \( I_{d_s} \) and synthesized image \( \hat I_{d_s} \):

- **Adversarial (PatchGAN) Loss**:
\[
\mathcal{L}_{\mathrm{GAN}} = \mathbb{E}_{I_{\mathrm{raw}}, I_{d_s}}[\log D(I_{\mathrm{raw}}, I_{d_s})] + \mathbb{E}_{I_{\mathrm{raw}}}[\log(1 - D(I_{\mathrm{raw}}, \hat I_{d_s}))]
\]

- **LPIPS Perceptual Loss**:
\[
\mathcal{L}_{\mathrm{LPIPS}} = \mathbb{E}\left[ \|\,\phi(I_{d_s}) - \phi(\hat I_{d_s}) \| \right]
\]

using a fixed VGG-based LPIPS extractor.

- **Culinary Image Similarity Loss**:
\[
\mathcal{L}_{\mathrm{CIS}} = \mathbb{E}[1 - F_{\mathrm{cul}}(I_{d_s}, \hat I_{d_s})]
\]

The full objective:

\[
\mathcal{L}_{\mathrm{gen}} = \lambda_1 \mathcal{L}_{\mathrm{GAN}} + \lambda_2 \mathcal{L}_{\mathrm{LPIPS}} + \lambda_3 \mathcal{L}_{\mathrm{CIS}},
\]
with \(\lambda_1=1\), \(\lambda_2=50\), \(\lambda_3=50\).

Temporal consistency arises from discrete state conditioning and the CIS term, which aligns the generator outputs to a continuous culinary progression in embedding space rather than using explicit 3D or recurrent modeling.

## 4. Dataset and Data Processing

The approach leverages a novel dataset consisting of 1,708 oven-based sessions spanning 30 recipes, with each session producing a raw starting image and frames every 30 seconds, culminating in at least three chef-annotated edible states:

- cs1 = "basic cook" (edible)
- cs2 = "standard" (ideal)
- cs3 = "extended" (extra texture/browning)

Images are uniformly sized at 224×224, captured from a top-mounted camera with the oven closed. Session partitioning is 70/10/20 for train/val/test splits. Training employs on-the-fly augmentations including random horizontal flipping and ±60° rotation.

## 5. Training Protocols and Hyperparameters

**Similarity Network \( f_{\mathrm{sim}} \):**
- Optimizer: Adam (lr=1e-4, weight decay=1e-5)
- Step decay: \( \gamma=0.6 \) every 10 epochs
- Epochs: 100; batch size 32 (all session pairs per batch)

**Generator and Discriminator:**
- Optimizer: Adam (lr=2e-4, \(\beta_1=0.5\))
- Batch size: 1
- Epochs: 100 (first 50 epochs constant lr, then linear decay)

The entire high-level training and inference pipeline is detailed in the following pseudocode:

```
# 1) Train similarity network f_sim
for epoch in 1..100:
  for each session S:
    let frames = [I_0, I_1, ..., I_T]
    for all pairs (i,j) in S:
      s_ij = 1 - |t_i - t_j|/T
      e_i = f_sim(I_i);  e_j = f_sim(I_j)
      sim = cosine(e_i, e_j)
      L_sim += (sim - s_ij)^2
  update f_sim via Adam

# 2) Train generator G_e (w/ fixed f_sim)
for epoch in 1..100:
  for each training example:
    I_raw, c, d_s, I_ds = load_batch()
    # Context embedding
    p_i = index(c,d_s)
    E_p = MLP_pos(SPE(p_i))
    # Forward G_e with FiLM at every layer using E_p
    I_hat = G_e(I_raw, E_p)
    # Discriminator loss
    L_GAN = AdvLoss(D, I_raw, I_ds, I_hat)
    # LPIPS loss
    L_LPIPS = LPIPS(I_hat, I_ds)
    # CIS loss
    L_CIS   = 1 - cosine(f_sim(I_hat), f_sim(I_ds))
    L_gen = L_GAN + 50*L_LPIPS + 50*L_CIS
    update G_e via Adam
    update D via Adam

# 3) Inference + cooking-progress monitoring
Given I_raw, user picks desired state d_s*
I_targets = G_e(I_raw, c, cs1..cs3)
display I_targets→user picks I_goal
start oven; every 30s capture I_t:
  p = cosine(f_sim(I_t), f_sim(I_goal))
  update sliding window of p; if local peak→STOP
```

## 6. Edge Deployment and Efficiency

Deployment is targeted toward edge hardware. Only the generator (\(8.68\)M parameters) and similarity network (\(5.4\)M trainable parameters) are retained at inference, exported via PyTorch→ONNX→NPU runtime. Hybrid quantization (float16 convolutional weights, int8 activations) yields an on-disk footprint of approximately 45 MB. On a 5 TOPS embedded NPU, generation requires ~1.2 s/image (3.6 s for all target states) and the CIS comparison ~0.3 s/pair.

## 7. Empirical Evaluation

Performance is benchmarked against established baselines on the in-domain cooking dataset (715 test set pairs) as well as public edge2shoes and edge2handbags datasets. The following table summarizes key findings:

| Method            | #Params | FID↓   | LPIPS↓  |
|-------------------|---------|--------|---------|
| Pix2Pix [9]       | 163 M   | 153.00 | 0.4711  |
| Pix2Pix-Turbo [19]| 1290 M  | 75.42  | 0.2523  |
| Ours              | 8.68 M  | 52.18  | 0.2145  |
| - no recipe guide | 8.68 M  | 58.74  | 0.2398  |
| - no CIS loss     | 8.68 M  | 54.98  | 0.2310  |

On public datasets, the FID improvement is approximately 60% over Pix2Pix. The ablation entries indicate that excluding recipe guidance or excluding CIS loss both degrade performance. This suggests strong contributions from both explicit conditioning and the culinary similarity metric.

---

The Edge-Efficient Recipe and Cooking State Guided Generator thus combines domain-specific conditioning, an efficient architecture, and a culinary progression metric to achieve practical, high-fidelity generative food synthesis and progress monitoring under edge hardware constraints [2511.16965].

Source: https://www.emergentmind.com/topics/edge-efficient-recipe-and-cooking-state-guided-generator