---
title: Euler's Elastica Inpainting
url: https://www.emergentmind.com/topics/euler-s-elastica-based-inpainting
type: topic
---

# Euler's Elastica Inpainting

Euler’s elastica-based inpainting is a variational approach for image restoration that reconstructs missing or corrupted regions of a grey-value image by minimizing an energy functional dependent on both total variation and level-line curvature. This model is particularly suited for shape completion tasks, producing results with sharp edges and high rotation invariance. While the classical Euler’s elastica inpainting involves solving a singular, anisotropic, and nonlinear fourth-order PDE—posing significant numerical challenges—recent advances propose leveraging neural algorithms, specifically incorporating deep energy and deep image prior frameworks. This CNN-based method achieves efficient, unsupervised inpainting with simple discretization, robust performance, and GPU parallelizability [2207.07921].

## 1. Variational Formulation of Euler’s Elastica Inpainting

Let $u : \Omega \to \mathbb{R}$ be the grey-value image, known on a set $K \subset \Omega$ and to be reconstructed on $\Omega \setminus K$. The elastica energy functional is given by
$$
E[u] = \int_{\Omega \setminus K} (\alpha + \beta \kappa^2) |\nabla u|\, dx\,dy
$$
where $|\nabla u| = \sqrt{u_x^2 + u_y^2}$, and the level-line curvature is defined as $\kappa = \operatorname{div}(\nabla u/|\nabla u|)$. The weights $\alpha,\beta \ge 0$ control the relative influence of total variation and curvature. Specific parameter settings recover special cases:
- $\alpha=1,\,\beta=0$: Total variation (TV) inpainting
- $\alpha=0,\,\beta=1$: Pure curvature minimization

In practice, $\alpha = b$, $\beta=1-b$, with $b$ chosen via cross-validation on mean absolute error (MAE).

## 2. Gradient Flow and Computational Challenges

Minimizing $E[u]$ subject to $u|_K = f$ leads to a highly nonlinear PDE stemming from its Euler–Lagrange equation, with principal terms involving up to fourth-order derivatives. To regularize the functional near singularities, a small parameter $\epsilon > 0$ is introduced, yielding $g(|\nabla u|) = \sqrt{u_x^2 + u_y^2 + \epsilon^2}$. The corresponding gradient flow reads:
$$
\partial_t u = -\frac{\delta E}{\delta u} 
= \operatorname{div}\left( (\alpha + \beta \kappa^2) \frac{\nabla u}{g} \right) 
- \beta\, \operatorname{div}( g P \nabla \kappa )
$$
where $P = I - (\nabla u \otimes \nabla u) / g^2$ projects onto the level-line direction. Boundary conditions are Dirichlet on $\partial K$ ($u=f$), with optional Neumann conditions on $\partial\Omega$. The system’s singular, anisotropic, fourth-order nature necessitates careful numerical treatment.

## 3. Discretization via Finite Differences

The function $u$ is placed on a uniform grid (spacing $h$), using 3×3 binomial-weighted central difference stencils for derivatives, exhibiting strong rotation invariance. For each pixel $i$:
- $g_i = \sqrt{(u_x)_i^2 + (u_y)_i^2 + \epsilon^2}$
- $\kappa_i = \frac{(u_y)_i^2 (u_{xx})_i - 2 (u_x)_i (u_y)_i (u_{xy})_i + (u_x)_i^2 (u_{yy})_i}{g_i^3}$

The discrete energy is
$$
E(u) = \sum_{i:\,c_i=0} g_i\, ( \alpha + \beta\,\kappa_i^2 )
$$
where $c$ is the binary mask indicating known regions.

## 4. Deep Energy and Deep Image Prior Methodology

Instead of explicit time-stepping of the PDE, $E(u)$ serves as the neural network loss function within a “deep energy” framework. Using autodiff, gradients are computed efficiently, and optimization proceeds via
$$
u^{k+1} = u^k - \tau\, \nabla_u E(u^k)
$$
Direct minimization of $E(u)$ can introduce high-frequency checkerboard artifacts due to the stencils’ structure. To suppress such spurious oscillations, a “deep image prior” approach is implemented: the image is reparameterized as $u = \mathcal{N}(c, f; \theta)$, where $\mathcal{N}$ is a neural network mapping the mask $c$, known values $f$, and (optionally) noise to the restored image. The minimized functional becomes $L(\theta) = E(\mathcal{N}(c, f; \theta))$, biasing reconstructions toward natural images and mitigating grid artifacts.

## 5. Network Architecture and Optimization Protocol

The architecture is a gated U-net with skip connections, typically featuring three to four scales and two convolutional layers per scale. The channel dimensionality starts at approximately 28 and doubles per down-sampling stage. On the coarsest scale, dilated convolutions support large-gap completion, and a gating mechanism (cf. Yu et al., 2019) enables handling of free-form masks. Activation functions are ReLU. Inputs consist of:
- The binary mask ($c$)
- Known image values ($f$)
- Random noise in the unknown region

Optimization employs Adam with typical learning rates $\tau = 10^{-3}$ (natural images) or $\tau = 5 \times 10^{-5}$ (shape tasks), subject to staged decreases. Masked regions initialize with uniform $[0,1]$ noise. Regularization $\epsilon$ is set in $[10^{-4}, 10^{-3}]$; parameter $b$ for elastica balance is selected via cross-validation on MAE. Training proceeds for 6–7k iterations for $256 \times 256$ images (natural scenes) or up to 60k iterations (shape completion). Early stopping is triggered at the MAE minimum on the known mask to suppress checkerboarding artifacts.

## 6. Empirical Evaluation and Comparative Analysis

### Natural-Image Inpainting

- Data: “trui,” “peppers” ($256\times256$), random $10\%$ known-pixel masks.
- Metrics: Mean absolute error (MAE) w.r.t. ground truth.
- Observations: After 6,000 iterations, MAE $\approx 1.6 \cdot 10^{-2}$. Prolonged optimization further decreases $E$ but increases MAE and can produce checkerboards. Energy and MAE drop rapidly in early iterations.

### Shape-Completion

- Geometries: Straight-edge stars, circular stars, Kanizsa triangles (128–400 px), large missing regions ($\gtrsim 150$ px).
- Rotation consistency: High rotation invariance ensured by stencil design.
- Comparative performance:
  - Augmented Lagrangian (Tai et al., 2011): 78 s (CPU, $300 \times 235$, 60% mask)
  - ADMM (Yashtini & Kang, 2016): 854 s (CPU, $220 \times 340$, 20% mask)
  - TSC lifting (Chambolle & Pock, 2019): similar visual quality.
  - Present approach: 49 s (GPU, GTX1080, $256 \times 256$, 6,000 iters).
- All scenarios yield sharp edge completion, effective gap closure, and clear rotation invariance.

## 7. Computational Efficiency and Methodological Advantages

Traditional elastica solvers require discretizing and iteratively solving a fourth-order PDE, often resulting in complex, sequentially limited algorithms. The CNN-based deep energy method discretizes only up to second-order derivatives. It leverages backpropagation and GPU computation for parallelizable, highly optimized optimization steps. The approach is unsupervised, requiring neither ground-truth images nor pre-trained weights—every inpainting instance is solved directly. This ensures simplicity, high population transferability, and efficient deployment, while maintaining competitive or superior quality compared to classical and state-of-the-art elastica solvers [2207.07921].

Source: https://www.emergentmind.com/topics/euler-s-elastica-based-inpainting