Euler's Elastica Inpainting

• The paper introduces a variational approach that minimizes an energy functional combining total variation and curvature to restore grey-value images.
• It leverages a deep energy framework and deep image prior, reparameterizing the image via a CNN to overcome numerical and artifact challenges.
• Empirical evaluations show efficient inpainting with sharp, rotation-invariant edge completion, outperforming traditional fourth-order PDE solvers in speed and robustness.

Euler’s elastica-based inpainting is a variational approach for image restoration that reconstructs missing or corrupted regions of a grey-value image by minimizing an energy functional dependent on both total variation and level-line curvature. This model is particularly suited for shape completion tasks, producing results with sharp edges and high rotation invariance. While the classical Euler’s elastica inpainting involves solving a singular, anisotropic, and nonlinear fourth-order PDE—posing significant numerical challenges—recent advances propose leveraging neural algorithms, specifically incorporating deep energy and deep image prior frameworks. This CNN-based method achieves efficient, unsupervised inpainting with simple discretization, robust performance, and GPU parallelizability (Schrader et al., 2022).

1. Variational Formulation of Euler’s Elastica Inpainting

Let $u : \Omega \to \mathbb{R}$ be the grey-value image, known on a set $K \subset \Omega$ and to be reconstructed on $\Omega \setminus K$. The elastica energy functional is given by

$$E[u] = \int_{\Omega \setminus K} (\alpha + \beta \kappa^2) |\nabla u|\, dx\,dy$$

where $|\nabla u| = \sqrt{u_x^2 + u_y^2}$, and the level-line curvature is defined as $\kappa = \operatorname{div}(\nabla u/|\nabla u|)$. The weights $\alpha,\beta \ge 0$ control the relative influence of total variation and curvature. Specific parameter settings recover special cases:

• $\alpha=1,\,\beta=0$: Total variation (TV) inpainting
• $\alpha=0,\,\beta=1$: Pure curvature minimization

In practice, $\alpha = b$, $K \subset \Omega$0, with $K \subset \Omega$1 chosen via cross-validation on mean absolute error (MAE).

2. Gradient Flow and Computational Challenges

Minimizing $K \subset \Omega$2 subject to $K \subset \Omega$3 leads to a highly nonlinear PDE stemming from its Euler–Lagrange equation, with principal terms involving up to fourth-order derivatives. To regularize the functional near singularities, a small parameter $K \subset \Omega$4 is introduced, yielding $K \subset \Omega$5. The corresponding gradient flow reads:

$K \subset \Omega$6

where $K \subset \Omega$7 projects onto the level-line direction. Boundary conditions are Dirichlet on $K \subset \Omega$8 ($K \subset \Omega$9), with optional Neumann conditions on $\Omega \setminus K$0. The system’s singular, anisotropic, fourth-order nature necessitates careful numerical treatment.

3. Discretization via Finite Differences

The function $\Omega \setminus K$1 is placed on a uniform grid (spacing $\Omega \setminus K$2), using 3×3 binomial-weighted central difference stencils for derivatives, exhibiting strong rotation invariance. For each pixel $\Omega \setminus K$3:

• $\Omega \setminus K$4
• $\Omega \setminus K$5

The discrete energy is

$\Omega \setminus K$6

where $\Omega \setminus K$7 is the binary mask indicating known regions.

4. Deep Energy and Deep Image Prior Methodology

Instead of explicit time-stepping of the PDE, $\Omega \setminus K$8 serves as the neural network loss function within a “deep energy” framework. Using autodiff, gradients are computed efficiently, and optimization proceeds via

$\Omega \setminus K$9

Direct minimization of $$E[u] = \int_{\Omega \setminus K} (\alpha + \beta \kappa^2) |\nabla u|\, dx\,dy$$0 can introduce high-frequency checkerboard artifacts due to the stencils’ structure. To suppress such spurious oscillations, a “deep image prior” approach is implemented: the image is reparameterized as $$E[u] = \int_{\Omega \setminus K} (\alpha + \beta \kappa^2) |\nabla u|\, dx\,dy$$1, where $$E[u] = \int_{\Omega \setminus K} (\alpha + \beta \kappa^2) |\nabla u|\, dx\,dy$$2 is a neural network mapping the mask $$E[u] = \int_{\Omega \setminus K} (\alpha + \beta \kappa^2) |\nabla u|\, dx\,dy$$3, known values $$E[u] = \int_{\Omega \setminus K} (\alpha + \beta \kappa^2) |\nabla u|\, dx\,dy$$4, and (optionally) noise to the restored image. The minimized functional becomes $$E[u] = \int_{\Omega \setminus K} (\alpha + \beta \kappa^2) |\nabla u|\, dx\,dy$$5, biasing reconstructions toward natural images and mitigating grid artifacts.

5. Network Architecture and Optimization Protocol

The architecture is a gated U-net with skip connections, typically featuring three to four scales and two convolutional layers per scale. The channel dimensionality starts at approximately 28 and doubles per down-sampling stage. On the coarsest scale, dilated convolutions support large-gap completion, and a gating mechanism (cf. Yu et al., 2019) enables handling of free-form masks. Activation functions are ReLU. Inputs consist of:

• The binary mask ($$E[u] = \int_{\Omega \setminus K} (\alpha + \beta \kappa^2) |\nabla u|\, dx\,dy$$6)
• Known image values ($$E[u] = \int_{\Omega \setminus K} (\alpha + \beta \kappa^2) |\nabla u|\, dx\,dy$$7)
• Random noise in the unknown region

Optimization employs Adam with typical learning rates $$E[u] = \int_{\Omega \setminus K} (\alpha + \beta \kappa^2) |\nabla u|\, dx\,dy$$8 (natural images) or $$E[u] = \int_{\Omega \setminus K} (\alpha + \beta \kappa^2) |\nabla u|\, dx\,dy$$9 (shape tasks), subject to staged decreases. Masked regions initialize with uniform $|\nabla u| = \sqrt{u_x^2 + u_y^2}$0 noise. Regularization $|\nabla u| = \sqrt{u_x^2 + u_y^2}$1 is set in $|\nabla u| = \sqrt{u_x^2 + u_y^2}$2; parameter $|\nabla u| = \sqrt{u_x^2 + u_y^2}$3 for elastica balance is selected via cross-validation on MAE. Training proceeds for 6–7k iterations for $|\nabla u| = \sqrt{u_x^2 + u_y^2}$4 images (natural scenes) or up to 60k iterations (shape completion). Early stopping is triggered at the MAE minimum on the known mask to suppress checkerboarding artifacts.

6. Empirical Evaluation and Comparative Analysis

Natural-Image Inpainting

• Data: “trui,” “peppers” ($|\nabla u| = \sqrt{u_x^2 + u_y^2}$5), random $|\nabla u| = \sqrt{u_x^2 + u_y^2}$6 known-pixel masks.
• Metrics: Mean absolute error (MAE) w.r.t. ground truth.
• Observations: After 6,000 iterations, MAE $|\nabla u| = \sqrt{u_x^2 + u_y^2}$7. Prolonged optimization further decreases $|\nabla u| = \sqrt{u_x^2 + u_y^2}$8 but increases MAE and can produce checkerboards. Energy and MAE drop rapidly in early iterations.

Shape-Completion

• Geometries: Straight-edge stars, circular stars, Kanizsa triangles (128–400 px), large missing regions ($|\nabla u| = \sqrt{u_x^2 + u_y^2}$9 px).
• Rotation consistency: High rotation invariance ensured by stencil design.
• Comparative performance:
  • Augmented Lagrangian (Tai et al., 2011): 78 s (CPU, $\kappa = \operatorname{div}(\nabla u/|\nabla u|)$0, 60% mask)
  • ADMM (Yashtini & Kang, 2016): 854 s (CPU, $\kappa = \operatorname{div}(\nabla u/|\nabla u|)$1, 20% mask)
  • TSC lifting (Chambolle & Pock, 2019): similar visual quality.
  • Present approach: 49 s (GPU, GTX1080, $\kappa = \operatorname{div}(\nabla u/|\nabla u|)$2, 6,000 iters).
• All scenarios yield sharp edge completion, effective gap closure, and clear rotation invariance.

7. Computational Efficiency and Methodological Advantages

Traditional elastica solvers require discretizing and iteratively solving a fourth-order PDE, often resulting in complex, sequentially limited algorithms. The CNN-based deep energy method discretizes only up to second-order derivatives. It leverages backpropagation and GPU computation for parallelizable, highly optimized optimization steps. The approach is unsupervised, requiring neither ground-truth images nor pre-trained weights—every inpainting instance is solved directly. This ensures simplicity, high population transferability, and efficient deployment, while maintaining competitive or superior quality compared to classical and state-of-the-art elastica solvers (Schrader et al., 2022).