Edge-Aligned Initialization of Kernels for Steered Mixture-of-Experts

Abstract: Steered Mixture-of-Experts (SMoE) has recently emerged as a powerful framework for spatial-domain image modeling, enabling high-fidelity image representation using a remarkably small number of parameters. Its ability to steer kernel-based experts toward structural image features has led to successful applications in image compression, denoising, super-resolution, and light field processing. However, practical adoption is hindered by the reliance on gradient-based optimization to estimate model parameters on a per-image basis - a process that is computationally intensive and difficult to scale. Initialization strategies for SMoE are an essential component that directly affects convergence and reconstruction quality. In this paper, we propose a novel, edge-based initialization scheme that achieves good reconstruction qualities while reducing the need for stochastic optimization significantly. Through a method that leverages Canny edge detection to extract a sparse set of image contours, kernel positions and orientations are deterministically inferred. A separate approach enables the direct estimation of initial expert coefficients. This initialization reduces both memory consumption and computational cost.

• The paper proposes a deterministic Edge-Aligned Initialization for SMoE models, using Canny edge detection to place kernels deterministically, reducing the total convergence time by 5% compared to classic standards
• The method uses importance-based clustering and closed-form expert estimation, leading to improvements in PSNR and SSIM against a uniform grid baseline but slightly lower performance compared to state-of-the-art segmentation-based initializers
• If implemented as a part of existing SMoE frameworks, kernel placement would be performed deterministically, running Canny edge detection, directional line-segment analysis will be done for placement, and then some closed-form calculations will be dedicated to the task making large gains over prior deterministic approaches and smaller ones over state-of-the-art initializers.

Overview

The paper proposes a deterministic, edge-based initialization scheme for Steered Mixture of Experts (SMoE) image regression models. SMoE represents an image as a weighted sum of spatially localized Gaussian kernels whose steering matrices encode orientation and scale, and it has previously outperformed JPEG, JPEG 2000, and HEVC-Intra in PSNR and SSIM at comparable bitrates (2602.02031). Its main practical drawback is that parameter estimation—kernel positions, scales, orientations, and expert amplitudes—has relied on stochastic gradient-based optimization performed per image, which is computationally expensive and memory-intensive. The authors address the initialization stage of this pipeline: rather than placing kernels randomly or on a uniform grid and letting gradient descent discover image structure, they extract structural priors directly from the image via Canny edge detection and use them to place kernels deterministically before any optimization begins.

The central claim is that a lightweight, gradient-free initialization can approach the reconstruction quality of state-of-the-art segmentation-based initializers while substantially reducing total convergence time. The empirical results support the time reduction clearly; the quality claim holds only relative to a grid baseline, with a small deficit against the strongest prior methods.

Method

The pipeline proceeds in four stages. First, Canny edge detection produces a binary edge mask $E_b$. A directional line-segmentation step scans this mask along four canonical directions ($0^\circ$, $\pm45^\circ$, $90^\circ$) and extracts maximal connected pixel runs of length at least two, each characterized by its geometric center $\underline{\mu}$ and orientation $\theta$. This yields a candidate set $\mathcal{P}$ of kernel locations compatible with the SMoE kernel structure.

Second, a reduction stage controls the kernel count. Each candidate segment receives an importance score combining its distance to the two nearest similarly oriented segments and to the two nearest dissimilarly oriented segments, weighted by a hyperparameter $\lambda$ (typically $0.1$). Low-scoring segments are clustered with DBSCAN; each cluster is replaced by its mean position and modal orientation. The clustering radius $\epsilon$ is grown iteratively until roughly $0^\circ$0 of the budget $0^\circ$1 is absorbed into clusters, while the top-scoring unclustered segments—the ones corresponding to small but salient structures such as pupils—are retained explicitly. This gives direct control over sparsity, which prior methods could only influence indirectly through Canny detector parameters.

Third, kernels are placed in pairs straddling each segment orthogonally to its orientation, separated by a fixed distance $0^\circ$2, with isotropic steering matrices initialized as $0^\circ$3. Fourth, expert amplitudes are initialized by sampling luminance at kernel centers and refined by a short fixed-step iterative error-minimization loop ($0^\circ$4) that requires no full gradient descent over all parameters.

After this setup, standard tiled stochastic optimization proceeds: the image is split into tiles, per-tile SMoE models are trained with L2 regularization on normalization constants and pruned when $0^\circ$5 falls below a threshold, tile models are merged by translating kernel centers, and a final unregularized fine-tuning converges. Because initialization is deterministic and structurally informed, the authors note that arbitrary tile sizes are supported without the rigid memory-layout constraints of prior approaches.

Experimental results

Evaluation uses grayscale Barbara, Flowers, and Parrots ($0^\circ$6) and Peppers ($0^\circ$7), comparing four initializations: the proposed edge-aware method, a uniform grid baseline, S-SMoE (Li et al., 2024), and AS-SMoE. Three findings emerge:

• Against the grid baseline, the proposed method yields substantial PSNR and SSIM improvements, with visibly better preservation of edges and texture. Notably, the method tends to retain more kernels than the baseline because well-placed kernels contribute more per component, which weakens the effect of regularization-driven pruning—a trade-off the authors acknowledge openly.
• Against S-SMoE and AS-SMoE, the proposed method achieves slightly lower PSNR and SSIM. The paper does not report exact numerical gaps in the text, so the magnitude of this deficit must be read from the plotted rate-distortion curves.
• In runtime, the proposed method converges significantly faster than all competitors on three of the four test images; the exception is Peppers, where S-SMoE converged more quickly. This is the paper's strongest quantitative result: reconstruction quality approaching the state of the art at a fraction of the computational cost.

The timing advantage follows directly from the design: deterministic placement eliminates the stochastic search overhead of segmentation-based methods like MDBSCAN-driven S-SMoE, and the closed-form expert initialization reduces the work left for gradient descent.

Limitations and open questions

The paper concedes several points. The quality gap relative to S-SMoE and AS-SMoE persists across the evaluated images, so the method trades a measurable amount of fidelity for speed; whether this gap closes with longer fine-tuning or larger kernel budgets is not established. The evaluation covers only four grayscale images, so generalization to color imagery, higher resolutions, or diverse content types remains untested. The importance-score weighting $0^\circ$8, the separation distance $0^\circ$9, and the 80/20 cluster-to-isolated-segment split are fixed heuristics whose sensitivity is not analyzed. Kernel steering matrices are initialized isotropically despite the framework's capacity for oriented, anisotropic kernels, meaning the orientation information extracted from line segments informs placement only, not shape. Finally, the reported convergence-time advantage fails on one of four images, indicating the speedup is content-dependent.

The authors identify nonlinear segment modeling for steering-matrix initialization as the immediate extension, which would address the isotropy limitation directly.

Conclusion

This paper contributes a deterministic, edge-aligned initialization for SMoE image regression that replaces stochastic kernel placement with Canny-derived line-segment analysis, importance-based clustering, orthogonal kernel-pair placement, and closed-form expert estimation. It delivers clear runtime reductions and large gains over uniform-grid initialization, while falling slightly short of the best segmentation-based initializers in reconstruction quality. The method's compatibility with arbitrary tiling and its reduced memory footprint make it a practical alternative where optimization cost dominates, though its evaluation scope and heuristic parameter choices leave room for further validation.