BPAM: Bilateral Grid-based Pixel-Adaptive MLP

• The paper introduces BPAM, a framework that combines bilateral-grid processing with per-pixel MLPs to enable non-linear, spatially adaptive color transformations.
• It replaces traditional affine transforms by using a tiny 3→8→3 MLP, where bilateral grids store neural parameters based on multi-channel guidance maps.
• Benchmark evaluations show BPAM achieves superior PSNR, SSIM, and perceptual metrics, effectively balancing efficiency with enhanced image quality.

Bilateral Grid-based Pixel-Adaptive Multi-layer Perceptron (BPAM) is a framework for real-time image enhancement that combines bilateral-grid processing with per-pixel multi-layer perceptrons (MLPs) in order to overcome two limitations of prior methods: bilateral-grid pipelines are typically restricted to linear affine transforms, and MLP-based enhancement methods typically use globally shared parameters that are hard to adapt to localized image variations. In BPAM, a lightweight CNN backbone predicts bilateral grids from a downsampled image, the grids store parameters of a tiny MLP rather than affine coefficients, and each pixel slices the grids through guidance maps to retrieve its own transformation parameters for RGB-to-RGB color mapping (Lou et al., 16 Jul 2025).

1. Conceptual basis and problem setting

BPAM is designed for image enhancement settings in which both nonlinear color mappings and local spatial adaptivity are required. Prior bilateral-grid methods, exemplified in the paper by HDRNet-style processing, are fast and spatially aware but usually apply only linear affine transforms at each pixel. Prior MLP-based enhancement methods can represent nonlinear mappings, but because they generally use globally shared parameters, they do not adapt well to local image variation. BPAM combines these two lines of work by using the bilateral grid as a spatial- and intensity-aware parameter bank for MLPs rather than for affine filters (Lou et al., 16 Jul 2025).

The framework is organized around four operations. A lightweight CNN backbone generates bilateral grids from a downsampled image. These grids store MLP parameters rather than affine coefficients. A guidance map determines where each pixel slices the grids. The sliced parameters define a distinct per-pixel MLP that maps the input RGB value to the enhanced RGB output. The MLP is extremely small, with architecture $3 \rightarrow 8 \rightarrow 3$, so the overall design preserves efficiency while introducing nonlinear expressiveness (Lou et al., 16 Jul 2025).

A recurring misconception in bilateral-grid enhancement is that the grid itself implies an affine output model. BPAM directly contradicts that assumption: the grid remains the mechanism for spatial/intensity indexing, but the transformation indexed by the grid is nonlinear because the stored quantities are neural parameters rather than affine coefficients.

2. Bilateral-grid parameterization and slicing

For an input image $I \in \mathbb{R}^{W \times H \times 3}$, BPAM uses the standard bilateral-grid formulation

$$\mathcal{G}\in\mathbb{R}^{W/s_x \times H/s_y \times L/s_r},$$

where $s_x$ and $s_y$ are spatial downsampling factors, $s_r$ is the intensity downsampling factor, and $L$ is the number of intensity levels, typically $255$. A pixel at $(x,y)$ with guidance or intensity value $I(x,y)$ is lifted to

$I \in \mathbb{R}^{W \times H \times 3}$0

with integer and fractional decomposition

$I \in \mathbb{R}^{W \times H \times 3}$1

The per-pixel value is then obtained by trilinear interpolation over the eight neighboring grid cells: $I \in \mathbb{R}^{W \times H \times 3}$2 where

$I \in \mathbb{R}^{W \times H \times 3}$3

and the interpolation weights sum to $I \in \mathbb{R}^{W \times H \times 3}$4 (Lou et al., 16 Jul 2025).

BPAM uses two bilateral grids. The first grid stores parameters for the first MLP layer, and the second grid stores parameters for the second MLP layer. Each pixel slices both grids to obtain its own MLP parameters. In the supplementary description, the depth dimension is unrolled according to

$I \in \mathbb{R}^{W \times H \times 3}$5

which specifies how 2D feature maps from the backbone are converted into 3D bilateral grids (Lou et al., 16 Jul 2025).

This parameterization preserves the classical bilateral-grid mechanism—lifting by spatial coordinates and a guidance/intensity value, followed by trilinear slicing—while changing the semantics of the stored values. The resulting grid is not a bank of affine coefficients but a bank of neural network parameters conditioned on spatial position and intensity context.

3. Pixel-adaptive MLP formulation

BPAM uses a tiny three-layer MLP with structure

$I \in \mathbb{R}^{W \times H \times 3}$6

For each pixel, the first layer is

$I \in \mathbb{R}^{W \times H \times 3}$7

where $I \in \mathbb{R}^{W \times H \times 3}$8 is the input RGB value, $I \in \mathbb{R}^{W \times H \times 3}$9, $$\mathcal{G}\in\mathbb{R}^{W/s_x \times H/s_y \times L/s_r},$$0, $$\mathcal{G}\in\mathbb{R}^{W/s_x \times H/s_y \times L/s_r},$$1 is ReLU, and $$\mathcal{G}\in\mathbb{R}^{W/s_x \times H/s_y \times L/s_r},$$2. The second layer is

$$\mathcal{G}\in\mathbb{R}^{W/s_x \times H/s_y \times L/s_r},$$3

where $$\mathcal{G}\in\mathbb{R}^{W/s_x \times H/s_y \times L/s_r},$$4, $$\mathcal{G}\in\mathbb{R}^{W/s_x \times H/s_y \times L/s_r},$$5, and $$\mathcal{G}\in\mathbb{R}^{W/s_x \times H/s_y \times L/s_r},$$6 is the enhanced RGB output (Lou et al., 16 Jul 2025).

Because $$\mathcal{G}\in\mathbb{R}^{W/s_x \times H/s_y \times L/s_r},$$7, $$\mathcal{G}\in\mathbb{R}^{W/s_x \times H/s_y \times L/s_r},$$8, $$\mathcal{G}\in\mathbb{R}^{W/s_x \times H/s_y \times L/s_r},$$9, and $s_x$0 are retrieved from bilateral grids at pixel-specific coordinates, every pixel receives its own MLP. The model is therefore pixel-adaptive rather than globally shared. The parameter counts per grid cell exactly match the layer dimensions: the first grid cell stores $s_x$1 parameters, consisting of $s_x$2 weights and $s_x$3 biases, and the second grid cell stores $s_x$4 parameters, consisting of $s_x$5 weights and $s_x$6 biases (Lou et al., 16 Jul 2025).

The contrast with an affine bilateral-grid baseline is explicit in the paper. In HDRNet-style output,

$s_x$7

with $s_x$8 and $s_x$9. BPAM replaces this linear transform with a pixel-specific nonlinear mapping. The distinction is central: the grid determines where and under what intensity context parameters are retrieved, while the MLP determines how colors are transformed once those parameters are available.

4. Grid decomposition and multi-channel guidance

A major contribution of BPAM is its grid decomposition strategy, which is motivated by the observation that prior bilateral-grid methods typically fuse RGB into a single guidance channel and then use that single channel to slice all coefficients. BPAM argues that this underuses color information during slicing. To address this bottleneck, the method groups MLP parameters by type and stores them in separate subgrids rather than in a single undifferentiated grid (Lou et al., 16 Jul 2025).

For the first MLP layer, which contains $s_y$0 weights and $s_y$1 biases, the first grid is segmented into $s_y$2 subgrids. Each subgrid corresponds to one weight group for a specific color channel plus a shared bias. For the second MLP layer, the second grid is segmented into $s_y$3 subgrids, each storing the weight for a corresponding hidden channel plus a shared bias. Slicing these decomposed grids requires multi-channel guidance maps rather than a single grayscale map. The first grid uses a $s_y$4-channel guidance map, and the second grid uses a $s_y$5-channel guidance map; each guidance channel corresponds to one subgrid, and slicing is performed separately for each channel/subgrid pair (Lou et al., 16 Jul 2025).

The guidance generation mechanism is sequential. A CNN takes the original image and produces the first guidance map. That guidance map slices the first grid and yields parameters for the hidden layer. The hidden features are then fed to a second CNN, which generates the second guidance map. This second map slices the second grid to obtain the final-layer parameters. In the paper’s formulation, this design enables channel-specific spatial guidance and more precise per-pixel parameter generation (Lou et al., 16 Jul 2025).

This decomposition clarifies that BPAM is not merely a bilateral-grid MLP with a richer output head. Its parameter storage, guidance design, and slicing procedure are jointly structured so that color information participates more directly in parameter retrieval.

5. Objective function, backbone, and real-time realization

BPAM is trained with a weighted combination of three losses: $s_y$6 where $s_y$7 is MSE loss, $s_y$8 is SSIM loss, and $s_y$9 is perceptual loss computed using VGG19 features. This training objective combines reconstruction fidelity, structural similarity, and perceptual regularization (Lou et al., 16 Jul 2025).

The implementation uses a three-layer U-Net-style NAFNet backbone, and each encoder/decoder layer contains two NAF-blocks. The slicing operation and MLP coefficient application are implemented with CUDA extensions. Grid depth is fixed to $s_r$0. Grid spatial resolution depends on the input regime: $s_r$1 for PPR10K $s_r$2, $s_r$3 for full-resolution FiveK, and $s_r$4 for all other images (Lou et al., 16 Jul 2025).

The reported runtime satisfies real-time constraints. The paper states that 4K images are processed at $s_r$5 FPS. In the high-resolution timing table, $s_r$6 inference takes $s_r$7 ms at $s_r$8 FPS, and $s_r$9 inference takes $L$0 ms at $L$1 FPS. Relative to prior methods, BPAM is slower than purely LUT-based methods such as 3DLUT, which lack spatial adaptivity, and LutBGrid is reported as faster than BPAM, while BPAM yields better quality. HDRNet, CSRNet, and LCDPNet are described as generally slower or weaker in quality, and CoTF is described as competitive but slower on 4K in the reported table (Lou et al., 16 Jul 2025).

The paper frames the efficiency–expressiveness tradeoff directly: 3D LUT methods are faster but less adaptive, whereas BPAM gives up some speed relative to the fastest LUT methods in exchange for greater nonlinear expressiveness and spatial adaptivity while remaining real-time.

6. Datasets, quantitative results, and ablations

BPAM is evaluated on three public datasets. MIT-Adobe FiveK uses $L$2 training images and $L$3 test images and is used for tone mapping and photo retouching, with evaluation at both $L$4 and full resolution. PPR10K is a portrait retouching dataset with $L$5 training images and $L$6 test images and includes three retouching targets, A/B/C. LCDP addresses non-uniform illumination and exposure correction and uses $L$7 training images, $L$8 validation images, and $L$9 test images. The reported metrics are PSNR, SSIM, and $255$0 in CIE LAB for tone mapping and retouching, LPIPS for exposure correction, and for PPR10K also PSNR$255$1 and $255$2 (Lou et al., 16 Jul 2025).

On FiveK tone mapping, BPAM reports $255$3 PSNR, $255$4 SSIM, and $255$5, exceeding LutBGrid’s $255$6, $255$7, and $255$8. At full resolution, BPAM achieves $255$9 PSNR, described as the best among compared methods, including a $(x,y)$0 dB gain over the closest baseline in full-resolution tone mapping. On FiveK photo retouching, BPAM reports $(x,y)$1 PSNR, $(x,y)$2 SSIM, and $(x,y)$3. On PPR10K, BPAM generally gives the best PSNR, $(x,y)$4, and human-centered metrics across the A/B/C retouching targets. On LCDP exposure correction, BPAM achieves $(x,y)$5 PSNR, $(x,y)$6 SSIM, and $(x,y)$7 LPIPS, which is reported as the best result among the compared methods (Lou et al., 16 Jul 2025).

The ablation studies support the two principal design choices. Replacing an affine transform with an MLP improves performance from $(x,y)$8 PSNR / $(x,y)$9 SSIM to $I(x,y)$0 / $I(x,y)$1. Adding grid decomposition improves both variants: affine + decomposition reaches $I(x,y)$2 / $I(x,y)$3, while MLP + decomposition reaches $I(x,y)$4 / $I(x,y)$5. In the supplementary comparison of guidance schemes, the full multi-channel guidance map outperforms using input channels separately plus a fused bias map, with $I(x,y)$6 PSNR / $I(x,y)$7 SSIM versus $I(x,y)$8 / $I(x,y)$9. Best performance occurs at grid depth $I \in \mathbb{R}^{W \times H \times 3}$00. Among tested MLP architectures, $I \in \mathbb{R}^{W \times H \times 3}$01 provides the best tradeoff; $I \in \mathbb{R}^{W \times H \times 3}$02 is slower with slightly worse performance, and $I \in \mathbb{R}^{W \times H \times 3}$03 adds cost without meaningful gain (Lou et al., 16 Jul 2025).

Qualitative comparisons in the paper show better brightness preservation, better color fidelity, more effective recovery of details in dark regions, colors closer to ground truth in illumination correction, and fewer color shifts than prior methods. In the tone-mapping examples, the paper specifically notes that BPAM reveals road and lawn details in dark rear areas better than competing approaches. The paper attributes these gains to a division of labor in which the bilateral grid encodes where and under what intensity context a transformation should occur, the MLP encodes how to map colors nonlinearly, and grid decomposition improves the use of RGB information instead of collapsing guidance to a single channel (Lou et al., 16 Jul 2025).