Papers
Topics
Authors
Recent
Search
2000 character limit reached

Decomposing multispectral face images into diffuse and specular shading and biophysical parameters

Published 18 Feb 2019 in cs.CV | (1902.06557v1)

Abstract: We propose a novel biophysical and dichromatic reflectance model that efficiently characterises spectral skin reflectance. We show how to fit the model to multispectral face images enabling high quality estimation of diffuse and specular shading as well as biophysical parameter maps (melanin and haemoglobin). Our method works from a single image without requiring complex controlled lighting setups yet provides quantitatively accurate reconstructions and qualitatively convincing decomposition and editing.

Citations (6)

Summary

  • The paper presents a novel biophysical dichromatic model that decomposes a single multispectral face image into diffuse and specular shading along with melanin and haemoglobin maps.
  • It employs a nonlinear least squares approach with reparameterization and trust-region-reflective optimization to perform per-pixel fitting under natural lighting conditions.
  • Experimental results indicate a mean relative absolute error of 7.4% and demonstrate superior accuracy compared to ICA-based methods, enabling effective image editing.

Decomposing Multispectral Face Images into Diffuse and Specular Shading and Biophysical Parameters

Introduction

This paper introduces a biophysical and dichromatic reflectance model designed to efficiently characterize spectral skin reflectance from multispectral face images. The aim is to decompose a single image into diffuse and specular shading and biophysical parameter maps, including concentrations of melanin and haemoglobin. Unlike existing methods that require controlled setups, this approach achieves high-quality estimation from a single image under natural lighting conditions. This innovative technique is tested both quantitatively and qualitatively, demonstrating its efficacy in reconstructing true skin appearance and enabling advanced image editing capabilities.

Biophysical Dichromatic Skin Model

The proposed model assumes the skin reflects light according to the dichromatic model, which separates scene radiance into diffuse and specular components. It further incorporates the biophysical properties of skin to describe spectral reflectance, focusing on melanin and haemoglobin distribution. The multispectral dichromatic model is modeled as:

L(λ)=E(λ)(idR(λ)+is)L(\lambda) = E(\lambda)(i_dR(\lambda) + i_s)

where L(λ)L(\lambda) represents wavelength-dependent radiance. Melanin and haemoglobin parameters are spatially variable, allowing realistic mapping of skin color variations. Figure 1

Figure 1

Figure 1: (a) Layered skin reflectance model. (b) sRGB render (see Sec.~4.3) of the skin colouration model.

Model Fitting

Fitting the model to multispectral data involves solving a nonlinear least squares problem. With known spectral properties of the illumination, and biophysical constraints, the technique optimizes spatial reflectance parameters. The problem is structured to avoid constraints by using reparameterization, effectively enabling per-pixel optimization without the need for spatial smoothness priors. The methodology leverages the trust-region-reflective algorithm to achieve precise fitting across different skin types. Figure 2

Figure 2: Col.~1: sRGB render of multispectral images. Col.~2-5: decomposition into: specular and diffuse shading, melanin and haemoglobin. Col.~6: albedo from biophysical maps. Col.~7: sRGB render of reconstructed radiance.

Experiments

A series of experiments validate the effectiveness of the model and fitting methodology. The quantitative evaluation shows a mean relative absolute error of 7.4% when applied to the ISET database. The approach qualitatively reproduces plausible decompositions and reconstructions. Comparisons with methods employing Independent Component Analysis reveal that the proposed technique yields more accurate and physically meaningful results. Figure 3

Figure 3: Qualitative comparison to \cite{Tsumura:2003:ISC}, from top: our result; bottom: \cite{Tsumura:2003:ISC}.

Model-Based Editing

With the decomposition maps established, intuitive editing becomes possible. By altering melanin or haemoglobin concentrations in these maps, various facial appearances can be simulated, such as tanning or flushing. Specular reflection edits enhance the ability to manipulate skin texture visually. This facility is enabled by a trained skin segmentation network and blending of original and edited radiance.

Conclusion

The paper successfully demonstrates a new method for face image decomposition using a hybrid spectral biophysical and dichromatic reflectance model. By fitting this model to multispectral images, the technique accurately explains skin reflectance and offers plausible parameter maps that facilitate realistic editing. Future work could explore the model's application to different datasets, like RGB + NIR, and develop statistical models of intrinsic facial parameters. This advancement in image decomposition significantly contributes to both research and practical applications in computer vision and graphics.

Paper to Video (Beta)

No one has generated a video about this paper yet.

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.