Learning Non-Lambertian Object Intrinsics across ShapeNet Categories (1612.08510v1)

Abstract: We consider the non-Lambertian object intrinsic problem of recovering diffuse albedo, shading, and specular highlights from a single image of an object. We build a large-scale object intrinsics database based on existing 3D models in the ShapeNet database. Rendered with realistic environment maps, millions of synthetic images of objects and their corresponding albedo, shading, and specular ground-truth images are used to train an encoder-decoder CNN. Once trained, the network can decompose an image into the product of albedo and shading components, along with an additive specular component. Our CNN delivers accurate and sharp results in this classical inverse problem of computer vision, sharp details attributed to skip layer connections at corresponding resolutions from the encoder to the decoder. Benchmarked on our ShapeNet and MIT intrinsics datasets, our model consistently outperforms the state-of-the-art by a large margin. We train and test our CNN on different object categories. Perhaps surprising especially from the CNN classification perspective, our intrinsics CNN generalizes very well across categories. Our analysis shows that feature learning at the encoder stage is more crucial for developing a universal representation across categories. We apply our synthetic data trained model to images and videos downloaded from the internet, and observe robust and realistic intrinsics results. Quality non-Lambertian intrinsics could open up many interesting applications such as image-based albedo and specular editing.

• The paper presents a novel CNN architecture for learning non-Lambertian object intrinsics across ShapeNet categories, moving beyond traditional Lambertian assumptions.
• Key findings show substantial improvements in reflectance and shading separation accuracy compared to Lambertian-focused benchmark models.
• This research has implications for improving applications like augmented reality, realistic rendering, and object recognition by better characterizing complex object surfaces.

Learning Non-Lambertian Object Intrinsics across ShapeNet Categories

The paper "Learning Non-Lambertian Object Intrinsics across ShapeNet Categories" by Jian Shi presents an innovative approach to understanding intrinsic images, specifically targeting non-Lambertian surfaces across various categories in the ShapeNet dataset. Intrinsic imaging, which aims to decompose images into reflectance and shading components, traditionally assumes Lambertian reflectance. However, this paper ventures beyond such assumptions, addressing a broader class of object surfaces to enhance the applicability of intrinsic decomposition.

The core contribution of the paper is the development of a Convolutional Neural Network (CNN) architecture tailored to extract non-Lambertian object intrinsics. This is significant as it counters the limitations imposed by the Lambertian assumption, thereby improving the model’s ability to handle real-world materials and lighting conditions. The incorporation of a diverse range of categories from the ShapeNet dataset further broadens the scope and robustness of the model.

Key findings from the paper include quantitative results that demonstrate substantial improvements over traditional Lambertian-focused models. The paper reports clear advancements in the accuracy of reflectance and shading separation, highlighting the importance of considering non-Lambertian properties in 3D object datasets. The experimental evaluation underscores the superiority of the proposed CNN model by comparing it against established benchmarks, where it consistently achieves higher fidelity in intrinsic image decomposition.

The implications of this research are manifold. Practically, the enhanced model can be integrated into augmented reality applications, realistic rendering in computer graphics, and object recognition systems that demand precise material and lighting characterization. Theoretically, the paper challenges existing paradigms by illustrating the feasibility and necessity of accounting for non-Lambertian characteristics.

Looking ahead, the research paves the way for more comprehensive studies in object intrinsics that could further refine the integration of non-Lambertian surfaces across different datasets. Future developments might explore the inclusion of dynamic lighting conditions, greater diversity in object material properties, and real-time processing capabilities. This work stands as a crucial step towards a more nuanced understanding of complex object surfaces in machine learning and AI-driven visual tasks.