An Inpainting-Infused Pipeline for Attire and Background Replacement
Abstract: In recent years, groundbreaking advancements in Generative Artificial Intelligence (GenAI) have triggered a transformative paradigm shift, significantly influencing various domains. In this work, we specifically explore an integrated approach, leveraging advanced techniques in GenAI and computer vision emphasizing image manipulation. The methodology unfolds through several stages, including depth estimation, the creation of inpaint masks based on depth information, the generation and replacement of backgrounds utilizing Stable Diffusion in conjunction with Latent Consistency Models (LCMs), and the subsequent replacement of clothes and application of aesthetic changes through an inpainting pipeline. Experiments conducted in this study underscore the methodology's efficacy, highlighting its potential to produce visually captivating content. The convergence of these advanced techniques allows users to input photographs of individuals and manipulate them to modify clothing and background based on specific prompts without manually input inpainting masks, effectively placing the subjects within the vast landscape of creative imagination.
- (2022). Midjourney.
- From pampas to pixels: Fine-tuning diffusion models for ga\\\backslash\’ucho heritage. arXiv preprint arXiv:2401.05520.
- Blended diffusion for text-driven editing of natural images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18208–18218.
- ediffi: Text-to-image diffusion models with an ensemble of expert denoisers. arXiv preprint arXiv:2211.01324.
- Image inpainting. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques, pages 417–424.
- Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf.
- Bradski, G. (2000). The opencv library. Dr. Dobb’s Journal: Software Tools for the Professional Programmer, 25(11):120–123.
- Diffusion models in vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence.
- A short introduction to boosting. Journal-Japanese Society For Artificial Intelligence, 14(771-780):1612.
- An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint arXiv:2208.01618.
- Deep learning. MIT press.
- Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851.
- The role of chatgpt in scientific communication: writing better scientific review articles. American Journal of Cancer Research, 13(4):1148.
- Multistage gan for fabric defect detection. IEEE Transactions on Image Processing, 29:3388–3400.
- Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11461–11471.
- Latent consistency models: Synthesizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04378.
- A review of deep learning based speech synthesis. Applied Sciences, 9(19):4050.
- Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499.
- OpenAI (2023). Gpt-4 technical report.
- Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952.
- Mirrorgan: Learning text-to-image generation by redescription. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1505–1514.
- Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. PMLR.
- Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485–5551.
- Zero-shot text-to-image generation.
- Vision transformers for dense prediction.
- Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer.
- High-resolution image synthesis with latent diffusion models.
- High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695.
- Chatgpt and open-ai models: A preliminary review. Future Internet, 15(6):192.
- Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22500–22510.
- Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 35:36479–36494.
- Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502.
- Consistency models. arXiv preprint arXiv:2303.01469.
- StabilityAI (2023). deep-floyd-if.
- Robust real-time face detection. International journal of computer vision, 57:137–154.
- Diffusers: State-of-the-art diffusion models.
- Challenges for higher education in the era of widespread access to generative ai. Economics and Business Review, 9(2):71–100.
- Perceptual adversarial networks for image-to-image transformation. IEEE Transactions on Image Processing, 27(8):4066–4079.
- Tacotron: Towards end-to-end speech synthesis. arXiv preprint arXiv:1703.10135.
- Smartbrush: Text and shape guided object inpainting with diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22428–22437.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.