Class-Conditional self-reward mechanism for improved Text-to-Image models (2405.13473v2)
Abstract: Self-rewarding have emerged recently as a powerful tool in the field of NLP, allowing LLMs to generate high-quality relevant responses by providing their own rewards during training. This innovative technique addresses the limitations of other methods that rely on human preferences. In this paper, we build upon the concept of self-rewarding models and introduce its vision equivalent for Text-to-Image generative AI models. This approach works by fine-tuning diffusion model on a self-generated self-judged dataset, making the fine-tuning more automated and with better data quality. The proposed mechanism makes use of other pre-trained models such as vocabulary based-object detection, image captioning and is conditioned by the a set of object for which the user might need to improve generated data quality. The approach has been implemented, fine-tuned and evaluated on stable diffusion and has led to a performance that has been evaluated to be at least 60\% better than existing commercial and research Text-to-image models. Additionally, the built self-rewarding mechanism allowed a fully automated generation of images, while increasing the visual quality of the generated images and also more efficient following of prompt instructions. The code used in this work is freely available on https://github.com/safouaneelg/SRT2I.
- The falcon series of open language models. arXiv:2311.16867.
- Direct preference optimization with an offset. arXiv:2402.10571.
- Yolo-world: Real-time open-vocabulary object detection. arXiv preprint arXiv:2401.17270 .
- Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv:2210.17323.
- Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 .
- Mistral 7b. arXiv:2310.06825.
- Multi-concept customization of text-to-image diffusion, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1931–1941.
- Large language models understand and can be enhanced by emotional stimuli. arXiv:2307.11760.
- LimeWare, . Blueillow.ai. https://www.bluewillow.ai/. Accessed: 2024-03-21.
- Awq: Activation-aware weight quantization for llm compression and acceleration. arXiv:2306.00978.
- Visual instruction tuning. arXiv:2304.08485.
- Peft: State-of-the-art parameter-efficient fine-tuning methods. https://github.com/huggingface/peft.
- Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems 35, 27730–27744.
- Application of large language models to software engineering tasks: Opportunities, risks, and implications. IEEE Software 40, 4–8. doi:10.1109/MS.2023.3248401.
- Diffusers: State-of-the-art diffusion models. https://github.com/huggingface/diffusers.
- Learning transferable visual models from natural language supervision, in: International conference on machine learning, PMLR. pp. 8748–8763.
- Direct preference optimization: Your language model is secretly a reward model. arXiv:2305.18290.
- High-resolution image synthesis with latent diffusion models, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10684–10695.
- Llama 2: Open foundation and fine-tuned chat models. arXiv:2307.09288.
- A survey on llm-generated text detection: Necessity, methods, and future directions. arXiv:2310.14724.
- Self-rewarding language models. arXiv:2401.10020.
- Sdxs: Real-time one-step latent diffusion models with image conditions. arxiv .
- Text-to-image diffusion models in generative ai: A survey. arXiv:2303.07909.
- Responsible task automation: Empowering large language models as responsible task automators. arXiv:2306.01242.
Sponsor
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.