2000 character limit reached
VLRM: Vision-Language Models act as Reward Models for Image Captioning
Published 2 Apr 2024 in cs.CV | (2404.01911v1)
Abstract: In this work, we present an unsupervised method for enhancing an image captioning model (in our case, BLIP2) using reinforcement learning and vision-LLMs like CLIP and BLIP2-ITM as reward models. The RL-tuned model is able to generate longer and more comprehensive descriptions. Our model reaches impressive 0.90 R@1 CLIP Recall score on MS-COCO Carpathy Test Split. Weights are available at https://huggingface.co/sashakunitsyn/vlrm-blip2-opt-2.7b.
- “Flamingo: a Visual Language Model for Few-Shot Learning”, 2022 arXiv:2204.14198 [cs.CV]
- “IC3: Image Captioning by Committee Consensus”, 2023 arXiv:2302.01328 [cs.CV]
- “Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts”, 2021 arXiv:2102.08981 [cs.CV]
- “Deep reinforcement learning from human preferences”, 2023 arXiv:1706.03741 [stat.ML]
- “OpenCLIP” If you use this software, please cite it as below. Zenodo, 2021 DOI: 10.5281/zenodo.5143773
- “Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision”, 2021 arXiv:2102.05918 [cs.CV]
- Diederik P. Kingma and Jimmy Ba “Adam: A Method for Stochastic Optimization”, 2017 arXiv:1412.6980 [cs.LG]
- “BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models”, 2023 arXiv:2301.12597 [cs.CV]
- “BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation”, 2022 arXiv:2201.12086 [cs.CV]
- “Microsoft COCO: Common Objects in Context”, 2015 arXiv:1405.0312 [cs.CV]
- “Asynchronous Methods for Deep Reinforcement Learning”, 2016 arXiv:1602.01783 [cs.LG]
- “Training language models to follow instructions with human feedback”, 2022 arXiv:2203.02155 [cs.CL]
- “Learning Transferable Visual Models From Natural Language Supervision”, 2021 arXiv:2103.00020 [cs.CV]
- “LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs”, 2021 arXiv:2111.02114 [cs.CV]
- “LAION-5B: An open large-scale dataset for training next generation image-text models”, 2022 arXiv:2210.08402 [cs.CV]
- “Proximal Policy Optimization Algorithms”, 2017 arXiv:1707.06347 [cs.LG]
- “Llama 2: Open Foundation and Fine-Tuned Chat Models”, 2023 arXiv:2307.09288 [cs.CL]
- “LLaMA: Open and Efficient Foundation Language Models”, 2023 arXiv:2302.13971 [cs.CL]
- “Image Captioners Are Scalable Vision Learners Too”, 2023 arXiv:2306.07915 [cs.CV]
- “CoCa: Contrastive Captioners are Image-Text Foundation Models”, 2022 arXiv:2205.01917 [cs.CV]
- “Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language”, 2022 arXiv:2204.00598 [cs.CV]
- “OPT: Open Pre-trained Transformer Language Models”, 2022 arXiv:2205.01068 [cs.CL]
- “ChatGPT Asks, BLIP-2 Answers: Automatic Questioning Towards Enriched Visual Descriptions”, 2023 arXiv:2303.06594 [cs.CV]
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
