MLLM as Retriever: Interactively Learning Multimodal Retrieval for Embodied Agents (2410.03450v2)

Abstract: MLLM agents demonstrate potential for complex embodied tasks by retrieving multimodal task-relevant trajectory data. However, current retrieval methods primarily focus on surface-level similarities of textual or visual cues in trajectories, neglecting their effectiveness for the specific task at hand. To address this issue, we propose a novel method, MLLM As ReTriever (MART), which enhances the performance of embodied agents by utilizing interaction data to fine-tune an MLLM retriever based on preference learning, such that the retriever fully considers the effectiveness of trajectories and prioritizes them for unseen tasks. We also introduce Trajectory Abstraction, a mechanism that leverages MLLMs' summarization capabilities to represent trajectories with fewer tokens while preserving key information, enabling agents to better comprehend milestones in the trajectory. Experimental results across various environments demonstrate our method significantly improves task success rates in unseen scenes compared to baseline methods. This work presents a new paradigm for multimodal retrieval in embodied agents, by fine-tuning a general-purpose MLLM as the retriever to assess trajectory effectiveness. All the code for benchmark tasks, simulator modifications, and the MLLM retriever is available at https://github.com/PKU-RL/MART.

• The paper introduces MART, a framework that uses preference-based fine-tuning and interactive learning to select effective trajectories for embodied agents.
• It proposes trajectory abstraction to compress data without losing critical task milestones, improving performance in long-horizon tasks.
• Empirical studies in AI2-THOR and LEGENT demonstrate a significant improvement, with MART achieving up to a 40% success rate compared to 26% for baselines.

Multimodal Retrieval for Embodied Agents: A Deep Dive into MLLM as Retriever (MART)

In recent years, the deployment of embodied agents in complex, real-world tasks has highlighted the critical role of trajectory data in enhancing task performance. However, traditional retrieval methods have predominantly focused on the surface-level similarity of textual or visual data, often missing the mark in assessing task-specific effectiveness. Addressing this gap, the paper "MLLM AS RETRIEVER: INTERACTIVELY LEARNING MULTIMODAL RETRIEVAL FOR EMBODIED AGENTS" introduces a novel methodology termed MLLM As ReTriever (MART). This approach aims to refine embodied agents' selection of the most effective trajectories by leveraging interactive learning combined with the inherent capabilities of Multimodal LLMs (MLLMs).

Core Contributions

MART innovatively integrates interactive learning to enhance the retrieval process, focusing on trajectory effectiveness rather than mere similarity. Key contributions include:

1. Preference-Based Fine-Tuning: The MART framework utilizes interaction data for preference learning. It constructs preference pairs based on the agent's task success with various reference trajectories. Fine-tuning utilizes a Bradley-Terry model to adjust the MLLM retriever, enabling it to prioritize trajectories that enhance task success in unseen scenarios.
2. Trajectory Abstraction: This paper introduces a mechanism to abstract trajectories, allowing them to be represented with fewer tokens without losing critical information. This abstraction aids agents in better comprehending the task-specific milestones, which is crucial for performance in long-horizon tasks.
3. Empirical Validation: The paper presents rigorous empirical studies across diverse environments, such as AI2-THOR and LEGENT. These validate MART's efficacy, revealing a significant improvement in task success rates over baseline methods, with over a 10% increase in success rates across different environments.

Numerical Findings and Implications

Experimental results demonstrate that MART significantly outperforms baseline models both in terms of success rates and task completion efficiency. Specifically, MART achieved a 40% success rate in AI2-THOR compared to a maximum of 26% in baselines. This improvement reflects MART's enhanced ability to retrieve trajectory data that genuinely contributes to successful task completions, rather than data that superficially appears relevant. Additionally, MART's introduction of Trajectory Abstraction reduces the context window needed, thus managing long-horizon tasks more efficiently.

The theoretical implications are substantial, as this work challenges the traditional paradigms of multimodal retrieval by exemplifying how embedded interactive learning can refine retrieval processes. Practically, MART's methodology could be extended to other complex real-world applications involving robotics and interactive AI systems, wherein task-specific retrieval and effective reasoning are paramount tasks.

Prospects for Future Directions

Future developments in AI could further explore expanding MART's capabilities to handle multiple trajectories simultaneously, akin to few-shot learning paradigms. This expansion could allow for more sophisticated splicing of diverse skill sets within trajectories, facilitating more complex and nuanced task execution. Developing methodologies to retrieve sub-segments of trajectories rather than full lengths could also optimize the retrieval process, enhancing both the efficiency and applicability of MART in increasingly intricate environments.

In conclusion, MART offers a compelling advancement in the field of embodied agents by not only improving the retrieval accuracy for task-specific data but also by setting a precedent for future research integrating interactive feedback into multimodal learning models. The paper lays a solid foundation for future explorations in dynamically adaptive retrieval systems poised to tackle the evolving demands in AI and robotics tasks.