Cross-Batch Memory for Embedding Learning (1912.06798v3)

Abstract: Mining informative negative instances are of central importance to deep metric learning (DML), however this task is intrinsically limited by mini-batch training, where only a mini-batch of instances is accessible at each iteration. In this paper, we identify a "slow drift" phenomena by observing that the embedding features drift exceptionally slow even as the model parameters are updating throughout the training process. This suggests that the features of instances computed at preceding iterations can be used to considerably approximate their features extracted by the current model. We propose a cross-batch memory (XBM) mechanism that memorizes the embeddings of past iterations, allowing the model to collect sufficient hard negative pairs across multiple mini-batches - even over the whole dataset. Our XBM can be directly integrated into a general pair-based DML framework, where the XBM augmented DML can boost performance considerably. In particular, without bells and whistles, a simple contrastive loss with our XBM can have large R@1 improvements of 12%-22.5% on three large-scale image retrieval datasets, surpassing the most sophisticated state-of-the-art methods, by a large margin. Our XBM is conceptually simple, easy to implement - using several lines of codes, and is memory efficient - with a negligible 0.2 GB extra GPU memory. Code is available at: https://github.com/MalongTech/research-xbm.

• The paper introduces a novel cross-batch memory architecture that leverages embeddings from multiple mini-batches to enrich training signals.
• It employs an efficient memory mechanism that significantly enhances deep metric learning, improving retrieval and clustering tasks.
• Experimental results show that incorporating cross-batch information reduces training time while increasing overall model accuracy.

Review of "LaTeX Author Guidelines for CVPR Proceedings"

The paper, "LaTeX Author Guidelines for CVPR Proceedings," provides comprehensive instructions to authors preparing manuscripts for submission to the IEEE Computer Society Press, specifically for the Conference on Computer Vision and Pattern Recognition (CVPR). It is a detailed technical guide aimed at ensuring consistency and adherence to the standards expected for CVPR submissions.

Structure and Formatting Recommendations

The paper outlines the essential components of a submission, such as the abstract, introduction, body, referencing, and formatting requirements. The abstract should adhere to a standardized format, emphasizing clear and concise summarization in a fully-justified italic style below the author information. The body text maintains a two-column format, optimizing the flow of content within defined margins.

The paper stipulates that submissions must not exceed a length of eight pages, excluding references, which do not have a limit. This constraint ensures brevity and precision in conveying research findings. Manuscripts must use Times or Times Roman fonts, reinforcing consistency across submissions.

Sections and equations are to be numbered consistently, allowing readers to reference content easily. The paper also delineates specific instructions for figure and table formatting, to maintain visual cohesion.

Blind Review and Anonymization

A critical section of the guidelines is dedicated to the double-blind review process. Authors are advised on maintaining anonymity, particularly when citing their previous work. The guide clarifies appropriate self-referencing techniques, illustrating acceptable formats without revealing author identities inadvertently.

Citation and Bibliographic Norms

The paper stresses the importance of correct citation styles, advocating for numerical ordering of citations to maintain an organized and professional bibliography. This aspect is vital for the academic integrity and facilitates peer review.

Practical and Theoretical Implications

Practically, the guidelines assist authors in preparing manuscripts that meet the CVPR's publication standards, reducing the likelihood of desk rejections due to formatting issues. Theoretically, adherence to these guidelines implies that submissions are more likely to focus on content quality rather than presentation, thereby promoting a level playing ground for manuscript evaluation based on scientific merit alone.

Future Developments

While this guide addresses the 2020 CVPR iteration, future developments could include updates to accommodate new publication templates, evolving submission platforms, or adjustments in response to feedback from the academic community. The evolution of manuscript preparation practices should ensure the integration of more advanced tools for collaborative editing and real-time formatting, potentially streamlining the submission process further.

In conclusion, the "LaTeX Author Guidelines for CVPR Proceedings" provides a robust framework for authors preparing to submit to one of the leading conferences in computer vision. By specifying detailed formatting, anonymization, and citation guidelines, it helps maintain the high publication standards necessary for the dissemination of impactful research in the field.