PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning (1711.05769v2)

Abstract: This paper presents a method for adding multiple tasks to a single deep neural network while avoiding catastrophic forgetting. Inspired by network pruning techniques, we exploit redundancies in large deep networks to free up parameters that can then be employed to learn new tasks. By performing iterative pruning and network re-training, we are able to sequentially "pack" multiple tasks into a single network while ensuring minimal drop in performance and minimal storage overhead. Unlike prior work that uses proxy losses to maintain accuracy on older tasks, we always optimize for the task at hand. We perform extensive experiments on a variety of network architectures and large-scale datasets, and observe much better robustness against catastrophic forgetting than prior work. In particular, we are able to add three fine-grained classification tasks to a single ImageNet-trained VGG-16 network and achieve accuracies close to those of separately trained networks for each task. Code available at https://github.com/arunmallya/packnet

CVPR Submission Guidelines: A Comprehensive Breakdown

The provided document details a comprehensive set of guidelines for authors preparing their manuscripts for submission to the IEEE Computer Society Press, specifically targeting the CVPR conference. This essay encapsulates the key elements of these guidelines with a focus on the explicit technical and formatting requirements that authors must adhere to.

Abstract and Introduction

The document begins with explicit instructions on drafting the abstract, emphasizing fully-justified, italicized text in a specific format and positioning on the page. This is followed by an introduction that outlines the overall steps and considerations for manuscript submission, including a notable update that discourages outdated practices such as using sticky tape for artwork attachment.

Language and Formatting

Authors are reminded that all manuscripts must be written in English. The document underscores strict formatting rules, such as maintaining an eight-page limit for the main content (excluding references), and setting the manuscript in a specified two-column format. The document provides detailed measurements for margins and column widths to ensure uniformity across all submissions.

Reviewer-Specific Considerations

A significant portion of the guidelines is dedicated to facilitating the review process. The document mandates the inclusion of a printed ruler to enable reviewers to quickly reference specific lines. It also discusses the importance of numbering sections and equations for easy reference, and the requirements for maintaining anonymity during the blind review process.

Citations and References

The guidelines provide explicit instructions on how to handle citations to maintain anonymity. Authors are advised to avoid using possessive pronouns when citing their previous work and to ensure proper formatting of multi-author citations using "et al." appropriately. The correct chronological order of multiple citations is emphasized to maintain consistency.

Mathematical Content

Requirements for the presentation of mathematical content include numbering all displayed equations and ensuring all math is set in Roman type. This is to avoid awkward clashes with surrounding text, and to facilitate clear, unambiguous referencing.

Figures, Tables, and Color Usage

The document outlines precise formatting for figures, tables, and their captions. This includes centering graphics, ensuring captions are in Roman type, and matching font sizes with the main body text. Authors are advised to consider the print readability of graphics and to refer to the CVPR 2018 webpage for guidelines on the use of color.

Blind Review and Anonymization

The guidelines provide extensive advice on maintaining anonymity. This includes proper citation practices and advice against including acknowledgments in the review copy. Authors are reminded that the paper must be able to stand alone without requiring the reviewer to consult additional materials, like technical reports.

Final Submission and Copyright

In the final section, the document stresses the importance of including a signed IEEE copyright release form with the finished manuscript. This is a critical step that must be completed before the paper can be published in the proceedings.

Implications and Speculations

These exhaustive guidelines ensure that all submissions are uniformly formatted and easily reviewable, enabling fair and efficient assessment. For experienced researchers, adherence to these guidelines facilitates the clear communication of their research contributions within a highly competitive venue. In future developments, it is anticipated that automation tools might further streamline the submission process, ensuring even stricter adherence to these guidelines and potentially integrating more advanced content-checking algorithms.

The document offers no numerical results or bold claims but provides an essential foundation for the methodological preparation of research submissions. While it may not directly contribute to theoretical advancements, its impact on the academic community is significant, setting a standard for scholarly communication within the computer vision research domain.

In summary, the meticulous attention to detail and the comprehensive nature of these guidelines reflect the stringent standards upheld by the CVPR conference, ensuring that only the highest quality research is communicated effectively to the broader scholarly community.