Entities as Experts: Sparse Memory Access with Entity Supervision (2004.07202v2)

Abstract: We focus on the problem of capturing declarative knowledge about entities in the learned parameters of a LLM. We introduce a new model - Entities as Experts (EAE) - that can access distinct memories of the entities mentioned in a piece of text. Unlike previous efforts to integrate entity knowledge into sequence models, EAE's entity representations are learned directly from text. We show that EAE's learned representations capture sufficient knowledge to answer TriviaQA questions such as "Which Dr. Who villain has been played by Roger Delgado, Anthony Ainley, Eric Roberts?", outperforming an encoder-generator Transformer model with 10x the parameters. According to the LAMA knowledge probes, EAE contains more factual knowledge than a similarly sized BERT, as well as previous approaches that integrate external sources of entity knowledge. Because EAE associates parameters with specific entities, it only needs to access a fraction of its parameters at inference time, and we show that the correct identification and representation of entities is essential to EAE's performance.

The document you provided is not an actual research paper, but rather a set of formatting and submission instructions for authors submitting manuscripts to the ACL (Association for Computational Linguistics) 2019 conference. The document outlines specific guidelines for preparing camera-ready manuscripts for publication in the conference proceedings. Below are the key elements discussed in these instructions:

Document Formatting

1. Page Layout:
   • Manuscripts must be prepared in a two-column format on A4 paper.
   • Specific margins are provided: left, right, top, and bottom must all be 2.5 cm.
   • The column width is 7.7 cm, and the height is 24.7 cm with a 0.6 cm gap between columns.
2. Font and Style:
   • The recommended font is Adobe's Times Roman; when unavailable, Computer Modern Roman is suggested.
   • Different font sizes are stipulated: 15 pt for titles, 12 pt for authors and section headings, and 11 pt for document text.
3. Sections and Headings:
   • Headings and subheadings should be numbered using Arabic numerals.
   • Sections must be structured to facilitate cross-referencing.
4. First Page Requirements:
   • Title, author names, and affiliations are to be centered across both columns and must adhere to specified formatting.
   • The abstract should be indented, and its text should use a 10-point font.

Submission Requirements

1. Submission Process:
   • Submissions must be double-blind, meaning author information should not be included in the review version.
   • Page numbers should not be included in the camera-ready version, although they are permissible in initial submission.
2. Additional Material:
   • Authors may submit supplementary materials separate from the main document. These materials should adhere to anonymity guidelines and should not be necessary to understand the paper’s main content.
3. PDF Production:
   • Authors must submit their manuscripts in PDF format. Detailed instructions are provided to ensure correct PDF generation, including font embedding and compatibility checks such as using pdflatex.
4. Review and Anonymity:
   • The review process is double-blind, requiring that submissions avoid any self-referential content that could disclose author identities.
   • An entity identifier should be filled in where indicated if preparing a camera-ready version.

Citation and References

• The instructions accommodate both natbib and older ACL citation styles, advising use of cross-compatible command options.
• DOI and permanent identifiers are emphasized to enhance the reach and citation potential of referenced works.

Submission Length

• Long papers have a page limit of eight pages (expandable to nine upon acceptance) plus references, while short papers are limited to four pages (expandable to five upon acceptance).

Accessibility and Illustrations

• Clear guidelines are provided on the inclusion of figures and tables, with an emphasis on readability, especially in grayscale, to accommodate various printing and accessibility needs.

These guidelines are critical for maintaining uniformity and professionalism in the preparation of submissions for the conference, ensuring both readability and fairness in the review process. The detailed sections and examples accommodate authors in adhering to these publication standards.