Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Federated Learning with Non-IID Data (1806.00582v2)

Published 2 Jun 2018 in cs.LG and stat.ML

Abstract: Federated learning enables resource-constrained edge compute devices, such as mobile phones and IoT devices, to learn a shared model for prediction, while keeping the training data local. This decentralized approach to train models provides privacy, security, regulatory and economic benefits. In this work, we focus on the statistical challenge of federated learning when local data is non-IID. We first show that the accuracy of federated learning reduces significantly, by up to 55% for neural networks trained for highly skewed non-IID data, where each client device trains only on a single class of data. We further show that this accuracy reduction can be explained by the weight divergence, which can be quantified by the earth mover's distance (EMD) between the distribution over classes on each device and the population distribution. As a solution, we propose a strategy to improve training on non-IID data by creating a small subset of data which is globally shared between all the edge devices. Experiments show that accuracy can be increased by 30% for the CIFAR-10 dataset with only 5% globally shared data.

User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Yue Zhao (394 papers)
  2. Meng Li (244 papers)
  3. Liangzhen Lai (21 papers)
  4. Naveen Suda (13 papers)
  5. Damon Civin (3 papers)
  6. Vikas Chandra (75 papers)
Citations (2,306)

Summary

A Scholarly Review of "Formatting instructions for NIPS 2017"

"Formatting instructions for NIPS 2017" by David S. Hippocampus provides a comprehensive set of guidelines for authors submitting papers to the 2017 Neural Information Processing Systems (NIPS) conference. This document curates specific formatting rules, stylistic requirements, and submission protocols to ensure consistency and high standards in the presentation of academic research within the conference proceedings.

Overview of Submission Requirements

The guidelines emphasize an electronic submission process facilitated through an online platform. The document specifies the maximum length for submitted papers, which is capped at eight pages inclusive of figures, but excluding references and acknowledgments.

The historical perspective included in the document notes a change in margin specifications implemented in 2007, which allowed for approximately 15% more text per paper. This adjustment has remained consistent through 2017 and highlights the conference’s commitment to accommodating more substantial content without compromising readability.

Style and Formatting Specifics

Authors are mandated to use the NIPS LaTeX style files provided, with strict instructions against altering these files to ensure uniformity. Any deviation from specified formatting will result in the rejection of the submission. Essential formatting elements include a title set in 17-point bold font flanked by horizontal rules, author names in boldface, and specific styles for various heading levels.

Figures and Tables

The document delineates requirements for figures and tables, emphasizing clarity, legibility, and proper labeling. Figures should be dark enough for reproduction, with captions and numbering positioned consistently. Tables must adhere to high-quality typesetting standards, avoiding vertical rules and employing the booktabs package to enhance presentation quality.

Citation and Reference Guidelines

The NIPS style mandates the use of the natbib package for citations, accommodating both author/year and numeric citation styles, provided internal consistency is maintained. Authors are advised to anonymize self-citations to support the double-blind review process.

Footnotes should be sparingly used, properly formatted, and placed at the bottom of the respective pages.

Practical Implications

This document serves dual functions: as a precise guide for authors and as an implicit quality control mechanism. By standardizing formatting, NIPS can streamline the review process, improve readability, and uphold the conference's reputation for high academic standards.

Theoretical and Future Implications

The normalization of submission guidelines across major conferences like NIPS contributes to the broader theoretical framework of academic publishing standards. Uniform formatting facilitates easier comparative analysis of research papers and ensures parity in how research is presented and evaluated.

Looking forward, it is conceivable that such guidelines will evolve to include specifications for increasingly sophisticated types of digital media and interactive content, reflecting the advancing capabilities of research dissemination technologies in AI and beyond.

Conclusion

The "Formatting instructions for NIPS 2017" paper meticulously outlines the requirements for submitting papers to the NIPS conference, ensuring a standardized and professional presentation of research. This structured approach underscores the importance of consistency and quality in academic publishing, fostering an environment where research can be rigorously and fairly evaluated. As the field of artificial intelligence continues to expand, it is likely these guidelines will adapt to encompass new forms of scholarly communication, maintaining their relevance and utility in future conferences.

X Twitter Logo Streamline Icon: https://streamlinehq.com