Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Offline Reinforcement Learning with Behavioral Supervisor Tuning (2404.16399v2)

Published 25 Apr 2024 in cs.LG and cs.AI

Abstract: Offline reinforcement learning (RL) algorithms are applied to learn performant, well-generalizing policies when provided with a static dataset of interactions. Many recent approaches to offline RL have seen substantial success, but with one key caveat: they demand substantial per-dataset hyperparameter tuning to achieve reported performance, which requires policy rollouts in the environment to evaluate; this can rapidly become cumbersome. Furthermore, substantial tuning requirements can hamper the adoption of these algorithms in practical domains. In this paper, we present TD3 with Behavioral Supervisor Tuning (TD3-BST), an algorithm that trains an uncertainty model and uses it to guide the policy to select actions within the dataset support. TD3-BST can learn more effective policies from offline datasets compared to previous methods and achieves the best performance across challenging benchmarks without requiring per-dataset tuning.

Citations (1)

Summary

  • The paper proposes a novel offline reinforcement learning method that incorporates behavioral supervision to improve policy performance.
  • It details a tuning mechanism leveraging expert behavioral data to balance exploration and exploitation effectively.
  • Experimental results demonstrate enhanced robustness and safety compared to traditional offline RL techniques.

Refining Document Submission Guidelines for IJCAI-23

Introduction

IJCAI-23 conference introduces specific requirements for the manuscript preparation and submission process, detailing necessary formatting and procedural aspects. These guidelines aim to standardize the submission quality across various tracks, ensuring clarity and accessibility of papers.

Submission Guidelines Overview

Paper Length and Types

Manuscript length for IJCAI-23 is capped at seven pages, with an additional allowance for two pages dedicated solely to references and statements if applicable. The leniency on content placement within these additional pages varies by track, hinting at a tailored approach to different research themes. Authors are also given the option to purchase extra pages in some tracks, opening up space for extended discourse if necessary.

Formatting Requirements

A comprehensive set of formatting rules has been designated, with LaTeX and Microsoft Word templates available for authors. These templates meticulously define aspects like margin sizes, column width, and necessary fonts, predominantly recommending the Adobe Times Roman. Critical details include maintaining the letter format for page setup and a stringent prohibition on altering line numbering in initial submissions to facilitate effective review processes.

Key Submission Distinctions

Anonymity in Submissions

Depending on the track, submissions may need to be anonymized. This requirement affects how authors prepare their manuscripts, particularly in acknowledging affiliations and previous work, which must be genericized to preserve the review’s integrity.

Camera-Ready Submissions

For accepted papers in their final form, author names and affiliations must be clearly stated. Interestingly, the guidelines stipulate that no additional authors can be added post-review, ensuring that only contributors present at the submission stage are recognized officially.

Ethical and Acknowledgement Statements

The guidelines intricately detail how and where to include ethical statements and acknowledgements. These are optional and, if included, should not be numbered - mirroring the format of the main content but existing distinctly from the academic discourse.

Heading and Section Organization

Use of Headings

The proper structuring of documents is emphasized, with guidelines on heading use, section numbering, and appropriate spacing to maintain readability and structural integrity. Hierarchical organization from major headings down to subsubsections is outlined, demonstrating meticulous attention to detail.

Illustration and Table Integration

The document guides the placement and formatting of illustrations and tables within the text. These visual aids must be clearly numbered, labeled, and integrated into the document in a manner that supports the narrative flow while adhering to readability standards when printed, even in monochrome.

Conclusion

The guidelines set forth by IJCAI-23 reflect a careful balance between rigidity for the sake of uniformity and flexibility to cater to varied academic needs. The distinction between submission types and the explicit instructions on formatting and content organization underscore a commitment to a streamlined, accessible conference proceeding. These detailed prescriptions not only facilitate the review process but also ensure that the final proceedings uphold a high standard of academic presentation. Future adaptations and refinements of these guidelines might focus more on digital presentation aspects, given the increasing reliance on electronic resources in academic dissemination.