Unity: A General Platform for Intelligent Agents (1809.02627v2)

Abstract: Recent advances in artificial intelligence have been driven by the presence of increasingly realistic and complex simulated environments. However, many of the existing environments provide either unrealistic visuals, inaccurate physics, low task complexity, restricted agent perspective, or a limited capacity for interaction among artificial agents. Furthermore, many platforms lack the ability to flexibly configure the simulation, making the simulated environment a black-box from the perspective of the learning system. In this work, we propose a novel taxonomy of existing simulation platforms and discuss the highest level class of general platforms which enable the development of learning environments that are rich in visual, physical, task, and social complexity. We argue that modern game engines are uniquely suited to act as general platforms and as a case study examine the Unity engine and open source Unity ML-Agents Toolkit. We then survey the research enabled by Unity and the Unity ML-Agents Toolkit, discussing the kinds of research a flexible, interactive and easily configurable general platform can facilitate.

• The paper shows Unity as a general platform for AI research, enabling versatile and high-quality simulation environments for intelligent agents.
• It introduces a taxonomy of simulation platforms, categorizing environments by complexity from single tasks to dynamic multi-agent systems.
• Benchmark results with algorithms like PPO and SAC highlight Unity’s capability to foster scalable and innovative deep reinforcement learning research.

Unity: A General Platform for Intelligent Agents

In their paper "Unity: A General Platform for Intelligent Agents," Juliani et al. explore the critical role of simulated environments in advancing AI research. They propose Unity, a game engine ubiquitously utilized in game development, as an ideal candidate for a general platform to create sophisticated and diverse learning environments for intelligent agents. The authors argue for the necessity of high-quality, flexible simulation environments to drive the progress of AI algorithms, particularly those based on deep reinforcement learning (DRL).

Taxonomy of Simulation Platforms

The paper introduces a novel taxonomy of existing simulation platforms, categorizing them based on their capacity for environmental complexity:

1. Single Environment - These are fixed environments, which act as black boxes for the agents.
2. Environment Suite - Consists of multiple environments designed for benchmarking algorithms.
3. Domain-Specific Platform - Platforms tailored to specific kinds of tasks such as locomotion or navigation.
4. General Platform - Highly flexible platforms capable of simulating environments with complex visuals, physics, tasks, and social interactions.

Juliani et al. position modern game engines, particularly Unity, as exemplary instances of general platforms. Unity’s flexibility allows it to span the full spectrum of possible simulation environments, from simple 2D tasks to complex multi-agent systems with rich physical interactions.

Unity Engine and Unity ML-Agents Toolkit

The Unity engine is highly suitable for AI research owing to its robust features:

• Sensory Complexity: Unity supports high-fidelity rendering, complex textures, and shaders, capable of generating near-photorealistic images. Custom sensory data, including depth information and infrared imagery, can also be integrated.
• Physical Complexity: Unity uses advanced physics engines like Nvidia PhysX and Havok, allowing simulations involving rigid bodies, soft bodies, particle dynamics, fluid dynamics, and ragdoll physics.
• Task Logic Complexity: Unity’s scripting system in C# can define dynamic, hierarchically structured tasks requiring advanced problem-solving strategies.
• Social Complexity: Unity’s infrastructure supports multi-agent scenarios through comprehensive abstractions for networking and agent interactions.

Moreover, the toolkit enables:

• Fast and Distributed Simulation: Accelerated data collection is possible through asynchronous physics and framerate-independent game logic.
• Flexible Control: Researchers can dynamically alter environment parameters, which is crucial for curriculum learning and domain randomization.

Benchmarks and Performance

Juliani et al. provide benchmark results for several example environments included with the Unity ML-Agents Toolkit. They demonstrate the efficiency and scalability of the platform and show how existing state-of-the-art algorithms like Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC) perform on these benchmarks.

Current and Future Research

The versatility of Unity has facilitated significant research. For example, AI2Thor leverages Unity for indoor navigation tasks, enabling notable advancements in sim-to-real transfer learning. Similarly, this platform has been used by researchers to explore topics like intrinsic motivation, neural attention, semi-parametric reinforcement learning, and even the co-evolution of control and morphology in agents.

The Obstacle Tower challenge serves as a case paper illustrating Unity’s potential in pushing the boundaries of DRL research. Featuring procedurally generated levels and sparse rewards, it presents substantial hurdles that standard algorithms struggle to overcome, thus fostering innovation.

Implications and Future Directions

The employment of Unity as a general platform promises numerous benefits. It enables the development of environments fundamental for research in areas such as:

• Effective Learning Environments: AI can evolve in more sophisticated and dynamically changing settings.
• Human-In-The-Loop Training: Unity's interactive environments facilitate human intervention during the agent's learning process.
• Agent-Human Collaboration: Researchers can train agents in mixed human-agent scenarios at scale, furthering advancements in human-AI interaction.

As for future work, Unity and the Unity ML-Agents Toolkit will continue evolving to support a broader array of research needs, potentially incorporating advanced user interfaces for hyperparameter tuning and extended algorithmic capabilities.

Conclusion

Juliani et al. argue convincingly for Unity’s role as a general AI research platform, underscoring its flexibility, scalability, and robustness. Their work provides a roadmap for leveraging game engines to create complex, diverse, and interactive simulation environments, thereby significantly contributing to the progress of AI research.