- The paper introduces DIAS, a novel domain-independent problem-solving system inspired by Artificial Life, designed to solve problems by mapping domains to a spatial 'geo' where a population of diverse actors cooperates.
- Experiments demonstrate DIAS's effectiveness and generality, successfully solving n-XOR problems and various OpenAI Gym games like CartPole and MountainCar without domain-specific adjustments, often outperforming direct evolution.
- DIAS exhibits adaptability by handling changing problems mid-run and shows signs of lifelong learning, solving subsequent problems more quickly than initial attempts, a capability not seen in direct evolution methods tested.
The paper "DIAS: A Domain-Independent Alife-Based Problem-Solving System" introduces DIAS, a novel problem-solving system inspired by Artificial Life (Alife) principles. DIAS aims to address the limitations of current AI systems, which are often highly specialized and require significant domain-specific tuning to adapt to changes or new problems. The core idea is to separate the AI from the problem domain, potentially leading to several benefits, including easier AI improvement, robustness to changes, knowledge transfer across domains, and resilience to noise and variations.
DIAS operates within a spatial medium called a "geo," where the input and output dimensions of the problem domain are mapped. A population of independent "actors" inhabits this geo, each perceiving only a portion of the environment. The actors cooperate to solve problems collectively. The system's architecture includes a three-dimensional grid where the x-axis represents domain actions, the y-axis represents domain states, and the z-axis is a messaging space for actors. This design allows the problem-solving process to remain domain-independent, enabling various types of actors to be utilized.
Each actor possesses an actor-state vector (input) and produces an actor-action vector (output) at each time step. Actor actions include writing domain-action suggestions, writing messages, indicating reproduction eligibility, moving within the geo, and changing the coordinates of a linked location within the geo. The linked location allows the actors to relate information across the grid. Actor states include energy levels, age, reproduction eligibility, coordinates, messages, domain-action suggestions, and domain-state values.
The problem-solving process involves initializing the actor population, loading domain states into the geo, allowing actors to process information and suggest actions, creating a domain-action vector by averaging suggestions across locations, executing the domain action, and obtaining a domain fitness score. Based on the system's performance, a credit assignment mechanism updates each actor's energy level based on its contribution to the overall solution. Actors with low energy are removed, while successful actors reproduce, creating new actors through crossover and mutation. The whole process goes on and on.
The paper elaborates on the importance of the credit assignment mechanism based on the 'contribution' metric, which evaluates how well the actor's suggested actions align with the actual actions taken by the system. Actors are rewarded for contributing to successful outcomes and penalized for hindering them.
Five different actor types are employed in the current version of DIAS: Random, Robot (pre-programmed), Bandit (multi-armed bandit), Q-Learning, and Rule-set Evolution.
The experiments section demonstrates DIAS's effectiveness in several benchmark problems. First, DIAS is tested on the n-XOR domain (predicting the output of n independent XOR gates), demonstrating scalability to problems of varying dimensionality and complexity. The results indicate that DIAS can reliably solve n-XOR problems with n = 1, 2, and 3, even with noise. DIAS is shown to outperform direct evolution (DE) in solving the 2-XOR and 3-XOR problems.
The second set of experiments focuses on generality by evaluating DIAS on OpenAI Gym games, including CartPole, MountainCar, Acrobot, and LunarLander. The same experimental setup and hyperparameters used for the n-XOR problems are applied without any domain-dependent adjustments, highlighting DIAS's ability to solve diverse problems out-of-the-box.
The third set of experiments assesses the ability of DIAS to adapt to changing problems. The system is tasked with switching between 1-XOR, 2-XOR, 3-XOR, and back to 1-XOR during a single run. DIAS demonstrates the ability to adapt to these changes and, in many cases, solve subsequent problems more quickly, showing lifelong learning capabilities. DE could not adapt to the 2-XOR and 3-XOR problems and didn't solve the second 1-XOR any faster than the first.
The paper concludes by emphasizing the importance of the common mapping between the problem domain and the ecosystem of actors. It allows for a separation of concerns, leading to a domain-independent problem-solving approach. Future work includes exploring parallel implementations, folding the axes of the geo for high-dimensional spaces, and designing additional actor types. The authors suggest creating benchmarks for domain-independent learning to facilitate comparison with other systems.
