LogiCity: Urban Simulation in NeSy AI

• LogiCity is a configurable simulation platform that simulates abstract urban environments using first-order logic and multi-agent dynamics.
• It enables long-horizon reasoning, multi-agent path planning, and perception integration by decoupling logical rules from concrete agents.
• LogiCity supports scalable evaluation of NeSy methods with customizable predicate libraries and benchmarks such as Safe Path Following and Visual Action Prediction.

LogiCity is a configurable simulation platform designed for advancing research in Neuro-Symbolic (NeSy) AI through abstract urban environments governed by first-order logic (FOL). It addresses critical limitations of prior NeSy benchmarks by enabling long-horizon, multi-agent, and perception-in-the-loop reasoning tasks with user-configurable logical abstractions over dynamic agent populations. LogiCity explicitly decouples logical rule specification from concrete entities, supporting compositional generalization and the scalable evaluation of NeSy methods under urban reasoning challenges (Li et al., 2024).

1. System Architecture and Simulation Pipeline

LogiCity models urban environments via a layered architecture:

• Static Semantic Grid: $M_s \in \{0,1\}^{W \times H \times B}$ encodes $B$ base categories (streets, sidewalks, intersections, buildings).
• Agents: Global agent set $\mathcal{A} = \{A_1, ..., A_N\}$, each annotated with type (e.g., Car, Pedestrian, Ambulance), demographic attributes (IsOld, IsYoung), right-of-way priority $p \in \mathbb{R}$, and dynamic state (position, heading, goal).
• Global Pathing: For each start/goal sample, agents compute collision-free trajectories: A* (pedestrian trajectories) or Dijkstra (vehicle/road network).
• Dynamic Semantic Map: $M^t \in \{0,1\}^{W \times H \times (B + N)}$ overlays per-agent heatmaps.
• Local Observation & Grounding: Each agent $A_n$ receives a cropped percept $M^t_n$ and filtered agent list $\mathcal{A}^t_n$. Grounding functions $\{\mathcal{G}_i\}$ parse these to a grounding vector $g^t_n \in \{0,1\}^D$ ($B$0 grows with predicate/agent arity).
• Logical Inference: A Z3 SAT/SMT solver instantiates $B$1 FOL rules $B$2 over $B$3, computing truth values for action predicates {Slow, Normal, Fast, Stop}, yielding $B$4.
• State Update and Rendering: Agents execute actions and update states; GPT-4-generated semantic icons are fused into $B$5 for RGB frame output $B$6.

This simulation loop produces rich, multi-modal traces for both symbolic and neuro-perceptual evaluation settings (Li et al., 2024).

2. First-Order Logic Abstractions

LogiCity's core innovation is its decoupled, customizable FOL abstraction:

• Predicates: Arbitrary set $B$7, comprising unary “semantic” (e.g., IsAmbulance(X), IsPedestrian(X), IsYoung(X)) and binary “spatial/relational” predicates (e.g., IsClose(X, Y), HigherPri(X, Y), CollidingClose(X, Y)), with formally specified arity and boolean outputs.
• Clause Syntax: Each FOL clause $B$8 takes the form

$B$9

Example: $\mathcal{A} = \{A_1, ..., A_N\}$0 corresponds to $\mathcal{A} = \{A_1, ..., A_N\}$1.

• Compositional Generalization: Rules are defined on abstract variables ($\mathcal{A} = \{A_1, ..., A_N\}$2), not fixed agent sets. Predicate and clause libraries are reusable across instantiations (“cities”) with varying agent rosters $\mathcal{A} = \{A_1, ..., A_N\}$3, ensuring logical structures remain invariant under different entity permutations.

This yields a benchmark setting for robust evaluation of rule-based, neural, and hybrid approaches on compositional abstraction tasks (Li et al., 2024).

3. User-Configurable Abstraction Space

LogiCity exposes several axes of expressivity control:

• Predicate Count ($\mathcal{A} = \{A_1, ..., A_N\}$4) and Rule Count ($\mathcal{A} = \{A_1, ..., A_N\}$5): Directly affect the grounding vector size and logical inference (SMT) complexity, scaling up combinatorial reasoning challenges.
• Rule Depth and Breadth: Users can specify literal count per body (rule depth) and the maximal multi-entity (arity) interactions (breadth).
• Predicate Types: Selection of semantic, spatial, or both predicate classes; toggling these shifts the balance between perception and logical complexity.

Example configurations include:

• Easy SPF mode: 7 unary + 2 binary predicates, $\mathcal{A} = \{A_1, ..., A_N\}$6 “Stop” rules.
• Medium/Hard: Increased predicate vocabulary, more complex constraints.
• Expert mode: 12 mixed clauses, governing all four action predicates (Slow/Normal/Fast/Stop), with nested negation and multi-agent relationships (Li et al., 2024).

4. Task Definitions: Safe Path Following and Visual Action Prediction

LogiCity specifies two central benchmarks:

4.1 Safe Path Following (SPF)

• Formulation: Partially-observable Markov decision process (POMDP) with state $\mathcal{A} = \{A_1, ..., A_N\}$7, action set $\mathcal{A} = \{A_1, ..., A_N\}$8, and observation $\mathcal{A} = \{A_1, ..., A_N\}$9 for the controlled agent.
• Reward:

$p \in \mathbb{R}$0

where $p \in \mathbb{R}$1 penalizes rule violations, $p \in \mathbb{R}$2 is action cost, and $p \in \mathbb{R}$3 penalizes overtime.

• Objective: Maximize expected cumulative reward over $p \in \mathbb{R}$440-step horizons.

4.2 Visual Action Prediction (VAP)

• Task: From RGB observation $p \in \mathbb{R}$5 and agent metadata $p \in \mathbb{R}$6, predict actions $p \in \mathbb{R}$7 for all agents.
• Pipeline: ResNet50+FPN encoder yields regional features; ROIAlign computes unary predicate scores, pairwise concatenation evaluates binary predicates to construct a scene graph; graph reasoning (GNN or NLM) produces agent action logits.
• Supervision: Supports modular (predicate/action) and end-to-end (action-only) learning (Li et al., 2024).

5. Scenario and City Generation

Multiple urban “cities” are procedurally instantiated:

• Agent compositions $p \in \mathbb{R}$8 and start/goal samples are randomized across runs.
• The constant rule/predicate definitions mean each city's instantiation shifts only the grounding, not the logical structure—facilitating systematic evaluation of compositional generalization.
• Example: One city may instantiate 2 ambulances + 3 pedestrians; another, 1 police + 2 tiro + 4 cars, but both use identical logical tables (Li et al., 2024).

6. Experimental Results and Benchmarks

6.1 Safe Path Following (SPF)

• Baselines: Include symbolic learners (Popper, MaxSynth, HRI), neural (GNN, MLP, NLM), and RL variants (DQN, A2C, PPO, DreamerV2, NLM-DQN).
• Metrics: Trajectory Success Rate (TSR), Decision Success Rate (DSR), normalized return.
• Findings:
  • Easy: Symbolic rule learners achieve TSR $p \in \mathbb{R}$9.
  • Medium/Hard: NLM and NLM-DQN surpass pure neural (GNN, MLP), indicating superior abstraction learning.
  • Compositional shift: Transfer to new agent mixes halves DQN’s TSR but reduces NLM-DQN’s only by $M^t \in \{0,1\}^{W \times H \times (B + N)}$020%.
  • Continual: NLM recovers expert-mode with 30% of new-rule data vs. 100% for MLP (Li et al., 2024).

6.2 Visual Action Prediction (VAP)

• Metrics: Per-action recall, average accuracy (aAcc), weighted accuracy (wAcc).
• Findings:
  • End-to-end GNN/NLM reach aAcc $M^t \in \{0,1\}^{W \times H \times (B + N)}$1 (Easy); modular variants drop to 0.60 due to perceptual noise.
  • Hard mode: modular NLM yields wAcc $M^t \in \{0,1\}^{W \times H \times (B + N)}$2 (vs. GNN 0.28), evidencing more robust logical generalization.
  • Perceptual noise: unary predicate recall 55%, binary 89%, emphasizing the challenge for vision-only systems (Li et al., 2024).

6.3 LLM and Human Benchmarking

• In-context LLMs (GPT-4o/GPT-4/GPT-3.5) evaluated on VAP: GPT-4o reaches 59% accuracy, below human performance (81%), especially on hard “Stop” rules, indicating the gap remaining for in-context symbolic generalization (Li et al., 2024).

7. Implications, Challenges, and Research Frontiers

LogiCity provides a scalable, FOL-driven urban simulation supporting complex, abstract NeSy reasoning tasks, high-dimensional vision, and multi-agent planning. Benchmark results reveal NLMs and related NeSy frameworks excel in abstraction and continual learning, but face significant limitations under combinatorial and perceptual complexity.

Key open challenges:

• Inducing compact, conflict-free rule sets in combinatorial spaces.
• Integrating noisy perception (sub-60% unary predicate recall) with reliable logical inference.
• Extending beyond FOL to temporal logics (e.g., LTL), fuzzy SMT, and differentiable reasoning.
• Leveraging real-world urban/traffic data for rule auto-generation and sim-to-real transfer.

Future directions:

• Bi-level planners learning abstract actions atop grounded sub-policies.
• LLM-assisted rule induction, with symbolic verifiers.
• Cross-domain transfer of abstractions to platforms such as CARLA, SCENIC, and PDDL-based planners (Li et al., 2024).

LogiCity, as a fully open-source platform, establishes itself as a pivotal resource for evaluating and advancing the state-of-the-art in Neuro-Symbolic AI with a focus on real-world-relevant complexity.