---
title: PCG Management Protocol
url: https://www.emergentmind.com/topics/pcg-management-protocol
type: topic
---

# PCG Management Protocol

The PCG (Procedural Content Generation) Management Protocol in City𝒳 is a unifying, multi-agent orchestration layer that enables scalable, controllable, and high-fidelity 3D city generation. It acts as an abstraction and management interface between LLM-driven agentic workflows and the low-level Blender Python API, ensuring extensibility, heterogeneity support, and robust execution of complex procedural generation pipelines. Through tightly specified plugin encapsulation, dynamic data-format adaptation, and multi-phase agent mediation—including visual feedback via GPT-4V—the protocol undergirds City𝒳’s ability to generate realistic, simulation-grade urban environments from multimodal inputs [2407.17572].

## 1. Protocol Architecture: Layers and Responsibilities

The PCG Management Protocol is structured in three principal logical layers:

- **Plugin Registry:** Maintains a global registry of all "action functions" representing PCG plugin calls. Each entry, termed an Action Plugin, encapsulates metadata—name, description, input/output formats, and constraints—enabling introspection and compositional planning. The registry ensures that all plugins are discoverable and invocable from a unified interface, decoupling agents and plugins.
- **Dynamic API Conversion Interface:** Provides a uniform way to resolve data format mismatches between LLM agent outputs and the heterogeneous plugin APIs. This is operationalized as a collection of conversion functions:
  $$
  \mathrm{conv}_{f_i \to f_j}: \mathcal{D}_{f_i} \longrightarrow \mathcal{D}_{f_j}
  $$
  where $f_i, f_j$ span agreed data formats like SceneLayout, Point, Mesh, and others. Chaining of converters is automatic, supporting invocation from any unified interface.
- **Infinite Asset Library & Retrieval:** Encodes every asset with a text description and a 768-dimensional CLIP embedding. Asset retrieval is performed by computing top-10 cosine similarities between a query embedding and asset embeddings, with a random selection from this set, ensuring scalable and semantically aligned asset acquisition.

A common message pool (simple RPC/message-bus) underpins all layers, facilitating asynchronous, decoupled communication among four primary agents.

## 2. Structured Plugin Registration and Discovery

Each plugin is "structuredly encapsulated" as a JSON-serializable descriptor paired with a Python callable. The registration schema explicitly declares fields for name, description, input specification, output specification, limitation, and the run function:

```python
class PluginSpec:
    def __init__(self, name: str, description: str,
                 inputs: Dict[str,TypeSpec],
                 outputs: Dict[str,TypeSpec],
                 limitation: str,
                 run_fn: Callable[...,Any]):
        self.name = name
        self.description = description
        self.inputs = inputs
        self.outputs = outputs
        self.limitation = limitation
        self.run = run_fn
```

Plugins are registered into a registry with the following interface:

```python
class PluginRegistry:
    def __init__(self):
        self.plugins: Dict[str,PluginSpec] = {}

    def register(self, spec: PluginSpec):
        self.plugins[spec.name] = spec

    def discover(self, required_inputs: Set[str],
                 required_outputs: Set[str]) -> List[PluginSpec]:
        return [p for p in self.plugins.values()
                if required_inputs.issubset(p.inputs.keys())
                and required_outputs.issubset(p.outputs.keys())]
```
This design enables capability-based querying, allowing agentic planners to compose workflows based on required input/output capabilities, rather than hardcoded tool names.

## 3. Inter-Plugin Communication and Data-Format Reconciliation

Data and messages between plugins conform to standardized "data-formats," such as SceneLayout, PointCloud, FaceList, Mesh, and Texture. The conversion interface is described by a lookup table:
$$
\mathrm{ConvMap} : F\times F \longrightarrow \bigl(\mathcal{D}_{f_i}\to\mathcal{D}_{f_j}\bigr)
$$
where $F$ is the set of all supported data formats.

Messages exchanged on the bus follow a canonical schema:
$$
M = \{
   \texttt{"sender"}: \mathrm{AgentID},
   \texttt{"action"}: \text{Name},
   \texttt{"args"}: \text{Dict}<\text{Name},\text{Value}>,
   \tau: \text{timestamp}
\}
$$

Upon message receipt, the Executor resolves the corresponding PluginSpec, converts each argument as needed via registered converters, executes the wrapped Blender/Python function, applies output conversion, and dispatches the results. This dynamic format reconciliation underpins the seamless integration of heterogeneous plugins.

## 4. Multi-Agent Orchestration and Choreography

The protocol adopts a four-agent mediation architecture:

- **Annotator:** Scans all PluginSpec entries, grouping them by conceptual category (e.g., layout tools, material assignment), and publishes a mapping $L$ of "labels → plugin names".
- **Planner:** Accepts the user goal $I$, label map $L$, and planning guide $D$, and produces a high-level workflow $W = [A_1, A_2, ...]$. At each step, the Planner adapts to changes in Blender state $S_t$, guiding the sequence of actions.
- **Executor:** Listens for execution requests, invokes the appropriate plugin with dynamically converted arguments, and emits completion messages along with the updated state $S_{t+1}$.
- **Evaluator:** After each execution step, renders a viewport image $img_{t+1}$, feeds $(img_{t+1}, A_{t+1})$ to GPT-4V, and returns an evaluation label $R \in \{\text{ok}, \text{needs\_fix}, ...\}$. If $R$ is not "ok," the Planner is prompted for refinement.

The overall orchestration is depicted as:

```python
W = Planner.plan_initial(L, I, D)
t = 0
while not user_satisfied and t < MAX_STEPS:
    A = Planner.next_action(L, I, D, W, S_t)
    publish({sender:'Planner', action:A.name, args:A.args})
    S_{t+1} = Executor.execute()
    R = Evaluator.eval(render(S_{t+1}), A)
    if R == 'needs_fix':
        Planner.refine(A, R)
    else:
        t += 1
```

This agentic management loop ensures robust, adaptively refined progress toward user-specified generative goals.

## 5. Visual Feedback and Inner Refinement Loops

The Evaluator introduces visual feedback through an automated GPT-4V classification mechanism:
$$
R = \mathrm{Evaluator}\bigl(\mathrm{GPT4V}(img_{t+1}, A, D_{\mathrm{eval}})\bigr)
$$
Should $R$ indicate any inconsistency or required correction, the Planner triggers a subtask loop, possibly inserting intermediary format conversions (e.g., "Point→Face") and rerunning the Executor:

```python
def subtask_loop(A_t):
    S', img = Executor.execute(A_t)
    R = Evaluator.eval(img, A_t)
    if R == 'ok':
        return S'
    else:
        A_t' = Planner.fix(A_t, R)
        return subtask_loop(A_t')
```

This recursive structure reinforces precision and correctness, supporting iterative refinement until all visual and semantic criteria are satisfied.

## 6. Extensibility, Scalability, and Empirical Outcomes

The protocol's architecture facilitates the insertion of new PCG plugins with only minimal wrapper and format specification, requiring no agent or workflow modification. Registry-based discovery eliminates the need for pre-specified tool names in agentic prompts, enabling flexible composition.

The asset retrieval layer—based on CLIP embeddings—permits scaling to unbounded asset collections, with each new asset registered through a simple embedding addition.

Empirical evaluation in [2407.17572] demonstrated that structured plugin encapsulation (inclusion of description, input specification, and limitation fields) yielded substantial execution and robustness benefits. Specifically, executability (ER@1) improved from approximately 30% to 94%, and success rate (SR@1) increased from roughly 40% to 83%. *A plausible implication is that protocolized registration and data-format control contribute directly to stability and success in LLM-planned PCG workflows*.

## 7. Summary and Significance

The PCG Management Protocol in City𝒳 provides a rigorous, extensible, and modular framework for procedural 3D city generation via LLM-driven agents. Key contributions include: a unified plugin registry with explicit specification, message-bus agent orchestration with clear roles, a comprehensive data-format conversion interface, scalable CLIP-based asset retrieval, and a self-healing execution loop grounded in multimodal visual feedback. Collectively, these design choices enable City𝒳 to support fine-grained, multi-modal, and scalable generation of 3D urban environments suitable for embodied intelligence research [2407.17572].

Source: https://www.emergentmind.com/topics/pcg-management-protocol