Perception Agent

• Perception agents are spatiotemporal patterns within dynamical systems that exhibit autonomous, goal-directed behavior defined by counterfactual variation.
• They are rigorously characterized using information-theoretic measures such as specific and complete local integration to assess statistical cohesion.
• The framework provides precise definitions for perception and action, enabling effective analysis of agent-environment interactions in complex systems.

A perception agent is formally characterized as an entity embedded within a larger dynamical system—specifically, a finite multivariate Markov chain—that exhibits the haLLMarks of agency: the capacity to perceive, to act, and to pursue goal-directed behavior. In the most rigorous sense, the definition of a perception agent arises from the identification of spatiotemporal patterns (STPs) in such systems that are distinguished by their internal statistical integration, their autonomy from environmental determinism, and their capacity for counterfactual variation. The modern formalization hinges on information-theoretic constructs—including specific local integration (SLI) and complete local integration (CLI)—to single out the boundaries and properties of entities that qualify as agents, whose "perception" and "action" are rigorously defined in relation to alternative trajectories and environmental couplings.

1. Formal Definition of Perception Agents in Multivariate Markov Chains

A perception agent is defined as a distinguished STP within a finite multivariate Markov chain, described by a set of nodes $V = J \times T$ (spatial indices $J$, time indices $T$), such that the agent is not a fixed subset of random variables but rather a collection of spatiotemporal assignments—entity-sets—$E(X) \subseteq \bigcup_{A \subset V} X_A$, with $X_A$ denoting all possible assignments over $A$ (1704.02716). An entity $x_O$ is designated an "entity" if it satisfies a strict integration condition: its complete local integration (CLI) is strictly positive:

$$\iota(x_O) = \min_{\pi \in L(O) \setminus \{\mathbf{1}_O\}} \mu_\pi(x_O) > 0$$

where $\mu_\pi(x_O)$ is the specific local integration for partition $\pi$ of $O$:

$$\mu_\pi(x_O) = \log \left[ \frac{p_O(x_O)}{\prod_{b \in \pi} p_b(x_b)} \right]$$

This formalism ensures that every part of the STP increases the likelihood of all other parts, providing a statistically "cohesive" pattern that corresponds to intuitive ideas of a persistent, structured agent (e.g., a "glider" in the Game of Life).

2. Perception and Action: Counterfactual Definitions

Within this framework, perception and action receive precise, trajectory-level definitions fundamentally distinct from those presupposing fixed agent-environment partitions.

Perception:

The perception of an agent is defined by the set of all co-perception entities—those STPs that share an identical past with the given entity up to time $t$ and then diverge at $t+1$. The set $S(x_{A,t})$ is partitioned into equivalence classes ("branches") via:

$y_B \sim z_C \iff y_{B_{t+1}} = z_{C_{t+1}}$

For each environment configuration, a "branch-morph" is defined:

$$p_{\eta(x_{A,t})}(b \mid e_t, x_{A_{\le t}}) = \frac{p(b \mid e_t, x_{A_{\le t}})}{\sum_{c \in \eta(x_{A,t})} p(c \mid e_t, x_{A_{\le t}})}$$

Perceptions are then the blocks of environmental configurations that yield identical branch-morphs. In standard perception-action loop (PA-loop) settings, this definition recovers the usual sensor extraction based on equivalence of conditional agent transitions (1706.03576):

$$\hat{e}_t \sim \bar{e}_t \iff \forall m_{t+1}, m_t : p(m_{t+1} \mid m_t, \hat{e}_t) = p(m_{t+1} \mid m_t, \bar{e}_t)$$

Action:

An entity $x_A$ acts at time $t$ in trajectory $x_V$ if there exists an alternative entity $y_B$ (in another trajectory $y_V$) such that:

• $A_t = B_t$ (identical variables occupied at $t$),
• the environmental variables $V_t \setminus A_t$ coincide for $x_V$ and $y_V$ at $t$,
• but at time $t+1$, $x_{A_{t+1}} \neq y_{B_{t+1}}$ (the entities diverge in future assignments).

This encapsulates non-heteronomy: an entity's state at $t+1$ is not entirely determined by the environment at $t$. In the PA-loop, this is equivalent to a nonzero conditional entropy $H(M_{t+1} | E_t)$.

3. Spatiotemporal Patterns and Entity-Sets

The theory requires that entities satisfy three criteria:

1. Spatiotemporal compositionality: Patterns span multiple spatial and temporal nodes.
2. Degree of freedom traversal: Entities can "move" in the index set, not being confined to fixed variables.
3. Counterfactual variation: The entity may appear in one trajectory but not another.

Formally, entity-sets generalize beyond sets of variables to subsets of all possible STPs, enabling entities with dynamic or shifting boundaries. This is essential for modeling agents like mobile automata or higher organisms, whose constituent "parts" are not static.

4. Information-Theoretic Integration: Specific and Complete Local Integration

The technical constructs are grounded in local information theory. For a given partition $\pi$ of nodes $O$:

• Specific Local Integration (SLI):

Quantifies the statistical dependency between the parts of a pattern defined by $\pi$:

$$\mu_\pi(x_O) = \log \left[ \frac{p_O(x_O)}{\prod_{b \in \pi} p_b(x_b)} \right]$$

• Complete Local Integration (CLI):

The minimum SLI over all nontrivial partitions:

$$\iota(x_O) = \min_{\pi \in L(O) \setminus \{\mathbf{1}_O\}} \mu_\pi(x_O)$$

An STP is an "entity" ($\iota$-entity) if $\iota(x_O) > 0$. This definition is robust under allowable symmetries in the underlying probability distributions: if permutations $g$ leave $p_A$ invariant, then $\mu_{g\pi}(x_A) = \mu_\pi(x^g_A)$, preserving integration values under spatial reordering.

5. Examples, Applications, and Implications

Simple cellular automata and small Markov chains illustrate the principles: for example, "glider" STPs in automata are identified as $\iota$-entities with positive CLI (1704.02716). Actions are evidenced both as value actions (altering assignments) and extent actions (altering variable supports). Perceptions are constructed by branching analyses from shared pasts to divergent futures.

The formalism supports practical applications including:

• Identification of autonomous substructures in artificial life.
• Quantitative evaluation of agent–environment coupling and information integration in robotics or distributed systems.
• Design and evaluation of perception systems in simulations, sensor fusion, or information processing networks.

By formulating perception, action, and entityhood within a precise, trajectory-centric, and information-theoretic model—eschewing the limitations of fixed variable sets or linear agent-environment separations—this framework extends classical agent definitions and enables rigorous analysis of agency and perception in complex, dynamical systems.