---
title: 'Internal Ideal Action: Theory & Applications'
url: https://www.emergentmind.com/topics/internal-ideal-action
type: topic
---

# Internal Ideal Action: Theory & Applications

Searching arXiv for the cited works and closely related usage of “internal ideal action” across domains.
“Internal ideal action” is not a single cross-disciplinary technical term. In the supplied arXiv literature, it denotes several distinct constructions that share a common contrast between internally generated structure and externally given data. In simulated agents, it names actions whose actual causes are predominantly hidden-state causes rather than sensor-driven causes; in reinforcement learning, it denotes the optimal internal bit-level choice inside an augmented MDP; in descriptive set theory, it is a narrative label for the action of an ideal forcing that is internally projectively absolute; and in category theory, it is a formal notion of action in an ideally exact context. Related papers on control and field theory combine the words “internal,” “ideal,” and “action” in adjacent but non-identical ways [1904.02995], [2109.15147], [2108.09688], [2507.06124].

## 1. Terminological range and common structure

Across the supplied literature, the expression organizes around a recurrent distinction: whether the operative source of an action lies “within” the system, or instead in external inputs, ambient forcing, or a surrounding medium. The precise meaning depends entirely on the mathematical setting.

| Domain | Core object | Sense of “internal ideal action” |
|---|---|---|
| Simulated agents | Motor action in a Markov Brain | Hidden-dominated causal ancestry [1904.02995] |
| Reinforcement learning | Internal bit choice $b\in\{0,1\}$ | Argmax of internal $Q$ in an augmented MDP [2109.15147] |
| Descriptive set theory | Forcing $\mathbb{P}_I$ from a $\sigma$-ideal | Internally absolute forcing action on countable models [2108.09688] |
| Ideally exact categories | Relative action $\xi:U(B)\flat X\to X$ | Internal ideal action generalizing unital ring and algebra actions [2507.06124] |

A plausible implication is that the phrase is best treated encyclopedically as a family of domain-specific notions rather than as a unified theory. What remains stable is the role of an “internal” mechanism—hidden states, internal code bits, internal generic extensions, or internal categorical action data—together with an “ideal” criterion that selects a distinguished subclass or optimum.

## 2. Actual causation and internally driven actions in simulated agents

In the animat study of Juel et al., the relevant contrast is between sensor-driven and hidden-node-driven causes of motor actions in an 8-node binary Markov Brain consisting of up to $2$ sensors $(S_1,S_2)$, up to $4$ hidden nodes $(A,B,C,D)$, and $2$ motors $(M_1,M_2)$ [1904.02995]. The animats solve a catch/avoid perceptual-categorization task under three evolution conditions: BL (“baseline”), 1S (“one-sensor”), and HT (“hard task”).

The causal analysis uses the Actual Causation framework. An occurrence $y_t$ is any subset of nodes in a particular state at time $t$, and a candidate cause $x_{t-1}$ is any subset at time $t-1$. The causal strength is defined as

$$
a(x_{t-1}\to y_t)
=\min_{\pi}\log_2
\frac{p(x_{t-1}\mid y_t)}
{\prod_{j=1}^{|\pi|}p(x^{(j)}_{t-1}\mid y^{(j)}_t)}.
$$

The actual cause of $y_t$ is the subset $x^*_{t-1}$ maximizing $a$. Once that cause purview is found, its causal strength is decomposed proportionally over sensor and hidden nodes. If $|H|$ of the $n$ purview nodes are hidden and $|S|$ are sensors, then

$$
a_{\text{hidden}}=a\times \frac{|H|}{n},\qquad
a_{\text{sensor}}=a\times \frac{|S|}{n},
$$

and the hidden-ratio is $a_{\text{hidden}}/a_{\text{total}}$ [1904.02995].

The direct analysis considers the transition

$$
X_{t-1}=(S_1,S_2,A,B,C,D)_{t-1}\Rightarrow Y_t=(M_1,M_2)_t,
$$

and searches over subsets $x_{t-1}\subseteq X_{t-1}$ for the direct actual causes of $M_{1,t}$, $M_{2,t}$, and the joint motor occurrence $(M_1,M_2)_t$. The framework is then extended temporally: once the direct causes at time $t$ are found, their union $z_{t-1}$ is treated as an effect occurrence, and the analysis is repeated over $X_{t-2}\Rightarrow Z_{t-1}$, then $X_{t-3}\Rightarrow Z_{t-2}$, and so on, until either the purview consists of sensors only or a look-back horizon such as $15$ time steps is reached [1904.02995].

From that backtracking pattern, the paper extracts total causal strength, sensor versus hidden strength, hidden ratio, complexity as the number of distinct purview-patterns encountered, and duration as the normalized area under the backtracking pattern. The direct-cause statistics vary systematically across conditions. Total causal strength is BL $\simeq 2.2 >$ HT $\simeq 1.6 \simeq$ 1S $\simeq 1.7$ bits. Hidden causal strength is BL $\simeq 1.0$, HT $\simeq 0.7$, 1S $\simeq 1.1$. Hidden ratio is BL $\simeq 0.45$, HT $\simeq 0.45$, 1S $\simeq 0.65$. Cause-purview size is HT $\simeq 4.5 >$ BL $\simeq 3.5 >$ 1S $\simeq 2.8$ [1904.02995].

The backtracking-chain statistics display a stronger separation. Total chain strength is 1S $\simeq 100 >$ HT $\simeq 90 >$ BL $\simeq 60$ bits across approximately $15$ steps. Chain hidden ratio is 1S $\simeq 0.50 >$ HT $\simeq 0.45 >$ BL $\simeq 0.30$. Chain complexity is 1S $\simeq 15 >$ HT $\simeq 12 >$ BL $\simeq 10$. Duration is 1S $\simeq 14 >$ HT $\simeq 12 >$ BL $\simeq 11$ [1904.02995].

Within this framework, an internal ideal action is characterized as an action “whose actual causes—and the full chain of ‘causes of causes’—reside predominantly in the agent’s internal (hidden) states, with minimal reliance on sensor inputs, and which reverberate as long as possible within the agent’s own network before grounding in the environment.” Quantitatively, the action approaches that ideal when the direct hidden ratio, chain hidden ratio, chain duration, and chain complexity are maximal. In the reported experiments, the one-sensor condition most closely realizes this characterization [1904.02995].

## 3. Sequential internal action in reinforcement learning

In the information-theoretic actuation framework, the internal action is not a motor primitive but one bit of an arithmetic code generated under an action model $\rho:S\to\Delta(A^{\le k})$ [2109.15147]. The original MDP has state space $S$, action space $A\subseteq \mathcal{A}^{\le k}$, transition-reward kernel $\mu:S\times A\to\Delta(S\times\mathbb{R})$, and external policy $\Pi(a\mid s)$. The action model provides a coding distribution over external-action strings, from which a binary arithmetic encoder $C_\rho(\cdot\mid s)$ and decoder $D_\rho(\cdot\mid s)$ are built.

The internal action space is $\mathbb{B}=\{0,1\}$. At internal time $i$, the agent observes an internal state $(s,q)\in S\times\mathbb{B}^{\le n}$, where $q$ is the partial bit-string so far, samples

$$
b\sim \pi(b\mid (s,q)),
$$

forms $q'=qb$, and queries $D_\rho(q'\mid s)=u\in A^{\le k}$. If $u$ contains the end-of-action symbol $\top$, then $\tau(u)$ is the external action sent to the environment; otherwise the agent continues decoding by taking another internal action [2109.15147].

This induces an augmented internal MDP with internal state space $I=S\times\mathbb{B}^{\le n}$, internal action space $\mathbb{B}$, and transition-reward kernel $\vartheta:I\times\mathbb{B}\to\Delta(I\times\mathbb{R})$. If decoding terminates, the next internal state is $(s',\varepsilon)$ and the reward is the external reward from $\mu$; if not, the next state is $(s,qb)$ and the reward is $0$. A stationary internal policy induces an external policy by summing over all bit-strings decoding to the same external action:

$$
\Pi(a\mid s)=\sum_{q\in D_s:\tau(D_\rho(q\mid s))=a}\prod_{i=1}^{|q|}\pi(q_i\mid s,q_{<i}).
$$

The paper’s main self-consistency result is that the internal and external action-value functions coincide whenever the internal step completes an external action:

$$
Q^\pi_\vartheta((s,q),b)=Q^\Pi_\mu(s,a)
\quad\text{whenever}\quad
\tau(D_\rho(qb\mid s))=a
$$

[2109.15147].

In this setting, the ideal internal action is defined pointwise by the internal Bellman optimum:

$$
b^*(s,q)=\arg\max_{b\in\mathbb{B}} Q_{\text{int}}((s,q),b).
$$

At the start of a decision epoch, $q=\varepsilon$, so the ideal first bit is $b^*(s,\varepsilon)=\arg\max_{b\in\mathbb{B}}Q_{\text{int}}((s,\varepsilon),b)$. If the best external action is

$$
a^*(s)=\arg\max_{a\in A}Q_{\text{ext}}(s,a),
$$

then the ideal bit-string is the arithmetic code $C_\rho(a^*\mid s)=b_1^*b_2^*\dots b_m^*$, recovered inductively from successive internal argmax choices [2109.15147].

The same paper also formulates KL-constrained and soft-regularized variants. In the Lagrangian form, the objective is

$$
J(\pi)=\mathbb{E}_\pi\Bigl[\sum_{t=0}^\infty \gamma^t\bigl(r_t-\beta D_{KL}(\pi(\cdot\mid s_t,q_t)\Vert \rho_{\text{bits}}(\cdot\mid s_t,q_t))\bigr)\Bigr].
$$

The corresponding soft-Bellman solution yields

$$
\pi^*(b\mid s,q)\propto \rho_{\text{bits}}(b\mid s,q)\exp\Bigl\{\frac{1}{\beta}Q^\beta_{\text{int}}((s,q),b)\Bigr\},
$$

which reduces to the greedy ideal internal action in the limit $\beta\to 0$ [2109.15147].

## 4. Internal ideal action in descriptive set theory

In the set-theoretic literature, “Internal Ideal Action” is used as a narrative description of the relationship between a $\sigma$-ideal $I$ on a Polish space $X$, the forcing $\mathbb{P}_I$ of Borel $I$-positive sets, and projective absoluteness [2108.09688]. The forcing is defined by

$$
\mathbb{P}_I=\{A\subseteq X\mid A\ \text{is Borel and}\ A\notin I\},
$$

ordered by reverse inclusion. The standing assumption is that $\mathbb{P}_I$ is proper.

Internal projective $\mathbb{P}_I$-absoluteness is formulated using countable elementary submodels $M\prec H_\theta$, their transitive collapse $\bar M$, and the collapsed forcing $\bar P$. If $g\subseteq \bar P$ is $\bar P$-generic over $\bar M$ and $x$ is the $\bar P$-generic real, then for every projective formula $\varphi(v)$, possibly with real parameters in $\bar M$,

$$
\bar M[g]\models \varphi(x)\iff V\models \varphi(x).
$$

This is the internal absoluteness principle [2108.09688].

The corresponding regularity property is uniformization up to $I$. For a Borel relation $R\subseteq X\times Y$ and a Borel set $A\subseteq X$ with $A\notin I$, there exists a Borel $B\subseteq A$, still not in $I$, such that either $B\cap p[R]=\varnothing$, or $B\subseteq p[R]$ and there exists a Borel function $f:B\to Y$ with $\operatorname{graph}(f)\subseteq R$ [2108.09688].

The central theorem states that, assuming $\mathbb{P}_I$ is proper, the following are equivalent:

1. Internal projective $\mathbb{P}_I$-absoluteness.
2. Projective uniformization up to $I$.
3. $1$-step absoluteness for $\mathbb{P}_I$ together with the statement that all projective subsets of $X$ are $I$-measurable.

A level-by-level refinement is also given: for each $n\ge 1$, internal $\Sigma^1_n$-absoluteness for $\mathbb{P}_I$, $\Sigma^1_n$-uniformization up to $I$, $\Pi^1_{n-1}$-uniformization up to $I$, and $1$-step $\Sigma^1_{n+1}$-absoluteness plus $I$-measurability of all $\Sigma^1_n$ sets are equivalent [2108.09688].

The paper specializes these equivalences to the meager ideal and Cohen forcing, and to the null ideal and random forcing. For Cohen forcing, internal projective absoluteness is equivalent to the statement that for every Borel relation $R\subseteq 2^\omega\times 2^\omega$ and every comeager Borel $A$, there is a comeager $B\subseteq A$ on which $R$ can be uniformized by a Borel map. For random forcing, the corresponding statement uses positive measure instead of comeagerness [2108.09688].

The narrative description “Internal Ideal Action” in this setting refers to the fact that, from inside a countable model, the forcing associated with the ideal acts on generic reals without creating new projective truths about them. The supplied account states that this “absolute behaviour is mirrored by a strong regularity property in $V$,” namely projective uniformization on an $I$-large Borel set [2108.09688].

## 5. Internal ideal action in ideally exact categories

The most formal use of the phrase occurs in category theory. Mancini, Metere, and Piazza introduce “internal coherent action” and “internal ideal action” in the context of ideally exact categories, as generalizations of different aspects of unital actions of rings and algebras [2507.06124].

An ideally exact category $U$ is Barr exact, Bourn protomodular, has finite coproducts, and has the property that the unique map $\iota:0\to 1$ is a regular epimorphism. In this setting there is a semi-abelian category $V$ and a monadic adjunction

$$
U \dashv F
$$

whose unit $\eta:1_U\Rightarrow UF$ is cartesian. For each object $B\in V$, one has the monad

$$
T^B(X)=B\flat X=\ker([1,0]:B+X\to B),
$$

and an equivalence between algebras over $B\flat$ and split extensions over $B$ [2507.06124].

A relative $U$-action, or $B$-action, is then an internal action in $V$ of the form

$$
\xi:U(B)\flat X\longto X.
$$

A coherent action is defined by requiring compatibility with the canonical action $\xi_0:UF(0)\flat X\to X$ induced by the initial object. Concretely, $\xi$ is coherent if the square with maps $\xi$, $\xi_0$, and $U(\iota)\flat 1_X$ commutes, equivalently if one can form a corresponding morphism of split extensions in $V$ [2507.06124].

An ideal action is defined by a lifting condition on the split extension corresponding to $\xi$. A split extension

$$
A \;\substack{\longrightarrow\\[-0.5ex]\longleftarrow}\; U(B)
$$

in $\operatorname{Pt}_V(U(B))$ is ideal if there exists a split extension

$$
A' \;\substack{\longrightarrow\\[-0.5ex]\longleftarrow}\; B
$$

in $\operatorname{Pt}_U(B)$ and an isomorphism $\sigma:U(A')\to A$ in $V$ compatible with the projections and splittings. When $U$ is faithful and full on isomorphisms, the choice of such a realization is essentially unique [2507.06124].

The structural result is Theorem 2.3: in any ideally exact context, every ideal action is coherent. The converse does not hold universally, but it holds in “BAT” contexts, where every coherent action and morphism is ideal. Proposition 2.7 and Corollary 2.8 characterize BAT by pullback and pseudopullback conditions for the relevant algebra and point categories [2507.06124].

The paper gives several examples in which coherence and ideality coincide. For unitary rings, a relative action $\xi:U(B)\flat X\to X$ of a unital ring $B$ on a possibly non-unital ring $X$ is ideal, hence coherent, if and only if the corresponding split extension in $\mathbf{Rng}$ actually lies in $\mathbf{Ring}$; equivalently, the middle ring $A$ admits a multiplicative unit $1_A$ with $s(1_B)=1_A$. Analogous BAT results are stated for unit-closed varieties of non-associative $F$-algebras, for MV-algebras via the adjunction $U:\mathbf{WHoop}\to\mathbf{MVAlg}$, for product algebras via $U:\mathbf{PHoop}\to\mathbf{BLAlg}$, and for a non-varietal example in the opposite of pointed sets [2507.06124].

The relation to semidirect products is expressed through Janelidze’s construction. At the algebra level there are monads $B\#(-)$ and $U(B)\flat(-)$ together with a monad morphism

$$
\gamma:B\#X\to U(B)\flat X,
$$

and a $U(B)\flat$-algebra is ideal precisely when it lifts along $\gamma$ to a $B\#$-algebra. This provides a categorical criterion for when internal actions are genuinely ideal actions [2507.06124].

## 6. Related neighboring usages in control and field theory

The supplied literature also contains several adjacent formulations in which “internal,” “ideal,” and “action” are combined, but not as the same formal notion.

In control theory, Quan and Cai study the “ideal internal dynamics” problem for unstable matrix differential equations and propose a causal dynamic IID generator [1401.1580]. The plant is

$$
\dot\eta(t)=A\,\eta(t)+B\,u(t),
$$

and an IID is a bounded solution $\eta(t)\in L_\infty$ that exactly satisfies the equation despite instability of $A$. The proposed generator augments the state by $x=[\,v,\hat\eta,e\,]^T$ and evolves

$$
\dot x=A_{cl}x+N_{cl}\xi,\qquad \hat\eta=C_{cl}^T x.
$$

The design uses mixed $\mathcal{H}_2/\mathcal{H}_\infty$ objectives, applies under the rank condition that $S$ and $A$ share no eigenvalue, does not invert $A$, and extends to slowly time-varying $A(t)$ without extra online computation [1401.1580]. This is an internal-dynamics construction rather than a definition of internal ideal action, but it is part of the same lexical neighborhood.

In field theory, Arai, Nitta, and Sakai derive an all-order effective action for the internal modulus $\epsilon$ of a generic domain wall [1508.00433]. The effective Lagrangian is

$$
L_{\text{eff}}(\epsilon)=g\!\bigl(m\sqrt{1+(\partial_\mu\epsilon)^2}\bigr),
$$

with an upper bound on $\partial_0\epsilon$, interpreted as a speed limit in internal space. In the massive $CP^1$ sigma model, $g(m)=-m$, so the effective action reduces to the Nambu–Goto form $L_{\text{eff}}=-m\sqrt{1+(\partial\epsilon)^2}$ [1508.00433]. Here the word “internal” refers to moduli space rather than agency, ideals, or forcing.

Two recent fluid papers connect “ideal” and “action” in a variational sense. Mauri and Giona formulate the action of an irrotational ideal fluid with non-local internal energy,

$$
\mathcal{L}[\rho,\theta]
=\rho\,\partial_t\theta
-\frac{\rho}{2m}|\nabla\theta|^2
-\rho\,U_{\rm int}[\rho]
-\rho\,V_{\rm ext},
$$

and show that, when the internal energy is taken as a non-local logarithmic functional and truncated after the second gradient term, the Bernoulli equation acquires the Bohm quantum potential and reduces to the Madelung equation [2503.14137]. Klusoň studies an action for an ideal fluid minimally coupled to Born–Infeld-inspired gravity,

$$
S[g,\Gamma;j^\mu,\phi]
=
S_{\rm BIMG}[g,\Gamma]+S_m[g,j^\mu,\phi],
$$

with

$$
S_m=\int d^4x\,\sqrt{-g}\,\bigl[F(j)+j^\mu\partial_\mu\phi\bigr],
$$

leading to the perfect-fluid stress tensor

$$
T^{\mu\nu}
=
p(n)g^{\mu\nu}
+
[\rho(n)+p(n)]u^\mu u^\nu
$$

and a canonical Hamiltonian form after a $3+1$ split [2512.00942]. These are action principles for ideal fluids, not definitions of internal ideal action in the stricter senses above.

Taken together, these neighboring usages show that the phrase’s components are technically mobile. “Internal” may refer to hidden causes, code bits, generic extensions of transitive collapses, categorical internal actions, internal moduli, or internal energy; “ideal” may refer to an optimum, a $\sigma$-ideal, a categorical ideal, ideal internal dynamics, or an ideal fluid; and “action” may mean a motor event, a policy decision, a forcing action, an algebraic action, or a variational functional. The exact meaning is therefore determined entirely by the surrounding formalism.

Source: https://www.emergentmind.com/topics/internal-ideal-action