Umwelt Engineering in Cognitive Systems

• Umwelt engineering is the deliberate design of a system's cognitive and sensory environment, focusing on key perceptual distinctions rather than a universal model.
• It employs both measure-theoretic and linguistic frameworks to specify ecological constraints and map alignment using metrics like cosine similarity, RSA, and CKA.
• The approach guides model design by aligning sensor data, task demands, and developmental history to create locally optimal representations within defined constraints.

Umwelt engineering denotes the deliberate specification of the conditions under which a system senses, represents, and acts. In the linguistic-agent formulation, it is “the deliberate design of the linguistic cognitive environment—the vocabulary, grammar, and conceptual primitives—within which a LLM thinks”; in the broader representational formulation derived from the Umwelt Representation Hypothesis, it is the design, training, and evaluation of artificial agents as locally optimal solutions under specified ecological constraints rather than as approximations to a single universal world model (Jehu-Appiah, 29 Mar 2026, Bosch et al., 20 Apr 2026). The underlying notion of Umwelt refers to “the specific way particular organisms perceive the world, influenced by what they find important,” or, more generally, to a system’s “particular slice of reality of what it can sense, understand and act upon” (Bosch et al., 20 Apr 2026).

1. Conceptual and historical foundations

The concept is rooted in Uexküll’s notion that organisms inhabit organism-specific worlds rather than a single transparently shared reality. In contemporary technical formulations, this becomes an engineering problem: the relevant question is not only what the world is, but which distinctions are available to a system given its sensors, effectors, developmental history, and operating constraints. In the linguistic setting, the same logic is applied upstream of prompt engineering and context engineering: prompt engineering concerns what the agent is asked, context engineering concerns what the agent knows, and Umwelt engineering concerns what the agent can think (Jehu-Appiah, 29 Mar 2026).

A rigorous formal counterpart appears in the measure-theoretic treatment of embodied agents. There, the sensorimotor loop is described with Markov kernels for sensing, internal state update, action, and world transition, and two $\sigma$-algebras are distinguished: the external observer’s $\mathcal{F}_W$ over world states and the agent’s intrinsic $\mathcal{G}_W \subseteq \mathcal{F}_W$. The smaller $\sigma$-algebra is proposed as the formalization of Umwelt, because not all aspects of the externally described world are accessible from the agent’s intrinsic perspective (Ay et al., 2016). This suggests that Umwelt engineering is not merely a metaphor for task design; it can be understood as the deliberate shaping of the observable and actionable event structure available to a system.

The same formal work defines sensory equivalence by requiring equality of induced sensor-sequence distributions under all action sequences, and it shows that, under continuity and compactness assumptions, the world can be simplified without changing the internal process (Ay et al., 2016). A plausible implication is that engineered systems need not model the external world in full detail; they need only preserve the distinctions required by the target Umwelt.

2. Ecological constraint space and the rejection of universality

In the representational formulation, Umwelt engineering follows directly from the Umwelt Representation Hypothesis. That hypothesis states that alignment arises “not from convergence toward a single global optimum, but from overlap in ecological constraints under which systems develop,” and that “cognition is not a matter of reaching a global representational optimum, but rather of finding balanced compromises between the many constraints imposed that determine a system’s Umwelt” (Bosch et al., 20 Apr 2026). Ecological constraints include local environment and sensory statistics, sensory apparatus and effector capacities, internal goals, energetic and wiring costs, interaction dynamics, and developmental history; for artificial neural networks, analogous constraints include architecture, training data, objectives, learning rules, and input–output interfaces.

This position is defined against “Universality,” including the “Platonic Representation Hypothesis” and the “Anna Karenina scenario.” On the universalist view, models recover progressively better representations of an objective world, and non-shared dimensions are treated as idiosyncratic noise. The URH rejects that inference. It predicts partial alignment: “some, but not all, systems share some, but not all, representations,” because representational overlap reflects overlapping constraint structure rather than convergence to a unique world model (Bosch et al., 20 Apr 2026).

The empirical case for this view is organized around systematic adaptive differences. Across species, examples include Bogong moth stellar-compass neurons, mantis shrimp’s sixteen photoreceptor types versus humans’ three, echolocation in bats and dolphins, and magnetoreception in birds. Across individuals, the cited differences include cultural susceptibility to illusions, color perception variations such as “#thedress,” and the emergence of new category-selective regions with learning for reading, Pokémon, and cars. Between humans and ANNs, many vision models remain texture-biased whereas humans favor global shape; improved task performance does not guarantee better brain alignment, and many models fail to reach noise ceilings. Under URH, such divergences are not treated as mere defects but as signatures of local optimality under different constraints (Bosch et al., 20 Apr 2026).

3. Formalizations, metrics, and the mapping of alignment

The technical program of Umwelt engineering relies on explicit representations of constraint structure and on sensitive alignment measurements. A derived formalization consistent with URH defines an ecological constraint vector

$c = [s, t, e, d, h],$

where $s$ denotes sensory statistics, $t$ task demands or goals, $e$ embodiment and effector constraints, $d$ interaction dynamics, and $h$ developmental history. For a model with parameters $\mathcal{F}_W$0 and input $\mathcal{F}_W$1, representation is written as

$\mathcal{F}_W$2

Constraint overlap can then be defined by cosine similarity,

$\mathcal{F}_W$3

with predicted alignment

$\mathcal{F}_W$4

These are explicitly presented as derived rather than canonical equations, but they provide a compact interface for constraint-space mapping (Bosch et al., 20 Apr 2026).

Alignment itself is measured through RSA, CKA, encoding and decoding models, and neural predictivity. The same source gives representative expressions such as

$\mathcal{F}_W$5

and

$\mathcal{F}_W$6

The engineering warning attached to these metrics is central: parametric flexibility, especially in high-parameter encoding models, can obscure important differences; averaging across stimuli or regions can mask misalignment; and apparent agreement that emerges only under high-flexibility mappings should be treated as suspect (Bosch et al., 20 Apr 2026).

Accordingly, the methodological emphasis shifts from searching for a single best model to “mapping clusters of alignment in ecological constraint space.” This requires targeted stimulus selection, controversial stimuli, metamers, out-of-distribution synthetic images, active-versus-passive task comparisons, fine-grained region-by-region reporting, and comparison to noise ceilings rather than sole reliance on aggregate means (Bosch et al., 20 Apr 2026). In this framework, model comparison functions less as a ranking exercise and more as a cartographic one.

4. Linguistic Umwelt engineering in LLMs

A distinct but closely related strand applies the concept to LLMs by manipulating the medium of reasoning itself. The core claim is that “the words do not report on cognition that happened elsewhere; they are the cognition,” so altering the vocabulary and grammar available to the model alters the form of reasoning (Jehu-Appiah, 29 Mar 2026). The paper operationalizes this with two constraint families: No-Have, which removes possessive “to have” as a main verb, and E-Prime, which removes “to be.”

In Experiment 1, three models—Claude Haiku 4.5, GPT-4o-mini, and Gemini 2.5 Flash Lite—were evaluated across seven tasks in 4,470 completed trials. No-Have yielded consistent improvements: ethical reasoning increased by 19.1 percentage points with $\mathcal{F}_W$7, classification by 6.5 percentage points with $\mathcal{F}_W$8, and epistemic calibration by 7.4 percentage points, while achieving 92.8% constraint compliance. E-Prime improved causal reasoning by 14.1 percentage points and ethical reasoning by 15.5 percentage points, both with $\mathcal{F}_W$9, but it was markedly model-dependent: cross-model correlations of effect profiles reached $\mathcal{G}_W \subseteq \mathcal{F}_W$0, GPT-4o-mini showed a 27.5 percentage-point drop in epistemic calibration, and overall compliance was 48.1% (Jehu-Appiah, 29 Mar 2026).

In Experiment 2, sixteen constrained agents were used for software debugging. No single constrained agent outperformed the control individually, yet the full ensemble achieved 100% ground-truth coverage, and a minimal three-agent ensemble also achieved 100% coverage while using 17.6% of the calls; the control achieved 88.2%. A permutation test showed that only 8.0% of random three-agent subsets reached full coverage, and every successful subset contained the counterfactual agent (Jehu-Appiah, 29 Mar 2026). The interpretation offered is twofold: cognitive restructuring, in which removing default linguistic operations forces more explicit operational reasoning, and cognitive diversification, in which different constraints activate complementary regions of latent capacity.

These results define linguistic Umwelt engineering as more than prompt phrasing. It is an intervention on the representational substrate of reasoning, with measurable effects on ethics, classification, calibration, debugging coverage, and ensemble complementarity (Jehu-Appiah, 29 Mar 2026).

5. Engineering methodology and design practice

As an engineering program, the field is organized around specifying the target Umwelt first and only then choosing data, objectives, and architecture. In the constraint-based formulation, this begins by defining sensory statistics, tasks and goals, embodiment and energy or wiring limits, interaction regime, and developmental trajectory. Data are then selected to match the target sensory statistics; objectives are aligned to ecological goals; architectures and inductive biases are chosen to fit embodiment; augmentations and curricula are shaped to mirror the target niche; and evaluation is performed with multi-metric alignment protocols tailored to the relevant cluster (Bosch et al., 20 Apr 2026).

The roadmap is explicitly iterative. First define the target constraints $\mathcal{G}_W \subseteq \mathcal{F}_W$1; then construct constraint-aligned datasets and tasks; choose models and training regimes; measure and map alignment; identify clusters and boundaries; diagnose misalignment; and finally test generalization and robustness across controlled shifts, cross-modal tasks, and active paradigms (Bosch et al., 20 Apr 2026). This procedure reorients optimization away from generic accuracy and toward “tighter locality within the target cluster.”

The case studies emphasize that design variables should track niche structure. For primate-like visual alignment, the recommendation is naturalistic high-resolution human-environment images and videos, active vision dynamics, shape-emphasizing curricula, recurrent architectures, and evaluation with controversial stimuli and metamers. For insect niches, the recommendations include UV channels, lower spatial resolution, wide-field motion, floral iridescence statistics, limited-resource architectures, and objectives tuned to nectar foraging and predator detection. For agents in language-rich environments, multimodal training with realistic co-occurrence distributions and cross-modal RSA or CKA is proposed; for RL and control, embodied agents with sensorimotor loops, saccade-like sampling, and affordance objectives are required, with evaluation during action rather than passive fixation (Bosch et al., 20 Apr 2026).

In the linguistic-agent branch, the same logic appears as exact constraint prompts, compliance monitoring with regex plus dependency parsing, multi-agent ensemble construction for orthogonality rather than redundancy, and union-based aggregation with greedy Shapley-weighted selection (Jehu-Appiah, 29 Mar 2026). The common thread is that Umwelt engineering begins upstream of specific outputs: it engineers the space of available distinctions and operations.

6. Limitations, controversies, and open questions

The central controversy concerns universality. The URH does not deny that some basic features may be shared across modalities or species; it explicitly allows that claims of universality may be “locally valid within broad human-centric clusters.” What it rejects is the stronger inference from observed alignment to a single global representational optimum (Bosch et al., 20 Apr 2026). This is therefore not a dispute about whether overlap exists, but about what overlap means.

Methodological limitations are prominent. In alignment studies, flexible mappings, human-centric datasets, static passive paradigms, and underexplored modalities such as olfaction and echolocation can all distort conclusions. In the linguistic experiments, the primary limitation is the absence of an active control matching constraint prompt elaborateness; E-Prime shows low compliance and marked model dependence; and semantic matching in the debugging experiment introduces judge noise (Jehu-Appiah, 29 Mar 2026). These constraints make it difficult to distinguish robust restructuring effects from prompt-specific self-monitoring overhead in every case.

The open problems are correspondingly precise. Which constraints most strongly govern alignment clusters remains unresolved; quantifying overlap across diverse modalities is still open; scalable incorporation of wiring and energy constraints is underdeveloped; it is unclear whether foundation models can host multiple Umwelts through subspace-selection mechanisms; and more sensitive metrics for detecting misalignment are still needed (Bosch et al., 20 Apr 2026). From the measure-theoretic side, finite-data identification of sensory equivalence classes and extension from open-loop to richer closed-loop behaviors remain nontrivial (Ay et al., 2016).

Taken together, these lines of work define Umwelt engineering as a constraint-centered alternative to universalist model comparison. Its operative question is not whether all capable systems converge on the same representation, but which cognitive worlds can be designed, how their boundaries can be mapped, and which representations are locally optimal for action within a specified slice of reality.