NormPerceptor: Norm-Aware Perceptual Interfaces

Updated 5 July 2026

NormPerceptor is a norm-aware module that generates social norm cues from first-person RGB observations and task goals, facilitating hidden-norm planning.
It boosts embodied planning performance by improving norm compliance and task success through supervised fine-tuning and dynamic margin updates.
Beyond planning, NormPerceptor represents a family of interfaces, including transformer optimization probes and multimodal norm alignment systems.

NormPerceptor is the name of a context-conditioned cue generator introduced for embodied planning under hidden social norms: given a first-person RGB observation $o_i^{\mathrm{rgb}}$ and an ordinary task goal $g_i$ , it produces a norm-aware cue $N_i=P_\theta(o_i^{\mathrm{rgb}},g_i)$ , which is then supplied to a downstream planner to generate an action sequence $\tau_i=\pi(o_i,g_i,N_i\mid A)$ (Zhao et al., 26 Jun 2026). In broader technical usage, the same label has also been used as an interpretive lens for systems whose update rules, internal measurements, or cross-modal interfaces are controlled by norms or normalization, including approximate maximum-margin perceptrons, transformer optimization probes built from normalization layers, and multimodal norm-alignment mechanisms (Panagiotakopoulos et al., 2011, Gray et al., 2024, Li et al., 9 Dec 2025). This suggests that NormPerceptor denotes both a specific module for hidden-norm planning and a wider family of norm-aware perceptual interfaces.

1. NormAct formulation and hidden-norm planning

In its explicit published form, NormPerceptor appears within the NormAct benchmark for embodied social-norm interactions. NormAct defines each instance as

$x_i = (o_i, g_i, n_i, A, R_{\mathrm{goal}}, R_{\mathrm{norm}})$

where $o_i$ is an egocentric observation, $g_i$ is an ordinary task goal, $n_i$ is a hidden social norm constraint, $A$ is a high-level action space, and $R_{\mathrm{goal}},R_{\mathrm{norm}}$ are evaluators. The observation includes an RGB view and a paired semantic segmentation view with object IDs, but the NormPerceptor interface itself is specified on RGB plus goal. The downstream planner receives the full observation, the goal, the action space, and the generated cue; NormPerceptor is therefore a front-end module rather than the planner itself (Zhao et al., 26 Jun 2026).

The hidden norm is not verbalized in the no-cue setting. It is embedded in an ordinary task and must be inferred from scene context. The benchmark organizes these situations into 5 norm dimensions and 11 task types, including road crossing, queue waiting, avoiding interruption, turning off faucets, avoiding private rooms, respecting belongings, and giving priority to an elder. Evaluation is process-oriented rather than purely terminal: Goal Achievement, Norm Compliance, and Task Success are defined respectively by

$g_i$ 0

$g_i$ 1

$g_i$ 2

This formalization targets a specific failure mode of multimodal planners: high literal task completion with weak compliance to latent social constraints.

2. Cue generation, supervision, and empirical behavior

NormPerceptor outputs a short textual cue that connects visible scene evidence to a likely relevant social norm. The training labels are generated offline by a GPT-4o-series model with a prompt that asks for a first-person scene description and the social norms contained in the scene in two sentences. The resulting module is initialized from Qwen3-VL-2B-Instruct and trained by supervised fine-tuning on 1,100 independently generated first-person RGB images, with 100 images for each of 11 task types. The reported training configuration is 60 epochs, batch size $g_i$ 3, and learning rate $g_i$ 4 (Zhao et al., 26 Jun 2026).

NormAct’s cue-condition diagnostics show why such a module is needed. In aggregate across GPT-5.4, Claude Opus 4.7, and Gemini 3 Pro, the no-cue condition achieves 67.3% Goal Achieved, 26.4% Norm Compliance, and 21.8% Task Success, with the largest no-cue outcome class being “goal achieved but norm violating,” accounting for 751 of 1,650 trials. This pattern supports the paper’s interpretation that the principal bottleneck is not the absence of general social knowledge, but failure to activate and ground the relevant norm from the current scene before action selection (Zhao et al., 26 Jun 2026).

With Gemini 3 Pro as the fixed downstream planner, the generated-cue condition improves Norm Compliance from 26.7% to 50.0% and Task Success from 24.2% to 46.7%, while Goal Achieved changes from 77.3% to 76.2%. Human-written specific cues remain stronger, at 70.7% Norm Compliance and 56.5% Task Success, and evidence cues also outperform generated cues, at 67.1% Norm Compliance and 50.2% Task Success. By contrast, RAG cues perform poorly, with 24.5% Norm Compliance and 23.1% Task Success. The strongest task-level generated-cue results are reported for giving way at 94.0%, avoiding interruption at 74.0%, and giving priority to an elder at 74.0%; the weakest are avoiding private rooms at 16.0%, queue waiting at 18.0%, road crossing at 20.0%, turning off faucets at 28.0%, and washing used dishes at 28.0%. The resulting picture is specific: automatically generated social context is effective, but precise grounding remains the dominant limitation.

3. NormPerceptor as a margin-aware perceptron

A distinct technical lineage interprets NormPerceptor as a norm-aware perceptron whose update trigger depends on normalized margin rather than raw score. In the perceptron with dynamic margin, the reflected and augmented patterns are $g_i$ 5, the maximum directional margin is

$g_i$ 6

and the current normalized direction is

$g_i$ 7

The crucial online observation is that the classical perceptron sequence obeys

$g_i$ 8

This dynamic upper bound allows the algorithm to avoid requiring prior knowledge of $g_i$ 9 (Panagiotakopoulos et al., 2011).

The update condition becomes

$N_i=P_\theta(o_i^{\mathrm{rgb}},g_i)$ 0

with $N_i=P_\theta(o_i^{\mathrm{rgb}},g_i)$ 1 initialized using threshold $N_i=P_\theta(o_i^{\mathrm{rgb}},g_i)$ 2, and the weight update itself remains the standard perceptron rule

$N_i=P_\theta(o_i^{\mathrm{rgb}},g_i)$ 3

The method is therefore structurally simple: the additive update is unchanged, but the decision to update is controlled by a normalized, dynamically adjusted margin requirement. At convergence, the achieved directional margin $N_i=P_\theta(o_i^{\mathrm{rgb}},g_i)$ 4 satisfies

$N_i=P_\theta(o_i^{\mathrm{rgb}},g_i)$ 5

so the final separator is an $N_i=P_\theta(o_i^{\mathrm{rgb}},g_i)$ 6-accurate approximation to the maximum directional margin. The paper proves finite-step convergence for all $N_i=P_\theta(o_i^{\mathrm{rgb}},g_i)$ 7 in the linearly separable hard-margin setting, and experimentally reports that the method is considerably faster in training time than the perceptron with fixed margin, orders of magnitude faster than ROMMA, much faster than decomposition SVMs such as SVM $N_i=P_\theta(o_i^{\mathrm{rgb}},g_i)$ 8, and typically slower than DCD by a factor of a few while remaining competitive as a purely online perceptron-like method. Under this reading, NormPerceptor denotes a perceptron whose “perception” of difficulty is normalized by the current weight norm rather than fixed in advance.

4. Normalization layers as compact sensors of transformer optimization

A second norm-aware interpretation treats normalization layers as perceptors of a model’s global optimization state. In transformer LLMs, the relevant quantity is the operational gradient noise scale

$N_i=P_\theta(o_i^{\mathrm{rgb}},g_i)$ 9

estimated online from gradients at two batch sizes. Exact low-variance estimation of $\tau_i=\pi(o_i,g_i,N_i\mid A)$ 0 ordinarily requires per-example gradient norms, but the reported result is that the total GNS of contemporary transformer models is predicted well by the GNS of only the normalization layers, specifically LayerNorm scale and bias parameters $\tau_i=\pi(o_i,g_i,N_i\mid A)$ 1 and $\tau_i=\pi(o_i,g_i,N_i\mid A)$ 2 (Gray et al., 2024).

For LayerNorm, the per-example gradients are

$\tau_i=\pi(o_i,g_i,N_i\mid A)$ 3

with squared norms

$\tau_i=\pi(o_i,g_i,N_i\mid A)$ 4

Because training usually averages the loss over a minibatch, the reported means apply a $\tau_i=\pi(o_i,g_i,N_i\mid A)$ 5 correction. The empirical finding is that LayerNorm and attention layers are both highly predictive of total GNS, but LayerNorm is preferred because it is much cheaper to instrument and has slope closest to one; the paper summarizes the relationship as

$\tau_i=\pi(o_i,g_i,N_i\mid A)$ 6

The authors emphasize that this is an empirical proxy rather than a formal sufficient statistic.

The implementation consequence is unusually strong. A LayerNorm-specific CUDA kernel computes the usual backward pass and the per-example gradient norms in tandem, with essentially zero throughput overhead relative to PyTorch’s native LayerNorm implementation on an Nvidia H100 GPU, and at larger hidden dimensions it can even outperform the native implementation. Using only fused LayerNorm statistics, the method is then used to guide a practical batch-size schedule that reduces training time by 18% on a Chinchilla-optimal LLM. In this usage, NormPerceptor designates a compact normalization-layer sensor that tracks whole-model noise dynamics well enough to control training.

In multimodal LLMs, NormPerceptor is also a useful lens for a different norm-driven pathology: the severe discrepancy between high-norm visual tokens and low-norm text tokens in Pre-Norm architectures. The residual update is written as

$\tau_i=\pi(o_i,g_i,N_i\mid A)$ 7

so the residual branch sees normalized input, but the residual sum is not renormalized. The reported interface measurements on 1000 samples from MMBench, POPE, and MM-Star show that vision encoder outputs are dramatically larger than text embeddings: for example, CLIP-ViT-L/14 produces $\tau_i=\pi(o_i,g_i,N_i\mid A)$ 8, SigLIP-SO400M produces $\tau_i=\pi(o_i,g_i,N_i\mid A)$ 9, and Moon ViT-SO-400M produces $x_i = (o_i, g_i, n_i, A, R_{\mathrm{goal}}, R_{\mathrm{norm}})$ 0, whereas text embedding norms are 0.80 for Qwen2.5-7B-Instruct, 1.38 for Qwen3-8B-Instruct, and 1.09 for Llama3.2-3B-Instruct (Li et al., 9 Dec 2025).

The theoretical claim is that this norm discrepancy induces an asymmetric update dynamic. If the residual update has roughly layer-specific magnitude $x_i = (o_i, g_i, n_i, A, R_{\mathrm{goal}}, R_{\mathrm{norm}})$ 1, the effective angular change obeys

$x_i = (o_i, g_i, n_i, A, R_{\mathrm{goal}}, R_{\mathrm{norm}})$ 2

Larger $x_i = (o_i, g_i, n_i, A, R_{\mathrm{goal}}, R_{\mathrm{norm}})$ 3 therefore implies smaller angular motion, so high-norm visual tokens exhibit representational inertia and transform semantically more slowly than text tokens. The paper further links this lag to reduced cross-modal alignment and attenuated useful covariance in attention, with the resulting claim that visual information is not erased but becomes harder to extract.

The proposed repair is to insert one LayerNorm immediately after the visual projector. The target norm is computed from the text embedding matrix as

$x_i = (o_i, g_i, n_i, A, R_{\mathrm{goal}}, R_{\mathrm{norm}})$ 4

and the LayerNorm gain is initialized to

$x_i = (o_i, g_i, n_i, A, R_{\mathrm{goal}}, R_{\mathrm{norm}})$ 5

with learnable affine parameters and bias initialized to zero. When the target scale is extremely small, the paper introduces Global Weight Compensation, using

$x_i = (o_i, g_i, n_i, A, R_{\mathrm{goal}}, R_{\mathrm{norm}})$ 6

and scaling the backward pass by the inverse mean gain. On LLaVA-1.5 with a Llama backbone, simple norm alignment raises the average benchmark score from 59.01 to 62.04 and MMLU from 45.19 to 53.21; on a Qwen backbone, norm-only alignment is mixed, but with Global Weight Compensation the average rises from 68.49 to 69.41 and MMLU from 71.02 to 71.74. The paper also notes an instructive counterexample: Ovis-2.5 exhibits a large norm discrepancy but much less update asymmetry, suggesting that norm discrepancy alone is not sufficient and that tokenization and fusion architecture matter.

A broader theory of NormPerceptor-like systems appears in work that separates perception from decision. Perception Learning defines a sensory interface

$x_i = (o_i, g_i, n_i, A, R_{\mathrm{goal}}, R_{\mathrm{norm}})$ 7

and a downstream decision module

$x_i = (o_i, g_i, n_i, A, R_{\mathrm{goal}}, R_{\mathrm{norm}})$ 8

with population task risk

$x_i = (o_i, g_i, n_i, A, R_{\mathrm{goal}}, R_{\mathrm{norm}})$ 9

Its central proposal is to optimize $o_i$ 0 only with task-agnostic signals—augmentations, temporal proximity, predictive targets, reconstruction targets, or weak metadata—while training $o_i$ 1 separately on frozen $o_i$ 2. The theory defines perceptual properties such as invariance, equivariance, informativeness, variance preservation, covariance control, and leakage control, and proves that if $o_i$ 3 remains an injective factorization of a sufficient invariant statistic $o_i$ 4, then along tangent directions preserving that structure,

$o_i$ 5

and in particular

$o_i$ 6

In this formalization, a NormPerceptor-like front end is justified when it preserves sufficient invariants while remaining task-agnostic (Sanyal, 28 Oct 2025).

Normalization itself also has explicit geometric and complexity consequences. For a downstream linear map $o_i$ 7, LayerNorm mean-centers representations into the codimension-one hyperplane

$o_i$ 8

whereas RMSNorm places them on a sphere with full linear span. In the single-layer realizable teacher–student setting, this distinction changes the Local Learning Coefficient of the next weight matrix by

$o_i$ 9

so LayerNorm gives $g_i$ 0 and therefore

$g_i$ 1

while RMSNorm gives $g_i$ 2 and therefore

$g_i$ 3

The paper further argues that the threshold is binary for the codimension-one manifolds studied: any non-zero curvature is sufficient to preserve the LLC, while only affinely flat manifolds cause the drop. This places normalization-aware perceptors inside a geometry-of-representations framework rather than treating them as mere training heuristics (Chun, 28 Mar 2026).

An older biological analogue makes the same point from a sensory-coding perspective. In a statistical model of olfaction, receptor excitations

$g_i$ 4

are passed through a global inhibition threshold

$g_i$ 5

so activity depends on normalized excitation rather than absolute concentration. The result is a concentration-invariant, sparse representation of odor composition. The model predicts two generic consequences of global inhibition: odors with many molecular species are more difficult to discriminate, and receptor arrays with heterogeneous sensitivities perform badly. This provides a biological prototype for NormPerceptor-like architectures that use pooled normalization to separate composition from intensity (Zwicker, 2016).

Taken together, these lines of work portray NormPerceptor as a recurring design pattern rather than a single mechanism. In its narrowest sense, it is the cue generator used in NormAct to surface hidden social constraints before planning. In a wider technical sense, it denotes perceptual interfaces whose key operation is to read, compare, or regulate signals through norms, normalization, or explicitly norm-grounded constraints.