Critical Local Perceptor in Multimodal Geometry
- Critical Local Perceptor is a training-time module that extracts 13 critical local geometric structures essential for precise multimodal problem-solving.
- It employs theory-based QA templates to generate noise-free perception pairs, achieving a 61% boost in local feature coverage on benchmark tests.
- The approach underlines both safety-critical auditing and accurate local failure evaluation, thereby enhancing overall reasoning and operational reliability.
Critical Local Perceptor is a term used most explicitly for a training-time module in multimodal geometry problem-solving and more broadly, in adjacent perception literature, for methods that identify, formalize, evaluate, or preserve local structures whose correctness is decisive for downstream reasoning or safety. In "GeoFocus: Blending Efficient Global-to-Local Perception for Multimodal Geometry Problem-Solving" (Deng et al., 9 Feb 2026), the Critical Local Perceptor (CLP) is a theory-grounded module that defines 13 types of critical local structure, generates noise-free perception QA pairs, and trains a Large Multimodal Model to identify and emphasize essential local structures before reasoning. In safety-critical vision, closely related work uses the same phrase interpretively for ontology-based auditing of rare local failure configurations, rubric-based evaluation of atomic visual facts, criticality scoring of detected objects or radar points, and runtime monitoring of perception modules (Polavaram et al., 18 Apr 2025, Wei et al., 26 Jun 2026, Gamerdinger et al., 17 Dec 2025, Brühl et al., 5 Jul 2025, Antonante et al., 2020).
1. Research lineages and scope
Current usage separates into several technically distinct lineages. One is a concrete model-training component for geometry diagrams; another is a family of safety-oriented formalisms for local scene-critical configurations, local factual auditing, and local threat prioritization. This suggests that the phrase denotes a shared design intuition—local structures can dominate downstream correctness—rather than a single standardized architecture across all domains (Deng et al., 9 Feb 2026, Polavaram et al., 18 Apr 2025, Wei et al., 26 Jun 2026, Molloy et al., 2022, Antonante et al., 2020, Gamerdinger et al., 17 Dec 2025, Brühl et al., 5 Jul 2025, Kaul et al., 30 Mar 2026).
| Research context | Role of the concept | Representative mechanism |
|---|---|---|
| Multimodal geometry | Training-time local perception module | Theory-based QA templates and DPO |
| Vision failure auditing | Human-assistive formalization layer | Ontology graphs, SWRL/SQWRL, OWL/XML |
| Multimodal evaluation | Strict local-fact evaluator | Must-Right / Easy-Wrong gated scoring |
| AV safety assessment | Hazard-analysis prompt set | HAZOP guidewords and HISS |
| Runtime supervision | Perception fault monitor | Temporal diagnostic graphs |
| AV relevance scoring | Post-detection criticality layer | Bidirectional and aggregated metrics |
| Radar front end | Safety-aware evidence preservation | Point criticality and criticality regions |
A useful disambiguation is that some mathematical papers use the lexical components “critical” and “local” in unrelated senses, such as locality of percolation thresholds or critical local nonlinearities, rather than as a perception architecture (Easo et al., 2023, Sakuma, 2023).
2. Critical Local Perceptor in GeoFocus
In GeoFocus, Critical Local Perceptor is one of two core modules. The other module, VertexLang Topology Perceptor, handles global topology perception, while CLP handles targeted local perception. The paper’s motivation is explicitly human-inspired: after first understanding the overall shape of a geometry diagram, people then attend to small local structures that encode theoretical cues, such as an angle bisector mark, a midpoint relation, a perpendicular foot, or whether two lines are parallel. The authors argue that current Large Multimodal Models are weak at this second step, because existing geometry-perception improvements mostly strengthen global perception or produce broad image summaries rather than explicitly training the model to extract local theory-bearing details (Deng et al., 9 Feb 2026).
The paper defines a critical local structure as a local geometric configuration containing information essential for problem-solving because it is directly linked to geometry theory or inference rules. Examples include angles, parallel lines, comparative distances, midpoints, angle bisectors, perpendicularity, collinearity, and point-circle incidence. CLP operationalizes this by defining 13 types of critical local structure grounded in geometry theory and common geometry problem-solving data, generating noise-free perception QA pairs from those structure types using theory-based templates, and using these instructional samples to train the model to identify and emphasize essential local structures. The module is not presented as a standalone detector used online at inference time; GeoFocus is “only employed during training, introducing no extra overhead at inference time” (Deng et al., 9 Feb 2026).
The 13 templates are divided into two main groups. Basic Measurement covers Angle Value, Angle Compare, Length Value, Length Compare, and Area Compare. Relational Reasoning covers Shape Check, Bisector Check, Perp. Check, Parallel Check, Midpoint Check, On Circle, Perp. Foot Check, and Collinearity. The role of these tasks is theorem-triggering rather than generic visual description: a model may correctly perceive a triangle or circle globally and still fail to see the exact relation that licenses the next reasoning step (Deng et al., 9 Feb 2026).
3. Template formalism, transformation modules, and training objective
GeoFocus gives CLP a lightweight but explicit formalization. For Basic Measurement, the paper defines a quantification template pool
$W_Q(K_i) = DT_{K_i} \mid K_i \in {\big[\text{Angle}, \text{Length}, \text{Area}\big],$
and a comparison pool
For Relational Reasoning, it defines line-line templates and point-line templates , covering shape properties, bisector, parallel, perpendicular, midpoint, On-Circle, Perp.-Foot, and Collinearity. These templates are instantiated by two transformation modules: Property Transformation
and Description Transformation
The paper’s examples are direct: “Length: AB = 5.02. AC = 5.02” is mapped by to a length-comparison equality template, and “Point F is the midpoint of line segment AC” is mapped by to the midpoint template (Deng et al., 9 Feb 2026).
The supervision format is preference-based rather than strict next-token imitation. For each instantiated template, CLP creates an image, a question, a correct answer, and an incorrect answer, and trains with a DPO objective: $\mathcal{L}_{\text{DPO} = -\mathbb{E}_{\mathbf{Q}, \mathbf{I}, \mathbf{R}^+}[\log P(\mathbf{R}^+ | \mathbf{Q, I})] \ + \mathbb{E}_{\mathbf{Q}, \mathbf{I}, \mathbf{R}^-}[\log P(\mathbf{R}^- | \mathbf{Q, I})].$ The intended behavior is to increase preference for correct local-structure interpretations and reduce preference for incorrect ones. The paper argues that DPO is preferable to supervised fine-tuning because many geometry answers have multiple valid surface forms. CLP is trained for 1 epoch using standard LlamaFactory setup with learning rate , and the paper provides no evidence that it is separately implemented as a region proposal network, detector head, cropping pipeline, or explicit visual attention-routing module (Deng et al., 9 Feb 2026).
A central misconception addressed by the paper is that “identify and emphasize” does not mean literal image highlighting. The implementation description supports a narrower interpretation: explicit surfacing of local theory-bearing structures as supervised QA tasks during training. The absence of crop sizes, local-region coordinates, detector architectures, or attention steering prompts reinforces that CLP is a template-instantiated synthetic instruction generator rather than a local-region perceptual subnetwork (Deng et al., 9 Feb 2026).
4. Empirical profile within GeoFocus
The most visible empirical claim attached to CLP is the “61% boost” in critical local feature coverage. The supplementary coverage table makes this precise: CLP has local coverage ratio 0, compared with 1 for CogAlign, 2 for GeoPep, and 3 for MAVIS. The 61% figure is the relative improvement over CogAlign,
4
The paper is explicit that this is a relative improvement in local task coverage, not a 61-point accuracy increase (Deng et al., 9 Feb 2026).
On the CogAlign-data perception test set, CLP outperforms CogAlign-trained baselines in both 3B and 7B settings. For 3B, Baseline is 5 avg, CogAlign 6, and Local Perceptor 7. For 7B, Baseline is 8, CogAlign 9, and Local Perceptor 0. Task-wise examples reported in the 3B setting include Angle Comparison improving from 1 to 2, Perpendicular Detection from 3 to 4, Parallel Comparison from 5 to 6, and Chart Projection from 7 to 8. The paper summarizes this as an average improvement of 9 over models trained on CogAlign’s own synthetic samples (Deng et al., 9 Feb 2026).
CLP also improves downstream geometry problem-solving when used alone. In general LMMs, 3B baseline total rises from 0 to 1, and 7B from 2 to 3. In GRPO-based geometry-specific LMMs, 3B rises from 4 to 5, and 7B from 6 to 7. Combined with VertexLang, gains are additive: for general 3B, baseline 8, Topo only 9, Local only 0, both 1; for general 7B, baseline 2, Topo only 3, Local only 4, both 5. Order matters: Topo 6 Local performs better than Local 7 Topo, with improvements of 8 in 3B and 9 in 7B. The paper also reports that expanding local coverage from 40% to 70% to 100% improves performance monotonically, and that DPO is stronger than supervised learning for CLP, with 3B 0 vs 1 and 7B 2 vs 3 (Deng et al., 9 Feb 2026).
These results support a narrow but important interpretation: within GeoFocus, a Critical Local Perceptor is a training-time capability builder for theory-aligned local extraction, not a separate online parser.
5. Context-aware local failure auditing and formalization
Outside geometry reasoning, the closest systems-level analogue is CAIRO, “Context-Awareness and Interpretability of Rare Occurrences,” which is explicitly described as a framework for discovering, describing, and formalizing critical perception failures in vision systems (Polavaram et al., 18 Apr 2025). Interpreted through the lens of a Critical Local Perceptor, CAIRO is not a new detector that recognizes objects better; it is a human-assistive auditing and formalization layer for identifying when and why local scene understanding breaks down. The focus is the gap between what a human would judge as obviously dangerous in a scene and what a black-box vision model actually perceives, especially in safety-critical domains such as autonomous driving, surveillance, and law enforcement (Polavaram et al., 18 Apr 2025).
CAIRO’s pipeline is explicitly stepwise: Detection, Segmentation, Feature extraction, Ontology graph construction, Formalization, Reasoning, Human-in-the-loop validation, and Feature modification / counterfactual validation. The front end uses Faster R-CNN, GroundingDINO, CLIP, SAM for box refinement / segmentation, and a Canny edge detector for driving-direction analysis. Features include physical properties, spatial attributes, statistical metrics, and ego-centric relations such as has_distance, has_yaw, and has_part. These become nodes and edges in a directed ontology graph whose nodes are scene entities such as Vehicle, Pedestrian, and TrafficLight, and whose edges include is_left_of, is_right_of, is_occluded_by, has_part, has_distance, and has_yaw. The ontology distinguishes Scene as a single frame and Scenario as a time series of scenes linked by temporal attributes such as timePosition, and it uses a T-box / A-box split (Polavaram et al., 18 Apr 2025).
The formal substrate is description logic, OWL, SWRL, and SQWRL. The ontology vocabulary is
4
with T-box axioms such as 5 and 6, A-box assertions such as 7 and 8, and logical operations including 9, 0, 1, 2, and 3. The general SWRL rule form is
4
CAIRO’s critical phenomena are rule-defined local configurations. Examples include CP 0001, a Vulnerable Road User with has_high_occlusion(true) and color Gray; CP 0003, a Bicycle near a Crossing Site and nearby vulnerable road users; CP 0004, a Passenger Car with no_plate = 1 and has_distance < 50.0; CP 0005, a detached Vehicle_Wheel near a Driveable_Lane; and an adversarial rule
5
The framework uses Pellet and HermiT for reasoning and stores outputs as explicit knowledge graphs in OWL/XML format, intended for sharing, downstream analysis, logical reasoning, and accountability (Polavaram et al., 18 Apr 2025).
CAIRO also makes explicit an architectural qualification often obscured in looser usage of the phrase. It is strong as an auditing framework, a failure formalization layer, a context-aware reasoning system, a human-centered explanation scaffold, and a knowledge-graph-based repository of critical local cases, but it is not a new robust detector, not an end-to-end online perception model, not a fully automated CP discovery engine, and not a quantitatively validated real-time safety monitor. Runtime reflects this role: the paper reports average elapsed times per frame of 1.69 s for Faster R-CNN, 2.14 s for GroundingDINO, 146.51 s for CLIP, 0.14 s for the lane detector, and 865.37 s for the Pellet reasoner, making CAIRO more suitable for offline auditing, validation, and forensic analysis than real-time control (Polavaram et al., 18 Apr 2025).
6. Evaluation and metricization of critical local facts
PerceptionRubrics provides a complementary perspective: not how to build a local perceptor, but how to evaluate whether one gets the crucial local facts right. The framework replaces holistic semantic matching with atomic, failure-sensitive auditing of image-grounded facts, using 1,038 images, 1,038 golden captions, and 10,718 rubrics, of which 4,053 are Must-Right and 6,665 are Easy-Wrong. The key metric is a gate on mandatory facts: 6 followed by
7
This encodes the claim that some local visual facts are logically prior: if even one Must-Right fact is wrong, the final score is zero. Empirically, the framework exposes a Reliability Gap between high Atomic Accuracy and much lower Must-Right Pass Rate; for example, Qwen2.5-VL-32B has Must-Right Item Accuracy 8 but Gate Pass 9, while Gemini-3-Pro has 0 and 1. The benchmark also reports an approximately 8.46-point open-vs-proprietary gap at the frontier, stronger human alignment than conventional captioning benchmarks, and correlations with Vision Arena of Pearson 2 and Spearman 3 (Wei et al., 26 Jun 2026).
A different metricization appears in effort-based criticality scoring for 3D perception errors in autonomous driving. That work proposes False Speed Reduction (FSR) for false positives, Maximum Deceleration Rate (MDR) for false negatives, and Lateral Evasion Acceleration (LEA) as a lateral complement, all gated by a reachability-based collision plausibility filter. FSR aggregates the cumulative speed loss caused by persistent phantom detections, MDR takes the peak required braking effort for a missed object, and LEA estimates minimum lateral acceleration needed to avoid a predicted collision. Across nuScenes and Argoverse 2, the paper reports that 65–93% of errors are non-critical, and Spearman analysis shows that MDR and LEA capture safety-relevant information not accessible to established time-based, deceleration-based, or normalized criticality measures (Kaul et al., 30 Mar 2026).
Taken together, these evaluator-centric works imply two different but compatible notions of local criticality. One is conjunctive factual sufficiency: all essential local facts must be correct jointly. The other is avoidance effort: the local error matters in proportion to the braking or steering burden it would induce. Both are narrower and less forgiving than global semantic similarity.
7. Safety assurance, runtime monitoring, and online criticality layers
In autonomous systems, critical local perception is also framed as an assurance and operations problem. "Safety Assessment for Autonomous Systems' Perception Capabilities" argues that perception should be analyzed through the notion of a Hazardous Internal System State (HISS), defined verbally as the condition in which the system’s model of the environment or its own state differs significantly from the real world. To analyze such failures, the paper adapts HAZOP for perception and introduces guidewords such as No or Not, More, Less, As well as, Part of, Other than/Instead, Reverse, Early, Late, and Intermittent. The method is scenario-based and derives Derived Safety Requirements for sensor architecture, ML architecture and training choices, self-diagnosis, fusion logic, degraded modes, and operational constraints (Molloy et al., 2022).
PerSyS addresses the runtime side. It models perception modules and their consistency checks as a diagnostic graph 4 with syndrome 5, extends classical 6-diagnosability to temporal diagnostic graphs, and uses this structure for fault detection and identification in heterogeneous perception systems. The key formal guarantee is that if the number of faults does not exceed 7, all faulty modules can be uniquely identified from the syndrome. In experiments with LGSVL and Apollo Auto, PerSyS detects failures in challenging scenarios, improves diagnosability when temporal edges are added, and incurs less than 5 ms overhead on a single-core CPU (Antonante et al., 2020).
At the post-detection layer, relevance classification asks which nearby objects actually matter for safety. The DeepAccident study of criticality metrics defines relevance as the need to perceive an object in order to avoid a safety-critical situation and evaluates metrics such as TTC, MTTC, TTB, TTA, LSM, RSS, SACRED, and SURE-Val. It proposes bidirectional criticality rating,
8
and OR-based multi-metric aggregation, reporting up to 100% improvement in criticality classification accuracy on its benchmark. This is a direct operationalization of a critical local perceptor as a post-detection criticality layer rather than a detector itself (Gamerdinger et al., 17 Dec 2025).
At the sensor front end, SAFERad pushes local criticality even earlier, to raw radar points. It computes a pointwise criticality score
9
defines temporary criticality regions around points with $\mathcal{L}_{\text{DPO} = -\mathbb{E}_{\mathbf{Q}, \mathbf{I}, \mathbf{R}^+}[\log P(\mathbf{R}^+ | \mathbf{Q, I})] \ + \mathbb{E}_{\mathbf{Q}, \mathbf{I}, \mathbf{R}^-}[\log P(\mathbf{R}^- | \mathbf{Q, I})].$0, and removes the $\mathcal{L}_{\text{DPO} = -\mathbb{E}_{\mathbf{Q}, \mathbf{I}, \mathbf{R}^+}[\log P(\mathbf{R}^+ | \mathbf{Q, I})] \ + \mathbb{E}_{\mathbf{Q}, \mathbf{I}, \mathbf{R}^-}[\log P(\mathbf{R}^- | \mathbf{Q, I})].$1 filter threshold inside those regions in subsequent frames. Using $\mathcal{L}_{\text{DPO} = -\mathbb{E}_{\mathbf{Q}, \mathbf{I}, \mathbf{R}^+}[\log P(\mathbf{R}^+ | \mathbf{Q, I})] \ + \mathbb{E}_{\mathbf{Q}, \mathbf{I}, \mathbf{R}^-}[\log P(\mathbf{R}^- | \mathbf{Q, I})].$2, the paper reports Recall $\mathcal{L}_{\text{DPO} = -\mathbb{E}_{\mathbf{Q}, \mathbf{I}, \mathbf{R}^+}[\log P(\mathbf{R}^+ | \mathbf{Q, I})] \ + \mathbb{E}_{\mathbf{Q}, \mathbf{I}, \mathbf{R}^-}[\log P(\mathbf{R}^- | \mathbf{Q, I})].$3 and Precision $\mathcal{L}_{\text{DPO} = -\mathbb{E}_{\mathbf{Q}, \mathbf{I}, \mathbf{R}^+}[\log P(\mathbf{R}^+ | \mathbf{Q, I})] \ + \mathbb{E}_{\mathbf{Q}, \mathbf{I}, \mathbf{R}^-}[\log P(\mathbf{R}^- | \mathbf{Q, I})].$4 on its constructed critical set, chooses the threshold for recall-oriented safety reasons, and states that its post-processing algorithm lowers the rate of non-clustered critical points by 74.8% in an exemplary setup compared to a moderate, generic filter. Across $\mathcal{L}_{\text{DPO} = -\mathbb{E}_{\mathbf{Q}, \mathbf{I}, \mathbf{R}^+}[\log P(\mathbf{R}^+ | \mathbf{Q, I})] \ + \mathbb{E}_{\mathbf{Q}, \mathbf{I}, \mathbf{R}^-}[\log P(\mathbf{R}^- | \mathbf{Q, I})].$5 baselines, clustered critical points rise to roughly $\mathcal{L}_{\text{DPO} = -\mathbb{E}_{\mathbf{Q}, \mathbf{I}, \mathbf{R}^+}[\log P(\mathbf{R}^+ | \mathbf{Q, I})] \ + \mathbb{E}_{\mathbf{Q}, \mathbf{I}, \mathbf{R}^-}[\log P(\mathbf{R}^- | \mathbf{Q, I})].$6–$\mathcal{L}_{\text{DPO} = -\mathbb{E}_{\mathbf{Q}, \mathbf{I}, \mathbf{R}^+}[\log P(\mathbf{R}^+ | \mathbf{Q, I})] \ + \mathbb{E}_{\mathbf{Q}, \mathbf{I}, \mathbf{R}^-}[\log P(\mathbf{R}^- | \mathbf{Q, I})].$7, while incorrectly filtered critical points are reduced by about $\mathcal{L}_{\text{DPO} = -\mathbb{E}_{\mathbf{Q}, \mathbf{I}, \mathbf{R}^+}[\log P(\mathbf{R}^+ | \mathbf{Q, I})] \ + \mathbb{E}_{\mathbf{Q}, \mathbf{I}, \mathbf{R}^-}[\log P(\mathbf{R}^- | \mathbf{Q, I})].$8–$\mathcal{L}_{\text{DPO} = -\mathbb{E}_{\mathbf{Q}, \mathbf{I}, \mathbf{R}^+}[\log P(\mathbf{R}^+ | \mathbf{Q, I})] \ + \mathbb{E}_{\mathbf{Q}, \mathbf{I}, \mathbf{R}^-}[\log P(\mathbf{R}^- | \mathbf{Q, I})].$9 (Brühl et al., 5 Jul 2025).
A consistent limitation across these operational papers is that the term does not resolve to one unified online architecture. GeoFocus CLP is a training-time module; CAIRO is human-assistive and largely offline; PerceptionRubrics is an evaluator; PerSyS is a monitor; criticality metrics and SAFERad are post-detection or point-level prioritizers. Rule curation, dependence on upstream detection quality, open-loop or synthetic evaluation regimes, and runtime cost remain recurring constraints. This suggests that “Critical Local Perceptor” is best understood not as a settled component class, but as a research program centered on one principle: local structures and local failures should be represented, supervised, audited, or scored in proportion to their theorem-level or safety-level consequence.