Semantic-Topological Expansion (STeX)

• Semantic-Topological Expansion (STeX) is a framework that combines semantic annotations with topological structures to disambiguate and enrich mathematical documents and robotic maps.
• It utilizes custom grammar synthesis and interactive AST disambiguation to convert ambiguous LaTeX expressions into machine-interpretable semantic macros.
• In robotics, STeX enhances navigational graphs with rich semantic metadata, allowing efficient foundation model-driven planning and robust task automation.

Semantic-Topological eXpansion (STeX) refers to a suite of methodologies that integrate semantic information with topological structures, enabling both human and machine agents to navigate, interpret, and manipulate complex domains—ranging from mathematical documents to physical environments—by enriching classical graph representations with domain-specific, machine-interpretable annotation. STeX frameworks have been applied across two key domains: the semantic markup of mathematical documents, where LaTeX is disambiguated into semantic macro structures, and the semantic-topological mapping of indoor spaces for lifelong robot autonomy via hybrid human-foundation model annotation. In each context, STeX resolves representational ambiguity, supports interoperability, and enables downstream automation by unifying symbolic graph connectivity with semantically enriched node and edge attributes.

1. Conceptual Foundations and Motivations

Classical topological or syntactic representations—such as navigation graphs in robotics or markup-less LaTeX in mathematics—lack sufficient semantic granularity for unambiguous interpretation or autonomous reasoning. In LaTeX, syntax like $P \times Q$ or "sin x/y–z" is context-sensitive and ambiguous: it can encode multiple distinct mathematical operations depending on authorial intent, surrounding definitions, or typographic convention. In robotics, a pure topological map $\mathcal{M} = G(V,E)$ only captures connectivities between waypoints; no information about object distributions, affordance, or the semantic intent of manipulation targets is present.

Semantic-Topological eXpansion (STeX) systematically addresses these limitations by augmenting base topological/syntactic layers with rich semantic metadata. In mathematical authoring, this enables the replacement of ambiguous expressions with explicit semantic macros (e.g., $\backslash$cart{P,Q} for Cartesian product, or $\backslash$matrixtimes[x]{P,Q} for matrix multiplication), supporting machine-precise markup and subsequent conversion to formats for proof assistants, computer algebra systems, and interactive digital textbooks (Vrečar et al., 2024). In semantic mapping, STeX extends nodes/edges with human- and LM-understandable attributes (object inventory, manipulation action, ownership, open/closed states) serialized in JSON, which serve as the substrate for LLM-driven planning and symbolic reasoning (Kathirvel et al., 5 Nov 2025).

2. Semantic Markup in Mathematical Documents

Ambiguities in mathematical LaTeX pose fundamental challenges to tool interoperability, semantic search, and automated verification. The STeX system in this domain operates through a workflow anchored in semi-automated grammar synthesis and interactive markup refinement:

1. Macro Extraction and Grammar Generation: STeX modules (archives of semantic macro definitions, e.g. for $\lambda$-calculus) specify, via $\backslash$symdef, a macro’s name, rendering template, arity (including flexary), argument types, and precedence. A bespoke compiler extracts this database and emits a context-free grammar suitable for a GLR parser (e.g., parglare in Python) (Vrečar et al., 2024).
2. Nonterminal Structure and Rule Priority: Each macro yields grammar nonterminals ($\text{main\_abs}$, $\text{notation\_abs\_N}$, etc.), with argument slots replaced by nonterminals reflecting declared types. Operator precedence information, if present, annotates rules to drive preferential parse resolution.
3. Parsing and Interactive Disambiguation: Inline LaTeX formulas are parsed into possibly multiple abstract syntax trees (ASTs); a GTK-based GUI assists the author in disambiguating cases where the grammar admits several plausible readings. Users select the intended AST per ambiguous formula; the system then reconstitutes the input as nested STeX macros in LaTeX, preserving both the original and the semantified rendering.
4. Case Study (Untyped $\lambda$-Terms): For lecture notes on $\lambda$-calculus, three macros (\texttt{var}, \texttt{abs}, \texttt{app}) sufficed to induce a full markup grammar. For example, $\lambda xyz.xy$ produces two ASTs corresponding to different binding structures. The GUI allows selection and confirmation, automating full markup in minutes (Vrečar et al., 2024).

This approach reduces the expert burden of semantic markup, enabling broad portions of mathematical corpora to benefit from enhanced linking, tree views, exporting, and accessibility without requiring granular knowledge of grammar engineering.

3. Semantic-Topological Mapping in Robotics (SENT-Map)

In robotics and autonomous manipulation, simple navigational graphs are extended by STeX principles—realized concretely as SENT-Map—by encoding semantic properties at graph nodes, making high-level queries and planning tractable for both humans and LMs:

1. Graph Formalism: The base map $\mathcal{M} = G(V,E)$ is augmented such that each $v \in V$ is accompanied by attributes $\mathcal{A}_V(v)$, encoding pose (position, orientation), type (navigational or semantic), entities (objects, furniture), and states (open, closed, empty, etc.). Semantic nodes $V_{SE} \subseteq V$ are distinguished by this metadata.
2. Data Representation (JSON Schema): All information is serialized in a human-parseable JSON format, simplifying both manual editing and FM-based interaction. Node entries include IDs, pose data, connectivity, semantic flags, entity types, interaction affordances, and per-node object lists (with states and owners).
3. Edge Attributes: Edges $e = (v_i \to v_j)$ receive cost annotations reflecting both navigation ($d(v_i,v_j)$) and manipulation effort components ($t_{\mathrm{manip}}(v_i,v_j)$).
4. Annotated Pipeline:
   • Stage I: During operator-guided mapping, the robot creates nodes and captures environmental snippets ($I_v$) at POIs. A Vision-FM (e.g., Llama 3.2 Vision Instruct) fills semantic JSON fields for each node, subject to human correction before finalization.
   • Stage II: For planning, a textual prompt to an FM includes the SENT-Map, skill API, constraints, and a natural-language query. The FM returns high-level action sequences anchored to navigable/operable nodes.

4. Planning and Reasoning over Semantic-Topological Maps

The STeX framework in robotics defines a planning model operating on the semantic-topological graph:

• State: Defined as $s = (v, \sigma)$, where $v \in V$ and $\sigma$ encodes the world state (e.g., object graspings, door status).
• Transition: $T((v,\sigma), a) = (v', \sigma')$, with $$a \in\{\mathtt{navigate}(v\to v'),\mathtt{open}(v),\mathtt{pick}(o), ...\}$$.
• Objective: Find $\mathbf{a} = (a_1, ..., a_k)$ minimizing $J(\mathbf{a}) = \sum_{t=1}^k c(s_{t-1}, a_t)$ such that the final state satisfies the user’s query $q$.
• Feasibility: Only allowed if the edge exists and semantic constraints (e.g., cabinet open state) are met.

Search (e.g., A*) finds minimal-cost paths to target semantic nodes $V_q$, with LLMs decomposing $\pi$ into skill calls for the robot controller.

5. Experimental Evaluations and Quantitative Results

Experiments with SENT-Map in indoor settings (office, kitchen, lounge) comprised nine semantic nodes and 23 FM-identified objects (Kathirvel et al., 5 Nov 2025). Task success rates strongly favored semantic enhancement:

Model	Baseline (room only)	Full SENT-Map (semantic)
All (average)	38.9%	100%
Small LLMs (27B)	38.9%	100%

Indirect queries (e.g. “I’m feeling sniffly” $\rightarrow$ tissue) failed under baseline but were successfully resolved with a full semantic SENT-Map. Even relatively small LLMs achieved perfect task completion when provided with the enriched representation. A plausible implication is that the explicit semantic context resolves ambiguities and grounds planning in executable, factual affordances.

6. Limitations, Integration Prospects, and Generalization

Identified limitations for STeX/SENT-Map include:

• Operator involvement: High annotation effort for large/complex domains or environments.
• Scaling: Large JSON maps can challenge both small LMs and manual correction workflows.
• Dynamics: Current approaches presume static or single-floor mappings; real-time update support remains an open question.

For mathematical documents, automatically generated grammars may contain cycles that parsers like parglare cannot manage, and some macros lack sufficient type or precedence data, necessitating further manual input (Vrečar et al., 2024).

Planned extensions comprise dynamic online updates, enhanced GUIs for real-time human–FM authoring, hierarchical abstractions (multi-floor/outdoor environments), and systematic integration of author-defined macros or multiple archives.

The STeX paradigm generalizes to diverse structured domains: warehouses (racks with SKU metadata), hospitals (patient and care-station info), and search-and-rescue (landmarks plus hazard markers) (Kathirvel et al., 5 Nov 2025).

7. Impact and Outlook

Semantic-Topological eXpansion unifies graph-centric symbolic reasoning with explicit semantic grounding, directly leveraging both algorithmic automation (GLR parsing, LLM planning) and human expertise in authoring, disambiguation, and correction. STeX in mathematical markup democratizes the production of high-precision, interoperable, and accessible technical documents, while SENT-Map in robotics enables robust, verifiable task planning for foundation model–driven agents.

This methodology underpins the transition from expert-only semantic tooling to scalable, author-friendly systems and closes gaps between human-intuitive, foundation model–interpretable, and formally actionable representations (Vrečar et al., 2024, Kathirvel et al., 5 Nov 2025).