LESS: Local Encoder for Spatial Sensing

• The paper introduces LESS, an object-centric tactile imaging method that replaces a single global latent with spatially organized local recurrent encoders.
• LESS leverages local receptive fields and a compositional design to achieve zero-shot generalization from single-inclusion training to complex tactile scenes.
• The system combines self-supervised force prediction with supervised MRI-based image reconstruction to generate detailed 2D/3D depictions and spatial uncertainty maps.

Searching arXiv for the LESS paper and closely related work to ground the article with current references. Searching arXiv for "Local Encoder for Spatial Sensing" and tactile imaging context. Local Encoder for Spatial Sensing (LESS) is an object-centric tactile representation and neural architecture for tactile imaging, introduced to reconstruct the internal structure of soft objects from sequential touch sensing rather than direct visual imaging (Rimon et al., 12 Jun 2026). In LESS, a tactile scene is not compressed into a single global latent vector. Instead, it is modeled as a spatially organized collection of local recurrent encoders with local receptive fields, whose latent states are fused to reconstruct $2$D or $3$D images of internal structure. This locality prior is motivated by the paper’s claim that tactile interactions are inherently local: the force measured at one point is mainly determined by nearby internal structure and is nearly independent of far-away structure. The resulting compositional design is reported to enable strong zero-shot generalization from single-inclusion training objects to objects with multiple inclusions and to larger objects with unseen coordinate extent, while also supporting spatial uncertainty estimation, hand-held tactile imaging through external pose tracking, and full volumetric reconstruction (Rimon et al., 12 Jun 2026).

1. Problem formulation and conceptual basis

LESS addresses a tactile imaging setting in which a sensor moves over a deformable object and produces a time series of force readings together with sensor poses. At each time $t$, the pose is $x_t \in \mathbb{R}^6$ and the force reading is $f_t \in \mathbb{R}^k$. The task is to infer internal structure such as the location, size, and shape of inclusions from those sparse sequential palpation measurements (Rimon et al., 12 Jun 2026).

The design is explicitly motivated by limitations of an earlier self-supervised tactile imaging method that learned a single latent vector $z_t$ from the entire sequence $\{x_0,f_0,\dots,x_t,f_t\}$. In the LESS formulation, a single global, unstructured latent vector is treated as statistically inefficient for compositional tactile scenes, because it must encode all combinations of all local material patterns. The paper argues that this leads to poor generalization when training is restricted to single-inclusion phantoms and test objects contain multiple inclusions or extend beyond the coordinate range seen during training (Rimon et al., 12 Jun 2026).

LESS therefore replaces one global latent state with a spatially indexed set of local latent states,

$$z_t = \left\{ z_t^{x_1}, \dots, z_t^{x_N} \right\},$$

where each $z_t^{x_i}$ is centered at a spatial position $x_i$. The representation is called object-centric because its latent units are tied to local physical parts of the tactile scene rather than to one entangled scene-level code. This introduces a compositional inductive bias: if the model learns what one local inclusion feels like, several inclusions can be represented by activating several local encoders rather than inventing a qualitatively new global code (Rimon et al., 12 Jun 2026).

2. Local representation and neural architecture

The tactile scene in LESS is implemented as a grid of neural “particles,” each associated with a spatial location and responsible only for measurements falling within its receptive field (Rimon et al., 12 Jun 2026). In the reported implementation, the particle centers $3$0 are placed on a uniform $3$1D grid, reflecting the fact that the phantoms vary mostly in planar inclusion position. The receptive field of particle $3$2 is

$3$3

with

$3$4

An ablation identifies $3$5 as the best tradeoff between local invariance and local information content (Rimon et al., 12 Jun 2026).

Each local encoder processes only the subsequence of pose-force pairs whose poses fall inside its receptive field. The processing sequence is described as filtering by receptive field, centering the coordinates by subtracting $3$6, encoding pose and force with a Force-Location Encoder (FLE), and updating a recurrent hidden state with a GRU. Equivalently, the local update can be written as

$3$7

$3$8

$3$9

for those $t$0 such that $t$1. Here $t$2 is sinusoidal positional encoding and $t$3 is the force encoder. The encoder weights are shared across all particle locations, so local tactile patterns encountered at different spatial coordinates are encoded similarly once expressed in local coordinates (Rimon et al., 12 Jun 2026).

The self-supervised force decoder is likewise local. To predict force at a query pose $t$4 from local latent $t$5, the query pose is centered around $t$6, embedded, combined with an MLP projection of the latent, and decoded by another MLP. The paper summarizes this as

$t$7

where $t$8 denotes the force decoder (Rimon et al., 12 Jun 2026).

For image reconstruction, LESS does not decode one global image from one scene vector. Each local latent is decoded into a local image patch, and patches are fused spatially. In $t$9D, a latent at location $x_t \in \mathbb{R}^6$0 is decoded into logits

$x_t \in \mathbb{R}^6$1

for a square patch of side length $x_t \in \mathbb{R}^6$2. The main text mentions $x_t \in \mathbb{R}^6$3 classes in experiments, corresponding to free space, hydrogel, or silicone inclusion, while the appendix describing the MRI pipeline uses $x_t \in \mathbb{R}^6$4 classes: background, insert, pillar, inclusion (Rimon et al., 12 Jun 2026). The full image is composed by summing overlapping patch logits and then applying softmax: $x_t \in \mathbb{R}^6$5 For $x_t \in \mathbb{R}^6$6D reconstruction, the same mechanism is used, except each latent decodes to a local column

$x_t \in \mathbb{R}^6$7

and overlapping volumetric logits are fused into a voxel grid (Rimon et al., 12 Jun 2026).

The paper explicitly distinguishes this design from generic ConvLSTM-style architectures: locality is enforced by a hard threshold outside each receptive field rather than by a soft convolutional receptive pattern (Rimon et al., 12 Jun 2026).

3. Learning objectives and supervision structure

LESS uses hybrid training. The local encoder-decoder is trained self-supervisedly from tactile data alone by predicting future local force outcomes, and the image or volume reconstruction module is then trained supervisedly with MRI-derived semantic labels (Rimon et al., 12 Jun 2026).

For the self-supervised stage, the loss is defined as an average over particles,

$x_t \in \mathbb{R}^6$8

The appendix gives the force reconstruction loss as an MSE objective over sampled context and target times, with $x_t \in \mathbb{R}^6$9. To make training tractable, each filtered local sequence is padded to a constant length $f_t \in \mathbb{R}^k$0, and reconstruction indices are sampled only from non-padded entries (Rimon et al., 12 Jun 2026).

The supervised imaging stage uses MRI scans acquired with a 3T Siemens Prisma MRI and a 64-channel coil, processed into voxelwise semantic labels aligned to the tactile coordinate frame (Rimon et al., 12 Jun 2026). Because the inclusion class occupies only about $f_t \in \mathbb{R}^k$1 of voxels in $f_t \in \mathbb{R}^k$2D, versus $f_t \in \mathbb{R}^k$3 in $f_t \in \mathbb{R}^k$4D, the reconstruction loss is not plain cross-entropy. The appendix gives a dynamically scaled combination of Dice loss and focal loss: $f_t \in \mathbb{R}^k$5

$f_t \in \mathbb{R}^k$6

$f_t \in \mathbb{R}^k$7

$f_t \in \mathbb{R}^k$8

The reported hyperparameters are $f_t \in \mathbb{R}^k$9, $z_t$0, $z_t$1, and $z_t$2, with $z_t$3 defined as inverse-frequency class weights within the batch (Rimon et al., 12 Jun 2026).

An important architectural implication is that the core LESS encoding mechanism does not change when moving from $z_t$4D to $z_t$5D. The change is in the decoder target and the supervised loss: each local latent continues to summarize local tactile history, but its decoder outputs a volumetric patch instead of a planar patch (Rimon et al., 12 Jun 2026).

4. Sensing pipeline, hardware, and hand-held deployment

The paper reports both robot-controlled and hand-held tactile imaging. In the robot setting, sensing uses a Franka Panda robot with an off-the-shelf Xela uSkin ALHA tactile sensor sampled at $z_t$6 Hz (Rimon et al., 12 Jun 2026). The datasets include:

• data-v1: an earlier poke dataset with vertical motions and fixed orientation.
• data-poke: a new poke dataset with varied fixed orientations.
• data-primitive: robot-executed motion primitives recorded from teleoperated, human-like palpation.
• data-handheld: manually collected hand-held data with external pose tracking.

Measurements are subsampled by $z_t$7 for data-poke and $z_t$8 for data-primitive because of its greater motion complexity. Each phantom in data-poke and data-primitive is scanned with $z_t$9 trajectories, and each phantom in data-handheld with $\{x_0,f_0,\dots,x_t,f_t\}$0. For all datasets, trajectories are concatenated as a single input sequence to LESS (Rimon et al., 12 Jun 2026).

A substantial practical contribution is the hand-held pose-estimation pipeline. Because robot kinematics are unavailable in that setting, the paper uses fiducial-based tracking with two AprilTags, one mounted on the hand-held sensor rig and one fixed near the phantom, observed by a calibrated Intel RealSense D415 camera. If $\{x_0,f_0,\dots,x_t,f_t\}$1 maps frame $\{x_0,f_0,\dots,x_t,f_t\}$2 to frame $\{x_0,f_0,\dots,x_t,f_t\}$3, then the sensor pose in the global tag frame is

$\{x_0,f_0,\dots,x_t,f_t\}$4

The system is further calibrated to the robot base using a Kabsch alignment satisfying

$\{x_0,f_0,\dots,x_t,f_t\}$5

where $\{x_0,f_0,\dots,x_t,f_t\}$6 and $\{x_0,f_0,\dots,x_t,f_t\}$7 come from AprilTags, $\{x_0,f_0,\dots,x_t,f_t\}$8 from CAD, and $\{x_0,f_0,\dots,x_t,f_t\}$9 from robot kinematics (Rimon et al., 12 Jun 2026).

The reported pose-tracking configuration uses two $$z_t = \left\{ z_t^{x_1}, \dots, z_t^{x_N} \right\},$$0 AprilTags and a RealSense D415 at $$z_t = \left\{ z_t^{x_1}, \dots, z_t^{x_N} \right\},$$1 FPS, $$z_t = \left\{ z_t^{x_1}, \dots, z_t^{x_N} \right\},$$2 ms exposure, gain $$z_t = \left\{ z_t^{x_1}, \dots, z_t^{x_N} \right\},$$3, left IR imager, emitter disabled. Empirical pose noise is reported as $$z_t = \left\{ z_t^{x_1}, \dots, z_t^{x_N} \right\},$$4 mm and $$z_t = \left\{ z_t^{x_1}, \dots, z_t^{x_N} \right\},$$5 rad, and the paper states that end-to-end F1 degradation is negligible when using estimated rather than ground-truth robot pose on a sample (Rimon et al., 12 Jun 2026).

The deployed system runs at about $$z_t = \left\{ z_t^{x_1}, \dots, z_t^{x_N} \right\},$$6 Hz, with model forward pass around $$z_t = \left\{ z_t^{x_1}, \dots, z_t^{x_N} \right\},$$7 ms (Rimon et al., 12 Jun 2026). This is not framed as a general real-time theorem, but it establishes online operation during sequential palpation.

5. Empirical behavior and generalization

The central empirical claim for LESS is zero-shot compositional generalization. The main training objects contain only single inclusions, but evaluation includes two out-of-distribution settings: a multiple set with multiple inclusions, including connected shapes, and a large set with the same height but base area $$z_t = \left\{ z_t^{x_1}, \dots, z_t^{x_N} \right\},$$8 larger than regular phantoms (Rimon et al., 12 Jun 2026).

The comparison against the global baseline is asymmetric. In-distribution, the global model is slightly better on single-inclusion F1, which the paper attributes to the information bottleneck imposed by locality. Out-of-distribution, LESS is markedly stronger because local particles can represent several inclusion-like neighborhoods independently and because local coordinate centering reduces dependence on the absolute coordinate range seen during training (Rimon et al., 12 Jun 2026).

Test setting	GLOBAL	LESS
Single inclusion	F1 $$z_t = \left\{ z_t^{x_1}, \dots, z_t^{x_N} \right\},$$9; diameter error $z_t^{x_i}$0 mm; CoM error $z_t^{x_i}$1 mm	F1 $z_t^{x_i}$2; diameter error $z_t^{x_i}$3 mm; CoM error $z_t^{x_i}$4 mm
Multiple inclusions	F1 $z_t^{x_i}$5; CoM error $z_t^{x_i}$6 mm; area error $z_t^{x_i}$7 mm$z_t^{x_i}$8	F1 $z_t^{x_i}$9; CoM error $x_i$0 mm; area error $x_i$1 mm$x_i$2
Large phantoms	F1 $x_i$3; CoM error $x_i$4 mm; diameter error $x_i$5 mm	F1 $x_i$6; CoM error $x_i$7 mm; diameter error $x_i$8 mm

The ability to extend the particle grid at test time is central to the large-object result. Because particles share weights, the paper states that one may add an arbitrary number of particles at test time so long as spatial resolution remains similar (Rimon et al., 12 Jun 2026). The receptive-field ablation reinforces this interpretation: increasing $x_i$9 improves single-inclusion performance and approaches the global baseline, but degrades multiple-inclusion performance by weakening local invariance. The chosen value $3$00 mm is therefore used throughout the experiments (Rimon et al., 12 Jun 2026).

The extension to $3$01D also produces a notable supervision effect. When the prior global baseline is trained with $3$02D labels instead of $3$03D, its $3$04D slice metrics improve from F1 $3$05 to $3$06, diameter error from $3$07 mm to $3$08 mm, and CoM error from $3$09 mm to $3$10 mm. The authors interpret this as evidence that volumetric supervision provides a stronger geometric prior (Rimon et al., 12 Jun 2026).

The paper also reports large-scale data collection: roughly $3$11 hours of tactile data, over $3$12 the size of the prior data-v1 dataset, using $3$13 motorized scissor lifts and a Franka Panda operating nearly continuously for about $3$14 days. Data were collected from $3$15 inserts, $3$16 shells, and $3$17 Xela sensors, with random yaw angles from $3$18 to $3$19 in the automatic system. The hand-held dataset contains $3$20 full scans and about $3$21 single trajectories (Rimon et al., 12 Jun 2026).

On pose-noise robustness, the paper injects Gaussian perturbations

$3$22

with

$3$23

and reports stability up to about $3$24 mm position error and $3$25 rad orientation error (Rimon et al., 12 Jun 2026).

Distribution shift between robot and human palpation is substantial. When testing on data-primitive, training on poke alone yields F1 $3$26, training on primitive-only gives $3$27, and training on primitive + poke reaches $3$28. On real hand-held data, adding primitive training improves F1 from $3$29 to $3$30, diameter error from $3$31 mm to $3$32 mm, and CoM error from $3$33 mm to $3$34 mm (Rimon et al., 12 Jun 2026). The result is a proof of concept rather than parity with robot-held sensing.

6. Uncertainty, limitations, and broader significance

LESS supports spatial uncertainty estimation through predictive entropy rather than through an explicit Bayesian or ensemble formulation. For each pixel,

$3$35

The paper’s interpretation is structural: because each pixel is influenced only by nearby local representations, its uncertainty depends only on measurements in its vicinity. Untouched or under-sampled regions remain high-entropy, whereas well-palpated areas become confident (Rimon et al., 12 Jun 2026). In qualitative comparison, the global baseline is reported to become overconfident too early, including in unobserved regions, because its global latent strongly encodes dataset-level priors such as “outside the insert is background” (Rimon et al., 12 Jun 2026).

Several limitations are explicit. The system is trained and validated on synthetic phantoms rather than human tissue. The paper states that clinical relevance remains an open question and that future work must test whether a sufficiently large synthetic dataset can transfer to real anatomy (Rimon et al., 12 Jun 2026). LESS also incurs a small in-distribution accuracy cost relative to the global model, reflecting the tradeoff induced by locality. Hand-held performance remains substantially worse than robot-held performance, especially in center-of-mass error. The uncertainty estimate is useful but not a principled Bayesian posterior. The present implementation uses a $3$36D particle grid and mostly planar locality assumptions, which may need extension for more general anatomy or manipulation tasks (Rimon et al., 12 Jun 2026).

In a broader representation-learning context, LESS belongs to a family of architectures that allocate explicit local processing instead of relying on a single global representation. Cross-domain examples include prior-guided local encoding in multi-organ segmentation (Tian et al., 30 Oct 2025), parallel local-global context aggregation through depthwise convolution and efficient attention in remote-sensing change detection (Noman et al., 2024), and lightweight local CNN branches added to Transformer features in multi-source remote sensing classification (Lin et al., 2023). LESS differs in task and modality, but it shares the same core architectural decision: locality is treated as a representational primitive rather than as an emergent by-product of a global encoder.

The distinctive contribution of LESS is that this localism is tied directly to tactile physics. The model assumes that nearby internal structure dominates local force response, organizes latent state accordingly, trains those local states self-supervisedly through force prediction, and then decodes them into anatomical image or volume patches. That combination of locality, recurrence, spatial organization, and patchwise reconstruction is what gives LESS its reported advantages in multiple-inclusion generalization, coordinate-range extrapolation, and spatially meaningful uncertainty (Rimon et al., 12 Jun 2026).