---
title: 'Sensor2Sensor: Cross-Sensor Interoperability'
url: https://www.emergentmind.com/topics/sensor2sensor
type: topic
---

# Sensor2Sensor: Cross-Sensor Interoperability

Sensor2Sensor denotes a family of methods for transferring information, geometry, or task competence between sensing systems that do not share the same raw observation space. Across the cited literature, the term covers at least three recurring operations: estimating rigid inter-sensor transforms for fusion, adapting one sensor’s measurements or features so that models trained on another sensor remain usable, and generating the observations of one sensing embodiment from another [1812.11445][2502.19638][2605.22809]. In robotics, tactile sensing, collaborative edge systems, and autonomous driving, the common objective is to replace sensor-specific pipelines with a transferable interface—geometric, statistical, representational, or generative—between heterogeneous devices.

## 1. Meanings and scope of Sensor2Sensor

Sensor2Sensor is not a single algorithmic framework. In the cited work, it appears as a broad design problem whose exact formulation depends on what is being transferred: coordinate frames, measurement values, latent representations, learned features, or full synthetic sensor logs. This diversity is already visible in the contrast between extrinsic calibration from egomotion [1809.03554], universal registration from targets of opportunity [1807.05072], runtime device-to-device translation for black-box sensing models [2012.06035], sensor-invariant tactile embeddings [2502.19638], sparse-voxel domain-robust collective perception [2504.17399], and cross-embodiment generation of multi-view cameras plus LiDAR from monocular dashcam video [2605.22809].

A useful reading is that Sensor2Sensor methods try to factor sensor-specific nuisance variables away from the task-relevant structure. In some settings this nuisance is purely geometric, as in rigid extrinsics; in others it is statistical, as in affine calibration transfer or CycleGAN-style device translation; and in others it is embodied in the full sensing physics, as in visuo-tactile transfer or monocular-to-AV-log generation.

| Problem class | Core transferred quantity | Representative papers |
|---|---|---|
| Geometric calibration | \(SE(3)\) transform or common frame | [1809.03554], [1807.05072], [2210.10365] |
| Measurement adaptation | Affine map, translated signal, common interface | [2309.11526], [2012.06035], [2107.10258] |
| Representation or domain transfer | Shared latent, sparse occupancy, generated target observations | [2502.19638], [2504.17399], [2510.09817], [2605.22809] |
| Indirect matching or cooperation | Shared world reference, causal traffic linkage, centralized fusion | [2208.06233], [2005.03068], [2606.29926] |

A common misconception is that Sensor2Sensor always means direct pairwise calibration. Several of the cited systems are explicitly indirect: sensor-to-pattern optimization recovers pairwise extrinsics only by composition through a common world frame [2210.10365], geomagnetic-inertial matching aligns sensors through a north-referenced world coordinate system [2208.06233], and distributed ISAC relies on centralized fusion at the gNB rather than direct sensor-to-sensor links [2606.29926].

## 2. Geometric Sensor2Sensor calibration and registration

In calibration-oriented work, Sensor2Sensor typically means recovering the rigid transform between sensor frames so that measurements can be fused in a common geometry. The canonical motion-based relation is the hand-eye equation
\[
\Theta V_{a,t} = V_{b,t}\Theta,
\]
where \(\Theta = T_{b,a}\in SE(3)\) is the unknown extrinsic transform and \(V_{a,t},V_{b,t}\) are paired egomotion increments [1809.03554]. "Certifiably Globally Optimal Extrinsic Calibration from Per-Sensor Egomotion" formulates this as a QCQP over \(R\in SO(3)\) and \(t\in\mathbb{R}^3\), eliminates translation in closed form, and solves the Lagrangian dual SDP. Its central practical claim is that the convex relaxation is often tight, yielding a certifiable global optimum without initialization, with solve times generally under half a second in simulation and "less than a second" in the abstract [1809.03554].

For multi-sensor target tracking, "On Universal Sensor Registration" treats sensor-to-sensor registration as alignment of target observations after sensor-specific rotations are applied. Its global objective is
\[
\min_{A_1,\ldots,A_S}\sum_{s,t}\sum_{i=1}^n
\left\|A_s p_s^i+\ell_s-\big(A_t p_t^i+\ell_t\big)\right\|^2
\]
subject to the \(A_s\) being rotation matrices [1807.05072]. The method estimates angular misalignment biases only, assumes known sensor locations, and avoids target dynamical models entirely by using synchronized targets of opportunity. For \(S\ge 3\) sensors, the paper reports that in practical nondegenerate settings unambiguous absolute calibration becomes possible [1807.05072].

The review "Calibration of Heterogeneous Sensor Systems" organizes geometric Sensor2Sensor methods into target-based, targetless feature-based, and motion-based families [1812.11445]. The survey’s central point is that extrinsic calibration has a common geometric form across sensor types even when intrinsic calibration does not, but correspondence registration becomes difficult when sensors measure different physical quantities. Target-based methods are generally more precise; motion-based methods are more practical and are the most common approach in online calibration; targetless feature methods occupy the middle ground when shared environmental structure exists [1812.11445]. This establishes a recurrent trade-off between accuracy, practicality, and online adaptability.

Large fixed sensor networks motivate indirect formulations. "A sensor-to-pattern calibration framework for multi-modal industrial collaborative cells" avoids pairwise calibration and instead optimizes each sensor pose relative to a common world together with the pattern pose for each collection:
\[
\arg\min_{\{{}^{s_i}T_w\},\{{}^{p}T_w\}}
\sum_{s_i\in S}\sum_{c\in C} e\!\left({}^{s_i}T_w,{}^{p}T_w,d\right).
\]
Sensor-to-sensor transforms are then recovered by composition through the world frame [2210.10365]. This is specifically advantageous when overlap is sparse or nonexistent between many sensor pairs.

Targetless joint calibration of a camera-LiDAR-radar trio extends the same idea to multimodal online recalibration. "Physics and semantic informed multi-sensor calibration via optimization theory and self-supervised learning" defines pairwise \(6\)-DoF transforms for camera-LiDAR, camera-radar, and LiDAR-radar, and imposes a closed-loop consistency constraint
\[
T_{\text{camera}}^{\text{lidar}}\,T_{\text{lidar}}^{\text{radar}}\,T_{\text{radar}}^{\text{camera}} = I_{4\times 4}.
\]
It then solves either a direct optimization problem
\[
\min L\big(L_P(T,\phi),L_G(T)\big)
\]
or a self-supervised learning analogue driven by semantic and physics-informed losses [2206.02856]. This suggests that modern Sensor2Sensor calibration is increasingly joint and structured, rather than reducible to isolated pairwise registration.

A more indirect geometric route appears in "Dynamic Sensor Matching based on Geomagnetic Inertial Navigation", where sensors are placed into a common world coordinate system referenced to the earth’s magnetic field [2208.06233]. Pairwise Sensor2Sensor alignment becomes a byproduct of world-frame localization:
\[
\mathbf T_{s_i\to s_j} = \mathbf T_{s_j\to w}^{-1}\mathbf T_{s_i\to w}.
\]
The paper explicitly presents this as a replacement for unreliable direct pairwise visual matching for initialization, although it also states that magnetic measurements are unsuitable for permanent dynamic position determination because of susceptibility to external influences [2208.06233].

## 3. Measurement-space adaptation and runtime translation

A second major meaning of Sensor2Sensor is calibration transfer in measurement space rather than geometry. "Likelihood-based Sensor Calibration using Affine Transformation" studies the case of two sensors of identical design measuring the same latent phenomenon, with source and target outputs related by
\[
T(\mathbf{x}) = \mathbf{A}\mathbf{x} + \mathbf{b}.
\]
Using latent variables \(\theta_i\) and Gaussian noise in both sensor domains, the paper derives a likelihood-based errors-in-variables formulation and a simple closed-form estimator
\[
\mathbf{B}=\mathbf{Y}\mathbf{X}^T(\mathbf{X}\mathbf{X}^T)^{-1}
\]
for the affine map in augmented coordinates [2309.11526]. On a real board with \(8\) identical Bosch BME688 sensors, the proposed affine transfer substantially outperforms feature-wise normalization [2309.11526]. In this line of work, Sensor2Sensor means software calibration transfer between nominally identical devices.

At system level, "SensiX: A Platform for Collaborative Machine Learning on the Edge" defines Sensor2Sensor as runtime device-to-device data translation so that a model trained for one device can execute on another without per-device model engineering [2012.06035]. SensiX sits between sensor streams and pre-trained models, learns pairwise mappings from each new device to the training distribution of a target model using CycleGAN with unlabelled, unpaired data, and then chooses among candidate execution paths using a quality-aware selection operator based on the margin between the top-1 and top-2 predicted class probabilities [2012.06035]. Its evaluation reports a \(7\)–\(13\%\) increase in overall accuracy and up to \(30\%\) increase across different environment dynamics at the expense of \(3\) mW power overhead [2012.06035]. Here Sensor2Sensor is operational rather than geometric: it is a runtime substitution mechanism under device variability and availability changes.

A hardware-level abstraction appears in "Multiple Sensor Interface by the same hardware to USB and serial connection", which reduces a variety of two-wire sensors to a common electrical front-end and host protocol [2107.10258]. The device supports sensors based on change of inductance, resistance, capacitance, and frequency using the same connector and same electronic interface circuit between the sensor and the microcontroller, with additional ADC channels for small voltage measurement [2107.10258]. The board uses a Schmitt-trigger-based oscillator/interface, EEPROM calibration tables of \((\text{raw\_value},\text{measurement})\), and host links over USB, RS-485, GPIO, and serial. This is Sensor2Sensor interoperability by interface unification rather than by learned translation.

Taken together, these methods show that measurement-space Sensor2Sensor transfer can be solved at multiple levels: analytically by affine estimation, generatively by unpaired domain translation, or electrically by constraining heterogeneous sensors to a common transduction and calibration pipeline.

## 4. Representation-invariant and generative Sensor2Sensor transfer

Recent work increasingly treats Sensor2Sensor as a representation-learning or generation problem. In tactile sensing, "Sensor-Invariant Tactile Representation" introduces Sensor-Invariant Tactile Representations (SITR), a transformer-based encoder trained on a diverse dataset of simulated optical tactile sensor designs so that the same physical contact maps to a shared latent regardless of which optical tactile sensor produced the image [2502.19638]. The stated result is zero-shot transfer across optical tactile sensors with only minimal calibration for a new real sensor [2502.19638]. The key design principle is that the latent should preserve contact geometry and physically meaningful interaction structure while suppressing lighting color, marker texture, viewpoint, membrane shape, and manufacturing idiosyncrasies.

A different strategy is explicit sensor-domain generation. "Cross-Sensor Touch Generation" asks for the tactile observation that one sensor would have produced given an observation from another sensor under the same contact [2510.09817]. It proposes a paired end-to-end diffusion model, Touch2Touch, and an unpaired depth-mediated pipeline, T2D2: Touch-to-Depth-to-Touch. The geometric core of T2D2 is the rigid transfer of predicted source depth into the target sensor frame,
\[
T_{S\rightarrow T}=T_{A\rightarrow T}\circ T_{S\rightarrow A},
\]
followed by projection and target-specific tactile generation [2510.09817]. The paper reports that Touch2Touch preserves fine-grained structure better, while T2D2 is more scalable because it does not require paired source-target tactile observations [2510.09817]. This establishes a recurrent Sensor2Sensor trade-off between fidelity and data efficiency.

For LiDAR-based vehicle-to-vehicle collective perception, "S2S-Net: Addressing the Domain Gap of Heterogeneous Sensor Systems in LiDAR-Based Collective Perception" treats Sensor2Sensor as robustness to unseen cooperative LiDAR domains [2504.17399]. Instead of transmitting raw point clouds or learned features, S2S-Net exchanges sparse voxel coordinates and fuses local and collective streams with a scatter operator that takes the voxel union and applies element-wise max on duplicates. On SCOPE, a model trained with HDL64 cooperative data still achieves Car AP \(52.89\) when tested with VLP32 cooperative data, compared with \(50.04\) in-domain, indicating that representation choice can mitigate domain shift without explicit adversarial alignment [2504.17399].

The most expansive generative formulation appears in "Sensor2Sensor: Cross-Embodiment Sensor Conversion for Autonomous Driving" [2605.22809]. The framework converts monocular in-the-wild driving video into a target AV log containing \(8\) synchronized surround-view camera streams and \(1\) top-mounted LiDAR scan stream. Because paired real dashcam-to-AV-log data do not exist at scale, the paper reconstructs real AV scenes with 4D Gaussian Splatting, renders dashcam-style virtual views, and trains a conditional latent diffusion model to map those monocular inputs back into the original real AV sensor suite [2605.22809]. The model uses multi-view attention, cross-sensor attention between image and LiDAR branches, and autoregressive temporal rollout with DAgger fine-tuning. On the paired benchmark, the full method achieves FID \(6.47\) for image generation and LiDAR Chamfer Distance \(8.68\), improving over the adapted X-Drive baseline’s \(10.02\) Chamfer Distance [2605.22809].

These papers suggest a sharp conceptual split inside modern Sensor2Sensor transfer. One line seeks sensor-invariant latent spaces so downstream models operate on shared features rather than raw signals [2502.19638][2504.17399]. Another line preserves sensor-specific downstream pipelines by generating the raw or near-raw target-domain observation itself [2510.09817][2605.22809]. Both strategies aim at portability, but they intervene at different stages of the sensing stack.

## 5. Indirect sensing, shared references, and network-mediated cooperation

Sensor2Sensor can also mean using one sensor or sensor network to infer the state, activity, or location of another sensing device. "I Always Feel Like Somebody's Sensing Me! A Framework to Detect, Identify, and Localize Clandestine Wireless Sensors" uses a trusted sensor carried by the user—typically phone IMU or phone microphone—as a ground-truth sensor to test whether an unknown wireless device is sensing the same physical event [2005.03068]. For continuous streams such as IP cameras, the method uses Granger-style predictive testing:
\[
y_t=a_0+a_1y_{t-1}+\dots+a_n y_{t-n}
\]
versus
\[
y_t=a_0+a_1y_{t-1}+\dots+a_n y_{t-n}+b_1x_{t-1}+\dots+b_m x_{t-m},
\]
where the hidden device’s traffic time series is \(Y\) and the trusted sensor time series is \(X\) [2005.03068]. The paper reports detection of causality for snooping devices \(95.2\%\) of the time and localization to a sufficiently reduced sub-space [2005.03068]. In this setting, Sensor2Sensor is neither calibration nor domain transfer, but cross-sensor inference through a shared environmental cause.

"Multi-Sensor Integrated Sensing and Communication for Critical Infrastructure Protection" presents another indirect form: several passive sniffing sensors cooperate through a common cellular illuminator and centralized fusion at the gNB rather than through direct peer-to-peer exchange [2606.29926]. Each sniffer estimates multistatic range-Doppler parameters under the Cooperative Passive Coherent Location principle, using the direct BS-to-sniffer path as a reference and uplinking target-related parameters for fusion [2606.29926]. The paper emphasizes that this is not true direct sensor-to-sensor collaboration; it is sensor-to-BS-to-sensor coordination with shared timing, shared waveform, and centralized fusion. From a Sensor2Sensor standpoint, the important point is that common reference structures can substitute for direct inter-sensor links.

This broader perspective clarifies that Sensor2Sensor interaction need not imply symmetric exchange between two sensors. It may be asymmetric, mediated by communication infrastructure, or even based on the fact that two devices are coupled to the same external event.

## 6. Trade-offs, limitations, and unresolved directions

Several stable trade-offs recur across the literature. The first is between directness and scalability. Pairwise calibration and paired translation often give the highest fidelity, but they scale poorly as the number of sensors grows. This is explicit in sensor-to-pattern calibration for collaborative cells [2210.10365], in Touch2Touch versus T2D2 for tactile transfer [2510.09817], and in SensiX’s avoidance of training every model for every device combination [2012.06035].

The second is between accuracy and operational practicality. The review of heterogeneous calibration states that target-based methods are generally more precise, while motion-based methods are more practical and are the most common basis for online recalibration [1812.11445]. This pattern reappears in geomagnetic-inertial matching, where the earth’s magnetic field can provide a shared frame for initialization, but magnetic disturbances and inertial drift limit permanent high-accuracy use [2208.06233].

The third is observability. Motion-based extrinsic calibration from egomotion has a strictly convex cost only if the measurement data contain rotations about at least two unique axes [1809.03554]. The broader review extends this theme to other sensor pairs, emphasizing that poor excitation, synchronization errors, and weak shared structure can make Sensor2Sensor parameters unidentifiable [1812.11445]. A plausible implication is that progress in optimization alone cannot compensate for fundamentally uninformative data.

The fourth is coverage of the sensor family. SITR explicitly targets optical tactile sensors rather than all tactile modalities [2502.19638]. S2S-Net shows strong robustness across HDL64, VLP32, and Blickfeld Cube LiDARs, but the largest degradation is associated with FoV mismatch rather than a purely statistical domain gap [2504.17399]. Cross-embodiment AV conversion can generate realistic short sensor rollouts, but the paper explicitly states that temporal drift remains for long sequences, especially beyond \(30\) seconds [2605.22809]. These limitations indicate that zero-shot Sensor2Sensor transfer is usually family-bounded rather than universal.

Finally, there is a persistent ambiguity about the appropriate transfer object. Some papers argue for a shared latent representation [2502.19638]; others preserve target-sensor pipelines by generating target observations directly [2510.09817][2605.22809]; others avoid raw signal transfer and standardize on sparse occupancy or affine calibration [2504.17399][2309.11526]. This suggests that Sensor2Sensor is less a single methodology than a design space organized around what must be preserved: geometry, task performance, physical realism, hardware compatibility, or runtime substitutability.

In that sense, Sensor2Sensor is best understood as a unifying label for cross-sensor interoperability problems. The literature shows that interoperability can be achieved by certifiable geometry, by statistical calibration transfer, by domain-robust shared representations, by explicit generative translation, by common electrical interfaces, or by centralized cooperative references. What unites these approaches is not their mechanism, but their objective: to make sensor identity less of a bottleneck for perception, learning, and deployment.

Source: https://www.emergentmind.com/topics/sensor2sensor