Lighting-driven Dynamic Active Sensing

• Lighting-driven Dynamic Active Sensing (LiDAS) is an active optical system that dynamically controls illumination placement to improve scene perception under a fixed power budget.
• It employs closed-loop feedback by integrating high-definition headlights with visual perception models to predict and reallocate optimal illumination in real time.
• LiDAS extends across various applications—from nighttime driving to non-line-of-sight localization—demonstrating gains in detection, segmentation, and energy savings.

Searching arXiv for the specified LiDAS and closely related active-sensing papers to ground the article. Searching arXiv for "Lighting-driven Dynamic Active Sensing". Lighting-driven Dynamic Active Sensing (LiDAS) denotes a class of active optical sensing systems in which illumination is treated as a controllable sensing action rather than as a fixed imaging condition. In its explicit nighttime-perception formulation, LiDAS is a closed-loop active illumination system that combines off-the-shelf visual perception models with high-definition headlights, dynamically predicts an optimal illumination field for the next frame, and reallocates fixed lighting power away from empty or already well-lit regions toward perceptually informative regions (Moreau et al., 9 Dec 2025). In a broader research lineage, closely related systems had already treated projector patterns, scan density, light-curtain placement, or onboard light intensity as task-driven control variables for estimation, localization, detection, or SLAM, even when they were not yet named LiDAS (Chandran et al., 2019, Turkar et al., 13 Feb 2026).

1. Definition and conceptual lineage

The defining claim of LiDAS is that optical emission itself belongs to the estimator. Rather than accepting scene lighting passively, the sensing system selects where light should be placed, how much power should be assigned, and, in some systems, how that decision should evolve across time. In the 2025 nighttime-perception formulation, this principle is expressed as closed-loop active illumination with equal-power reallocation by high-definition headlights, using the current image, the previous illumination map, and image coordinates to predict the next illumination field (Moreau et al., 9 Dec 2025).

A precursor appears in steady-state non-line-of-sight perception. “Adaptive Lighting for Data-Driven Non-Line-of-Sight 3D Localization and Object Identification” treats the illumination pattern over visible line-of-sight surface patches as the control variable and chooses the patch or patches with the highest predicted NLOS radiosity, optionally under total-power and per-patch-power constraints (Chandran et al., 2019). That system is not closed-loop in the sequential Bayesian sense; it precomputes geometry-aware patterns and cycles through them at inference. Even so, it already establishes a central LiDAS proposition: hidden-scene observability can be improved by optimizing where light is placed.

This suggests a broader interpretation of LiDAS as a family of systems in which active optical power, spatial allocation, or scan density is selected according to downstream task utility. Under that interpretation, adaptive projector-camera NLOS sensing, event-guided structured light, light curtains, adaptive coherent LiDAR, robot illumination control, and integrated visible-light sensing-communication-illumination all occupy positions on the same continuum, with different degrees of closed-loop adaptivity and different optical substrates (Muglikar et al., 2021, Ancha et al., 2020, Chen et al., 2024, Xie et al., 26 Nov 2025).

2. Control formulations and sensing objectives

The closed-loop LiDAS nighttime framework represents illumination as an image-space field

$M_t \in [0,1]^{H \times W},$

predicts a residual correction

$\Delta M_t \in [-1,1]^{H\times W},$

and updates illumination by

$$\widetilde{M}_t = \min\!\big(\max(M_{t-1} + \Delta M_t,\, 0),\, 1\big).$$

Equal-power reallocation is then imposed by

$$M_t = \frac{\eta}{\max(\epsilon,\;\bar{m}_t)}\, \widetilde{M}_t,$$

where $\eta$ is the target energy budget and $\bar m_t$ is the mean intensity of $\widetilde M_t$. This normalization is the formal mechanism by which light is removed from uninformative regions and concentrated on regions that matter for downstream perception under fixed power (Moreau et al., 9 Dec 2025).

Training is task-driven rather than photometrically driven. The relit image is passed to frozen downstream heads, and the lighting model is optimized through the weighted sum of downstream task losses from YOLO11L, YOLOv8L, YOLOv8L-Worldv2, and Mask2Former. Deployment can then use arbitrary downstream heads, and the paper reports transfer to unseen downstream models such as YOLOv5L, YOLO12L, and SegFormer without retraining LiDAS (Moreau et al., 9 Dec 2025). The system is therefore designed to improve the image presented to a frozen perception stack rather than to retrain that stack.

Earlier LiDAS-style work uses more explicit physics-based optimization. In the NLOS adaptive-lighting system, the visible scene is divided into $N$ patches, the base radiosity model is

$$B_i = E_i + \rho_i \sum_{j=1}^{N} F_{ij} B_j, \qquad F_{ij} = \left(\frac{\cos \theta_i \cos \theta_j}{\pi r^2}\right) V(x_i, x_j) A_i,$$

and the illumination-patch selection problem is

$$\argmax_{[1, \dots, m] \subseteq [1,\dots,N]} \sum_{i=1}^{m} B_i^{NLOS}.$$

Under a power budget, the intended control problem becomes

$\Delta M_t \in [-1,1]^{H\times W},$0

This makes the sensing variable explicit twice over: first in the choice of illuminated patches, and second in the allocation of radiant exitance across those patches (Chandran et al., 2019).

A related robot-perception framework, Lightning, uses a different control variable—discrete intensity rather than a full image-space illumination field—but the same LiDAS logic. Its offline Optimal Intensity Schedule minimizes

$\Delta M_t \in [-1,1]^{H\times W},$1

with

$\Delta M_t \in [-1,1]^{H\times W},$2

and

$\Delta M_t \in [-1,1]^{H\times W},$3

Here the objective is not brightness, but sequence-level SLAM utility balanced against power consumption and temporal smoothness (Turkar et al., 13 Feb 2026).

Indoor visible-light systems provide another optimization form. In the activity area, LED transmit powers solve a linear program minimizing total transmit power under illumination and SNR constraints; in the non-activity area, the system solves a convex quadratic program minimizing SNR variance under illuminance and power constraints. The runtime switch between these objectives is triggered by user localization from NLOS optical sensing (Xie et al., 26 Nov 2025).

3. Representative architectures and controlled sensing variables

Only one system is explicitly titled LiDAS, but several representative systems can be read as LiDAS-style because each treats active optical allocation as part of inference rather than as a fixed acquisition condition.

Representative system	Controlled optical variable	Reported task
“Adaptive Lighting for Data-Driven Non-Line-of-Sight 3D Localization and Object Identification” (Chandran et al., 2019)	illumination patch set and power allocation over LOS patches	NLOS 3D localization and object identification
“Event Guided Depth Sensing” (Muglikar et al., 2021)	dense versus sparse laser scanning over event-defined ROIs	event-based structured-light depth sensing
“Active Perception using Light Curtains for Autonomous Driving” (Ancha et al., 2020)	light-curtain surface placement	sequential 3D object recognition
“Integrated adaptive coherent LiDAR for 4D bionic vision” (Chen et al., 2024)	ROI location by center wavelength and imaging granularity by comb spacing	adaptive gaze and dynamic zoom-in FMCW LiDAR
“Adaptive Illumination Control for Robot Perception” (Turkar et al., 13 Feb 2026)	co-located onboard light intensity schedule	visual SLAM
“Adaptive Lighting Control in Visible Light Systems: An Integrated Sensing, Communication, and Illumination Framework” (Xie et al., 26 Nov 2025)	LED transmit powers	integrated sensing, communication, and illumination

The nighttime LiDAS system is architecturally minimal in sensing hardware and explicit in control loop structure. It sits between a front-facing camera and a high-definition headlight, takes the current RGB frame, the previous illumination map, and CoordConv-style coordinates as input, and outputs the next image-space illumination field. Training uses a differentiable relighting operator built from a fully illuminated image and a no-headlight image; deployment requires only the LiDAS policy plus a precomputed warp from camera to headlight pixels (Moreau et al., 9 Dec 2025).

The NLOS adaptive-lighting system is architecturally different but conceptually aligned. It uses a conventional projector and a standard camera, assumes known line-of-sight geometry, and computes a physics-guided bank of illumination patterns for candidate hidden voxels using radiosity, diffuse transport, and third-bounce path approximations. The CNN is trained on measurements already tailored to each voxel hypothesis, and inference cycles through the precomputed patterns (Chandran et al., 2019).

Event-guided structured light replaces global rastering by motion-aware spatial allocation. One event camera identifies active image regions, and a laser point projector scans densely inside those ROIs and sparsely elsewhere. Light curtains make the control surface itself the action: a deep 3D detector produces anchor uncertainty, and the next light curtain is placed by maximizing information gain under galvanometer constraints (Muglikar et al., 2021, Ancha et al., 2020).

Adaptive coherent LiDAR and metasurface-enhanced LiDAR occupy the hardware end of the LiDAS spectrum. The former supports dynamic gazing by center-wavelength tuning and dynamic zoom-in by elastic channel spacing; the latter combines acousto-optic deflectors and metasurfaces to create simultaneous peripheral and foveal sensing zones (Chen et al., 2024, Martins et al., 2022). These systems do not yet demonstrate the full task-loss-driven closed loop of nighttime LiDAS, but they show that nonuniform active optical allocation can be embedded directly in sensing hardware.

4. Empirical behavior and performance across domains

In nighttime driving, LiDAS is reported to improve frozen daytime-trained perception models without retraining those models. In synthetic testing at the same power as low beam, LiDAS$\Delta M_t \in [-1,1]^{H\times W},$4 improves detection from 36.9 to 47.3 mAP$\Delta M_t \in [-1,1]^{H\times W},$5 and segmentation from 66.0 to 72.8 mIoU. In real closed-loop deployment on unlit urban and rural roads, Low Beam$\Delta M_t \in [-1,1]^{H\times W},$6 gives 15.8 mAP$\Delta M_t \in [-1,1]^{H\times W},$7 and 10.3 mIoU, while LiDAS$\Delta M_t \in [-1,1]^{H\times W},$8 reaches 34.5 mAP$\Delta M_t \in [-1,1]^{H\times W},$9 and 15.3 mIoU, corresponding to +18.7 mAP$$\widetilde{M}_t = \min\!\big(\max(M_{t-1} + \Delta M_t,\, 0),\, 1\big).$$0 and +5.0 mIoU over standard low beam at equal power. The same paper also reports that LiDAS maintains performances while reducing energy use by 40% (Moreau et al., 9 Dec 2025).

The NLOS precursor demonstrates the value of physics-guided illumination selection even without transient hardware. On real data from a conventional projector-camera prototype, it achieves average object identification of 87.1% for four classes and centroid localization mean-squared error of 1.97 cm in the occluded region. In simulation, for single-patch illumination, the adaptive optimal patch achieves average localization MSE 2.09 cm across four walls and four object classes, versus 3.60 cm for the second-best patch and 5.08 cm for the third-best patch; identification averages 91.02% for the adaptive optimal patch versus 82.28% for the next-best patch (Chandran et al., 2019).

Event-guided depth sensing quantifies a different LiDAS advantage: the area that truly needs dense active illumination is often small. In 116 EventScape autonomous-driving sequences, the average fraction of active pixels is 7.1% of image resolution, and in real indoor experiments active-pixel percentages range from 0.73% to 10.34%. The paper’s headline interpretation is that moving edges correspond to less than 10% of the scene on average, implying nearly 90% less illumination-source power if dense scanning is restricted to that fraction. On a planar wall, dense scanning yields a plane-fitting reconstruction error of 1.1 cm, sparse scanning yields 0.76 cm, and event-guided scanning yields 0.89 cm while keeping event rates between 600 kEv/s and 5 MEv/s depending on scene motion (Muglikar et al., 2021).

Uncertainty-guided light curtains show that active optical allocation can materially improve 3D detection. On Virtual KITTI, a single-beam LiDAR baseline reaches 39.91 3D mAP @0.5 and 15.49 3D mAP @0.7; after one light curtain the scores rise to 58.01 and 35.29, and after three light curtains to 68.52 and 38.47. On SYNTHIA, the same progression goes from 60.49 and 47.73 to 68.79 and 55.99 after one curtain, and to 69.16 and 57.30 after three (Ancha et al., 2020).

Adaptive coherent LiDAR emphasizes a different metric regime: selective high-acuity sensing. It reports a best angular resolution of 0.012°, up to 15 times the resolution of conventional 3D LiDAR sensors, 115 equivalent scanning lines, 4D parallel imaging, point precision with 90% of points below 1.3 cm and average precision 0.9 cm, and a maximum acquisition rate of 1.7 million pixels/s. In the colorized road-scene demonstration, focused rescanning makes the number “80” on a road sign visible only after targeted densification (Chen et al., 2024).

Robot illumination control for SLAM shows that optimal light is often neither minimal nor maximal. Lightning improves trajectory robustness over fixed 0% and 100% illumination baselines and often uses moderate mean light levels rather than always selecting maximum intensity. In online deployment, the learned controller reaches trajectory ratio 0.89 in 113dark and 0.91 in kitchenloop while using substantially less light than the always-on baseline, and it runs on a Jetson Orin at 0.5 Hz after offline distillation from oracle schedules (Turkar et al., 13 Feb 2026).

5. Relation to adjacent active optical sensing fields

LiDAS is not exhausted by nighttime automotive headlights. Several adjacent research areas instantiate the same control principle under different names or hardware assumptions.

First, adaptive LiDAR can be interpreted as LiDAS when scan density or beam placement is the optical resource being reallocated. The adaptive coherent FMCW LiDAR for “4D bionic vision” explicitly performs dynamic gazing, zoom-in sensing, and region-of-interest selection over a wide field of view by tuning the external-cavity-laser center wavelength and the comb repetition rate. The paper does not present a fully closed-loop autonomous saliency engine, but it does demonstrate measurement-informed densification of cones, barricades, and road signs (Chen et al., 2024).

Second, active sensing can be guided by uncertainty rather than by direct task loss. Light curtains maximize information gain from detector entropy under device constraints, while gaze-aware adaptive LiDAR for ADAS reallocates range and angular resolution away from the driver’s region of focus and toward unattended regions. In 50% fog, the range-controlled variants detect the right vehicle approximately 1 second earlier than the non-adaptive baseline and the resolution-controlled adaptive LiDAR, while preserving average power and scan-time fairness constraints (Scarì et al., 24 Feb 2025, Ancha et al., 2020).

Third, visible-light infrastructures can become context-aware sensing actuators. The integrated sensing, communication, and illumination framework uses NLOS user localization to switch LED power optimization objectives, achieving 53.59% energy savings over a non-adaptive system and improving SNR uniformity by 57.79% while maintaining a mean localization error of 0.071 m (Xie et al., 26 Nov 2025). This is not task-loss-driven perception in the LiDAS nighttime sense, but it is a direct example of sensed context driving online lighting control.

Fourth, some works are better understood as adjacent foundations than as LiDAS proper. “Super LiDAR Reflectance for Robotic Perception” argues for LiDAR as an active optical imaging device and reconstructs dense reflectance images from sparse non-repeating scanning LiDAR data, but it improves the representation layer rather than the control layer (Gao et al., 14 Aug 2025). MARS and Robot Active Neural Sensing are viewpoint-active systems that use multimodal perception and next-best-view planning in cluttered environments, including low ambient lighting conditions, yet they do not explicitly optimize illumination (Zeng et al., 2024, Ren et al., 2022). BOUNDS and AI-KF are not optical or lighting methods, but they provide a principled way to discover actuation motifs that improve observability in nonlinear systems; a plausible implication is that analogous tools could quantify which illumination motifs improve estimation of depth, normals, reflectance, or material parameters in future LiDAS systems (Cellini et al., 11 Nov 2025).

6. Misconceptions, limitations, and research directions

A recurring misconception is that LiDAS is merely “brighter lighting.” The central result of the nighttime LiDAS paper is the opposite: gains come from where light is placed, not from increasing total light flux, and equal-power redistribution is the key mechanism. The same principle appears in the NLOS adaptive-lighting paper, which argues against broad spatially varying or floodlighting patterns under a fixed power budget and shows that concentrating power on a few high-yield patches produces greater NLOS radiosity than spreading the same total energy over many patches (Moreau et al., 9 Dec 2025, Chandran et al., 2019).

A second misconception is that every LiDAS-style system is already fully dynamic in the closed-loop sense. The original NLOS adaptive-lighting system is geometry-adaptive and power-aware, but not online or uncertainty-aware; it precomputes a bank of physics-optimized patterns and, at inference, projects one image per candidate hidden voxel. Its adaptive optimization takes 1–2 minutes on a CPU, depends on known LOS geometry and scene-specific calibration, assumes diffuse transport, handles LOS occlusions through ray-cast visibility, and does not model hidden-scene occlusions (Chandran et al., 2019). By contrast, nighttime LiDAS predicts the next illumination field online, runs in 6.8 ms on an RTX 4090, and is evaluated in real closed-loop driving scenarios, but it still depends on HD/projectable headlights, camera-headlight calibration, and a differentiable relighting surrogate that does not explicitly model reflectance changes (Moreau et al., 9 Dec 2025).

A third limitation concerns deployment constraints. Lightning assumes a co-located camera-light geometry and raw sensor-space relighting, and its online light runs at 0.5 Hz rather than at image rate; the method was tested mostly indoors and depends on the fidelity of its relighting model (Turkar et al., 13 Feb 2026). Adaptive coherent LiDAR remains only partially integrated, still uses a Galvo mirror for horizontal scan, demonstrates its main imaging result at about 3.2 m, and does not yet show an onboard saliency or AI loop for ROI selection (Chen et al., 2024). Visible-light integrated sensing frameworks must manage calibration, single-user assumptions, and future transition algorithms for flicker and communication interruptions (Xie et al., 26 Nov 2025). Automotive nighttime LiDAS adds anti-glare and regulatory concerns: the supplementary text explicitly notes that the policy does not currently enforce glare avoidance toward other road users, and many jurisdictions do not yet allow fully dynamic HD headlight functions (Moreau et al., 9 Dec 2025).

These limitations suggest several converging research directions. One is tighter integration of sensing and control: sequential closed-loop policies that reason directly about uncertainty, rather than precomputed pattern banks or mode switches. Another is broader optical actuation: combining illumination field control, scan density control, wavelength or polarization modulation, and viewpoint change. A third is stronger forward modeling, since physically based automotive LiDAR simulation now makes it possible to model blooming, echo pulse width, ambient light, beam spread, and receiver sensitivity in the near-infrared; this suggests that future LiDAS policies could be trained or stress-tested against illumination-dependent sensing failures before deployment (Dudzik et al., 5 Dec 2025). Across all of these directions, the consistent design insight is that if illumination is controllable, then light placement, energy allocation, and actuation timing should be treated as part of the estimator rather than as fixed background conditions.