RoadSight: Road Safety Perception Systems

• RoadSight is a family of road-safety perception systems that transform noisy sensor data into real-time safety judgments for pedestrians and vehicles.
• It leverages techniques like optical flow, YOLO-based object detection, and signal processing to assess traffic, road distress, and lane visibility.
• Deployment ranges from fixed Android crossing aids to mobile, edge-assisted networks, emphasizing calibration, latency reduction, and multimodal integration.

Searching arXiv for RoadSight-related and roadside perception papers to ground the article in current literature. RoadSight denotes a family of road-safety and road-infrastructure perception systems that transform roadside or vehicle-borne sensing into actionable safety judgments, warnings, maps, or assessments. In the narrow sense, it is the name of a stationary Android-based crossing aid mounted near an unsignalized crossing, using a camera aimed at approaching traffic to indicate either TRAFFIC or GAP for pedestrians (Perry et al., 2016). In a broader research sense, the same design logic appears in dashboard-mounted smartphone pipelines for road-distress detection, roadside edge-cloud perception for connected automated vehicles, multimodal road-surface sensing, road-scene text reasoning, overtaking assistance based on sight distance, and visual road assessment workflows (Heitzmann, 2022, Zhang et al., 2023, Tayeb et al., 2024, Tom et al., 2023, Bauske et al., 25 Feb 2025, Jongwiriyanurak et al., 2024). The common thread is the conversion of noisy real-world roadway observations into low-latency, deployment-oriented safety intelligence.

1. Conceptual scope and problem domain

RoadSight research targets safety-critical situations in which human judgment is expensive, inconsistent, delayed, or unavailable. In the crossing-aid formulation, the problem is deciding whether there is a safe moment to cross a road without pedestrian lights, especially for visually impaired users, elderly people, and children; the system must detect a vehicle sufficiently early—about 7 seconds before contact—because a pedestrian typically needs roughly 4.5–7 seconds to cross common road widths at about 1.2 m/s (Perry et al., 2016). In road infrastructure management, the target shifts to potholes and cracks that are otherwise identified through human visual inspection or specialized inspection vehicles, both described as expensive, labor-intensive, hard to scale, and prone to human error (Heitzmann, 2022).

A second branch of the literature extends the same paradigm from local warning to scene understanding. RoadTextVQA treats road-scene text and road signs as temporally sparse but safety-relevant signals for driver assistance and in-vehicle multimodal reasoning, using 3,222 driving videos and 10,500 question-answer pairs grounded in text or road signs (Tom et al., 2023). MSight moves the sensing locus to elevated roadside infrastructure and frames RoadSight-like functionality as infrastructure-based cooperative perception for connected automated vehicles, with real-time vehicle detection, localization, tracking, and short-term trajectory prediction (Zhang et al., 2023). V-RoAst pushes the idea further toward road-safety coding under the iRAP standard, asking whether zero-shot vision-LLMs can serve as road safety assessors without labeled training data (Jongwiriyanurak et al., 2024).

A common misconception is to treat RoadSight as a single product category or as necessarily smartphone-centric. The literature does not support that simplification. One instantiation is a stand-alone Android device attached to a traffic sign or lighting pole (Perry et al., 2016), but conceptually similar systems include dashboard-mounted smartphones for road-distress mapping (Heitzmann, 2022), fisheye-camera roadside cabinets with V2X dissemination (Zhang et al., 2023), multimodal camera-plus-microphone road-surface classifiers (Tayeb et al., 2024), and single-camera edge-computing services for urban collision-risk inference (Mendes et al., 15 Apr 2025). This suggests that “RoadSight” is best understood as a systems pattern: fixed or mobile sensing, calibrated roadway context, and a decision layer that produces safety-relevant output.

2. Sensing modalities and computational primitives

The sensing stack varies by deployment scenario, but several computational primitives recur. The original crossing device is based on optical flow $\vec{u}_n(x,y)$, a learned influx map $\vec{m}(x,y)$, and an Activity signal defined as

$A_n=\sum_{x,y}\vec{m}(x,y)\cdot\vec{u}_n(x,y).$

The influx map is obtained during a brief calibration period by temporally averaging optical flow, suppressing random motion while preserving the systematic motion of incoming vehicles. Dense optical flow is computed in training using Farnebäck’s algorithm in OpenCV4Android, while runtime uses sparse Lucas–Kanade flow evaluated at about 2000 image points sampled from a Gaussian density centered at the point of first appearance (Perry et al., 2016).

Object-detection variants replace motion templates with learned detectors. The road-distress pipeline built on the IEEE Global Road Damage Detection Challenge uses YOLOv5-x, YOLOv5-l, and Faster R-CNN for multi-class object detection over the four most frequent classes—D00 longitudinal cracks, D10 lateral cracks, D20 alligator cracks, and D40 potholes. Predictions are filtered by Non-Maximum Suppression (NMS) under an Intersection over Union (IoU) criterion, with tuned confidence and NMS thresholds, optional test-time augmentation, and model ensembling (Heitzmann, 2022). SmartRSD similarly uses a classifier architecture rather than a detector, but its core pattern is multimodal: an improved MobileNet image branch, an improved YAMNet audio branch, and weighted fusion with initial weights of 60% for image predictions and 40% for audio predictions (Tayeb et al., 2024).

Roadside cooperative-perception systems add calibration, localization, tracking, and forecasting layers. MSight uses landmark-based fisheye calibration, a generic radial distortion model

$$r(\theta)=k_1\theta + k_2\theta^3 + k_3\theta^5 + \dots,$$

a homography $\mathbf{H}$ from undistorted image space to world coordinates, and an Enhanced Correlation Coefficient (ECC) image-alignment step to compensate for pole drift over time. Object detection is performed with YOLOX-nano, localization maps detections to latitude/longitude, tracking uses a SORT-style Kalman filter plus the Hungarian algorithm, and short-term prediction is handled by a transformer over historical trajectories (Zhang et al., 2023). A single-camera smart-city service follows a lighter-weight version of the same logic—camera calibration, geoframing from four non-collinear road points, YOLOv8n detection, coordinate transformation to world coordinates, Haversine distance, and a rule-based collision-risk layer (Mendes et al., 15 Apr 2025).

The literature also contains prompt-driven and generative instantiations. V-RoAst recasts road assessment as visual question answering, with prompts that include task specification, local context, attribute details for 52 iRAP attributes, and a JSON output constraint (Jongwiriyanurak et al., 2024). The road-redesign work begins with hotspot classification, GradCAM, GradCAM++, or ScoreCAM for accident-prone feature localization, and then uses DreamBooth plus Textual inversion on top of Hugging Face stable-diffusion-v1-5 to inpaint safer roadway elements (Mishra et al., 2023). In that sense, RoadSight research is not confined to perception in the narrow recognition sense; it also includes explanation, redesign, and planning support.

3. The canonical crossing-aid system

The clearest literal RoadSight system is the Android-based safety device for unsignalized crossings (Perry et al., 2016). It is mounted near the crossing on a traffic sign or lighting pole, points its camera toward approaching traffic, and exposes a simplified decision variable—TRAFFIC or GAP—through sound, display, vibration, or communication to another receiver or smartphone. By being fixed in place, it avoids ego-motion, reduces computational burden, and makes calibration practical.

Its decision rule is a matched-filter test over the temporal Activity signal. The system learns a representative Activity pulse template $s_n$ and noise variance $\sigma^2$, then tests

$$\tilde{y} \equiv \sum_{n=0}^{N-1} A_n s_n \gtrless \gamma,$$

under the hypotheses $\theta_1$: traffic present, $A_n = s_n + \nu_n$, and $\vec{m}(x,y)$0: gap / no traffic, $\vec{m}(x,y)$1, where $\vec{m}(x,y)$2 is zero-mean Gaussian noise. The threshold is selected by the Neyman–Pearson criterion and expressed as

$\vec{m}(x,y)$3

The corresponding probability of detection is

$\vec{m}(x,y)$4

This formulation is notable because the system does not attempt full object detection; it converts the scene into a one-dimensional temporal signal whose rising pulse structure is sufficient for a safety decision.

The prototype was implemented on a Samsung Galaxy K-zoom running Android 4.4.2 (KitKat), using OpenCV4Android SDK 2.4.10, FFmpeg, and JavaCPP. The CPU alone was sufficient, and the system achieved about 8 frames per second in real time; operation could be reduced to 2 fps without major degradation, while the template-matching decision stage ran at 30 fps using temporal interpolation. A 35-minute training period was used to generate the influx map and Activity pulse template. At a typical test site, the device warned of approaching traffic about 8–10 seconds before the Activity peak, which corresponds to warning before the vehicle reaches the camera and gives pedestrians sufficient time to cross safely.

The paper is also explicit about false alarms and edge cases. Alerts were triggered by a person on the road, a cat, slow bicycles, vehicles appearing late from parking spaces or turning from driveways, and a car that appeared, stopped, and then turned into a side street. Performance in rain or snow was not established, and night-time operation is challenging because headlights can saturate the camera and create imaging artifacts. These limitations are central rather than incidental: they delimit the circumstances under which a motion-template RoadSight can substitute for a pedestrian signal.

4. Mobile roadway monitoring and semantic road understanding

A major extension of RoadSight is mobile road-condition sensing from ordinary vehicles. The road-distress system built on the IEEE Global Road Damage Detection Challenge (GRDC) 2020 dataset uses 21,041 road images of resolutions 600×600 and 720×720, captured by a smartphone camera mounted on a vehicle dashboard at about 25 mph, with hand annotations by researchers from the Indian Institute of Technology Roorkee and the University of Tokyo (Heitzmann, 2022). The dataset contains images from Japan: 10,506, India: 7,706, and the Czech Republic: 2,829, and the paper splits the training data into 20,621 training images and 420 validation images. Baseline scores were F1 = 0.52 for YOLOv5x, 0.52 for YOLOv5l, and 0.50 for Faster R-CNN; a six-model YOLOv5 ensemble reached 0.57, test-time augmentation reached 0.59, and exhaustive tuning of confidence and NMS thresholds produced F1 = 0.68 with confidence threshold = 0.25 and NMS threshold = 0.999, ranking top 5 among 121 teams.

The operational significance of that pipeline lies in its deployment model. Images or video from a dashboard-mounted smartphone are fed to a trained detector, which outputs bounding boxes, class labels, and confidence scores; NMS resolves overlaps; optional test-time augmentation and model ensembling improve accuracy; and detections can be displayed in real time, stored for offline review, geotagged using GPS from EXIF metadata, and aggregated into maps of road quality for infrastructure management. The paper further notes possible integration with road quality maps, government maintenance databases, multiple phone viewpoints, accelerometer-based roughness estimation, and the International Roughness Index (IRI) (Heitzmann, 2022). This places RoadSight within infrastructure management as much as within online warning.

Road-surface state detection adds a different sensing dimension. SmartRSD argues that vision-only surface classification is brittle under fog, debris, low light, and reflective wet surfaces, and therefore combines road images with vehicle–road interaction audio (Tayeb et al., 2024). Its dataset contains 8,580 images—2,868 dry, 2,548 wet, 2,354 snow—and 75 minutes of audio collected over one year. The best-performing configuration, Improved MobileNet + Improved YAMNet, achieved 0.9491 accuracy, 0.9387 precision, 0.9356 recall, and 0.9384 F1-score. The paper does not report standalone latency figures, but it frames the architecture as suitable for deployment on a mini PC and emphasizes real-time road-surface detection for safer driving.

A more interpretive branch is represented by the road-redesign paper, which identifies accident hotspots using DBSCAN, trains hotspot classifiers on 5,088 hotspot images and 4,908 non-hotspot images, localizes accident-prone features with CAM methods, and then inpaints safer structures such as chicanes, chokers, curb extensions, raised medians, roundabouts, street plazas, and road markings on large intersections (Mishra et al., 2023). It reports an average inpainting time of 2 minutes per image and, for SqueezeNet-ABM, a reduction in hotspot classification probability from 0.97 on original scenes to 0.86 on inpainted scenes, described as an 11.85% average drop. This suggests a RoadSight trajectory from detection toward counterfactual redesign.

The semantic end of the spectrum is visible in RoadTextVQA and V-RoAst. RoadTextVQA focuses on transient road-scene text and road-sign semantics, reporting that the strongest learned baseline, GIT + MSRVTT-QA pretrain, achieved 29.58% Acc and 35.23 ANLS, while an OCR-aware transformer, M4C, reached 38.20% Acc and 44.22 ANLS when given the timestamp frame (Tom et al., 2023). V-RoAst, by contrast, uses Gemini-1.5-flash and GPT-4o-mini in zero-shot mode to code iRAP attributes from road images, and the paper concludes that VLMs remain weaker than CNN baselines on most attributes but can generalize to unseen classes and support star-rating inference from crowdsourced imagery (Jongwiriyanurak et al., 2024). Together, these works broaden RoadSight from geometric hazard sensing to multimodal semantic assessment.

5. Roadside infrastructure perception, work-zone warning, and benchmarking

RoadSight acquires a different character when sensing is moved from the vehicle to the roadside. MSight is a full-stack edge-cloud roadside perception system operating 24/7 at a two-lane roundabout in Ann Arbor, Michigan, with four GridSmart fisheye cameras mounted at the four corners of the roundabout, a Nuvo edge device in a traffic cabinet, and a Cohda MK5 roadside unit (RSU) radio for V2X dissemination (Zhang et al., 2023). Calibration error across the four cameras stays below 0.6 m—NE 0.53 m, NW 0.47 m, SE 0.39 m, SW 0.46 m—and runtime image alignment prevents drift from growing to values such as 2.97 m. With four cameras, the system achieved FN rate 11.83%, FP rate 4.51%, lateral error 0.63 m, longitudinal error 0.90 m, 0.38 ID switches, longest track 90.25%, and MOTA 0.82. The real-time pipeline averaged 35 ms for edge-side perception and 40 ms for roadside-to-vehicle transmission.

The same roadside logic appears in short-term roadwork protection. The vision-based warning system for maintenance personnel uses two conventional RGB cameras mounted on the maintenance vehicle, YOLOv4 detection, detection-based tracking with a Kalman filter and Hungarian assignment, and a worker-worn warning peripheral providing acoustic and visual signals (Ni et al., 2022). The detector was trained by transfer learning on 1200 annotated images with a 60/20/20 split, learning rate: 0.001, and 20 epochs. In laboratory evaluation it reached Average Precision (AP): 96.2%, 26.9 ms per frame, and 37.2 FPS. Field tests showed warning lead times of 3 to 7 seconds before the vehicle passed the construction site, with 1 to 2 seconds in curved or occluded cases. Its traffic flow check algorithm uses a threshold duration of 10 s to suppress redundant alerts and reduced warning counts from 1308 warnings/day to 316 warnings/day, filtering about 80% of useless notices.

A lighter smart-city variant is the single static camera service implemented at the Aveiro Tech City Living Lab (ATCLL) on an NVIDIA Jetson Xavier AGX (Mendes et al., 15 Apr 2025). It decomposes into three concurrent services—frame acquisition and lens distortion correction, YOLOv8n detection/tracking, and geographic position estimation plus MQTT publication. The reported end-to-end latency had median total service latency: 0.44 s, minimum: 0.35 s, maximum: 0.99 s, and standard deviation: 0.05 s; the GStreamer stream service alone had mean latency: 0.08 s and standard deviation: 0.01 s. For distance validation, a crosswalk of actual length 8 m was estimated as 8.45 m and 9.44 m. The system is explicitly described as near-real-time rather than hard real-time, and its collision layer relies on geometric inference rather than a learned risk predictor.

The need for standardized evaluation is itself now part of RoadSight research. A field-testing methodology at Mcity defines latency estimation, point matching, association matching, and a metric suite including MOTP, MOTA, IDF1, and HOTA, using a 1.5 m threshold tied to SAE J2945/1 (Zhang et al., 2024). In comparative trials, System A (LiDAR-based) had mean latency 48 ms, System B 145 ms, and System C 1715 ms. For vehicle detection in daytime, System A achieved FP 0.0%, FN 2.1%, MOTA 92.0%, MOTP 0.402 m, IDF1 96.3%, and HOTA 96.3%. The study’s key methodological point is that visually plausible trajectories can still fail quantitative evaluation once temporal alignment and localization error are handled correctly. For RoadSight-like platforms, benchmark design is therefore part of the technical core, not merely an afterthought.

6. Visibility, sight distance, communication, and predictive warning

Some RoadSight systems do not detect objects directly; instead, they estimate the visibility or communication conditions that determine whether a maneuver is safe. The road-pathway visibility diagnostic literature defines safety in terms of required visibility distance versus available visibility distance along the road (Charbonnier et al., 2016). The paper contrasts two-vehicle following systems with one-vehicle alternatives based on binocular stereovision and Terrestrial LIDAR Mobile Mapping System data. In the LIDAR formulation, the observer is centered on the lane axis at 1 m height, and visibility is computed geometrically by simulating a target at increasing distances on the road axis. The volumetric target has dimensions

$\vec{m}(x,y)$5

and is treated as visible if at least 5% of its surface is visible. On a 2 km road section, the LIDAR 3D-model method was in good agreement with the vehicle-following reference, whereas stereovision tended to underestimate visibility on long sections.

The overtaking-assistance system on country roads applies the same logic to a predictive warning problem (Bauske et al., 25 Feb 2025). It runs once per second, identifies three warning cases—insufficient room between two vehicles ahead, available sight distance smaller than required sight distance, and unsafe nearby intersections—and compares map-derived visibility against a model-based minimum required sight distance

$\vec{m}(x,y)$6

The overtaking model includes three acceleration models, but the simulator warnings use the constant acceleration model

$\vec{m}(x,y)$7

with $\vec{m}(x,y)$8, a global acceleration scaling factor $\vec{m}(x,y)$9, and conservative assumptions about oncoming traffic and braking. In a VR driving simulator study with N = 25 recruited, 20 completed, and 19 included in analysis, the paper compares monitoring-focused and scheduling-focused UIs. The monitoring UI obtained SUS 85.0, the scheduling UI 78.6; both made drivers more patient, but the study found no significant difference in the measured available-to-required sight distance ratio at the point-of-no-return. The authors further report that prediction accuracy improved more by correcting assumptions about driver behavior—especially raising $A_n=\sum_{x,y}\vec{m}(x,y)\cdot\vec{u}_n(x,y).$0 from 0.8 to 1 and relaxing the speed cap—than by using more complex acceleration models.

Communication-oriented visibility appears in the optical intelligent reflecting surface work for blocked urban intersections (Pal et al., 2022). There, the safety problem is dissemination of vehicular visible-light communication messages when line of sight is blocked by vehicles, buildings, roadside structures, or signboards. The proposed mirror-array O-IRS is compared against optical relay and non-line-of-sight road reflection. For a BER target of $A_n=\sum_{x,y}\vec{m}(x,y)\cdot\vec{u}_n(x,y).$1, the maximum allowable $A_n=\sum_{x,y}\vec{m}(x,y)\cdot\vec{u}_n(x,y).$2 is reported as 6 m for NRR, 73 m for optical relay, 45 m for $A_n=\sum_{x,y}\vec{m}(x,y)\cdot\vec{u}_n(x,y).$3 O-IRS, 184 m for $A_n=\sum_{x,y}\vec{m}(x,y)\cdot\vec{u}_n(x,y).$4 O-IRS, and 459 m for $A_n=\sum_{x,y}\vec{m}(x,y)\cdot\vec{u}_n(x,y).$5 O-IRS. The ordering is stated explicitly as

$A_n=\sum_{x,y}\vec{m}(x,y)\cdot\vec{u}_n(x,y).$6

for sufficiently large mirror arrays. Although this work is not a vision system, it extends the RoadSight theme from scene perception to safety-message propagation under intersection occlusion.

7. Limitations, interpretation, and research trajectory

Across the literature, RoadSight systems are effective only within carefully delimited sensing and deployment assumptions. The original crossing aid does not establish performance in rain or snow, and night-time operation is difficult because headlights can saturate the camera (Perry et al., 2016). The smartphone road-distress detector remains imperfect even at F1 0.68, and its accuracy-speed trade-off worsens when ensembling and test-time augmentation are added (Heitzmann, 2022). SmartRSD considers only dry, wet, and snow classes and does not report detailed latency figures or cross-dataset generalization (Tayeb et al., 2024). The single-camera urban monitoring service does not model the Z-axis, and its 0.44 s median end-to-end latency does not always satisfy the cited 300 ms warning target (Mendes et al., 15 Apr 2025).

A second limitation is that calibration and timing errors can dominate perceived algorithm quality. MSight introduces runtime image alignment precisely because roadside poles drift over time, and without that alignment calibration error can reach 2.97 m (Zhang et al., 2023). The Mcity evaluation paper shows that asynchronous outputs and latency variation can make an image-based system appear visually accurate while failing the 1.5 m localization threshold after proper temporal compensation (Zhang et al., 2024). This suggests that RoadSight evaluation must jointly model sensing, synchronization, and geospatial error rather than treating “detection” as a purely semantic problem.

Human factors remain central. The overtaking assistant explicitly warns that “no warning” does not mean overtaking is recommended, only that the system has not detected insufficient sight distance (Bauske et al., 25 Feb 2025). The roadwork warning system had to add a 10 s traffic-flow check because continuous warnings would otherwise create nuisance and warning fatigue (Ni et al., 2022). The road-redesign work adds saliency enhancement because accident-prone features may not be visually salient enough to drivers, even when a classifier identifies them (Mishra et al., 2023). V-RoAst likewise emphasizes that VLM performance is poor when key visual features are absent in imagery, so human labeling remains necessary in some cases (Jongwiriyanurak et al., 2024).

The trajectory of the field is therefore additive rather than substitutive. Recent work points toward more geographically diverse datasets, explicit road-sign modeling, additional phone viewpoints, integration with IRI, improved handling of repeated or recurrent questions in road-scene VQA, fine-tuning or parameter-efficient adaptation for VLMs, and hybrid systems in which semantic coding, geometric visibility, forecasting, and infrastructure communication are combined rather than optimized separately (Heitzmann, 2022, Tom et al., 2023, Jongwiriyanurak et al., 2024, Pal et al., 2022). A plausible implication is that future RoadSight systems will be judged less by any single detector score than by how reliably they connect calibrated sensing, interpretable reasoning, and deployment-specific safety action.