PaveVQA: Pavement & Privacy VQA

Updated 5 July 2026

PaveVQA is an overloaded term with two distinct interpretations: one for pavement distress analysis within PaveBench and another for privacy-preserving visual question answering using symbolic bottlenecks.
In the pavement analysis context, PaveVQA supports interactive multimodal reasoning by integrating classification, localization, segmentation, and quantitative estimation on top-down highway inspection images.
The privacy-preserving variant employs a hybrid edge/cloud architecture that transforms raw images into symbolic representations to protect sensitive visual data.

PaveVQA is an overloaded term in the arXiv literature. In the contemporary pavement-analysis setting, it denotes the vision-language question answering component of PaveBench: a real-image QA dataset for pavement distress analysis on real-world highway inspection images, built from top-down orthographic pavement images and intended to support recognition, localization, quantitative estimation, and maintenance-oriented reasoning (Li et al., 3 Apr 2026). In an earlier and unrelated usage, “PaveVQA” abbreviates “Privacy Preserving Visual Question Answering,” a hybrid edge/cloud architecture that replaces raw images or dense visual features with a symbolic, non-differentiable scene representation extracted on-device (Bara et al., 2022). The term therefore requires contextual disambiguation.

1. Terminological scope

Two distinct usages are documented.

Usage	Description
PaveVQA in PaveBench	A real-image QA dataset for pavement distress analysis (Li et al., 3 Apr 2026)
PaveVQA as Privacy Preserving Visual Question Answering	A privacy-preserving VQA architecture with a symbolic bottleneck (Bara et al., 2022)

The pavement-analysis usage is explicitly tied to a broader benchmark for pavement distress perception and interactive vision-language analysis. The privacy-preserving usage is instead a systems architecture for edge deployment, with privacy framed as abstraction and non-differentiability rather than as a domain-specific dataset. Because both uses are technically substantive and semantically unrelated, citations are necessary whenever the acronym appears in mixed-domain discussion.

2. PaveVQA within PaveBench

Within PaveBench, PaveVQA is the multimodal reasoning and interaction layer built on top of a unified benchmark that covers classification, object detection, semantic segmentation, and vision-language question answering. It is described as a real-image QA dataset for pavement distress analysis, built from top-down orthographic pavement images captured by a highway inspection vehicle, and intended to support recognition, localization, quantitative estimation, and maintenance-oriented reasoning (Li et al., 3 Apr 2026).

The source imagery originates from Liaoning Province, China, collected using a highway inspection vehicle traveling at 80 km/h. The acquisition system uses a high-resolution line-scan camera capturing vertical orthographic views. The paper emphasizes that this top-down orthographic setup preserves geometric properties of pavement distress, including crack width and length, and thereby supports reliable downstream quantification. Before annotation, the continuous scan data were processed by denoising, sharpening, contrast enhancement, and histogram equalization.

The visual annotation pipeline is hierarchical and multi-stage. Classification labels were cross-validated by multiple annotators; detection boxes were annotated with LabelMe; and pixel-level segmentation was manually traced by four domain experts in Photoshop. The paper reports about 10 minutes per image on average for segmentation, with up to 1 hour for complex alligator cracks. All annotations were then reviewed through a multi-stage expert verification process to ensure label consistency and accurate boundaries. During annotation, the benchmark retained hard distractors such as stains and shadows rather than filtering them out, explicitly treating them as robustness-critical cases.

PaveVQA is grounded by multiple supervision levels: image-level classification labels, instance-level bounding boxes, pixel-level segmentation masks, derived geometric metadata, and textual QA/dialogue supervision. The broader visual subset of PaveBench contains 20,124 high-resolution images at $512 \times 512$ resolution. The paper does not state explicitly that PaveVQA uses exactly the full same image pool; the reported counts imply it uses a subset.

3. Dataset construction and interaction design

The PaveVQA construction pipeline links raw images, structured metadata, prompt templates, and an LLM. The authors first design a question pool around practical inspection needs: presence verification, classification, localization, quantitative analysis, severity assessment, and maintenance recommendation. To support grounded numerical reasoning, high-fidelity visual annotations are converted into structured JSON metadata containing bounding box coordinates, pixel area, and skeleton length. These metadata fields function as verifiable evidence for reasoning rather than leaving geometry to be inferred only from pixels (Li et al., 3 Apr 2026).

In the generation stage, raw images, structured metadata, and manually designed prompt templates are jointly fed into ChatGPT-5.2. For each image, the pipeline creates 10 single-turn questions and 2 rounds of multi-turn dialogue, yielding approximately 20 QA pairs per image. Quality control includes negative queries about non-existent distresses, adversarial and error-correction pairs, and human-in-the-loop expert review by pavement domain experts to correct logical inconsistencies and ensure domain fidelity.

The dataset supports three interaction settings: single-turn, multi-turn, and expert-corrected interaction. The paper states that dialogue history and visual states are preserved, enabling multi-turn, context-aware analysis from distress identification to quantitative assessment and decision support. It does not, however, provide a formal context-window representation, exact conversation serialization, role tags, turn separators, or answer constraints.

The question space is repeatedly grouped into four primary areas: recognition, localization, quantitative estimation, and maintenance reasoning. Recognition covers distress presence verification and distress type classification. Localization addresses where the distress is and uses short-text spatial description. Quantitative estimation includes pixel-level numerical estimation, with examples such as crack length and area. Maintenance reasoning includes severity assessment and maintenance recommendation. The dataset is organized into four primary tasks and 14 fine-grained sub-categories, but the paper does not enumerate all 14 sub-categories explicitly in text.

The reported PaveVQA scale is 32,160 question-answer pairs in total, comprising 10,050 single-turn queries, 20,100 multi-turn interactions, and 2,010 error-correction pairs. The paper mentions a PaveVQA training set for LoRA fine-tuning, so a train split clearly exists, but it does not report train/validation/test sizes, whether splits are image-disjoint, or how multi-turn and error-correction samples are partitioned.

4. Evaluation protocol and empirical results

PaveBench evaluates PaveVQA under interactive VQA and agent-augmented VQA settings. For strict numerical and factual queries, the stated protocol uses classification accuracy for categorical queries on distress presence and type, localization token-F1 for short-text spatial descriptions, and segmentation quantification MAE for pixel-level numerical estimation such as crack length and area. For descriptive responses, including severity assessment and maintenance recommendation, the reported metrics are ROUGE-L, BLEU, METEOR, and BERTScore (Li et al., 3 Apr 2026).

The paper also contains a terminology inconsistency. The protocol section says segmentation quantification MAE, whereas Table 5 reports Quant. MAPE $(\downarrow)$ . The experiments table therefore operationally reports Quant. MAPE, but the discrepancy is not explained.

Three VLMs are evaluated in three paradigms each: Base (zero-shot), + LoRA FT, and + Agent Aug. For Qwen2.5-VL-3B, Base reports Cls. Acc. 65.18, Loc. Token-F1 16.39, Quant. MAPE 116.20, ROUGE-L 8.32, BLEU 0.79, METEOR 14.39, and BERTScore 83.69. + LoRA FT improves these to 88.24, 43.77, 47.01, 52.40, 20.82, 44.66, and 92.86. + Agent Aug. reports 89.68, 42.66, 35.40, 53.41, 21.95, 44.22, and 92.78.

For DeepSeek-VL2-small, Base reports Cls. Acc. 55.48, Loc. Token-F1 40.36, Quant. MAPE 531.21, ROUGE-L 24.82, BLEU 4.77, METEOR 23.89, and BERTScore 88.17. + LoRA FT reaches 92.98, 59.23, 48.69, 52.67, 20.57, 43.40, and 93.21. + Agent Aug. reports 92.80, 42.08, 26.91, 50.39, 18.08, 37.79, and 92.65.

For LLaVA-OneVision-7B, Base reports Cls. Acc. 60.91, Loc. Token-F1 27.71, Quant. MAPE 158.24, ROUGE-L 11.01, BLEU 1.69, METEOR 15.95, and BERTScore 85.38. + LoRA FT reaches 83.04, 55.19, 66.95, 50.82, 17.20, 38.52, and 92.76. + Agent Aug. reports 83.33, 46.64, 26.14, 46.09, 13.27, 33.35, and 91.24.

The main empirical patterns are explicit. Zero-shot VLMs perform poorly on domain-specific pavement QA. LoRA fine-tuning yields consistent improvements across all models, indicating that general models are not naturally aligned to the structured taxonomy and response formats of the domain. Agent augmentation is especially effective for quantitative reasoning: Qwen improves Quant. MAPE from 47.01 to 35.40, DeepSeek from 48.69 to 26.91, and LLaVA from 66.95 to 26.14. By contrast, localization token-F1 and some text-generation metrics are often stronger under LoRA than under agent augmentation. The paper therefore characterizes the agent framework as particularly useful for geometry-grounded numerical analysis and for reducing numerical hallucinations.

5. Agent-augmented VQA and operational use

The paper presents a simple and effective agent-augmented VQA framework in which the VLM acts as an interactive controller and specialized perception models act as external tools. The integrated domain-specific tools are OverLoCK-T for classification, DEIM for localization, and SCSegamba for segmentation (Li et al., 3 Apr 2026).

The described workflow is procedural rather than training-centric. A user query is received, the VLM interprets user intent, the query is decomposed into executable subtasks, and each subtask is routed to the corresponding specialized model. Visual outputs such as bounding boxes and pixel masks are then converted into explicit geometric quantities and fed back into the model context as textual evidence before final answer generation. The paper links this design to tool-calling capabilities inspired by prior work such as ReAct and Visual ChatGPT, but does not provide explicit tool-call syntax, prompt templates, planner prompts, decision logic for tool invocation, or pseudocode.

The claimed advantages are reduced multimodal hallucinations, separation of semantic reasoning from geometric measurement, interpretability through inspectable intermediate outputs, and multi-turn support via preservation of dialogue history and visual states. In application terms, these properties matter because PaveVQA is intended not merely for distress recognition but also for inspection dialogue, quantitative analysis, severity assessment, maintenance decision support, and explanation. The benchmark therefore operationalizes a transition from unimodal perception toward interactive, fact-grounded roadway analysis.

6. Privacy Preserving Visual Question Answering as an earlier usage

In the 2022 paper “Privacy Preserving Visual Question Answering,” “PaveVQA” refers to a hybrid architecture for performing VQA on the edge while avoiding transmission of raw images or dense differentiable features. The method constructs a symbolic representation of the visual scene using a lightweight modified EfficientDet-D0 that predicts object bounding boxes, object classes, and attributes; that representation is then sent to the cloud-side QA model, instantiated as MCAN (Deep Modular Co-Attention Network) (Bara et al., 2022).

The privacy motivation is framed through three privacy levels: Not Private, At risk, and Private. Raw images or end-to-end intermediate feature representations are “Not Private”; raw prediction distributions for objects, attributes, and bounding boxes are “At risk”; and a symbolic representation is “Private.” The paper does not provide a formal differential privacy or information-theoretic guarantee. Its central privacy claim is instead that the symbolic representation is non-differentiable and therefore cannot be used to recover the original image, thereby breaking the differentiable link from image to cloud-side reasoning.

The edge-side model starts from EfficientDet-D0 and adds an attribute prediction head for multi-label attribute classification. It is trained on Visual Genome using the same vocabulary setup as Bottom-Up: 1600 object classes and 400 attribute classes. The training procedure is staged: object classification and localization heads are trained to convergence, the attribute head is reintroduced, and all three heads are then trained jointly to convergence. The reported setup uses AdamW, an adaptive learning rate starting at 0.001, an 8-GPU P3.8 instance, 3 days for object classification head training, and 2 additional days for attribute head training.

The symbolic representation is the core bottleneck. For each detected object region, the method forms a vector in $\mathbf{R}^{2048}$ . The class representation takes the top 5 predicted object classes, converts each class name into a GloVe embedding, and concatenates them to yield $\mathbf{R}^{1500}$ . The attribute representation takes the top 5 predicted attributes, converts attribute names into GloVe embeddings, weights each by its confidence score, and sums them to yield $\mathbf{R}^{300}$ . The bounding box representation contributes $\mathbf{R}^{8}$ , using one box normalized to the full image and one box normalized to the “encompassing bounding box.” Concatenation yields $\mathbf{R}^{1808}$ , which is then padded to $\mathbf{R}^{2048}$ . An alternative BERT-based symbolic embedding was also tested, but it underperformed the standard GloVe-based setup.

The paper evaluates both the perception module and end-to-end VQA on VQA 2.0. Parameter counts are reported as 153M for the Bottom-Up visual model and 5.75M for the EfficientDet-based visual model, matching the claim that the latter is more than 25 times smaller. On vision evaluation, the proposed detector reports AP 39.30 and AR 50.48 versus Bottom-Up AP 32.50 and AR 45.67, but its class prediction on detected objects is weaker: Accuracy 47.43, Precision 84.62, Recall 67.13, and F1 74.87 versus Bottom-Up Accuracy 62.53, Precision 87.02, Recall 71.24, and F1 78.34. On attribute prediction, it is stronger: Accuracy 33.51, Precision 64.33, Recall 41.16, and F1 50.20 versus Bottom-Up Accuracy 16.23, Precision 25.62, Recall 30.71, and F1 27.93.

For VQA without captions, the Bottom-Up symbolic private setting reaches Overall 62.49, Other 53.78, Yes/No 81.05, and Count 42.10, whereas the EfficientDet symbolic private setting reaches Overall 55.41, Other 42.95, Yes/No 77.27, and Count 41.89. The authors’ interpretation is that the privacy bottleneck penalty is small relative to the small vision model penalty: raw predictions to symbolic representation causes only a small drop, while replacing Bottom-Up with EfficientDet-D0 causes the larger accuracy loss. Captions, tested using MSCOCO ground-truth captions as a proxy for an ideal captioner, recover some lost holistic and relational information and improve the low-footprint system by more than 4 points overall.

This earlier PaveVQA usage is therefore not a pavement benchmark but a privacy-preserving systems proposal. Its main contribution is to reframe VQA privacy as a representation problem: the critical design decision is what leaves the edge device.

Several adjacent benchmarks help situate PaveVQA as a domain-specific VQA construct. “Polar-VQA: Visual Question Answering on Remote Sensed Ice sheet Imagery from Polar Region” introduces what the authors describe as the first VQA dataset and benchmark built specifically for remote-sensed polar ice-sheet imagery, with questions scoped to sensor identification, place recognition, and zone recognition (Sarkar et al., 2023). Its results show that domain transfer of VQA into scientific imaging is feasible, but also that a benchmark can be scientifically meaningful while still being relatively easy for standard VQA models. This is relevant because PaveVQA similarly anchors VQA in a specialized inspection workflow rather than in generic scene understanding.

“VISTAQA: Benchmarking Joint Visual Question Answering and Pixel-Level Evidence” is presented as a conceptual reference for PaveVQA-style benchmarks, especially where answer generation and evidence localization should be treated as a single coupled task rather than as two independent capabilities (Azadani et al., 20 May 2026). VistaQA requires a free-form textual answer and a set of segmentation masks that constitute the visual evidence supporting that answer, and introduces Grove as a joint metric. This suggests a concrete future direction for evidence-grounded variants of PaveVQA: coupling answer correctness to pixel-level support and explicitly handling no-evidence cases.

“Urban Risk-Aware Navigation via VQA-Based Event Maps for People with Low Vision” is directly relevant to pedestrian-path hazard VQA, but the paper explicitly does not call its dataset “PaveVQA”; it consistently refers to “our dataset” or “the dataset,” without giving it a branded standalone name (Valls et al., 12 May 2026). Its framework uses a three-level hierarchical multicategory query structure over street imagery and aggregates VQA responses into route-level risk maps. The relation to PaveVQA is therefore conceptual rather than nominal: it shows how a hazard-centric VQA dataset can be embedded in a decision-support pipeline for assistive navigation.

Across these neighboring efforts, a common pattern emerges. Domain-specific VQA benchmarks are increasingly organized around grounded metadata, structured reasoning targets, and operational decision support rather than around generic recognition alone. In that landscape, “PaveVQA” most precisely denotes either a pavement-distress QA dataset within PaveBench or, in earlier work, a privacy-preserving symbolic-bottleneck VQA architecture. Context determines which meaning is intended.