CAD2DMD-SET: Synthetic Dataset & Geometric PTAS

• The paper introduces a synthetic dataset pipeline that generates 100,000 realistic DMD composites to robustly fine-tune LVLMs for digital measurement recognition.
• The work presents a local-search PTAS for minimum dominating set and set cover problems on 2D convex objects, achieving near-optimal (1+ε)-approximations.
• CAD2DMD-SET uniquely bridges computer vision and computational geometry, offering scalable tools for both robust visual question answering and efficient geometric optimization.

CAD2DMD-SET encompasses two distinct research pillars: (1) a synthetic dataset and tooling framework for fine-tuning Large Vision-LLMs (LVLMs) on Digital Measurement Device (DMD) recognition tasks (Valente et al., 29 Aug 2025), and (2) a formal geometric optimization framework addressing minimum dominating set and set cover problems for 2D convex objects via local-search PTAS (Polynomial Time Approximation Schemes) (De et al., 2016). Both share the CAD2DMD-SET acronym, yet originate from separate threads: the first grounded in machine learning and computer vision, the second in computational geometry. The following entry rigorously details both lines, their formalisms, methodologies, evaluations, and interrelations.

1. CAD2DMD-SET in Vision-LLM Fine-Tuning

Modern LVLMs exhibit substantial deficits in fine-grained visual question answering (VQA) tasks involving DMDs, particularly under real-world image degradations such as clutter, occlusion, extreme viewpoints, and motion blur. CAD2DMD-SET was developed in response to the inability of existing image–text corpora to expose LVLMs to the distributional variety and photometric complexity inherent to DMD scenes, which are frequent in Augmented Reality and head-mounted camera contexts (Valente et al., 29 Aug 2025). The absence of sufficient data leads to brittle optical character recognition (OCR) capabilities within these models, especially when extracting segmented digital readings.

CAD2DMD-SET addresses this data scarcity by generating large-scale, high-diversity, and well-labeled synthetic datasets, supporting performance-robust fine-tuning for LVLMs in realistic DMD VQA scenarios.

2. Synthetic Data Generation and Dataset Design

The CAD2DMD-SET pipeline comprises multiple stages to ensure visually plausible, label-rich datasets for DMD-focused VQA:

• 3D CAD Model Acquisition: Six devices—industrial multimeter, power supply, musical metronome, medical thermometer, pulse oximeter, blood-pressure monitor—are modeled using photogrammetric reconstruction (Apple’s Object Capture), prioritizing mesh quality and UV-mapping for downstream rendering.
• Display Image Generator: For each operational mode, a dictionary of physically valid readings is defined. Blank display photos are used to annotate Regions of Interest (ROIs) and mask display segments. Synthetic readings, stochastically sampled and rendered with 24 DSEG seven-segment fonts, are inserted into these ROIs; associated metadata (mode, ROI, value, unit) are exported per image.
• Renderer: Utilizing Blender (Eevee Next), the CAD model display faces are UV-mapped with the synthetic textures. Each image instance is rendered via randomized camera parameters (distance, rotation, focal length) and physical scene variables (background color, illumination). RGB outputs undergo convolution with randomly-selected motion blur kernels; depth and mask passes facilitate later compositing.
• Image Composition: Device renders are overlaid on 2,900 contextually rich indoor backgrounds from ADE20k. Bounding-box placements employ the FOPA model, maintaining aspect ratio and background coverage constraints (≥10%). Optionally, diffusion harmonization aligns color statistics. Complete montage parameters are logged for traceability.

The resulting dataset: approximately 100,000 DMD composites, balanced across device types and operational modes, each instance annotated with bounding boxes, segmentation masks, scene configuration, and VQA pairs linking device readings to semantically relevant questions (“What is the pulse rate?”).

3. Real-World Validation and Performance Evaluation

To benchmark model generalization, the DMDBench set was constructed—1,000 painstakingly annotated real-world DMD images, sampled under challenging conditions (motion blur, low light, oblique angles, glare). Images are labeled using the CAD2DMD-SET tool and visually cross-checked, constituting a gold-standard for DMD VQA under real operational constraints (Valente et al., 29 Aug 2025).

Evaluation leverages:

• Primary Metric: Average Normalised Levenshtein Similarity (ANLS):

$$\mathrm{ANLS} = \frac{1}{N}\sum_{i=1}^{N} \max\left(0,\ 1 - \frac{\mathrm{lev}(a_i,\,\hat a_i)}{\max(|a_i|,\,|\hat a_i|)}\right)$$

where $a_i$ is the ground-truth answer, $\hat a_i$ the predicted answer, and $\mathrm{lev}$ denotes Levenshtein distance.

• Secondary Metrics (prompt-independent):
  • Word-level accuracy (mean of numeric and unit token accuracies),
  • Numeric accuracy,
  • Unit accuracy.

Fine-tuning employs Low-Rank Adaptation (LoRA) modules added to pretrained LVLMs. Training proceeds in two stages: initial instruction tuning, followed by LoRA fine-tuning with a data blend (typically 10% or 50% synthetic CAD2DMD-SET, the rest from canonical corpora). Experiments on NVIDIA A100 (rank $r=8$, 3 epochs, batch size 32) show that LVLMs (InternVL, LLaVA) experience orders-of-magnitude ANLS improvement post-fine-tuning. Notably, InternVL gains nearly 200% in ANLS (32.92 to 95.16) without loss on external VQA tasks (TextVQA ANLS remains ~76.8%) (Valente et al., 29 Aug 2025).

4. Geometric Optimization: Minimum Dominating Set and Set Cover

Separately, CAD2DMD-SET denotes the theoretical analysis of geometric covering and domination problems in the plane (De et al., 2016). The objective is: given a family of compact, simply connected 2D sets, select the minimal subset that dominates all objects (dominating-set) or covers given points (set-cover).

• Dominating Set: Given $\mathscr S = \{S_1, \ldots, S_n\}$ (planar sets), $A\subseteq\mathscr S$ is a dominating set if for every $S\in\mathscr S$ there exists $A_i\in A$ such that $S\cap A_i\neq\emptyset$.
• Set Cover: Given points $a_i$0 and sets $a_i$1, select minimal $a_i$2 such that $a_i$3.

For homothets of a fixed convex body (dom-set) and convex pseudodisks (set-cover), the local-search PTAS achieves a $a_i$4-approximation.

5. Local-Search PTAS Framework: Structure and Analysis

The framework executes iterative $a_i$5-exchange local search:

1. Initialization: Construct a feasible covering or dominating set $a_i$6.
2. $a_i$7-Exchange Iteration: Seek subsets $a_i$8 and replacements $a_i$9 that lower the solution size and preserve feasibility.
3. Termination: Declare $\hat a_i$0 as $\hat a_i$1-locally optimal when no such exchange exists.

Setting $\hat a_i$2, the total algorithmic complexity is $\hat a_i$3.

Key Structural Lemmas:

• Construction of a planar bipartite "locality" graph between $\hat a_i$4-locally optimal and globally optimal solutions, respecting the coverage/dominance via interior-disjoint "traces."
• Existence of planar separators (Frederickson's theorem) for the locality graph, ensuring each component is small and amenable to exchange arguments.
• The final size guarantee: $\hat a_i$5, where $\hat a_i$6 is the global optimum.

The PTAS is fully combinatorial: no geometric objects beyond basic intersection/containment needs to be manipulated; Voronoi-based duals are analytic, not algorithmic artifacts. The approach generalizes prior PTAS results for disks and unit squares to arbitrary convex homothets (dom-set) and convex pseudodisks (set-cover).

6. Implementation, Extensibility, and Limitations

• Vision-Language Edition: CAD2DMD-SET is implemented as open-source tooling, with pipeline modules (Display, Render, Compose, Label), configuration files, and API documentation for extensibility. Users can import new device CADs, define operational modes, and expand the synthetic dataset coverage to new device types, including analog and graphical measurement interfaces (Valente et al., 29 Aug 2025).
• Geometric Optimization Edition: The local search PTAS is simple to implement, relying only on $\hat a_i$7-local exchange heuristics and standard geometric primitives. No sophisticated data structures are mandated, aside from spatial indexing for efficiency.

Limitations:

• The geometric PTAS handles only unweighted versions; extensions to weighted covering/dominating problems are nontrivial due to breakdowns in the separator structure (De et al., 2016).
• For the synthetic vision-language datasets, real-to-synthetic domain gap mitigation is empirically validated but may remain for extremely complex or out-of-distribution DMD artifacts.

7. Research Impact and Future Prospects

• Vision-Language Pipeline: CAD2DMD-SET elevates DMD VQA accuracy in LVLMs by simulating the spectrum of real-world acquisition conditions, bridging the gap between synthetic pretext datasets and operational deployment (Valente et al., 29 Aug 2025). The benchmark (DMDBench) standardizes evaluation, guiding further advances in OCR-for-instrumentation and robust multimodal reasoning under adverse imaging.
• Geometric Optimization: The locality-separator analysis unlocks practical near-optimality for a broad swath of geometric covering/dominating problems where prior approximability was severely limited; it provides a unifying template for extension to other planar-locality structures (De et al., 2016).

Collectively, CAD2DMD-SET exemplifies the confluence of synthetic data-driven methodology for machine learning robustness and deep algorithmic advances in planar geometric optimization, each branch shaping its research community’s standards and toolkits.