Uirapuru: Edge Analytics for PTZ Cameras

• Uirapuru is an edge-based framework for real-time analytics on high-resolution PTZ cameras that dynamically adapts to camera actuation.
• It employs a dynamic programming algorithm over a quad-tree partition to generate per-frame tile plans under strict latency constraints.
• The system transforms historical global object distributions into local, per-frame plans, yielding up to 1.45× accuracy improvements and 4.53× faster inference.

Uirapuru is an edge-based framework for real-time video analytics on high-resolution steerable cameras, specifically pan–tilt–zoom (PTZ) cameras whose actuation continuously changes the scene geometry, object density, and apparent object scale. It is designed for latency-bounded detection on 4K and 8K streams, where conventional static-camera methods such as down-sampling, static tiling, adaptive tiling based on historical statistics, and frame-skipping become unreliable because camera motion invalidates spatial and temporal locality assumptions. Uirapuru addresses this setting by modeling camera actuation explicitly, transforming a historical global object distribution into a per-frame local distribution, and generating a fresh per-frame tile plan online with a dynamic-programming algorithm that respects latency Service-Level Objectives (SLOs). In reported experiments, it achieves up to $1.45\times$ improvement in accuracy at fixed latency SLOs and up to $4.53\times$ faster inference at on-par accuracy relative to state-of-the-art static-camera approaches (Apostolo et al., 1 Sep 2025).

1. Problem formulation and operating assumptions

Uirapuru is motivated by the computational difficulty of running object detection in real time on high-resolution cameras at the edge. Prior work for static-viewpoint cameras commonly relies on down-sampling or static/adaptive tiling, and these methods depend on the premise that object sizes and object distributions do not change abruptly across adjacent frames. Under PTZ actuation, that premise breaks down: pan shifts object density across image regions, while tilt and zoom can change average object size by orders of magnitude between adjacent frames. As a result, tiles that historical plans classify as sparse may become crowded, dense tiles may become empty, and objects may grow beyond model input bounds during zoom (Apostolo et al., 1 Sep 2025).

The framework therefore formulates the problem as per-frame latency-constrained optimization under view-dependent non-stationarity. Its stated goals are to maximize detection accuracy under a per-frame latency budget, remain responsive to pan, tilt, and zoom by updating plans every frame, and operate within edge constraints on GPU, CPU, memory, and energy. This problem framing distinguishes Uirapuru from static-camera approaches such as Remix, whose offline or historical optimization becomes mismatched when actuation alters the field of view.

A central assumption is that the PTZ camera is fixed in position and undergoes rotation and focal scaling with negligible translation. Uirapuru also assumes that only 2D bounding boxes are available for historical objects, so reprojection is approximate rather than a full 3D scene reconstruction. This makes the method computationally tractable on edge devices, but it also delimits the class of camera systems to which it applies directly.

2. System organization and execution pipeline

Uirapuru is organized into a bootstrap phase and a runtime phase. In the bootstrap phase, a model profiler measures, for each detector in a model family, the mean and 99th percentile inference latencies together with an accuracy vector across object-size bins. Historical frames object extraction then gathers objects from a short window of frames representing the global scene, and these objects define the global object distribution. Finally, plan runtime estimation measures tile-planning overhead on historical frames and reserves this overhead from the runtime SLO using $p99 + 10\%$ safety (Apostolo et al., 1 Sep 2025).

In the runtime phase, a distribution controller tracks accumulated camera actuation and maps historical global objects into the current local frame. A tile planner then builds a quad-tree partition for each incoming frame and uses dynamic programming, termed DP-DP, to select tiles and assign models under the available latency budget. Uirapuru also constructs uniform tiling plans and chooses whichever plan—adaptive or uniform—has higher estimated accuracy. After plan selection, the execution stage crops tiles with padding, resizes them if needed, runs inference, and merges detections with Non-Maximum Suppression (NMS).

The runtime data flow is explicitly specified as sensor capture, acquisition of current actuation $\Delta T_t$, computation of local object distribution from historical global objects, plan generation under $L_{SLO}$ minus planning overhead, tile inference, NMS merging, and output of detections. The significance of this organization is that actuation understanding is not treated as an auxiliary signal; it directly informs both distribution estimation and tile selection at frame granularity.

3. Camera actuation model and field-of-view dynamics

Uirapuru models the steerable camera as a pinhole camera and tracks a time-varying 3D transformation $T_t$ initialized at identity for the global view. Each actuation contributes an incremental transformation $\Delta T_t$, yielding

$T_t = T_{t-1} \cdot \Delta T_t.$

With intrinsics $K$, a rotation matrix $R(\theta_t,\phi_t)$ for pan $4.53\times$0 and tilt $4.53\times$1, and a zoom factor $4.53\times$2 scaling focal length, the effective focal length becomes

$4.53\times$3

The horizontal and vertical fields of view are given by

$4.53\times$4

where $4.53\times$5 and $4.53\times$6 denote sensor width and height, or equivalently frame sampling dimensions. The mapping from the global image plane to the local image plane is approximated by the homography

$4.53\times$7

with zoom captured through the scaling of $4.53\times$8 (Apostolo et al., 1 Sep 2025).

Historical bounding-box corners are transformed by $4.53\times$9, boxes outside the local field of view are filtered out, and visible boxes have their areas updated by zoom scaling. Under small rotations, the framework uses the approximations

$p99 + 10\%$0

to estimate motion-induced pixel displacement. This provides a computationally lightweight mechanism for anticipating tile movement and object re-sizing. The paper notes that occlusion and distortion are not explicitly modeled and that more advanced methods such as visual odometry or deep PTZ calibration remain future work. A plausible implication is that Uirapuru prioritizes fast control-oriented reprojection over geometric exactness, which is consistent with its edge-deployment target.

4. Adaptive tiling, model profiling, and optimization objective

Uirapuru uses a hierarchical quad-tree to partition the frame into non-overlapping tiles; for 4K input, a maximum depth of $p99 + 10\%$1 is reported to work well. This hierarchical partitioning decouples spatial decomposition from detector input sizes and reduces the combinatorial search space. For each node $p99 + 10\%$2, with local object set $p99 + 10\%$3, the framework constructs a size-distribution vector over $p99 + 10\%$4 bins,

$p99 + 10\%$5

where $p99 + 10\%$6 is the normalized count of objects in size bin $p99 + 10\%$7. The bins are relative and non-linear: object area is normalized by model input or tile area, the binning allocates finer resolution to small sizes, and it covers relative areas exceeding $p99 + 10\%$8 to handle oversized objects induced by zoom (Apostolo et al., 1 Sep 2025).

Each model $p99 + 10\%$9 has an accuracy profile vector

$\Delta T_t$0

The expected AP for tile $\Delta T_t$1 under model $\Delta T_t$2 is the dot product

$\Delta T_t$3

For a plan $\Delta T_t$4, consisting of selected leaf tiles and assigned models, the estimated plan accuracy is

$\Delta T_t$5

The optimization problem is to maximize $\Delta T_t$6 subject to the latency budget and hierarchical constraints. If $\Delta T_t$7 denotes per-tile latency for model $\Delta T_t$8 and $\Delta T_t$9 the remaining inference budget after subtracting planning overhead, the plan must satisfy

$L_{SLO}$0

Two constraints are enforced: non-overlap, meaning a selected node excludes all descendants, and single-model assignment, meaning each selected node receives exactly one model. The paper characterizes this as a hierarchical knapsack over tile–model pairs with values $L_{SLO}$1 and weights $L_{SLO}$2.

The DP-DP algorithm performs depth-first post-order dynamic programming over the quad-tree. At each node, it combines three possibilities under every discretized latency budget: selecting the node itself with some model, retaining the best solution accumulated from the left path, or taking the optimal combination encoded in the rightmost child’s dynamic-programming state. The complexity is reported as

$L_{SLO}$3

assuming latency budgets are discretized with unit interval. The practical effect is that Uirapuru can replan every frame while keeping planning overhead within a small fraction of the SLO.

5. Scheduling semantics, hardware footprint, and runtime behavior

Uirapuru explicitly reserves planning time from the end-to-end latency budget. Planning overhead is estimated during bootstrap as

$L_{SLO}$4

and the available inference budget is then

$L_{SLO}$5

Two strictness modes are provided. The non-conservative mode uses mean model latencies, $L_{SLO}$6, and therefore targets good average latency with occasional per-frame misses. The conservative mode uses $L_{SLO}$7 and is intended to ensure that at least $L_{SLO}$8 of frames meet the latency SLO (Apostolo et al., 1 Sep 2025).

The implementation uses Python 3.8, TensorFlow, and OpenCV. Evaluations are reported on NVIDIA Jetson AGX Xavier, with a 512-core Volta GPU, 64 Tensor Cores, and an 8-core ARM v8.2 CPU, and on Jetson AGX Orin, with a 2048-core Ampere GPU, 64 Tensor Cores, and a 12-core ARM CPU. The detector family is EfficientDet D0–D6, with input sizes from $L_{SLO}$9 to $T_t$0 and model sizes from $T_t$1 to $T_t$2 MB. Models are fine-tuned on the PANDA Image dataset for 600 epochs. During inference, tiles are padded uniformly, resized to model input, and merged by NMS.

The reported memory footprint reaches up to $T_t$3 GB during inference with all models loaded, with D5 and D6 accounting for about $T_t$4 GB combined. Average per-frame energy is approximately $T_t$5 W on Xavier and $T_t$6 W on Orin, with GPU power dominating at $T_t$7 W and $T_t$8 W respectively. Runtime overhead remains within $T_t$9 to $\Delta T_t$0 of SLO for dynamic programming, while most execution time—more than $\Delta T_t$1 on Xavier and more than $\Delta T_t$2 on Orin—is spent in inference. These measurements indicate that the framework’s control logic is not the dominant bottleneck; detector execution remains the primary cost center.

6. Experimental evaluation, ablations, and comparative behavior

The experimental setup includes two datasets. The first, Steerable PANDA, is derived from PANDA gigapixel images of $\Delta T_t$3 pixels and synthesizes 4K cutout videos with random PTZ sequences per clip, each sequence containing at least one actuation type, actuation durations of 15–50 frames, and 15-frame pauses while respecting image boundaries. The second is a real PTZ case study using 4K video from a deployed PTZ camera at 10:00, 13:00, 16:00, and 19:00 under sunny, cloudy, and dark conditions, with three fixed actuation loops per sequence, the first loop used as historical input and the remaining two for evaluation. Because ground truth is unavailable for that case study, labels are generated with the strongest feasible oracle, namely uniform tiling with EfficientDet D6 (Apostolo et al., 1 Sep 2025).

Baselines are down-sampling (DS), uniform tiling (UT), and Remix (R). SLOs span 500–7000 ms across 12 points on Xavier and 500–3500 ms across 13 points on Orin. The evaluation metrics are COCO-style mAP and AP50. The headline results report up to $\Delta T_t$4 accuracy improvement at the same latency SLOs and up to $\Delta T_t$5 faster inference at on-par accuracy relative to static-camera approaches. One cited example is Xavier, where non-conservative Uirapuru reaches $\Delta T_t$6 mAP at 750 ms versus UT-D3 at $\Delta T_t$7 mAP and 3404 ms, corresponding to approximately $\Delta T_t$8 speedup. On Orin, speedup reaches up to $\Delta T_t$9 at comparable mAP.

Per-actuation analysis at SLO 3500 ms shows Uirapuru obtaining the highest per-frame mAP under pan, tilt, and zoom. Under tilt and zoom, Remix and uniform tiling degrade sharply, whereas down-sampling improves when objects become large. Uirapuru adapts by moving toward DS-like or UT-like plans depending on the local scale and density regime. In the real PTZ case study at AP50 and SLO 2000 ms, it outperforms Remix and Uniform at all times, with gains of $T_t = T_{t-1} \cdot \Delta T_t.$0 to $T_t = T_{t-1} \cdot \Delta T_t.$1 AP50 over Remix and $T_t = T_{t-1} \cdot \Delta T_t.$2 to $T_t = T_{t-1} \cdot \Delta T_t.$3 over Uniform. The cloudy setting yields the best performance and the dark setting the worst, attributed in the source to lighting conditions and artifacts.

The ablation studies isolate three mechanisms. First, the relative non-linear size profile substantially improves adaptive methods in steerable settings; Remix improves when given Uirapuru’s profile, yet Uirapuru still remains superior, with mAP improvement ranges of $T_t = T_{t-1} \cdot \Delta T_t.$4–$T_t = T_{t-1} \cdot \Delta T_t.$5 on Xavier and $T_t = T_{t-1} \cdot \Delta T_t.$6–$T_t = T_{t-1} \cdot \Delta T_t.$7 on Orin. Second, DP-DP dominates under tight SLOs, while uniform plans become more common as SLO increases; on Xavier, DP-DP is chosen in roughly $T_t = T_{t-1} \cdot \Delta T_t.$8–$T_t = T_{t-1} \cdot \Delta T_t.$9 of frames at high SLOs up to 7000 ms and exclusively below 6500 ms, while on Orin it is exclusive below 2750 ms and selected in approximately $K$0–$K$1 of frames at larger SLOs. Third, using only $K$2–$K$3 of historical frames yields similar mAP, and extracting historical objects with uniform tiling and the best model performs similarly to ground-truth extraction. This suggests that representativeness of the historical distribution matters more than the volume of historical frames.

Latency miss-rate analysis further differentiates Uirapuru from static-camera baselines. Conservative mode nearly eliminates misses, with Xavier miss rates typically between $K$4 and $K$5 and Orin between $K$6 and $K$7. Non-conservative mode remains low but higher, with Xavier ranging from $K$8 to $K$9 depending on SLO and Orin from $R(\theta_t,\phi_t)$0 to $R(\theta_t,\phi_t)$1. Remix exhibits substantially higher miss rates at tight SLOs, including up to $R(\theta_t,\phi_t)$2 at 750 ms on Xavier and $R(\theta_t,\phi_t)$3 at 500 ms on Orin. These results reinforce the paper’s central claim that per-frame actuation-aware replanning is more reliable than static-camera planning assumptions under PTZ motion.

7. Limitations, deployment conditions, and broader significance

Uirapuru’s limitations are explicit. It assumes fixed mounting, so UAVs or mobile cameras would require 6-DoF modeling and more complex distribution control. Its homography-based reprojection uses only 2D boxes and does not explicitly handle occlusions or lens distortion. Historical frames must represent the global scene adequately; stale histories only slightly reduce accuracy when distributions remain similar, but larger changes in object density can degrade performance. Memory can also become a constraint, since loading large models such as D5 and D6 may exceed the capacity of smaller edge devices. The framework does not implement explicit runtime tile skipping; instead, skipping emerges implicitly through DP-DP’s tile selection. Under aggressive zoom, tiling may become suboptimal, and the system may shift toward few large tiles with large models, effectively approaching down-sampling behavior (Apostolo et al., 1 Sep 2025).

Deployment guidance in the source targets traffic control and crowd monitoring. Recommended practice includes collecting one loop of wide-angle shots as historical frames, using quad-tree depth 3 for 4K, selecting conservative mode for strict per-frame deadlines and non-conservative mode for throughput-oriented operation, loading a broad model pool such as D0–D6 when memory permits, applying uniform padding with an initial range of 16–32 pixels, synchronizing actuation telemetry with frame timestamps, calibrating camera intrinsics $R(\theta_t,\phi_t)$4, limiting excessive zoom, and periodically refreshing historical frames when scenes change significantly. The source also notes that when actuation is minimal, reusing the previous plan or reducing planning frequency may reduce overhead.

The broader significance of Uirapuru lies in its formulation of steerable-camera analytics as a coupled geometry-and-scheduling problem. The work identifies a failure mode of static-camera assumptions under PTZ, introduces an actuation-aware distribution controller and a relative non-linear profiling scheme adapted to wide object-scale variability, and combines these with a hierarchical dynamic program that enforces latency SLOs frame by frame. The paper characterizes Uirapuru as the first end-to-end, actuation-aware, edge-based real-time analytics framework tailored for high-resolution steerable PTZ cameras. This suggests a methodological shift from historically optimized but view-static planning to continual replanning under explicit camera-state dynamics, with future extensions anticipated in deep PTZ calibration, robust skipping, memory and energy optimization, and multi-camera actuation-aware analytics.