OnFly: Online Processing in Real-Time Systems

Updated 4 July 2026

OnFly is a systems paradigm that enforces immediate data processing and state updates during data acquisition, diverging from batch methods.
It integrates real-time inference, optimization, and control across applications like Structure-from-Motion, aerial navigation, and large language model serving.
OnFly frameworks employ dynamic, local decision-making strategies to balance latency, resource constraints, and system robustness in distributed environments.

Searching arXiv for the specific "OnFly" entries and closely related papers to ground the article in current records. OnFly is a reused research term rather than a single canonical system. In recent arXiv literature it denotes, or is used as shorthand for, several online or on-the-fly computational frameworks whose common premise is that inference, optimization, communication, or control should occur during data acquisition or service execution rather than after a static preprocessing stage. Prominent instantiations include online Structure-from-Motion during image capture (Zhan et al., 2023), fully onboard zero-shot aerial vision-language navigation (Zheng et al., 11 Mar 2026), live data-parallel to tensor-parallel switching for LLM serving (Gao et al., 26 Feb 2026), and per-round flying-master selection for federated learning in heterogeneous edge settings (Nguyen et al., 2021). A related lineage appears in on-the-fly distributed tensor decomposition over MIMO channels, where progressive sketch aggregation replaces one-shot decomposition (Chen et al., 2023). This suggests that “OnFly” functions less as a domain-specific algorithm name than as a recurring systems principle: immediate processing under online constraints.

1. Terminological scope and recurring meaning

Across the cited works, the term consistently marks a departure from deferred, batch, or deployment-time processing. In each case, a system receives new observations, requests, or local updates and must produce an actionable state before the next event or scheduling boundary. The technical implementations differ sharply, but the design pressure is similar: bounded latency, incremental state maintenance, and local rather than global optimization (Zhan et al., 2023).

Usage	Domain	Core online mechanism
"On-the-Fly SfM" (Zhan et al., 2023)	3D vision	Immediate image registration and local BA during capture
"OnFly" (Zheng et al., 11 Mar 2026)	Aerial robotics	Dual-agent onboard AVLN with verifier and RHP
"Flying Serving" (Gao et al., 26 Feb 2026)	LLM systems	Online DP↔TP switching without restarting workers
OnFly shorthand (Nguyen et al., 2021)	Federated learning	Per-round flying-master election
"FlyCom $^2$ " (Chen et al., 2023)	Wireless edge analytics	Streaming random sketches with progressive AirComp

The literature also contains naming collisions. OpenFly is a distinct aerial VLN platform and agent, and the paper explicitly notes that it is “sometimes misread as ‘OnFly’” (Gao et al., 25 Feb 2025). That distinction matters because OpenFly is primarily a benchmark and toolchain, whereas the OnFly systems are online execution frameworks.

2. OnFly as online Structure-from-Motion

In "On-the-Fly SfM: What you capture is What you get" (Zhan et al., 2023), OnFly denotes an online SfM paradigm in which each newly captured image is processed immediately upon arrival. The pipeline retrieves the new image against previously registered images, performs robust local feature matching and multi-model geometric verification, initializes pose through PnP with $2\text{D}$ – $3\text{D}$ correspondences or epipolar geometry when only $2\text{D}$ – $2\text{D}$ matches are available, triangulates visible $3\text{D}$ points, updates tracks and covisibility, and then refines local states by hierarchical weighted local bundle adjustment. The defining claim is operational rather than merely algorithmic: the current image’s pose and points are output before the next image arrives (Zhan et al., 2023).

A central component is fast global retrieval through a vocabulary tree trained unsupervised on learning-based global features. The implementation uses a fine-tuned multi-NetVLAD model from Hou et al., hierarchical $k$ -means tree construction, and TF–IDF scoring over an inverted index. Retrieval returns top- $N$ candidates for local matching; in the reported implementation, the vocabulary tree uses depth $L=5$ and branching factor $k=5$ , each new image retrieves top-30 candidates, and only top-8 similar images per node are retained for BA window control (Zhan et al., 2023).

Local registration combines SIFT matching, RANSAC-based estimation of $2\text{D}$ 0, $2\text{D}$ 1, or $2\text{D}$ 2, GRIC-based model selection, and Least Squares Matching. LSM is used both to improve $2\text{D}$ 3 positional accuracy and suppress outliers, and to densify $2\text{D}$ 4– $2\text{D}$ 5 observations on the new image. The photometric refinement is formulated over a local window, with a first-order radiometric model and an affine warp, and is solved by Gauss–Newton or Levenberg–Marquardt. Pose estimation then prioritizes LSM-refined $2\text{D}$ 6– $2\text{D}$ 7 matches inside EPnP-in-RANSAC (Zhan et al., 2023).

The optimization stage is explicitly local and hierarchical. Neighboring images are selected from retrieval and covisibility graphs, layers are weighted according to

$2\text{D}$ 8

and the BA uses inverse weighting so that nearer layers adapt more strongly while farther layers are regularized. The weighted LM system is solved with Schur complement reduction. This is intended to preserve the accuracy of local refinement while preventing the per-image cost of global BA (Zhan et al., 2023).

Empirically, the method is validated on two self-collected unordered datasets, SX with 221 images and YX with 349 images, together with simulated online runs on TUM sequences fr1_desk, fr1_xyz, and fr3_st_far. The reported outcomes are that vocabulary-tree retrieval is consistently faster than exhaustive Euclidean comparison of global descriptors and Colmap’s exhaustive local-feature-based matching; weighted local BA remains low and stabilizes as the map grows; mean reprojection error is typically $2\text{D}$ 9 px; mean track length is comparable to baselines; and OpenMVG fails on some unordered sets where OnFly and Colmap succeed (Zhan et al., 2023). The paper also stresses a conceptual boundary: unlike SLAM, the method does not require spatiotemporal continuity.

In "OnFly: Onboard Zero-Shot Aerial Vision-Language Navigation toward Safety and Efficiency" (Zheng et al., 11 Mar 2026), OnFly is a fully onboard, real-time framework for zero-shot AVLN. The UAV receives multimodal inputs

$3\text{D}$ 0

and the objective is to generate a safe and efficient trajectory that satisfies the natural-language instruction, reaches the goal, and avoids obstacles and collisions (Zheng et al., 11 Mar 2026).

The architecture is organized around a shared-perception dual-agent design. The Target Generation Agent runs at higher frequency and predicts the next image-space target pixel $3\text{D}$ 1, while the Progress Monitoring Agent runs at lower frequency and outputs task-state signals $3\text{D}$ 2. Both agents share a ViT backbone and cached visual features, but each maintains an independent KV-cache matched to its own prompt structure and operating rate. The stated motivation is to avoid the disparate-frequency interference and objective conflicts of single-stream VLM systems (Zheng et al., 11 Mar 2026).

Long-horizon monitoring is handled by a hybrid keyframe–recent-frame memory

$3\text{D}$ 3

where $3\text{D}$ 4 is the initial observation, $3\text{D}$ 5 is an ordered keyframe list, and $3\text{D}$ 6 is the most recent frame. Keyframes are de-duplicated by cosine similarity, organized by odometry distance into $3\text{D}$ 7 segments, and serialized so that the prompt prefix remains stable for KV-cache reuse. In the reported configuration, $3\text{D}$ 8 (Zheng et al., 11 Mar 2026).

Target refinement is delegated to a semantic–geometric verifier. Semantic consistency is scored with VLM features, depth feasibility is enforced after obstacle dilation, and the final refined pixel $3\text{D}$ 9 is selected from the intersection of semantic locality and safe geometry. A bearing-aware gating stage then modulates forward range as a function of target bearing and camera intrinsics before lifting the target into odometry coordinates. The refined target is passed to an ESDF-constrained receding-horizon planner that minimizes tracking deviation, control effort, control smoothness, and yaw misalignment under dynamics and clearance constraints (Zheng et al., 11 Mar 2026).

The framework is explicitly zero-shot. It uses Qwen3-VL-4B-AWQ as the default backbone, with ablations for 2B and 30B variants, and no task-specific finetuning. Deployment-time optimizations include AWQ quantization, FP16 ViT execution, TensorRT acceleration, CUDA Graphs, cached image features, and KV-cache reuse. On Jetson Orin NX, the reported per-step latencies are $2\text{D}$ 0 s for the Decision Agent and $2\text{D}$ 1 s for the Monitoring Agent with hybrid memory, compared with $2\text{D}$ 2 s and $2\text{D}$ 3 s for baseline Transformers (Zheng et al., 11 Mar 2026).

The main simulation benchmark spans ten Unreal Engine 4.27 scenes and 150 tasks. Against the strongest baseline, SPF, task success rises from $2\text{D}$ 4 to $2\text{D}$ 5, collision rate drops from $2\text{D}$ 6 to $2\text{D}$ 7, and flight time decreases from $2\text{D}$ 8 s to $2\text{D}$ 9 s. Ablations show degradation without the dual-agent design, without verification, and especially without the planner. Real-world flights further validate pedestrian tracking, long-range corridor navigation, precise outdoor navigation, and multi-level navigation with obstacle avoidance, all executed onboard on the Orin NX platform (Zheng et al., 11 Mar 2026).

4. OnFly in LLM serving

In "FLYING SERVING: On-the-Fly Parallelism Switching for LLM Serving" (Gao et al., 26 Feb 2026), OnFly refers to a vLLM-based runtime that can switch between data parallelism and tensor parallelism online, without restarting engine workers. The stated motivation is that static DP-only and TP-only deployments are each suboptimal under non-stationary traffic: DP maximizes concurrency during bursts, whereas TP reduces per-request latency and pools KV memory for long-context inference (Gao et al., 26 Feb 2026).

The implementation rests on four components. The zero-copy Model Weights Manager exposes logical TP shard views on resident full weights, so switching activates rank-aware tensor views rather than moving or reloading tensors. The KV Cache Adaptor keeps physical blocks fixed-size while changing logical capacity and stride with TP degree; with local hidden width $2\text{D}$ 0, block capacity scales as $2\text{D}$ 1, which keeps block memory invariant. The Communicator Pool eagerly initializes NCCL groups for supported contiguous device sets. The scheduler coordinates safe transitions and matched collectives so that execution skew does not induce deadlock (Gao et al., 26 Feb 2026).

The system exposes runtime DP↔TP binding as a scheduling primitive driven by queue pressure, priority, and context-length or memory pressure. During bursts it can remain DP-like to suppress queue growth; under low load or for latency-sensitive or long-context requests it can merge engines into TP groups. Switching occurs at scheduler iteration boundaries and completes in $2\text{D}$ 2 ms live, compared with $2\text{D}$ 3– $2\text{D}$ 4 s for cold restarts to a different TP degree (Gao et al., 26 Feb 2026).

Across Llama-3-70B, GPT-OSS-120B, and Nemotron-8B, the reported gains reach up to $2\text{D}$ 5 under high load and $2\text{D}$ 6 under low load. Peak throughput retains approximately $2\text{D}$ 7– $2\text{D}$ 8 of static DP while substantially outperforming static TP, and dynamic merging extends maximum context length on Llama-3-70B to $2\text{D}$ 9M tokens, compared with $3\text{D}$ 0K for static $3\text{D}$ 1 and $3\text{D}$ 2K for $3\text{D}$ 3 (Gao et al., 26 Feb 2026). This makes OnFly a serving-time control layer rather than a model-architecture change.

5. Distributed and edge interpretations

A related but not identical usage appears in "On-the-Fly Communication-and-Computing for Distributed Tensor Decomposition Over MIMO Channels" (Chen et al., 2023). There the formal system name is FlyCom $3\text{D}$ 4, not OnFly, yet the paper is explicitly about on-the-fly streaming computation. Devices generate Gaussian random sketches $3\text{D}$ 5, upload them progressively through MIMO AirComp, and the server accumulates and whitens received sketches to obtain an effective observation $3\text{D}$ 6. The online subspace estimator is then

$3\text{D}$ 7

Both deterministic and probabilistic analyses show decomposition error decays as $3\text{D}$ 8, and threshold-based sketch selection improves reliability under noise (Chen et al., 2023). This suggests a broader systems interpretation of OnFly: computation is decomposed into streaming low-cost updates rather than one-shot heavy local eigendecompositions.

In federated learning, "On-the-fly Resource-Aware Model Aggregation for Federated Learning in Heterogeneous Edge" uses a fixed phrase—“flying master”—and the provided summary explicitly introduces OnFly as shorthand for the per-round master-election framework (Nguyen et al., 2021). Instead of a permanent server, each FL round elects a master from either the current participants or all devices. Election uses least-distance or least-stress metrics through gossip-based min-consensus, or a proof-of-work race when resource sharing is undesirable. The underlying FL objective and FedAvg aggregation remain standard; only the aggregator placement changes (Nguyen et al., 2021).

The reported benefit is a reduction in wall-clock round time on heterogeneous edge hardware and real 5G traces. The paper states that the fixed central server can be up to $3\text{D}$ 9 slower than Optimal, Least-stress, and PoW-based selections, especially for small participant sets, and that least-stress via gossip and PoW closely match the oracle selector in total time (Nguyen et al., 2021). Here OnFly denotes dynamic aggregation topology rather than online model adaptation.

6. Naming collisions, adjacent usages, and technical boundaries

A persistent misconception is that OnFly and OpenFly are interchangeable. They are not. OpenFly is a comprehensive aerial VLN platform comprising rendering engines, a toolchain, a dataset with $k$ 0 trajectories across $k$ 1 scenes, and OpenFly-Agent; the paper states that OpenFly is “sometimes misread as ‘OnFly,’” but it remains a separate name and a different contribution profile (Gao et al., 25 Feb 2025).

A second boundary concerns descriptive rather than formal naming. "Dom, cars don't fly! -- Or do they? In-Air Vehicle Maneuver for High-Speed Off-Road Navigation" does not present a system formally named OnFly, but the provided explanation uses “OnFly” descriptively for airborne maneuvering with throttle and steering during off-road jumps (Pokhrel et al., 24 Mar 2025). The actual method is a hybrid forward kinodynamic model with a fixed-horizon, sampling-based planner and state-dependent bounds on $k$ 2 and $k$ 3, evaluated on a $k$ 4-scale buggy. A plausible implication is that “OnFly” has become a generic label for timing-critical online adjustment even when the paper’s official title uses different nomenclature.

Across these works, the common pattern is precise but narrow. OnFly systems are not merely “real-time” in a loose sense; they explicitly restructure pipelines so that state is updated online and immediately exploited. In SfM, the image is reconstructed before the next frame arrives (Zhan et al., 2023). In AVLN, target generation, verification, and replanning run fully onboard during flight (Zheng et al., 11 Mar 2026). In serving, DP↔TP reconfiguration occurs live without worker restart (Gao et al., 26 Feb 2026). In federated learning and wireless tensor decomposition, aggregation itself becomes progressive and topology-aware (Nguyen et al., 2021, Chen et al., 2023). The term therefore identifies a systems stance toward latency, incremental state, and bounded local decision windows rather than a single transferable algorithm.