---
title: 'OnFly: Online Processing in Real-Time Systems'
url: https://www.emergentmind.com/topics/onfly
type: topic
---

# OnFly: Online Processing in Real-Time Systems

Searching arXiv for the specific "OnFly" entries and closely related papers to ground the article in current records.
OnFly is a reused research term rather than a single canonical system. In recent arXiv literature it denotes, or is used as shorthand for, several online or on-the-fly computational frameworks whose common premise is that inference, optimization, communication, or control should occur during data acquisition or service execution rather than after a static preprocessing stage. Prominent instantiations include online Structure-from-Motion during image capture [2309.11883], fully onboard zero-shot aerial vision-language navigation [2603.10682], live data-parallel to tensor-parallel switching for large language model serving [2602.22593], and per-round flying-master selection for federated learning in heterogeneous edge settings [2112.11485]. A related lineage appears in on-the-fly distributed tensor decomposition over MIMO channels, where progressive sketch aggregation replaces one-shot decomposition [2302.14297]. This suggests that “OnFly” functions less as a domain-specific algorithm name than as a recurring systems principle: immediate processing under online constraints.

## 1. Terminological scope and recurring meaning

Across the cited works, the term consistently marks a departure from deferred, batch, or deployment-time processing. In each case, a system receives new observations, requests, or local updates and must produce an actionable state before the next event or scheduling boundary. The technical implementations differ sharply, but the design pressure is similar: bounded latency, incremental state maintenance, and local rather than global optimization [2309.11883].

| Usage | Domain | Core online mechanism |
|---|---|---|
| "On-the-Fly SfM" [2309.11883] | 3D vision | Immediate image registration and local BA during capture |
| "OnFly" [2603.10682] | Aerial robotics | Dual-agent onboard AVLN with verifier and RHP |
| "Flying Serving" [2602.22593] | LLM systems | Online DP↔TP switching without restarting workers |
| OnFly shorthand [2112.11485] | Federated learning | Per-round flying-master election |
| "FlyCom$^2$" [2302.14297] | Wireless edge analytics | Streaming random sketches with progressive AirComp |

The literature also contains naming collisions. OpenFly is a distinct aerial VLN platform and agent, and the paper explicitly notes that it is “sometimes misread as ‘OnFly’” [2502.18041]. That distinction matters because OpenFly is primarily a benchmark and toolchain, whereas the OnFly systems are online execution frameworks.

## 2. OnFly as online Structure-from-Motion

In "On-the-Fly SfM: What you capture is What you get" [2309.11883], OnFly denotes an online SfM paradigm in which each newly captured image is processed immediately upon arrival. The pipeline retrieves the new image against previously registered images, performs robust local feature matching and multi-model geometric verification, initializes pose through PnP with \(2\text{D}\)–\(3\text{D}\) correspondences or epipolar geometry when only \(2\text{D}\)–\(2\text{D}\) matches are available, triangulates visible \(3\text{D}\) points, updates tracks and covisibility, and then refines local states by hierarchical weighted local bundle adjustment. The defining claim is operational rather than merely algorithmic: the current image’s pose and points are output before the next image arrives [2309.11883].

A central component is fast global retrieval through a vocabulary tree trained unsupervised on learning-based global features. The implementation uses a fine-tuned multi-NetVLAD model from Hou et al., hierarchical \(k\)-means tree construction, and TF–IDF scoring over an inverted index. Retrieval returns top-\(N\) candidates for local matching; in the reported implementation, the vocabulary tree uses depth \(L=5\) and branching factor \(k=5\), each new image retrieves top-30 candidates, and only top-8 similar images per node are retained for BA window control [2309.11883].

Local registration combines SIFT matching, RANSAC-based estimation of \(E\), \(F\), or \(H\), GRIC-based model selection, and Least Squares Matching. LSM is used both to improve \(2\text{D}\) positional accuracy and suppress outliers, and to densify \(2\text{D}\)–\(3\text{D}\) observations on the new image. The photometric refinement is formulated over a local window, with a first-order radiometric model and an affine warp, and is solved by Gauss–Newton or Levenberg–Marquardt. Pose estimation then prioritizes LSM-refined \(2\text{D}\)–\(3\text{D}\) matches inside EPnP-in-RANSAC [2309.11883].

The optimization stage is explicitly local and hierarchical. Neighboring images are selected from retrieval and covisibility graphs, layers are weighted according to
\[
p_i = k^{i-1},
\]
and the BA uses inverse weighting so that nearer layers adapt more strongly while farther layers are regularized. The weighted LM system is solved with Schur complement reduction. This is intended to preserve the accuracy of local refinement while preventing the per-image cost of global BA [2309.11883].

Empirically, the method is validated on two self-collected unordered datasets, SX with 221 images and YX with 349 images, together with simulated online runs on TUM sequences fr1_desk, fr1_xyz, and fr3_st_far. The reported outcomes are that vocabulary-tree retrieval is consistently faster than exhaustive Euclidean comparison of global descriptors and Colmap’s exhaustive local-feature-based matching; weighted local BA remains low and stabilizes as the map grows; mean reprojection error is typically \(< 1\) px; mean track length is comparable to baselines; and OpenMVG fails on some unordered sets where OnFly and Colmap succeed [2309.11883]. The paper also stresses a conceptual boundary: unlike SLAM, the method does not require spatiotemporal continuity.

## 3. OnFly in onboard aerial vision-language navigation

In "OnFly: Onboard Zero-Shot Aerial Vision-Language Navigation toward Safety and Efficiency" [2603.10682], OnFly is a fully onboard, real-time framework for zero-shot AVLN. The UAV receives multimodal inputs
\[
O_t = \{ I_t, D_t, pose_t, L \},
\]
and the objective is to generate a safe and efficient trajectory that satisfies the natural-language instruction, reaches the goal, and avoids obstacles and collisions [2603.10682].

The architecture is organized around a shared-perception dual-agent design. The Target Generation Agent runs at higher frequency and predicts the next image-space target pixel \((p_x,p_y)\), while the Progress Monitoring Agent runs at lower frequency and outputs task-state signals \(s_t \in \{\text{CONTINUE}, \text{STOP}, \text{LOST}\}\). Both agents share a ViT backbone and cached visual features, but each maintains an independent KV-cache matched to its own prompt structure and operating rate. The stated motivation is to avoid the disparate-frequency interference and objective conflicts of single-stream VLM systems [2603.10682].

Long-horizon monitoring is handled by a hybrid keyframe–recent-frame memory
\[
M_t = [o_1, Q_t, o_t],
\]
where \(o_1\) is the initial observation, \(Q_t\) is an ordered keyframe list, and \(o_t\) is the most recent frame. Keyframes are de-duplicated by cosine similarity, organized by odometry distance into \(S\) segments, and serialized so that the prompt prefix remains stable for KV-cache reuse. In the reported configuration, \(S=4\) [2603.10682].

Target refinement is delegated to a semantic–geometric verifier. Semantic consistency is scored with VLM features, depth feasibility is enforced after obstacle dilation, and the final refined pixel \(p'\) is selected from the intersection of semantic locality and safe geometry. A bearing-aware gating stage then modulates forward range as a function of target bearing and camera intrinsics before lifting the target into odometry coordinates. The refined target is passed to an ESDF-constrained receding-horizon planner that minimizes tracking deviation, control effort, control smoothness, and yaw misalignment under dynamics and clearance constraints [2603.10682].

The framework is explicitly zero-shot. It uses Qwen3-VL-4B-AWQ as the default backbone, with ablations for 2B and 30B variants, and no task-specific finetuning. Deployment-time optimizations include AWQ quantization, FP16 ViT execution, TensorRT acceleration, CUDA Graphs, cached image features, and KV-cache reuse. On Jetson Orin NX, the reported per-step latencies are \(0.81\) s for the Decision Agent and \(1.15\) s for the Monitoring Agent with hybrid memory, compared with \(3.83\) s and \(7.47\) s for baseline Transformers [2603.10682].

The main simulation benchmark spans ten Unreal Engine 4.27 scenes and 150 tasks. Against the strongest baseline, SPF, task success rises from \(26.4\%\) to \(67.8\%\), collision rate drops from \(42.7\%\) to \(2.7\%\), and flight time decreases from \(39.2\) s to \(27.1\) s. Ablations show degradation without the dual-agent design, without verification, and especially without the planner. Real-world flights further validate pedestrian tracking, long-range corridor navigation, precise outdoor navigation, and multi-level navigation with obstacle avoidance, all executed onboard on the Orin NX platform [2603.10682].

## 4. OnFly in large language model serving

In "FLYING SERVING: On-the-Fly Parallelism Switching for Large Language Model Serving" [2602.22593], OnFly refers to a vLLM-based runtime that can switch between data parallelism and tensor parallelism online, without restarting engine workers. The stated motivation is that static DP-only and TP-only deployments are each suboptimal under non-stationary traffic: DP maximizes concurrency during bursts, whereas TP reduces per-request latency and pools KV memory for long-context inference [2602.22593].

The implementation rests on four components. The zero-copy Model Weights Manager exposes logical TP shard views on resident full weights, so switching activates rank-aware tensor views rather than moving or reloading tensors. The KV Cache Adaptor keeps physical blocks fixed-size while changing logical capacity and stride with TP degree; with local hidden width \(D_{\text{local}}(p)=D/p\), block capacity scales as \(B(p)=p\times B_{\text{base}}\), which keeps block memory invariant. The Communicator Pool eagerly initializes NCCL groups for supported contiguous device sets. The scheduler coordinates safe transitions and matched collectives so that execution skew does not induce deadlock [2602.22593].

The system exposes runtime DP↔TP binding as a scheduling primitive driven by queue pressure, priority, and context-length or memory pressure. During bursts it can remain DP-like to suppress queue growth; under low load or for latency-sensitive or long-context requests it can merge engines into TP groups. Switching occurs at scheduler iteration boundaries and completes in \(15\) ms live, compared with \(146\)–\(292\) s for cold restarts to a different TP degree [2602.22593].

Across Llama-3-70B, GPT-OSS-120B, and Nemotron-8B, the reported gains reach up to \(4.79\times\) under high load and \(3.47\times\) under low load. Peak throughput retains approximately \(95\%\)–\(96\%\) of static DP while substantially outperforming static TP, and dynamic merging extends maximum context length on Llama-3-70B to \(1.9\)M tokens, compared with \(264\)K for static \(4\text{DP}\times2\text{TP}\) and \(959\)K for \(2\text{DP}\times4\text{TP}\) [2602.22593]. This makes OnFly a serving-time control layer rather than a model-architecture change.

## 5. Distributed and edge interpretations

A related but not identical usage appears in "On-the-Fly Communication-and-Computing for Distributed Tensor Decomposition Over MIMO Channels" [2302.14297]. There the formal system name is FlyCom\(^2\), not OnFly, yet the paper is explicitly about on-the-fly streaming computation. Devices generate Gaussian random sketches \(S_{t,k}=X_kL_{t,k}\), upload them progressively through MIMO AirComp, and the server accumulates and whitens received sketches to obtain an effective observation \(P_t\). The online subspace estimator is then
\[
\widehat{U}_t = \mathsf{S}_r(P_tP_t^T).
\]
Both deterministic and probabilistic analyses show decomposition error decays as \(1/(tM)\), and threshold-based sketch selection improves reliability under noise [2302.14297]. This suggests a broader systems interpretation of OnFly: computation is decomposed into streaming low-cost updates rather than one-shot heavy local eigendecompositions.

In federated learning, "On-the-fly Resource-Aware Model Aggregation for Federated Learning in Heterogeneous Edge" uses a fixed phrase—“flying master”—and the provided summary explicitly introduces OnFly as shorthand for the per-round master-election framework [2112.11485]. Instead of a permanent server, each FL round elects a master from either the current participants or all devices. Election uses least-distance or least-stress metrics through gossip-based min-consensus, or a proof-of-work race when resource sharing is undesirable. The underlying FL objective and FedAvg aggregation remain standard; only the aggregator placement changes [2112.11485].

The reported benefit is a reduction in wall-clock round time on heterogeneous edge hardware and real 5G traces. The paper states that the fixed central server can be up to \(4\times\) slower than Optimal, Least-stress, and PoW-based selections, especially for small participant sets, and that least-stress via gossip and PoW closely match the oracle selector in total time [2112.11485]. Here OnFly denotes dynamic aggregation topology rather than online model adaptation.

## 6. Naming collisions, adjacent usages, and technical boundaries

A persistent misconception is that OnFly and OpenFly are interchangeable. They are not. OpenFly is a comprehensive aerial VLN platform comprising rendering engines, a toolchain, a dataset with \(100{,}000\) trajectories across \(18\) scenes, and OpenFly-Agent; the paper states that OpenFly is “sometimes misread as ‘OnFly,’” but it remains a separate name and a different contribution profile [2502.18041].

A second boundary concerns descriptive rather than formal naming. "Dom, cars don't fly! -- Or do they? In-Air Vehicle Maneuver for High-Speed Off-Road Navigation" does not present a system formally named OnFly, but the provided explanation uses “OnFly” descriptively for airborne maneuvering with throttle and steering during off-road jumps [2503.19140]. The actual method is a hybrid forward kinodynamic model with a fixed-horizon, sampling-based planner and state-dependent bounds on \(\dot{rpm}\) and \(\dot{\psi}\), evaluated on a \(1/5\)-scale buggy. A plausible implication is that “OnFly” has become a generic label for timing-critical online adjustment even when the paper’s official title uses different nomenclature.

Across these works, the common pattern is precise but narrow. OnFly systems are not merely “real-time” in a loose sense; they explicitly restructure pipelines so that state is updated online and immediately exploited. In SfM, the image is reconstructed before the next frame arrives [2309.11883]. In AVLN, target generation, verification, and replanning run fully onboard during flight [2603.10682]. In serving, DP↔TP reconfiguration occurs live without worker restart [2602.22593]. In federated learning and wireless tensor decomposition, aggregation itself becomes progressive and topology-aware [2112.11485][2302.14297]. The term therefore identifies a systems stance toward latency, incremental state, and bounded local decision windows rather than a single transferable algorithm.

Source: https://www.emergentmind.com/topics/onfly