See, Point, Fly: Spatial Action Paradigm

• SPF is a paradigm that integrates sensory observation, explicit spatial grounding, and dynamic action to drive robotics, vision-language navigation, and point cloud forecasting.
• Its architecture employs modules like pre-trained vision–language models for 2D-to-3D waypoint projection, Gaussian Process Regression for distributed fusion, and recurrent cost-volume networks for sequential prediction.
• Empirical results demonstrate high success rates in UAV navigation and significant error reduction in forecasting tasks, confirming SPF's scalability and efficacy in real-world applications.

The See, Point, Fly (SPF) paradigm encompasses a family of approaches integrating observation, spatial grounding or representation, and dynamic action in diverse domains. In recent literature, “See, Point, Fly” designates deterministic control and perception pipelines in vision-language navigation (Hu et al., 26 Sep 2025), distributed point-cloud fusion for edge AI (Chen et al., 2024), and multi-step prediction in spatiotemporal point cloud modeling (He et al., 2021). This article surveys the core methodologies and implications of SPF systems, focusing on technical architecture, algorithmic components, and empirical results.

1. Conceptual Underpinnings and Scope

The SPF paradigm formalizes a three-phase process:

1. See: Sensing or observation using input modalities such as RGB images, point clouds, or multi-modal sensor arrays.
2. Point: Spatial grounding or representation—either as explicit (2D/3D) waypoints, attribution of 3D spatial structure, or distributed feature estimation.
3. Fly: On-the-fly decision-making or streaming actions, involving either physical actuation (robotics, UAVs), in-network computation, or future state prediction.

Distinct instantiations of SPF each employ this sequence, but differ in their technical implementations and scientific objectives. In aerial vision-language navigation, SPF directs UAVs via zero-shot vision-language grounding (Hu et al., 26 Sep 2025). In distributed point cloud learning, SPF describes a pipeline for low-latency, progressive fusion across edge devices and network infrastructure (Chen et al., 2024). For scene dynamics, SPF denotes the sequential prediction of future point cloud frames (He et al., 2021).

2. Vision-Language Navigation via SPF

The SPF system for universal unmanned aerial navigation (Hu et al., 26 Sep 2025) epitomizes a deterministic, training-free control framework atop pre-trained vision–LLMs (VLMs). The navigation policy is formalized as

$\pi(\ell, I_t)\;\longrightarrow\;m_t,$

where $\ell$ is a free-form textual instruction, $I_t$ the current image, and $m_t$ a 3D motion command.

Spatial Grounding and Action

Action selection operates as a 2D spatial grounding task:

• The VLM $G$ predicts candidate 2D waypoints $(u,v)$ on $I_t$ for each instruction $\ell$,

$$O_t = \arg\max_{w\in\mathcal W} P_G(w \mid \ell, I_t) = \{u, v, d_{\rm VLM}\},$$

where $d_{\rm VLM}$ is a discretized step-length estimator.

Waypoints are unprojected into 3D UAV body-frame displacements: $\ell$0 with $\ell$1 the camera intrinsic matrix and $\ell$2 an adaptively scaled distance.

Controller primitives are derived per cycle: $\ell$3 A notable feature is real-time step-size adaptation: $\ell$4 enabling coarse global planning and fine obstacle negotiation.

Evaluation and Generality

SPF exhibits high generalization and robustness:

• In simulation (23 tasks), SPF achieved a 93.9% success rate (SR), substantially exceeding TypeFly (0.9%) and PIVOT (28.7%).
• Real-world experiments (11 tasks) yielded 92.7% SR, compared to TypeFly's 23.6% and PIVOT's 5.5%.
• Ablations reveal the superiority of explicit 2D waypoint labeling versus text-based policies.
• SPF is interface-agnostic, performing strongly across modern VLMs (100% SR on Gemini 2.0 Flash, Gemini 2.5 Pro, GPT-4.1).

3. On-the-Fly Distributed Point Cloud Fusion

Within the Integrated Sensing and Edge AI (ISEA) framework, SPF is realized as the FlyCom$\ell$5 system (Chen et al., 2024), designed for distributed 3D representation learning:

• See: Edge devices (cameras, LiDAR, etc.) sense and generate local occupancy and attribute measurements.
• Point: At the server, a global attribute field is constructed via Gaussian Process Regression (GPR) using streams of projected local observations.
• Fly: On-the-fly local projection with low-latency, over-the-air aggregation (AirComp), optimizing communication and computation.

Modules and Algorithmic Details

• Each device $\ell$6 encodes occupancy via octree traversal, sending bit sequences $\ell$7 to reconstruct shapes at the server.
• Local attributes $\ell$8 are projected: $\ell$9 with $I_t$0, and $I_t$1 adaptively optimized to maximize predictive value at the fusion center.
• AirComp fuses attribute projections over a fading SIMO uplink using an optimized combiner $I_t$2, yielding an unbiased estimate robust to channel noise.
• The global field estimator is the minimum-MSE predictor: $I_t$3 with covariances calculated from device observation models and channel statistics.
• Progress is measured by reduction in MSE: $I_t$4 with provable convergence as observation streams grow.

Joint Optimization

The system alternates between optimizing the projection weights $I_t$5 and the receive combiner $I_t$6, guided by a generalized eigenproblem maximizing information gain, subject to communication constraints.

A distinguishing aspect is the fully progressive, on-the-fly data upload and fusion, tightly integrating sensing, representation learning, and communication.

4. Sequential Point-Cloud Forecasting (SPF) in Scene Dynamics

SPF is also defined as Sequential Point-cloud Forecasting in the context of 4D point cloud modeling (He et al., 2021). The target is to learn $I_t$7 to predict future frames $I_t$8, minimizing

$I_t$9

where $m_t$0 is mean squared error, Chamfer Distance, or Earth Mover’s Distance, depending on the supervision regime.

SPCM-Net Architecture

• Intra-frame Feature Pyramid (IFFP): Multi-scale geometric encoding at each temporal step via FPS sampling and local PointConv.
• Inter-frame Spatio-temporal Correlation (IFSC): Recurrent cost-volume units encode cross-frame matchings, updating hidden/cell states in a permutation-invariant, LSTM-like block, but replacing linear layers with cost-volume operators.
• Multi-scale Coarse-to-Fine Prediction: Recursively predicts displacements at multiple levels; for SPF, each forecast step $m_t$1 computes a displacement $m_t$2 and uses recurrent rollout for sequence prediction.

Benchmarks and Metrics

Synthetic and real-world datasets include SFT3D, Virtual KITTI Sequence (VKS), and Sequential Argoverse (SAG). Metrics comprise Average Displacement Error (ADE), Final Displacement Error (FDE), and per-frame divergence measures.

Key Results

• On SFT3D, SPCM-Net pretrained on sequential scene flow (SSFE) achieves ADE $m_t$3, FDE $m_t$4, roughly halving the error seen from random initialization.
• On VKS, SPCM-Net is competitive with PointRNN (ADE $m_t$5).
• On SAG, Sinkhorn Distance (SD) is used for self-supervised evaluation: SPCM-Net SD $m_t$6, outperformed slightly by PointRNN ($m_t$7).

This suggests that set-to-set correlation via cost-volume layers with permutation-invariant sequence modeling remains advantageous for capturing dynamic 3D structure.

5. On-the-Fly Point Feature Representation in Point Cloud Analysis

While not an “SPF” acronym, recent work on On-the-fly Point Feature Representation (OPFR) (Wang et al., 2024) further contextualizes the demand for instantaneous geometric feature estimation in point cloud pipelines:

• CFGen module emulates Point Feature Histogram (PFH) using efficient local coordinate construction via Local Reference Constructor (LRCon).
• Hierarchical Sampling ensures robust triangle selection for geometric descriptors.
• When integrated with point-based backbones (PointNet++ or Point Transformer), OPFR yields substantial gains in ModelNet40 (+3.8% OA, +3.2% mAcc) and S3DIS Area-5 (+3.6% OA, +13.1% mIoU).

A plausible implication is that on-the-fly geometric reasoning at the point or patch level synergizes with SPF-style progressive or sequential inference, particularly where latency or resource efficiency is critical.

6. Methodological Challenges and Open Directions

SPF-style systems must address several inherent difficulties:

• For VLM navigation, robust and interpretable spatial grounding is crucial; step-size adaptation is needed to handle environmental diversity and dynamic obstacles (Hu et al., 26 Sep 2025).
• In distributed fusion, balancing communication noise, heterogeneity, and temporal correlation requires multi-step joint optimization of both local and global parameters (Chen et al., 2024).
• Real-world sequential forecasting suffers from outlier points and view-dependent incompleteness; current self-supervised metrics like CD/EMD are imperfect proxies for frame-to-frame consistency (He et al., 2021).

Future work may involve integrating stronger physical and semantic priors, exploring generative or adversarial losses, and deepening the theoretical treatment of progressive information accumulation in dynamic environments.

7. Comparative Summary

SPF Instance	Domain	Core Technical Elements	Notable Results
(Hu et al., 26 Sep 2025)	Vision-Language Navigation	VLM spatial grounding; closed-loop 2D–3D control	93.9% sim./92.7% real-world SR; strong generalization
(Chen et al., 2024)	Distributed Point-Cloud Fusion	Streaming local projection; AirComp fusion; GPR	Progressive MSE bounds; scalable multi-agent fusion
(He et al., 2021)	Sequential Point-Cloud Forecast	Perm.-inv. recurrent cost-volumes (SPCM-Net)	50% ADE reduction (pretrain) on SFT3D; new benchmarks
(Wang et al., 2024)*	Point Cloud Feature Extraction	Efficient geometric descriptor via LRCon & CFGen	3–13% OA/mIoU improvement at 1.56ms/sample

• OPFR is not named SPF but addresses a cognate technical challenge.

The See, Point, Fly paradigm thus underlies a spectrum of methods for linking perception, efficient spatial representation, and rapid, adaptive action across vision, robotics, and cloud-edge applications. Each instantiation reflects the tension between immediate, on-the-fly computation and the requirements of robust, accurate real-world decision making.