Collaborative Intelligence for AIoT

• CISAIOT is a framework that distributes AI tasks across terminal, edge, and cloud layers to enable efficient inference and real-time decision making.
• It employs collaborative inference, federated and incremental learning, and dynamic model partitioning to optimize latency, energy, and accuracy.
• The system enhances security and resilience by integrating trust frameworks, policy enforcement, and adaptive resource allocation over heterogeneous infrastructures.

Collaborative Intelligence Systems for AIoT (CISAIOT) denote the practical realization of cloud-edge-terminal collaborative intelligence in AI-enabled Internet of Things deployments. In this paradigm, sensing, communication, computation, and learning are jointly organized across terminal devices, edge or fog nodes, and cloud infrastructure so that inference, adaptation, and control are no longer confined to a single locus. The resulting system integrates task offloading, distributed inference, federated or incremental learning, model life-cycle management, policy enforcement, and trust mechanisms over heterogeneous hardware and protocol stacks. Across the literature, CISAIOT appears both as an architectural program for distributing intelligence in IoT and as a family of concrete mechanisms for collaborative inference, collaborative learning, security enforcement, and resilience management in resource-constrained environments (Wu et al., 26 Aug 2025, Shlezinger et al., 2022, Ramos et al., 2019).

1. Conceptual foundations

The core idea of CISAIOT is that AI execution in IoT is distributed rather than monolithic. Collaborative inference is defined as the joint execution of a deep-learning inference task across multiple computational entities—edge IoT devices, fog or edge servers, and/or a remote cloud—by exploiting both local computing and inter-node communications. In the AIoT setting, this means that IoT devices capture raw data, share either raw data or intermediate feature representations, and jointly apply an inference model across devices and infrastructure. By orchestrating who computes which part of the model and what information is communicated, CISAIOT systems explicitly trade off latency, energy, accuracy, privacy, and connectivity (Shlezinger et al., 2022).

A related formulation, presented under the label “Intelligence Everywhere,” defines collaborative or collective intelligence as a decision-making process in which distributed IoT devices, edge, fog, and cloud nodes, services, and end-users continuously exchange data streams, models, and insights to arrive at global, collectively optimal outcomes. This framing is broader than split inference alone. It includes ultra-low-latency analytics at the far-edge, orchestration at the fog, and high-capacity aggregation in the cloud, together with federated learning (FL), incremental learning (IL), and data life-cycle control (Cao et al., 2023).

Within the cloud-edge-terminal collaborative intelligence literature, CISAIOT is further described as the practical instantiation of a three-tiered computing paradigm in which AIoT tasks are dynamically partitioned and executed across the Cloud layer, the Edge layer, and the Terminal layer. This definition emphasizes joint task offloading, resource orchestration, and distributed learning. A common misconception is that collaborative intelligence in AIoT is equivalent to sending data to the cloud. The reviewed taxonomies show instead that full cloud processing is only one option among split edge-cloud inference, compact on-device models, multi-device computation partitioning, edge ensembles, and collaborative learning across heterogeneous infrastructures (Wu et al., 26 Aug 2025).

2. Layered architectures and interface structure

A foundational architectural proposal is Ramos and Morabito’s “Intelligence Stratum,” which describes a single intelligence layer with four logical interfaces—Northbound, Southbound, Eastbound, and Westbound—rather than isolated edge, fog, and cloud modules. This intelligence compass can nonetheless be projected onto three cooperating strata. At the edge stratum, “Device-Local Intelligence” hosts local inference execution, on-device model provisioning, hardware-specific acceleration, and policy enforcement on data collection and transmission. Key components include a Southbound device manager that discovers CPU, GPU, or TPU resources and available runtimes, an Actor Repository plus Actors Engine that stores and runs Atomic Intelligent Services (AIS) or Fine-Grained Intelligent Services (FGIS), and a Northbound exposure interface that publishes REST or hypermedia APIs for local applications such as object recognition and voice-to-text (Ramos et al., 2019).

At the fog stratum, “Gateway or Edge-Cluster Intelligence” provides distributed inference orchestration, dynamic off-loading, peer-to-peer intelligence sharing, proxying for model updates, and lightweight model training on aggregated IoT data. Its Eastbound interpreter negotiates master-slave or peer associations among intelligence-layer instances and routes inference requests to optimal local nodes, while the Management Engine enforces regional policies on latency, throughput, and data privacy. At the cloud stratum, “Centralized Intelligence and Brokerage” provides heavy-weight model training and re-training, large-scale data aggregation, semantic cataloging of services and data sources, trust and certification authority, and life-cycle management for versioning, dependency resolution, certification, and push-to-edge procedures (Ramos et al., 2019).

The four interfaces define the operational grammar of the architecture. Northbound exposes intelligent-service APIs to applications and operations and management systems; Southbound handles device capability probes and actor deployment adaptation; Westbound provides life-cycle management, including on-boarding new models, updates, versioning, and secure provenance checks via digital signatures; and Eastbound enables distributed execution and cooperative inference across peers, fog nodes, and clouds. Underpinning these interfaces are a rule-engine in the Management Engine, a trust framework such as IETF SUIT-style secure updates or blockchain or ledger mechanisms for provenance, and semantic annotations such as JSON-LD or RDF so that applications and brokers can match capabilities, data types, and QoS metrics without manual encoding. The architecture is therefore simultaneously a deployment model, a management framework, and an interoperability layer (Ramos et al., 2019).

The broader CETCI literature places this within a three-layer stack in which the cloud supplies model training and data warehousing, the edge provides inference, cache, and pre-processing, and the terminal performs sensing and lightweight AI. The same body of work identifies enabling technologies beyond AI models themselves, notably network virtualization, container orchestration, and software-defined networking. This suggests that CISAIOT is not only about distributed model placement, but also about the control substrate required to embed AI services into heterogeneous communication and compute fabrics (Wu et al., 26 Aug 2025).

3. Collaboration modes and distributed learning mechanisms

The literature distinguishes several collaboration modes for inference. In non-collaborative cloud inference, an edge device obtains a sample $x_i$, transmits it over the uplink, and the cloud runs the full DNN $f(\cdot;\theta)$, yielding $\hat y_i = f(x_i;\theta)$. In collaborative edge-cloud split inference, the edge computes a front-end $u_i = f_{\rm front}(x_i;\theta_f)$, compresses and sends $u_i$, and the cloud computes the back-end $\hat y_i = f_{\rm back}(u_i;\theta_b)$. Edge-centric variants include compact DNN inference on each device, multi-device computation partitioning in which devices $\{1,\dots,K\}$ share layer blocks $\{L_1,\dots,L_K\}$, and edge ensembles in which multiple devices run diverse compact models and aggregate predictions through an ensemble rule such as majority vote or averaging (Shlezinger et al., 2022).

Algorithmically, the principal techniques supporting these modes are model compression, early-exit architectures, dynamic layer partitioning, and ensemble aggregation. Model compression includes knowledge distillation, pruning, and quantization. Early-exit methods embed auxiliary classifiers at intermediate layers so that the model can exit early if a confidence threshold is reached. Dynamic layer partitioning adapts the split layer in real time according to measured bandwidth, device load, and required latency. Over-the-air aggregation is surveyed as another mechanism, in which synchronized analog transmission of local logits uses the wireless channel to compute their average with proper pre-coding. These techniques show that collaboration in CISAIOT is not a single protocol but a design space for relocating computation and communication burdens (Shlezinger et al., 2022).

Collaborative learning extends the same logic from inference to model adaptation. In the CETCI formulation, FL trains local models $\theta_k$ on private data and aggregates them using

$$\theta^{t+1} = \sum_{k=1}^K \frac{n_k}{n}\,\theta_k^t, \qquad n=\sum_k n_k.$$

IL, by contrast, exchanges model updates peer-to-peer on a sliding window of continuous data, enabling cross-node knowledge transfer without a central server. The “Intelligence Everywhere” framework presents these as complementary modalities: FL provides iterative aggregation through a fog or cloud aggregator, whereas IL supports decentralized adaptation on real-time streams (Wu et al., 26 Aug 2025, Cao et al., 2023).

AdaptiveFL addresses a specific obstacle in collaborative learning for AIoT: heterogeneity in computing capacity, memory size, and operating conditions. It introduces a fine-grained width-wise model pruning strategy that generates various heterogeneous local models and uses a reinforcement learning-based device selection mechanism to dispatch suitable heterogeneous models to corresponding AIoT devices on the fly based on their available resources for local training. The abstract reports that AdaptiveFL can achieve up to $f(\cdot;\theta)$0 inference improvements for both IID and non-IID scenarios. Its architecture avoids forcing all clients to train a single heavyweight model and instead learns device capacities on the fly through “curiosity” and “resource” tables (Jia et al., 2023).

A further collaborative mechanism appears in secure IoT computing with deep reinforcement learning and blockchain. In the surveyed four-tier framework, IoT devices submit observations to edge nodes; edge DRL agents query the blockchain for current policy parameters or aggregate statistics; agents decide actions such as local computation, edge offloading, cloud offloading, bandwidth allocation, or transaction submission; and smart contracts collect contributions and may distribute a global update. In this setting, policy aggregation, reward-signal sharing, smart-contract-based access control, and immutable logging become part of the collaborative intelligence loop rather than external administrative functions (Far et al., 2024).

4. Quantitative models and optimization frameworks

The mathematical treatment of CISAIOT is diverse. Ramos and Morabito’s intelligence stratum remains entirely conceptual and does not introduce explicit mathematical models or LaTeX-formatted equations for resource allocation, workload distribution, or communication overhead. Much of the later literature can be read as supplying the formal optimization machinery that the architectural blueprint leaves open (Ramos et al., 2019).

For collaborative inference, the standard quantitative model starts with latency, communication, accuracy, and energy. For full cloud inference,

$f(\cdot;\theta)$1

For split inference at layer $f(\cdot;\theta)$2,

$f(\cdot;\theta)$3

Accuracy under compression is modeled as $f(\cdot;\theta)$4, with compression ratio $f(\cdot;\theta)$5, and energy is decomposed into computation and transmission terms such as $f(\cdot;\theta)$6. These formulations formalize the central CISAIOT trade-off: moving inference toward the edge reduces communication dependence but increases local compute burden (Shlezinger et al., 2022).

For feature transmission in split models, Ranjbar Alvar and Bajić model task distortion as a convex function of feature-coding rates: $f(\cdot;\theta)$7 In the single-task case, minimizing distortion under a total-rate budget yields the closed-form allocation

$f(\cdot;\theta)$8

with $f(\cdot;\theta)$9 chosen so that $\hat y_i = f(x_i;\theta)$0. The interpretation is “reverse water-filling”: streams with small $\hat y_i = f(x_i;\theta)$1 receive zero rate. The same paper characterizes the full Pareto set for 2-stream $\hat y_i = f(x_i;\theta)$2-task systems and derives analytic bounds for 3-stream 2-task systems, demonstrating that collaborative intelligence can be optimized at the level of encoded intermediate representations rather than only at the level of model partitioning (Alvar et al., 2020).

The broader CETCI literature supplies generic offloading and resource allocation models. Binary offloading is written as a minimization over $\hat y_i = f(x_i;\theta)$3, while multi-objective formulations optimize weighted sums of latency and energy. Resource allocation uses shares $\hat y_i = f(x_i;\theta)$4 across cloud, edge, and terminal layers under per-layer capacity constraints. Optimization techniques span convex and non-convex optimization, Stackelberg or Nash models, heuristics and meta-heuristics, and deep RL agents that learn offloading and allocation policies end-to-end (Wu et al., 26 Aug 2025).

In large-area sensor deployments, optimization can be even more communication-centric. The ISA+CI+CAS framework formulates a real-time co-optimization problem that minimizes total energy $\hat y_i = f(x_i;\theta)$5 over design variables such as anomaly and compression thresholds, cluster size, and LoRa hop-count, subject to information-fidelity, network-lifetime, and coverage constraints. This formulation is notable because it treats collaborative intelligence as a coupled computation-communication design problem rather than as DNN placement alone (Chatterjee et al., 2020).

5. Representative applications and empirical results

Use cases in the intelligence-stratum literature illustrate how CISAIOT decomposes service execution. In voice-to-text on a smart headset, the requirement is low-latency $\hat y_i = f(x_i;\theta)$6 inference, privacy of voice streams, and periodic model updates. The proposed response is a local AIS for acoustic feature extraction on the device’s DSP, discovery of the FGIS chain through Northbound hypermedia queries, periodic Westbound updates from the Intelligence Broker, and Eastbound off-loading of phoneme classification to a nearby fog node whenever DSP load spikes. In factory robot arm calibration, the requirement is a real-time control loop, high-accuracy model calibration, and secure firmware+model deployment; the response combines GPU-accelerated controller-board inference, certified signed models from an approved provider, and Eastbound replication of residual-error estimation modules across peer controllers. In a smart thermostat in a shared office, occupancy data are streamed to a local edge node, temperature forecasting is discovered Northbound, re-training is orchestrated Eastbound on a fog gateway, and the Westbound life-cycle manager provisions the new model back to the edge under operator-configured data policies (Ramos et al., 2019).

The “Intelligence Everywhere” program generalizes this to vertical sectors. In digital health, wearable body area networks collect EoG, ECG, and EMG streams; edge inference performs anomaly detection; local incremental updates personalize the model; periodic FL rounds aggregate updates through a smartphone or home gateway; and the cloud refines a global diagnostic model. In infrastructure, smart homes and buildings use fog nodes for predictive maintenance and prescriptive optimization, while incremental knowledge from occupancy patterns can be shared peer-to-peer among neighboring homes. In transportation and mobility, vehicles perform local CNN-based perception, exchange snapshot updates of traffic density models over PC5 links, use fog aggregation for micro-zone congestion forecasting, and rely on the cloud for large-scale traffic simulation (Cao et al., 2023).

Empirical evidence from large-area sensing shows the magnitude of the communication-computation trade-off. In a mesh architecture over a 2400-acre university campus for temperature, humidity, and water nitrate sensing, in-sensor analytics (ISA) consumes $\hat y_i = f(x_i;\theta)$7 lower energy than traditional BLE communication and $\hat y_i = f(x_i;\theta)$8 lower energy than LoRa communication. When ISA is implemented in conjunction with LoRa, node lifetime increases from 4.3 hours to 66.6 days with a 230 mAh coin cell battery while preserving more than $\hat y_i = f(x_i;\theta)$9 of the total information. The CI and CAS algorithms extend the worst-case node lifetime by an additional $u_i = f_{\rm front}(x_i;\theta_f)$0, yielding an overall network lifetime of $u_i = f_{\rm front}(x_i;\theta_f)$1 days, which is $u_i = f_{\rm front}(x_i;\theta_f)$2 of the theoretical limits posed by leakage currents in the system, while effectively transferring information sampled every second (Chatterjee et al., 2020).

Empirical evidence on collaborative learning under heterogeneity appears in AdaptiveFL. On CIFAR-10 with VGG16 under IID data, average submodel accuracy increases from $u_i = f_{\rm front}(x_i;\theta_f)$3 for ScaleFL to $u_i = f_{\rm front}(x_i;\theta_f)$4 for AdaptiveFL; on CIFAR-10 VGG16 with Dir$u_i = f_{\rm front}(x_i;\theta_f)$5, it increases from $u_i = f_{\rm front}(x_i;\theta_f)$6 to $u_i = f_{\rm front}(x_i;\theta_f)$7. On CIFAR-100 with ResNet18, the reported gains include $u_i = f_{\rm front}(x_i;\theta_f)$8 to $u_i = f_{\rm front}(x_i;\theta_f)$9 for IID, $u_i$0 to $u_i$1 for Dir$u_i$2, and $u_i$3 to $u_i$4 for Dir$u_i$5. In extreme cases, such as comparison to HeteroFL on CIFAR-100 ResNet18 IID, the gain reaches $u_i$6. Real-device testbeds using Pi4B, Jetson Nano, and Xavier AGX show very similar accuracy improvements and a $u_i$7 reduction in communication waste (Jia et al., 2023).

6. Security, resilience, and open research directions

CISAIOT research increasingly treats security and resilience as first-class system properties. In secure smart-city IoT computing, a canonical integration uses a blockchain layer to record device or agent registration, DRL agent reports such as parameter updates and reward statistics, and smart contracts for access control, resource-sharing auctions, or incentive distribution. Permissioned networks often employ PBFT or Tendermint, while public testbeds use Proof of Work or delegated Proof of Stake. Smart contracts implemented in Solidity or Chaincode on Hyperledger Fabric support device and agent registration, key management, storage of DRL agent gradients or value-function summaries, and automatic incentive or reward distribution. Transactions are built from headers and signed payloads, using SHA-256 for hashing and ECDSA over secp256k1 for signatures, with optional PUF-based device attestation (Far et al., 2024).

Security, however, does not eliminate the need for performance tracking under disruption. Rimawi et al. model collaborative AI system performance through the Autonomous Classification Ratio,

$u_i$8

If confidence $u_i$9, the system acts autonomously and records $\hat y_i = f_{\rm back}(u_i;\theta_b)$0; otherwise it enters the Learning State, requires human demonstration, records $\hat y_i = f_{\rm back}(u_i;\theta_b)$1, and updates the model. During the first fully autonomous phase, the minimum acceptable performance is

$\hat y_i = f_{\rm back}(u_i;\theta_b)$2

Resilience during a Disruptive State is then described by State Length $\hat y_i = f_{\rm back}(u_i;\theta_b)$3, PUT, PAT, PUT-to-PAT ratio, and $\hat y_i = f_{\rm back}(u_i;\theta_b)$4. In a robot-human case study with $\hat y_i = f_{\rm back}(u_i;\theta_b)$5 iterations, $\hat y_i = f_{\rm back}(u_i;\theta_b)$6, and $\hat y_i = f_{\rm back}(u_i;\theta_b)$7, the system exhibits $\hat y_i = f_{\rm back}(u_i;\theta_b)$8, $\hat y_i = f_{\rm back}(u_i;\theta_b)$9, $\{1,\dots,K\}$0, $\{1,\dots,K\}$1, $\{1,\dots,K\}$2, and $\{1,\dots,K\}$3 during the first disruptive state; after $\{1,\dots,K\}$4 iterations past the disruption, $\{1,\dots,K\}$5 climbs above $\{1,\dots,K\}$6 and the system enters a recovered state (Rimawi et al., 2024).

Open challenges remain extensive and are distributed across architecture, algorithms, and governance. The intelligence-stratum literature highlights provisioning and composition of AI models in a layered architecture, protocol selection and architectural style trade-offs, semantic API discovery for intelligence services, secure, private yet interoperable service composition, policy-driven orchestration, trust and provenance propagation, runtime monitoring and capability management, and balancing model complexity, performance, and resource usage (Ramos et al., 2019). Collaborative inference research adds privacy guarantees, mobility and connectivity dynamics, hybrid collaboration, over-the-air collaborative computing, joint hardware-algorithm co-design, and model diversity for ensembles (Shlezinger et al., 2022). The broader CETCI surveys emphasize scalability, heterogeneity, interoperability, security and privacy, and future trends including 6G+, agents, digital twins, quantum computing, and explainable AI (Wu et al., 26 Aug 2025). The “Intelligence Everywhere” perspective extends these concerns to human-machine collaboration, infrastructure supervision, context awareness, and cross-sector marketplaces, while the blockchain-DRL literature foregrounds privacy-preserving DRL, lightweight consensus for ultra-resource-constrained nodes, and formal verification of smart contracts that implement RL parameter updates (Cao et al., 2023, Far et al., 2024).

Taken together, these strands define CISAIOT as a multi-layer intelligence continuum rather than a single algorithmic pattern. Its distinguishing features are the coordinated placement of AI functions across terminal, edge, and cloud layers; the co-optimization of communication, computation, and learning; the incorporation of life-cycle, trust, and policy mechanisms into the inference loop; and the explicit treatment of heterogeneity, uncertainty, and disruption as normal operating conditions.