Virtual Sensor Middleware: Managing IoT Data for the Fog-Cloud Platform (2210.10481v1)

Abstract: This paper introduces the Virtual Sensor Middleware (VSM), which facilitates distributed sensor data processing on multiple fog nodes. VSM uses a Virtual Sensor as the core component of the middleware. The virtual sensor concept is redesigned to support functionality beyond sensor/device virtualization, such as deploying a set of virtual sensors to represent an IoT application and distributed sensor data processing across multiple fog nodes. Furthermore, the virtual sensor deals with the heterogeneous nature of IoT devices and the various communication protocols using different adapters to communicate with the IoT devices and the underlying protocol. VSM uses the publish-subscribe design pattern to allow virtual sensors to receive data from other virtual sensors for seamless sensor data consumption without tight integration among virtual sensors, which reduces application development efforts. Furthermore, VSM enhances the design of virtual sensors with additional components that support sharing of data in dynamic environments where data receivers may change over time, data aggregation is required, and dealing with missing data is essential for the applications.

• The paper introduces Virtual Sensor Middleware that distributes sensor data processing across fog nodes using a publish-subscribe approach to reduce development efforts.
• It details a modular virtual sensor design with components for data aggregation, fault handling, and dynamic configuration via JSON-based setups.
• Evaluation using Raspberry Pi simulations shows that distributing virtual sensors across fog nodes reduces CPU load, though memory usage remains a challenge.

Virtual Sensor Middleware for Fog-Cloud Platforms

This paper introduces the Virtual Sensor Middleware (VSM), designed to facilitate distributed sensor data processing across multiple fog nodes. The core component of VSM is a redesigned virtual sensor that extends beyond simple sensor virtualization to enable the deployment of virtual sensor sets representing IoT applications and the distribution of sensor data processing across multiple fog nodes. VSM leverages the publish-subscribe pattern for seamless data consumption between virtual sensors, thus reducing application development efforts.

Core Concepts and Architecture

Virtual Sensor Design

The virtual sensor design is enhanced with components that support data sharing in dynamic environments, manage data aggregation, and handle missing data. (Figure 1) illustrates the components of a virtual sensor.

Figure 1: Virtual Sensor Components.

Each virtual sensor is instantiated based on a configuration file specifying attributes such as input sources, output destinations, data reading rates, fault handling policies, and database addresses. The key components include:

• Consumer: Establishes connections with input sources, subscribing to message brokers for data from other virtual sensors or using adapters for physical sensors.
• Data Aggregator: Combines data from various input sources, managing priority queues based on timestamps and employing a fault handling mechanism for incomplete data.
• Fault Handler: Activated by the Data Aggregator, it applies fault handling policies to manage missing or malformed data.
• Processor: Performs the core functionality of the virtual sensor, processing incoming data tuples.
• Publisher: Publishes processed data to a message broker, making it accessible to subscribers.
• Data Manager: Connects to a database for data storage, if required.
• VS Monitor: Transmits regular status messages to a cloud-based VSM Monitor for fault detection.

Middleware Architecture

The VSM architecture, as depicted in (Figure 2), comprises components distributed across cloud and fog nodes.

Figure 2: VS Middleware Architecture.

On the fog node level, a publish-subscribe message broker accepts data from hosted virtual sensors. A knowledge base stores settings for all deployed virtual sensors, and the VS Orchestrator uses these settings to instantiate or update virtual sensors. Cloud components include a VS Configurator, which generates and verifies virtual sensor configurations, a VS Deployer for distributing configuration files to fog nodes, and a VSM Monitor for tracking the status of deployed virtual sensors.

Implementation Details

The Virtual Sensor Configurator Interface is implemented using Java Server Pages (JSP), a JSON editor, and the Bootstrap framework. The Virtual Sensor Configurator component is a Java Servlet, storing configurations in JSON files. The Virtual Sensor Deployer component, also a Java Servlet, uses Apache Commons Net v3.6 to transfer configurations to fog nodes via FTP. RabbitMQ, an open-source message broker using AMQP 0-9-1, is used for the publish-subscribe messaging, deployed on each fog node using a federation topology. The Knowledge-Base uses the file system for storing virtual sensor configurations. The VS Orchestrator, implemented in Java, monitors the Knowledge-Base for updates and instantiates virtual sensors accordingly.

Evaluation and Results

The evaluation assesses VSM's performance with varying numbers of virtual sensors, inputs per sensor, deployment levels, and fog nodes, using Raspberry Pi devices to simulate fog nodes. (Figure 3) and (Figure 4) illustrates the evaluation environment.

Figure 3: Evaluation Environment using Raspberry Pi serving as fog nodes and other scripts used for execute the simulation.

Figure 4: Evaluation Environment - Creating and deploying VS Configurations.

Experiments show that CPU and memory utilization increase with the number of virtual sensors (Figure 5) and input sources per sensor (Figure 6). Distributing virtual sensors across multiple fog nodes reduces resource utilization on individual nodes.

Figure 7: Evaluation Environment - Load Testing Script simulating data generated by physical sensors.

Figure 5: Number of Virtual Sensor Impact on Performance.

Figure 6: Number of Inputs per Virtual Sensor Impact on Performance.

Impact of Virtual Sensor Distribution

When distributing virtual sensors across multiple levels of fog nodes, the paper observed that distributing virtual sensors equally into three levels decreased the CPU utilization, but the memory utilization almost remained the same compared to the first setup. (Figure 8)

Figure 8: Deployment levels Impact on Performance.

Conclusion

The VSM design incorporates fault-handling policies to address data loss and node failures. It decouples IoT devices and applications via the publish-subscribe pattern, enabling runtime application changes through dynamic configuration updates. Future work includes developing a more flexible approach for software deployment to virtual sensors. This work assumes that each fog node has a publish-subscribe broker and they want to compare different approaches to placing brokers.