MLaaS Service Instances: Benchmark & Composition

• MLaaS service instances are rigorously defined, independently-deployable ML units characterized by functional attributes, QoS metrics, and composition indicators.
• They enable reproducible selection, benchmarking, and federated orchestration across cloud-based, edge-oriented, and domain-specific deployments.
• They support integration in heterogeneous environments such as IoT by leveraging detailed per-instance data for realistic service composition and improved quality.

A Machine Learning as a Service (MLaaS) service instance is a rigorously defined, independently-deployable unit of machine learning functionality accessible as a remote service—characterized not only by its learnable model, task type, and input/output schemas but critically by empirically measured functional attributes, quality of service (QoS) metrics, and composition-aware indicators. The modern paradigm of MLaaS service instances spans cloud-based, edge-oriented, and domain-specific deployments, supporting reproducible selection, benchmarking, federated orchestration, composition, and lifecycle management under diverse system, data, and regulatory constraints.

1. Formal Definitions and Taxonomy

In contemporary frameworks such as the MDG (MLaaS Dataset Generator) for IoT, each MLaaS service instance $S$ is precisely defined as a tuple:

$S = \langle F, Q, C \rangle$

where:

• $F = \{f_1, f_2, \ldots, f_m\}$ — functional attributes (e.g., accuracy, latency).
• $Q = \{q_1, q_2, \ldots, q_p\}$ — QoS metrics (throughput, reliability).
• $C = \{c_1, c_2, \ldots, c_r\}$ — composition-specific indicators (e.g., inter-service transfer cost, model utility, scalability).

The instance is further indexed by a unique run identifier associated with a distinct dataset split, model family, and training configuration. Each training "run" generates a time series of metrics over multiple rounds and clients, supporting granular empirical benchmarking (Kanneganti et al., 18 Jan 2026).

2. Service Instance Generation and Empirical Properties

MDG’s instance generation protocol encompasses:

a) Data split preparation (IID or non-IID, e.g., Dirichlet, shard, quantity-skew). b) Model instantiation among supported families (CNN, RNN/LSTM-GRU, MLP/ANN, Logistic Regression, MobileNetV2, Random Forest, K-means). c) Execution of $T$ training rounds (including federated scenarios), recording per-round, per-client traces of all metrics. d) Export to relational (SQLite), tabular (CSV), and hierarchical (JSON) data formats (Kanneganti et al., 18 Jan 2026).

Core attributes and metrics:

• Accuracy (classification):

$$\mathrm{Acc} = \frac{1}{N}\sum_{i=1}^N \mathbf{1}(y_i = \hat y_i)$$

• Latency (average round time, $L_{\mathrm{avg}}$):

$$L_{\mathrm{avg}} = \frac{1}{T} \sum_{t=1}^T t_{\mathrm{round}, t}$$

• Throughput:

$$\mathrm{Thr} = \frac{\sum_{t,i} n_{t,i}}{\sum_t t_{\mathrm{round}, t}}$$

where $n_{t,i}$ is the sample count for client $i$ in round $t$.

• Reliability:

$$\mathrm{Rel} = \frac{|\{ t \mid \text{round t completed} \}|}{T}$$

• Inter-service transfer cost:

$$\mathrm{Cost}_{i\to j} = D_{i\to j} \times \ell_{i \to j}$$

($D_{i \to j}$ = data size, $\ell_{i \to j}$ = network latency).

Composition-specific indicators such as Historical Quality Score (HQS), Service Reliability Score (SRS), Data Utility (DUM), Model Utility (MUM), and Scalability (SM) are captured for advanced orchestration (Kanneganti et al., 18 Jan 2026).

3. Built-in Composition and Orchestration

A distinguishing property of advanced frameworks is simulation of instance-level composition behaviors under real-world constraints. The MDG, for instance, iterates:

• Candidate filtering based on composability (DUM, MUM, SRS thresholds).
• Aggregated parameter computation (for neural models):

$$\Theta_{\mathrm{comp}} = \sum_{i=1}^k w_i \Theta_i, \quad w_i = \frac{n_i}{\sum_j n_j}$$

• Non-parametric ensemble merging (e.g., majority voting for Random Forest/K-means; centroid averaging for unsupervised).
• Stochastic injection of network delay ($\ell \sim \mathrm{Uniform}(10, 100)$ ms or log-normal).

Composite instances record not only post-composition accuracy and latency, but also communication and true composition times, supporting further analysis and downstream optimization (Kanneganti et al., 18 Jan 2026).

4. Empirical Scale, Diversity, and Benchmarking

The MDG instance corpus comprises 10,432 distinct service instances spanning:

Category	Details
Datasets (7)	MNIST, Fashion-MNIST, Digits, CIFAR-10, Iris, Wine, California Housing
Model Families	CNN, RNN (LSTM/GRU), MLP/ANN, Logistic Regression, MobileNetV2, Random Forest, K-means
Task Types (3)	Classification, Regression, Clustering
Service Instances	10,432
Data Distributions	IID, non-IID (Dirichlet, shard, quantity-skew)
Rounds/Run	5–50 (avg. 20)
Compositions	740 unique multi-service compositions

A sample cross-section is shown for MNIST/CIFAR-10:

Model Family	MNIST	Fashion-MNIST	CIFAR-10	Iris/Wine	California Housing	Total
CNN	1024	1024	512	–	–	2560
RNN (LSTM/GRU)	1024	1024	256	–	–	2304
MLP/ANN	512	512	512	256	256	2048
Logistic Reg.	256	256	–	128	–	640
MobileNetV2	–	–	512	–	–	512
Random Forest	–	–	–	256	256	512
K-means	–	–	–	256	256	512
Total	2816	2816	1792	896	768	10,432

Instances are distributed across IID and non-IID settings to mimic federated and real-world data phenomena (Kanneganti et al., 18 Jan 2026).

5. Impact on Service Selection, Composition, and Benchmarking

Empirical evaluation establishes that rich, multidimensional instance metrics directly improve automated service selection and composition processes. Applying three canonical MLaaS selection schemes to the MDG-generated benchmark produces satisfaction rate improvements of 15%–25% over prior QWS and incomplete MLaaS collections:

Technique	QWS	In-MLaaS	MDG-Generated
Rule-based	0.82	0.85	0.92
Distance-based	0.88	0.96	0.99
Skyline-based	0.50	0.70	0.81

Moreover, composition quality (mean solution quality across multiple services) is higher and less volatile with dense instance metrics, yielding mean composability of ~0.68 versus ~0.58 for incomplete data (10% gain) (Kanneganti et al., 18 Jan 2026).

This validates the importance of a reproducible, functionally and contextually rich MLaaS instance benchmark for fair and scalable evaluation of orchestration and selection techniques.

6. Integration in IoT and Heterogeneous Environments

MDG-generated service instances are engineered for plug-and-play composition in heterogeneous, distributed, and resource-constrained networks such as IoT environments. The inclusion of per-instance composition indicators (e.g., transfer cost, historical reliability) enables:

• Simulation and optimization of realistic service pipelines.
• Data-driven orchestration that takes into account federated, non-IID, and adversarial data configurations.
• Systematic benchmarking of both micro (single-instance) and macro (composed multi-instance) MLaaS deployments (Kanneganti et al., 18 Jan 2026).

7. Reproducibility, Extensibility, and Research Applications

By encapsulating all per-run functional, system, and composition metrics in a transparent schema and supporting export to relational and hierarchical formats, MDG and similar frameworks provide a foundation for:

• Large-scale, reproducible research into MLaaS selection algorithms.
• Data-driven studies on composition under nonstationary, distributed, or adversarial workloads.
• Comparative benchmarking across model families, datasets, and architectures in federated, IoT, or enterprise-scale contexts.

The approach enables rapid extension to novel model classes, data regimes, or emerging composability paradigms (Kanneganti et al., 18 Jan 2026).

---

For IoT-focused, federated, and composite MLaaS research, the MDG instance schema—$S = \langle F, Q, C \rangle$ with complete round-wise, client-wise, and composition-metric logging—constitutes a comprehensive template for both empirical and theoretical advancement in the design, selection, and orchestration of MLaaS service instances.