DQ-Net: Deep Q-Learning for Network Selection

• DQ-Net is a deep Q-network model that dynamically selects among WiFi, 4G LTE, 5G, and satellite networks to maintain high QoS/QoE in changing conditions.
• It uses reinforcement learning with Q-learning, leveraging normalized QoS metrics and dynamic reward functions to guide network selection.
• Simulations in NS-3 show DQ-Net outperforms traditional MADM methods, achieving up to 93% accuracy in selecting the optimal network over multiple epochs.

DQ-Net denotes a Deep Q-Network-based network-selection model for heterogeneous wireless systems in which a user equipment learns which radio access technology to choose at each decision point instead of relying on a static multi-criteria ranking rule. In the formulation reported in “A DQN-based model for intelligent network selection in heterogeneous wireless systems,” the candidate radio access technologies are WiFi, 4G LTE, 5G, and satellite / LEO (Starlink), and the objective is to maintain good service continuity and QoS/QoE while adapting to changing network conditions such as bandwidth, latency, jitter, packet loss, congestion/load, and cost (Bendaoud et al., 8 Jan 2026).

1. Heterogeneous access-network selection problem

DQ-Net is situated in the classic heterogeneous wireless-network problem: multiple radio access technologies coexist, each has different and time-varying QoS properties, mobile users may need to switch networks to keep service quality high, and static methods struggle because network conditions change over time. The model is explicitly framed as dynamic and adaptive, with the optimal choice depending on the current state of the networks and the user context rather than on a fixed ranking rule (Bendaoud et al., 8 Jan 2026).

The practical role of the model is to select the best access network for a user equipment among WiFi, 4G LTE, 5G, and satellite / LEO. In the paper’s presentation, 5G is assumed to be the best candidate in most cases, which is why the main reported evaluation metric is the percentage of 5G selection over epoch intervals. This choice of metric does not redefine the task itself; the broader task remains network selection under changing QoS and cost conditions. A plausible implication is that the reported results are best interpreted as a proxy for correct RAT choice in the particular simulation regime rather than as a universal criterion for all deployments.

2. Reinforcement-learning formulation

The model uses reinforcement learning because an agent can learn from interaction with the environment, receives rewards and punishments based on selected actions, can improve through repeated experience, and can handle changing conditions better than static MADM methods. The agent is the user equipment, the environment is the set of available RATs/networks, the action is to choose one network to connect to, the state is formed from QoS and device/network metrics, and the reward is the quality of the selected connection (Bendaoud et al., 8 Jan 2026).

The state vector for a network $net_i$ is defined as

$$S_{(net_i)}=[Throughput_{(net_i )},latency_{(net_i)},jitter_{(net_i)}, PLR_{(net_i )},U_{(net_i )},C_{(net_i)}]$$

where Throughput is bandwidth/data rate, Latency is delay, Jitter is delay variation, $PLR$ is packet loss rate, $U$ is user load / utilization, and $C$ is cost. This representation makes the decision rule explicitly contingent on both performance and economic variables rather than on throughput alone (Bendaoud et al., 8 Jan 2026).

The reward is defined as a weighted quantity that prefers high bandwidth and penalizes latency, jitter, packet loss, congestion, and cost. The paper specifies dynamic weights

$$W_b = 4 + \frac{b}{100}, \qquad W_l = 4 + \frac{l}{100}, \qquad W_j = 2.5 + \frac{j}{20},$$

$$W_p = 4 + \frac{p}{5}, \qquad W_u = 3 + \frac{u}{50}, \qquad W_c = 2 + \frac{c}{10}.$$

The reported interpretation is that higher bandwidth increases reward, while higher latency, jitter, packet loss rate, load, and cost decrease reward; the authors note that the reward is linear and monotone in the expected direction. The temporal-difference update is the standard Q-learning form

$$Q(s_t,a_t )=Q(s_t,a_t )+\alpha[r_{(t+1)}+\gamma \max_{a} Q(s_{(t+1)},a_t )-Q(s_t,a_t )]$$

with $\alpha$ the learning rate, $r_{t+1}$ the reward, and $$S_{(net_i)}=[Throughput_{(net_i )},latency_{(net_i)},jitter_{(net_i)}, PLR_{(net_i )},U_{(net_i )},C_{(net_i)}]$$0 the discount factor (Bendaoud et al., 8 Jan 2026).

3. Decision architecture and control flow

The proposed selector is described as having three main parts: an input layer taking normalized QoS/network data, hidden layers in which convolutional layers are mentioned as being used to learn complex state-action relationships, and an output layer that produces a Q-value for each candidate network, enabling ranking and selection. In operational terms, the network-selection decision is either a random action during exploration or the argmax-Q action during exploitation (Bendaoud et al., 8 Jan 2026).

The control loop is specified stepwise. At each step or epoch, the model builds a state array from the current fixed state, chooses an action using $$S_{(net_i)}=[Throughput_{(net_i )},latency_{(net_i)},jitter_{(net_i)}, PLR_{(net_i )},U_{(net_i )},C_{(net_i)}]$$1-greedy, executes the action via take_action(action, fixedstate), observes the reward, next state, done flag, and selected network, stores the transition $$S_{(net_i)}=[Throughput_{(net_i )},latency_{(net_i)},jitter_{(net_i)}, PLR_{(net_i )},U_{(net_i )},C_{(net_i)}]$$2 in replay memory, trains or updates the DQN, periodically updates the target network, and decays $$S_{(net_i)}=[Throughput_{(net_i )},latency_{(net_i)},jitter_{(net_i)}, PLR_{(net_i )},U_{(net_i )},C_{(net_i)}]$$3. In the pseudocode, exploration occurs when rand(0,1) < epsilon; otherwise the model computes qvalues = model.predict(statearray) and selects action = argmax(qvalues) (Bendaoud et al., 8 Jan 2026).

This design makes DQ-Net sequential rather than static. It does not merely rank networks once; it repeatedly selects and, if necessary, switches to another RAT. That property is central to the paper’s claim that the approach is better suited than typical MADM methods to changing network conditions.

4. Training regime and simulation setup

The training procedure includes random initialization of network weights, experience replay, a separate target network for more stable Q-target computation, and epsilon decay so that exploration decreases over time and exploitation becomes dominant later. The paper explicitly states that the model starts by making poor choices during exploration and then learns to choose better actions after enough epochs. Early epochs therefore feature more random network choices, whereas later epochs feature more greedy selection of the best-known RAT (Bendaoud et al., 8 Jan 2026).

Simulations were run using NS-3 under different scenarios, and the learning process was run for up to 2000 epochs. The network characteristics table specifies bandwidths of 50–500 Mbps for 5G, 10–50 for 4G, 20–80 for WiFi, and 50–200 for LEO (Starlink); latencies of 5–10 ms, 10–30 ms, 10–50 ms, and 30–70 ms; jitter ranges of 1–5 ms, 5–15 ms, 1–8 ms, and 5–20 ms; packet loss ranges of 0–1%, 0.1–2%, 0–5%, and 2–10%; user load ranges of 10–50%, 30–70%, 20–60%, and 40–80%; and cost ranges of \$S_{(net_i)}=[Throughput_{(net_i )},latency_{(net_i)},jitter_{(net_i)}, PLR_{(net_i )},U_{(net_i )},C_{(net_i)}]$42–5, \$S_{(net_i)}=[Throughput_{(net_i )},latency_{(net_i)},jitter_{(net_i)}, PLR_{(net_i )},U_{(net_i )},C_{(net_i)}]$54–8, respectively (Bendaoud et al., 8 Jan 2026).

The comparison set consists of MADM methods AHP, WSM, WPM, and TOPSIS. Because the principal metric is 5G selection percentage, the experiments are organized around epoch intervals rather than around latency, handover frequency, or packet-level application outcomes. This suggests that the evaluation is primarily a policy-selection study within the simulation environment.

5. Reported performance and comparative results

The abstract reports that the approach reaches 93% accuracy, especially after the exploration phase, whereas typical MADM methods do not exceed 75 percent. The detailed epoch-wise results are consistent with that claim. Over epochs 1–500, DQN records 74.08, compared with 77.2 for AHP, 74.2 for WSM, 12.4 for WPM, and 50.4 for TOPSIS. Over epochs 500–1000, DQN rises to 91.3, while AHP is 72.8, WSM 67.0, WPM 17.1, and TOPSIS 41.8 (Bendaoud et al., 8 Jan 2026).

The same pattern persists later in training. Over epochs 1000–1500, DQN records 92.5; over 1500–2000 it records 90.9; and over 1900–2000 it records 93.0. The corresponding 1900–2000 baselines are 78.0 for AHP, 72.0 for WSM, 17.0 for WPM, and 47.0 for TOPSIS. Over the full run from 1–2000 epochs, DQN averages 87.03, compared with 75.4 for AHP, 71.2 for WSM, 15.3 for WPM, and 44.05 for TOPSIS (Bendaoud et al., 8 Jan 2026).

The interpretation supplied in the paper is direct: DQN is weaker in the first 500 epochs because it is exploring, after exploration decreases its performance rises above 90%, and once trained it becomes more accurate than MADM methods. The best MADM result is AHP at about 75.4–78%, so the proposed model outperforms the static alternatives in the reported scenarios. This suggests that the advantage arises less from a different ranking heuristic than from the capacity to adapt policy through repeated interaction.

6. Limitations, assumptions, and nomenclature

Several caveats are explicit. The method is computationally heavy, is not yet real-time, requires substantial training data and epochs, and is evaluated in NS-3 scenarios rather than in a deployed live system. The authors also state that they expect similar behavior across VoIP, video streaming, video calls, web browsing, and downloads, but that expectation is not presented as a separately validated experiment. They further note that more epochs should improve results, making training duration a material assumption of the reported performance (Bendaoud et al., 8 Jan 2026).

The term “DQ-Net” is also nomenclaturally ambiguous. In the cited wireless-systems paper, it functions as a descriptive label for a Deep Q-Network used as the core decision engine for dynamic network selection rather than as a universally standardized method name (Bendaoud et al., 8 Jan 2026). Other arXiv works use related but different abbreviations: “Deep Quality-Value (DQV) Learning” introduces a two-network $$S_{(net_i)}=[Throughput_{(net_i )},latency_{(net_i)},jitter_{(net_i)}, PLR_{(net_i )},U_{(net_i )},C_{(net_i)}]$$6 and $$S_{(net_i)}=[Throughput_{(net_i )},latency_{(net_i)},jitter_{(net_i)}, PLR_{(net_i )},U_{(net_i )},C_{(net_i)}]$$7 architecture and explicitly notes that it does not use the exact term “DQ-Net” (Sabatelli et al., 2018); “Dataset Quantization” is a dataset compression framework and is described as being referred to as “DQ-Net” in some discussions, but it addresses compression of ImageNet-1K and Alpaca rather than wireless access selection (Zhou et al., 2023). Additional unrelated uses of “DQ” include DQ-modules in deformation quantization (Petit, 2015), DQM for decentralized optimization (Mokhtari et al., 2015), and dq modeling for wireless power transfer systems (Jiang, 28 Jul 2025). Accordingly, in wireless-network literature, DQ-Net is most precisely understood as a DQN-based selector for radio access technology choice in heterogeneous systems rather than as a general-purpose label for all “DQ” methods.