---
title: 'DQ-Net: Deep Q-Learning for Network Selection'
url: https://www.emergentmind.com/topics/dq-net
type: topic
---

# DQ-Net: Deep Q-Learning for Network Selection

DQ-Net denotes a Deep Q-Network-based network-selection model for heterogeneous wireless systems in which a user equipment learns which radio access technology to choose at each decision point instead of relying on a static multi-criteria ranking rule. In the formulation reported in “A DQN-based model for intelligent network selection in heterogeneous wireless systems,” the candidate radio access technologies are WiFi, 4G LTE, 5G, and satellite / LEO (Starlink), and the objective is to maintain good service continuity and QoS/QoE while adapting to changing network conditions such as bandwidth, latency, jitter, packet loss, congestion/load, and cost [2601.04978].

## 1. Heterogeneous access-network selection problem

DQ-Net is situated in the classic heterogeneous wireless-network problem: multiple radio access technologies coexist, each has different and time-varying QoS properties, mobile users may need to switch networks to keep service quality high, and static methods struggle because network conditions change over time. The model is explicitly framed as dynamic and adaptive, with the optimal choice depending on the current state of the networks and the user context rather than on a fixed ranking rule [2601.04978].

The practical role of the model is to select the best access network for a user equipment among WiFi, 4G LTE, 5G, and satellite / LEO. In the paper’s presentation, 5G is assumed to be the best candidate in most cases, which is why the main reported evaluation metric is the percentage of 5G selection over epoch intervals. This choice of metric does not redefine the task itself; the broader task remains network selection under changing QoS and cost conditions. A plausible implication is that the reported results are best interpreted as a proxy for correct RAT choice in the particular simulation regime rather than as a universal criterion for all deployments.

## 2. Reinforcement-learning formulation

The model uses reinforcement learning because an agent can learn from interaction with the environment, receives rewards and punishments based on selected actions, can improve through repeated experience, and can handle changing conditions better than static MADM methods. The agent is the user equipment, the environment is the set of available RATs/networks, the action is to choose one network to connect to, the state is formed from QoS and device/network metrics, and the reward is the quality of the selected connection [2601.04978].

The state vector for a network \(net_i\) is defined as

\[
S_{(net_i)}=[Throughput_{(net_i )},latency_{(net_i)},jitter_{(net_i)}, PLR_{(net_i )},U_{(net_i )},C_{(net_i)}]
\]

where Throughput is bandwidth/data rate, Latency is delay, Jitter is delay variation, \(PLR\) is packet loss rate, \(U\) is user load / utilization, and \(C\) is cost. This representation makes the decision rule explicitly contingent on both performance and economic variables rather than on throughput alone [2601.04978].

The reward is defined as a weighted quantity that prefers high bandwidth and penalizes latency, jitter, packet loss, congestion, and cost. The paper specifies dynamic weights

\[
W_b = 4 + \frac{b}{100}, \qquad
W_l = 4 + \frac{l}{100}, \qquad
W_j = 2.5 + \frac{j}{20},
\]

\[
W_p = 4 + \frac{p}{5}, \qquad
W_u = 3 + \frac{u}{50}, \qquad
W_c = 2 + \frac{c}{10}.
\]

The reported interpretation is that higher bandwidth increases reward, while higher latency, jitter, packet loss rate, load, and cost decrease reward; the authors note that the reward is linear and monotone in the expected direction. The temporal-difference update is the standard Q-learning form

\[
Q(s_t,a_t )=Q(s_t,a_t )+\alpha[r_{(t+1)}+\gamma \max_{a} Q(s_{(t+1)},a_t )-Q(s_t,a_t )]
\]

with \(\alpha\) the learning rate, \(r_{t+1}\) the reward, and \(\gamma\) the discount factor [2601.04978].

## 3. Decision architecture and control flow

The proposed selector is described as having three main parts: an input layer taking normalized QoS/network data, hidden layers in which convolutional layers are mentioned as being used to learn complex state-action relationships, and an output layer that produces a Q-value for each candidate network, enabling ranking and selection. In operational terms, the network-selection decision is either a random action during exploration or the argmax-Q action during exploitation [2601.04978].

The control loop is specified stepwise. At each step or epoch, the model builds a state array from the current fixed state, chooses an action using \(\epsilon\)-greedy, executes the action via `take_action(action, fixedstate)`, observes the reward, next state, done flag, and selected network, stores the transition \((s,a,r,s',done)\) in replay memory, trains or updates the DQN, periodically updates the target network, and decays \(\epsilon\). In the pseudocode, exploration occurs when `rand(0,1) < epsilon`; otherwise the model computes `qvalues = model.predict(statearray)` and selects `action = argmax(qvalues)` [2601.04978].

This design makes DQ-Net sequential rather than static. It does not merely rank networks once; it repeatedly selects and, if necessary, switches to another RAT. That property is central to the paper’s claim that the approach is better suited than typical MADM methods to changing network conditions.

## 4. Training regime and simulation setup

The training procedure includes random initialization of network weights, experience replay, a separate target network for more stable Q-target computation, and epsilon decay so that exploration decreases over time and exploitation becomes dominant later. The paper explicitly states that the model starts by making poor choices during exploration and then learns to choose better actions after enough epochs. Early epochs therefore feature more random network choices, whereas later epochs feature more greedy selection of the best-known RAT [2601.04978].

Simulations were run using NS-3 under different scenarios, and the learning process was run for up to 2000 epochs. The network characteristics table specifies bandwidths of 50–500 Mbps for 5G, 10–50 for 4G, 20–80 for WiFi, and 50–200 for LEO (Starlink); latencies of 5–10 ms, 10–30 ms, 10–50 ms, and 30–70 ms; jitter ranges of 1–5 ms, 5–15 ms, 1–8 ms, and 5–20 ms; packet loss ranges of 0–1%, 0.1–2%, 0–5%, and 2–10%; user load ranges of 10–50%, 30–70%, 20–60%, and 40–80%; and cost ranges of \$3–6, \$2–5, \$1–4, and \$4–8, respectively [2601.04978].

The comparison set consists of MADM methods AHP, WSM, WPM, and TOPSIS. Because the principal metric is 5G selection percentage, the experiments are organized around epoch intervals rather than around latency, handover frequency, or packet-level application outcomes. This suggests that the evaluation is primarily a policy-selection study within the simulation environment.

## 5. Reported performance and comparative results

The abstract reports that the approach reaches 93% accuracy, especially after the exploration phase, whereas typical MADM methods do not exceed 75 percent. The detailed epoch-wise results are consistent with that claim. Over epochs 1–500, DQN records 74.08, compared with 77.2 for AHP, 74.2 for WSM, 12.4 for WPM, and 50.4 for TOPSIS. Over epochs 500–1000, DQN rises to 91.3, while AHP is 72.8, WSM 67.0, WPM 17.1, and TOPSIS 41.8 [2601.04978].

The same pattern persists later in training. Over epochs 1000–1500, DQN records 92.5; over 1500–2000 it records 90.9; and over 1900–2000 it records 93.0. The corresponding 1900–2000 baselines are 78.0 for AHP, 72.0 for WSM, 17.0 for WPM, and 47.0 for TOPSIS. Over the full run from 1–2000 epochs, DQN averages 87.03, compared with 75.4 for AHP, 71.2 for WSM, 15.3 for WPM, and 44.05 for TOPSIS [2601.04978].

The interpretation supplied in the paper is direct: DQN is weaker in the first 500 epochs because it is exploring, after exploration decreases its performance rises above 90%, and once trained it becomes more accurate than MADM methods. The best MADM result is AHP at about 75.4–78%, so the proposed model outperforms the static alternatives in the reported scenarios. This suggests that the advantage arises less from a different ranking heuristic than from the capacity to adapt policy through repeated interaction.

## 6. Limitations, assumptions, and nomenclature

Several caveats are explicit. The method is computationally heavy, is not yet real-time, requires substantial training data and epochs, and is evaluated in NS-3 scenarios rather than in a deployed live system. The authors also state that they expect similar behavior across VoIP, video streaming, video calls, web browsing, and downloads, but that expectation is not presented as a separately validated experiment. They further note that more epochs should improve results, making training duration a material assumption of the reported performance [2601.04978].

The term “DQ-Net” is also nomenclaturally ambiguous. In the cited wireless-systems paper, it functions as a descriptive label for a Deep Q-Network used as the core decision engine for dynamic network selection rather than as a universally standardized method name [2601.04978]. Other arXiv works use related but different abbreviations: “Deep Quality-Value (DQV) Learning” introduces a two-network \(V(s)\) and \(Q(s,a)\) architecture and explicitly notes that it does not use the exact term “DQ-Net” [1810.00368]; “Dataset Quantization” is a dataset compression framework and is described as being referred to as “DQ-Net” in some discussions, but it addresses compression of ImageNet-1K and Alpaca rather than wireless access selection [2308.10524]. Additional unrelated uses of “DQ” include DQ-modules in deformation quantization [1507.04315], DQM for decentralized optimization [1510.07356], and dq modeling for wireless power transfer systems [2507.20921]. Accordingly, in wireless-network literature, DQ-Net is most precisely understood as a DQN-based selector for radio access technology choice in heterogeneous systems rather than as a general-purpose label for all “DQ” methods.

Source: https://www.emergentmind.com/topics/dq-net