Agent-Level Admission Control in Networks

• Agent-level admission control is a framework where local agents decide to accept or block requests based on network state, QoS, and resource constraints.
• It utilizes methods like MDPs, contextual bandits, and reinforcement learning to balance utilization, reliability, and safety in dynamic network environments.
• The paradigm is applied in wireless, edge computing, network slicing, and federated systems, enhancing throughput and reducing drop rates through optimized reward structures.

Agent-level admission control is a paradigm in networked systems wherein decision-making over the admittance or rejection of connection requests, resource reservation, or service instantiation is performed by individual agents. These agents may correspond to base stations, edge nodes, virtual infrastructure managers, or logical control points representing a network slice or service instance. The agent receives local or aggregated network state and traffic measurements, selects actions (usually “accept” or “block” per request), and optimizes a system-level utility function under resource, quality-of-service (QoS), fairness, safety, or incentive constraints. Modern research frames agent-level admission control as a sequential decision process under uncertainty, often solving it through reinforcement learning (RL), contextual bandits, or incentive-compatible mechanism design, with emergent applications in wireless radio access, edge computing, network slicing, virtual network embedding, and federated service orchestration.

1. Markov Decision Process and Bandit Formulations

Agent-level admission control generally formalizes the decision process as a Markov decision process (MDP) or, where feedback is more limited, as a contextual bandit. The states typically encode local system load, characteristics of the incoming request (type, rate/bandwidth demand, QoS, deadline), and possibly network-wide summaries or context:

• In the 5G wireless context, state $s_i$ at arrival $i$ encapsulates serving cell load, requested resource $R_i$ of the new user equipment (UE), one-hot encoding of its type, and instantaneous arrival rate $\lambda$—aggregation that sufficiently summarizes the observable process for RL (Raaijmakers et al., 2021).
• Bandit-based schemes for URLLC admit/reject user equipment based on feature vectors $x_t$ combining per-applicant SINR, per-UE traffic statistics, cell-level load, and active-UE feature averages. Here, each “arm” is a choice of which subset to admit, and reward depends on empirical reliability (latency constraint satisfaction) and throughput (Semiari et al., 2024).
• For service federation and slice admission, MDP states encode local and remote resource consumptions, arrival/departure events, and request profiles (Bakhshi et al., 2021, Batista et al., 2019).

Typical action sets are binary (accept/block), multi-class (accept locally, federate, block), or set-valued (admit a subset of simultaneous applicants).

The system's dynamic transitions are modeled either via explicit event rates (e.g., Poisson arrivals, exponential holding/completion), simulated rollouts, or are left model-free for RL to learn.

2. Reward Structures and Optimization Objectives

Optimization objectives in agent-level admission control reflect a trade-off between system utilization, user/service satisfaction, and operational or safety constraints. Reward forms vary by domain:

• Wireless 5G networks penalize blocking new UEs and heavily penalize dropping ongoing sessions, rewarding successful acceptance by class (Raaijmakers et al., 2021):

$r(s_i,a_i) = \begin{cases} r_{B,C(i)}, & a_i=a^- \ (\text{block}) \ r_{A,C(i)} + \gamma^{\Delta t_{D,i}} r_{D,C(i)} \mathbbm{1}_{\{\text{UE }i\text{ eventually dropped}\}}, & a_i=a^+ \ (\text{accept}) \end{cases}$

The cumulative discounted reward $G=\sum_i \gamma^{t_i} r(s_i,a_i)$ is maximized.

• URLLC admission expresses reward as the fraction of admitted users that jointly satisfy per-UE reliability constraints:

$r_t = \frac{|\text{admitted UEs}|}{|K'|} \cdot \mathbbm{1}_{c}(x_t, z_t)$

with an optimization constraint that $Pr\{\text{delay}_i\leq D_{th}\}\geq\delta_i$ (Semiari et al., 2024).

• Slice and flow admission (edge, VNE, federated) models often define reward as long-term profit (admission revenue minus costs/penalties for SLA violation and orchestration) (Batista et al., 2019), or average reward per episode (Bakhshi et al., 2021, Wang et al., 2024).

Objectives may be discounted-sum, average-reward (ergodic), multi-objective (e.g., reward–cost or reward–reliability tradeoff), or subject to hard/soft constraints (e.g., constrained MDP).

3. Learning Algorithms: RL, Contextual Bandits, and Safe RL

Contemporary agent-level admission control relies extensively on machine learning methodologies for policy optimization:

• Deep Q-Learning: Applies to discrete-action MDPs typical in wireless admission (Raaijmakers et al., 2021, Afifi et al., 2021). Neural networks approximate the state-action value function $Q(s,a)$; experience replay and target networks stabilize training.
• Policy Gradient/Actor-Critic: Used when action spaces or constraints are more complex (e.g., slice admission (Batista et al., 2019), hierarchical policies in VNE (Wang et al., 2024)), with REINFORCE or Proximal Policy Optimization (PPO) optimizing parameterized policies.
• R-Learning: Optimizes for average, rather than discounted, reward—robust in scenarios where discount factor sensitivity is problematic (e.g., service federation (Bakhshi et al., 2021)).
• Constrained RL (Safe RL): Primal-dual approaches (e.g., DR-CPO (Fox et al., 2024)) handle constrained MDPs by optimizing a Lagrangian with dual variables for each constraint.
• Neural Contextual Bandits: For settings with rich contexts and arm-dependent rewards (e.g., URLLC), neural UCB or Thompson sampling drives admission under uncertainty, maintaining empirical regret bounds (Semiari et al., 2024).

Key Algorithmic Elements Table

Domain	Formulation	Learning Type	Notable Feature
5G Wireless	Tabular/DQN	Deep Q-Learning	Model-free, multiple UE classes, adapts to $\lambda$
URLLC	Contextual Bandit	Neural UCB	Empirical reliability, per-arm neural models
Edge/Service	CMDP	Primal-dual RL	Reward decomposition, decentralized safe RL
Virtual Nets	HRL	PPO (actor-critic)	Hierarchical control (admit/embed), GNN policies

4. Architectural and Representation Considerations

Agent-level policies in modern implementations exploit compact and expressive state representations:

• State Feature Encodings: Inclusion of traffic class, resource vectorization, and load summaries; application of one-hot and bit-field encodings for categorical variables, such as tenant ID and priority in slice admission (Batista et al., 2019).
• Graph Neural Networks (GNNs): Employed to represent physical and virtual network topologies for virtual network embedding problems, capturing node/link demands and residual capacities (Wang et al., 2024).
• Neural Decoders: Sequence-to-sequence decoders generate node embeddings for resource allocation (Wang et al., 2024), while compact feed-forward networks suffice for scalar context settings.
• Target Networks, Embedding Layers: Adoption of target networks for Q-value stabilization (Raaijmakers et al., 2021), and embedding layers in multi-dimensional feature spaces (Semiari et al., 2024).

Training is typically episodic, with buffer-based experience replay, mini-batch SGD/Adam for neural architectures, and on-policy updates for actor-critic setups.

5. Practical Examples and Benchmarking

Extensive simulation-based evaluations span wireless, edge, and network slicing domains:

• 5G Wireless: Deep Q-learning automatically adapts thresholds to time-varying loads and heterogeneous UE types, outperforming static threshold policies in both accept/drop trade-off and cumulative reward. For example, high-value users are favored (accept ≈74%), and drop rates are sharply reduced (≤3%) (Raaijmakers et al., 2021).
• URLLC: Neural contextual bandits yield near-optimal cell reliability, missing stringent reliability targets <14% of the time (vs >40% with no admission control), and incidentally achieve up to 15× reduction in link-drop rates (Semiari et al., 2024).
• Edge Computing: Safe RL with reward decomposition achieves 15% higher normalized reward and 2× lower convergence time than unconstrained deep RL, while adhering more tightly to system constraints (Fox et al., 2024).
• Service Federation: R-Learning achieves a robust near-optimality gap (3–5%) across offered loads and cost scales, outperforming both greedy and Q-learning baselines (Bakhshi et al., 2021).
• Virtual Network Embedding: Hierarchical RL with average-reward admission control outperforms all baselines on both acceptance ratio and long-term revenue, with dense reward shaping and GNNs for graph-centric state representation (Wang et al., 2024, Afifi et al., 2021).

6. Incentive Mechanisms, Safety, and Decentralization

In multi-agent settings or systems with strategic actors, agent-level admission control must be incentive-aligned or provably safe:

• Strategy-Proof Mechanisms: In non-monetary wireless access control, any deterministic strategy-proof mechanism must be a highest-winning-bid mechanism. The uniform-threshold “dropping trick” achieves such truthfulness within a constant approximation of optimal utilization (Kang et al., 2010).
• Decentralized Admission Control: For wireless links, decentralized policies such as Active Link Protection (ALP) ensure each admitted user’s SIR remains above its target without centralized coordination, generalizable to arbitrary standard interference functions (0907.2896).
• Safe RL: In multi-agent environments (e.g., edge, federated), primal-dual safe RL methods (e.g., DR-CPO) decompose the objective to provide provable constraint satisfaction with only local exchange of dual variables (Fox et al., 2024).

7. Generalizations and Emerging Directions

Recent agent-level admission control research generalizes across multiple technical axes:

• Scalability to Heterogeneous Agents: Mechanisms are extensible to systems with arbitrary numbers of service/resource classes, domains, users, or edge nodes, scaling via state aggregation, feature engineering, or distributed learning (Bakhshi et al., 2021, Fox et al., 2024, Semiari et al., 2024).
• Network Slicing and Multi-Context Admission: Slice-level KPIs, multi-cell context features, and context-rich per-tenant policy learning extend the agent-level paradigm to multi-domain, multi-slice, and multi-tenant orchestration (Batista et al., 2019, Semiari et al., 2024).
• Hierarchical and Multi-Level Policies: Hierarchical RL separates admission from embedding or resource allocation, allowing each “level” of the agent to optimize under its own temporal and structural abstraction (Wang et al., 2024).
• Safety, Robustness, and Fairness: Safe-RL, constrained MDP, and fairness-aware extensions address operational guarantees and regulatory compliance in resource-limited or mission-critical applications (Fox et al., 2024, Semiari et al., 2024).

A plausible implication is that the agent-level framework—supported by expressive state representations, model-free RL, and/or strategic-incentive mechanism design—can be transferred to any domain where real-time, resource-constrained, multi-agent admission must be traded off against dynamic utility objectives under uncertainty.