Channel-wise Partial Linearization

• Channel-wise partial linearization is a method that selectively linearizes individual variable blocks to efficiently solve block-separable composite optimization problems.
• It reduces per-iteration complexity by updating only one channel at a time, significantly enhancing scalability in large-scale and distributed systems.
• Adaptive tolerance sequencing balances computational accuracy with efficiency, ensuring provable convergence in applications like network equilibrium, multi-agent optimization, and machine learning.

Channel-wise partial linearization is a family of decomposition-oriented optimization algorithms for problems with block-separable objective function structure and product-form feasible sets. The approach consists in selectively applying first-order (partial) linearizations to only one “channel” (block) of variables per iteration, accompanied by adaptive control of optimality tolerances. This facilitates reduced per-iteration complexity and scalability for large-scale composite and distributed optimization, with provable convergence guarantees and practical benefit in several applications including network equilibrium, large-margin classification, and multi-agent systems (Konnov, 2016, Scutari et al., 2013). In information theory, channel-wise partial linearization enables the transformation of multi-terminal perturbative problems into linear-algebraic form through the blockwise treatment of tangent-space coordinates and quadratic approximations (Huang et al., 2014).

1. Problem Setting and Preliminaries

Channel-wise partial linearization addresses composite minimization problems of the form

$\min_{x \in X} \, f(x) := g(x) + h(x)$

where $g : \mathbb{R}^N \to \mathbb{R}$ is smooth (continuously differentiable), $h : \mathbb{R}^N \to (-\infty, +\infty]$ is convex, proper, lower semicontinuous, and separable by blocks (channels): $$h(x) = \sum_{i=1}^n h_i(x_i), \quad x = (x_1, ..., x_n), \quad x_i \in \mathbb{R}^{N_i}, \sum N_i = N,$$ with constraints

$$X = X_1 \times X_2 \cdots \times X_n, \quad X_i \subset \mathbb{R}^{N_i} \text{ closed and compact}.$$

This block-structured framework encompasses many high-dimensional applications, such as multi-agent system design, wireless network scheduling, path-based traffic equilibrium, and decomposable statistical learning (Konnov, 2016, Scutari et al., 2013).

2. Channel-wise (Block-wise) Partial Linearization Update

At each iteration, a single channel (block) is chosen, and the smooth part $g$ is partially linearized with respect to this block. The key steps are:

• Block/Channel selection: Select block index $s \in \{1, \ldots, n\}$.
• Partial model: Locally linearize $g$ in $x_s$:

$$L_s^k(y) = g(x^k) + \langle \nabla_s g(x^k), y - x_s^k \rangle.$$

• Auxiliary block problem: Compute

$g : \mathbb{R}^N \to \mathbb{R}$0

and define the block gap,

$g : \mathbb{R}^N \to \mathbb{R}$1

The search direction $g : \mathbb{R}^N \to \mathbb{R}$2 is zero for all $g : \mathbb{R}^N \to \mathbb{R}$3, with $g : \mathbb{R}^N \to \mathbb{R}$4.

• Stepsize/linesearch: Either an inexact Armijo rule via backtracking, or explicit stepsize for block-Lipschitz gradient,

$g : \mathbb{R}^N \to \mathbb{R}$5

for block-Lipschitz constants $g : \mathbb{R}^N \to \mathbb{R}$6.

• Block update: Set

$g : \mathbb{R}^N \to \mathbb{R}$7

Optionally, a proximal regularizer can be applied:

$g : \mathbb{R}^N \to \mathbb{R}$8

This selectivity and block-separability produce efficient coordinate descent variants, often termed “adaptive partial linearization” (Konnov, 2016), or “channel-wise SCA” in distributed optimization (Scutari et al., 2013).

3. Adaptive Tolerance Sequencing and Staging

To balance inexactness and computational effort, channel-wise partial linearization incorporates a stagewise tolerance regime. One introduces a decreasing sequence $g : \mathbb{R}^N \to \mathbb{R}$9, e.g., $h : \mathbb{R}^N \to (-\infty, +\infty]$0 with $h : \mathbb{R}^N \to (-\infty, +\infty]$1, and at each stage iterates until all block gaps $h : \mathbb{R}^N \to (-\infty, +\infty]$2. This staged refinement ensures progress in a controlled manner and prevents unnecessary minimization accuracy in early iterations (Konnov, 2016).

4. Convergence, Complexity, and Rates

Under typical assumptions (block-Lipschitz continuity of $h : \mathbb{R}^N \to (-\infty, +\infty]$3, convexity and compactness of $h : \mathbb{R}^N \to (-\infty, +\infty]$4, properness and lower semicontinuity of $h : \mathbb{R}^N \to (-\infty, +\infty]$5), channel-wise partial linearization converges to stationary points that solve the mixed variational inequality: $h : \mathbb{R}^N \to (-\infty, +\infty]$6 In the convex case, the function values decrease at a sublinear $h : \mathbb{R}^N \to (-\infty, +\infty]$7 rate, and for accuracy $h : \mathbb{R}^N \to (-\infty, +\infty]$8, the required number of block updates is $h : \mathbb{R}^N \to (-\infty, +\infty]$9 (Konnov, 2016). Each iteration requires only one partial gradient and one block-local minimization—contrasting with full-gradient linearization (e.g., Frank-Wolfe/conditional gradient) which needs full gradient information and updates. For large $$h(x) = \sum_{i=1}^n h_i(x_i), \quad x = (x_1, ..., x_n), \quad x_i \in \mathbb{R}^{N_i}, \sum N_i = N,$$0, per-iteration computational cost is reduced by up to a factor $$h(x) = \sum_{i=1}^n h_i(x_i), \quad x = (x_1, ..., x_n), \quad x_i \in \mathbb{R}^{N_i}, \sum N_i = N,$$1. For nonconvex settings, as in partial-linearization-based best-response algorithms, accumulation points are stationary under diminishing stepsizes and strong convexity of subproblems (Scutari et al., 2013).

5. Methodological Variants in Related Domains

In distributed and multi-agent optimization, channel-wise partial linearization is realized through parallel Jacobi schemes that construct block-local strongly convex surrogate objectives using dynamic pricing (“price mechanisms”) for coupling terms. Each agent solves its own block-separable quadratic program in parallel, communicating only local prices or interference statistics. The result is provably convergent parallel optimization with minimal signaling overhead, applicable to complex resource allocation and interference management (Scutari et al., 2013).

In local perturbative information theory, channel-wise partial linearization is realized via quadratic expansion of information functionals (KL divergence, mutual information) in a tangent-space of probability measures, with each channel treated as a separate block in the Euclideanized geometry. The channel transition operates linearly in this space, and broadcast/multiple access problems become matrix- and singular value-based optimizations over these block coordinates (Huang et al., 2014).

6. Representative Applications

• Large-Scale Machine Learning: Channel-wise partial linearization applies to problems such as set-based SVM duals, where each data object is a set and block updates correspond to sparse dual variable adjustments (Konnov, 2016).
• Traffic Equilibrium: In multi-OD network equilibrium, flows for each origin-destination pair form separate channels; channel-wise updates are implemented via shortest-path solutions and tolerance checking (Konnov, 2016).
• Penalty Decomposition: Quadratic-penalized decompositions in separable resource allocation reduce to composite objectives suited for blockwise linearization and update (Konnov, 2016).
• Wireless Multiuser Optimization: Power control and resource allocation in parallel channels utilize channel-wise SCA/partial linearization with dynamic pricing, enabling scalable distributed computation (Scutari et al., 2013).
• Network Information Theory: Local mutual information and common-message tradeoff problems are mapped to tangent Euclidean space, where channel maps are linear and partial linearization yields the core structure for singular-value-based analysis (Huang et al., 2014).

7. Numerical Performance and Practical Considerations

Empirical studies comparing classic full-gradient linearization and adaptive channel-wise schemes indicate that channel-wise partial linearization can dramatically reduce the number of total partial-gradient evaluations while maintaining near-equivalent total numbers of major iterations. This computational savings is more pronounced as the number of blocks (channels) increases, validating the method’s suitability for high-dimensional, block-structured problems. In practical distributed systems, the communication and computation per agent is minimized, typically requiring solution to low-dimension (often one-dimensional) convex problems per iteration, and only local exchange of price or interference signals (Konnov, 2016, Scutari et al., 2013).