---
title: Block Coordinate Descent Methods
url: https://www.emergentmind.com/topics/block-coordinate-descent
type: topic
---

# Block Coordinate Descent Methods

Block coordinate descent (BCD) is a fundamental algorithmic paradigm for high-dimensional optimization problems, wherein variables are partitioned into blocks and optimization proceeds by successively updating one or several blocks while keeping the others fixed. By decomposing large-scale, often structured objectives into tractable subproblems, BCD achieves scalability, memory efficiency, and the ability to exploit problem structure across a wide range of convex and nonconvex settings.

## 1. Fundamental Principles and Variants

The defining characteristic of BCD is the block-wise update mechanism. For a problem of the form
\[
\min_{x} f(x), \qquad x = (x^{(1)},\dots,x^{(p)}),\ x^{(s)} \in \mathbb{R}^{n_s}
\]
the method iterates by selecting a block \(s_k\) at step \(k\) and performing a block-specific update, e.g.,
\[
x^{(s_k)}_{k+1} = \arg\min_{z} f(x^{(1)}_{k+1},\dots,x^{(s_k-1)}_{k+1},z,x^{(s_k+1)}_k,\dots,x^{(p)}_k)
\]
with various rules to select the block and perform the update (exact minimization, gradient, Newton, or proximal steps).

Block selection schemes include:
- **Cyclic**: Scan through blocks in a fixed order.
- **Randomized**: Sample blocks i.i.d. or using Markov chains [1811.08990].
- **Greedy**: Select the block expected to yield maximal decrease, e.g., Gauss-Southwell-type rules [1712.08859].
- **Flexible/Variable**: Priority or arbitrary deterministic scheduling subject to regular access (e.g., K-cyclic) [2510.26477].

Block updates can be:
- **Exact minimization** on the

Source: https://www.emergentmind.com/topics/block-coordinate-descent