Localized Cost Functions: Theory & Applications

• Localized cost functions are objective formulations that limit cost evaluation to specific subsets of variables, coordinates, orbitals, or samples.
• They are applied in distributed optimization, cost function networks, Wannier orbital localization, and high-dimensional regression to exploit local structure while solving global problems.
• These methods balance local computational benefits with global consistency, addressing trade-offs in tractability, communication overhead, and practical convergence.

Searching arXiv for the cited papers to ground the article in current records. Localized cost functions are objective constructions in which cost evaluation, optimization, or consistency enforcement is restricted to a subset of variables, coordinates, orbitals, or samples rather than being expressed only as a single globally coupled term. In the literature considered here, the notion appears in at least four technically distinct forms: distributed convex optimization with local domains, global cost functions in cost function networks under local transformations, selectively localized spread functionals for Wannier orbitals, and sample-wise local regression objectives coupled by graph regularization (Mota et al., 2013, Allouche et al., 2015, Wang et al., 2014, Yamada et al., 2016). The common theme is not a single formalism but a recurring design principle: exploit structure that is intrinsically local while still solving a global optimization or inference problem.

1. Conceptual scope and formal uses

In distributed optimization, locality is defined by a family of index sets $S_p \subseteq \{1,\dots,n\}$, one per node $p$, so that each local function $f_p$ depends only on $x_{S_p}$. The resulting problem is

$\min_x \sum_{p=1}^P f_p(x_{S_p}),$

with $x \in \mathbb{R}^n$, and is explicitly described as a “localized” or “partial-variable” optimization problem (Mota et al., 2013).

In cost function networks, the relevant object is a global cost function: a mapping assigning an integer cost to every assignment of an unbounded set of variables. Locality enters through scope-restricted cost functions $W_S$, unary costs, and Equivalence Preserving Transformations (EPTs), which shift cost between a scope $S_1$ and a sub-scope $S_2 \subset S_1$ without changing the cost of any full assignment (Allouche et al., 2015).

In electronic-structure theory, localization refers to the spread functional of Wannier functions. The standard Marzari–Vanderbilt spread $\Omega_{MV}$ is replaced by a selectively localized functional $p$0 that minimizes spread only for a chosen subset $p$1 of “objective Wannier functions,” while the remaining functions are left as spectators (Wang et al., 2014).

In high-dimensional regression, the “localized Lasso” assigns a distinct coefficient vector $p$2 to each sample $p$3. Its objective combines per-sample squared loss, sample-wise network regularization, and sample-wise exclusive group sparsity: $p$4 Here locality is sample-indexed rather than coordinate-indexed (Yamada et al., 2016).

A plausible implication is that “localized cost function” is best understood as a family resemblance term. Across these domains, locality may refer to restricted scope, restricted transformation, restricted orbital subset, or restricted sample-specific parametrization.

2. Partial-variable objectives in distributed optimization

The distributed optimization formulation of "Distributed Optimization With Local Domains: Applications in MPC and Network Flows" begins with a connected communication graph $p$5, $p$6, and local functions $p$7 that are closed, proper, convex, and known only to node $p$8 (Mota et al., 2013). The assumption $p$9 excludes the case in which every node depends on the same coordinate. Because each $f_p$0 depends only on some components of $f_p$1, each node is interested only in some components of the optimizer $f_p$2, not the entire vector.

For each coordinate $f_p$3, the paper defines

$f_p$4

and the induced subgraph $f_p$5. A coordinate is called connected if $f_p$6 is connected and non-connected otherwise. This coordinate-wise notion of locality governs both storage and communication: node $f_p$7 stores its private function $f_p$8, its index set $f_p$9, and estimates $x_{S_p}$0 for $x_{S_p}$1; at each iteration it sends neighbor $x_{S_p}$2 only those components for which $x_{S_p}$3. The paper’s communication model is therefore not based on broadcasting the full decision vector, but on exchanging only shared coordinates.

The connected-variable case is rewritten by introducing local copies $x_{S_p}$4 and enforcing equality along edges of each $x_{S_p}$5: $x_{S_p}$6 A proper coloring of $x_{S_p}$7 partitions the nodes into color classes $x_{S_p}$8, after which the reformulation becomes suitable for Extended ADMM. With condensed dual variables $x_{S_p}$9, each color class updates in parallel, computes

$\min_x \sum_{p=1}^P f_p(x_{S_p}),$0

solves

$\min_x \sum_{p=1}^P f_p(x_{S_p}),$1

and then exchanges the updated coordinate copies with neighbors. After all colors update, the dual step is

$\min_x \sum_{p=1}^P f_p(x_{S_p}),$2

For non-connected variables, the method introduces Steiner paths by solving a centralized minimum-Steiner-tree problem with terminals $\min_x \sum_{p=1}^P f_p(x_{S_p}),$3, producing an augmented induced subgraph $\min_x \sum_{p=1}^P f_p(x_{S_p}),$4. The same color-ADMM scheme is then applied on enlarged local domains $\min_x \sum_{p=1}^P f_p(x_{S_p}),$5. For Steiner nodes, the corresponding coordinate update reduces to a simple quadratic and has closed form because $\min_x \sum_{p=1}^P f_p(x_{S_p}),$6 does not depend on that coordinate.

The convergence theorem states that, under the standing convexity assumptions and assuming either $\min_x \sum_{p=1}^P f_p(x_{S_p}),$7 or strong convexity of each group-cost $\min_x \sum_{p=1}^P f_p(x_{S_p}),$8, the Extended ADMM scheme converges to a primal–dual solution. Communication complexity is described in terms of locality: per iteration each node $\min_x \sum_{p=1}^P f_p(x_{S_p}),$9 sends $x \in \mathbb{R}^n$0 real numbers to each neighbor in $x \in \mathbb{R}^n$1, only for shared coordinates, so one ADMM iteration costs $x \in \mathbb{R}^n$2 scalar transmissions. In the reported experiments, the algorithm required fewer communications to converge than prior distributed methods.

3. Local transformations and tractability in cost function networks

The theory developed in "Tractability and Decompositions of Global Cost Functions" places locality at the level of cost scope and cost movement rather than distributed communication (Allouche et al., 2015). A Cost Function Network is $x \in \mathbb{R}^n$3, where $x \in \mathbb{R}^n$4 is a set of discrete variables with domains $x \in \mathbb{R}^n$5, and $x \in \mathbb{R}^n$6 contains a zero-arity constant $x \in \mathbb{R}^n$7, unary costs $x \in \mathbb{R}^n$8, and higher-arity cost functions $x \in \mathbb{R}^n$9. The cost of a full assignment $W_S$0 is

$W_S$1

with $W_S$2 defined by $W_S$3.

Locality is operationalized through EPTs. An $W_S$4-EPT shifts cost between a larger scope $W_S$5 and a sub-scope $W_S$6 with $W_S$7, while preserving the cost of every complete assignment. In the Project operation, for a tuple $W_S$8 and shift amount $W_S$9 satisfying

$S_1$0

one updates

$S_1$1

and

$S_1$2

for every $S_1$3 extending $S_1$4, where $S_1$5 if $S_1$6, else $S_1$7.

The central question is tractable projection-safety. A global cost function $S_1$8 is tractable if minimizing $S_1$9 can be done in polynomial time, and it is tractable $S_2 \subset S_1$0-projection-safe if it remains tractable after any sequence of $S_2 \subset S_1$1-EPTs. The paper gives a sharp dependence on $S_2 \subset S_1$2. Any tractable global cost function is $S_2 \subset S_1$3-projection-safe. If $S_2 \subset S_1$4 is tractable and returns only finite costs, then for any $S_2 \subset S_1$5, $S_2 \subset S_1$6 is not $S_2 \subset S_1$7-projection-safe unless $S_2 \subset S_1$8. The case $S_2 \subset S_1$9 is explicitly identified as borderline: some tractable global cost functions remain tractable after unary EPTs, others do not.

The positive side of the $\Omega_{MV}$0 boundary is represented by flow-based global cost functions such as soft AllDifferent variable-based and soft GCC, which remain flow-based after unary EPTs and therefore are tractable $\Omega_{MV}$1-projection-safe. The negative side is illustrated by a cost function $\Omega_{MV}$2 encoding Max-2-SAT: minimizing $\Omega_{MV}$3 is polynomial, but $\Omega_{MV}$4-extensions can encode Weighted-SAT with weight limit $\Omega_{MV}$5, which is NP-complete. Local cost movement is therefore not innocuous; its tractability depends decisively on the arity of the scope onto which cost is projected.

Two structural mechanisms preserve tractability. The first is polynomial DAG-filterability. A safe DAG-filter represents a cost function $\Omega_{MV}$6 as a directed acyclic graph whose internal nodes aggregate sub-costs through associative–commutative functions and whose leaves are unary costs. If the graph is polynomial in $\Omega_{MV}$7 and projections and extensions distribute safely through the DAG, then minimization can be done by bottom-up dynamic programming and tractability is preserved under unary EPTs. The paper provides examples based on soft grammar cost, soft Among cost, soft Regular cost, and $\Omega_{MV}$8, with explicit polynomial complexities.

The second mechanism is decomposition into a Berge-acyclic bounded-arity network. A $\Omega_{MV}$9-network-decomposition of $p$00 is a polynomial-size CFN over $p$01 with cost functions of arity at most $p$02 such that

$p$03

Under this form, soft local consistencies such as Terminal Directional Arc Consistency (T-DAC*) and Virtual Arc Consistency (VAC) can emulate dynamic programming. The paper proves that, with a suitable variable order, T-DAC* on $p$04 yields the same updated unary cost on the last variable as T-DAC* on the full decomposition, and that VAC yields the same lower bound $p$05 on either representation.

A plausible implication is that locality in cost function networks is ambivalent: it can be the source of tractability when confined to unary projections or safe decompositions, but it can also destroy tractability when cost is pushed onto larger scopes.

4. Selective localization of spread functionals in Wannier theory

In "Selectively Localized Wannier Functions," localization is not attached to variable subsets in an optimization network, but to the spatial spread of a chosen orbital subspace (Wang et al., 2014). The standard Marzari–Vanderbilt spread functional for $p$06 Wannier functions $p$07 is

$p$08

with $p$09 and $p$10. On a discrete $p$11-mesh,

$p$12

where

$p$13

$p$14

and $p$15.

The selectively localized functional restricts the outer summation to a subset $p$16 of “objective Wannier functions”: $p$17 Only the objective orbitals are optimized for maximal localization; the remaining $p$18 orbitals are unconstrained spectators. This is a localized cost function in the sense that the spread penalty acts on a designated subfamily of orbitals rather than on the full basis.

The framework then introduces localized constraints. To fix centers $p$19 at prescribed positions $p$20, the cost is augmented as

$p$21

For one-dimensional point-group symmetry about $p$22, a further term is added: $p$23 Here $p$24 is reflection about $p$25, and the first $p$26 objective Wannier functions are chosen to be even while the rest are odd.

Optimization proceeds through infinitesimal unitary rotations $p$27 with $p$28. Writing

$p$29

the gradient $p$30 is used in the update

$p$31

with $p$32 in practice, $p$33. The implementation embeds this step in a nonlinear conjugate-gradient line-search scheme and declares convergence when $p$34 falls below a chosen tolerance, for example $p$35.

The reported material examples show how selective localization changes the balance between target and spectator orbitals. In GaAs with $p$36 bands and $p$37, MLWF spreads are all approximately $p$38; under SLWF the objective spread decreases to $p$39 while the remaining three spreads increase to approximately $p$40; under SLWF+C centered at As the objective spread is approximately $p$41. The center ratio $p$42 shifts from $p$43 for MLWF to $p$44 for SLWF and to $p$45 for SLWF+C. In SrMnO$p$46, the reported reductions are small: MLWF $p$47-spreads are $p$48 and $p$49, while SLWF gives $p$50 and $p$51. In Co, the method materially alters the $p$52-subspace: MLWF $p$53-spreads are $p$54 and $p$55, while SLWF gives $p$56 and $p$57. The paper states that SLWF preserves clean $p$58 symmetry and reduces unwanted $p$59 hybridization.

These results show a domain-specific meaning of localization: not decomposition of a sum over local functions, but selective concentration of the cost functional on orbitals of interest, together with optional center and symmetry constraints.

5. Sample-wise localized objectives in high-dimensional regression

The localized Lasso introduces one linear model per sample rather than one model per dataset (Yamada et al., 2016). For samples $p$60, features $p$61, responses $p$62, coefficient vectors $p$63, and a symmetric nonnegative similarity matrix $p$64 with $p$65, the objective is

$p$66

Each term has a distinct localized interpretation. The squared loss fits each local linear model to its own sample. The network regularizer $p$67 borrows strength across neighboring samples and induces a soft clustering of local models. The exclusive group-sparsity penalty $p$68, also described as $p$69, promotes sparsity within each $p$70 while encouraging diversity of supports across samples and avoiding the trivial all-zeros solution.

The objective is convex. The squared loss is convex and smooth, and both $p$71 and $p$72 are convex though non-smooth. The paper states that the cost function is convex and thus has a globally optimal solution; the solution is unique under strict convexity in the loss term, for example if the $p$73 are linearly independent.

Optimization is carried out by an iterative least-squares scheme rather than a generic ADMM. Stacking the coefficients into $p$74, the method forms at iteration $p$75 two weighting matrices, $p$76 from the network regularizer and $p$77 from the exclusive group-sparsity term, and solves

$p$78

where $p$79 is a suitable block-diagonal design matrix built from the $p$80. The paper describes the procedure as simple, efficient, parameter-free, monotonically convergent, and guaranteed to converge to a globally optimal solution. With the Woodbury identity, the cost of the linear solve is $p$81 rather than $p$82. Forming $p$83 and $p$84 costs $p$85, so the total per-iteration cost is $p$86, and the total cost is $p$87, where $p$88 is the number of iterations to convergence.

The empirical results reported in the paper are explicitly tied to both prediction and interpretability. On synthetic high-dimensional regression with $p$89 samples, $p$90 true features, and $p$91 clusters of $p$92 samples each, Localized Lasso recovers the three true support patterns almost perfectly, while Network Lasso without $p$93 is dense and Network Lasso+$p$94 either fails or shrinks entire models to zero under heavy regularization. The method converges in approximately $p$95 iterations, and runtime scales linearly in $p$96. In personalized medicine using toxicogenomics data with $p$97-drug activity prediction based on $p$98 gene expressions, the reported RMSE is approximately $p$99, compared with $f_p$00 for Network Lasso, $f_p$01 for Lasso, $f_p$02 for FORMULA, and $f_p$03 for a kernel method; the average number of selected genes is approximately $f_p$04, compared with approximately $f_p$05 for Network Lasso, $f_p$06 for FORMULA, and $f_p$07 for Elastic Net. The paper also reports higher Adjusted Rand Index on synthetic convex clustering and real-world clustering benchmarks such as COIL20 and NCII lymphoma gene data.

In this setting, localization is explicitly sample-specific. The model does not merely impose local penalties on a global parameter vector; it instantiates a collection of local sparse models connected by graph regularization.

6. Common patterns, distinctions, and recurring misconceptions

Across these four literatures, localized cost functions share a common structural objective: exploit partial dependence while preserving a meaningful notion of global solution. In distributed optimization, the global minimizer is reconstructed from coordinate copies constrained to agree on induced subgraphs (Mota et al., 2013). In cost function networks, local transformations preserve the value of full assignments while potentially changing tractability (Allouche et al., 2015). In Wannier theory, a subset of orbitals is designated for localization while the optimization still proceeds over the gauge degrees of freedom of the full subspace (Wang et al., 2014). In localized Lasso, every sample receives its own coefficient vector, but these vectors are tied by a network regularizer and solved jointly in one convex objective (Yamada et al., 2016).

One recurring misconception is to treat “localized” as synonymous with “approximate” or “heuristic.” The sources do not support that identification. The distributed ADMM method has formal convergence under stated assumptions. The Localized Lasso objective is convex and has a globally optimal solution. In the cost-function-network setting, locality may preserve tractability exactly for $f_p$08 and for some $f_p$09 classes, while for $f_p$10 it can render a previously tractable class non-projection-safe unless $f_p$11. In the Wannier setting, selective localization is still implemented by an explicit gradient-based variational procedure.

A second misconception is that localization always decreases computational burden without trade-offs. The evidence is more specific. In distributed optimization, communication is reduced because each node exchanges only shared coordinates, and the reported benchmarks show fewer communication rounds than several alternatives. In Wannier optimization, concentrating the spread objective on a subset of orbitals can decrease the spread of those orbitals while increasing the spreads of spectator orbitals, as the GaAs example makes explicit. In cost function networks, local transformations can either preserve or destroy tractability depending on the projection arity. In localized Lasso, interpretability arises from sparse local models, but the optimization still requires repeated solution of a regularized least-squares system.

A plausible implication is that the principal scientific value of localized cost functions lies less in a single canonical definition than in a methodological pattern: identify the part of the model where locality is structurally meaningful, localize the objective or transformation there, and then recover global coherence through consensus constraints, decomposition theorems, variational gauge updates, or coupled convex regularization.