Blockwise & Grouped Patterns: Regression & Combinatorics

• Blockwise and grouped patterns are a framework combining structural sparsity in regression with symmetry in combinatorial patterns to enhance model interpretability.
• They use hierarchical priors and smooth surrogate optimization to achieve simultaneous inter- and intra-group sparsity, reducing false positives in high-dimensional data.
• Group actions in pattern theory enable tractable analysis of cyclic and symmetric structures, facilitating applications like wait-time analysis and nontransitive game strategies.

Blockwise and grouped patterns represent a foundational paradigm for structuring and analyzing both statistical models (notably in high-dimensional regression) and combinatorial objects (notably in the study of pattern-generating systems with symmetry). Two principal lines of research exemplify this paradigm: (1) the use of block-structured priors for grouped variable selection in regression models via nested spike-and-slab constructions, and (2) the study of patterns arising from group actions on words, particularly as applied to nontransitive games and wait-time analysis. These approaches exploit blockwise or group-level symmetries and constraints, often yielding substantial advantages in model selection, interpretability, and analytic tractability.

1. Blockwise Patterns in Grouped Variable Selection

In regression with grouped covariates, suppose $p$ features are partitioned into $m$ nonoverlapping groups $G_1, \dots, G_m$ of cardinalities $q_1, \dots, q_m$, with regression coefficients $\beta_g \in \mathbb{R}^{q_g}$. To model both between- and within-group sparsity, a nested spike-and-slab prior is introduced (Yen et al., 2011):

• Group-level prior: For each group $g$, a Bernoulli indicator $\gamma_g \sim \mathrm{Bern}(\theta_g)$ determines whether the group is active. Marginally, $\beta_g$ follows

$$f(\beta_g) = \theta_g\, \text{(slab density)} + (1-\theta_g)\, \delta_0(\|\beta_g\|_2),$$

where, with probability $1-\theta_g$, the entire subvector $\beta_g$ is exactly zero.

• Within-group prior: Conditional on $\gamma_g = 1$, each coordinate $j \in G_g$ receives its own spike-and-slab prior via a Bernoulli $\alpha_j | \gamma_g \sim \mathrm{Bern}(\omega_j)$, i.e.,

$$\beta_j | \gamma_g, \alpha_j \sim \alpha_j\, \mathcal{N}(0, \sigma^2/\lambda) + (1-\alpha_j)\, \delta_{(-\xi,\xi)}(\beta_j),$$

with $\xi\to0$.

This hierarchical construction can induce both exact block sparsity (entire groups zeroed out) and within-group sparsity (individual zeros within active blocks).

2. MAP Objective and Surrogate Optimization

The posterior mode estimation problem leads to an objective comprising both block- and coordinate-level penalties. For Gaussian regression $y\sim \mathcal{N}(X\beta, \sigma^2 I)$, the negative log-posterior (up to additive constants) is:

$$V(\beta) = \frac{1}{2\sigma^2}\|y - X\beta\|_2^2 + \lambda \sum_{g=1}^m \|\beta_g\|_2^2 + \rho_1 \sum_{j=1}^p \mathbb{I}\{\beta_j \neq 0\} + \rho_2 \sum_{g=1}^m \sqrt{q_g\, \mathbb{I}\{\|\beta_g\|_2 \neq 0\}}.$$

To render the problem tractable, each indicator is approximated by a smooth log-sum surrogate:

$$\mathbb{I}\{a \neq 0\} \approx g_\tau(a) = \frac{\ln(1 + |a|/\tau)}{\ln(1+\tau^{-1})}, \qquad \tau \to 0,$$

which majorizes to weighted $\ell_1$ and group-$\ell_2$ penalties. The resulting surrogate, at iterate $\beta^{(d)}$, is convex in $\beta$:

$$Q^{(d)}(\beta) = \frac{1}{2\sigma^2}\|y - X\beta\|_2^2 + \lambda \sum_g \|\beta_g\|_2^2 + \lambda_1 \sum_j \nu_j^{(d)} |\beta_j| + \lambda_2 \sum_g \phi_g^{(d)} \|\beta_g\|_2,$$

where $\nu_j^{(d)} = (|\beta_j^{(d)}| + \tau)^{-1}$ and $\phi_g^{(d)} = (\|\beta_g^{(d)}\|_2 + \tau)^{-1}$.

3. Blockwise Coordinate-Descent Algorithms

The surrogate objective facilitates minimization via blockwise coordinate descent. For each group $g$:

1. Zero-block test: The KKT subgradient at $b = 0$ yields the criterion

$$\left\|\, \mathrm{ST}_{\lambda_1 \nu_{G_g}}\left(2X_g^T r_{-g}\right) \,\right\|_2 \leq \lambda_2\,\phi_g^{(d)},$$

where $r_{-g} = y - \sum_{h\neq g} X_h \beta_h$ and $$\mathrm{ST}_{\lambda v}(z)_j = \mathrm{sign}(z_j)\max\{|z_j|-\lambda v_j, 0\}$$.

If true, set $\beta_g = 0$.

2. Nonzero-block update: Otherwise, a strictly convex quadratic problem yields

$$\beta_g^{\rm new} = \left(X_g^T X_g + w_g I \right)^{-1} \mathrm{ST}_{\lambda_1 \nu_{G_g}/2}\left( X_g^T r_{-g} \right),$$

with $$w_g = \lambda + \lambda_2 \phi_g^{(d)} / (2\|\beta_g^{(d)}\|_2)$$.

This two-stage procedure ensures exact block zeros and soft-thresholded updates within active blocks. Majorization-minimization is iterated until convergence.

4. Theoretical Guarantees and Label-Invariance

Under standard regularity on the design matrix $X$ (e.g., restricted eigenvalue conditions) and appropriate growth rates for $\lambda, \rho_1, \rho_2 = O(\sqrt{n})$, key properties can be established (Yen et al., 2011):

• Estimation error bound: If the true support lies in $r$ blocks covering $q_R$ coordinates,

$$\| \hat{\beta} - \beta^* \|_2 = O\left( \frac{1}{\sqrt{n}} \sqrt{ q_R \ln m } \right)$$

with high probability. When $q_g \ll p$ and $r \ll m$, this can improve upon the lasso rate $O(\sqrt{s \ln p / n})$.

• Label-invariance: Provided $\rho_2 \max_g \sqrt{q_g} = o(\ln n)$, the estimator becomes asymptotically invariant to the choice of grouping as $\tau \to 0$.
• Sign-consistency: Under Gaussian errors and $p = o(n/\ln^2 n)$, with no irrepresentable-type condition, $$\Pr[\,\mathrm{sign}(\hat{\beta}) = \mathrm{sign}(\beta^*)\,] \to 1$$.

These results indicate that block-structured priors can induce simultaneous inter- and intra-group sparsity with favorable finite-sample and asymptotic guarantees.

5. Pattern Formation by Group Action: Blockwise Reductions

A parallel formalism emerges in the combinatorics of patterns under group action (Khovanova et al., 2020). Let $\mathcal{A}$ be an alphabet of size $q$, and $G \subset S_q$ a group acting on $\mathcal{A}$ by permuting letters, which extends letterwise to words $w = w_1\cdots w_\ell \in \mathcal{A}^\ell$: $g \cdot w = (g\cdot w_1)\cdots (g\cdot w_\ell)$.

• The orbit $G \cdot w = \{g \cdot w : g \in G\}$ and its stabilizer $G_w = \{g \in G : g \cdot w = w\}$.
• The set of patterns of length $\ell$ is identified with the set of orbits.

When $G$ factors as a product or acts on blocks, there is often a bijection between patterns of length $\ell$ and words of reduced length. The cyclic group $C_q$ acting on $Z/qZ$ under Caesar shift exemplifies this principle: every pattern of length $\ell$ is determined by its adjacency signature $S(p) \in (Z/qZ)^{\ell-1}$. Thus, analysis of avoidance, generating functions, and waiting times for blockwise group patterns can be reduced to lower-dimensional classical problems.

6. Statistical and Combinatorial Consequences

Blockwise and grouped structures in both regression and pattern theory enforce structural constraints that shape model selection and pattern occurrence statistics:

• In regression, simulation [(Yen et al., 2011), Table 1] shows that the grouped variable selection via nested spike-and-slab (gvsnss) outperforms lasso and group lasso when support lies within a few groups, particularly when needing to detect within-group zeros. Specifically, in a scenario with $p=100$, $m=10$, $r=2$ active groups, and five within-group nonzeros per active group, gvsnss yields lower false positive rates (7.8%) and $L_2$ error (0.95), with 68% correct detection of within-group zeros, compared to higher false positive rates and lower within-group specificity for standard lasso and group lasso.
• In combinatorial pattern matching, blockwise group actions enable explicit calculation of pattern-based Conway leading numbers, expected wait times, and non-transitive game strategies, especially under cyclic and symmetric group action (see, e.g., Section 8 and 9 of (Khovanova et al., 2020)).

Method	FPR (%)	$\|\widehat\beta-\beta^*\|_2$	Within-group zero detections
lasso	24.5	1.04	0.21
group lasso	9.2	1.07	0.00
gvsnss	7.8	0.95	0.68

A plausible implication is that methodologies exploiting blockwise or grouped patterns, whether via hierarchical priors or group actions, support refined inference and analytic tractability in structured high-dimensional or symmetric settings.

7. Synthesis and Broader Implications

Blockwise and grouped patterns, manifested as either hierarchical priors in regression or as group actions partitioning word spaces, provide a unifying abstraction for imposing and exploiting structural constraints. In both settings, algorithms and theoretical results leverage block structure to improve selection specificity, estimation accuracy, and enable tractable computation or exact enumeration. These principles, demonstrated respectively by the gvsnss estimator for regression and group-action pattern theory in combinatorics, suggest broad applicability for model selection, symmetry exploitation, and the design of algorithms that require discrimination at multiple hierarchical or group levels (Yen et al., 2011, Khovanova et al., 2020).