Bi-Dimensional Decomposition (BoRA)

• Bi-dimensional Decomposition (BoRA) is a method that exploits two-dimensional block or symmetric decompositions to enhance fine-tuning and logic synthesis.
• It extends low-rank adaptation techniques by partitioning weight updates to raise effective rank with only a small parameter overhead.
• Empirical results show BoRA's superior performance in natural language understanding and digital logic, outperforming standard approaches with minor extra cost.

Bi-dimensional Decomposition (BoRA) refers to a class of strategies for enhancing expressivity and efficiency in neural network adaptation and Boolean logic synthesis by exploiting structured, two-dimensional (bi-dimensional) decompositions—typically involving block diversity, symmetric magnitude modulation, or optimal variable partitioning. Three prominent lines in the literature employ the "BoRA" terminology: Block-Diversified Low-Rank Adaptation for parameter-efficient fine-tuning in deep learning (Li et al., 9 Aug 2025), Bi-dimensional Weight-Decomposed Low-Rank Adaptation for magnitude-symmetric neural adaptation (Wang et al., 2024), and QBF-based Boolean Function Bi-Decomposition in logic synthesis (Chen et al., 2011). All BoRA approaches share the use of two-way decompositions that increase structural or functional diversity and improve task performance under resource constraints.

1. Background and Motivation

Parameter-efficient fine-tuning (PEFT) has become essential for adapting large neural models with minimal overhead, typically using low-rank adaptation (LoRA). LoRA decomposes an update to a pretrained weight $W \in \mathbb{R}^{m \times n}$ as $BA$ with $$A \in \mathbb{R}^{r \times n}, B \in \mathbb{R}^{m \times r}$$, $r \ll \min(m, n)$, yielding an update of rank at most $r$ with $(m+n)r$ parameters. Increasing $r$ improves capacity but quickly increases costs.

Block-Diversified Low-Rank Adaptation (BoRA) (Li et al., 9 Aug 2025) extends LoRA by imposing a two-dimensional block structure and introducing block-wise diagonal modulations to maximize expressivity with only a small increase in parameter count. Separately, the bi-dimensional weight-decomposed BoRA approach (Wang et al., 2024) introduces symmetric row- and column-wise magnitude modulation on adapter weights, and QBF-based BoRA in logic synthesis (Chen et al., 2011) seeks optimal variable partitioning for Boolean function bi-decomposition. In each context, bi-dimensionality (via explicit row/column or blockwise structure) enables a richer adaptation or decomposition.

2. Block-Diversified Low-Rank Adaptation (BoRA) in Neural Networks

BoRA (Li et al., 9 Aug 2025) generalizes LoRA by treating its low-rank parameter update as a grid of $b \times b$ blocks. Specifically, $A$ is partitioned by columns and $B$ by rows:

• $BA$0, $BA$1
• $BA$2, $BA$3

The update becomes

$BA$4

Each sub-block $BA$5. To break inter-block coupling and raise the effective rank, BoRA inserts for each block a diagonal matrix $BA$6: $BA$7 With $BA$8 such matrices, total parameters increase only by $BA$9, since each $$A \in \mathbb{R}^{r \times n}, B \in \mathbb{R}^{m \times r}$$0 is diagonal. The update generalizes to

$$A \in \mathbb{R}^{r \times n}, B \in \mathbb{R}^{m \times r}$$1

where $$A \in \mathbb{R}^{r \times n}, B \in \mathbb{R}^{m \times r}$$2 concatenates all row blocks, $$A \in \mathbb{R}^{r \times n}, B \in \mathbb{R}^{m \times r}$$3 stacks the column blocks, and $$A \in \mathbb{R}^{r \times n}, B \in \mathbb{R}^{m \times r}$$4 is block-diagonal from all $$A \in \mathbb{R}^{r \times n}, B \in \mathbb{R}^{m \times r}$$5.

Effective rank: BoRA raises the upper bound of the update rank from $$A \in \mathbb{R}^{r \times n}, B \in \mathbb{R}^{m \times r}$$6 (the LoRA limit) to $$A \in \mathbb{R}^{r \times n}, B \in \mathbb{R}^{m \times r}$$7. This allows higher expressivity at fixed or slightly increased parameter count.

Parameter overhead: Adding $$A \in \mathbb{R}^{r \times n}, B \in \mathbb{R}^{m \times r}$$8 parameters for $$A \in \mathbb{R}^{r \times n}, B \in \mathbb{R}^{m \times r}$$9 to $r \ll \min(m, n)$0 for $r \ll \min(m, n)$1 and $r \ll \min(m, n)$2 is negligible when $r \ll \min(m, n)$3 and $r \ll \min(m, n)$4 are small and $r \ll \min(m, n)$5 are large.

Implementation: For each input, the computation is blockwise: project each input slice through its $r \ll \min(m, n)$6, modulate via $r \ll \min(m, n)$7, combine across $r \ll \min(m, n)$8 for each output block $r \ll \min(m, n)$9, then project via $r$0 and sum.

3. Bi-dimensional Weight-Decomposed Low-Rank Adaptation (BoRA) for Symmetric Modulation

The variant introduced in (Wang et al., 2024) addresses the asymmetry of DoRA, which only introduces column-wise (vertical) magnitude scaling. This BoRA symmetrically modulates both rows and columns:

1. Adapter construction: Given frozen $r$1,

$r$2

with learnable $r$3 and $r$4.

2. Row-wise normalization and scaling: Normalize each row of $r$5 and scale by $r$6:

$r$7

3. Column-wise normalization and scaling: Normalize each column of $r$8 and scale by $r$9:

$(m+n)r$0

This approach aligns the adaptation to both input (column) and output (row) sensitivities, providing a bi-dimensional symmetry that matches empirical patterns in full-parameter fine-tuning.

4. Rank, Parameter, and Compute Characteristics

Method	Update Rank	Trainable Parameters	Extra Overhead
LoRA	$(m+n)r$1	$(m+n)r$2	$(m+n)r$3 FLOPs
BoRA (block)	$(m+n)r$4	$(m+n)r$5	$(m+n)r$6
DoRA	$(m+n)r$7	$(m+n)r$8	$(m+n)r$92x LoRA
BoRA (symm)	$r$0	$r$1	2 normalizations

With $r$2, $r$3 small, and $r$4, additional costs from BoRA in either formulation are minor relative to gains in expressivity or transfer performance (Li et al., 9 Aug 2025, Wang et al., 2024).

5. Empirical Results and Comparative Performance

Block-partitioned BoRA with $r$5 and $r$6–$r$7 consistently outperforms vanilla LoRA (same rank) by $r$8–$r$9 absolute across natural language understanding (GLUE/commonsense reasoning) and reasoning (Gemma/LLaMA/Qwen) tasks, and often matches or exceeds LoRA at $b \times b$0 the rank (Li et al., 9 Aug 2025).

Bi-dimensional magnitude-modulated BoRA outperforms LoRA and DoRA across MT-Bench and commonsense NLU with only slight parameter increase, e.g., Llama-2-7B: BoRA (2.35% params) achieves 6.76 (MT-Bench), compared to LoRA (2.32%) at 6.16 and DoRA (2.33%) at 6.38 (Wang et al., 2024).

Ablation studies indicate both diagonal parameterization and normalization in $b \times b$1 are crucial, with weak utilization or gradient issues if omitted. Singular value analysis confirms that BoRA achieves substantially more effective singular values in $b \times b$2, aligning with its theoretical rank increase.

6. QBF-Based BoRA in Boolean Function Bi-Decomposition

In digital logic, bi-decomposition refers to splitting a Boolean function $b \times b$3 into two subfunctions combined by a binary gate. The QBF-based BoRA scheme (Chen et al., 2011) achieves optimal variable partitioning for

$b \times b$4

where $b \times b$5 is partitioned into $b \times b$6, $b \times b$7, $b \times b$8, and $b \times b$9. The method uses Quantified Boolean Formulas (QBF) over partition indicator variables $A$0 and universally-quantified copies of inputs to enforce the decomposition and desired metrics (disjointness, balancedness).

Empirically, this method yields strictly higher-quality decompositions versus prior SAT/BDD-based approaches, with optimal guarantees and scalable performance on industrial circuits. Disjointness, balancedness, and joint metrics improve across 13%–80% of test cases versus SAT-based baselines, at modest additional computational cost.

7. Extensions, Limitations, and Perspectives

BoRA offers a principled route to increased expressivity with controllable overhead in both neural PEFT and Boolean logic synthesis. In the adaptation setting:

• Scaling $A$1 increases effective update rank until overfitting occurs.
• Extra diagonal parameters are negligible for practical $A$2, $A$3.
• The block/symmetric decompositions integrate with existing architectures without architectural redesign.

Potential extensions include grouping rows/columns for structured sharing, learning asymmetric row/column ranks, and combining with quantized or further-structured modules. Limitations include the possibility of overfitting if $A$4 is too large or if not paired with sufficient regularization. Slight increases in computation (1–5%) are observed due to block or normalization operations.

In summary, bi-dimensional decomposition via BoRA defines a family of methods wherein blockwise or symmetric modulation/gating multiplies capacity and expressivity at minimal additional cost, with robust empirical advantages across architectures and domains (Li et al., 9 Aug 2025, Wang et al., 2024, Chen et al., 2011).