Lightweight Additive Modules

• Lightweight additive modules are compact interventions applied to unchanged base structures to enable domain adaptation, task specialization, and efficient knowledge injection.
• They employ low-dimensional, additive changes—such as low-rank adapters and algebraic constructs—that facilitate parameter efficiency and modular interoperability.
• Empirical findings highlight improved performance, like accuracy boosts in federated learning and enhanced mechanical properties in additive manufacturing, with minimal resource cost.

Lightweight additive modules are a class of constructions—algebraic, geometric, statistical, or architectural—that achieve adaptation, knowledge injection, or modularization via compact additive interventions applied to a base structure. They arise in modern machine learning, representation theory, federated optimization, and additive manufacturing, sharing the unifying trait that the underlying system remains largely unchanged while specialized functionality is introduced or composed through parameter-efficient additive alterations. This approach enables applications ranging from domain-robust LLMs to physical components with enhanced mechanical performance and from modular representations in algebraic structures to cross-device model interoperability in federated systems.

1. Algebraic Frameworks: Modules with Additive Dimension

The notion of “additive modules” in algebra arises centrally in the classification of group representations. Classic results recast minimal representations of symmetric $\operatorname{Sym}(n)$ and alternating $\operatorname{Alt}(n)$ groups using only an “additive dimension”—a generalization of linear dimension encompassing modules over fields, abelian $p$-groups (Prüfer rank), o-minimal dimension, or finite Morley rank (Corredor et al., 2021, Chin et al., 2024). Formally, a module $V$ over a modular universe $U$ is equipped with a dimension function $\dim$ satisfying $\dim V = \dim\ker f + \dim\operatorname{im} f$ for any morphism $f$.

Minimal lightweight additive modules are then either “standard” modules defined as

$$\mathrm{std}(n,L) = \left\{(x_1,\dots,x_n) \in L^n : \sum_{i=1}^n x_i = 0\right\}$$

of dimension $(n-1)\dim L$, or their reduced quotients when the characteristic divides $n$. For $n \geq 7$, every faithful irreducible $\operatorname{Alt}(n)$- or $\operatorname{Sym}(n)$-module of minimal dimension is essentially of this form or its sign-twisted variant. Specific small-$n$ cases correspond to exceptional isomorphisms (e.g., $\operatorname{Alt}(5) \cong \operatorname{SL}(2,4)$), but the overarching paradigm is dimension-optimality, faithfulness, and direct sum decomposability in a setting where only the additive structure is assumed (Corredor et al., 2021, Chin et al., 2024).

2. Lightweight Additive Modules in Model Finetuning

The “additive module” paradigm is fundamental in parameter-efficient finetuning (PEFT) for large pretrained models (Zhang et al., 2023, Xu et al., 2023). Here, a lightweight module is formalized as an additive perturbation $\Delta\theta_M$ applied to frozen parameters $\theta_0$ of a base model: $\theta = \theta_0 + \Delta\theta_M$ where $\Delta\theta_M$ is typically low-dimensional (e.g., LoRA, adapters, scaling vectors) and learned for specific domains. Novel contributions exploit the linearity of such updates: given modules $M_A, M_B$ corresponding to perturbations $\Delta\theta_A, \Delta\theta_B$, addition and negation are defined by

$$M_A \oplus M_B \to \Delta\theta_{A\oplus B} = \Delta\theta_A + \Delta\theta_B,\quad -M_A \to -\Delta\theta_A$$

This arithmetic supports a spectrum of inference-time operations:

• Distribution Generalization: Merging modules trained on disjoint label distributions yields a convex combination module that outperforms individuals, with empirical gains (e.g., GLUE: +1.65 pts) (Zhang et al., 2023).
• Multi-Task Learning: Interpolated merges produce joint modules with higher average accuracy, even when specialized accuracy may drop slightly per task.
• Unlearning: Negating a “toxic” module removes unwanted learned behaviors more cleanly than full retraining (toxicity reduced to near zero) (Zhang et al., 2023).
• Domain Transfer: Additive transformations map modules between domains by leveraging shared features and counterfactual arithmetic.
• Model Composition: Linear combinations with convex or arbitrary coefficients create interpolated or extrapolated skills with no additional gradient steps.

No further retraining is necessary provided modules share a base, and the resulting composite modules retain properties of all constituents—though potential amplification of biases is a known limitation (Zhang et al., 2023).

3. Lightweight Additive Modules via Low-Rank and Adapter Structures

Lightweight additive modules also manifest as low-rank or bottleneck adapters that interface with large backbone models. Examples include:

• Language-Specific Matrix Synthesis (LMS) (Xu et al., 2023): Each language in multilingual MT is assigned a low-rank additive matrix $W_{\text{ls}} = W_v W_f$ (with $$W_v \in \mathbb{R}^{r \times d}, W_f \in \mathbb{R}^{d \times c}$$, $d \ll \min(r, c)$), enabling per-language specialization with $O(d(r+c))$ parameters instead of $O(rc)$. This decomposition is parallel to the base weight and supports both language-wise and pair-wise slotting, dramatically reducing parameter count while maintaining translation quality (e.g., on OPUS-100, LMS+FD achieves BLEU gains at a fraction of MoE’s cost).
• Adapter-based Finetuning in Vision Transformers (Chen et al., 2024): Lightweight “adapter” modules with two-layer bottleneck structure $$(\text{Norm}(h_{\text{in}}) W_{\text{down}}) \xrightarrow{\text{ReLU}} W_{\text{up}}$$ are inserted pre-attention and in the residual branch of each ViT block, increasing trainable parameter count by only 1–2%. All but the adapters remain frozen, enabling rapid convergence (OIS/ODS >0.87 in 50–100 epochs on seismic fault data) and improved generalization while retaining the priors of the base model.

These modules are architecturally orthogonal to their backbones, supporting parallel adaptation (as in FedUNet (Seo et al., 18 Aug 2025)) or injection of per-language or per-domain knowledge (Xu et al., 2023, Chen et al., 2024).

4. Federated and Modular Learning: Additive Modules for Heterogeneity

Lightweight additive modules play a central role in federated learning under device and architecture heterogeneity (Seo et al., 18 Aug 2025). In the “FedUNet” framework, each client attaches a compact U-Net–inspired additive module alongside its arbitrary backbone (e.g., MobileNet, VGG, ResNet). Only the narrow bottleneck of the U-Net (two conv layers) is synchronized globally per round, minimizing communication (0.89 MB/round in the compact setup), while the backbone and other U-Net parameters remain local. The joint output

$$f_{\text{joint}} = f_{\text{base}} + f_{\text{unet}}$$

enables model-agnostic participation across clients. Comparisons show superior or equivalent accuracy (92.68–93.11%) compared to full backbone sharing, but with >99% communication reduction and high robustness to non-IID data (Seo et al., 18 Aug 2025).

5. Additive Modules in Physical Manufacturing and Quantum Technology

In applied physics and engineering, additive manufacturing (AM) enables the construction of “lightweight additive modules” for components where classical subtractive processes are infeasible (Hilpert et al., 2018, Madkhaly et al., 2021). The key design pipeline employs topology optimization to minimize mass $M = \int_\Omega \rho\, dx$ of a structure $\Omega$ subject to stiffness, eigenmode, and manufacturability constraints: $$\text{minimize}\; V(\rho) = \int_\Omega \rho(x)\, dx\ \text{subject to}\; K(\rho)u_i = \lambda_i M u_i,\ \lambda_1 \geq (2\pi f_{\text{min}})^2$$ For optical mirrors (Hilpert et al., 2018), this results in non-manufacturable (by cutting) “honeycomb” structures, realized by SLM from AlSi12 and finished to sub-100 nanometer precision after plating and polishing. An optimized UHV chamber for quantum devices is constructed by SLM with internal lattice infill, yielding 75% mass reduction and UHV compatibility (Madkhaly et al., 2021). Additive modules in this context exploit the AM design envelope for bespoke mechanical and thermal properties, integrated functional geometries, and minimized adjustment or assembly steps.

6. General Principles, Performance Metrics, and Limitations

The defining traits of lightweight additive modules across domains include:

• Parameter- or Mass-Efficiency: Each module represents a compact update—e.g., $<2\%$ of model parameters (Chen et al., 2024), or $O(d(r + c))$ in Transformer's FFN layers (Xu et al., 2023).
• Additivity: Both theoretical (vector space sum, module sum) and practical (arithmetic in weight space, superposition of physical fields/structures) additivity underlies their composition and integration.
• Interoperability: Additive modules can be composed, layered, or exchanged without full retraining or re-architecting, supporting task/domain transfer, federated fusion, or modular upgrades.
• Empirical Validation: Such modules yield statistically significant gains (e.g., +1.65 avg. points on GLUE via composition, or 0.4% accuracy drop for compression with 99% communication reduction in FedUNet), and robust physical performance (e.g., UHV, thermal/mechanical stability, sub-nm finish) (Zhang et al., 2023, Seo et al., 18 Aug 2025, Madkhaly et al., 2021, Hilpert et al., 2018).
• Limitations: All current approaches require compatible base structures (identical frozen base models or compatible manufacturing primitives), and composition can propagate or amplify undesirable features inherent in constituent modules. Safety-oriented validation and further study of mode connectivity remain open (Zhang et al., 2023).

7. Synthesis and Outlook

Lightweight additive modules constitute a cross-disciplinary paradigm for efficient specialization, adaptation, and modularization across algebra, machine learning, federated optimization, and engineered systems. Their theoretical foundations—additive dimension, modular universes, linear composition—support robust practical implementations: adapter-based finetuning, architecture-agnostic federated learning, knowledge distillation via low-rank additive synthesis, and physically optimized, additively manufacturable components. Emerging directions include fine-grained weighting schemes for module merging, extension to heterogeneous base models, enhanced safety testing, and expansion to novel domains such as quantum-classical hybrid control and interactive human-in-the-loop architectures (Zhang et al., 2023, Chen et al., 2024, Seo et al., 18 Aug 2025, Corredor et al., 2021, Madkhaly et al., 2021).