Domain Mixed Unit in Neural Arithmetic

• Domain Mixed Unit (DMU) is a neural arithmetic layer that mixes linear-space and log-space computations via a two-element gate for effective arithmetic extrapolation.
• It integrates addition/subtraction in linear domain and multiplication/division in log domain, achieving 100% success in NALM benchmarks.
• Its minimalist design with only two trainable parameters ensures a smooth optimization landscape and reliable handling of magnitudes and signs.

Searching arXiv for the cited DMU paper and key related neural arithmetic papers. First, retrieving the Domain Mixed Unit paper. Searching for "Domain Mixed Unit neural arithmetic" on arXiv. The Domain Mixed Unit (DMU) is a neural arithmetic layer that learns a single gate mixing linear-space and log-space computations while applying either addition or subtraction over two inputs. In its benchmark configuration, the layer is instantiated as DMU_add and DMU_sub, keeps the operand selector fixed, trains only a two-element gate, and is designed to improve out-of-distribution arithmetic generalization, particularly for multiplication and division on the NALM benchmark (Curry, 9 Sep 2025). The acronym is overloaded across fields: in Data Envelopment Analysis, DMU conventionally denotes Decision Making Unit, not Domain Mixed Unit (Kitahara et al., 2024, Mehdiloozad et al., 2015, L. et al., 2015, Liu et al., 2023).

1. Terminological scope and research context

Within neural arithmetic, the Domain Mixed Unit belongs to the family of differentiable modules that encode arithmetic inductive biases into neural networks. The stated motivation is the long-standing difficulty of learning addition, subtraction, multiplication, and division with extrapolation capability rather than merely interpolating within the training range. The DMU addresses this by restricting the learning problem to a single question: in which domain should the primitive operation be executed? Under this formulation, addition and subtraction are handled in linear space, while multiplication and division are obtained by performing addition or subtraction in log space and exponentiating the result (Curry, 9 Sep 2025).

This design positions the DMU against architectures such as NAU, NALU, NMU, NPU/RealNPU, and related variants, which are described as often learning brittle circuits or unstable mixtures of operations. The NALM benchmark provides the paper’s evaluation setting, with standardized protocols and success criteria across input ranges including negative ranges and ranges near zero. A central claim is that reducing the model to a single learned gate avoids competing sub-networks and saddle points, thereby yielding a smoother optimization landscape and more reliable extrapolation (Curry, 9 Sep 2025).

The acronym overlap with DEA is not a minor terminological detail. In operations research, “DMU” is consistently used for observed productive entities such as firms, banks, hospitals, or branches, and several DEA papers explicitly note that “Domain Mixed Unit” is a misexpansion in that literature (Kitahara et al., 2024, Mehdiloozad et al., 2015, L. et al., 2015, Liu et al., 2023). In contemporary machine learning, however, the term “Domain Mixed Unit” denotes the neural arithmetic layer introduced in 2025 (Curry, 9 Sep 2025).

2. Architectural principle

The DMU is defined for two inputs, $x = [x_1, x_2] \in \mathbb{R}^2$, and has two operational initializations. The architecture differs between these variants only through a frozen operand selector $O$; the trainable component is a two-element gate $G = [G_{\text{lin}}, G_{\text{log}}]$ that mixes linear-domain and log-domain paths (Curry, 9 Sep 2025).

Variant	Frozen $O$	Covered operations
DMU_add	$[1, 1]$	addition and multiplication
DMU_sub	$[1, -1]$	subtraction and division

The key restriction is that the unit does not learn a complex operator-selection circuit in the benchmark configuration. Instead, it applies the same primitive structure in two domains and lets the gate interpolate between them. For DMU_add, the linear path corresponds to addition and the log path corresponds to multiplication. For DMU_sub, the linear path corresponds to subtraction and the log path corresponds to division. With $G_{\text{lin}} \rightarrow 1$ and $G_{\text{log}} \rightarrow 0$, the unit reduces to the linear-space behavior; with $G_{\text{lin}} \rightarrow 0$ and $G_{\text{log}} \rightarrow 1$, it reduces to the log-space behavior (Curry, 9 Sep 2025).

This formulation makes the layer unusually small. In the benchmark configuration, the gate values are the only trainable parameters, the total parameter count is two, and the forward-pass computational complexity is constant for the two-input case. The reported memory footprint is correspondingly minimal relative to multi-branch units such as NALU and NPU (Curry, 9 Sep 2025).

3. Forward map and sign handling

The forward computation explicitly separates magnitudes from signs. Inputs are decomposed as

$O$0

Using the frozen selector $O$1, the layer computes linear-space and log-space operations on magnitudes,

$O$2

The sign terms are then computed pathwise:

$O$3

where $O$4 is a temperature on the linear-path sign (Curry, 9 Sep 2025).

The log-path sign is determined by the parity of the number of negative inputs. Writing

$O$5

the term $O$6 yields $O$7 for an even number of negative inputs and $O$8 for an odd number, covering both multiplication and division sign behavior. The mixed sign is

$O$9

Magnitude mixing is performed in log space for numerical stability:

$G = [G_{\text{lin}}, G_{\text{log}}]$0

$G = [G_{\text{lin}}, G_{\text{log}}]$1

$G = [G_{\text{lin}}, G_{\text{log}}]$2

The smooth absolute value prevents undefined logarithms when $G = [G_{\text{lin}}, G_{\text{log}}]$3 (Curry, 9 Sep 2025).

The two variants follow directly from the choice of $G = [G_{\text{lin}}, G_{\text{log}}]$4. For DMU_add, $G = [G_{\text{lin}}, G_{\text{log}}]$5, so the linear path computes $G = [G_{\text{lin}}, G_{\text{log}}]$6 and the log path computes $G = [G_{\text{lin}}, G_{\text{log}}]$7, whose exponentiation yields $G = [G_{\text{lin}}, G_{\text{log}}]$8. For DMU_sub, $G = [G_{\text{lin}}, G_{\text{log}}]$9, so the linear path computes $O$0 and the log path computes $O$1, whose exponentiation yields $O$2 (Curry, 9 Sep 2025).

The log path uses $O$3 and therefore expects non-zero inputs. The benchmark ranges avoid exact zero. For implementations beyond the benchmark, the paper recommends ensuring strictly positive inputs to the log path or adding small input-side epsilons or clipping (Curry, 9 Sep 2025).

4. Initialization, optimization, and benchmark behavior

Two initializations are proposed. The addition/multiplication initialization fixes $O$4 and biases the gate toward either addition or multiplication depending on whether $O$5 or $O$6. The subtraction/division initialization fixes $O$7 and analogously biases the gate toward subtraction or division. In the benchmark configuration, $O$8 remains frozen and only $O$9 is trained (Curry, 9 Sep 2025).

The reported training setup follows the NALM benchmark with one substantive modification: increasing the learning rate from $[1, 1]$0 to $[1, 1]$1 was required for full extrapolation on all tasks. A temperature scalar can also be applied to the gate to encourage discretization. The paper notes that a gate temperature of $[1, 1]$2 helps convergence on the smallest negative multiplication range by avoiding a saddle point where a 20/80 lin/log mixture approximates most multiplication operations. Tighter temperatures may be needed for ranges closer to zero, but may also introduce instability (Curry, 9 Sep 2025).

On the NALM benchmark, the paper reports that DMU_add and DMU_sub “both perform optimally” for their respective operation pairs and that the DMU has “no differing behavior between different seeds.” The reported success rates are 100% across all reported ranges for addition and subtraction, and 100% across all reported ranges for multiplication and division as well. The abstract further states that the DMU achieves “the highest percentage solved over all seeds” on multiplication and division (Curry, 9 Sep 2025).

The comparative framing is specific. For addition, the DMU matches NAU and iNALU and exceeds NALU and G-NALU on the reported ranges. For subtraction, it matches NAU and typically exceeds iNALU, NALU, and G-NALU. For multiplication, the reported 100% scores across all ranges exceed NMU, iNALU, RealNPU, NALU, and G-NALU on the paper’s benchmarks, including challenging ranges such as $[1, 1]$3 and $[1, 1]$4. For division, the reported 100% scores exceed iNALU, RealNPU, NPU, NALU, and NAC* in the same evaluation setting (Curry, 9 Sep 2025).

5. Interpretation, ablations, and limitations

The paper’s main explanatory claim is that the DMU is reliable because it simplifies the optimization problem. Freezing $[1, 1]$5 and learning only the domain-mixing gate $[1, 1]$6 appears to provide a smooth descent direction, whereas unfreezing $[1, 1]$7 or adding compositional structure introduces extra degrees of freedom and additional saddle points. In the reported ablations, convergence reliability declines when the unit is extended compositionally or when $[1, 1]$8 is no longer fixed (Curry, 9 Sep 2025).

Theoretical analysis in the paper is primarily intuitive rather than formal. The central intuition is that addition and subtraction are naturally linear-domain operations, while multiplication and division become addition and subtraction in log space. By forcing the network to decide only which domain to use for the same primitive, the DMU reduces the competition between sub-units. This suggests that the layer’s inductive bias is concentrated in domain selection rather than in jointly learning domain, operation, and operand structure (Curry, 9 Sep 2025).

Several practical limitations remain. The log-domain path is sensitive near zero because it depends on $[1, 1]$9, so exact zeros are excluded in the benchmark and require special handling in broader deployments. Tight gate temperatures can encourage sharper domain selection, but may destabilize training on very small ranges. The benchmark configuration is also intentionally narrow: two inputs, frozen $[1, -1]$0, and only two trainable parameters. The paper notes that strict reliability degrades when this simplicity is relaxed (Curry, 9 Sep 2025).

Sign handling is an additional design constraint rather than a secondary implementation detail. The linear-path sign is a smooth approximation based on $[1, -1]$1, while the log-path sign is computed deterministically by $[1, -1]$2. This separation of sign from magnitude is central to the DMU’s ability to handle negative inputs while preserving the multiplicative and divisive interpretation of the log path (Curry, 9 Sep 2025).

6. Extensions, applications, and resources

The proposed applications are tasks that require reliable arithmetic reasoning and compositional generalization within larger models, including program induction components, numerical reasoning in NLP, and differentiable calculators embedded in end-to-end systems. The layer is also positioned as useful where multiplication and division extrapolation is critical and where stability and interpretability of arithmetic behavior matter (Curry, 9 Sep 2025).

A sketched extension generalizes the unit from a single two-input operation to directed acyclic graphs of depth $[1, -1]$3. In that setting, $[1, -1]$4 would be unfrozen, $[1, -1]$5 would become a length-$[1, -1]$6 vector choosing the domain at each step, and masking would enforce valid connectivity. The paper describes a procedure in which working magnitudes and signs are propagated across DAG steps, enabling expressions such as $[1, -1]$7 through three steps with appropriate rows of $[1, -1]$8 and corresponding gate choices (Curry, 9 Sep 2025).

The implementation resources are concrete. The DMU is intended for submission as a pull request to the open-source NALM benchmark, and code is available at the author’s GitHub repository namespace. Reproduction of the reported benchmark results requires fixing $[1, -1]$9 for DMU_add or $G_{\text{lin}} \rightarrow 1$0 for DMU_sub, training only $G_{\text{lin}} \rightarrow 1$1, setting the learning rate to $G_{\text{lin}} \rightarrow 1$2, and applying a gate temperature of $G_{\text{lin}} \rightarrow 1$3 for the smallest negative multiplication range (Curry, 9 Sep 2025).

In summary, the Domain Mixed Unit is a deliberately constrained neural arithmetic layer whose defining mechanism is a single learned gate between linear-space and log-space computation. Its contribution lies less in operator expressivity than in a narrow but effective inductive bias: keep the primitive fixed, choose the domain, and handle sign and magnitude explicitly. Within the reported NALM setting, that restriction is sufficient to produce state-of-the-art extrapolative arithmetic performance with only two trainable parameters (Curry, 9 Sep 2025).