SmoothSplit: Topology-Aware Activation

• SmoothSplit is a topology-aware activation function defined as x + c · tanh(αx), providing a smooth approximation to a discontinuous split operator.
• It uses learnable parameters to introduce a sign-dependent shift that asymptotically translates positive inputs by c and negative inputs by -c, reducing topological entanglement.
• Empirical studies show that while SmoothSplit improves class separability in low-dimensional latent spaces, it may exhibit instability in deeper or higher-dimensional networks.

SmoothSplit is a topology-aware activation function introduced in "Topology-Aware Activation Functions in Neural Networks" (Snopov et al., 17 Jul 2025). It is defined as a differentiable, smooth approximation to a discontinuous split operator and is intended to endow neural networks with a topology "cutting" capability that complements the "gluing"/compression behavior associated with activations such as $\mathrm{ReLU}$. In the formulation proposed in that work, SmoothSplit is aimed particularly at low-dimensional latent spaces, where separating data components along coordinate axes can reduce topological entanglement and improve class separability.

1. Formal definition and parameterization

SmoothSplit is defined by

$\mathrm{SmoothSplit}(x)=x+\tanh(\alpha x)\,c,$

where $x \in \mathbb{R}$ is the input, $c \in \mathbb{R}$ is a learnable scalar parameter, and $\alpha \in \mathbb{R}$ is a learnable scalar parameter that determines the sharpness of the split. Both $c$ and $\alpha$ are initialized randomly between $0$ and $1$.

The activation can be interpreted as a near-identity transform augmented by a sign-dependent translation. The additive term $c\,\tanh(\alpha x)$ applies an asymptotic shift of approximately $\mathrm{SmoothSplit}(x)=x+\tanh(\alpha x)\,c,$0 to positive inputs and $\mathrm{SmoothSplit}(x)=x+\tanh(\alpha x)\,c,$1 to negative inputs, with a smooth transition near the origin controlled by $\mathrm{SmoothSplit}(x)=x+\tanh(\alpha x)\,c,$2. As $\mathrm{SmoothSplit}(x)=x+\tanh(\alpha x)\,c,$3 increases, the transition steepens and SmoothSplit approaches the discontinuous operator

$\mathrm{SmoothSplit}(x)=x+\tanh(\alpha x)\,c,$4

The paper also situates SmoothSplit relative to ParametricSplit. It states that $\mathrm{SmoothSplit}(x)=x+\tanh(\alpha x)\,c,$5 can approximate $\mathrm{SmoothSplit}(x)=x+\tanh(\alpha x)\,c,$6 when $\mathrm{SmoothSplit}(x)=x+\tanh(\alpha x)\,c,$7 and $\mathrm{SmoothSplit}(x)=x+\tanh(\alpha x)\,c,$8, with $\mathrm{SmoothSplit}(x)=x+\tanh(\alpha x)\,c,$9 “uniquely defined by $x \in \mathbb{R}$0,” although the exact mapping is not provided.

2. Analytic properties

SmoothSplit has explicit first and second derivatives:

$x \in \mathbb{R}$1

and

$x \in \mathbb{R}$2

Because it is a composition of smooth functions, SmoothSplit is $x \in \mathbb{R}$3 on $x \in \mathbb{R}$4. It is also odd:

$x \in \mathbb{R}$5

For typical use with $x \in \mathbb{R}$6 and $x \in \mathbb{R}$7, the derivative satisfies $x \in \mathbb{R}$8 for all $x \in \mathbb{R}$9, with $c \in \mathbb{R}$0 and $c \in \mathbb{R}$1. Under this positive-parameter regime, the activation is strictly increasing and bijective. Since $c \in \mathbb{R}$2, one also has

$c \in \mathbb{R}$3

when $c \in \mathbb{R}$4, so a global Lipschitz constant is $c \in \mathbb{R}$5 in that regime.

These properties distinguish SmoothSplit from saturating odd activations in one direction and from clipping activations in another. The map remains globally smooth and invertible on $c \in \mathbb{R}$6, but its local slope near the origin increases with $c \in \mathbb{R}$7, which may have optimization implications.

3. Topology-cutting mechanism

The conceptual motivation for SmoothSplit is given through comparison with the discontinuous SignSplit operator. SignSplit produces an explicit cut because it is non-surjective:

$c \in \mathbb{R}$8

maps the positive half-line to $c \in \mathbb{R}$9 and the negative half-line to $\alpha \in \mathbb{R}$0, leaving the interval $\alpha \in \mathbb{R}$1 خارج the image. In the paper’s terminology, this divides the manifold along coordinate axes (Snopov et al., 17 Jul 2025).

SmoothSplit replaces $\alpha \in \mathbb{R}$2 with $\alpha \in \mathbb{R}$3 and therefore approximates the same behavior smoothly. Its asymptotics are

$\alpha \in \mathbb{R}$4

$\alpha \in \mathbb{R}$5

Near the origin, $\alpha \in \mathbb{R}$6, so

$\alpha \in \mathbb{R}$7

Globally, SmoothSplit is a homeomorphism of $\alpha \in \mathbb{R}$8, and more specifically a diffeomorphism. The intended cutting effect therefore concerns the dataset rather than the ambient space. Coordinate-wise shifts of opposite sign asymptotically separate data lying on opposite sides of coordinate hyperplanes, producing gaps in the transformed point cloud and increasing connected-component separations in the representation. This suggests that SmoothSplit is best understood as a representation-level topological manipulator rather than a literal topological surgery on the full input space.

The paper illustrates this mechanism on the Circles dataset: with $\alpha \in \mathbb{R}$9 and $c$0, SmoothSplit separates rings along a coordinate and visually increases separability (Snopov et al., 17 Jul 2025).

4. Relation to other activation functions

SmoothSplit is framed as complementary to standard activations rather than as a universal replacement.

Activation	Characterization in the paper	Contrast with SmoothSplit
ReLU	Non-injective; primarily “glues” points by collapsing negative inputs to zero	SmoothSplit is injective, odd, and asymptotically shifts opposite halves
$c$1	Smooth and homeomorphic on $c$2; tends to preserve input topology	SmoothSplit uses $c$3 inside an additive shift to induce sign-dependent separation
PReLU	Compressive variant of ReLU; mitigates dead neurons but still primarily glues topology	SmoothSplit avoids clipping and exposes a controllable split via $c$4 and $c$5
ParametricSplit	Piecewise-defined; can emulate ReLU, SignSplit, or approximate SmoothSplit	More flexible and empirically more consistently strong

Against ReLU, the central distinction is topological. ReLU compresses topology by clipping and by collapsing negative activations, whereas SmoothSplit does not clip and instead asymptotically translates positive and negative halves in opposite directions. In low-dimensional hidden layers, the paper argues that this splitting can improve class separability when compression alone is insufficient.

Against $c$6, the difference is not smoothness but mechanism. Plain $c$7 is symmetric and saturating and is described as less capable of explicit splitting. SmoothSplit preserves a smooth odd structure while adding a sign-dependent displacement.

Against PReLU, the claim is again topological rather than purely optimization-centric. PReLU still belongs to the compressive family, whereas SmoothSplit is designed to create separation without clipping.

Against ParametricSplit, SmoothSplit is the less general construction. ParametricSplit can emulate several behaviors and, in the reported experiments, is more consistently strong in low-dimensional settings.

5. Empirical behavior

The empirical study in (Snopov et al., 17 Jul 2025) evaluated binary classification on Circles in $c$8, CurvesOnTorus in $c$9, and Breast Cancer Wisconsin. The architectures were fully connected networks with activations applied to all layers except the last, which used ReLU. Depth ranged from $\alpha$0 to $\alpha$1 hidden layers. Layer widths were $\alpha$2 for Circles, $\alpha$3 for CurvesOnTorus, and $\alpha$4 for Breast Cancer. Training used $\alpha$5 epochs, Xavier normal initialization for weights, Xavier uniform for ReLU networks, learning rate $\alpha$6, binary cross-entropy loss, a $\alpha$7 train/test split, and $\alpha$8 runs per configuration.

Several representative outcomes were reported:

Dataset and setting	SmoothSplit result	Observation
Circles, 1 hidden layer, width 3	$\alpha$9 validation loss	Best average validation loss
CurvesOnTorus, width 5	$0$0	Competitive but generally behind ParametricSplit
Breast Cancer, 1 layer, width 80	$0$1	Performed well, rivaling PReLU
Breast Cancer, 2 layers, width 100	$0$2	High variability
Breast Cancer, 3 layers, width 80	$0$3	Instability
Breast Cancer, 3 layers, width 100	$0$4	Instability

On Circles, SmoothSplit was strongest in a narrow low-dimensional regime and remained competitive at nearby widths. On CurvesOnTorus, it generally trailed ParametricSplit, $0$5, and PReLU at larger widths, although it remained competitive in some cases. On Breast Cancer, performance was mixed: one-layer configurations could be strong, but deeper and wider configurations exhibited large validation losses and very high variance.

The paper’s overall conclusion is that SmoothSplit “delivers competitive results, often matching ReLU and tanh,” but is less consistently strong than ParametricSplit and can be unstable in some higher-dimensional settings (Snopov et al., 17 Jul 2025).

6. Implementation and optimization considerations

A straightforward PyTorch implementation follows directly from the definition: one introduces two learnable scalar parameters per activation, $0$6 and $0$7, and computes

$0$8

The paper indicates that both parameters are learnable and initialized uniformly in $0$9.

Within the reported pipeline, SmoothSplit is applied to all hidden layers, while the final layer uses ReLU to impose a “gluing” effect. This design choice is part of the experimental protocol and is not separately ablated in the paper.

From a computational standpoint, SmoothSplit adds one $1$0 per activation together with a few scalar multiplications and additions. Its cost is described as comparable to other smooth activations such as $1$1, ELU, and GELU, and typically negligible relative to matrix multiplications.

The principal practical issue is stability. Because $1$2, large $1$3 steepens the transition near zero and can amplify gradients locally. The paper connects this to the high variance observed on Breast Cancer in deeper and wider networks. It therefore suggests, as practical mitigations derivable from the form, constraining $1$4, clipping its value, or using smaller learning rates when many layers use SmoothSplit.

For reproducibility, the paper provides a code repository at https://github.com/Snopoff/Topology-Aware-Activations. The license is not stated.

7. Limitations, open questions, and terminological scope

SmoothSplit is explicitly a smooth approximation to a non-surjective splitter. Because it remains bijective, its cut is necessarily softer than that of SignSplit. The paper notes that in some datasets and architectures this softer behavior may be insufficient to achieve the desired topological separation.

Another limitation is pipeline dependence. The experiments always used ReLU in the final layer to enforce a gluing effect, and the paper states that ablations are needed to assess the impact of this choice. Reported results therefore reflect a specific composition of splitting and gluing operations rather than SmoothSplit in isolation.

The open questions identified in the paper are both theoretical and architectural. These include broader applicability to convolutional and transformer-based models, a deeper assessment of how explicit topological transformations affect generalization, ablations isolating the role of the final ReLU layer, formalization of the relationship between ParametricSplit’s parameter $1$5 and SmoothSplit’s $1$6, and more theoretical work on manifold-splitting effects in multi-dimensional layers, including changes in connected components or homology under realistic data distributions (Snopov et al., 17 Jul 2025).

The label “SmoothSplit” is also used, or informally applied, in unrelated arXiv contexts, including split factorizations in scattering amplitudes (Arkani-Hamed et al., 2024), linear-time split decoding for vehicle routing (Gibbons et al., 24 Jan 2026), progressive Gaussian splitting for head avatars (Liao et al., 25 May 2026), DAG-based AI model splitting over edge networks (Li et al., 23 Jun 2025), lossless a priori demand splitting in split-delivery routing (Jones et al., 4 Apr 2025), and Hamiltonian splitting in HMC (Shahbaba et al., 2011). This suggests that the neural-network activation should be disambiguated by domain when cited.