---
title: 'wSTL-NN: Weighted Signal Temporal Logic Neural Networks'
url: https://www.emergentmind.com/topics/weighted-signal-temporal-logic-neural-networks-wstl-nn
type: topic
---

# wSTL-NN: Weighted Signal Temporal Logic Neural Networks

Weighted Signal Temporal Logic Neural Networks (wSTL-NN) combine the expressive temporal, Boolean, and quantitative semantics of weighted Signal Temporal Logic (wSTL) with the trainability and representational power of neural networks. By encoding wSTL subformulas as differentiable neurons and learning parameters via gradient-based optimization, wSTL-NN provides an interpretable, formally grounded, and computationally efficient framework for time-series classification and other temporal logic specification learning tasks. The approach is characterized by explicit importance weighting of logic and temporal operators, smooth differentiable relaxations of min/max semantics, and full end-to-end trainability, delivering models whose final output is an explicit, human-readable wSTL formula.

## 1. Weighted Signal Temporal Logic: Syntax and Quantitative Semantics

Weighted Signal Temporal Logic (wSTL) extends classical STL by assigning nonnegative importance weights to each Boolean and temporal operator in the formula. Let $s:\mathbb{N}\to\mathbb{R}^d$ denote a discrete $d$-dimensional signal.

A wSTL formula $\tilde{\varphi}$ admits the following recursive grammar [2104.05435, 2204.07579, 2210.01910]:
- Atomic predicate: $a^\top s(k)\le c$
- Negation: $\lnot\tilde{\varphi}$
- Weighted conjunction/disjunction: ${}^{w_1}\!\tilde{\varphi}_1 \wedge {}^{w_2}\!\tilde{\varphi}_2$, ${}^{w_1}\!\tilde{\varphi}_1 \vee {}^{w_2}\!\tilde{\varphi}_2$
- Weighted temporal operators:
  - Always: $\mathbf{G}_{I}^{\mathbf{w}}\tilde{\varphi}$
  - Eventually: $\mathbf{F}_{I}^{\mathbf{w}}\tilde{\varphi}$

Here $w_1,w_2>0$ are clause weights, $\mathbf{w}=(w_{k_1},...,w_{k_2})^\top$ are interval weights on time window $I=[k_1,k_2]$, and $a\in\mathbb{R}^d, c\in\mathbb{R}$.

Quantitative semantics assign a real-valued robustness to each formula, with higher weights amplifying the quantitative satisfaction of their associated subformulae. Standard min/max are replaced by soft, differentiable aggregations respecting the weights:
- For conjunction: $\otimes^{\wedge}([w_i,r_i],\sigma) = \frac{\sum \bar{w}_i s_i r_i}{\sum \bar{w}_i s_i}$, where $s_i = \exp(-r_i/\sigma)/\sum_j \exp(-r_j/\sigma)$ and $\bar{w}_i = w_i / \sum_j w_j$
- For temporal operators: Aggregation is performed over the window, with smooth weighting; as $\sigma\to0$, soft operations converge to the true weighted min/max [2104.05435].

## 2. Neural Network Architectures for wSTL-NN

Each wSTL formula can be compiled into a neural network in which each node corresponds to a wSTL subformula and each edge encodes the logical/temporal structure [2210.01910, 2104.05435, 2209.09114, 2204.07579]. The general architectural principles are as follows:

- **Predicate Layer**: Each atomic predicate $a^\top s(\tau) - b$ becomes a neuron, with only threshold $b$ (and optionally $a$) trainable [2210.01910]. Predicates are evaluated at each relevant time index.
- **Temporal Layer**: Implements weighted “always”/“eventually” via smooth, differentiable aggregation functions. Temporal windows are parameterized by endpoints $(t_1, t_2)$ and realized as soft selection vectors, e.g.,
  $$
  f_t(t_1,t_2;\eta) = \frac{1}{\eta} \min(
    \operatorname{ReLU}(\mathbf{n}-\mathbf{1}(t_1-\eta)) - \operatorname{ReLU}(\mathbf{n}-\mathbf{1}t_1),
    \operatorname{ReLU}(-\mathbf{n}+\mathbf{1}(t_2+\eta)) - \operatorname{ReLU}(-\mathbf{n}+\mathbf{1}t_2)
  )
  $$
  where $\mathbf{n} = [0, 1, ..., l]^\top$ and $\eta>0$ provides smoothing [2210.01910].
- **Boolean (Gate) Layers**: Conjunction/disjunction are modeled as soft, sparse activations. Boolean gate matrices are learned empirically, often with straight-through Bernoulli quantization for discrete selection [2210.01910, 2204.07579].
- **Output Layer**: Aggregates the results via conjunction and disjunction layers (in fixed or learned normal forms) to produce the overall wSTL robustness.
- **Graph-based Extensions**: For graph-temporal logic (wGSTL-NN), neurons compute over spatially connected nodes with per-neighbor and per-interval importance weights, leveraging the graph structure (e.g., region, neighbor) in both the input and operator semantics [2109.08078].

## 3. Differentiability and Smooth Relaxations

All key operators in wSTL-NN employ differentiable approximations of non-differentiable logical functions to facilitate gradient-based optimization. Sparse-softmax functions smoothly and soundly approximate min/max within the chosen temporal or logical aggregation, ensuring correct sign preservation for classification [2210.01910].

For instance, the soft maximum over a weighted window is
$$
F(\mathbf{r}, \mathbf{w}; \beta, h)
= \frac{\sum_i r_i w_i q_i}{\sum_i w_i q_i}
$$
where $q_i = \exp(\beta r_i'')/(\sum_j \exp(\beta r_j''))$ and $r_i'' = h r_i'/r_{im}$, with $r_i' = r_i w_i$, $r_{im} = |\max_i r_i'|$ (or $1$ if this is zero) [2210.01910].

Boolean combination weights and structural gating variables are often handled using straight-through estimators or soft quantization to maintain end-to-end differentiability while enforcing network sparsity and interpretability [2204.07579, 2208.05440].

## 4. Training, Optimization, and Sparsification

All weights (predicate coefficients, threshold biases, time window endpoints, clause weights, temporal weights, and logic-gate parameters) are learned via back-propagation on a smooth loss function. Common choices are the exponential classification loss,
$$
\ell(s, y) = \exp(- y \, r_\chi(s))
$$
with $y \in \{\pm1\}$ and $r_\chi(s)$ the wSTL robustness [2210.01910], or mean-squared-error for regression-style training [2204.07579].

Sparsification methods are deployed post-training or during training:
- **Thresholding**: Small normalized weights are set to zero, either by absolute threshold or by retaining the top-$s$ largest weights [2104.05435].
- **Gate-variable Regularization**: Each weight is multiplied by a stochastic gate $g_i\sim\mathrm{Bernoulli}(p_i)$; the network learns $p_i\in[0,1]$ and applies $\ell_1$ penalty for overall sparsity. This approach enables pruning with minimal accuracy loss [2104.05435].

For structure learning, certain frameworks (e.g., Fronda & Abbas [2208.05440]) employ differentiable gating blocks at each possible logic/temporal branch, allowing the network to jointly infer formula structure and parameters. Quantized gating vectors determine the final logical structure after training.

## 5. Interpretation and Extraction of STL Formulas

wSTL-NN uniquely enables extraction of an explicit, readable STL or wSTL formula after training. The final model directly corresponds to a formula in DNF (or other normal form as imposed by the architecture) whose parameters and structure are dictated by the learned weights and gate selections:
- Predicate thresholds define atomic predicates.
- Time-interval endpoints and temporal weights define the precise windows for $\mathbf{G}/\mathbf{F}$.
- Clause and gate weights specify which conjunctions and disjunctions are present and their importance.

Interpretability is supported both by the ability to read the formula and by the monotonic influence of each weight on formula satisfaction. Compactness is achieved by sparsification and grow-and-prune cycles in the network [2204.07579, 2208.05440, 2210.01910].

## 6. Representative Applications and Empirical Evaluation

wSTL-NN has been validated in a variety of temporal logic classification tasks:

- **Time-Series Classification**: Models have achieved high classification accuracy (e.g., 99.46% in UCI occupancy detection) and test performance equivalent to or better than classical ML methods, while providing explicit temporal logic rules [2104.05435, 2210.01910].
- **Fault Diagnosis**: TLNN (a wSTL-NN instantiation) enables interpretable and efficient diagnosis of bearing faults, with quantitative robustness scores and formula readability revealing physical mechanisms [2204.07579].
- **Time-Incremental Prediction**: For tasks where signals are revealed incrementally—such as urban driving or surveillance—wSTL-NN achieves low incremental misclassification rates and fast runtime compared to non-weighted or brute-force baselines [2112.14300].
- **Neuro-symbolic TSC**: NSTSC leverages wSTL-NN within decision trees to solve multiclass classification on biosignals and UCR datasets, often within 1-2% of SOTA purely statistical methods, while maintaining full formula-based interpretability [2209.09114].
- **Graph-temporal Learning**: wGSTL-NN captures spatial dependencies and regional heterogeneity in tasks such as rainfall and COVID-19 event prediction, yielding human-understandable structure and state-of-the-art accuracy [2109.08078].

## 7. Key Advantages, Limitations, and Future Directions

Key advantages of wSTL-NN include:
- End-to-end differentiability: All formula parameters, including structure and weights, are learned via standard optimization methods.
- Interpretability: The final model is a human-readable STL or wSTL formula, facilitating post-hoc analysis, validation, and deployment in safety-critical domains.
- Soundness: Differentiable approximations retain the key semantics of STL operators, ensuring correct label sign for classification [2210.01910].
- Efficiency: Compact formulas and weight-sharing reduce the search space compared to template enumeration or combinatorial logic synthesis.

Notable limitations include:
- For some variants, the formula structure must be specified a priori; fully structure-learned models are more computationally demanding [2104.05435, 2208.05440].
- As the joint parameter space is highly non-convex, careful initialization and regularization are necessary to avoid degenerate solutions and excessive model complexity [2208.05440].
- For graph-based scenarios, scalability with respect to number of nodes and multi-hop dependencies remains a challenge [2109.08078].

Future research directions include:
- Automating structure search and formula architecture selection, potentially via reinforcement learning or hierarchical model selection.
- Extending to multi-modal, multi-graph, and higher-order spatial-temporal logics.
- Incorporating online, one-class, or positive-only learning paradigms.
- Integration of learned wSTL-NN formulas into deep reinforcement learning agents for interpretability and constraint satisfaction.

**References**

- [2210.01910] Learning Signal Temporal Logic through Neural Network for Interpretable Classification
- [2104.05435] Neural Network for Weighted Signal Temporal Logic
- [2204.07579] Interpretable Fault Diagnosis of Rolling Element Bearings with Temporal Logic Neural Network
- [2112.14300] Time-Incremental Learning from Data Using Temporal Logics
- [2208.05440] Differentiable Inference of Temporal Logic Formulas
- [2109.08078] Weighted Graph-Based Signal Temporal Logic Inference Using Neural Networks
- [2209.09114] Neuro-symbolic Models for Interpretable Time Series Classification using Temporal Logic Description

Source: https://www.emergentmind.com/topics/weighted-signal-temporal-logic-neural-networks-wstl-nn