Penalty Learning for Optimal Partitioning using Multilayer Perceptron (2408.00856v4)

Abstract: Changepoint detection is a technique used to identify significant shifts in sequences and is widely used in fields such as finance, genomics, and medicine. To identify the changepoints, dynamic programming (DP) algorithms, particularly Optimal Partitioning (OP) family, are widely used. To control the changepoints count, these algorithms use a fixed penalty to penalize the changepoints presence. To predict the optimal value of that penalty, existing methods used simple models such as linear or tree-based, which may limit predictive performance. To address this issue, this study proposes using a multilayer perceptron (MLP) with a ReLU activation function to predict the penalty. The proposed model generates continuous predictions -- as opposed to the stepwise ones in tree-based models -- and handles non-linearity better than linear models. Experiments on large benchmark genomic datasets demonstrate that the proposed model improves accuracy and F1 score compared to existing models.

• The paper introduces a deep learning method using MLPs to dynamically predict penalty parameters for precise changepoint detection.
• The study demonstrates that the MLP-based approach outperforms traditional linear and tree-based models, achieving higher accuracy on benchmark datasets.
• The research underlines the importance of careful feature selection and highlights computational challenges in optimizing model configurations.

Deep Learning Approach for Changepoint Detection: Penalty Parameter Optimization

The paper introduces a deep learning-based method for optimizing penalty parameters in changepoint detection algorithms, aiming to enhance the accuracy of identifying significant shifts in data sequences across various applications such as finance, genomics, and medicine. Traditional methods such as Optimal Partitioning (OPART), Functional Pruning Optimal Partitioning (FPOP), and Labeled Optimal Partitioning (LOPART) rely on a fixed penalty parameter $\lambda$, which influences the number of detected changepoints and their locations. Existing techniques for predicting the optimal $\lambda$ employ simple models like linear models and decision trees, which might fail to capture intricate data patterns. This paper proposes utilizing deep learning, specifically Multi-Layer Perceptrons (MLPs), to predict $\lambda$ dynamically, thereby improving changepoint detection accuracy.

Methodology

Problem Setting

The objective is to predict the penalty parameter $\lambda$ that optimizes the detection of changepoints in a given data sequence $\mathbf{d}$. Each sequence has predefined labels indicating the expected number of changepoints within particular regions. The goal is to minimize the discrepancy between the detected changepoints and the expected number of changepoints, minimizing false positives and false negatives.

Previous Methods

The paper evaluates several conventional methods for predicting $\lambda$, including:

• Bayesian Information Criterion (BIC): An unsupervised method predicting $\log\lambda_i = \log\log N_i$ for the $i^{th}$ sequence.
• Linear Models: Utilizing features such as sequence length ($N_i$), variance ($\sigma_i$), range ($r_i$), and sum of absolute differences ($s_i$) to predict $\log \lambda_i$ through a linear combination.
• Maximum Margin Interval Trees (MMIT): A tree-based method that minimizes the hinge loss within each region, differing from standard regression trees that minimize squared error within regions.

Proposed Method

The paper proposes using MLPs with carefully selected features to predict $\lambda$. The features include sequence length, variance, range, and the sum of absolute differences. The MLPs are trained to minimize a squared hinge loss function, which is more appropriate for interval regression problems than squared error loss. The model configurations, such as the number of hidden layers and neurons, are optimized using cross-validation.

Experiments

The methodology was evaluated on three large benchmark datasets: neuroblastoma tumors (detailed and systematic sequences) and a large epigenomic dataset. The main evaluation metric was the accuracy of changepoint detection, measured as the proportion of correctly predicted changepoints. A cross-validation setup ensured robust and reliable results.

Results

The experiments demonstrated that the proposed MLP-based method outperforms traditional methods in terms of accuracy. Specifically, the MLP models with four chosen features consistently achieved higher accuracy across all datasets compared to linear models and decision trees. Notably, while models with more features occasionally offered improvements, they often complicated the model without significantly enhancing performance, suggesting the selected four-feature set's efficacy.

Discussion and Conclusion

Feature Selection:

• The analysis highlighted the importance of selecting relevant features. Simple features like sequence length proved insufficient in isolation, while the addition of variance, range, and the sum of absolute differences enhanced prediction accuracy.

MLP Performance:

• While MLPs generally outperformed linear models and decision trees, they required careful configuration and were computationally intensive. The optimal MLP configurations typically included 2-3 hidden layers with fewer than 64 neurons per layer.

Comparison with MMIT:

• Decision trees like MMIT often did not surpass linear models, possibly due to the linear relationships between features and target intervals. This suggests that, for certain datasets, linear models are more suitable than tree-based approaches.

Limitations:

• The paper's approach might not generalize well to other types of sequence datasets where the chosen features are less relevant. The process of identifying the optimal MLP configuration and extensive feature set evaluations were computationally demanding.

Future Work

Future research could explore alternative neural network architectures, such as Recurrent Neural Networks (RNNs), Gated Recurrent Units (GRUs), or Long Short-Term Memory (LSTM) networks, which might handle raw sequence data more effectively. Additionally, advanced feature engineering techniques or automated feature selection processes could further improve the model's performance.

Reproducible Research

The paper emphasizes reproducibility by providing all code and materials necessary to replicate the research results, available at the provided GitHub repository.

By presenting a robust deep learning approach to optimize penalty parameters in changepoint detection, this paper significantly contributes to improving the accuracy of identifying critical shifts in various data sequences, leveraging the strengths of deep learning techniques.