An Artificial Neural Network-based Stock Trading System Using Technical Analysis and Big Data Framework

Abstract: In this paper, a neural network-based stock price prediction and trading system using technical analysis indicators is presented. The model developed first converts the financial time series data into a series of buy-sell-hold trigger signals using the most commonly preferred technical analysis indicators. Then, a Multilayer Perceptron (MLP) artificial neural network (ANN) model is trained in the learning stage on the daily stock prices between 1997 and 2007 for all of the Dow30 stocks. Apache Spark big data framework is used in the training stage. The trained model is then tested with data from 2007 to 2017. The results indicate that by choosing the most appropriate technical indicators, the neural network model can achieve comparable results against the Buy and Hold strategy in most of the cases. Furthermore, fine tuning the technical indicators and/or optimization strategy can enhance the overall trading performance.

• The paper introduces an ANN-based trading system that leverages technical indicators and a big data framework to automatically generate buy, sell, and hold signals for Dow 30 stocks.
• It employs a four-stage pipeline—data ingestion, feature computation with RSI, MACD, Williams %R, class imbalance adjustment, and a four-layer MLP model—evaluated against a Buy and Hold benchmark.
• Empirical results show mixed performance with an average annualized return of 10.3% and 67.33% profitable trades, highlighting both its potential and limitations such as lookahead bias and trend capturing issues.

Artificial Neural Network-Based Stock Trading System Integrating Technical Analysis and Big Data Processing

Problem Formulation and Motivation

This work tackles algorithmic trading via predictive modeling of stock price action using technical analysis (TA) indicators within an artificial neural network (ANN) framework. The central objective is to automate the identification of buy/sell/hold signals for liquid US equities—specifically, the Dow 30 constituents—by leveraging technical indicator features integrated into a multilayer perceptron (MLP) and implemented using a scalable big data framework (Apache Spark). The approach is motivated by persistent deficiencies in rule-based and heuristic trading systems (e.g., low adaptability, high human intervention cost, suboptimal out-of-sample performance) as well as the high computational requirements in financial markets characterized by extensive and noisy time series.

Technical Design and Architecture

The system operationalizes a four-stage pipeline:

1. Data Ingestion and Labeling: Daily OHLCV data are acquired for Dow 30 equities (period: 1997–2017). Labeling of each day's close price is automated: local price minima (“valleys”) map to "Buy", maxima (“peaks”) to "Sell", and all other values to "Hold." This approach emulates an optimal hindsight strategy for labeling (oracle labeling).
2. Feature Computation and Preprocessing: The feature set comprises three TA indicators: Relative Strength Index (RSI), Moving Average Convergence Divergence (MACD), and Williams %R. These are calculated for each data point and normalized along with close prices.
3. Imbalance Adjustment: To counteract severe class imbalance intrinsic to the problem domain (Hold signals outnumbering Buy/Sell by orders of magnitude), minority classes (Buy, Sell) are resampled (oversampled) to constitute a balanced training set. This resampling is critical for meaningful model training and avoids majority class dominance in finite-capacity classifiers.
4. Model Training and Execution: The core predictive engine is a four-layered MLP (architecture: [4,5,4,3], corresponding to inputs, two hidden layers, and three output classes), trained via Spark MLlib for 200 epochs on 1997–2006 data, and evaluated on held-out 2007–2017 data. Trading simulation logic applies a prescribed algorithmic rule: buy (using full capital) on “Buy", sell on “Sell", and maintain position on “Hold". The backtest incorporates transaction costs and stop-loss logic.

Empirical Results

The resulting system achieves the following:

• Classification Accuracy: For Walmart (WMT), the overall classification accuracy is 65.52%, with significant class-dependent variability (e.g., precision for Buy 0.20, for Sell 0.14), indicating limitations in detecting sharp regime shifts under balanced but noisy labeling.
• Strategy-Level Performance: When evaluated in simulated trading against a Buy and Hold (BaH) benchmark, results are mixed. The average annualized return across Dow 30 names is 10.3% for the model vs. 13.83% for BaH. However, in 9/29 cases, the ANN-based strategy outperforms BaH, in some instances substantially (e.g., JPM: 26.17% vs. 12.26%).
• Transaction Analysis: The average proportion of profitable trades is 67.33% across all assets, underscoring the model's capability to exploit subperiod-specific market inefficiencies, although this advantage is dampened by lack of parameter optimization and the adverse effect of flat/volatile periods.

Crucially, notable underperformance in strong bull market conditions (e.g., AAPL, HD, NKE) arises due to the inability of the neural model, as constructed, to hold through extended trends, in contrast to BaH which capitalizes fully on persistent upward moves.

Theoretical and Practical Implications

The findings highlight several important theoretical considerations:

• Feature Selection Constraints: Limiting the feature set to three canonical TA indicators may provide insufficient nonlinearity for MLPs to capture regime transitions, especially in the presence of market microstructure noise and structural breaks.
• Labeling and Real-World Tradeability: The use of peak/valley-based oracle labeling for Buy/Sell generation creates a nontrivial lookahead bias, as true peaks and valleys are only knowable ex post. This constrains the direct application of tested results in live settings and may partially explain overly optimistic in-sample success rates.
• Model Regularization and Overfitting: The adoption of synthetic oversampling (for class balancing) mitigates underrepresentation of minority classes, but may inadvertently introduce sample correlation and exacerbate overfitting, particularly in a shallow MLP with static hyperparameters.
• Big Data Integration: Demonstration of Spark-based pipeline underscores practicality and feasibility of deploying neural trading models at scale, capable of handling production-grade datasets typical of major equities universes.

Limitations and Future Work

Key limitations, by authors' own admission, include:

• Absence of per-stock indicator parameter optimization: default lookback windows and thresholds may not generalize across securities with varying volatility, trendiness, and liquidity characteristics.
• Homogeneous model hyperparameters: all assets are treated identically, without per-symbol calibration or meta-learning adaptation.
• Restriction to shallow MLPs: more expressive architectures (e.g., LSTM, CNN, hybrid attention models) may yield stronger results, especially when incorporating multimodal data (e.g., fundamental, sentiment).
• Absence of explicit risk control: no dynamic position sizing or portfolio-level risk constraints are implemented.

Future research directions should prioritize deep learning architectures, indicator and hyperparameter optimization (possibly via evolutionary methods or Bayesian optimization), and robust cross-validated out-of-sample experiments designed to minimize lookahead bias and overfitting. Incorporation of modern big data infrastructure remains essential.

Conclusion

The paper provides a comprehensive evaluation of an MLP-based, TA-driven trading system on the Dow 30 universe, implemented atop a big data framework. While competitive with BaH for a subset of assets, and boasting robust transaction-level profitability, the approach's inability to surpass BaH on average is largely attributable to feature and model simplicity, non-optimized parameters, and oracle bias in signal labeling. The methodology and large-scale evaluation contribute valuably to the corpus of neural trading system literature and highlight the multifaceted interplay of architecture choice, signal engineering, and deployment infrastructure in real-world quantitative finance. Future enhancements will likely necessitate advanced deep learning, adaptive parameterization, and rigorous methodology for realistic financial backtesting.