Hybrid LSTM-Transformer Model for Stock Market Prediction: A Deep Learning Approach

Stock market prediction is a complex and dynamic task due to the volatile nature of financial markets, influenced by economic, social, and geopolitical factors. Traditional machine learning models, including Long Short-Term Memory (LSTM) networks, have shown potential but often fall short in capturing both short-term price fluctuations and long-term dependencies. This paper proposes a novel LSTM-Transformer hybrid model that integrates the sequential modeling capabilities of LSTM with the attention-based long-range pattern recognition of Transformers. To enhance predictive performance, we incorporate key technical indicators—Relative Strength Index (RSI), Moving Average Convergence Divergence (MACD), Bollinger Bands—as well as sentiment features derived from FinBERT, a finance-specific large language model.The model is trained on historical stock data spanning 2015 to 2024 and evaluated using an $80 \%-20 \%$ training-testing split. Performance is assessed using Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Percentage Error (MAPE), and Sharpe Ratio to capture both prediction accuracy and risk-adjusted returns. A rolling-window backtesting approach is used to simulate real-world trading behavior across varying market conditions. Our hybrid model outperforms standalone LSTM, GRU, and Transformer baselines, achieving an MSE of 0.0021, RMSE of 0.0467, and a directional accuracy of $76.4 \%$. These findings highlight the value of combining deep learning, financial indicators, and sentiment analysis for robust stock market forecasting. A conceptual extension discussing the role of generative models like GPT for unstructured financial data is also presented.