Multi-Time Scale Prediction of US Stock Index: A Feature Selection Based Approach Using LSTM and Random Forest

Xiaomin Xia

2025

Abstract

The prediction of stock market indices presents significant challenges owing to their inherent complexity and nonlinearity. Nevertheless, the swift advancement of artificial intelligence, coupled with its extensive utilization within the financial sector, equips investors with robust tools for the analysis of market trends. This research employs the Yahoo Finance API to obtain historical data pertaining to the S&P 500 index. It subsequently implements data preprocessing techniques, feature selection methodologies, and machine learning models, specifically Random Forest (RF) and Long Short-Term Memory (LSTM), to forecast trends over short-term (1-week), mid-term (5-week), and long-term (30-week) horizons. Experimental results indicate that RF performs better for short-term predictions, while LSTM excels in mid- and long-term forecasting. The study also compares different methods for handling missing data, this paper uses removing missing values method in order to simplify the preprocessing workflow. By eliminating irrelevant variables through feature selection, the prediction accuracy is further improved. This study demonstrates an effective workflow combining automated feature selection and machine learning algorithms, aiding investors in making more informed decisions and providing a basis for future research on hybrid models.

Paper Citation

in Harvard Style

Xia X. (2025). Multi-Time Scale Prediction of US Stock Index: A Feature Selection Based Approach Using LSTM and Random Forest. In Proceedings of the 2nd International Conference on Data Science and Engineering - Volume 1: ICDSE; ISBN 978-989-758-765-8, SciTePress, pages 604-611. DOI: 10.5220/0013702700004670

in Bibtex Style

@conference{icdse25,
author={Xiaomin Xia},
title={Multi-Time Scale Prediction of US Stock Index: A Feature Selection Based Approach Using LSTM and Random Forest},
booktitle={Proceedings of the 2nd International Conference on Data Science and Engineering - Volume 1: ICDSE},
year={2025},
pages={604-611},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013702700004670},
isbn={978-989-758-765-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 2nd International Conference on Data Science and Engineering - Volume 1: ICDSE
TI - Multi-Time Scale Prediction of US Stock Index: A Feature Selection Based Approach Using LSTM and Random Forest
SN - 978-989-758-765-8
AU - Xia X.
PY - 2025
SP - 604
EP - 611
DO - 10.5220/0013702700004670
PB - SciTePress