Deep Learning Models for Price Prediction in Travertine Stone Mines: A Comparison of LSTM, Transformer, and Hybrid Models

Abstract

This study aims to develop and evaluate four deep learning models for forecasting the monthly prices offered by stone suppliers in Iran. Given the simultaneous influence of temporal factors (such as exchange rate, inflation, fuel price, and order volume) and static attributes (including block quality, brand reputation, and cooperation history), each model was designed with a dual-input structure to separately process sequential and non-sequential features, which are then integrated at a later stage. The implemented architectures include LSTM, Bi-LSTM, Transformer, and a hybrid Transformer+LSTM+CLS model. The models were trained and evaluated using data collected from five different mines over several months, ensuring robustness and generalizability across diverse supply sources and time periods. Model performance was assessed using key evaluation metrics including Mean Absolute Error (MAE), Mean Squared Error (MSE), Root Mean Squared Error (RMSE), the coefficient of determination (R²), and validation loss. The results indicate that the Transformer model achieved the highest accuracy, with the lowest prediction errors and the best generalization capability. The hybrid model also performed comparably well, making it a robust alternative for more complex forecasting tasks. In contrast, the Bi-LSTM model underperformed and is less recommended for this application. Overall, the findings highlight that combining attention-based architectures with sequential analysis provides an effective solution for price forecasting in data-driven and competitive industrial contexts.