--- license: mit pipeline_tag: time-series-forecasting tags: - ml-lab - statsmodels - keras - time-series-forecasting - finance - arima - holt-winters metrics: - mape - rmse - mae - mase ml_lab: title: EU Stock Market Forecasting order: 17 summary: Forecasts the DAX, SMI, CAC and FTSE stock indices from any date in 1991-1998 with classical time-series models. framework: statsmodels (+ TensorFlow/Keras compared) architecture: 'per-index classical forecaster (DAX: Holt-Winters (additive trend); SMI: Holt-Winters (additive trend); CAC: VAR(10); FTSE: Holt-Winters (additive trend))' dataset: EuStockMarkets (R datasets) space: shalev396/eu-stock-forecasting runtime: cpu-basic ui_kind: timeseries colab: https://colab.research.google.com/github/shalev396/ml-lab/blob/main/eu-stock-forecasting/training/notebook.ipynb github: https://github.com/shalev396/ml-lab/tree/main/eu-stock-forecasting metric: name: mape value: 1.046944 split: test params: 4 params_human: '4' trained_at: '2026-09-25' model-index: - name: eu-stock-forecasting results: - task: type: time-series-forecasting dataset: name: EuStockMarkets (R datasets) type: eustockmarkets-r-datasets split: test metrics: - type: me value: 2.799135 name: me - type: rmse value: 72.945636 name: rmse - type: mae value: 57.798856 name: mae - type: mpe value: 0.045626 name: mpe - type: mape value: 1.046944 name: mape - type: mase value: 3.196867 name: mase - type: backtest_mae_h20 value: 174.267212 name: backtest_mae_h20 - type: backtest_mape_h20 value: 3.122334 name: backtest_mape_h20 --- # EU Stock Market Forecasting Forecasts the four European stock indices of R's classic `EuStockMarkets` dataset (**DAX, SMI, CAC, FTSE**, 1,860 business days, 1991-07-01 to 1998-08-14 on a synthetic calendar) from any cutoff date, with a classical time-series model that only sees the history up to that date. Five forecasters were compared on every index (ARIMA, Holt-Winters, VAR, a Keras MLP and a Keras LSTM) against the naive "tomorrow = today" baseline. Per index, the deployed model is the classical model with the **fewest fitted parameters** among those whose **validation** error is within 2 % of the best: **Holt-Winters** (4 parameters) on DAX, SMI and FTSE, and **VAR(10)** (164 parameters) on CAC, where nothing smaller comes close. On the DAX test window (last 100 business days), the deployed Holt-Winters has a one-step-ahead MAPE of **1.047 %** (RMSE 72.9 index points). The honest headline: on DAX all three classical models land within 0.02 MAPE points of naive persistence (1.048 %). Daily index closes are close to a random walk, and nothing here beats that by a meaningful margin. Numbers are from the run of 2026-09-25 (`metrics.json`). ## Model - **Input**: `cutoff` (a `YYYY-MM-DD` date), `horizon` (1-120 business days) and `index` (`DAX`, `SMI`, `CAC` or `FTSE`). - **What it ships**: `config.json` (the deployed method per index, plus every classical method's settings: ARIMA order + coefficients, VAR lag, Holt-Winters trend) and `eustocks.csv` (the ~82 KB dataset the forecasts run on). There are no pickled fits. Each request refits on the history up to the cutoff in well under a second: - **Holt-Winters** (deployed for DAX, SMI and FTSE), additive trend and no seasonality, with 4 parameters (smoothing level and trend, initial level and trend). It is refit per request, and `predict(..., method="holt_winters")` selects it. It is also the model behind the Space's bring-your-own-CSV tab (`predict_series`). - **VAR(10)** (deployed for CAC): a vector autoregression on the four index levels together, 10 lags picked by AIC (max 12), refit by OLS on every request. It has 164 coefficients (10 lag matrices of 4x4 + 4 intercepts). Select it with `method="var"`. - **ARIMA(p,1,q)**, with the order picked by AIC on the fit window per index (DAX (2,1,3), SMI (2,1,3), CAC (3,1,3), FTSE (3,1,2)). Serving keeps the trained coefficients and re-runs the Kalman filter over the history up to the cutoff, because MLE refits of these near-cancelling ARMA orders take 1-4 s and often do not converge. Select it with `method="arima"`. - **Output**: a JSON dict `{model, history, forecast, actual, naive, mae, naiveMae}`. `history` holds the last 250 closes up to the cutoff. `forecast`, `actual` and `naive` cover the `horizon` business days after the cutoff: the model's forecast, the real closes (empty past 1998-08-14), and the last close carried forward. `mae` and `naiveMae` are the mean absolute errors against `actual` (`null` without actuals). - **Dates are synthetic.** The R ts object has no calendar (start 1991.496, 260 observations a year), so row 1 is dated 1991-07-01 and each following row is the next weekday (Mon-Fri, no holidays). Forecast dates continue that calendar. The order and spacing of the data are real; the exact dates are approximate. ## Usage ```python from huggingface_hub import snapshot_download import sys path = snapshot_download("shalev396/eu-stock-forecasting") sys.path.insert(0, path) import model predictor = model.load(path, device="cpu") out = predictor.predict("1997-10-01", 30, "DAX") # deployed Holt-Winters print(out["model"], out["forecast"][:3], out["mae"], out["naiveMae"]) predictor.predict("1997-10-01", 30, "DAX", method="var") # or "arima" / "holt_winters" ``` - **Space API** (free): `POST https://shalev396-eu-stock-forecasting.hf.space/gradio_api/call/predict` with `{"data": ["1997-10-01", 30, "DAX"]}`. See the [Space](https://huggingface.co/spaces/shalev396/eu-stock-forecasting). - **Inference Endpoint**: `handler.py` makes this repo deployable (Deploy -> Inference Endpoints, CPU). Request body: `{"inputs": {"cutoff": "1997-10-01", "horizon": 30, "index": "DAX"}}`. ## Training Chronological split per index, never shuffled: **fit** = 1,660 days (1991-07-01 .. 1997-11-07), **validation** = 100 days (.. 1998-03-27), **test** = the last 100 days (1998-03-30 .. 1998-08-14). The ADF test finds a unit root in every index level (p >= 0.76) and none in the first differences (p < 0.001), which is why ARIMA uses d = 1. All parameters are estimated on the fit window only: the ARIMA(p,1,q) grid (p, q <= 3, 15 fits per index, ranked by AIC), Holt-Winters, the VAR lag (by AIC), and the neural nets. The nets are an MLP 64-32 on 20 lagged closes (3,457 parameters) and an LSTM(64) on the same 20-day window (16,961 parameters), trained with Adam + MSE on MinMax-scaled closes (scaler fit on the fit window) and early-stopped on validation. Evaluation is **one-step-ahead walk-forward** with frozen parameters: each forecast sees the actual closes up to the previous day. This is the setting of R's `forecast::accuracy()`, whose table (ME, RMSE, MAE, MPE, MAPE, MASE) is reproduced below. The deployed method per index follows a rule fixed before the test window was looked at (see Experiments): fewest fitted parameters within 2 % of the best validation MAPE. The test window is only reported. A multi-step backtest then replays the Space itself: from 9 origins, one every 10 days of the test window, `model.forecast` sees the history up to the origin and predicts 20 business days ahead. Hardware: CPU (a shared 20-thread desktop). The full notebook run took about 3.5 min of wall time, 168 s of it for fitting (ARIMA grids and Keras). Full code: [training/](https://github.com/shalev396/ml-lab/tree/main/eu-stock-forecasting/training) ยท [Colab](https://colab.research.google.com/github/shalev396/ml-lab/blob/main/eu-stock-forecasting/training/notebook.ipynb) ## Evaluation Deployed model (Holt-Winters, additive trend) on the DAX test window: one-step-ahead metrics, plus the 20-day backtest (`backtest_*_h20`). MAPE/MPE are in %, the other metrics are in index points. | metric (test) | value | |---|---| | me | 2.7991 | | rmse | 72.9456 | | mae | 57.7989 | | mpe | 0.0456 | | mape | 1.0469 | | mase | 3.1969 | | backtest_mae_h20 | 174.2672 | | backtest_mape_h20 | 3.1223 | ## Experiments **Selection rule** (decided before, and independent of, the test window; test metrics are never used): rank the classical models (ARIMA, Holt-Winters, VAR) by one-step **validation** MAPE; among all models within **2 % (relative)** of the best, deploy the one with the **fewest fitted parameters** (ties go to the lower validation MAPE). The classical models differ by hundredths of a percent, so a 164-parameter VAR(10) should only be deployed when no small model is close. Fitted parameters: Holt-Winters 4, ARIMA 6-7 (AR + MA coefficients + noise variance), VAR(10) 164. Result: VAR(10) has the lowest validation MAPE on every index, but Holt-Winters is within 2 % of it on DAX (0.967 vs 0.952, +1.6 %), SMI (+1.5 %) and FTSE (+0.4 %), so Holt-Winters is deployed there. On CAC, Holt-Winters is 6.3 % behind (0.898 vs 0.845) and ARIMA 8.4 %, so VAR(10) stays. Naive persistence has no fitted parameters and is within 2 % on DAX too, but it is the reference, not a candidate. **DAX, every variant.** Validation MAPE (through the rule above) picks the deployed model; test columns are the R `accuracy()` table on the last 100 days (one-step-ahead). MASE is scaled by the in-sample naive MAE of the calm early 1990s, which is why it is > 1 for every model, naive included. | variant | val MAPE % | test ME | test RMSE | test MAE | test MPE % | test MAPE % | test MASE | |---|---|---|---|---|---|---|---| | ARIMA(2,1,3) | 0.995 | 4.01 | 72.47 | 58.22 | 0.067 | 1.054 | 3.22 | | **Holt-Winters** (deployed) | 0.967 | 2.80 | 72.95 | 57.80 | 0.046 | 1.047 | 3.20 | | VAR(10) | 0.952 | 0.95 | 74.05 | 58.60 | 0.009 | 1.061 | 3.24 | | MLP | 2.757 | 124.96 | 168.21 | 148.76 | 2.220 | 2.657 | 8.23 | | LSTM | 1.919 | 68.98 | 171.69 | 147.43 | 1.169 | 2.622 | 8.15 | | Naive persistence | 0.970 | 4.07 | 73.01 | 57.88 | 0.068 | 1.048 | 3.20 | **Every index**, `validation MAPE / test MAPE` (%), deployed in bold. VAR has the lowest validation MAPE on all four indices, yet the lowest test MAPE on none: the differences between the classical models (and naive) are a few hundredths of a percent, well inside day-to-day noise. That Holt-Winters is also the best classical model on the DAX and FTSE test windows is not why it was chosen; the rule never reads the test columns. | index | ARIMA | Holt-Winters | VAR | MLP | LSTM | Naive persistence | deployed | |---|---|---|---|---|---|---|---| | DAX | 0.995 / 1.054 | **0.967 / 1.047** | 0.952 / 1.061 | 2.757 / 2.657 | 1.919 / 2.622 | 0.970 / 1.048 | Holt-Winters | | SMI | 0.844 / 0.934 | **0.771 / 0.943** | 0.760 / 0.951 | 3.280 / 2.954 | 1.533 / 2.246 | 0.778 / 0.943 | Holt-Winters | | CAC | 0.916 / 0.989 | 0.898 / 0.969 | **0.845 / 1.012** | 1.737 / 1.596 | 1.212 / 1.282 | 0.903 / 0.968 | VAR(10) | | FTSE | 0.808 / 0.795 | **0.779 / 0.788** | 0.776 / 0.812 | 2.906 / 2.265 | 1.413 / 2.475 | 0.782 / 0.788 | Holt-Winters | **Multi-step backtest** (what the Space serves): MAE in index points of 20-business-day forecasts from 9 origins in the test window, deployed in bold. Over this short window, the deployed Holt-Winters beats naive on DAX (174.3 vs 195.7) and SMI (205.8 vs 208.4) and loses on FTSE (160.4 vs 148.1); the deployed VAR(10) loses to naive on CAC (132.7 vs 129.8). Holt-Winters is the best on DAX and SMI, ARIMA on CAC, and naive persistence on FTSE. | index | Holt-Winters MAE | ARIMA MAE | VAR MAE | Naive persistence MAE | origins | |---|---|---|---|---|---| | DAX | **174.3** | 195.0 | 184.5 | 195.7 | 9 x 20 days | | SMI | **205.8** | 216.0 | 239.8 | 208.4 | 9 x 20 days | | CAC | 135.8 | 124.7 | **132.7** | 129.8 | 9 x 20 days | | FTSE | **160.4** | 149.5 | 166.4 | 148.1 | 9 x 20 days | ![Series and split](assets/series.png) ![All variants](assets/model_comparison.png) ![DAX one-step predictions on the test window](assets/holdout_overlay.png) ![DAX multi-step backtest](assets/forecast_backtest.png) ![Neural-net training curves (DAX)](assets/training_curves.png) The neural nets are much worse here (test MAPE 1.3-3.0 %) than in the original course run. Their scaler is fit on the fit window only, and the 1998 validation / test levels lie far above anything in it (DAX tops out at about 4,460 in the fit window; the test window reaches 6,186). The nets cannot extrapolate, and on DAX early stopping keeps the weights from epoch 6 (MLP) and epoch 2 (LSTM). The classical models work on levels and differences and do not have this problem. ## Limitations - **Near random walk.** One-step test errors of every classical model are within 0.05 MAPE points of "tomorrow = today" on every index (0.02 on DAX). This model does not predict the market; it is a teaching example of the classical workflow. - **Selection is noisy.** With 100 validation days, the ranking of ARIMA / Holt-Winters / VAR can flip between windows. VAR won validation on every index but was the weakest classical model on the test window, which is why the rule prefers a 4-parameter model whenever its validation MAPE is within 2 %. - **Synthetic dates.** Weekdays from 1991-07-01, no holidays. Treat the calendar dates as approximate. - **Old, closed data.** 1991-1998 index closes only. Cutoffs after 1998-08-14 just forecast from the last row; there is no live data. - **No intervals.** Point forecasts only. Uncertainty grows fast with the horizon (the 20-day backtest MAPE is about 3 % vs about 1 % one step ahead). - ARIMA(2,1,3) on DAX did not converge in training (its AR and MA roots nearly cancel). It is kept because it had the lowest AIC, and its forecasts are almost identical to naive.