A demand forecast becomes useful when it helps decide how much stock to order. Order too much and inventory accumulates. Order too little and sales are lost. The cost of those decisions gives us a practical way to compare forecasting models.
We tested TimeGPT 2.1 against optimized SES, AutoETS and AutoTheta on 599 store and product series from the VN2 inventory planning challenge. We tuned each model's safety stock factor on the same validation period, then simulated replenishment over the same 17 week holdout.
TimeGPT produced the lowest total inventory cost, EUR 14,710.80. That represents savings of 15.8% to 18.3% compared with the statistical baselines. Most of the saving came from holding less inventory.
What we tested
The supplied VN2 data contains 157 weeks of sales through April 8, 2024. We kept the complete weekly calendar, including leading zeros, for all 599 series.
The first 125 weeks provide history. The next 15 weeks form the validation period for policy selection. The final 17 weeks form the holdout period for the cost comparison. At every ordering week, each model sees only earlier observations and predicts demand for the next three weeks.
TimeGPT 2.1 uses local inference without additional features or task specific training. Optimized SES fits an exponential smoothing level. AutoETS selects a nonseasonal error and trend specification. AutoTheta selects among standard, optimized and dynamic Theta models, also without seasonality. All models clip negative forecasts to zero.
Turn forecasts into replenishment orders
Each model uses the same policy. Its target inventory equals the sum of the next three demand forecasts plus a safety buffer of k times RMSE. We calculate that RMSE separately for each model and item from the latest 26 observed errors for forecasts made one week ahead.
The order fills the gap between target inventory and stock already on hand or on order. Orders are rounded up to whole units and cannot be negative. Deliveries arrive two weeks after ordering.
We search k from 0.0 to 1.5 in steps of 0.1, minimizing total cost over the validation period. Each model gets its own k, which remains fixed during holdout.
| Model | Selected k |
|---|---|
| Optimized SES | 0.3 |
| AutoETS | 0.3 |
| AutoTheta | 0.3 |
| TimeGPT 2.1 | 1.0 |
The selected k values differ because the models have different forecasts and historical errors. This compares each forecast with its tuned replenishment policy.
Every model starts each episode with the same inventory, three times the mean sales over the preceding 13 weeks, rounded up. The incoming delivery pipeline starts empty. Inventory carries forward within each episode and resets at the validation and holdout boundary.
We charge EUR 0.20 per unit of ending inventory per week and EUR 1.00 per unit of lost sales. There are no backorders or ordering costs. Orders stop two weeks before the episode ends so every ordered unit can arrive within the scored period.
Compare inventory costs on holdout
| Model | Holding cost, EUR | Shortage cost, EUR | Total cost, EUR | TimeGPT saving, EUR | TimeGPT saving |
|---|---|---|---|---|---|
| Optimized SES | 13,377.80 | 4,102.00 | 17,479.80 | 2,769.00 | 15.8% |
| AutoETS | 13,881.40 | 4,128.00 | 18,009.40 | 3,298.60 | 18.3% |
| AutoTheta | 13,709.60 | 4,009.00 | 17,718.60 | 3,007.80 | 17.0% |
| TimeGPT 2.1 | 10,299.80 | 4,411.00 | 14,710.80 |
For each baseline, percentage savings equal the baseline cost minus TimeGPT cost, divided by baseline cost. These are simulated holding and shortage savings. They exclude model inference and implementation costs.
TimeGPT's holding cost is lower than every baseline's. Its shortage cost is higher. Under the chosen cost assumptions, the reduction in inventory outweighs the additional lost sales.
Read accuracy alongside service
| Model | RMSE | WAPE | Fill rate | Average ending stock per item |
|---|---|---|---|---|
| Optimized SES | 7.27 | 72.34% | 87.02% | 6.57 |
| AutoETS | 7.52 | 72.59% | 86.94% | 6.82 |
| AutoTheta | 7.32 | 73.50% | 87.31% | 6.73 |
| TimeGPT 2.1 | 5.01 | 57.26% | 86.04% | 5.06 |
TimeGPT has the lowest RMSE and WAPE in this holdout. Accuracy covers all items and three forecast horizons at each of the 15 ordering origins. WAPE divides pooled absolute error by pooled sales. Inventory cost scores each week once along the 17 week inventory trajectory.
Fill rate measures the share of demand fulfilled from stock. TimeGPT's fill rate is about one percentage point below the baselines, despite its lower forecast error and total cost. But this is fine, as we only optimized for costs and not for fill rate.
What these savings establish
For this sales history, starting inventory and cost function, TimeGPT with its tuned safety buffer reduced total inventory cost compared with all three baselines. That gives a concrete business result to evaluate alongside forecast accuracy.
The comparison uses one validation episode and one holdout episode. Starting inventory is synthetic, and sales during stockouts can understate actual demand.
The analysis follows the VN2 inventory planning challenge, uses StatsForecast for the statistical baselines, and calls a pinned replenishment policy implementation.