Nixtla Blog

Categorical Exogenous Variables in NeuralForecast: Embeddings, Configuration and a Hands On Example

Learn how NeuralForecast turns categorical exogenous variables into learned embeddings, why that beats one hot or ordinal encoding, and how to configure it for your own models.

Aug 21, 2026 • 9 min read

Marco Peixeiro

Applied AI Scientist, Core maintainer of NeuralForecast

Table of Contents

Introduction

Most real forecasting problems come with more than a target column and a timestamp. A retailer's sales series carries a store type, a product category, a region. A demand series for a marketplace carries a seller tier. These are categorical exogenous variables, and how you hand them to a model changes what the model can actually learn from them.

NeuralForecast now handles categorical features through learned embeddings rather than manually encoding them yourself before the data reaches the model. Instead of turning a category into a single number or a wall of binary columns, it learns a dense vector representation for each category, jointly with the rest of the network, as part of training.

This post covers how that works end to end: declaring categorical exogenous variables, how NeuralForecast encodes them internally, why an embedding is the right tool for the job instead of simple encoding, which model families and capabilities support them today, and a hands on example you can run yourself.

Setup

Install the required libraries:

Bash

pip install datasetsforecast neuralforecast utilsforecast

Python

from datasetsforecast.m5 import M5

Y_df, X_df, S_df = M5.load("./data")

M5.load returns three dataframes: Y_df (unique_id, ds, y, one row per item per store per day), X_df (the same keys plus calendar and price exogenous columns), and S_df (one row per unique_id with its static hierarchy: item_id, dept_id, cat_id, store_id, state_id). unique_id is item_id and store_id concatenated, and y is daily units sold.

Declaring Categorical Exogenous Variables

NeuralForecast already lets you mark a column as future known (futr_exog_list) or historical only (hist_exog_list). Categorical features build on top of that with two more arguments:

cat_exog_list: which of those columns should be treated as categorical rather than continuous.
categorical_cardinalities: a dictionary giving the number of unique values for each categorical column.
cat_emb_dim: how wide the learned embedding for each categorical feature should be.

Python

from neuralforecast import NeuralForecast
from neuralforecast.models import NHITS

model = NHITS(
    h=12,
    input_size=24,
    max_steps=500,
    futr_exog_list=["month"],
    cat_exog_list=["month"],
    categorical_cardinalities={"month": 12},
    cat_emb_dim="fastai",
)

nf = NeuralForecast(models=[model], freq="ME")
nf.fit(df=train)

futr = test[["unique_id", "ds", "month"]]
preds = nf.predict(futr_df=futr)

A column can carry the same values whether it is static, historical, or future known. What changes is which list you put it in, plus adding its name to cat_exog_list and giving its cardinality.

How NeuralForecast Encodes Categories Internally

Under the hood, NeuralForecast builds a panel wide vocabulary at fit time: a mapping from each category value to an integer index, built across every series in the dataset at once, not per series. That single shared vocabulary is what lets the model reuse what it learns about, say, "month equals December" across every store in the panel instead of relearning it from scratch for each one.

Unlike numerical exogenous features, categorical columns are not scaled or normalized. Each category index is looked up in an embedding table, one learned vector per category, and that vector is concatenated with the continuous features before the combined tensor is fed into the model. The embedding table itself is a set of trainable weights, so it is optimized by the same loss and the same optimizer as every other parameter in the network, at the same time.

Choosing an Embedding Dimension

cat_emb_dim controls how wide each category's vector is, and NeuralForecast gives you four ways to set it, all capped at fifty dimensions:

Strategy	Formula	When to use it
`"fastai"` (default)	`min(50, ceil(1.6 * cardinality^0.56))`	A balanced default that works well for most feature sets.
`"sqrt"`	`min(50, ceil(sqrt(cardinality)))`	Features with very high cardinality and limited training data.
`"half"`	`min(50, (cardinality + 1) // 2)`	A small number of categories that each carry a lot of signal.
An integer, e.g. `8`	The exact value you pass	Uniform sizing across features, or when you are tuning embedding width as a hyperparameter.

Start with "fastai". It is derived from a rule of thumb that scales sublinearly with cardinality, so a feature with two categories does not get the same width as a feature with two thousand, and revisit it only if you have a specific reason to, such as running low on data relative to your cardinality, or wanting exact control while tuning.

Why Embeddings Instead of Just Encoding

It is fair to ask why NeuralForecast bothers with a learned embedding table at all, when pandas gives you ordinal encoding and one hot encoding for free. The short answer is that both of those alternatives throw away information that an embedding preserves.

Ordinal or label encoding assigns each category an arbitrary integer: Monday becomes 0, Tuesday becomes 1, and so on. That works only when the categories genuinely have an order the model should respect. For a feature like store type or product category, there is no natural order, so the model has to either ignore the numeric relationship between 0 and 1 or, worse, learn a spurious one, treating "category 5" as somehow closer to "category 4" than to "category 1" simply because of how you happened to number them.

One hot encoding avoids the false ordering problem by giving each category its own binary column, but it comes with two costs. First, dimensionality grows linearly with cardinality: a feature with two thousand unique store IDs becomes two thousand extra columns, most of them zero for any given row. Second, every category starts out, and stays, equally distant from every other category. The model has no way to represent that two store types tend to behave similarly unless it independently rediscovers that fact through separate weights for each column.

A learned embedding solves both problems at once. Cardinality maps to a modest, chosen width instead of one column per category, so it scales gracefully even into the thousands. And because the embedding vectors are trained jointly with the forecasting objective, categories that behave similarly with respect to the target end up with similar vectors, purely as a side effect of minimizing the same loss everything else in the network is minimizing. The model effectively discovers its own notion of distance between categories, informed by the actual forecasting task, rather than having one imposed on it or denied to it upfront. That is also why the embedding is concatenated with the continuous features rather than kept separate: the network can learn interactions between a category and a continuous driver, such as a particular store type reacting more strongly to a promotion than others do.

What Categorical Support Covers Today

Categorical exogenous variables, configured exactly as shown above, are supported across NeuralForecast's model families: univariate models, multivariate models, and recurrent models. As of the current release, that support also extends to the explainability and simulation paths, so you can use categorical exogenous variables end to end, including when inspecting feature contributions or running simulation based workflows, not only during a plain fit and predict cycle.

Not every model in NeuralForecast supports exogenous features at all, categorical or numerical, so check the specific model's documentation before assuming cat_exog_list is available to it.

Hands On Example

The M5 dataset gives us two kinds of categorical exogenous variables in one place: event_type_1 and event_name_1 (future known, since the calendar of holidays and sporting events is fixed in advance) and the static hierarchy in S_df (item_id, dept_id, cat_id, store_id, state_id). We will use both, and compare cat_emb_dim="fastai" against cat_emb_dim="sqrt" on a 5 series subset.

Pick 5 series and merge in the calendar and static columns:

Python

totals = Y_df.groupby("unique_id")["y"].sum().sort_values(ascending=False)
meta = S_df.merge(totals.rename("total"), on="unique_id")

# top seller per category, then fill by volume while spreading across stores
chosen_ids = ["FOODS_3_090_CA_3", "FOODS_3_586_TX_2", "FOODS_3_586_TX_3",
              "HOBBIES_1_234_CA_3", "HOUSEHOLD_1_118_CA_3"]

df = Y_df.merge(X_df, on=["unique_id", "ds"], how="left")
df = df[df["unique_id"].isin(chosen_ids)]
static_df = S_df[S_df["unique_id"].isin(chosen_ids)]

Both categorical exogenous columns need a cardinality. Because NeuralForecast builds its vocabulary from the panel it is fit on, that cardinality has to come from this 5 series subset, not from the full 30,490 series dataset:

Python

FUTR_CAT = ["event_type_1", "event_name_1"]
STAT_CAT = ["item_id", "dept_id", "cat_id", "store_id", "state_id"]

cardinalities = {c: df[c].nunique() for c in FUTR_CAT}
cardinalities.update({c: static_df[c].nunique() for c in STAT_CAT})
# {'event_type_1': 5, 'event_name_1': 31, 'item_id': 4, 'dept_id': 3,
#  'cat_id': 3, 'store_id': 3, 'state_id': 2}

event_name_1's cardinality stays at 31 regardless of subset size, because the same five years of calendar events apply to every series. The static columns, on the other hand, are capped by how many series we picked, since with only 5 series item_id cannot have more than 5 distinct values. Keep that in mind: it means this demo cannot show off the high cardinality case (item_id has 3,049 values across the full dataset) the "sqrt" strategy is really meant for.

Now build the two embedding variants and fit them together, on the M5 competition's own horizon (h=28):

Python

from neuralforecast import NeuralForecast
from neuralforecast.models import NHITS

common = dict(h=28, input_size=84, max_steps=1000, random_seed=0, scaler_type="robust")

model_fastai = NHITS(
    **common,
    stat_exog_list=STAT_CAT,
    futr_exog_list=["snap_CA", "snap_TX", "snap_WI", "sell_price"] + FUTR_CAT,
    cat_exog_list=STAT_CAT + FUTR_CAT,
    categorical_cardinalities=cardinalities,
    cat_emb_dim="fastai",
    alias="embed_fastai",
)
model_sqrt = NHITS(
    **common,
    stat_exog_list=STAT_CAT,
    futr_exog_list=["snap_CA", "snap_TX", "snap_WI", "sell_price"] + FUTR_CAT,
    cat_exog_list=STAT_CAT + FUTR_CAT,
    categorical_cardinalities=cardinalities,
    cat_emb_dim="sqrt",
    alias="embed_sqrt",
)

nf = NeuralForecast(models=[model_fastai, model_sqrt], freq="D")
nf.fit(df=train_df, static_df=static_df)
preds = nf.predict(futr_df=futr_df)

We score with RMSSE exactly as the M5 Competitors' Guide defines it: RMSE of the 28 day forecast, scaled by the RMSE of a lag 1 naive forecast computed in sample, then rolled up into a WRMSSE weighted by each series' share of dollar sales over the last 28 training days. This is a WRMSSE over our 5 series only, not the official 42,840 series leaderboard metric, so treat it as a fair comparison between the two configurations, not a benchmark score.

On this subset, cat_emb_dim="sqrt" (WRMSSE 0.6051) came out ahead of cat_emb_dim="fastai" (0.6500). That lines up with the guidance from earlier: "sqrt" is suited to limited training data, and 5 series for 1,000 steps is a limited training data regime, even though the raw cardinalities here are too small to be the "high cardinality" case "sqrt" was designed for. Do not read this as "sqrt" being generally better than "fastai"; it is a reasonable outcome given how little data this particular demo trains on, and it is worth trying both on your own data rather than assuming the default is always best.

Here is actual versus forecast for both configurations, across all 5 series:

Conclusion

Categorical exogenous variables in NeuralForecast are now fully supported across the library. Declaring cat_exog_list and categorical_cardinalities alongside futr_exog_list or hist_exog_list tells the model to build a shared vocabulary across your whole panel and learn a dense representation for each category, jointly with everything else, instead of relying on an ordering that is not there or a one hot column that cannot see similarity between categories. If your series carry categorical drivers such as store type, product family, or region, this is the mechanism to reach for, and cat_emb_dim="fastai" is a reasonable place to start before you tune further.