Re-using fine-tuned models

​

Re-using Fine-tuned Models

Reusing previously fine-tuned TimeGPT models can help reduce computation time and costs while maintaining or improving forecast accuracy. This guide walks you through the steps to save, fine-tune, list, and delete your TimeGPT models effectively.

​

How to Re-use Fine-tuned Models

​

Step 1: Import Packages

First, we import the required packages and initialize the Nixtla client

import pandas as pd
from nixtla import NixtlaClient
from utilsforecast.losses import rmse
from utilsforecast.evaluation import evaluate

nixtla_client = NixtlaClient(
    # defaults to os.environ["NIXTLA_API_KEY"]
    api_key='my_api_key_provided_by_nixtla'
)

​

Step 2: Load Data

Load the forecasting dataset and prepare the train/validation split.

df = pd.read_parquet('https://datasets-nixtla.s3.amazonaws.com/m4-hourly.parquet')

h = 48

valid = df.groupby('unique_id', observed=True).tail(h)
train = df.drop(valid.index)

train.head()

​

Step 3: Zero-shot forecast

We can try forecasting without any finetuning to see how well TimeGPT does.

fcst_kwargs = {
    'df': train,
    'freq': 1,
    'model': 'timegpt-1-long-horizon'
}

fcst = nixtla_client.forecast(h=h, **fcst_kwargs)

zero_shot_eval = evaluate(fcst.merge(valid), metrics=[rmse], agg_fn='mean')
zero_shot_eval

​

Step 4: Fine-tune the model

We can now fine-tune TimeGPT a little and save our model for later use. We can define the ID that we want that model to have by providing it through output_model_id. This ID is also returned as the output of the finetune method.

first_model_id = 'my-first-finetuned-model'

nixtla_client.finetune(output_model_id=first_model_id, **fcst_kwargs)

'my-first-finetuned-model'

We can now forecast using this fine-tuned model by providing its ID through the finetuned_model_id argument.

first_finetune_fcst = nixtla_client.forecast(
    h=h,
    finetuned_model_id=first_model_id,
    **fcst_kwargs
)

first_finetune_eval = evaluate(
    first_finetune_fcst.merge(valid),
    metrics=[rmse],
    agg_fn='mean'
)

zero_shot_eval.merge(
    first_finetune_eval,
    on=['metric'],
    suffixes=('_zero_shot', '_first_finetune')
)

We can see the error was reduced.

​

Step 5: Further fine-tune the model

We can now take this model and fine-tune it a bit further by using the NixtlaClient.finetune method but providing our already fine-tuned model as finetuned_model_id, which will take that model and fine-tune it a bit more. We can also change the fine-tuning settings, like using finetune_depth=3, for example. As before, the new finetuned model ID is returned by the finetune method.

second_model_id = nixtla_client.finetune(
    finetuned_model_id=first_model_id,
    finetune_depth=3,
    **fcst_kwargs
)

second_model_id

'468b13fb-4b26-447a-bd87-87a64b50d913'

Since we didn’t provide output_model_id this time, it got assigned an UUID.

We can now use this model to forecast.

second_finetune_fcst = nixtla_client.forecast(
    h=h,
    finetuned_model_id=second_model_id,
    **fcst_kwargs
)

second_finetune_eval = evaluate(
    second_finetune_fcst.merge(valid),
    metrics=[rmse],
    agg_fn='mean'
)

first_finetune_eval.merge(
    second_finetune_eval,
    on=['metric'],
    suffixes=('_first_finetune', '_second_finetune')
)

We can see the error was reduced a bit more.

​

Step 6: List fine-tuned models

We can list our fine-tuned models with the NixtlaClient.finetuned_models method.

finetuned_models = nixtla_client.finetuned_models()
finetuned_models

[FinetunedModel(id='468b13fb-4b26-447a-bd87-87a64b50d913', created_at=datetime.datetime(2024, 12, 30, 17, 57, 31, 241455, tzinfo=TzInfo(UTC)), created_by='user', base_model_id='my-first-finetuned-model', steps=10, depth=3, loss='default', model='timegpt-1-long-horizon', freq='MS'),
 FinetunedModel(id='my-first-finetuned-model', created_at=datetime.datetime(2024, 12, 30, 17, 57, 16, 978907, tzinfo=TzInfo(UTC)), created_by='user', base_model_id='None', steps=10, depth=1, loss='default', model='timegpt-1-long-horizon', freq='MS')]

While that representation may be useful for programmatic use, in this exploratory setting it’s nicer to see them as a dataframe, which we can get by providing as_df=True.

nixtla_client.finetuned_models(as_df=True)

We can see that the base_model_id of our second model is our first model, along with other metadata.

​

Step 7: Delete fine-tuned models

In order to keep things organized, and since there’s a limit of 50 fine-tuned models, you can delete models that weren’t so promising to make room for more experiments. For example, we can delete our first finetuned model. Note that even though it was used as the base for our second model, they’re saved independently so removing it won’t affect our second model, except for the dangling metadata.

nixtla_client.delete_finetuned_model(first_model_id)

nixtla_client.finetuned_models(as_df=True)

WARNING: Deleting a fine-tuned model is irreversible. Make sure to back up any necessary information before removal.

​

Conclusion

Congratulations! You have successfully learned how to save, refine, and manage your fine-tuned TimeGPT models. This workflow helps optimize your forecasting pipelines by leveraging previously generated insights.