Evaluating time series forecasting models helps you understand how well your predictions match the actual data. It’s crucial to assess the accuracy of your model to ensure reliable forecasts. Commonly used metrics for evaluation include Mean Squared Error (MSE) and Mean Absolute Error (MAE). MSE measures the average squared difference between predicted and actual values, highlighting larger errors more significantly. MAE calculates the average absolute difference, providing a straightforward view of the forecast accuracy.

In this guide, we’ll show you how to evaluate time series forecasting models using Python.

Creating Synthetic Data

To illustrate model evaluation, we first need synthetic time series data. We’ll generate data that includes a trend and some noise, which will be useful for testing our forecasting model.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

# Set random seed for consistency
np.random.seed(0)

# Create a range of dates
date_range = pd.date_range(start='2021-01-01', end='2023-01-01', freq='M')

# Generate synthetic time series data
trend = np.linspace(0, 10, len(date_range))
seasonal_effect = 10 * np.sin(np.linspace(0, 2 * np.pi, len(date_range)))
noise = np.random.normal(loc=0, scale=1, size=len(date_range))
data = trend + seasonal_effect + noise

# Create a DataFrame
df = pd.DataFrame(data, index=date_range, columns=['Value'])

# Plot the synthetic data
plt.figure(figsize=(10, 6))
plt.plot(df.index, df['Value'], label='Synthetic Data')
plt.title('Synthetic Time Series Data')
plt.xlabel('Date')
plt.ylabel('Value')
plt.legend()
plt.show()

# Save the data for evaluation
df.to_csv('time_series_data.csv')

Evaluating the Forecasting Model

Now that we have our synthetic data, we will evaluate the performance of a time series forecasting model using this data. For this example, we’ll assume you already have predictions from your model. We will calculate MSE and MAE to gauge the accuracy of these predictions.

import pandas as pd
import numpy as np
from sklearn.metrics import mean_squared_error, mean_absolute_error
import matplotlib.pyplot as plt

# Load the data
data = pd.read_csv('time_series_data.csv', index_col='Date', parse_dates=True)
series = data['Value']

# Example predictions and actual values
# For demonstration, use the last 10 values of the series as both predictions and actual values
predictions = np.array([value for value in series[-10:]]) # Replace with model predictions
actual = np.array([value for value in series[-10:]])

# Calculate metrics
mse = mean_squared_error(actual, predictions)
mae = mean_absolute_error(actual, predictions)

# Print metrics
print(f'Mean Squared Error: {mse}')
print(f'Mean Absolute Error: {mae}')

# Plot the results
plt.figure(figsize=(10, 6))
plt.plot(series.index[-20:], actual, label='Actual')
plt.plot(series.index[-10:], predictions, label='Predicted', color='red')
plt.title('Actual vs. Predicted Values')
plt.xlabel('Date')
plt.ylabel('Value')
plt.legend()
plt.show()

Wrapping Up

Evaluating time series forecasting models is essential for ensuring your predictions are accurate and reliable. By using metrics like Mean Squared Error (MSE) and Mean Absolute Error (MAE), you can quantify how closely your model’s predictions match the actual data.

In this guide, you learned how to create synthetic data, compute these metrics, and visualize the results using Python. With these tools and techniques, you can effectively assess and improve your forecasting models, leading to more accurate and dependable predictions. As you continue to work with time series data, regularly evaluating your models will help you achieve better forecasting performance.