Time series data is everywhere. From stock prices and weather patterns to your daily step count on a fitness tracker, if it has a timestamp, it’s a time series. But how do you analyze such data to uncover patterns, trends, or even predict future values? Enter time series analysis.

In this article, we’ll walk through essential time series analysis techniques using SciPy, a popular Python library for scientific computing. We’ll use a sample dataset that mimics real-world seasonal temperature trends and explore the following:

• Smoothing the time series data 
• Filtering out noise with a low-pass filter
• Detrending to isolate patterns
• Performing spectral analysis to Identify Patterns
• Autocorrelation to understand data dependencies
• Cross-correlation for relationships between series

Let’s get started.

Creating Sample Time Series Data

To make this tutorial practical, let’s create a sample time series that represents daily temperature data for one year. Our simulated data will:

• Show a seasonal pattern—this means a repeating cycle of warmer and cooler days, similar to summer and winter variations.
• Contain a very gradual upward trend, mimicking a (possible) case where a city gradually gets warmer over the year.
• Include random noise—representing unpredictable changes, like sudden temperature fluctuations.

Let’s use NumPy and Matplotlib to generate and visualize this dataset, respectively:

import numpy as np
import matplotlib.pyplot as plt

# Set a random seed for reproducibility
np.random.seed(0)

# Create daily data points for a year (365 days)
days = np.arange(365)

# Generate a seasonal pattern using a sine wave (mimicking annual temperature cycles)
seasonal = 10 * np.sin(2 * np.pi * days / 365)

# Add random noise to make it more realistic
noise = np.random.normal(0, 2, days.size)

# Simulate a gradual upward trend (e.g., 0.05 degrees increase per day)
trend = 0.05 * days

# Combine all parts to create our final temperature data
temperature_data = 20 + seasonal + trend + noise

# Plot the data
plt.figure(figsize=(10, 5))
plt.plot(days, temperature_data, label="Temperature")
plt.title("Simulated Daily Temperature Data Over One Year")
plt.xlabel("Day")
plt.ylabel("Temperature (°C)")
plt.legend()
plt.show()

You’ll see a similar plot:

Smoothing the Data with a Moving Average Filter

Raw data often includes noise, which can make it hard to spot trends. One way to make the time series clearer is by using a moving average filter to smooth out fluctuations by averaging values within a specific window (say weekly or monthly). Let’s use a window size of 7 days (so we can smooth out daily noise).

We can write a simple helper function using the convolution function (np.convolve) from NumPy:

def moving_average(data, window_size):
    return np.convolve(data, np.ones(window_size) / window_size, mode='valid')

# Apply the MAF
smoothed_data = moving_average(temperature_data, 7)

# Plot the original vs. smoothed data
plt.figure(figsize=(10, 5))
plt.plot(days, temperature_data, label="Original Data")
plt.plot(days[3:-3], smoothed_data, label="7-Day Moving Average", color="orange")
plt.title("Temperature Data with 7-Day Moving Average Smoothing")
plt.xlabel("Day")
plt.ylabel("Temperature (°C)")
plt.legend()
plt.show()

The original data and the data after moving average smoothing are shown:

Filtering Out the Noise with a Low-Pass Filter

While a moving average smooths data over time, a low-pass filter is more useful for isolating the main trend from high-frequency noise. You can use a Butterworth low-pass filter from SciPy’s signal module for this.

The low-pass filter allows low-frequency components to pass through while filtering out high-frequency noise.

The butter function creates the filter coefficients based on the specified cutoff frequency and the sampling frequency (fs=1), with a filter order of 5. The filtfilt function applies the filter in both forward and reverse directions to avoid phase distortion.

from scipy.signal import butter, filtfilt

def low_pass_filter(data, cutoff, fs, order=5):
    nyquist = 0.5 * fs
    normal_cutoff = cutoff / nyquist
    b, a = butter(order, normal_cutoff, btype='low', analog=False)
    return filtfilt(b, a, data)

# using the low-pass filter
filtered_data = low_pass_filter(temperature_data, cutoff=0.1, fs=1)

plt.figure(figsize=(10, 5))
plt.plot(days, temperature_data, label="Original Data")
plt.plot(days, filtered_data, label="Low-Pass Filtered", color="green")
plt.title("Temperature Data with Low-Pass Filter")
plt.xlabel("Day")
plt.ylabel("Temperature (°C)")
plt.legend()
plt.show()

We then plot the filtered series alongside the original data for comparison:

Detrending to Isolate Patterns

Sometimes, we’re interested in studying only the cyclical or seasonal patterns in the data without the long-term trend. Detrending removes the trend component, letting us focus on the repeating patterns. Let’s use SciPy’s detrend function for this:

from scipy.signal import detrend

# Remove the trend component
detrended_data = detrend(temperature_data)

# Plot the original vs. detrended data
plt.figure(figsize=(10, 5))
plt.plot(days, temperature_data, label="Original Data")
plt.plot(days, detrended_data, label="Detrended Data", color="purple")
plt.title("Detrended Temperature Data")
plt.xlabel("Day")
plt.ylabel("Temperature (°C)")
plt.legend()
plt.show()

If the original time series data doesn’t have a strong linear trend, the detrending process will not have much effect. Detrending typically removes linear trends, so if the data has a sinusoidal, constant, or very shallow trend, the original and detrended data may appear quite similar.

Performing Spectral Analysis to Identify Patterns

Spectral analysis breaks down a signal into its component frequencies. This helps us spot cycles within the data—such as weekly, monthly, or yearly patterns.

Let’s perform spectral analysis on the time series data. The periodogram function computes the power spectral density (PSD) of the temperature_data, which reveals the distribution of power across different frequencies. The fs=1 specifies that the data is sampled once per day.

from scipy.signal import periodogram

frequencies, power = periodogram(temperature_data, fs=1)

plt.figure(figsize=(10, 5))
plt.semilogy(frequencies, power)
plt.title("Spectral Analysis of Temperature Data")
plt.xlabel("Frequency (cycles per day)")
plt.ylabel("Power")
plt.show()

The resulting frequencies represent the cycles per day, and the power indicates the strength of each frequency component. The plot is then visualized using a logarithmic scale for power:

Using Autocorrelation to Understand Data Dependencies

Autocorrelation helps us understand if current values in a time series are related to past values. Positive autocorrelation indicates that values are similar at certain intervals, which is common in seasonal data.

from scipy.stats import pearsonr

lags = range(1, 31)
autocorrs = [pearsonr(temperature_data[:-lag], temperature_data[lag:])[0] for lag in lags]

plt.figure(figsize=(10, 5))
plt.stem(lags, autocorrs)
plt.title("Autocorrelation of Temperature Data")
plt.xlabel("Lag (days)")
plt.ylabel("Autocorrelation")
plt.show()

This calculates and plots the autocorrelation of a time series for various time lags (from 1 to 30 days). For each lag, we calculate the Pearson correlation between the original data and a version of itself shifted by the lag amount.

Cross-Correlation for Relationships Between Series

Cross-correlation measures how one time series relates to another at different time lags, which is useful if you want to see how one variable might lead or follow another.

Let’s say we have another time series, like humidity, and we want to see if changes in temperature affect it.

# Generate synthetic humidity data with a lagged relationship to temperature
humidity_data = 70 + 0.5 * temperature_data[:-5] + np.random.normal(0, 1, 360)

# Calculate cross-correlation
cross_corrs = [pearsonr(temperature_data[:360], np.roll(humidity_data, shift))[0] for shift in range(-10, 11)]

# Plot cross-correlation
plt.figure(figsize=(10, 5))
plt.stem(range(-10, 11), cross_corrs)
plt.title("Cross-Correlation between Temperature and Humidity")
plt.xlabel("Lag (days)")
plt.ylabel("Cross-Correlation")
plt.show()

You should see a similar output:

Wrapping Up

And there you have it! We’ve covered some key techniques for time series analysis using SciPy, from detrending to filtering and cross-correlation. You can use these tools to explore patterns in time series data.

So what’s next? You could dive deeper into time series forecasting and build a predictive model. But before you do so, make sure you’ve got a good grip on these basics.

Happy time series analysis!