Data Analysis Advanced: Joins, Reshaping and Time Series
Dates and Time Series
In this lesson you will
- Convert text columns to datetimes and extract date parts
- Summarise by period with groupby and resample
- Smooth noisy data with a rolling window
Introduction
Dates are everywhere in data, and they are full of traps: different formats, time zones, uneven gaps. pandas has strong tools for them. Outputs below come from real runs with pandas 3.1.
Convert to real dates
Dates read from files are often plain text. Convert them first:
Python
import pandas as pd
d = pd.DataFrame({
"date": pd.to_datetime(["2026-01-05", "2026-01-20", "2026-02-03",
"2026-02-25", "2026-03-10"]),
"sales": [10, 20, 15, 5, 30],
})Once a column is a datetime, you can compare, subtract and extract parts. For ambiguous formats such as 03/04/2026, state the format explicitly, using format= or dayfirst=True, because day-first and month-first dates are easy to mix up. Use errors="coerce" to turn unreadable values into missing values rather than crashing.
Extract parts
Python
print(d["date"].dt.day_name().tolist())['Monday', 'Tuesday', 'Tuesday', 'Wednesday', 'Tuesday']Other parts include .dt.year, .dt.month, .dt.dayofweek and .dt.quarter. These let you ask, for example, whether sales are higher on weekends.
Quick quiz
Why convert a date column from text to a real datetime type?
Group by month
Python
d["month"] = d["date"].dt.to_period("M")
print(d.groupby("month")["sales"].sum())month
2026-01 30
2026-02 20
2026-03 30
Freq: M, Name: sales, dtype: int64Periods such as 2026-01 sort in true calendar order, unlike text month names.
Resample: regular time steps
When the date is the index, resample groups by regular time periods and is the standard time-series tool:
Python
s = d.set_index("date")["sales"]
print(s.resample("MS").sum())date
2026-01-01 30
2026-02-01 20
2026-03-01 30
Freq: MS, Name: sales, dtype: int64"MS" means month start. Other codes include "D" for day, "W" for week and "QS" for quarter start. Resampling also inserts empty periods, which is useful to see gaps.
Rolling windows
A rolling average smooths short-term noise so the trend stands out:
Python
print(s.rolling(2).mean())date
2026-01-05 NaN
2026-01-20 15.0
2026-02-03 17.5
2026-02-25 10.0
2026-03-10 17.5
Name: sales, dtype: float64Each value is the mean of the current and previous observation. The first is missing, because there is no previous one. Real analyses use windows such as 7 days. Here the points are uneven in time, so a time-based window such as rolling("30D") can be more accurate.
Good habits with time data
- Sort by date before using rolling or difference calculations.
- Handle time zones deliberately. Store in UTC, and convert for display.
- Look at missing periods, since no data may mean zero or may mean unknown.
- Compare like with like: seasonal patterns, such as December sales, need year-on-year comparisons.
- Plot the series before modelling, and look for trends, cycles and outliers.
- Be careful with forecasting. Simple baselines, such as "same as last year", are surprisingly hard to beat.
Try it yourself
Create a year of daily numbers, such as random sales. Resample to weeks, compute a 4-week rolling mean and plot both lines. Which one shows the trend more clearly?
Summary
- Convert text to datetimes, then extract parts such as the day name, month or year.
- Group by period or use
resamplefor regular time steps, androllingto smooth noise. - Sort by date, handle time zones and missing periods carefully, and compare seasons fairly.
Lesson complete
+20 XP
Nice work. Keep going while it is fresh.
Sources and further reading