Meet pandas and DataFrames

Introduction

Data analysis means asking questions of data: which product sells best, which month was busiest, what is the average price? pandas is the most popular Python library for this. It works on tables, much like a spreadsheet, but you control it with code so the work is repeatable.

This course assumes you know basic Python (variables, lists and dictionaries). The code here is shown for reading. To run it, install Python and pandas on your computer with pip install pandas, then use a notebook or a script. The outputs shown were produced by real runs.

The simple explanation

A DataFrame is a table. A Series is one column of it.

Each row is one record, and each column is one kind of information, just like a table in a database.

Your first DataFrame

import pandas as pd

df = pd.DataFrame({
    "product": ["Notebook", "Pen Set", "Backpack", "Bottle", "Notebook", "Backpack"],
    "category": ["Stationery", "Stationery", "Bags", "Kitchen", "Stationery", "Bags"],
    "price": [3.5, 5.0, 29.9, 12.0, 3.5, 29.9],
    "units": [10, 4, 2, 6, 8, 3],
})

We gave pandas a dictionary. Each key becomes a column name and each list holds that column's values. We will reuse this df through the course. In real work you will usually load a file instead, with pd.read_csv("sales.csv").

Take a first look

print(df.head(3))   # the first 3 rows
print(df.shape)     # (rows, columns)

    product    category  price  units
0  Notebook  Stationery    3.5     10
1   Pen Set  Stationery    5.0      4
2  Backpack        Bags   29.9      2
(6, 4)

The numbers on the left (0, 1, 2) are the index, a label for each row.

Quick statistics

print(df["price"].mean())
print(df.describe().loc[["mean", "max"]])

13.966666666666667
          price  units
mean  13.966667    5.5
max   29.900000   10.0

df["price"] picks one column. .mean() gives its average. describe() summarises all the number columns at once.

Try it yourself

Build a DataFrame of five of your own items with at least one text column and one number column. Print its head(), shape and the average of the number column.

Summary

• pandas is the main Python library for working with tables of data.
• A DataFrame is a table, and one column is a Series.
• head(), shape, mean() and describe() give a quick first look.