Selecting and Filtering Data

Introduction

Most analysis starts with narrowing down: just these columns, just the rows that match a rule. This lesson uses the same df from the last lesson (products, category, price and units).

Choose columns

print(df[["product", "price"]].head(2))

    product  price
0  Notebook    3.5
1   Pen Set    5.0

A single name in brackets gives a Series. A list of names, with double brackets, gives a smaller DataFrame.

Filter rows

Put a condition inside the brackets. pandas keeps only the rows where it is true.

print(df[df["price"] > 10])

    product category  price  units
2  Backpack     Bags   29.9      2
3    Bottle  Kitchen   12.0      6
5  Backpack     Bags   29.9      3

Read it as "the rows of df where price is more than 10".

Combine conditions

Use & for and, | for or, and put each condition in its own parentheses:

print(df[(df["category"] == "Stationery") & (df["units"] >= 8)])

    product    category  price  units
0  Notebook  Stationery    3.5     10
4  Notebook  Stationery    3.5      8

Do not use the words and or or here. They do not work on whole columns.

Sort

print(df.sort_values("units", ascending=False).head(3))

    product    category  price  units
0  Notebook  Stationery    3.5     10
4  Notebook  Stationery    3.5      8
3    Bottle    Kitchen   12.0      6

ascending=False puts the biggest first.

Try it yourself

Show only the product and units of items that cost less than 10, sorted by units from highest to lowest.

Summary

• df[["a", "b"]] selects columns, and df[condition] selects rows.
• Combine conditions with & and |, each inside parentheses.
• sort_values orders rows by a column.