Inspecting a DataFrame

The first four commands

df.head()        # first 5 rows
df.shape         # (rows, columns)
df.info()        # column names, non-null counts, dtypes
df.describe()    # summary statistics for numeric columns

Run all four on any dataset before doing anything else. Two minutes here saves an hour of confusion later.

Reading info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 1000 entries, 0 to 999
Data columns (total 4 columns):
 #   Column      Non-Null Count  Dtype
---  ------      --------------  -----
 0   order_id    1000 non-null   int64
 1   order_date  1000 non-null   object
 2   revenue     982 non-null    float64
 3   region      1000 non-null   object

Two problems are visible here:

• order_date is object, meaning text, not a date. Date arithmetic will fail and sorting will be alphabetical.
• revenue has 982 non-null out of 1000 — 18 missing values that will quietly skew any mean.

dtypes

An integer column becoming float64 is the usual sign that a null crept in.

Looking closer

df.columns              # column names
df['region'].unique()   # distinct values
df['region'].nunique()  # how many distinct
df['region'].value_counts()   # frequency, most common first
df.isnull().sum()       # missing values per column
df.sample(5)            # random rows, not just the first ones

value_counts() on every categorical column is the fastest way to find inconsistent labels — TX, Tx and Texas all present in one field.