Part 1 argued for learning the statistics before the models. Part 2 covers what actually consumes the time in real work: getting the data into a state where the statistics mean anything.
Estimates vary, but most practitioners put cleaning somewhere between half and four-fifths of a project. It is also the part where the decisions that determine your conclusions get made, usually without being recorded.
The checklist
1. Count things before you look at them
Rows, columns, unique values per column, missing values per column. Do this first, every time. It catches the loading errors — a file parsed with the wrong delimiter, a header row read as data, a truncated download — that would otherwise produce a confident analysis of half a dataset.
2. Check the types
Numbers stored as text, dates stored as text, identifiers stored as numbers. The third is the dangerous one: an ID read as a number can silently lose leading zeros or be averaged by accident.
3. Find the duplicates
Exact duplicates are easy. Near-duplicates are the problem — the same entity recorded twice with a trailing space, a different capitalisation, or a slightly different spelling. Check unique counts against what you expect before assuming there are none.
4. Look at the extremes
Sort each numeric column and look at the top and bottom twenty values. This finds sentinel values pretending to be data — ages of 999, prices of -1, dates in 1900 — which are far more common than genuine outliers and much more damaging, because they average in silently.
5. Decide about missing values, deliberately
The question that matters is why the value is missing. Missing at random is a different problem from missing because something happened.
If income is missing more often for high earners, dropping those rows biases your result. If a sensor reading is missing because the sensor failed during an event, the missingness is the signal.
Options: drop the rows, drop the column, fill with a central value, fill with a model, or add a flag column marking that it was missing. All are defensible; the mistake is choosing silently.
6. Standardise categories
“UK”, “U.K.”, “United Kingdom”, “uk ” are four categories to a computer and one to you. Trim whitespace, normalise case, then look at the unique values by eye. This step is tedious and there is no substitute for looking.
7. Check dates properly
Ambiguous day-month ordering is the classic failure, and it is silent for the first twelve days of every month. Verify against a known record, check the range is plausible, and look for dates in the future.
Make it reproducible
Never clean by hand in a spreadsheet. Write the cleaning as code, in order, in one script that runs from the raw file to the clean one.
Three reasons. You will receive an updated file. You will need to explain a decision. And you will find a mistake in step three after doing steps four to nine — which is a ten-second re-run with a script and an afternoon by hand.
Keep the raw file untouched. Always.
Document the decisions
Keep a short log alongside the script: what you dropped and why, what you filled and how, what you standardised. Four or five lines.
This is the part that separates an analysis someone can trust from one they cannot, because every cleaning decision is a judgement that could have gone the other way. Making them visible is how the work becomes checkable.
The sanity check at the end
Re-run your counts. Compare row totals before and after. If you lost thirty percent of your data cleaning it, that is a finding worth understanding rather than a step to move past.
Next in this track
Part 3 covers your first model and how to evaluate it honestly.