Fairness

1Bias is a fact, fairness is a judgment call

Bias is the observation that "this model's results lean one way." Measure it and a number comes out. Fairness is the question that follows: "is this much of a lean acceptable, and if not, which direction should it be corrected toward?" Measuring and deciding are connected, but they aren't the same job.

Blur that line and strange requests start showing up, like asking the tool that measures the lean to also "make it fair." A tool can tell you how far apart two groups are on some axis. Whether closing that gap is the right call is still something a person has to decide.

Put a thermometer in front of the fan and you'll learn exactly how many degrees each seat is. The thermometer won't tell you which seat deserves the breeze.

2Fairness isn't just one definition

The first answer is matching outcome rates: fair means every group passes at roughly the same rate. The second looks at people who got the same verdict: fair means the actual outcome for people the model rated similarly should look similar across groups. Meaning: among everyone marked as a pass, the share who genuinely did well should be the same no matter the group.

The third answer looks at the direction of the mistakes: fair means the rate of wrongly rejecting someone who deserved to pass is even across groups. The fourth looks at individuals directly: similar people should get similar treatment.

All four sound reasonable, and each has people who back it. The trouble is that on the same data, these answers can point to different decisions.

3Some of them can't all be satisfied at once

If the underlying rates already differ across groups, then once outcome rates are matched, it is usually hard to also match the actual rate within each verdict, pass and reject alike, across groups. Fix that one instead, and it becomes hard to match both the rate of wrongly rejecting and the rate of wrongly passing. Nailing one exactly tends to push another out of line, and it's a known result that satisfying all three at once is impossible except in the special case where the underlying rates are the same across groups.

So the real work isn't "make it fair." It's "choose what gets protected first." For example, where the cost of a wrongful rejection is high, some choose to minimize missed cases. Which side is right is disputed, though, and the limits the law sets differ by country and field.

It's like the fan: rotating and staying fixed can't both happen at once. Not because there's no way to do it, but because a next step only becomes possible once something has been decided as more important.

4Covering your eyes doesn't make it fair

The first idea that comes to mind is dropping the sensitive category entirely. But the remaining categories very often carry that same information anyway. Neighborhood, school, way of speaking, time of day someone logs in, all of these leak a little bit of the same signal. Drop the category and the model tends to land on a similar conclusion regardless.

Worse, dropping the category also removes any way to measure the lean. Checking whether one group is doing worse requires being able to tell that group apart in the first place. That's also where the view comes from that, for auditing, the information is better kept under tight access than deleted.

5Writing down what got chosen is part of the job

Fairness work is three steps: decide which axis to look across, measure how far apart things are on that axis, and choose which standard to prioritize, then write down that choice and why. Skip the last step and the first two are just numbers sitting in a drawer.

Writing it down makes two things possible: someone reaching a different conclusion can point to exactly where the disagreement is, and the choice can be reopened later once conditions change. Fairness isn't a setting you configure once and leave; it's a decision that keeps getting reopened.