Key points
- An ethics dilemma is a situation where there are several reasons worth honoring, but you can't follow them all at once. Whichever way you choose, you often lose something.
- People mostly split because they're using different yardsticks: the size of the outcome, a line that shouldn't be crossed, and a fair turn each pull a different way.
- This gets heavier for AI because the standard has to be settled in advance. Whether it's written as rules or built in through training data and goals, the direction of the judgment is fixed before the moment arrives, unlike a person weighing it as it happens.
- Once a standard is set, it repeats identically, millions of times over. A person's judgment wobbles case by case; a machine leans the same direction every time.
- Since it's hard for everyone to agree on what's right, many argue for also deciding who sets the standard, what gets disclosed, and how it gets changed.
The analogy
Snow fell all night, and there's exactly one plow. Someone has to decide where it starts, and three people each argue for a different answer.
One says clear the main road first, since that gets the most people moving the fastest. Another says go to the neighborhood that came last the previous time, since the same area shouldn't always end up at the back of the line. A third says clear the road in front of the hospital first, since the ambulance route was promised to stay open no matter what.
All three have a reason, and none of them is acting in bad faith. But there's only one plow, so an order has to be picked, and arguing about it on a snowy morning is already too late. An AI ethics dilemma looks like this scene. Good reasons collide with each other, and the answer has to be written down ahead of time.
In detail
Why people split
Each of the three arguments comes from a different yardstick. The first is the size of the outcome: pick whichever choice produces more good, added up across everyone. It's easy to compute and persuasive, but a weakness hides inside the sum, since harm concentrated on a few people can get buried in the total.
The second is a line that shouldn't be crossed regardless of outcome: not treating people as means to an end, not using someone's information without consent. Principles like these don't bend, but when two lines collide with each other, the argument stalls.
The third is a fair share: who benefits, who bears the cost, and whether the same group keeps ending up at the back. It rarely shows up in a single decision and only becomes visible after many decisions pile up. Most disagreements come down to which yardstick gets put first, and rarely to one side meaning harm.
Deciding in the moment versus writing it down ahead of time
A dilemma a person runs into is usually settled on the spot: they read the situation, hesitate, choose, sometimes regret it later. Judgment shifting a little each time isn't seen as a flaw.
AI doesn't get that option. What to prioritize has to be set in advance, and if it isn't, the system drifts along whatever pattern sat in the training data. Not deciding is itself a decision.
A standard written down ahead of time has a real upside: what got chosen is visible from the outside, can be questioned, and can be revised. A judgment made in the moment is hard to trace back later, but a written policy stays put. Putting a dilemma into writing feels awkward, but it's what makes the argument possible in the first place.
One choice, repeated millions of times
Line up a hundred people to judge the same situation and the answers scatter. Some lean one way, some the other, and that scatter often helps keep any single direction from fully taking over.
A machine works differently. Once a standard is set, it applies identically to every case. If that standard leans slightly in one direction, the lean repeats identically, millions of times over. One person's bias stays within the judgments that person makes; a system's bias spreads everywhere the system reaches.
So the ethical question for AI isn't only "is this choice right." It's also "is it right to repeat this choice at this scale." A call that would be fine once can turn into something else entirely when it runs a million times.
What can be decided instead of the answer
Getting everyone to agree on what's right is hard. What can still be decided: who sets the standard, whether the people affected by it get a voice, and whether the choice gets disclosed to the outside.
Two more things belong here: what process changes the standard once it's shown to be wrong, and where exceptions the standard doesn't handle well get routed. A dilemma can't be erased, but a way back from a bad call can still be built in.
More precisely
The three yardsticks map onto some of the long-standing views in ethics: weighing outcomes, weighing duties and rights, and weighing fair process and distribution. Others exist too, such as virtue ethics, which looks at what kind of person would make a given choice, and views centered on relationships and care. None has been settled as beating the others. Real-world judgment usually blends several views, and disagreement tends to be about which gets more weight.
Where to draw the line on what counts as a dilemma is also contested. Some hold that if one reason is clearly heavier, it isn't a dilemma at all; others separate apparent dilemmas, which resolve once you think them through, from real ones where something is lost whichever way you go.
The analogy breaks down in one place: a plow route has a countable number of options, and there's time to sit down and decide before the snow falls. What AI runs into branches out endlessly, so every case can't be written down in advance. A written standard always eventually meets an exception, and how that exception gets handled becomes another standard in its own right.
One more gap: a standard shaped by training isn't written in a document, it's woven into the data and the training process itself. Often even the people who built it can't fully say what standard went in, since it emerged from millions of examples rather than a rule anyone sat down and wrote. Because of that, handling a dilemma takes surfacing what went in to begin with, alongside picking an answer.
Try it yourself
- Moral Machine (self-driving ethics) ailearn.space Make your own calls across several scenarios, then see how far your choices drift from everyone else's
- Fuzzy (self-driving ethics) ailearn.space Work through several scenarios in turn and watch for the point where your own standard starts to waver, and how others chose
Common misconceptions
It's easy to think that with no right answer, any answer will do, but actually a clearly bad answer does exist, and the disagreement sits among answers that each have their reasons.
It's easy to think whatever most people choose is the right answer, but actually a majority vote is only a reference point. Plenty of preferences overlap widely, but how much weight each one gets shifts across regions and cultures.
It's easy to think better technology makes the dilemma go away, but actually it only shrinks the ambiguous cases; the job of choosing what to prioritize stays exactly where it was.
One-line summary
An AI ethics dilemma is a place where good reasons collide and no single answer emerges, so disclosing what got chosen and building in a way to reverse it matter just as much as the choice itself.
Spotted an error or have a better analogy? Suggest an edit · Last updated 2026-10-18