Mode Collapse

The inspection only ever happens one at a time

Approaches that learn through competition pair up a maker and a checker. The maker tries to produce something that passes as real; the checker tries to tell real from fake apart. The tighter this competition stays, the better the results get.

The catch is that the checker only ever receives one item at a time. It can answer "does this one look real," but it can't answer "are today's outputs different from each other." Variety was never on the scorecard to begin with.

So the maker has no reason to produce variety. Once it finds the one output that scores best, repeating that same output is the most profitable move available. This isn't laziness — it's exactly what the scorecard rewards.

Chasing and getting caught, stuck in one spot

The checker doesn't just sit still either. Once the same thing keeps showing up, it learns to flag that one thing as fake. The maker then shifts to a different single thing, the checker catches up again, and the maker shifts once more.

This game of tag is fine as long as it spreads out widely, but it's common for it to just circle around a handful of narrow spots instead. This isn't diversity — it's swapping wholesale from one single thing to another single thing. Line up the samples and it's all one shape yesterday, all a different shape today.

The numbers don't show it

When training goes wrong, the loss value usually spikes or swings wildly, which shows up right away. Mode Collapse doesn't work that way. Both sides' scores sit at plausible-looking values, so a glance at the chart alone shows nothing wrong.

So this problem has to be caught by eye. Lay out a few dozen generated samples on one screen, and it shows immediately. The poses, colors, and compositions all resemble each other, and changing the input a lot barely changes what comes out. Making a habit of periodically sampling a batch of results during training is the surest check there is.

Partial collapse is far more common than total collapse

The extreme case of shrinking down to one single thing is actually rare. What shows up more often is a state where the original data had ten types but the output only produces three or four. On the surface this looks plausible enough to slide past unnoticed, and it's only later that an entire category turns out to be missing altogether.

What tends to disappear is usually whatever was rare in the data to begin with. A good score comes just from making the common stuff well, so there's no real incentive to learn the rare stuff at all. Mode Collapse ends up erasing minority types from the output, which ties it right back into problems with bias.

What gets fixed

The most direct fix is rewriting the scorecard. Show the checker a whole batch at once, and have it weigh how similar the items inside are to each other. If everything looks alike, that costs points, which finally gives the maker a reason to produce variety.

There are ways to rebalance the competition too. Giving each side a different learning speed keeps one from getting strong too fast, or the scoring itself gets softened so nothing collapses into a single point. Running several makers, each covering a different region, is also used.

Switching the approach entirely is also an option. The approach that starts from a blur and gradually finds shape trains to imitate the whole dataset evenly, so it runs into this problem much less. That's one reason it's so widely used for work where variety matters.