Sound Classification

The bins get fixed before anything else

It can feel backward, but the label list gets settled first, before anything else is built. Decide to build it with three bins — clap, whistle, cough — and this classifier stays a three-bin system for its whole life. Wanting it to recognize footsteps instead of claps later means adding a bin and retraining from scratch.

Gathering example sounds for each bin is the actual teaching. The clap bin gets filled with claps recorded from many people, many rooms, many distances. Teach it with sound recorded by one person in one spot, and it ends up learning that room's echo as part of what a clap sounds like.

Sound gets spread into a picture first

What a mic captures is one line wiggling up and down. That line alone makes it hard to tell a clap apart from a door closing. So a clip of sound gets cut into very short pieces, and for each piece it measures how strongly each pitch is present.

Lay these values out with time across and pitch up-down, and one grid picture comes out. A clap shows up as a short, broad smear; a whistle shows up as a thin line stretching sideways. The traits are distinct enough that they look different even to a human eye.

From here on, it's nearly the same as working with photos. What a sound classifier actually does is recognize the pattern in this grid picture — it converts the sound in front of the mic into a picture and classifies the picture.

Every bin gets a share of the score

The classifier's output isn't the single word "clap." A score comes out for every bin, and the scores are set to add up to 1 — clap 0.7, whistle 0.2, cough 0.1, say. What shows on screen is just the name of whichever bin scored highest.

How the score splits matters as much as the answer itself. One bin taking 0.9 means the sound was distinct; three bins splitting it roughly evenly is a signal that the sound was ambiguous. That's why real services are built to react only once the top score clears a set line.

An absent bin never appears

This is the most common problem. Ring a bell in front of a three-bin classifier and, since there's no bell bin, one of the three still gets picked — sometimes with a fairly high score too. That's because the classifier isn't asking "is this a clap," it's asking "which of these three bins is this closest to."

So in practice, an extra none-of-these bin gets added. Fill it with plenty of sounds that shouldn't trigger a response — a room's quiet hum, passing chatter, a chair scraping — and false triggers drop sharply.

It shakes when the environment changes

The same clap draws a different grid pattern once the mic or the room changes. A high-ceilinged room drags the sound out longer, and a laptop mic captures less of the low end. That's why a classifier tuned well in a quiet room ends up picking the wrong bin in a café.

So example sounds get deliberately recorded at varying distances and locations. Background noise gets mixed in, or copies stretched and compressed slightly are added to training too. Showing it sounds close to where it'll actually be used is the most reliable fix.