Evaluation & Data
42
How we measure whether AI is any good, and what data we teach it with. Accuracy, precision and recall, benchmarks, labeling, bias, data augmentation: the words you need to read a results table.
Level
A/B Test
A test that shows two versions to real people and compares results
AccuracyThe share of all judgments a model got right
AI BiasSkew in the data and how it's built tilting an AI's judgment one way
AI Content DetectionWorking out whether something was made by an AI
BenchmarkA shared set of problems used to measure candidates the same way
Bounding BoxThe smallest box around an object, and where that box sits
CalibrationMaking a model's stated confidence match how often it's right
Class ImbalanceA dataset where one category vastly outnumbers the rest
Confidence ScoreThe number a model attaches to how sure it is about its own answer
Confusion MatrixA table pairing truth with judgment in right-and-wrong boxes
Cross-ValidationSplitting data into pieces, checking each in turn, then averaging
Data AugmentationTransforming data you already have to multiply how much of it you have
Data CleaningFixing collected data up into a usable state
Data LeakageInfo that shouldn't be in training slips in, inflating scores
Data QualityHow trustworthy the data used for training actually is
Data ScienceTurning scattered data into an answer worth acting on
DatasetA bundle of material gathered and organized to teach an AI
Dataset BiasWhen the data you collected already leans to one side
Elo RatingA skill number built up from head-to-head results
Evaluation MetricThe yardstick that decides what counts as doing well
ExplainabilityHow much an AI can show for why it decided what it did
F1 ScorePrecision and recall folded into a single score
FeatureOne ingredient an AI looks at when it makes a judgment
Ground TruthThe answer agreed on as the standard for training and grading
Human EvaluationEvaluation where a person looks and scores it directly
LabelThe answer paired with data for training, usually attached by a person
LabelingThe work of a person attaching an answer to each piece of data
LeaderboardA table that lines evaluation scores up in order
Learning CurveA line showing how the score changes as rounds or data grow
LLM-as-a-JudgeLetting a language model rate an answer instead of a person
Mean Squared ErrorA penalty that squares how far off you were, then averages
OutlierA single value sitting far apart from the rest
OverfittingMemorizing the practice answers and failing on anything new
PerplexityA score for how many directions a model is torn between
Precision & RecallHow many picks were right, and how many real ones got caught
Prompt AdherenceThe yardstick for whether a result matches what was requested
RobustnessStill holding up when conditions get shaken
ROC & AUCScores across every cutoff, folded into one curve and one number
Sampling BiasA skew in data caused by who got picked
Synthetic DataData made up because it was needed, not because it happened
UnderfittingNot getting even the practice data right
Validation SetData set aside for checking only, never used to train