Vision & Audio

47

How AI recognizes objects in photos, tracks body poses and turns speech into text. CNNs, object detection, segmentation, pose estimation, TTS, STT: many of these you can try with just a webcam and a microphone.

Level
Action Recognition

Watching a run of frames to name what action is happening

Basics
Background Removal

Erasing a photo's background and keeping just the subject

Intro
Bounding Box

The smallest box around an object, and where that box sits

Intro
CNNConvolutional Neural Network

A neural network built to spot patterns in a photo, layer by layer

Basics
Colorization

Adding fitting color to a black-and-white picture

Generative Intro
Computer Vision

Technology that gets a computer to make sense of photos and video

Basics Intro
Confidence Score

The number a model attaches to how sure it is about its own answer

Intro
Convolution

Sliding a small tile across a photo or other data to score each spot

Basics
Denoising

Removing only the flecks mixed in, leaving the original untouched

Generative Basics
Depth Estimation

Working out near and far from a single flat photo

Basics
Face Detection

Drawing a box around every face in a photo

Intro
Face Recognition

Comparing faces to work out who someone is

Intro
Feature Map

A picture that lays scan results out in their original spots

Advanced
Filter / Kernel

A small value tile that carries what to look for

Basics
Gaussian Splatting

A 3D scene held as millions of scattered color blobs

Generative Advanced
Gesture Recognition

Reading a hand or body movement as one predefined meaning

Intro
Hand Tracking

Following hand joint locations from one moment to the next

Intro
Image Captioning

Looking at a picture and writing what's in it as a sentence

Generative Basics
Image Classification

Putting one label on an entire photo

Intro
Keypoint

A point that pins down one location in an image

Basics
Landmark

A numbered reference point at a fixed spot

Basics
Lip Sync

Fitting mouth shapes to the beat of a sound

Basics
Motion Capture

Turning body movement into a record of point locations

Intro
Music Generation

Turning a request or opening phrase into a song with AI

Generative Intro
NeRFNeural Radiance Field

A 3D scene that answers with light for any spot and direction

Generative Advanced
Object Detection

Finding every object in a photo and boxing each one

Intro
Object Tracking

Keeping the same number attached to the same thing over time

Basics
OCR

Reading the text in a picture and turning it into characters

Intro
Pattern Recognition

Spotting the regularity that keeps repeating

Basics Intro
Pitch Detection

Pinpointing how high or low a sound is

Intro
Pooling

Shrinking a picture by keeping one representative value per region

Basics
Pose Estimation

Marking joint locations with points to work out a body's pose

Intro
Real-Time Inference

Finishing a judgment on incoming input within a short, fixed window

Basics
Resolution

The number of dots a picture is made of

Generative Intro
Segmentation

Putting a label on every single pixel of a photo

Basics
Sound Classification

Sticking one label on a short clip of sound

Intro
Source Separation

Splitting sound that's been mixed together back into its parts

Basics
Speaker Recognition

Telling who spoke from the sound of their voice

Basics
Spectrogram

A picture drawn so you can see sound with your eyes

Basics
Speech Recognition

The technology that transcribes spoken words into text

Basics Intro
STTSpeech-to-Text

The conversion that turns sound into written words

Intro
TTSText-to-Speech

The technology that reads text out loud in a voice

Intro
Upscaling

Using AI to enlarge a picture by inventing the detail it never had

Generative Intro
Vision Transformer

Cutting a photo into pieces and treating it like text

Advanced
Vocoder

The part that turns a sound blueprint into an actual waveform

Advanced
Voice Assistant

An AI you speak commands to, and hear answers from

Tools Intro
Voice Cloning

Imitating a person's voice from a short recording

Basics