Learning Goals
5 minBy the end of this lesson you can:
- Explain what a confidence score means.
- Read a model's confidences and say which it is sure about.
- Explain why high confidence does not always mean correct.
Warm-Up · How Sure Are You?
8 minLast lesson AIs learned by reward. Today we look at how sure an AI is when it guesses.
If someone asked "Will it rain in Penang this afternoon?", you might say "pretty sure" or "maybe". We use confidence too. How could you put that as a number?
Reveal the thinking
You could say "80% sure". AI does the same — it gives its best guess and a percentage for how confident it is.
New Concept · The Confidence Score
18 minA guess plus a number
When an AI makes a prediction, it also gives a confidence — how sure it is, shown as a percentage. High means very sure; low means unsure.
It's like naming a friend in a photo. A clear photo → "definitely Wei Jie". A blurry one → "maybe Wei Jie".
High vs low confidence
- One big number (92% vs 8%) → the AI is sure.
- Two close numbers (55% vs 45%) → the AI is unsure; treat it carefully.
Confident does not mean correct. An AI can be 99% sure and still wrong if it learned the wrong pattern. Always sense-check important answers.
Worked Example · Sure vs Unsure
18 minOur food classifier looks at two photos and reports its confidences.
Photo A — a clear plate of nasi lemak
nasi lemak 92% roti canai 8%
One big number — the AI is confident. We can trust it.
Photo B — a blurry, half-eaten plate
nasi lemak 55% roti canai 45%
Two close numbers — the AI is unsure. This is a good moment for a human to take a look.
The confidence number is a tool for us. It tells us when to trust the AI and when to double-check.
Try It Yourself
20 minUse your worksheet.
For each, say if the AI is sure or unsure: (a) dog 88%, cat 12%; (b) apple 51%, pear 49%; (c) bus 97%, car 3%.
Hint
One number much bigger = sure. Two close numbers = unsure.
A medical AI says "healthy 60%, needs a doctor 40%". Should a human check? Explain why in one sentence.
Hint
The numbers are close and the decision is important — that's a clear yes.
Mini-Challenge · Set a Confidence Rule
12 minYou're in charge of an AI that sorts recycling. Decide a confidence rule: below what percentage should a human check the item by hand?
Pick a number, then explain why you chose it — too low wastes people's time, too high lets mistakes through.
It works if you can defend your threshold with a sensible reason about the cost of mistakes.
Show one good answer
"Check anything below 80%. Recycling mistakes spoil a whole batch, so I want the AI to be quite sure before it sorts on its own. 80% leaves a safe margin without bothering a human for every item."
Recap
5 minAI gives a prediction and a confidence — a percentage for how sure it is. Close numbers mean unsure. But high confidence is not the same as correct, so we still check important answers.
Vocabulary Card
- confidence
- How sure an AI is about its prediction, shown as a percentage.
- prediction
- The AI's best guess for a new example.
- threshold
- A cut-off — e.g. "below 80%, ask a human to check".
Homework · When Must It Be Sure?
≤ 20 minThink of two AI decisions: one where you'd want the AI to be very confident before acting, and one where it's fine for it to be unsure. Write each, and explain why in a sentence.
Sample · Confidence Choices
Must be very confident: an AI deciding if a medicine is safe — a wrong guess could hurt someone.
Fine to be unsure: an AI suggesting a song — if I don't like it, I just skip.
Yours will be different — match the confidence needed to how costly a mistake would be.