Accuracy, Precision, Recall

• Read a confusion matrix (TP, FP, FN, TN).
• Define accuracy, precision, recall, F1 — and when each matters.
• See why accuracy is misleading on imbalanced data.
• Use classification_report and confusion_matrix.

A rare disease affects 1 in 100 people. A "model" that always predicts "healthy" is 99% accurate — and catches zero sick patients. Useless, despite the headline number.

y_true = [0]*99 + [1]      # 99 healthy, 1 sick
y_pred = [0]*100           # model says everyone is healthy
accuracy = sum(t == p for t, p in zip(y_true, y_pred)) / len(y_true)
print(f"accuracy: {accuracy:.0%}")   # 99% — and it missed the one sick person

Four outcomes (for a yes/no model)

                    PREDICTED
                  positive   negative
ACTUAL positive    TP ✅       FN ❌ (missed it)
       negative    FP ❌       TN ✅
                   (false alarm)

TP = true positive   FN = false negative (miss)
FP = false positive  TN = true negative

The metrics

accuracy  = (TP + TN) / everything       "overall correctness"
precision = TP / (TP + FP)               "when it says YES, is it right?"
recall    = TP / (TP + FN)               "of all real YESes, how many caught?"
F1        = harmonic mean of P and R     "balance of the two"

Which to optimise?

• Recall matters most when missing a positive is costly: cancer screening, fraud, security alerts. Better a false alarm than a miss.
• Precision matters most when a false alarm is costly: spam filter (don't bin a real email), "you may be pregnant" ads.
• F1 when you need a single number balancing both.

In scikit-learn

from sklearn.metrics import confusion_matrix, classification_report

print(confusion_matrix(y_test, preds))
print(classification_report(y_test, preds))

              precision    recall  f1-score   support
           0       0.97      0.99      0.98        71
           1       0.98      0.95      0.96        43
    accuracy                           0.97       114

support is how many real samples of each class there were — your imbalance check, built in.

# scorecard.py — full evaluation, not just accuracy
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import (confusion_matrix, classification_report,
                             ConfusionMatrixDisplay)
import matplotlib.pyplot as plt

X, y = load_breast_cancer(return_X_y=True)
# In this dataset: 0 = malignant, 1 = benign
Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.2,
                                      stratify=y, random_state=0)

model = LogisticRegression(max_iter=5000).fit(Xtr, ytr)
preds = model.predict(Xte)

print("accuracy:", model.score(Xte, yte).round(3))
print("\nconfusion matrix:\n", confusion_matrix(yte, preds))
print("\n", classification_report(yte, preds,
                                   target_names=["malignant", "benign"]))

ConfusionMatrixDisplay.from_predictions(yte, preds,
    display_labels=["malignant", "benign"])
plt.savefig("confusion.png", dpi=150)

Sample output

accuracy: 0.965

confusion matrix:
 [[40  2]
 [ 2 70]]

              precision    recall  f1-score   support
   malignant       0.95      0.95      0.95        42
      benign       0.97      0.97      0.97        72
    accuracy                           0.96       114

Read the diff

Here, a false negative means calling a malignant tumour "benign" — the dangerous error. The matrix shows 2 such misses. For a medical tool you'd push recall on the malignant class higher even at the cost of more false alarms. Accuracy alone (96.5%) would never reveal that trade-off.

Accuracy hides the type of mistake. The confusion matrix shows TP/FP/FN/TN. Precision = "when it says yes, is it right"; recall = "did it catch all the real yeses"; F1 balances them. Pick the metric the real-world cost demands. Always read classification_report, not just the headline number.