Project — Cyberbullying Comment Filter

• Build a TF-IDF + classifier pipeline for "harmful vs OK".
• Evaluate with precision/recall — both errors matter here.
• Design it as a flag for human review, not an auto-ban.
• Discuss bias, false positives, and the limits of the model.

false negative: harmful comment slips through → someone gets hurt
false positive: harmless comment flagged       → free speech chilled,
                                                  innocent user frustrated

Data shape

comment (text)                          label
"great video, thanks for posting"        ok
"you're an idiot and should quit"        harmful
"i disagree but respect your view"        ok
...

Real datasets exist (Jigsaw Toxic Comments on Kaggle). For class, a small hand-labelled set works to learn the mechanics — but be honest that it's a toy.

The pipeline

from sklearn.pipeline import make_pipeline
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression

model = make_pipeline(
    TfidfVectorizer(stop_words="english", ngram_range=(1, 2), min_df=1),
    LogisticRegression(max_iter=2000, class_weight="balanced"),
)

class_weight="balanced" matters: harmful comments are rare, so we tell the model to weight the minority class up — otherwise it just predicts "ok" for everything.

Evaluate the RIGHT way

from sklearn.metrics import classification_report
print(classification_report(y_test, model.predict(X_test)))
# focus on recall and precision for the "harmful" class

Frame it as assistance

Output a SCORE + a recommendation, not a verdict:
  score > 0.8  → "flag for urgent review"
  score > 0.5  → "queue for review"
  else         → "looks ok"
A HUMAN makes the final call. Always.

# comment_filter.py — flag-for-review, not auto-ban
from sklearn.pipeline import make_pipeline
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report

# tiny illustrative dataset (use a real one for anything serious)
data = [
    ("thanks so much, this helped a lot", "ok"),
    ("great explanation, subscribed!", "ok"),
    ("i respectfully disagree with point 2", "ok"),
    ("nice work, keep it up", "ok"),
    ("you are so stupid, just give up", "harmful"),
    ("nobody likes you, get lost loser", "harmful"),
    ("worst person ever, you should quit", "harmful"),
    ("shut up idiot", "harmful"),
]
texts, labels = zip(*data)
Xtr, Xte, ytr, yte = train_test_split(texts, labels, test_size=0.25,
                                      stratify=labels, random_state=0)

model = make_pipeline(
    TfidfVectorizer(stop_words="english", ngram_range=(1, 2)),
    LogisticRegression(max_iter=2000, class_weight="balanced"),
).fit(Xtr, ytr)

def review(comment):
    proba = model.predict_proba([comment])[0]
    harmful_p = proba[list(model.classes_).index("harmful")]
    if harmful_p > 0.8:
        action = "🚨 flag for URGENT human review"
    elif harmful_p > 0.5:
        action = "⚠️  queue for human review"
    else:
        action = "✅ looks ok"
    return harmful_p, action

for c in ["you are amazing, thank you",
          "you should just quit, loser",
          "i don't agree but ok"]:
    p, action = review(c)
    print(f"[{p:.0%}] {action:<32} {c!r}")

Sample output

[ 8%] ✅ looks ok                      'you are amazing, thank you'
[91%] 🚨 flag for URGENT human review  'you should just quit, loser'
[34%] ✅ looks ok                      "i don't agree but ok"

Read the diff

Note the design: it outputs a probability and a recommendation to a human, never a ban. The thresholds (0.5/0.8) are policy decisions, not technical ones — a person chooses how cautious to be. This framing is the responsible way to ship any consequential classifier.

A toxicity classifier is a TF-IDF + balanced classifier — technically easy. The hard parts are ethical: accuracy lies on imbalanced data (use precision/recall), models inherit bias from their training data, and the system should flag for humans, never auto-punish. Build it to assist, measure it honestly, disclose its limits. Next: real LLMs.