Random Forests — Many Trees, One Vote

• Explain bagging: many trees on bootstrap samples, voting together.
• Train a RandomForestClassifier in two lines.
• Read forest feature importances (more stable than one tree).
• Tune n_estimators and max_depth.

Ask one person to guess the number of sweets in a jar — wildly off. Ask 100 people and average — eerily close. That's the random forest principle: many imperfect, diverse guessers beat one expert.

Random forest = 100s of decision trees, where each tree:
  - trains on a random BOOTSTRAP sample of rows (with replacement)
  - considers only a random SUBSET of features at each split
Final prediction = majority vote (classification) or average (regression)

Two lines to a strong model

from sklearn.ensemble import RandomForestClassifier

rf = RandomForestClassifier(n_estimators=100, random_state=0)
rf.fit(X_train, y_train)
print(rf.score(X_test, y_test))

Why it beats a single tree

single tree:  high accuracy on train, twitchy on test (high variance)
forest:       averages many trees → much lower variance, similar bias
              → better test accuracy, more stable across splits

Key knobs

n_estimators   how many trees (more = better but slower; ~100-300 is plenty)
max_depth      cap each tree's depth (None = grow fully, usual default)
max_features   features considered per split ("sqrt" is the classic default)
n_jobs=-1      use all CPU cores (trees train independently → parallel)

Feature importances — now trustworthy

import pandas as pd
imp = pd.Series(rf.feature_importances_, index=feature_names)
print(imp.sort_values(ascending=False).head())

Averaged over many trees, forest importances are far more stable than a single tree's — a real tool for understanding which features matter.

Regression too

from sklearn.ensemble import RandomForestRegressor
rfr = RandomForestRegressor(n_estimators=200, random_state=0)
# same fit / predict / score — score returns R²

# tree_vs_forest.py
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import cross_val_score
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
import pandas as pd

X, y = load_breast_cancer(return_X_y=True)

tree   = DecisionTreeClassifier(random_state=0)
forest = RandomForestClassifier(n_estimators=200, random_state=0, n_jobs=-1)

for name, m in [("single tree", tree), ("random forest", forest)]:
    s = cross_val_score(m, X, y, cv=5)
    print(f"  {name:<14} {s.mean():.1%} ± {s.std():.1%}")

# Top features from the forest
forest.fit(X, y)
names = load_breast_cancer().feature_names
imp = pd.Series(forest.feature_importances_, index=names)
print("\ntop features:")
print(imp.sort_values(ascending=False).head(5).round(3))

Sample output

  single tree    91.4% ± 2.3%
  random forest  96.0% ± 1.6%

top features:
worst perimeter      0.142
worst concave points 0.139
worst radius         0.111
mean concave points  0.104
worst area           0.092

Read the diff

The forest beats the single tree on both mean (96% vs 91%) and stability (smaller std). And the importances make medical sense — "worst" (largest) cell measurements dominate. Random forests are the sensible default for tabular data: strong, robust, and barely need tuning.

A random forest grows many diverse trees (random rows + random features) and votes. The diversity cancels each tree's overfitting, giving higher, more stable accuracy with almost no tuning. Importances are trustworthy. For most tabular problems, a random forest is the right default. Next: predicting numbers with linear regression.