Process Inspection & Anomaly Detection

By the end of this lesson you can:

• Diff a live host snapshot against a baseline to find new processes/ports/connections.
• Inspect process lineage (parent → child) and binary paths for red flags.
• Apply layered anomaly heuristics and score suspicion.
• Reason about false positives and the limits of heuristic detection.

You can't maintain a list of every possible piece of malware — there are millions and new ones daily. But you can know what your own clean system looks like. Anything that deviates from that known-good baseline is worth a look.

1. Diff against the baseline

import json, psutil

def current_names() -> set:
    names = set()
    for p in psutil.process_iter(["name"]):
        names.add(p.info["name"])
    return names

def diff_processes(baseline_path: str) -> set:
    base = set(json.loads(open(baseline_path).read())["process_names"])
    now = current_names()
    new = now - base                 # processes present NOW but not in baseline
    return new

print("new since baseline:", diff_processes("host_baseline.json"))

Set subtraction does the work: now - baseline = what appeared. Do the same for listening ports and established remotes. New items aren't necessarily malicious (you may have launched a new app), but they're where to look first.

2. Inspect process lineage

import psutil

def lineage(pid: int) -> list[str]:
    """Walk parent chain: who launched this process?"""
    chain = []
    p = psutil.Process(pid)
    while p:
        chain.append(f"{p.name()}({p.pid})")
        try:
            p = p.parent()
        except psutil.Error:
            break
    return chain        # e.g. ['evil.bin(900)', 'bash(880)', 'sshd(440)', ...]

Lineage is forensic gold. A shell (bash, cmd, powershell) spawned by a web server process is a huge red flag — it often means a webshell or RCE gave an attacker command execution. Legitimate apps have predictable parents; attacker payloads often don't.

3. Anomaly heuristics (layered, scored)

import psutil

SUSPICIOUS_PARENTS = {"nginx", "apache2", "httpd", "java", "node"}  # shouldn't spawn shells
SHELLS = {"sh", "bash", "cmd.exe", "powershell.exe"}

def suspicion_score(p: psutil.Process) -> tuple[int, list[str]]:
    score, reasons = 0, []
    try:
        exe = (p.exe() or "").lower()
        name = p.name().lower()
        parent = p.parent().name().lower() if p.parent() else ""

        if "/tmp" in exe or "\\temp\\" in exe or "/dev/shm" in exe:
            score += 3; reasons.append("binary in a temp/world-writable dir")
        if name in SHELLS and parent in SUSPICIOUS_PARENTS:
            score += 5; reasons.append(f"shell spawned by {parent} (RCE?)")
        if p.username() in ("root", "SYSTEM") and "/tmp" in exe:
            score += 3; reasons.append("privileged process from temp dir")
        if not exe:
            score += 1; reasons.append("no backing executable (memory-only?)")
    except psutil.Error:
        pass
    return score, reasons

No single heuristic is proof — so we score and combine. A binary in /tmp (+3) that's a shell spawned by nginx (+5) running as root scores high enough to demand investigation. Each rule encodes a known attacker behaviour (TTP).

Goal: combine the baseline diff with the heuristic scorer into a small host intrusion-detection scan that ranks processes by suspicion and explains why — so a human can triage fast.

import json, psutil

SHELLS = {"sh", "bash", "zsh", "cmd.exe", "powershell.exe"}
SHOULDNT_SPAWN_SHELL = {"nginx", "apache2", "httpd", "node", "java", "python"}

def score(p: psutil.Process, baseline_names: set) -> tuple[int, list[str]]:
    s, reasons = 0, []
    try:
        name = p.name().lower()
        exe = (p.exe() or "").lower()
        parent = p.parent().name().lower() if p.parent() else ""
        if name not in baseline_names:
            s += 1; reasons.append("not in baseline")
        if any(d in exe for d in ("/tmp", "\\temp\\", "/dev/shm")):
            s += 3; reasons.append("runs from temp/world-writable dir")
        if name in SHELLS and parent in SHOULDNT_SPAWN_SHELL:
            s += 5; reasons.append(f"shell child of {parent} — possible RCE")
        if p.cpu_percent() > 85:
            s += 2; reasons.append("very high CPU (miner?)")
    except psutil.Error:
        pass
    return s, reasons

def scan(baseline_path: str) -> None:
    baseline = set(n.lower()
                   for n in json.loads(open(baseline_path).read())["process_names"])
    findings = []
    for p in psutil.process_iter():
        s, reasons = score(p, baseline)
        if s > 0:
            findings.append((s, p.pid, p.name(), reasons))

    print("Host IDS scan — investigate highest scores first:\n")
    for s, pid, name, reasons in sorted(findings, reverse=True)[:10]:
        flag = "🔴" if s >= 5 else "🟡" if s >= 3 else "⚪"
        print(f"{flag} score {s:2}  {name} (pid {pid})")
        for r in reasons:
            print(f"            • {r}")

scan("host_baseline.json")

Host IDS scan — investigate highest scores first:

🔴 score  9  sh (pid 9123)
            • not in baseline
            • runs from temp/world-writable dir
            • shell child of nginx — possible RCE
🟡 score  3  xmrig (pid 9201)
            • not in baseline
            • very high CPU (miner?)
⚪ score  1  newapp (pid 9300)
            • not in baseline

Read the code

This is the shape of real host-based detection: combine baseline-diff ("is it new?") with behavioural heuristics ("does it look like an attacker TTP?"), score, and rank so the analyst's attention goes to the 🔴 nine before the ⚪ one. The top finding — a shell, running from /tmp, spawned by nginx — is a textbook web-RCE signature worth dropping everything for. The bottom finding (a new app you installed) is almost certainly benign noise. The tool doesn't decide guilt; it focuses the human, which is exactly what detection tooling is for.

Anomaly detection recognises "not normal" rather than enumerating "bad." Diff a live snapshot against a known-good baseline (processes, ports, remotes) to surface what's new; inspect process lineage (a shell spawned by a web server screams RCE) and binary paths (temp dirs are suspicious); then combine layered heuristics into a score and rank, so an analyst's attention goes to the worst first. It's the core of host-based IDS/EDR — and it's inherently noisy, so a high score means "investigate," never "auto-act." Tuning the false-positive/false-negative balance is the real skill.