PY-L8-45 · Logging & Audit Trails for Forensics

Learning Goals

3 min

By the end of this lesson you can:

Decide which security events to log (and at what level) — and which never to.
Write structured audit records: who, what, when, from where, outcome.
Make logs tamper-evident (append-only, hash-chained, shipped off-host).
Avoid the A09 logging/monitoring failures and pair logs with alerts.

Warm-Up · The Breach You Couldn't Investigate

5 min

OWASP A09 — Security Logging & Monitoring Failures — is on the list because the worst part of many breaches is that nobody could tell what happened: the attacker was in for months, and there were no usable logs to reconstruct the damage or even confirm the scope. You built log analysis in Lesson 25; this lesson is producing logs worth analysing.

Today's big idea

An audit trail is the record that lets you answer, after an incident, "who did what, when, from where, and what was the outcome?" Good audit logging is deliberate: log the security-relevant events (logins, privilege use, changes), never the secrets, make the trail tamper-evident (an attacker shouldn't be able to quietly erase their tracks), and alert on the events that matter. Logs are both your forensic memory and your detection signal.

New Concept · What to Log, Safely & Tamper-Evidently

14 min

What to log (security events)

LOG these:   logins (success AND failure), logouts, password/role changes,
             access-control denials (403s), privileged actions (delete, admin),
             input-validation failures, use of security features, config changes
DON'T log:   passwords, tokens, API keys, full card/SSN/PII, session contents
             (mask or omit — a log full of secrets is a breach waiting to happen)

The shape of a good audit record

import logging, json
from datetime import datetime, timezone

audit = logging.getLogger("audit")

def audit_log(action: str, *, actor, outcome, target=None, ip=None, **extra):
    record = {
        "ts": datetime.now(timezone.utc).isoformat(),   # WHEN (UTC, L8-... time)
        "actor": actor,                                   # WHO
        "action": action,                                # WHAT
        "target": target,                                # on what
        "outcome": outcome,                               # success / denied / error
        "ip": ip,                                         # FROM where
        **extra,
    }
    audit.info(json.dumps(record))      # structured (JSON) → easy to parse/search

# usage:
audit_log("login", actor="aisha", outcome="failure", ip="203.0.113.9")
audit_log("post.delete", actor="admin", target="post:42", outcome="success",
          ip="10.0.0.5")

The five Ws — who, what, when, where, outcome — make a record actionable. Structured JSON (one event per line) is far better than free text: it's greppable, parseable (Lesson 25), and ships cleanly to a SIEM. Use UTC timestamps so events correlate across servers (Lesson 12).

Tamper-evidence — attackers erase logs

⚠️ A log an attacker can edit is worthless for forensics

A skilled attacker's first move after access is often to delete or edit logs to hide their tracks. Defences: ship logs off-host in real time (to a separate log server/SIEM the attacker can't reach), make them append-only, and/or make them hash-chained so any deletion/edit is detectable. The integrity of the audit trail is as important as the trail itself — same lesson as protecting the FIM baseline (Lesson 21).

import hashlib, json

# hash-chain: each record includes the hash of the previous one.
# Deleting/altering any record breaks the chain → tampering is detectable.
class AuditChain:
    def __init__(self):
        self.prev_hash = "0" * 64        # genesis
    def append(self, record: dict) -> dict:
        record["prev_hash"] = self.prev_hash
        line = json.dumps(record, sort_keys=True)
        record["hash"] = hashlib.sha256(line.encode()).hexdigest()
        self.prev_hash = record["hash"]  # next record chains to this
        return record                    # write/ship it; verify the chain later

Retention, privacy, and alerting

Retain long enough to investigate (breaches are often found months later) — but mind storage and privacy law.
Privacy — logs may contain personal data; protect and minimise them too (GDPR etc.).
Alert — logging without monitoring is half the job. Pair the trail with alerts (Lesson 34): many failed logins, a privilege escalation, a 403 spike → notify someone (Level 7 notifier).
Synchronised clocks (NTP) + UTC so timelines line up across systems.

Worked Example · A Tamper-Evident Audit Logger

12 min

Goal: an audit logger that writes structured, hash-chained records, masks secrets, and can verify the chain — so you can both investigate and prove the log wasn't tampered with.

import hashlib, json, logging
from datetime import datetime, timezone
from pathlib import Path

class AuditLogger:
    def __init__(self, path: str = "audit.log"):
        self.path = Path(path)
        # resume the chain from the last record if the file exists
        self.prev = self._last_hash()

    def _last_hash(self) -> str:
        if not self.path.exists():
            return "0" * 64
        last = self.path.read_text(encoding="utf-8").splitlines()[-1]
        return json.loads(last)["hash"]

    def _mask(self, d: dict) -> dict:
        # never write secret-ish fields
        return {k: ("***" if k.lower() in
                    ("password", "token", "secret", "pw", "api_key") else v)
                for k, v in d.items()}

    def log(self, action: str, *, actor, outcome, **fields) -> None:
        record = {
            "ts": datetime.now(timezone.utc).isoformat(),
            "actor": actor, "action": action, "outcome": outcome,
            **self._mask(fields), "prev_hash": self.prev,
        }
        line = json.dumps(record, sort_keys=True)
        record["hash"] = hashlib.sha256(line.encode()).hexdigest()
        self.prev = record["hash"]
        # append-only write (and in prod: also ship off-host in real time)
        with open(self.path, "a", encoding="utf-8") as f:
            f.write(json.dumps(record) + "\n")

    def verify(self) -> bool:
        prev = "0" * 64
        for line in self.path.read_text(encoding="utf-8").splitlines():
            rec = json.loads(line)
            stored = rec.pop("hash")
            if rec["prev_hash"] != prev:
                return False                 # chain broken → tampered/deleted
            expected = hashlib.sha256(
                json.dumps(rec, sort_keys=True).encode()).hexdigest()
            if expected != stored:
                return False                 # record edited
            prev = stored
        return True

log = AuditLogger()
log.log("login", actor="aisha", outcome="failure", ip="203.0.113.9")
log.log("post.delete", actor="admin", outcome="success", target="post:42",
        ip="10.0.0.5", token="should-not-appear")
print("chain intact:", log.verify())            # True
# now simulate an attacker editing a line → verify() returns False

audit.log (one JSON record per line):
  {"ts":"2026-...Z","actor":"aisha","action":"login","outcome":"failure",
   "ip":"203.0.113.9","prev_hash":"000...","hash":"a1b2..."}
  {"ts":"2026-...Z","actor":"admin","action":"post.delete","outcome":"success",
   "target":"post:42","ip":"10.0.0.5","token":"***","prev_hash":"a1b2...","hash":"c3d4..."}
chain intact: True
# after editing/deleting any line → verify() == False (tampering detected)

Read the code

Three security properties in one logger. Records are structured with the five Ws (greppable, SIEM-ready). Secrets are masked — note the token field becomes ***, so a leaked log isn't a leaked credential. And the hash chain makes tampering evident: each record commits to the previous one's hash, so editing or deleting any line breaks verify(). In production you'd also ship each line off-host in real time (so the attacker can't reach the only copy). This is an audit trail that holds up in an investigation — the A09 defence.

Try It Yourself

13 min

01 🟢 Structured audit records

Write audit_log producing JSON records with who/what/when/where/outcome. Log a few events (login success/failure, a delete) and confirm they parse cleanly (feed them to your Lesson 25 analyzer).

02 🟡 Mask secrets & choose levels

Add masking so passwords/tokens never reach the log, and assign levels (failed login → WARNING, privilege use → INFO, repeated failures → ERROR). Confirm a token passed in is recorded as ***.

03 🔴 Prove tamper-evidence

Implement the hash-chained logger, write 10 records, and confirm verify() is True. Then edit one record (as an attacker would) and confirm verify() returns False, pinpointing the break.

Hint

# tamper: change a byte in one line of audit.log, then:
print(AuditLogger("audit.log").verify())   # → False
# extend verify() to print WHICH line index first failed.

Mini-Challenge · Audit + Alert Integration

8 min

Close the loop: wire the audit logger to detection + alerting. As records are written, watch for patterns (N failed logins from one IP, any privilege escalation, a burst of 403s) and fire an alert (Lesson 34's notifier) — so the trail isn't just forensic, it's real-time monitoring (the full A09 fix).

Show a sample solution

from collections import defaultdict
import time

class MonitoredAudit(AuditLogger):
    def __init__(self, path="audit.log"):
        super().__init__(path)
        self.fails = defaultdict(list)
    def log(self, action, *, actor, outcome, **fields):
        super().log(action, actor=actor, outcome=outcome, **fields)
        # real-time detection on top of the trail:
        if action == "login" and outcome == "failure":
            ip = fields.get("ip", "?")
            now = time.time()
            self.fails[ip] = [t for t in self.fails[ip] if t > now - 60]
            self.fails[ip].append(now)
            if len(self.fails[ip]) >= 5:
                alert(f"brute force: {len(self.fails[ip])} failed logins "
                      f"from {ip}", "critical")          # Lesson 34 notifier
        if action.endswith("role.change") and fields.get("to") == "admin":
            alert(f"privilege escalation: {actor} → admin", "critical")

Non-negotiables: writes the tamper-evident trail AND alerts in real time on brute force / privilege escalation.

Recap

3 min

You can't investigate what you didn't log — A09 (logging/monitoring failures) is on the OWASP list for a reason. Log the security-relevant events (logins success+failure, privilege use, access denials, changes), with the five Ws — who, what, when (UTC), where, outcome — as structured JSON. Never log secrets/PII; mask them. Make the trail tamper-evident (append-only, hash-chained, shipped off-host) so an attacker can't erase their tracks. And pair the trail with alerting — logging without monitoring is half a defence. Good audit logs are both your forensic memory and your detection signal.

Vocabulary Card

audit trail: A record of security-relevant events: who/what/when/where/outcome.
tamper-evident log: Append-only/hash-chained/off-host so edits or deletions are detectable.
A09: OWASP Logging & Monitoring Failures — can't detect or investigate attacks.
five Ws: Who, What, When, Where, outcome — the anatomy of a useful audit record.

Homework

4 min

Add a tamper-evident audit logger to your Lesson 41 secured blog: log logins (success/failure), privileged actions, and access denials as masked, hash-chained JSON. Wire it to alerting for brute force and privilege escalation. Simulate an attack, then use your Lesson 25 analyzer on the audit log to reconstruct it — and prove the chain detects a tampered line. Write a note on what you log, what you never log, and how you keep the trail trustworthy.

Sample · audit logging note

What I log (security events, as masked hash-chained JSON):
  - login success + FAILURE (failures are the brute-force signal)
  - logout, password change, ROLE change
  - access-control denials (403s) and privileged actions (post.delete,
    user.manage) — with actor, target, UTC ts, ip, outcome.

What I NEVER log: passwords, tokens, API keys, full PII/card numbers.
The logger masks these fields to "***" before writing.

Keeping the trail trustworthy:
  - hash-chained: each record commits to the previous hash, so any
    edit/deletion breaks verify() (tampering detected).
  - append-only + shipped off-host in real time, so an attacker who
    gets the box can't quietly erase their tracks.
  - UTC timestamps + NTP so events correlate across servers.
  - paired with ALERTS: 5 failed logins/min or any →admin escalation
    pages on-call immediately (logging without monitoring = half a defence).

Reconstruction: fed audit.log to my L8-25 analyzer → it surfaced the
brute force → success → delete sequence; verify() flagged a line I
tampered with as a test.

Non-negotiables: tamper-evident masked audit logging wired to the blog + alerts, an attack reconstructed from the log, chain-tamper detection, and the what/never/trustworthy note.

LOG these: logins (success AND failure), logouts, password/role changes, access-control denials (403s), privileged actions (delete, admin), input-validation failures, use of security features, config changes DON'T log: passwords, tokens, API keys, full card/SSN/PII, session contents (mask or omit — a log full of secrets is a breach waiting to happen)

import logging, json from datetime import datetime, timezone audit = logging.getLogger("audit") def audit_log(action: str, *, actor, outcome, target=None, ip=None, **extra): record = { "ts": datetime.now(timezone.utc).isoformat(), # WHEN (UTC, L8-... time) "actor": actor, # WHO "action": action, # WHAT "target": target, # on what "outcome": outcome, # success / denied / error "ip": ip, # FROM where **extra, } audit.info(json.dumps(record)) # structured (JSON) → easy to parse/search # usage: audit_log("login", actor="aisha", outcome="failure", ip="203.0.113.9") audit_log("post.delete", actor="admin", target="post:42", outcome="success", ip="10.0.0.5")

import hashlib, json # hash-chain: each record includes the hash of the previous one. # Deleting/altering any record breaks the chain → tampering is detectable. class AuditChain: def __init__(self): self.prev_hash = "0" * 64 # genesis def append(self, record: dict) -> dict: record["prev_hash"] = self.prev_hash line = json.dumps(record, sort_keys=True) record["hash"] = hashlib.sha256(line.encode()).hexdigest() self.prev_hash = record["hash"] # next record chains to this return record # write/ship it; verify the chain later

import hashlib, json, logging from datetime import datetime, timezone from pathlib import Path class AuditLogger: def __init__(self, path: str = "audit.log"): self.path = Path(path) # resume the chain from the last record if the file exists self.prev = self._last_hash() def _last_hash(self) -> str: if not self.path.exists(): return "0" * 64 last = self.path.read_text(encoding="utf-8").splitlines()[-1] return json.loads(last)["hash"] def _mask(self, d: dict) -> dict: # never write secret-ish fields return {k: ("***" if k.lower() in ("password", "token", "secret", "pw", "api_key") else v) for k, v in d.items()} def log(self, action: str, *, actor, outcome, **fields) -> None: record = { "ts": datetime.now(timezone.utc).isoformat(), "actor": actor, "action": action, "outcome": outcome, **self._mask(fields), "prev_hash": self.prev, } line = json.dumps(record, sort_keys=True) record["hash"] = hashlib.sha256(line.encode()).hexdigest() self.prev = record["hash"] # append-only write (and in prod: also ship off-host in real time) with open(self.path, "a", encoding="utf-8") as f: f.write(json.dumps(record) + "\n") def verify(self) -> bool: prev = "0" * 64 for line in self.path.read_text(encoding="utf-8").splitlines(): rec = json.loads(line) stored = rec.pop("hash") if rec["prev_hash"] != prev: return False # chain broken → tampered/deleted expected = hashlib.sha256( json.dumps(rec, sort_keys=True).encode()).hexdigest() if expected != stored: return False # record edited prev = stored return True log = AuditLogger() log.log("login", actor="aisha", outcome="failure", ip="203.0.113.9") log.log("post.delete", actor="admin", outcome="success", target="post:42", ip="10.0.0.5", token="should-not-appear") print("chain intact:", log.verify()) # True # now simulate an attacker editing a line → verify() returns False

audit.log (one JSON record per line): {"ts":"2026-...Z","actor":"aisha","action":"login","outcome":"failure", "ip":"203.0.113.9","prev_hash":"000...","hash":"a1b2..."} {"ts":"2026-...Z","actor":"admin","action":"post.delete","outcome":"success", "target":"post:42","ip":"10.0.0.5","token":"***","prev_hash":"a1b2...","hash":"c3d4..."} chain intact: True # after editing/deleting any line → verify() == False (tampering detected)

from collections import defaultdict import time class MonitoredAudit(AuditLogger): def __init__(self, path="audit.log"): super().__init__(path) self.fails = defaultdict(list) def log(self, action, *, actor, outcome, **fields): super().log(action, actor=actor, outcome=outcome, **fields) # real-time detection on top of the trail: if action == "login" and outcome == "failure": ip = fields.get("ip", "?") now = time.time() self.fails[ip] = [t for t in self.fails[ip] if t > now - 60] self.fails[ip].append(now) if len(self.fails[ip]) >= 5: alert(f"brute force: {len(self.fails[ip])} failed logins " f"from {ip}", "critical") # Lesson 34 notifier if action.endswith("role.change") and fields.get("to") == "admin": alert(f"privilege escalation: {actor} → admin", "critical")