Web Automation: Playwright (Modern Alternative)

By the end of this lesson you can:

• Install Playwright and its bundled browsers.
• Drive a page with the sync API: navigate, click, fill, read.
• Use auto-waiting locators and never write a manual wait again.
• Decide between Playwright and Selenium for a given task.

Recall the Selenium dance: import By, WebDriverWait, expected_conditions; wrap every interaction in an explicit wait; remember to quit(). Powerful, but verbose — and forgetting a wait makes tests flaky.

pip install playwright
playwright install        # downloads bundled Chromium/Firefox/WebKit

Launch with a context manager

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True)
    page = browser.new_page()
    page.goto("https://example.com")
    print(page.title())
    browser.close()

The with block manages startup/teardown. headless=True by default; set headless=False to watch it run while developing. Three browser engines are built in: p.chromium, p.firefox, p.webkit.

Locators that auto-wait

# build a locator — nothing happens yet
heading = page.locator("h1")
search  = page.get_by_role("textbox", name="Search")
button  = page.get_by_role("button", name="Submit")

# acting on it auto-waits for the element to be ready:
button.click()                  # waits until clickable, then clicks
search.fill("playwright")       # waits, clears, types
print(heading.inner_text())     # waits until present, then reads

A locator is a recipe for finding an element, re-evaluated each time you use it. The action (click, fill) waits for the element to be actionable — so timing bugs largely disappear.

User-facing selectors (recommended)

page.get_by_role("button", name="Login")     # by accessible role + name
page.get_by_text("Add to cart")              # by visible text
page.get_by_label("Email")                   # form field by its label
page.get_by_placeholder("Search…")           # by placeholder
page.locator("#results .item")               # CSS, when needed

Playwright nudges you toward selectors a user would recognise (role, text, label). These are robust (Lesson 23's lesson again) and double as accessibility checks.

Explicit assertions and waits (when you do need them)

from playwright.sync_api import expect

expect(page.locator(".result")).to_be_visible()      # auto-retries up to a timeout
expect(page.locator(".count")).to_have_text("3 found")

page.wait_for_url("**/dashboard")                     # wait for navigation
page.wait_for_load_state("networkidle")               # wait for network to settle

expect(...) retries the assertion until it passes or times out — perfect for "wait until the result appears." You rarely need raw waits.

Handy extras

page.screenshot(path="shot.png", full_page=True)
page.pdf(path="page.pdf")                  # save the page as PDF (chromium)
content = page.content()                   # rendered HTML → BeautifulSoup
page.goto(url, wait_until="domcontentloaded")

Goal: the same JS-rendered quote scrape from Lesson 26, now in Playwright — notice how the auto-waiting removes the explicit-wait boilerplate.

from playwright.sync_api import sync_playwright
import csv, logging

logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
log = logging.getLogger("playwright")

def scrape_quotes() -> list[dict]:
    quotes = []
    with sync_playwright() as p:
        browser = p.chromium.launch(headless=True)
        page = browser.new_page()
        page.goto("https://quotes.toscrape.com/js/")

        # locator auto-waits for the quotes to render — no WebDriverWait needed
        cards = page.locator(".quote")
        cards.first.wait_for()              # ensure at least one exists
        count = cards.count()
        log.info("found %d quotes", count)

        for i in range(count):
            card = cards.nth(i)
            quotes.append({
                "text": card.locator(".text").inner_text(),
                "author": card.locator(".author").inner_text(),
            })
        browser.close()
    return quotes

rows = scrape_quotes()
with open("quotes.csv", "w", newline="", encoding="utf-8") as f:
    w = csv.DictWriter(f, fieldnames=["text", "author"])
    w.writeheader(); w.writerows(rows)
log.info("wrote %d quotes", len(rows))

INFO found 10 quotes
INFO wrote 10 quotes

Read the code

Compare with Lesson 26: no By, no WebDriverWait/expected_conditions imports, no try/finally (the with block closes the browser). The cards.first.wait_for() is the only explicit wait, and even that's often unnecessary because acting on a locator auto-waits. cards.nth(i) indexes the matched set. Same result, noticeably less ceremony — which is why Playwright has become many teams' default for new browser automation.

Playwright is the modern take on browser automation: with sync_playwright() manages the browser, bundled engines avoid driver headaches, and locators auto-wait for elements to be ready — so flaky timing bugs mostly vanish and you rarely write an explicit wait. Prefer user-facing selectors (get_by_role, get_by_text, get_by_label), and use expect(...) for retrying assertions. The concepts are identical to Selenium (Lesson 26) with far less ceremony. Choose Playwright for new work and flaky-test relief; Selenium for legacy suites and the widest ecosystem.