CLI route-list command
Use 'playwright-cli route-list' to display all active routes that are currently mocking requests.
ci and best practices
65 notes in this subject, read out of this brain and free to use. This is page 1 of 2.
CLI unroute command
Use 'playwright-cli unroute' with a URL pattern to remove a specific route, or use 'playwright-cli unroute' with no arguments to remove all active routes. Example: playwright-cli unroute "**/*.jpg" removes mocking for JPG files; playwright-cli unroute removes all routes.
URL pattern matching - exact path and wildcard
URL patterns support exact path matching (e.g., **/api/users), wildcard matching in the path (e.g., **/api/*/details), file extension matching (e.g., **/*.{png,jpg,jpeg}), and query parameter matching (e.g., **/search?q=*).
Conditional response based on request body
Use 'playwright-cli run-code' with page.route() to inspect the request body and return conditional responses. Call route.request().postDataJSON() to parse the request body, and route.fulfill() to return different responses based on the body content.
Modify real response from server
Use 'playwright-cli run-code' with page.route() to intercept a real response from the server. Call route.fetch() to get the actual response, modify it, and pass the modified response or json to route.fulfill(). Example modifies the isPremium field of a user API response.
Simulate network failures with route.abort()
Use 'playwright-cli run-code' with route.abort() to simulate network failures. Supported error codes are: connectionrefused, timedout, connectionreset, internetdisconnected. Example: route.abort('internetdisconnected') simulates internet disconnection.
Delayed response with setTimeout
Use 'playwright-cli run-code' with page.route() and setTimeout to add a delay before returning a mocked response. This simulates slow network conditions or server latency. Example adds a 3000ms delay before fulfilling the response.
CLI route command - mock with custom status
Use 'playwright-cli route' with the --status flag to mock requests and return a custom HTTP status code. Example: playwright-cli route "**/*.jpg" --status=404 returns 404 for all JPG files.
CLI route command - mock with JSON body
Use 'playwright-cli route' with the --body flag to return a mocked JSON response. The --content-type flag specifies the response content type. Example: playwright-cli route "**/api/users" --body='[{"id":1,"name":"Alice"}]' --content-type=application/json
CLI route command - mock with custom headers
Use 'playwright-cli route' with the --body and --header flags to return a mocked response with custom headers. Example: playwright-cli route "**/api/data" --body='{"ok":true}' --header="X-Custom: value"
CLI route command - remove headers from requests
Use 'playwright-cli route' with the --remove-header flag to strip headers from outgoing requests. Multiple headers are comma-separated. Example: playwright-cli route "**/*" --remove-header=cookie,authorization removes cookie and authorization headers from all requests.
playwright-cli detach command
Use 'playwright-cli detach' to tear down an attached session without affecting the external browser. For the default attached session, use detach without arguments; for a specific session, use 'playwright-cli -s=sessionname detach'. The detach command only works on sessions created via attach, not on sessions created via open.
playwright-cli open command configuration options
The open command accepts --config=<file> to use a config file, --browser=<name> to specify a browser type, --headed to run in headed mode, and --persistent to persist the browser profile to disk.
Best practice: name browser sessions semantically
Use descriptive, semantic names for browser sessions that clearly indicate their purpose (e.g., 'github-auth', 'docs-scrape') rather than generic names (e.g., 's1', 's2').
Best practice: always clean up browser sessions
Stop browsers when done using them. Use individual session close commands (playwright-cli -s=auth close) or 'playwright-cli close-all' to stop all sessions. If browsers become unresponsive or zombie processes remain, use 'playwright-cli kill-all'.
Best practice: delete stale browser data
Remove old browser profile data periodically using 'playwright-cli -s=oldsession delete-data' to free disk space from sessions that are no longer needed.
Concurrent scraping example with playwright-cli
Multiple browser sessions can be started concurrently with background execution (&) and wait to coordinate operations across sites. Example: #!/bin/bash playwright-cli -s=site1 open https://site1.com & playwright-cli -s=site2 open https://site2.com & playwright-cli -s=site3 open https://site3.com & wait playwright-cli -s=site1 snapshot playwright-cli -s=site2 snapshot playwright-cli -s=site3 snapshot playwright-cli close-all
Browser session isolation properties
Each browser session created with the -s flag has independent cookies, LocalStorage, SessionStorage, IndexedDB, cache, browsing history, and open tabs.
A/B testing sessions example with playwright-cli
Browser sessions can be used to test different user experiences concurrently. Example: playwright-cli -s=variant-a open "https://app.com?variant=a" playwright-cli -s=variant-b open "https://app.com?variant=b" playwright-cli -s=variant-a screenshot playwright-cli -s=variant-b screenshot
playwright-cli list command
The command 'playwright-cli list' displays all currently running browser sessions.
playwright-cli -s flag for named browser sessions
Use the -s flag with playwright-cli to create named, isolated browser sessions. Each session maintains separate cookies, LocalStorage, SessionStorage, IndexedDB, cache, browsing history, and open tabs. When -s is omitted, commands use the default browser session.
playwright-cli close and close-all commands
Use 'playwright-cli close' to stop the default browser session, 'playwright-cli -s=mysession close' to stop a named session, and 'playwright-cli close-all' to stop all browser sessions.
playwright-cli kill-all command
The command 'playwright-cli kill-all' forcefully terminates all daemon processes, used for removing stale or zombie processes when browsers become unresponsive.
playwright-cli delete-data command
Use 'playwright-cli delete-data' to delete default browser session user data (profile directory), or 'playwright-cli -s=mysession delete-data' to delete data for a named session.
PLAYWRIGHT_CLI_SESSION environment variable
Set the PLAYWRIGHT_CLI_SESSION environment variable to specify a default browser session name that will be used automatically by playwright-cli commands when the -s flag is not provided.
playwright-cli persistent profile with --persistent flag
By default, browser profiles are kept in memory only. Use the --persistent flag on the open command to persist the browser profile to disk at an auto-generated location, or use --profile=/path/to/profile to specify a custom directory.
playwright-cli attach command for connecting to running browsers
Use 'playwright-cli attach' to connect to a browser that is already running instead of launching a new one. Supports attaching by channel name (--cdp=chrome, --cdp=msedge, etc.), CDP endpoint (--cdp=http://localhost:9222), or browser extension (--extension).
playwright-cli attach with channel names
Supported channel names for attach with --cdp flag are: chrome, chrome-beta, chrome-dev, chrome-canary, msedge, msedge-beta, msedge-dev, msedge-canary. Target browsers must have remote debugging enabled at chrome://inspect/#remote-debugging.
playwright-cli attach automatic session naming
When --session is not provided with attach, the session is named after the channel (e.g., --cdp=msedge creates a session named 'msedge'). This prevents naming collisions when attaching to different browsers. Use --session=<name> to override the automatic name.
Setting up side-by-side Playwright installs for regression testing
Create two separate directories with independently installed Playwright versions. Run: mkdir -p ~/tmp/<good>/tests ~/tmp/<bad>/tests, then ( cd ~/tmp/<good> && npm init -y && npm install @playwright/test@<good-ver> && npx playwright install chromium) and ( cd ~/tmp/<bad> && npm init -y && npm install @playwright/test@<bad-ver> && npx playwright install chromium ). Place identical test files in both folders and run npx playwright test in each.
Bisecting regressions across published Playwright versions
When a user reports a regression between two published Playwright versions, reproduce both side by side from npm by installing from different version tags. Do not try to bisect against the monorepo source. The compiled JavaScript in node_modules/playwright/lib/ is faster to analyze and avoids build and branch confusion.
Use ~/tmp for side-by-side version installs, not /tmp
When setting up side-by-side installations of different Playwright versions for bisecting, use ~/tmp/<version-tag>/ directories, not /tmp/. The user's shell sessions live in ~/tmp/, making it the correct location for interactive work.
Avoid npm init playwright@latest for regression testing
Do not use npm init playwright@latest when setting up a regression reproduction because it is interactive and the scaffold pulls in 3 projects (chromium/firefox/webkit) which produces 6 test runs from a single spec, creating confusing output. Instead use npm init -y followed by npm install @playwright/test@<version>.
How to report a confirmed Playwright regression root cause
When the root cause is confirmed: 1) Quote the offending lines from node_modules/.../lib/... of the bad version with file path. 2) Show the equivalent code from the good version for contrast. 3) Explain why the change breaks the user's case, not just point at the diff. 4) Propose and verify a minimal fix by patching the bad install in place. Post the writeup as a comment on the original issue using: gh issue comment <number> --repo microsoft/playwright --body "$(cat <<'EOF' ... EOF)".
Don't attempt to map bugs to monorepo source first when bisecting
When bisecting regressions, do not try to map the bug to monorepo source first. The shipped JavaScript in node_modules/ is what the user is running; the source may have already been refactored or fixed on main. Investigate node_modules/ first, then map the fix back to source only when proposing the upstream patch.
Use && to chain cd commands in bash to maintain working directory
Do not cd between commands in a single Bash call without &&. The shell cwd resets between tool invocations. Use && to chain commands in sequence to maintain the correct working directory.
Be cautious when deleting existing ~/tmp directories during regression work
Do not rm -rf an existing ~/tmp/<ver>/ without checking what it contains. It may contain the user's prior work. Edit the regression test files in place instead.
Playwright Test features compared to Testing Library
Playwright Test provides: full zero-configuration TypeScript support; running tests across all web engines (Chrome, Firefox, Safari) on any popular operating system (Windows, macOS, Ubuntu); full support for multiple origins, frames, tabs and contexts; tests run in isolation in parallel across multiple browsers; built-in test artifact collection; Visual Studio Code integration; UI mode for debugging with time travel; Playwright Inspector; Playwright Test code generation; Playwright Tracing for post-mortem debugging.
CI health report generation and analysis
CI health reports are generated for the last commit on the main branch of microsoft/playwright to show the full picture of what's failing, grouped by root cause. The process involves two phases: fetching logs using a provided script, then analyzing and compiling the report. The report distinguishes between possible regressions introduced by the specific commit, pre-existing flakes and infrastructure issues.
Fetch commit logs script and output structure
Run 'bash .claude/skills/playwright-devops/fetch-commit-logs.sh [<sha>]' to download failed job logs. If no SHA is provided, it fetches the last commit on main. The script creates ~/tmp/commit-<short-sha>/ directory containing: summary.json with commit info, failed workflows, and failed job metadata; and <workflow-name>/<job-name>.log files with failed log output for each failed job. The script fetches failed jobs from both failed AND in-progress workflows.
CI health report structure and content
A CI health report should include: a title with short SHA, commit message, summary section with overview of workflows/jobs/test failures and note of in-progress workflows, grouped bullets for possible regressions and pre-existing issues, infrastructure issues section, and detailed failures organized by workflow with tables containing test paths and error messages. The summary should appear first for immediate visibility of what matters, grouping related failures (same test across browsers) into single bullets rather than listing individually.
CI report analysis workflow
To analyze CI failures: read summary.json to get commit message and list of failed workflows/jobs; read each .log file and extract failing test names and error messages; compile the report starting with summary section, then detailed tables grouped by workflow and job; save the report to ci-failures-<short-sha>.md in the repo root.
Playwright test results database schema - test_results table columns
The test_results table contains one row per test result (one row per retry). Columns are: run_id (GitHub Actions run identity), run_attempt (GitHub Actions run attempt number), run_started_at (when the run started), workflow_name (e.g. 'tests 1', 'tests 2', 'tests others', 'MCP'), event ('push' or 'pull_request'), head_sha (what was tested), head_branch (what was tested), pr_number (what was tested), bot_name (CI bot identifier, e.g. 'chromium-ubuntu-22.04-node20', 'webkit-macos-15-large'; OS and arch are encoded here), project_name (CI project = browser + suite, e.g. 'chromium-page', 'webkit-library', 'playwright-test'), test_title (title path within the file, joined by ' › ' for describe › test), file (source location relative to repo root), line (source location), column_number (source location), expected_status ('passed', 'skipped', etc.), status (actual result: 'passed', 'failed', 'timedOut', 'skipped', 'interrupted'), retry (0 = first attempt), result_started_at (when this attempt started), duration_ms (result duration), error_message (all errors joined, ANSI-stripped, NULL when none), tags (list of strings, e.g. ['@slow', '@flaky']; use list functions / list_contains), annotations (list of {type, description} structs, e.g. [{'type': 'skip', 'description': 'flaky on CI'}]; empty list when none), artifact_id (the GitHub artifact this row came from, used as dedupe key), ingested_at (debug only — when this row was imported).
Test identification in Playwright test results database
A test is identified by the tuple (project_name, file, test_title). Group results on that tuple to identify a unique test. Playwright's test_id hash is deliberately not stored; these three columns are its pre-image.
Flakiness signal in Playwright test results database
Flakiness is derived, not stored as a column. The signal that matters most is cross-run flakiness: a test whose final verdict (after retries) flips between runs — green in some runs, red in others. A separate within-run flake is a test where a retry rescued it inside a single run (status changed from 'failed' to 'passed'). Use arg_max(status, retry) to get the final status after retries.
Filtering real test failures in Playwright test results
To filter out intentional test failures, scope queries to expected_status = 'passed'. Tests marked test.fail() record status='failed' with expected_status='failed' and would otherwise dominate any 'most failing' list.
Playwright test results database retention policy
The database is size-capped by run count: the oldest whole runs are evicted over time, so it holds a recent window, not full history.
Download and update Playwright CI test results database
To get the latest test results snapshot, run 'npm ci' (first time only from repo root), then run 'GITHUB_TOKEN=$(gh auth token) node utils/test-results-db/cli.ts download'. The snapshot may be missing the newest runs. To top it up locally, run 'GITHUB_TOKEN=$(gh auth token) node utils/test-results-db/cli.ts update --lookback-days 3'.
Query Playwright test results database with DuckDB
Query the test results through the bundled @duckdb/node-api binding — no separate DuckDB install needed, it ships in node_modules after npm ci. Run: node --input-type=module -e 'import { DuckDBInstance } from "@duckdb/node-api"; const conn = await (await DuckDBInstance.create("utils/test-results-db/test-results.duckdb")).connect(); console.table((await conn.runAndReadAll(process.argv[1])).getRowObjectsJson());' "SELECT count(*) FROM test_results"
Integer columns in Playwright test results database come back as strings
Integer columns come back as strings from the DuckDB Node API (JSON-safe), so do ranking and filtering in SQL, not in JavaScript.
Example query for flaky tests across runs in Playwright CI
To find tests that are flaky across runs (where the final verdict flips between runs), use a query that groups by (project_name, file, test_title, run_id, run_attempt), computes the final status using arg_max(status, retry), then groups again by (project_name, test_title) to count failed_runs and passed_runs. Filter for expected = 'passed' to exclude intentional failures, then order by least(failed_runs, passed_runs) DESC to rank genuinely bimodal tests highest. Example: WITH per_run AS (SELECT project_name, file, test_title, run_id, run_attempt, arg_max(status, retry) AS final_status, any_value(expected_status) AS expected FROM test_results GROUP BY project_name, file, test_title, run_id, run_attempt) SELECT project_name, test_title, count(*) AS runs, count(*) FILTER (WHERE final_status IN ('failed','timedOut')) AS failed_runs, count(*) FILTER (WHERE final_status = 'passed') AS passed_runs, round(100.0 * count(*) FILTER (WHERE final_status IN ('failed','timedOut')) / count(*), 1) AS fail_pct FROM per_run WHERE expected = 'passed' GROUP BY project_name, test_title HAVING failed_runs > 0 AND passed_runs > 0 AND runs >= 10 ORDER BY least(failed_runs, passed_runs) DESC, failed_runs DESC LIMIT 20;
Example query for filtering by tag in Playwright test results
To filter tests by tag (tags is a list, not a string), use list_contains function. Example: SELECT project_name, test_title, count(*) AS runs FROM test_results WHERE list_contains(tags, '@slow') GROUP BY project_name, test_title ORDER BY runs DESC LIMIT 20;
Generate linked emoji run history for Playwright test results
To generate a compact emoji run history for a specific test (e.g. for a GitHub comment), use a query that groups by (run_id, run_attempt) and picks the final status with arg_max(status, retry). Map each final verdict to an emoji: 🟧 (orange) for a rescued failure (final_status='passed' with earlier 'failed'/'timedOut' attempts), 🟩 (green) for passed, 🟥 (red) for failed/timedOut. Link each emoji to the run/attempt URL: https://github.com/<repository>/actions/runs/<run_id>/attempts/<run_attempt>. Each square is one workflow run attempt, oldest first.
Fetch full test details from Playwright CI blob reports
The test results database stores per-result summaries only. For the full step tree, attachments, and stdio, fetch the original blob report for that run if the run uploaded one. A row identifies it by run_id + bot_name: the run's blob artifact is named 'blob-report-<bot_name>'. List the run's blob artifacts with 'gh api /repos/microsoft/playwright/actions/runs/<run_id>/artifacts --jq '.artifacts[] | select(.name | startswith("blob-report")) | {id, name}'', then download it with 'gh api /repos/microsoft/playwright/actions/artifacts/<artifact_id>/zip > blob.zip'. Blob and parquet artifacts have 7-day retention, so this works only for recent runs; the database itself retains summaries longer (until run-count eviction).
Issue triage goal: verify status, not fix
The goal of triaging a Playwright bug report is to reach a clear, verified status through reproduction, not to jump to a fix. For bugs, the status should be: reproduced, fixed-on-latest, cannot-reproduce, or not-a-bug. For feature requests or upstream/environment issues, provide a short verdict (already-possible, valid request, or upstream — owned by X) with supporting evidence.
Classify issues by content, not by label
Judge GitHub issues by their actual content rather than their assigned label. A '[Feature]' label often masks a bug (something that should already work), and a '[Bug]' label sometimes describes expected behaviour. Determine the true nature: Bug (reproduce it), Feature request (check if it already exists, verify cited sources, surface the design question), Upstream/environment (find the real external owner, verify any cited upstream issue, point at the real fix path), or Question/usage (answer it or point at documentation).
Playwright family repositories are not upstream
The Playwright family — @playwright/mcp (source in packages/playwright-core/src/tools/mcp/), playwright-vscode, playwright-python, playwright-java, playwright-dotnet — are all part of Playwright, not external. When triaging issues targeting these repositories, check out that repo and reproduce there in its own language/toolchain when needed. Never tell a reporter the issue belongs in a different Playwright repo or should be refiled there; that is an internal routing detail, not the reporter's problem.
Reproducing bugs: be exhaustive before giving up
When reproducing a bug, be thorough before reporting that it cannot be reproduced. Play around with things the reporter might have forgotten to mention: all three browsers (chromium, firefox, webkit), headed/headless modes, a few recent versions, and variations of the snippet or trigger. When you give up, report 'cannot reproduce' only after genuinely exploring, and say what you tried.
Watch for browser-specific divergence in bug reproduction
When reproducing bugs, run across all three browsers and watch for divergence — a bug that only reproduces in webkit, or everywhere except firefox, is a strong signal worth leading with. Plenty of bugs are browser-agnostic though, and those are just as real: reproducing on every browser is a good result to report, not a non-finding.
Bug reproduction process: read thread, pull inputs, reproduce on tip-of-tree
When reproducing a bug, follow these steps: (1) Read the whole thread including comments — the missing repro or narrowed trigger is often there. (2) Pull the inputs: version, browser(s), OS, repro repo/snippet, Expected-vs-Actual oracle. If something is missing, guess and try anyway; note assumptions in the report. (3) Reproduce on tip-of-tree first in ~/tmp/issue-<number>/: clone the linked repo or scaffold 'npm install @playwright/test@next' with a single-project config. Use PLAYWRIGHT_HTML_OPEN=never. If it reproduces on ToT, it is a live bug — record the exact version/sha tested, and if it looks like a regression, bisect it.