ci

199 notes in this subject, read out of this brain and free to use. This is page 1 of 4.

Google Cloud Build: Playwright Docker image

Google Cloud Build configuration for Playwright: Use steps with name: mcr.microsoft.com/playwright:v%%VERSION%%-noble (JavaScript), set env CI=true. This runs Playwright tests in the Google Cloud Build environment using the official Docker image.

GitLab CI: Sharding with parallel:matrix

GitLab CI sharding using parallel:matrix with multiple variable combinations: stages: [test], image: mcr.microsoft.com/playwright:v%%VERSION%%-noble, parallel: matrix with PROJECT: ['chromium', 'webkit'] and SHARD: ['1/10', '2/10', ..., '10/10']. Script: npm ci, npx playwright test --project=$PROJECT --shard=$SHARD. This creates 2 projects × 10 shards = 20 parallel jobs. Available for JavaScript.

GitHub Actions: Run tests after deployment

GitHub Actions can run Playwright tests after a GitHub Deployment reaches the 'success' state using the deployment_status trigger. Example: on: deployment_status with condition if: github.event.deployment_status.state == 'success'. Set PLAYWRIGHT_TEST_BASE_URL environment variable using ${{ github.event.deployment_status.target_url }}. This pattern is used by services like Vercel to run end-to-end tests on deployed environments.

Drone CI: Playwright Docker configuration

Drone CI configuration for Playwright: Define kind: pipeline, name: default, type: docker. Add steps with name: test, image: mcr.microsoft.com/playwright:v%%VERSION%%-noble, commands: [npx playwright test] for JavaScript. This runs tests in the Playwright Docker container on Drone.

GitLab CI: Playwright Docker image setup

GitLab CI configuration uses Playwright Docker images as the test environment. For JavaScript: image: mcr.microsoft.com/playwright:v%%VERSION%%-noble. For Python: mcr.microsoft.com/playwright/python:v%%VERSION%%-noble. For Java: mcr.microsoft.com/playwright/java:v%%VERSION%%-noble. For C#: mcr.microsoft.com/playwright/dotnet:v%%VERSION%%-noble.

Azure Pipelines: Python Playwright setup

Azure Pipelines configuration for Python: Set trigger to main, pool to vmImage ubuntu-latest. Steps: UsePythonVersion@0 task with versionSpec 3.13, run pip install --upgrade pip and pip install -r requirements.txt, run playwright install --with-deps, run pytest.

GitLab CI: Sharding with parallel keyword

GitLab CI sharding using the parallel keyword: stages: [test], image: mcr.microsoft.com/playwright:v%%VERSION%%-noble, parallel: 7, script: npm ci, npx playwright test --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL. The test job is split into 7 jobs named sequentially from 'job_name 1/7' to 'job_name 7/7'. This is available for JavaScript.

Azure Pipelines: Basic Playwright setup for JavaScript

Azure Pipelines configuration for JavaScript: Set trigger to main, pool to vmImage ubuntu-latest. Steps: UseNode@1 task with version 22, run npm ci, run npx playwright install --with-deps, run npx playwright test with env CI=true. This makes the pipeline fail if any Playwright tests fail.

Parallel testing in CI with workers and sharding

Playwright supports two approaches to parallelize CI tests: (1) Increasing the workers configuration to run multiple test files in parallel within a single CI job (requires powerful CI system); (2) Sharding to distribute tests across multiple CI machines/jobs. Sharding is recommended for wider parallelization across CI infrastructure.

GitHub Actions: C# Playwright workflow

GitHub Actions workflow example for C#: Create .github/workflows/playwright.yml with on: push and on: pull_request triggers for main/master branches. The workflow uses actions/checkout@v6, actions/setup-dotnet@v5 with dotnet-version 8.0.x, runs dotnet build, pwsh bin/Debug/net8.0/playwright.ps1 install --with-deps, and dotnet test. timeout-minutes is 60 and runs-on is ubuntu-latest.

GitHub Actions: Java Playwright workflow

GitHub Actions workflow example for Java: Create .github/workflows/playwright.yml with on: push and on: pull_request triggers for main/master branches. The workflow uses actions/checkout@v6, actions/setup-java@v5 with distribution 'temurin' and java-version '25', runs mvn -B install -D skipTests --no-transfer-progress, mvn exec:java -e -D exec.mainClass=com.microsoft.playwright.CLI -D exec.args="install --with-deps", and mvn test. timeout-minutes is 60 and runs-on is ubuntu-latest.

Azure Pipelines: Publishing test results and reports

To integrate test results with Azure DevOps and upload reports, use PublishTestResults@2 task with searchFolder 'test-results', testResultsFormat 'JUnit', testResultsFiles 'e2e-junit-results.xml', mergeTestResults true, failTaskOnFailedTests true, and condition succeededOrFailed(). Also use PublishPipelineArtifact@1 to upload playwright-report. Configure JUnit reporter in playwright.config.ts: reporter: [['junit', { outputFile: 'test-results/e2e-junit-results.xml' }]].

Running headed tests on Linux with Xvfb in CI

On Linux CI agents, headed test execution requires Xvfb (X Virtual Framebuffer) to be installed. The official Playwright Docker image and GitHub Action have Xvfb pre-installed. To run browsers in headed mode with Xvfb, prefix the test command with xvfb-run, for example: xvfb-run npx playwright test (JavaScript), xvfb-run pytest (Python), xvfb-run mvn test (Java), xvfb-run dotnet test (C#).

Azure Pipelines: Java Playwright setup

Azure Pipelines configuration for Java: Set trigger to main, pool to vmImage ubuntu-latest. Steps: JavaToolInstaller@1 task with versionSpec 25 and jdkArchitectureOption x64, run mvn -B install -D skipTests --no-transfer-progress, run mvn exec:java -e -D exec.mainClass=com.microsoft.playwright.CLI -D exec.args="install --with-deps", run mvn test.

GitHub Actions: Fail-Fast with --only-changed flag

The --only-changed flag analyzes a test suite's dependency graph to run only test files likely to fail first, providing faster feedback and lower CI consumption on pull requests. Usage: npx playwright test --only-changed=origin/$GITHUB_BASE_REF. This is a heuristic that might miss tests, so always run the full test suite after the preliminary run. Requires fetch-depth: 0 in actions/checkout to reference $GITHUB_BASE_REF. Only available for JavaScript.

Caching browser binaries in CI is not recommended

Caching browser binaries is not recommended in CI because the time to restore the cache is comparable to the time to download the binaries. This is especially true on Linux, where operating system dependencies also need to be installed and are not cacheable. If caching is necessary, cache the browser binary directories against a hash of the Playwright version.

GitHub Actions: Python Playwright with tracing

GitHub Actions workflow example for Python: Create .github/workflows/playwright.yml with on: push and on: pull_request triggers for main/master branches. The workflow uses actions/checkout@v6, actions/setup-python@v6 with python-version 3.13, installs dependencies from requirements.txt, runs python -m playwright install --with-deps, executes pytest --tracing=retain-on-failure, and uploads test-results/ as playwright-traces artifact. timeout-minutes is 60 and runs-on is ubuntu-latest.

Debugging browser launch errors with DEBUG environment variable

Playwright supports the DEBUG environment variable to output debug logs during execution. Setting DEBUG=pw:browser is helpful when debugging 'Error: Failed to launch browser' errors. This works across all languages: DEBUG=pw:browser npx playwright test (JavaScript), DEBUG=pw:browser pytest (Python), DEBUG=pw:browser mvn test (Java), DEBUG=pw:browser dotnet test (C#).

Bitbucket Pipelines: Playwright Docker images

Bitbucket Pipelines uses the Playwright Docker image as the build environment. For JavaScript: image: mcr.microsoft.com/playwright:v%%VERSION%%-noble. For Python: mcr.microsoft.com/playwright/python:v%%VERSION%%-noble. For Java: mcr.microsoft.com/playwright/java:v%%VERSION%%-noble. For C#: mcr.microsoft.com/playwright/dotnet:v%%VERSION%%-noble.

CircleCI: Playwright Docker image configuration

CircleCI executors for Playwright use the official Docker images: pw-noble-development with docker image mcr.microsoft.com/playwright:v%%VERSION%%-noble for JavaScript, mcr.microsoft.com/playwright/python:v%%VERSION%%-noble for Python, mcr.microsoft.com/playwright/java:v%%VERSION%%-noble for Java, mcr.microsoft.com/playwright/dotnet:v%%VERSION%%-noble for C#. Note: The default 'medium' tier has 2 CPU cores. Overriding workers to greater than 2 causes unnecessary timeouts and failures.

Jenkins: Playwright Docker pipeline configuration

Jenkins Groovy pipeline for JavaScript: agent { docker { image 'mcr.microsoft.com/playwright:v%%VERSION%%-noble' } } with stage('e2e-tests') containing steps sh 'npm ci' and sh 'npx playwright test'. For Python: use mcr.microsoft.com/playwright/python:v%%VERSION%%-noble with sh 'pip install -r requirements.txt' and sh 'pytest'. For Java: use mcr.microsoft.com/playwright/java:v%%VERSION%%-noble with sh 'mvn -B install -D skipTests --no-transfer-progress' and sh 'mvn test'. For C#: use mcr.microsoft.com/playwright/dotnet:v%%VERSION%%-noble with sh 'dotnet build' and sh 'dotnet test'.

CircleCI: Sharding with zero-indexed parallelism

CircleCI sharding is indexed with 0, so add 1 to CIRCLE_NODE_INDEX when passing to --shard. Example for parallelism: 4, the shard command is: SHARD="$((${CIRCLE_NODE_INDEX}+1))"; npx playwright test --shard=${SHARD}/${CIRCLE_NODE_TOTAL}. This ensures shards are numbered 1-4 instead of 0-3.

Three steps to run Playwright tests in CI

To run Playwright tests in CI environments, follow these three steps: (1) Ensure CI agent can run browsers by using the official Docker image for Linux agents or installing dependencies using the CLI; (2) Install Playwright by running npm ci and npx playwright install --with-deps for JavaScript, or the equivalent for Python, Java, or C#; (3) Run tests using npx playwright test for JavaScript or the equivalent for other languages.

Azure Pipelines: Container-based Playwright

Azure Pipelines supports containerized jobs using the container keyword. For JavaScript: container: mcr.microsoft.com/playwright:v%%VERSION%%-noble. For Python: mcr.microsoft.com/playwright/python:v%%VERSION%%-noble. For Java: mcr.microsoft.com/playwright/java:v%%VERSION%%-noble. For C#: mcr.microsoft.com/playwright/dotnet:v%%VERSION%%-noble. This provides a consistent environment without needing to install dependencies on the host.

Azure Pipelines: Sharded tests with matrix strategy

Azure Pipelines sharding example for JavaScript: Use strategy.matrix to define project (chromium, firefox, webkit) and shard (1/3, 2/3, 3/3) variables. Run npx playwright test --project=$(project) --shard=$(shard) with env CI=true. This creates a job for each matrix combination.

GitHub Actions: Playwright tests on push/pull request

GitHub Actions workflow example for JavaScript: Create .github/workflows/playwright.yml with on: push and on: pull_request triggers for main/master branches. The workflow uses actions/checkout@v6, actions/setup-node@v6, runs npm ci, npx playwright install --with-deps, npx playwright test, and uploads playwright-report/ as an artifact with 30-day retention. timeout-minutes is set to 60 and runs-on is ubuntu-latest.

Workers setting for CI stability

In CI environments, it is recommended to set the workers configuration to 1 to prioritize stability and reproducibility. Setting workers to 1 ensures each test gets full system resources and avoids potential conflicts. This is configured in playwright.config.ts using: workers: process.env.CI ? 1 : undefined.

Azure Pipelines: C# Playwright setup

Azure Pipelines configuration for C#: Set trigger to main, pool to vmImage ubuntu-latest. Steps: UseDotNet@2 task with packageType sdk and version 8.0.x, run dotnet build --configuration Release, run pwsh bin/Release/net8.0/playwright.ps1 install --with-deps, run dotnet test --configuration Release.

GitHub Actions: Container-based Playwright for consistent environments

GitHub Actions supports running jobs in containers using the jobs.<job_id>.container option. For JavaScript, use container image mcr.microsoft.com/playwright:v%%VERSION%%-noble with options --user 1001. For Python, use mcr.microsoft.com/playwright/python:v%%VERSION%%-noble. For Java, use mcr.microsoft.com/playwright/java:v%%VERSION%%-noble. For C#, use mcr.microsoft.com/playwright/dotnet:v%%VERSION%%-noble. This approach is useful to avoid polluting the host environment and ensures consistent environments for screenshots and visual regression testing across operating systems.

Publishing HTML reports to Azure Storage static website

To publish Playwright HTML reports from GitHub Actions to a web-accessible location, create an Azure Storage account with static website hosting enabled, create a Service Principal with Storage Blob Data Contributor role, set up GitHub Actions secrets (AZCOPY_SPA_APPLICATION_ID, AZCOPY_SPA_CLIENT_SECRET, AZCOPY_TENANT_ID), and add a workflow step using azcopy to upload the playwright-report directory to the $web container. Access the reports via the public URL of the Azure website. This approach does not work for pull requests from forked repositories due to lack of access to secrets.

Handling sensitive data in CI artifacts

Artifacts like trace files, HTML reports, and console logs can contain sensitive data including test user credentials, access tokens, testing or application source code. Treat these files with the same care as sensitive data. Only upload to trusted artifact stores, encrypt files before upload, and use trusted file shares when sharing with team members.

Playwright tests run on any CI provider

Playwright tests can be executed on any CI provider, not limited to GitHub Actions. Detailed documentation on configuring various CI providers is available in the Continuous Integration guide.

Accessing trace files from HTML report in GitHub Actions

After serving the HTML report using 'npx playwright show-report', click on the trace icon next to a test's file name to view the trace and inspect each action to identify why tests are failing.

Downloading and viewing HTML report from GitHub Actions

HTML reports are uploaded to GitHub Actions artifacts. In the Artifacts section, click on 'playwright-report' to download the report as a zip file. To view the report locally, extract the zip and use the command 'npx playwright show-report name-of-my-extracted-playwright-report' to serve it with a web server, which enables viewing in a browser.

Viewing test logs in GitHub Actions

After clicking on a workflow run in the Actions tab, click on 'Run Playwright tests' to view detailed error messages, expected vs. actual results, and the call log for failed tests.

GitHub Actions workflow for Playwright JavaScript tests

A basic GitHub Actions workflow for Playwright JavaScript tests uses the file `.github/workflows/playwright.yml`. The workflow triggers on push and pull request to main/master branches. It performs these steps: (1) Clone repository using actions/checkout@v6, (2) Install Node.js LTS using actions/setup-node@v6, (3) Install dependencies with npm ci, (4) Install Playwright browsers with npx playwright install --with-deps, (5) Run tests with npx playwright test, (6) Upload HTML report to GitHub UI using actions/upload-artifact@v4 with name 'playwright-report', path 'playwright-report/', and retention-days set to 30. The job has a timeout of 60 minutes and runs on ubuntu-latest.

Playwright Docker base images Ubuntu versions

Playwright publishes Docker images based on: (1) Ubuntu 26.04 LTS (Resolute Raccoon) with image tags including 'resolute'. (2) Ubuntu 24.04 LTS (Noble Numbat) with image tags including 'noble'. (3) Ubuntu 22.04 LTS (Jammy Jellyfish) with image tags including 'jammy'.

Build custom Playwright Docker image with Node.js

To run Playwright inside Docker with Node.js, use: FROM node:20-bookworm\nRUN npx -y playwright@%%VERSION%% install --with-deps

Alpine Linux not supported for Playwright browsers

Browser builds for Firefox and WebKit are built for the glibc library. Alpine Linux and other distributions based on the musl standard library are not supported.

Build custom Playwright Docker image with Python

To run Playwright inside Docker with Python, use: FROM python:3.12-bookworm\nRUN pip install playwright==@%%VERSION%% && playwright install --with-deps

Playwright version must match between tests and Docker container

When running tests remotely, ensure the Playwright version in your tests matches the version running in the Docker container.

Docker network configuration for accessing local servers

To access local servers from within the Docker container, use: docker run --add-host=hostmachine:host-gateway -p 3000:3000 --rm --init -it --workdir /home/pwuser --user pwuser mcr.microsoft.com/playwright:v%%VERSION%%-noble /bin/sh -c "npx -y playwright@%%VERSION%% run-server --port 3000 --host 0.0.0.0". This makes 'hostmachine' point to the host's localhost; use 'hostmachine' instead of 'localhost' when accessing local servers.

Connect to remote Playwright server using BrowserType.connect API

To connect to a remote Playwright server using the API: const browser = await playwright['chromium'].connect('ws://127.0.0.1:3000/');

Connect to remote Playwright server using environment variable

To connect to a remote Playwright server using the environment variable with @playwright/test: PW_TEST_CONNECT_WS_ENDPOINT=ws://127.0.0.1:3000/ npx playwright test

Run Playwright Server in Docker for remote execution

Start the Playwright Server in Docker with: docker run -p 3000:3000 --rm --init -it --workdir /home/pwuser --user pwuser mcr.microsoft.com/playwright:v%%VERSION%%-noble /bin/sh -c "npx -y playwright@%%VERSION%% run-server --port 3000 --host 0.0.0.0"

Recommended Docker configuration for Playwright

When running Playwright in Docker: (1) Use the --init flag to avoid special treatment for processes with PID=1 and prevent zombie processes. (2) Use --ipc=host when using Chromium to prevent it from running out of memory and crashing. (3) If seeing weird errors when launching Chromium, try running the container with docker run --cap-add=SYS_ADMIN when developing locally.

Seccomp profile for Chromium sandbox in Docker

The seccomp_profile.json file is needed to run Chromium with sandbox in Docker. It is based on the default Docker seccomp profile with additional user namespace cloning permissions for the 'clone', 'setns', and 'unshare' system calls with action SCMP_ACT_ALLOW.

Web scraping Docker command with separate user and seccomp

For untrusted websites when crawling and scraping, use: docker run -it --rm --ipc=host --user pwuser --security-opt seccomp=seccomp_profile.json mcr.microsoft.com/playwright:v%%VERSION%%-noble /bin/bash

End-to-end tests Docker command with root user

For trusted websites and end-to-end tests, use: docker run -it --rm --ipc=host mcr.microsoft.com/playwright:v%%VERSION%%-noble /bin/bash

Root user disables Chromium sandbox in Docker

Running Docker image with the root user will disable the Chromium sandbox, which is not available with root permissions. Root user may be acceptable for trusted code like end-to-end tests, but for web scraping or crawling, create a separate user inside the Docker container and use the seccomp profile.

Docker image published to Microsoft Artifact Registry

The Playwright Docker image is published to Microsoft Artifact Registry at mcr.microsoft.com. Image tags are available for JavaScript, Python, .NET, and Java versions.

Playwright Docker image only for testing and development

The Playwright Docker image is intended for testing and development purposes only and is not recommended for visiting untrusted websites.

Playwright Docker image includes browsers and system dependencies

The Playwright Docker image (Dockerfile.noble) includes Playwright browsers and browser system dependencies. The Playwright package/dependency is not included in the image and must be installed separately.

Pin Docker image to specific Playwright version

It is recommended to always pin your Docker image to a specific version. If the Playwright version in your Docker image does not match the version in your project/tests, Playwright will be unable to locate browser executables.

Use noVNC viewer in Docker for codegen and test recording

In Docker and GitHub Codespaces environments, you can view and generate tests using the noVNC viewer built into the Docker image. Enable the desktop-lite feature and specify webPort in .devcontainer/devcontainer.json to make the VNC webviewer accessible outside the container. Example: {"image": "mcr.microsoft.com/playwright:v1.57.0", "forwardPorts": [6080], "features": {"desktop-lite": {"webPort": "6080"}}}. This enables recording tests, picking selectors, and using codegen directly on the container.

VS Code extension installation and setup

Install the Playwright VS Code extension from Microsoft by opening the Extensions view with Ctrl+Shift+X (or Cmd+Shift+X) and searching for 'Playwright'. After installation, open the Command Palette with Ctrl+Shift+P (or Cmd+Shift+P) and run the 'Test: Install Playwright' command. Select the browsers you want for your tests (Chromium, Firefox, WebKit) and optionally add a GitHub Actions workflow for CI. These settings can be changed later in the playwright.config.ts file.

Install system dependencies for browsers

Run `npx playwright install-deps` to automatically install system dependencies required for running browsers. You can target a specific browser: `npx playwright install-deps chromium`. Combine both commands with `npx playwright install --with-deps chromium` to install browsers and OS dependencies in one command.

Install browsers with CLI

Run `npx playwright install` to download and install default browsers. To install a specific browser, pass its name as an argument: `npx playwright install webkit`. Running `npx playwright install --help` shows all supported browsers.

Playwright version update cycle

Each version of Playwright requires specific versions of browser binaries. With every Playwright release, browser versions are updated. This means re-running the install CLI command is often necessary after updating Playwright to fetch new browser versions.

Projects configuration for multi-browser testing

Define projects in the Playwright config to run tests on multiple browsers and configurations. Each project specifies a name and use options that can include device settings and browser channel. All projects run by default when executing tests.

Give your agent this brain