testing-philosophy

29 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

Test user-visible behavior, not implementation details

Automated tests should verify that application code works for end users and avoid relying on implementation details such as function names, array types, or CSS classes. Tests should only see and interact with rendered output that users see or interact with.

Make tests isolated and independent

Each test should run independently with its own local storage, session storage, data, and cookies. Test isolation improves reproducibility, makes debugging easier, and prevents cascading test failures. Tests should not rely on other tests running first.

Avoid testing third-party dependencies

Only test what you control. Do not test links to external sites or third-party servers you do not control. Testing third-party content is time consuming, slows down tests, and you cannot control the content or page structure (cookie banners, overlays, etc.) that might cause test failures.

Mock third-party API responses with page.route()

Instead of testing third-party dependencies, use the Playwright Network API to intercept and mock responses. Use page.route() to handle requests to external APIs and return controlled test data.

Example of mocking third-party API response

await page.route('**/api/fetch_data_third_party_dependency', route => route.fulfill({ status: 200, body: testData, })); await page.goto('https://example.com');

Control data when testing with databases

When working with a database, make sure you control the data. Test against a staging environment that does not change. For visual regression tests, ensure the operating system and browser versions are the same.

Page hydration and Playwright timing

Playwright operates as a very fast user and will start interacting with the page the moment it sees an element. If a page has poor hydration where interactive controls appear enabled but listeners have not yet been added, Playwright will perform actions that may have no effect. The correct fix is to ensure all interactive controls are disabled until after the hydration, when the page is fully functional. This can be verified by opening Chrome DevTools, setting network emulation to 'Slow 3G', reloading the page, and attempting to interact with elements.

Mock browser APIs with addInitScript before page load

Use page.addInitScript() to set up mocks before the page starts loading, since the page may call APIs very early during navigation. This is the easiest way to mock browser APIs.

Mock battery API example with addInitScript

Example of mocking the battery API using page.addInitScript() to override window.navigator.getBattery with a mock object containing level, charging, chargingTime, dischargingTime, and addEventListener properties: ```js await page.addInitScript(() => { const mockBattery = { level: 0.75, charging: true, chargingTime: 1800, dischargingTime: Infinity, addEventListener: () => { } }; window.navigator.getBattery = async () => mockBattery; }); ```

Override read-only navigator properties with Object.defineProperty

For read-only navigator properties that are configurable, use Object.defineProperty() to override them in addInitScript. For example, direct assignment like navigator.cookieEnabled = true has no effect, but Object.defineProperty can override it: ```js await page.addInitScript(() => { Object.defineProperty(Object.getPrototypeOf(navigator), 'cookieEnabled', { value: false }); }); ```

Verify API calls with page.exposeFunction

Use page.exposeFunction() to pass messages from the page back to test code, allowing you to record and verify which API methods were called. The exposed function can be called from page code and will execute in the Node.js test context.

Verify battery API calls example with exposeFunction

Example showing how to verify API calls by exposing a logCall function to the page, recording invocations of mocked APIs, and comparing against expected calls: ```js test('log battery calls', async ({ page }) => { const log = []; await page.exposeFunction('logCall', msg => log.push(msg)); await page.addInitScript(() => { const mockBattery = { level: 0.75, charging: true, chargingTime: 1800, dischargingTime: Infinity, addEventListener: (name, cb) => logCall(`addEventListener:${name}`) }; window.navigator.getBattery = async () => { logCall('getBattery'); return mockBattery; }; }); await page.goto('/'); await expect(page.locator('.battery-percentage')).toHaveText('75%'); expect(log).toEqual([ 'getBattery', 'addEventListener:chargingchange', 'addEventListener:levelchange' ]); }); ```

Make mock APIs fire events like browser implementations

To test that an app correctly reflects API status updates, the mock object should fire the same events that the browser implementation would. Store the mock object on window so test code can access and update it via page.evaluate().

Stateful mock battery API example that fires events

Example of a mock battery API that maintains event listeners and fires events when state changes: ```js test('update battery status (no golden)', async ({ page }) => { await page.addInitScript(() => { class BatteryMock { level = 0.10; charging = false; chargingTime = 1800; dischargingTime = Infinity; _chargingListeners = []; _levelListeners = []; addEventListener(eventName, listener) { if (eventName === 'chargingchange') this._chargingListeners.push(listener); if (eventName === 'levelchange') this._levelListeners.push(listener); } _setLevel(value) { this.level = value; this._levelListeners.forEach(cb => cb()); } _setCharging(value) { this.charging = value; this._chargingListeners.forEach(cb => cb()); } } const mockBattery = new BatteryMock(); window.navigator.getBattery = async () => mockBattery; window.mockBattery = mockBattery; }); await page.goto('/'); await expect(page.locator('.battery-percentage')).toHaveText('10%'); await page.evaluate(() => window.mockBattery._setLevel(0.275)); await expect(page.locator('.battery-percentage')).toHaveText('27.5%'); await expect(page.locator('.battery-status')).toHaveText('Battery'); await page.evaluate(() => window.mockBattery._setCharging(true)); await expect(page.locator('.battery-status')).toHaveText('Adapter'); await expect(page.locator('.battery-fully')).toHaveText('00:30'); }); ```

Update mock state in tests with page.evaluate

Use page.evaluate() to access and modify mock objects stored on the window object during a test. This allows you to simulate API state changes and verify that the application responds correctly.

Page object models: definition and purpose

Page object models are an approach to structure test suites for large applications. A page object represents a part of a web application, such as a home page, listings page, or checkout page. Page objects simplify authoring by creating a higher-level API suited to the application and simplify maintenance by capturing element selectors in one place and creating reusable code to avoid repetition.

Page object model implementation: TypeScript/JavaScript class structure

In TypeScript/JavaScript, a page object is implemented as a class that wraps a Page object. The class constructor accepts a Page instance and defines Locator properties for page elements. Methods on the class encapsulate user interactions and assertions. Example: a PlaywrightDevPage class with getStartedLink, gettingStartedHeader, pomLink, and tocList Locator properties, and methods like goto(), getStarted(), and pageObjectModel() that perform navigation and verification actions.

Page object model: combining user actions and assertions

Page object methods can combine multiple user actions and assertions. For example, a getStarted() method might click a link and then assert that an expected header is visible using expect(this.gettingStartedHeader).toBeVisible().

Page object model: TypeScript type imports

When implementing page objects in TypeScript, import Locator and Page types from @playwright/test: import { expect, type Locator, type Page } from '@playwright/test';

Parameterize tests with forEach loop

You can parameterize tests on a test level by using forEach on an array of data objects. Each iteration creates a test with a unique name based on the parameters. The test name must be unique across all iterations.

Generate tests from CSV file

Playwright Test runs in Node.js, allowing you to read files from the file system and parse them to generate tests dynamically. Use libraries like csv-parse to read CSV files and create tests based on the data.

CSV to tests generation example

Example of generating tests from CSV using csv-parse: import fs from 'fs'; import path from 'path'; import { test } from '@playwright/test'; import { parse } from 'csv-parse/sync'; const records = parse(fs.readFileSync(path.join(__dirname, 'input.csv')), { columns: true, skip_empty_lines: true }); for (const record of records) { test(`foo: ${record.test_case}`, async ({ page }) => { console.log(record.test_case, record.some_value, record.some_other_value); }); }

Best practice: isolate tests for efficient retry

It is usually better to make tests isolated, so they can be efficiently run and retried independently, rather than grouping them with serial mode.

Component tests in React and Vue

Playwright Test can test React and Vue.js components using real browsers with all Playwright Test features like parallelization and emulation.

test.step API for splitting long tests

The `test.step()` API (introduced in 1.14) allows splitting long tests into multiple steps. Step information is exposed in the reporters API.

test.step example usage

Example of using test.step: await test.step('Log in', async () => { /* ... */ }); await test.step('news feed', async () => { /* ... */ });

Playwright Test new in 1.12

Playwright Test (introduced in 1.12) is a new test runner built specifically for end-to-end testing. It supports running tests across all browsers, executing tests in parallel, context isolation with sensible defaults, and capturing videos, screenshots, and other artifacts on failure.

Basic Playwright Test example

Example of a basic Playwright Test: import { test, expect } from '@playwright/test'; test('basic test', async ({ page }) => { await page.goto('https://playwright.dev/'); const name = await page.innerText('.navbar__title'); expect(name).toBe('Playwright'); });

testInfo.titlePath property

The `testInfo.titlePath` property (introduced in 1.17) provides the title path of the test.

Give your agent this brain