new·The score now tells you which way it movedA brain's exam only ever grows: its own material writes questions, and so does every question a real caller asked and did not get answered. The score is a percentage over that growing set, so a brain that learned more could post a smaller number — and this week three did. One of them answered two MORE questions than the week before and showed eighteen points less. Printed as a single percentage, that reads as decline to a reader and as punishment to anyone who contributes material.all news →
mozg.beta
Sign in

Vitest · Config reference · all subjects

benchmark options

7 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

BenchmarkProvider registration object structure

Each registration object in the registrations array contains: name (string identifier), fn (the benchmark function), and optional fnOpts (an object for lifecycle hooks, cancellation, async behavior, and sample retention).

Custom benchmark provider lifecycle hooks

The provider is responsible for running every benchmark function and its beforeAll, beforeEach, afterEach, and afterAll hooks according to the benchmarking engine's lifecycle. If execution fails, the provider must throw the error to fail the test.

Custom benchmark provider lifetime and caching

Vitest imports the provider module on first use and caches its default export for the lifetime of the worker. The API does not have separate setup or teardown hooks. Worker-scoped state can be kept on the provider object when needed.

BenchResult format requirements

Results returned from the provider.run() method must be an array of BenchResult objects, one for each registration, matched to registrations by name. The BenchResult shape must be compatible with Tinybench's result format, which is exported by vitest.

Benchmark provider error handling

If a benchmark result has state 'errored', the provider must throw the associated error. If a benchmark result has a state other than 'completed' (aside from 'errored'), the provider must throw an error indicating the unexpected state.

bench.from() registrations not passed to provider

Registrations created by bench.from() are loaded by Vitest and are not passed to the provider's run method.

Custom benchmark provider example with Tinybench

Example of a custom benchmark provider wrapping Tinybench: the provider receives registrations, creates a new Bench instance with config options like signal and retainSamples, adds each registration using bench.add(name, fn, fnOpts), runs the benchmarks with await bench.run(), maps the task results to BenchResult format by checking state and extracting the result object with the task name, and throws errors if any benchmark fails.

Give your agent this brain