These documents contains guidelines for contributors to the WebGPU CTS (Conformance Test Suite) on how to write effective tests, and on the testing philosophy to adopt.
The WebGPU CTS is arguably more important than the WebGPU specification itself, because it is what forces implementation to be interoperable by checking they conform to the specification. However writing a CTS is hard and requires a lot of effort to reach good coverage.
More than a collection of tests like regular end2end and unit tests for software artifacts, a CTS needs to be exhaustive. Contrast for example the WebGL2 CTS with the ANGLE end2end tests: they cover the same functionality (WebGL 2 / OpenGL ES 3) but are structured very differently:
Below are guidelines based on our collective experience with graphics API CTSes like WebGL’s. They are expected to evolve over time and have exceptions, but should give a general idea of what to do.
Testing tasks are tracked in the CTS project tracker. Go here if you’re looking for tasks, or if you have a test idea that isn’t already covered.
If contributing conformance tests, the directory you’ll work in is
This directory is organized according to the goal of the test (API validation behavior vs
actual results) and its target (API entry points and spec areas, e.g. texture sampling).
The contents of a test file (
src/webgpu/**/*.spec.ts) are twofold:
Please read the following short documents before contributing.
This section provides an example of the planning process for a test. It has not been refined into a set of final test plan descriptions. (Note: this predates the actual implementation of these tests, so doesn’t match the actual tests.)
Somewhere under the
api/operation node are tests checking that running
the device using the
GPURenderEncoderBase.draw family of functions works correctly. Render
pipelines are composed of several stages that are mostly independent so they can be split in
several parts such as
Vertex input itself has several parts that are mostly separate in hardware:
GPUVertexFormat to the datatype for the input variable
in the shader
Each of these are tested separately and have cases for each combination of the variables that may
affect them. This means that
api/operation/render/vertex_input/id_generation checks that the
correct operation is performed for the cartesian product of all the following dimensions:
GPURenderPassEncoder or a
offset argument to
“Various values” above mean several small values, including
0 and the second smallest valid
value to check for corner cases, as well as some large value.
An instance of the test sets up a
draw* call based on the parameters, using point rendering and
a fragment shader that outputs to a storage buffer. After the draw the test checks the content of
the storage buffer to make sure all expected vertex shader invocation, and only these ones have