Testing strategy

bash

make test            # unit and integration tests on the CPU (not the service or slow tests)
make test-service    # end-to-end: a real server with worker and session processes
make lint            # ruff
GYRE_DEVICE=cpu .venv/bin/python -m pytest tests/test_operator.py -q -k entropy   # one test
make verify          # the verification studies (slower; see Verification)
make ci              # all of the above plus the documentation build and a deployment

Tests answer “is it still right?” and gate every change; verification studies answer “how right is it?” and are published with their provenance. Most test time is JIT compilation, not arithmetic: the cost column below is what to expect per file.

Layer	File(s)	What it proves	Cost
operators	`test_basis.py`, `test_physics.py`	quadrature, SBP, flux consistency/symmetry, entropy conservation condition, safe derivatives	seconds
mesh and geometry	`test_mesh.py`	connectivity in 2-D/3-D, neighbour traces coincide, GCL, differentiable metrics	seconds
mesh files	`test_meshio.py`	Gmsh node ordering on genuine Gmsh files, MSH 2.2 against 4.1, periodic node pairs, curved boundaries, round trips through `.msh` and `.npz`, re-oriented hexahedra through a file, refusal of unusable, truncated and malformed files, size checks of NPZ archives, a case on a mesh file	seconds to a minute
spatial operator	`test_operator.py`	free-stream preservation, conservation, entropy balance, split-argument blocks versus dense Jacobian, dot-product identity	~2 min
line smoother	`test_lines.py`	lines follow the thin direction, none on an isotropic mesh, loops across a periodic direction cut once, the colouring isolates every line neighbour; the line solve against a dense solve of the block-tridiagonal part of an AD Jacobian, forward and transposed, in 2-D, 3-D and with the turbulence model; forward and transposed multigrid solves with line smoothing	~3 min
single-precision blocks	`test_precision.py`	the single-precision blocks are the rounded double-precision ones and are inverted in double precision; forward and transposed multigrid solves with them, with block and with line smoothing, satisfy the double-precision system; the option reaches the hierarchy	~2 min
Krylov solver	`test_linear.py`	FGMRES against direct solves, flexibility, restarts	seconds
discipline	`test_discipline.py`	nonlinear convergence, loads sum to forces, adjoint/direct totals versus finite differences, transposed solves, failed-solve state retention and residual reference, refusal of derivatives after a failed solve, error-estimate force normalisation, periodic mesh deformation	minutes (compile-bound)
order selection	`test_adapt.py`	prolongation exact for polynomials, argument checks before any solve, multigrid levels following the degree, retries of a failed degree, the degree raised until the estimate meets the tolerance	minutes (compile-bound)
integration	`test_integrations.py`	coupled aero-structural adjoint; OpenMDAO adapter totals; OpenMDAO adapter after a failed point	minutes
command line	`test_cli.py`	overrides, `--target-error` and its exit statuses, `gyre mesh`, the `gyre verify` summary line, generated references	seconds
pipelines	`test_pipeline.py`	DAG validation, reference resolution, caching, gates	< 1 s
service	`test_service.py`	real server on a free port: jobs, SSE, cache, cancellation, sessions, MCP workflow, error shapes, non-loopback reachability	~1–2 min
turbulence closure	`test_turbulence.py`	closure functions and their smoothness, wall distance to curved walls, free stream and conservation with the model, exact linearisation on both model branches in 2-D and 3-D, row-scaled Krylov solves, a turbulent plate from the free stream, adjoint against finite differences, diagnostics and field output of a turbulent state	minutes
three dimensions	`test_threed.py`	cubed-sphere meshes and their face orientations, 3-D force and moment functionals, sideslip, wind axes, 3-D adjoint products	minutes
sessions	`test_sessions.py`	checkpoints, restore and rebuild of a session, the remote example script (which contains no optimizer)	minutes
documentation	`test_docsite.py`	builder: levels, shortcodes, links, search index, expanded Markdown for agents, build-time mathematics	seconds
documentation content	`test_docs_content.py`	the pages against the code: solver failure reasons, pipeline stage inputs, every output defined, example scripts referenced, the executed Python sample, the effort page against the ledger, no host-specific data	seconds
tooling	`test_tooling.py`, `test_worklog.py`	deployment manager, smoke test, bootstrap and lock scripts, CI definitions, the effort ledger tool	seconds
verification studies	`test_verify.py`	the suite runner, record format, study helpers with stubbed solves (a wrong Jacobian, a stagnated residual and NaN must fail their checks)	seconds

Invariants first. Round-off-level identities are the cheapest and sharpest tests, so they form the base of the pyramid.
Share expensive fixtures. JIT compilation dominates, so a converged aerofoil and a converged cylinder are module-scoped fixtures reused by a dozen assertions each.
Real processes for the service. Service tests start the actual server and let it spawn worker and session processes; nothing is mocked. A tiny case keeps this affordable.
Negative tests. Invalid meshes, unknown inputs, bad overrides and forbidden artefact names are asserted to fail with the documented error.
External formats are tested against the external tool. The .msh fixtures under tests/data/meshes are genuine Gmsh output, generated by tools/make_mesh_fixtures.py: the node-ordering tables are tested against Gmsh itself, not against Gyre’s own writer. Gmsh is not a dependency of Gyre; only that script needs it, and the fixtures are kept in the repository. Files Gmsh would not write (truncated, with lost section lines, with counts that do not match) are derived from the fixtures inside the tests.
Failure paths without long solves. Tests of a solve that does not converge limit the solver to one step on an existing fixture, or replace the solve, instead of constructing a flow that diverges.
CPU by default. Tests set GYRE_DEVICE=cpu for reproducibility; the GPU path is exercised by the verification suite.

Tests gate every commit and answer “is it still right?”. Verification studies are slower, produce charts and tables with provenance, answer “how right is it?”, and are published in the Verification section. The CI pipeline runs both and refuses to deploy if either fails.