Verification

Each study produces checks (a measured value, the expectation, pass or fail), tables and charts, and a provenance stamp: the source revision, the compute device and the time of the run. A page built from a different revision than the one deployed says so in its stamp. Nothing on these pages is typed in by hand.

gyre verify --suite quick -o var/verification      # all studies
gyre verify --only isentropic_vortex,mms_euler     # a subset
gyre docs build                                    # re-render these pages from the new results

The quick suite is not short: its last complete run on the development host (39 studies in three parallel lanes, CPU plus an RTX 3060 Ti) took 104 minutes, and the studies’ own wall times, shown on each study card, add up to 273 minutes. A subset such as --only free_stream,entropy_conservation takes about a minute. --suite full adds finer meshes and higher degrees. Each study runs in its own process on the device suited to its size; results land in var/verification/<study>.json. The timing studies run last, one at a time, and the GPU timing study is skipped on a host without a GPU.

index.json lists every result file in the directory, not only those of the last invocation. Each entry carries the level, commit, device and timestamp of the run that produced it, and stale: true if that was an earlier invocation. requested, skipped and summary describe the last invocation. gyre verify ends with the line P/R studies of this invocation passed and exits with status 1 only when a study of this invocation failed; failing results that an earlier invocation left in the directory are named separately (; stale failing results not re-run: ...) and do not change the exit status.

On these pages a status block or study card of a result that the last invocation did not re-run carries the marker “Earlier run”; its tooltip names the commit and time of the run that produced the result. Such a block shows the checks, thresholds, tables and notes of that run. The text of a page describes the studies as they are in the code now, so the two can differ until the study is run again with gyre verify --only <id>.

Order checks have one of three kinds of threshold, and each names its basis in its expected text:

• the design rate, for example “observed order at least $p + 1 - 0.2$”, where that rate is established;
• the guaranteed rate, “at least $p$” with a tolerance, where only consistency is guaranteed;
• an empirical regression floor between the two, where a rate above $p$ was measured but not derived: $p + 0.35$ for the manufactured Euler solution on coarse curved meshes and $p + 0.4$ for the velocity in the diffusion-dominated Navier–Stokes regimes. A floor of this kind guards against regressions; it is not a property of the scheme.

Bounds on timing ratios and on the ratio of actual to estimated error in the order-selection study are regression bounds of the same kind and say so. When a study fails, its badge on these pages turns red and lists the failing checks, the CI pipeline stops before deployment, and the study’s log is kept under var/verification/logs/. Failures during development were instructive:

• The vortex study at $p = 4$ measured order 4.05 against an expectation of at least 4.4. The first explanation — time-integration error — was tested by shrinking the time step five-fold and was refuted: the error did not change. The expectation was wrong, not the measurement; the study now checks the guaranteed rate, reports the observed one, and keeps the time-step test as a permanent check.
• The mesh-deformation derivative check compared against a finite difference of the whole domain area, a badly conditioned reference; it now uses a weighted area concentrated near the surface.
• A finite-difference step sweep started with a step of 0.1 rad, which pushed the flow into a regime with shocks; the failed solve then contaminated every later point. Perturbed solves now restart from the baseline state and must converge.
• The output-error study included a mesh so coarse that the bump was sampled at a single node; its “errors” were those of a different geometry.

In each case the suite did its job: it made a wrong assumption visible.

1. Prefer exact properties over convergence rates. Round-off-level identities (conservation, entropy conservation, symmetry, dot-product tests) cannot be passed by accident.
2. Test the negative. Fluxes that should not conserve entropy are checked to violate the condition; otherwise a broken test would pass everything.
3. Independent references. Derivatives are compared against finite differences and against the other differentiation mode; the error estimator is compared against an actually converged finer solution; block preconditioners against a dense Jacobian.
4. Manufactured forcing from the solver’s own physics. Source terms are generated by automatic differentiation of the flux kernels, eliminating transcription errors.
5. Separate regimes. Where behaviour differs by regime (diffusion- versus convection-dominated), the study reports them separately instead of averaging them away.