Gyredocs
Depth Practice
API

Gyre/User Guide

Troubleshooting

In one minuteWhat to do when a solve does not converge — every reason the solver can report, what it means and which control to change — and the first-run and service symptoms most often met.

Written at L1 Essentials goes to L3 MechanicsCFD practitionersMDO integratorsoperators 1515 words Markdown

A solve that does not converge is reported, not raised: the result carries converged: false and a reason. gyre run exits with status 2; its printed summary shows converged: false but not the reason, which is in result.json (run with -o DIR). A job result, a session evaluation and SolveHistory.reason in Python carry the same text.

The reasons a solve can end with#

reason Converged? Meaning
residual tolerance reached yes the residual fell below tol_rel × the reference residual
residual stagnated at round-off floor yes the residual stopped halving, at or below 10−810^{-8} of the reference residual
maximum number of steps reached no max_steps nonlinear steps were used
residual stalled above tolerance no the residual stopped halving above 10−810^{-8} of the reference residual
residual is not finite no the residual of the initial state is nan or inf
pseudo-time step collapsed (no admissible update) no no step was accepted even at a pseudo-time CFL below 10−610^{-6}

These six strings are the complete list (gyre.implicit.SteadySolver.solve).

The controls are the fields of solver and discretization in the case; on the command line --set solver.max_steps=400. The remedies are listed in the order to try them. They follow from how the iteration works (see the block below); they are not the outcome of a study of failures.

maximum number of steps reached. Look at the residual history first (gyre run prints it; result.json and the job events hold it).

  • The residual is still falling and the CFL still growing: the solve needed more steps. Raise solver.max_steps (default 150).
  • The CFL stays small or keeps being cut: the start is too far from the solution. Start from a converged neighbour instead (a session or flow.solve() after set_inputs warm-starts from the last converged state), approach the target condition in smaller increments, or lower solver.cfl0 (default 5).
  • The Krylov iteration count per step sits at its limit (krylov_dim × restarts, default 40 × 2) and the residual falls slowly: the linear systems are not being solved. Raise solver.krylov_dim, or use solver.smoother: gmres; keep solver.levels: auto. On a wall-resolved mesh (cells much longer than thick at the wall) use solver.smoother: line, which solves exactly along the lines of thin cells (measured on the turbulent flat plate).

residual stalled above tolerance. The residual did not halve in six consecutive steps although the CFL had reached 10410^4. Usual causes: an unsteady or marginally stable flow that has no steady state at this resolution (a wake that wants to shed, a shock that moves between nodes), or linear solves that do not reach lin_tol.

  • Check the last entries of the history: linear_reduction well above lin_tol means the linear solver is the limit — raise solver.krylov_dim or solver.restarts.
  • A flow with a shock needs discretization.shock_capturing: true; a larger discretization.av_coef (default 1) smooths the shock further at the price of accuracy.
  • A coarser tolerance is a decision about the result, not a fix: with tol_rel above the plateau the solve “converges” to the same state. Compare the plateau with the accuracy you need before doing that.

residual is not finite. This reason describes the starting point, not the iteration: the residual of the initial state — the uniform free stream, or the warm-start state evaluated with the new inputs — is nan or inf. The line search never accepts an update with a non-finite residual, so the reason cannot appear after the first step, and no solver control (cfl0, cfl_growth, max_steps) changes it.

  • Check the inputs: the Mach number, gamma, and the boundary data in flow.extra must give positive density and pressure.
  • Check the mesh: gyre mesh info reports inverted or badly scaled elements (messages and remedies).
  • Check the start state: a state restored from a checkpoint or set by hand, or the iterate of a failed solve passed on as a start, may itself be inadmissible.

pseudo-time step collapsed (no admissible update). Every update, at every step length, either produced negative density or pressure, gave a non-finite residual, or failed to reduce the residual, and cutting the pseudo-time step did not help. This is how a transient that leaves the admissible range ends (the scheme does not enforce positivity); if the step limit is reached first, the same cause is reported as maximum number of steps reached.

  • Start from a converged neighbouring condition, or lower solver.cfl0 and solver.cfl_growth (default 10) so that the transient is followed more closely.
  • Under-resolved strong gradients: enable discretization.shock_capturing, lower the polynomial degree (discretization.order), or refine the mesh where the gradient is.
  • In a design loop it usually means the last step in the design variables was too large — the kept state is still the last converged one, so take a smaller step.

Converged, but residual stagnated at round-off floor. Not a failure: the residual reached the conditioning-limited floor before tol_rel. The result is as converged as double precision allows on this mesh; a smaller tol_rel cannot be met.

  • The outputs in the result are those of the iterate the failed attempt ended at; they are not a solution.
  • The discipline keeps the last converged state and warm-starts the next solve from it. Repeating the same request repeats the same failure.
  • Gradients and error estimates are not computed for a failed point: the result record (result.json, a job result) and the summary printed by gyre run list them under skipped, next to the reason the solve ended with; sessions answer 422, and the Python calls raise RuntimeError (Python API, MDO integration).
  • Order selection (--target-error) ends at the first degree that does not converge and reports the last degree that did.

Each step solves a linearised pseudo-time problem to the relative tolerance lin_tol and tries the update with step lengths 1, ½, …, down to 1/32. A step length is accepted if the new state has positive density and pressure, a finite residual, and a smaller pseudo-transient residual. After an accepted full step the CFL grows by the factor by which the residual fell, limited to between 1.5 and cfl_growth; after a shortened step it stays (a step shortened to a quarter or less multiplies it by solver.cfl_shrink, which is 1 by default); if no step length is accepted the CFL is divided by ten and the step is repeated. Stagnation is declared after six steps without halving the best residual once the CFL is at least 10410^4. The reference residual is that of the uniform free stream, also for warm starts. See the implicit solver.

First run and installation#

Symptom Cause Remedy
import jax aborts with “illegal instruction” the CPU has no AVX2 and a newer JAX wheel was installed make bootstrap detects this and pins jax < 0.12 (constraints/avx-only.txt); with pip, install with -c constraints/avx-only.txt
gyre: command not found the virtual environment is not activated call .venv/bin/gyre, or source .venv/bin/activate once per shell
nothing is printed for tens of seconds after the case … line kernels are being compiled; the CLI prints nothing while JAX compiles wait: see the measured times in Getting started. Later runs of the same mesh size and degree reuse the cache in ~/.cache/gyre/jit
a small case is slower on the GPU than on the CPU compilation dominates and is slower for the GPU GYRE_DEVICE=cpu (set before the process starts; the choice is per process)
a 3-D solve exhausts device memory while the preconditioner is assembled the element blocks of the smoother grow like nvar2(p+1)6n_\text{var}^2 (p+1)^{6} per element set solver.precond_precision: single (half the memory, same answers); lower the degree or the number of elements
a derivative call exhausts memory forward mode over many inputs (jax.jacfwd with respect to all mesh nodes) builds one tangent per input use reverse mode (jax.grad, flow.totals, adjoint_product) for one output and many inputs (Python API)
stale or corrupt compiled kernels are suspected the persistent cache GYRE_JIT_CACHE=off disables it for a process; GYRE_JIT_CACHE=<dir> uses another directory

Service symptoms#

Answer Meaning What to do
429 QueueFull the job queue is at its limit wait for jobs to finish and submit again, or raise GYRE_MAX_QUEUED_JOBS
429 TooManySessions the session limit is reached close sessions that are no longer used (DELETE /api/v1/sessions/{id}); the operator can raise GYRE_MAX_SESSIONS
504 Timeout a session command, or the start-up of a session, exceeded its time limit check the size of the case with POST /api/v1/cases/validate; the first evaluation of a new size compiles kernels
410 SessionDead the solver process of the session has ended and its state with it open a new session
409 NotReady the job has not finished, or did not succeed poll GET /api/v1/jobs/{id}; read error in the job record if its state is failed
422 with type: RuntimeError on a session command derivatives or an error estimate were requested after an evaluation that did not converge evaluate a point that converges first

The complete list of error types is in the service guide; the limits and their environment variables are in the deployment guide.

    ↑ ↓ moveEnter open[ ] change depthResults show section and depth level