TurtLES: A Large-Scale Benchmark for Turbulent 3D Neural PDE Surrogates
Machine-learning surrogates promise to make computational fluid dynamics faster by replacing repeated numerical simulations with learned predictions. The difficulty is that many existing benchmarks simplify the problem: they use two-dimensional or regular-grid flows, steady-state solutions, coarse meshes, or dynamics that are unsteady without containing the physical mechanisms of three-dimensional turbulence.
In TURTLES, Jarno Platenburg, Armand Kassaï Koupaï, Paola Cinnella, and Patrick Gallinari introduce a benchmark intended to expose those gaps. It contains 400 simulated trajectories of turbulent three-dimensional wakes behind objects with varied shapes, Reynolds numbers, and angles of attack. The authors position the dataset as a test of whether neural operators can remain accurate and stable when realistic turbulence, irregular meshes, long time horizons, and millions of spatial points appear together.
Why this benchmark is different
The dataset is generated with high-fidelity implicit large-eddy simulations of compressible Navier–Stokes flow. Each trajectory has 400 saved time steps, and each frame contains roughly 3–11 million irregularly distributed solution points. The stored variables are the three velocity components, pressure, and density.
Across the cases, the Reynolds number ranges from 1,000 to 3,900, the angle of attack from 0 to 45 degrees, and the obstacle’s half-length from 1/6 to 3/2 of the object thickness. Four super-ellipse families represent diamond, elliptical, rounded-rectangle, and rectangular shapes. A Sobol sequence is used to cover the four-dimensional parameter space, with 320 trajectories for training, 40 for validation, and 40 for testing.
The authors emphasize that the flow contains an active three-dimensional energy cascade driven by vortex stretching. That distinction matters: a visually complex or chaotic flow is not necessarily turbulent in the physical sense. TURTLES also provides velocity spectra from wake probes, allowing researchers to examine whether a model preserves structures across large, intermediate, and small scales instead of relying only on pointwise error.
How the evaluation works
The benchmark asks a model to predict the next state from one frame, conditioned on the Reynolds number. Models learn a temporal residual and then generate a full trajectory autoregressively, using their own previous prediction at each step. This setup directly tests error accumulation, rather than measuring only a one-step prediction.
The paper evaluates four neural-operator baselines: UPT, GINO, Transolver++, and a scaled implementation of CALM-PDE. All models use the same broad training budget: 100,000 optimization steps, an effective batch size of two, and approximately 20 hours on one NVIDIA H100 GPU. Because the complete meshes are too large for a single forward pass, training samples points and inference processes the full mesh in chunks.
The authors report four main classes of measurements: relative L2 error, variance-scaled RMSE, spectral error over low-, middle-, and high-frequency bands, and GPU time needed to produce a complete trajectory. This combination is important because a model can achieve a reasonable field-wise error while smoothing away fine-scale turbulence or becoming unstable during a long rollout.
What the results show
For next-frame prediction, CALM-PDE performs best among the reported baselines. Its relative L2 error is 6.80 × 10-2, and its variance-scaled RMSE is 7.26 × 10-2. UPT is close on relative L2 error at 7.01 × 10-2, while GINO and Transolver++ are weaker under the shared training setup. Their variance-scaled RMSE values are at or above the paper’s mean-field reference level.
Long-horizon behavior separates the models more clearly. CALM-PDE’s variance-scaled RMSE rises from 0.461 over the first 50 rollout steps to 0.789 over the final window. UPT reaches 5.12 in the final window, while Transolver++ reaches 31.24 and is described as unstable from the beginning. These results suggest that short-horizon accuracy is not a reliable proxy for whether a surrogate can track the evolving wake.
The spectral analysis reinforces that point. The authors report that CALM-PDE most closely follows the reference spectrum across the resolved range. UPT follows the overall spectral decay but has larger high-frequency errors. Transolver++ produces excessive spectral energy, particularly at intermediate and high frequencies. A persistence baseline can match some time-averaged spectral behavior simply by repeating a real turbulent frame, showing why spectral agreement must be combined with temporal and field-level metrics.
Speedups are meaningful but limited at this scale. The solver requires 13,160 GPU-seconds for one trajectory under the reported protocol. Surrogate costs range from 835 GPU-seconds for UPT to 4,097 for GINO, with CALM-PDE at 1,073. The authors report approximately 3× to 16× speedups, considerably less dramatic than results often claimed on smaller or simpler problems.
Limitations and practical implications
TURTLES is deliberately demanding, but it covers only one physical setting: turbulent cylinder wakes in a shear-layer transition regime. The authors caution that its failure modes may not transfer to other geometries or Reynolds-number ranges. The reference data come from implicit LES rather than DNS or direct experimental ground truth, so reported surrogate errors measure disagreement with the chosen solver.
Temporal subsampling also removes some of the highest-frequency information. The trajectories are too short to provide fully converged long-term statistics such as Reynolds stresses, limiting some physical analyses. Finally, the four baselines are an initial survey, not a definitive ranking of neural operators.
For researchers, the practical lesson is to evaluate neural PDE models on more than average field error. Long autoregressive rollouts, energy spectra, and full-mesh latency can reveal failures hidden by conventional metrics. For engineering teams, the benchmark is a useful stress test before trusting a surrogate for design exploration, but its reported speedups do not eliminate the cost of high-resolution inference. The full dataset is 14 TB, so the authors also provide standardized MINI-TURTLES subsets, including tiers down to approximately 0.8 TB and a 5 GB debugging subset.
TURTLES does not demonstrate that neural surrogates are ready to replace verified CFD in safety-critical workflows. It does demonstrate, according to its authors, that realistic three-dimensional turbulence and dense irregular meshes expose weaknesses that simpler benchmarks can conceal. That makes the dataset valuable not only for comparing models, but also for clarifying what a useful CFD surrogate must preserve: stable dynamics, multiscale energy, geometric variation, and enough computational efficiency to matter in practice.
Sources
AI-assisted research summary. Check the original paper for complete methods, results, and limitations.