Dynamic resource allocation for efficient parallel CFD simulations

• Published in: Computers & fluids 2022-09, Vol.245, p.105577, Article 105577
• Main Authors: Houzeaux, G., Badia, R.M., Borrell, R., Dosimont, D., Ejarque, J., Garcia-Gasulla, M., López, V.
• Format: Article
• Language: English
• Subjects: CFD; Elastic computing; High performance computing; MPI; Parallel efficiency
• ISSN: 0045-7930; 1879-0747
• DOI: 10.1016/j.compfluid.2022.105577

CFD users of supercomputers usually resort to rule-of-thumb methods to select the number of subdomains (partitions) when relying on MPI-based parallelization. One common approach is to set a minimum number of elements or cells per subdomain, under which the parallel efficiency of the code is “known” to fall below a subjective level, say 80%. The situation is even worse when the user is not aware of the best practice for a given code and a huge amount of resources can thus be wasted. This work presents an elastic computing methodology that adapts at runtime the resources allocated to a simulation automatically. The criterion to control the required resources is based on a runtime measure of the communication efficiency of the execution. According to some analytical estimates, the resources are then expanded or reduced to fulfill this criterion and eventually execute an efficient simulation. •A speedups campaign may give a crude approximation to parallel efficiency.•Ideally, resources should adapt automatically: elastic computing can adapt resources at runtime, by adding or removing cores.•By measuring real parallel efficiency at runtime, elastic computing is used to obtain a target efficiency for CFD simulations•Target parallel efficiency is reached automatically, inside the same SLURM job.