Questions on Computational Efficiency

[guanxuefeng1996]

Hello everyone, I am currently trying to solve a relatively large problem using Exec/Production/JetFlame. I only modified the geometric dimensions and mesh distribution in the example case for debugging. The basic mesh distribution is 96×96×192. A localized refinement is applied in the central region, and the total number of mesh cells is around 70 million. The time-stepping method uses a fourth-order SDC method, with non-reactive flow and CFL=0.5. However, after testing with 1024 cores for one hour, it only advanced by one step. Could you please provide guidance in this regard? Are there any methods to accelerate the calculation? Thank you very much!

[jrood-nrel]

I have two main questions:

1. Did you build with MPI enabled?
2. Are you building in DEBUG mode?

Can you also show the line you're using to run the application?

Here is why I am asking this: If MPI is not enabled correctly, you could be running 1024 instances of the same simulation, rather than one simulation using 1024 cores. If you're building with DEBUG mode, the application is not using compiler optimizations, which significantly slows down the code.

[guanxuefeng1996]

Thank you very much for your reply, I think I used MPI and turned off DEBUG mode, and that's how it shows up inside the GNUmakefile:

# Performance
USE_MPI = TRUE
USE_OMP = FALSE
USE_CUDA = FALSE
USE_HIP = FALSE
USE_SYCL = FALSE

# Debugging
DEBUG = FALSE
FSANITIZER = FALSE
THREAD_SANITIZER = FALSE

I used this line of command to run the program:
mpirun -np 1024 PeleC3d.gnu.MPI.ex inputs.inp