Task List Orchestrator Refactor

[jfields7]

Currently the Z4c and DynGRMHD modules use the NumericalRelativity class to queue their tasks, sort them according to dependencies, and add them to the appropriate task lists in a way that doesn't require rewriting a completely separate task list for doing GRMHD+Z4c. The motivation behind this was that eventually we'll also couple features like neutrino radiation to the spacetime solver, and we didn't want to write different task lists for every possible combination.

The current logic is as follows:

1. Tasks for each physics module are queued with QueueTask rather than added directly to a task list with AddTask. Each task specifies required dependencies (e.g., MHD_RecvFlux will always depend on MHD_SendFlux) via a vector. Another vector provides optional dependencies, which are enabled if specific physics modules are available (e.g., Z4c_ExplRK has an optional dependency on MHD_EField which is enabled if MHD is also present to keep from updating the lapse and shift vectors too early).
2. While each task is queued, NumericalRelativity looks at which physics modules are enabled to determine if optional dependencies should be added to the dependency list. This check is done by doing an index check against each optional dependency's TaskName to determine which physics module it belongs to, then checking if that module is available.
3. After all tasks have been queued, AssembleNumericalRelativityTasks is called. It loops over each task queue and adds a task if its dependencies are satisfied. This loop is iterated either until all tasks have been added (success) or it completes a single loop without adding any new tasks (missing and/or circular dependencies).

There was some interest in generalizing this infrastructure so that it can be used for all the other physics modules, too, such as two-fluid, rad hydro, etc. This is a basic proposal from the PSU AthenaK developers' workshop for how we might go about doing this incrementally.

1. Rename from NumericalRelativity to TaskListOrchestrator

The current name isn't really reflective of what the class does, and we're generalizing it so that it will work for more than just NR physics modules. This would also rename AssembleNumericalRelativityTasks to AssembleAllTasks or something similar.

2. Update task queues to mirror current TaskList infrastructure

NumericalRelativity was written before the current task list infrastructure using std::map was completed. Some small modifications were made to make it work after these updates, but there is no support for the before_timeintegrator or after_timeintegrator TaskList objects. We should add support for these. It would also be nice to have a map of TaskQueue objects that mirrors the map used for TaskList rather than the current logic of checking an enumerator explicitly.

3. Move optional dependency checks to right before tasks are assembled

Right now we explicitly check if physics modules are enabled, but the code can be extended much more easily if we simply wait until all tasks have been queued, then use the tasks in that list to enable optional dependencies. This will allow us to remove all the infrastructure for checking which physics modules have been enabled explicitly.

4. Replace the task enumerators with strings

Currently tasks are added and sorted according to an enumerator, numrel::TaskName, which describes all possible tasks in the GRMHD and Z4c solvers. This is fine for a small number of tasks, but it will quickly become impractical when we generalize this to all possible physics modules in the code. A more general approach might be to name each task with a string (which is already done for debugging purposes), then compare strings instead of enumerators. Combined with the previous changes, this would make it so that combining physics modules isn't usually much more than ensuring they're all queued with the correct optional dependencies enabled.

Is this something that people would like in the code? Are there any concerns or suggestions before someone starts working on this? As far as I'm aware, the only people it would immediately affect are those running with Z4c and/or DynGRMHD. All physics modules would need their task lists rewritten to use this infrastructure, but this can be done incrementally rather than all at once.

[jfields7]

One suggestion from David: there should be an option to serialize certain tasks such that they never overlap. This is important for tasks which call MPI reductions, such as the compact object trackers, which could potentially execute out of order on different ranks. The simplest policy is just to collect all of these events and insert dependencies if they do already exist.