Post Timestep Calculations

[spaceman-spence]

Hi there, I have recently started using PeleC for detonation simulations and I am trying to extract data and do some manipulations at runtime with the problem_post_timestep function, however I'm not quite sure where to begin to access the state data. Any pointers or suggestions would be much appreciated. Thanks!

[baperry2]

Here's very simple example problem_post_timestep that shows how to access the state data

void
PeleC::problem_post_timestep()
{
    amrex::MultiFab& S_new = get_new_data(State_Type);
    amrex::Print() << "Max density on lev " << level << " is " << S_new.max(URHO) << std::endl;
}

To access pointwise data, you could use an amrex::ParallelFor over the S_new MultiFab. This should work fine if you just want to access the state to do some on-the-fly data processing. It could potentially also work for manipulating the state, but I'd proceed with caution if you do that, as it would be easy to set state variables in an inconsistent way that leads to problems on the next timestep.

[spaceman-spence]

I see. Thank you! I think that and the Parallel For should be enough to get started. Thank you!

[RSuryaNarayan]

Not sure if this is particularly helpful here, I usually try to wrap up the post-timestep calculations by creating a function similar to pc_initdata in prob.cpp:

void
PeleC::problem_post_timestep()
{
  //Get old and new states
  amrex::MultiFab& S_new = get_new_data(State_Type);
  amrex::MultiFab& S_old = get_old_data(State_Type);

  //MFIter to iterate across the domain
  for (amrex::MFIter mfi(S_new, amrex::TilingIfNotGPU()); mfi.isValid();++mfi)
  {
    const amrex::Box& box = mfi.tilebox();
    auto sfab_new = S_new.array(mfi);
    auto sfab_old = S_old.array(mfi);

    const auto geomdata = geom.data();

    const amrex::Real* prob_lo = geomdata.ProbLo();
    const amrex::Real* dx = geomdata.CellSize();

    const ProbParmDevice* lprobparm = d_prob_parm_device;

    if(lprobparm->PostStep>0)
    {
      amrex::ParallelFor(box, [=] AMREX_GPU_DEVICE(int i, int j, int k) noexcept {
         //Defined at prob.H, just like pc_initdata
         pc_poststep(i, j, k, sfab_new, sfab_old, geomdata, *lprobparm);
        });
    }
  }
}

This allows you to add a function like pc_initdata in prob.H like so:

//-------------------------------After every timestep----------------------------------//
AMREX_GPU_DEVICE
AMREX_FORCE_INLINE
void
pc_poststep(
  int i,
  int j,
  int k,
  amrex::Array4<amrex::Real> const& state_new,
  amrex::Array4<amrex::Real> const& state_old,
  amrex::GeometryData const& geomdata,
  ProbParmDevice const& prob_parm)
{
  amrex::Real P_old, massfrac_old[NUM_SPECIES], T_old, rho_old, P_new, massfrac_new[NUM_SPECIES], T_new, rho_new;
  
 // Dummy code to get old and new temperatures for example
   T_old = state_old(i, j, k, UTEMP);
   T_new = state_new(i, j, k, UTEMP);

}