romanisim support for very large catalogs

[stscijgbot-rstdms]

Issue RCAL-985 was created on JIRA by Eddie Schlafly:

We need to be able to simulate large surveys (e.g., HLWAS) with romanisim.  We don't want to pass a single file as the catalog for that.  Instead, we'd want to pass a directory with files in some kind of format that say what part of the sky they correspond to is.  I would propose the following: * Download all of Gaia to ██████████ here: http://cdn.gea.esac.esa.int/Gaia/gdr3/gaia_source/ (looks like ~600 GB)

• Go through each of those files and figure out which healpix pixels they touch (nside = 128)
• Build a new set of files for each healpix
  ** From the previous step, we know which files touch each healpix.
  ** For each healpix, loop over the csv files that touch it.
  ** Extract the rows in the CSV file that correspond to this healpix
  ** Concatenate them and write them out to a new file along the lines of gaia-{healpixno}.fits.
• In romanisim, allow specifying catalog by either a directory or by a file (as now).
  ** If a directory, then figure out which healpix touch this image (healpix that touch some circle around WFI_CEN).  Then read the name-{healpixno} files in the directory in and in and concatenate.  Then proceed as usual in the simulation.

If we had something like this, then we could simulate any test program that anyone wanted (albeit only with Gaia stars).
On top of this, we can also add artificial simulated galaxies based on distribution from the COSMOS field to have a more realistic full-sky catalog that we could feel good about!