add client.scatter?

[matt-long]

When runningFEISTY_driver I get the following message in the log file:

Starting compute at 10:54:30
2022-09-25 10:54:30,721 - distributed.scheduler - INFO - Receive client connection: Client-b929bd22-3cf2-11ed-82f1-3cecef1a5636
2022-09-25 10:54:30,721 - distributed.core - INFO - Starting established connection
/glade/work/mclong/miniconda3/envs/dev-feisty/lib/python3.9/site-packages/distributed/worker.py:2936: UserWarning: Large object of size 2.29 MiB detected in task graph: 
  ([('X',), <xarray.IndexVariable 'X' (X: 300129)>
a ... =object), {}],)
Consider scattering large objects ahead of time
with client.scatter to reduce scheduler burden and 
keep data on workers

    future = client.submit(func, big_data)    # bad

    big_future = client.scatter(big_data)     # good
    future = client.submit(func, big_future)  # good
  warnings.warn(
Starting run() at 10:54:42
Starting run() at 10:54:42
Starting run() at 10:54:42
Starting run() at 10:54:42
Starting run() at 10:54:42
Starting run() at 10:54:42
...

The relevant block of code is here:

    print(f'Starting compute at {time.strftime("%H:%M:%S")}')

    with Client() as c:
        # map_blocks lets us run in parallel over our dask cluster
        ds_out = xr.map_blocks(
            feisty.config_and_run_from_dataset,
            ds,
            args=(
                run_settings.nsteps,
                run_settings.start_date,
                run_settings.ignore_year_in_forcing,
                run_settings.settings_in,
                run_settings.diagnostic_names,
                run_settings.max_output_time_dim,
                run_settings.method,
            ),
            template=template,
        ).compute()