Dataset files on Globus seem to be unaccessible

[menggedu]

Great work and paper. But I got this error when I tried to fetch the datafile on globus. Could you help me fix it?

Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/fsspec/mapping.py", line 143, in __getitem__
    result = self.fs.cat(k)
  File "/opt/conda/lib/python3.8/site-packages/fsspec/asyn.py", line 115, in wrapper
    return sync(self.loop, func, *args, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/fsspec/asyn.py", line 100, in sync
    raise return_result
  File "/opt/conda/lib/python3.8/site-packages/fsspec/asyn.py", line 55, in _runner
    result[0] = await coro
  File "/opt/conda/lib/python3.8/site-packages/fsspec/asyn.py", line 414, in _cat
    raise ex
  File "/opt/conda/lib/python3.8/asyncio/tasks.py", line 455, in wait_for
    return await fut
  File "/opt/conda/lib/python3.8/site-packages/fsspec/implementations/http.py", line 231, in _cat_file
    self._raise_not_found_for_status(r, url)
  File "/opt/conda/lib/python3.8/site-packages/fsspec/implementations/http.py", line 213, in _raise_not_found_for_status
    raise FileNotFoundError(url)
FileNotFoundError: https://g-402b74.00888.8540.data.globus.org/jet/forcing2.zarr/.zmetadata

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test.py", line 21, in <module>
    jet_high_res = get_dataset('jet/forcing2').isel(run=0)
  File "test.py", line 17, in get_dataset
    return xr.open_zarr(mapper, consolidated=True)
  File "/opt/conda/lib/python3.8/site-packages/xarray/backends/zarr.py", line 688, in open_zarr
    ds = open_dataset(
  File "/opt/conda/lib/python3.8/site-packages/xarray/backends/api.py", line 501, in open_dataset
    backend_ds = backend.open_dataset(
  File "/opt/conda/lib/python3.8/site-packages/xarray/backends/zarr.py", line 736, in open_dataset
    store = ZarrStore.open_group(
  File "/opt/conda/lib/python3.8/site-packages/xarray/backends/zarr.py", line 336, in open_group
    zarr_group = zarr.open_consolidated(store, **open_kwargs)
  File "/opt/conda/lib/python3.8/site-packages/zarr/convenience.py", line 1300, in open_consolidated
    meta_store = ConsolidatedStoreClass(store, metadata_key=metadata_key)
  File "/opt/conda/lib/python3.8/site-packages/zarr/storage.py", line 2861, in __init__
    meta = json_loads(self.store[metadata_key])
  File "/opt/conda/lib/python3.8/site-packages/zarr/storage.py", line 724, in __getitem__
    return self._mutable_mapping[key]
  File "/opt/conda/lib/python3.8/site-packages/fsspec/mapping.py", line 147, in __getitem__
    raise KeyError(key)
KeyError: '.zmetadata'

[georgemilosh]

Yeah I looked at another repo of m2lines and saw a potential solution, could it be that the filename was changed?
In this case there is a typo in the notebook dataset_description.ipynb and it should say

get_dataset('eddy/hires')

Also it seems the requirements.txt did not have all the requirements, I also had to manually install some packages like aiohttp and zarr