2022-01-24 PyAerocom meeting

[avaldebe]

Follow up from last meeting

Jan 17th #570

New Issues

None

Get list of observations/data

How to find available observations/catalogues of information? (Michael S. will open GH issue/feature request)
https://pyaerocom.readthedocs.io/en/latest/api.html?highlight=browse#pyaerocom.io.utils.browse_database

Pyaerocom mostly meant for using known data, not as a metadata store

Where is documentation of readers / code of the readers? https://github.com/metno/pyaerocom/tree/main-dev/pyaerocom/io

In script

>>> pya.browse_database('*')
Found more than 20 matches for input pattern *:

Matches: ['AeronetSunV2Lev1.5.daily', 'AeronetSun_2.0_NRT', 'AeronetSunV2Lev2.daily', 'AeronetSunV2Lev2.AP', 'AeronetSDAV2Lev2.daily', 'AeronetInvV2Lev1.5.daily', 'AeronetInvV2Lev1.5.AP', 'AeronetInvV2Lev2.daily', 'AeronetInvV2Lev2.AP', 'AeronetSunV3Lev1.5.daily', 'AeronetSunV3Lev1.5.AP', 'AeronetSunV3Lev2.daily', 'AeronetSunV3Lev2.AP', 'AeronetSDAV3Lev1.5.daily', 'AeronetSDAV3Lev1.5.AP', 'AeronetSDAV3Lev2.daily', 'AeronetSDAV3Lev2.AP', 'AeronetInvV3Lev1.5.daily', 'AeronetInvV3Lev2.daily', 'EBASMC', 'EEAAQeRep', 'EARLINET', 'GAWTADsubsetAasEtAl', 'DMS_AMS_CVO', 'GHOST.EEA.daily', 'GHOST.EEA.hourly', 'GHOST.EEA.monthly', 'GHOST.EBAS.daily', 'GHOST.EBAS.hourly', 'GHOST.EBAS.monthly', 'CERES', 'Z3D', 'MPI_OBSBASED', 'lost+found', 'BHJC', 'MPI_OM-HAMOCC', 'AEROCOM_MEDIAN_DUST', 'NMMB-BSC', 'Sulphur_Trend', 'AAA-Python', 'bin', 'MPIHAM-r714_EUC000JK', 'CRESCENDO_WP6', 'LOA_SV', 'AEROCOM_tools', 'AAA-QUESTIONABLE-DATA', 'CMIP6', 'OBSDATA', 'AEROCOM_A', 'SPLA', 'AEROCOM_LIDARDATA', 'AEROCOM_MEDIAN', 'LOTOS_EUROS.ASSIM', 'AEROCOM_B', 'Aeolus_CalVal', 'AAA-JOBS', 'MPIHAM-r716_EUC003JK', 'AEROCOM_FILE_EXTRACT', 'ECHAM5HAM2_R46D', 'SPLA_NEW', 'MPIHAM-r481_EUC94144LP', 'LOTOS_EUROS.BCKGRD', 'AERONET_TESTBED-SK', 'AEROCOM_PRESCRIBED-OPTICS', 'MPIHAM-r578_EUC002JK', 'OSLO_OBS', 'CCSM4', 'ECMWF_OSUITE_NRT', 'ECMWF_OSUITE_NRT.old', 'AEROCOM_MEDIAN-SK', 'AEROCOM_SOURCE', 'AEROCOM_MEDIAN_B', 'ECMWF_OSUITE_NRT_test', 'ADM_CALIPSO_TEST', 'AEROCOM_ADJUSTED-BC', 'ECHAM5HAM2_R46', 'CAMS-50', 'pyaerocom-synthesis', 'AEROCOMLIB-METNO', 'INCA', 'AAA-mic-jobs', 'CALIOP_SylvainPetiotWork', 'PopulationDensity', 'EMEP_SVN_TEST', 'pyaerocom_out', 'NorESM_SVN_TEST', 'BSC_DREAM8b', 'DATA', 'EMEP_COPERNICUS', 'EMEP_GLOBAL', 'cams-aerosol-alert', 'CHIMERE_DUST', 'SURFACE_ALBEDO', 'EMEP_TFMM', 'BCC-ESM1_historical', 'MIROC-SPRINTARS_historical', 'UKESM1-0-LL_historical', 'CESM2_historical', 'CNRM-ESM2-1_historical', 'E3SM-1-0_historical', 'CESM2-WACCM_historical', 'MPI-ESM-1-2-HAM_historical', 'NorESM2-LM_historical', 'IPSL-CM6A-LR_historical', 'GFDL-CM4_historical', 'MRI-ESM2-0_historical', 'MIROC6_historical', 'CNRM-CM6-1_historical', 'CanESM5_historical', 'SLSTR_SU_v1.11', 'SLSTR_SDV.v2.10'...

In CLI

pya -h

usage: pya [-h] [-b BROWSE] [--clearcache] [--ppiaccess]

pyaerocom command line interface

optional arguments:
  -h, --help            show this help message and exit
  -b BROWSE, --browse BROWSE
                        Browse database
  --clearcache          Delete cached data objects
  --ppiaccess           Check if MetNO PPI can be accessed

pya -b *

usage: pya [-h] [-b BROWSE] [--clearcache] [--ppiaccess]
pya: error: unrecognized arguments: ACTRIS aerocomevaluation aerocom.met.no aeronet aerosolalerts aerosoltrends AeroTweets aeroval aframe alerts AOD-COVID19 A-Profiles aq.met.no aq-portal ash basic Bootstrap Bruce CALIOP Calitoo cams84 Climato EMEP_logo e-science Fonctions_Scilab FR Inversion KTV LOA michaels Mie model-scoring mwr_raw2l1 MySQL NegiCourse policysupport pyaerocom pygame Rayleigh scripts S-Trends Svetlana traefik_v2.5.4_linux_amd64 trends uEMEP VAAC V-Profiles web-bak

pya -b Aeronet*Heinesen

Searching database for matches of Aeronet*
Reading failed for AeronetSunV2Lev1.5.daily. Error: NetworkNotImplemented('Could not find reading class for dataset AeronetSunV2Lev1.5.daily')
Reading failed for AeronetSun_2.0_NRT. Error: NetworkNotImplemented('Could not find reading class for dataset AeronetSun_2.0_NRT')

Dataset name: AeronetSunV2Lev2.daily
Data directory: /lustre/storeA/project/aerocom/aerocom1/AEROCOM_OBSDATA/AeronetRaw2.0/renamed
Supported variables: ['od1640aer', 'od1020aer', 'od870aer', 'od675aer', 'od667aer', 'od555aer', 'od551aer', 'od532aer', 'od531aer', 'od500aer', 'od440aer', 'od380aer', 'od340aer', 'ang4487aer', 'od550aer', 'ang4487aer_calc']
Last revision: 20180519

New PRs

• calculate the new variable ratpm10pm25 using the calculation infrastructure of the ReadUngridded class #550

New DeprecationWarnings

 # xarray calling index.get_loc on Pandas 1.4.0
 "ignore:Passing method to Float64Index.get_loc is deprecated:FutureWarning",
 # numpy calling calling distutils.ccompiler
 "ignore:The distutils package is deprecated:DeprecationWarning",

Should we continue the minutes on Google Docs?

Maybe we could move to github discussions

For next meeting

• Daniel will take a look at PR CAMS2-83 surface observations #546
• Python packaging mini workshop with Álvaro, Krister, Ove, Daniel and Charlie
• Michael S. open an issue with his question about available obs Observation catalogue / overview #551
• Talk about moving away from google docs into github discussions