CF: decibel unit no yet implemented in UDUNITS

[lbesnard]

The ADCP variable backscatter_intensity_from_acoustic_beam has a unit attribut decibel.

The netcdf-checker ouput is

units                                  :3:    10/22 :
    ABSI1                              :3:     0/ 1 :
        known                          :3:     0/ 1 : unknown units type
                                                      (decibel)

1. why ABSI1 is set as known although there is no standard name, and the variable doesn't exist in the CF convention ?

2. the CF FAQ says decibel will be in future version http://cfconventions.org/faq.html

There are two units acceptable to CF that are not in the UDUNITS library: sverdrup, and decibel or dB. These have been requested for inclusion in future versions of the UDUNITS library.
[....]
 decibel: a logarithmic measure of relative acoustic or energy intensity; symbol dB, db, or dbel (the reference level, needed for logarithmic units, is specified in the standard names that use this canonical unit)

What do to in the meantime ?

[mhidas]

@lbesnard

1. The left column just lists the hierarchy of checks being applied. So "units > ABSI1 > known" means we're checking units, currently looking at the ABSI1 variable, and asking whether the given unit is "known", i.e. defined in the UDUNITS package. Unfortunately it is not.
   This is independent of standard names. All variables representing dimensional quantities should have valid units.

2. Technically this is a bug in the IOOS code, because it only uses UDUNITS to check the validity of units, while this says there are CF acceptable units that are not in UDUNITS. I will raise this issue with them. However, as this is supposed to be a temporary situation, they may prefer to wait for it to be fixed in UDUNITS. We may want to do the same. I've asked them what the status is (Requesting new Decibels (dB) Unit in Udunits Database Unidata/UDUNITS-2#33).

@ggalibert @smancini According to Geonetwork the unit "Decibels" is only used in ABOS and ANMN current velocity data. Do you know of others?

[mhidas]

Reported to IOOS: ioos#137

[ggalibert]

Don't know any other facility using decibel unit.

[mhidas]

This issue has also been reported against the cf_units package, a wrapper for UDUNITS2, which is what the checker uses to assess the validity of units: SciTools/cf-units#30

[mhidas]

Short-term solution: Create our own fork of UDUNITS2 and add "decibel" and any other units we think should be in there. The modified UDUNITS XML files in cf-python are a useful example of how to do this.

[ggalibert]

Ok, just to take some notes for myself :
-UDUNITS2 is installed on 10-nsp via the ubuntu official repos.
-This version of UDUNITS2 doesn't have an up to date database so a copy of the latest UDUNITS2 has been included in the data_services github repo as a hack so that when the netcdf-checker is called from the pipeline, it is set to point to this version of the database.
-We will need to edit this database for our needs. UDUNITS and CF are distinct organisations and there are some units that are CF (like decibel) and not yet included in UDUNITS2 (and decibel might never be included!). If we fork UDUNITS2 then we would be able to edit/maintain it and at the same time benefit from any updates from UDUNITS.
-At the moment the netcdf-checker on 10-nsp points to the ubuntu version of UDUNITS2 database by default. It would be good if it could point by default to our edited version of the database.

Would be good to talk about that with @danfruehauf .

[danfruehauf]

As discussed yesterday, the tradeoff is between more control vs. more maintenance.

If we instantly fork the repository and maintain it ourselves, we're taking on ourselves the task of maintaining it automatically. We need to think carefully whether we want this responsibility. Once we do it, going back is hard.

Ideally, it'd be good to put pressure on UDUNITS to include those units from CF. Since we cannot, I suggest we pull from upstream both UDUNITS and CF and make them play along, perhaps auto-generating combined files or something. This way we can enjoy reduced maintenance and also more control (we can choose what UDUNITS and CF versions we want as long as they don't collide).

At the moment, UDUNITS is in the data-services repository, which is not ideal, but that's because the packaged UDUNITS version is too old I think @mhidas can tell us more about it.

[ggalibert]

@mhidas let's discuss this on Monday. In the meantime I created a pull request on data-services to add decibel there : aodn/data-services#189 .

[mhidas]

For now this is fixed via aodn/data-services#189