Keep original dtype in DayNightCompositor

[pnuu]

This PR is a start to keep the dtype of the input data until saving. To have the tests pass, another PR is needed for Trollimage so that the enhancements don't upscale the data.

The tests require pytroll/trollimage#150 to be merged (done) and a new Trollimage release including it. This is a side-effect of the Satpy feature applying a linear stretch by default, and in the current release this up-casts float32 to float64.

• Closes #xxxx
• Tests added
• Fully documented
• Add your name to AUTHORS.md if not there already

[pnuu]

The tests will require pytroll/trollimage#150

[pnuu]

The composite tests will require also pytroll/trollimage#151 and a new release of Trollimage.

[pnuu]

The changes in Trollimage might have affected the "thruth" values in also other tests, I'll have a look.

[pnuu]

2ff0b2a should fix three of the tests that were affected by floating point inaccuracy. There's a remaining Ninjo geotiff failure and three CF failures, I'd guess they are again because of a new XArray release without the compression fix.

[pnuu]

Oh, now there's only the single Ninjo tiff failure. I'll compare the results with previous Trollimage version with the current one tomorrow and see what could be the problem. I don't know what the NinJoTagGenerator does, @gerritholl do you have time to have a look?

[gerritholl]

Some background info to the failing test:

The NinjoTagGenerator class derives NinJo tags from data and metadata. In the failing test, it calculates the minimum grey value for an RGB image that was originally with data ranged from 0 to 255, but that should become 1 to 255 because we used fill_value=0, such that the rest of the data have to be stretched from 1 to 255. I haven't run the test with this branch yet to dig deeper, but it looks like it became 255 rather than 1, which is not good.

I have no clue what that has to do with dtype preservation in the DayNightCompositor :O

[pnuu]

I have no clue what that has to do with dtype preservation in the DayNightCompositor :O

Nothing, the change is due to my updates to Trollimage (most likely pytroll/trollimage#150) which were released today and needed for the DNC tests to pass in this PR.

[pnuu]

Ok.

#!/usr/bin/env python

import numpy as np
import xarray as xr

from trollimage import xrimage

arr = np.arange(75, dtype=np.uint8).reshape(5, 5, 3)
arr[4, 4, :] = 255
data = xr.DataArray(arr.copy(), dims=['y', 'x', 'bands'],
                    coords={'bands': ['R', 'G', 'B']})
img = xrimage.XRImage(data)
print("img", img.data.values.min(), img.data.values.max())
img.stretch_linear()
print("stretched img", img.data.values.min(), img.data.values.max())

With tag Trollimage v1.21.0 this prints

$ python test_xr_uint8.py
img 0 255
stretched img -0.0015614156835530892 1.0960743801652901

and with main

$ python test_xr_uint8.py
img 0 255
stretched img 0.0 2.2130435

I'll create an issue on Trollimage.

[pnuu]

pytroll/trollimage#152

[pnuu]

The integer stretching is fixed in pytroll/trollimage#153 . So we need a patch release of Trollimage to resolve this.

[mraspaud]

The integer stretching is fixed in pytroll/trollimage#153 . So we need a patch release of Trollimage to resolve this.

We will do a patch release as soon as the PR is merged.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (a0b8a7b) 95.20% compared to head (2ff0b2a) 95.21%.
Report is 80 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff            @@
##             main    #2640    +/-   ##
========================================
  Coverage   95.20%   95.21%            
========================================
  Files         354      356     +2     
  Lines       51393    51603   +210     
========================================
+ Hits        48930    49132   +202     
- Misses       2463     2471     +8     

Flag	Coverage Δ
behaviourtests	4.22% <0.00%> (-0.02%)	⬇️
unittests	95.83% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[coveralls]

Pull Request Test Coverage Report for Build 6971000025

Warning: This coverage report may be inaccurate.

We've detected an issue with your CI configuration that might affect the accuracy of this pull request's coverage report.
To ensure accuracy in future PRs, please see these guidelines.
A quick fix for this PR: rebase it; your next report should be accurate.

• 31 of 31 (100.0%) changed or added relevant lines in 4 files are covered.
• 59 unchanged lines in 17 files lost coverage.
• Overall coverage increased (+0.001%) to 95.784%

---

Files with Coverage Reduction	New Missed Lines	%
satpy/modifiers/spectral.py	1	92.13%
satpy/tests/reader_tests/test_scmi.py	1	99.07%
satpy/tests/reader_tests/test_vii_base_nc.py	1	98.77%
satpy/tests/reader_tests/test_vii_l1b_nc.py	1	97.22%
satpy/tests/multiscene_tests/test_save_animation.py	2	99.1%
satpy/tests/test_demo.py	2	99.36%
satpy/tests/test_writers.py	2	99.05%
satpy/tests/test_yaml_reader.py	2	99.61%
satpy/tests/utils.py	2	93.36%
satpy/tests/reader_tests/gms/test_gms5_vissr_l1b.py	3	98.67%

Totals
Change from base Build 6891901381:	0.001%
Covered Lines:	49258
Relevant Lines:	51426

---

💛 - Coveralls

[djhoese]

This looks good to me. I'm not sure I followed the discussion over the last week or two on slack and on github but the final result looks reasonable.

[mraspaud]

LGTM