coverage methods

[dpancic]

In GitLab by @laureD19 on Oct 19, 2022, 08:28

In the coverage flag notebook, there are still a few things to adjust to be able to write back to the marketplace in a meaningful way.

This issue is meant to support the discussion about the methods used to raise the coverage flag and refine the related notebook.

notify @cesareconcordia @KlausIllmayer @aureon249 @kreetrapper

[dpancic]

In GitLab by @laureD19 on Oct 19, 2022, 08:31

while running the current notebook 3.4CurationFlagCoverage, Martin and I got the same error (already encountered last year as far as I remember) about "Returning a view versus a copy". See screenshot below for example, but this happens not only on the cell previewed here.

[dpancic]

In GitLab by @laureD19 on Oct 19, 2022, 09:05

Two rationales to raise the coverage flag were developed in this notebook: one based on a "max number of null values allowed" in section 1; and one based on specific properties with null values in the second section of the notebook.

Because I think we should raise the coverage flag using - for now - the second section, I've tried to adapt the notebook and to raise the flag with

res_cov=mpdata.setPropertyFlags(df_flag_dataset, curation_flag_property, curation_detail_property)

but run into some errors with the setPropertyFlags function that I don't understand. I attached here what I've tried out: 3.4CurationFlagCoverage.ipynb

Any guidance more than welcome!

[dpancic]

In GitLab by @cesareconcordia on Oct 19, 2022, 10:55

Thanks, it should have been fixed in 3.4 now, let me know if it persists.
This Warning appears also in other notebooks, I'm going to check them.

[dpancic]

In GitLab by @cesareconcordia on Oct 19, 2022, 11:41

Hi Laure, the code you wrote is correct, just to make the example work you should replace the line

df_flag_dataset=df_items_coverage_null_values_tools

with:

df_flag_dataset=df_coverage_sp

the df_coverage_sp is the dataframe containing items to be flagged and flagging information (number of null properties), this dataframe must be passed to the mpdata.setPropertyFlags function.

I attach the fixed nb, I also changed the code to avoid the "Returning a view versus a copy" warning in the code you wrote.

3.4CurationFlagCoverageL.ipynb

[dpancic]

In GitLab by @laureD19 on Oct 19, 2022, 17:11

wonderful! It works as expected to raise the flag!

But I still got the "Returning a view versus a copy" error :(

[dpancic]

In GitLab by @cesareconcordia on Oct 20, 2022, 16:57

Try with the nb attached, I've tried to fix the error

3.4CurationFlagCoverageL.ipynb