process duplicates identified by notebook 4.1

[laureD19]

duplicates mainly due to re-ingest errors of the CRF

notebook 4.1 currently gives back:

• Using the attribute(s) "label" as filter, there are: 26 duplicated tools, 6 duplicated publications, 2 duplicated training materials, 0 duplicated workflows, 264 duplicated datasets
• Using the attribute(s) "accessibleAt" as filter, there are: 510 duplicated tools, 26 duplicated publications, 20 duplicated training materials, 277 duplicated datasets

@carikan @mkrzmr - the suggestion would be to go through notebook 4.1 together and decide how we can share the work for the merges needed

[mkrzmr]

dup_label.csv
dup_url.csv

Ran the same and uploaded the files. Suggest we divide the work and merge the items
Q: What to do with item sources? Might create issues with ingest

[laureD19]

If I'm not wrong, the new item created during a merge doesn't have an item source, but keep in its history the two (or more) items merged (including their sources).
With DACE, in case of reingest, in theory, the ingest pipeline notices the difference between the first ingest and the one happenings and marks the problematic item(s) for moderators to have a look at it before their are approved.