[Bug] Sanitizing institution data harvested from GrSciColl breaks functionality

[mickley]

In the institution editor, there is now a call to sanitize all institution data before displaying it using htmlspecialchars(), see:

Symbiota/collections/misc/institutioneditor.php

Line 201 in 06f6cb6

$instArr = $instManager->cleanOutArray($instArr);

That seems reasonable, however, when I wrote the code to harvest data from GBIF's GrSciColl API, I incorporated several GrSciColl/indexHerbariorum fields and links to Index Herbariorum and GrSciColl in the notes field, using some html. Santitizing with htmlspecialchars unfortunately breaks that functionality. See:

Symbiota/js/symb/collections.grscicoll.js

Line 297 in 06f6cb6

// Add taxonomic coverage to notes, if included

Here's what it should look like:

Here's what the sanitizing does:

@Atticus29: There's different ways this could be fixed: sanitizing all except the notes, allowing limited html, having more institution editor fields to avoid this need, etc. Up to you as to what works best.

[mickley]

Another note in relation to this:
The notes field in the Institutions table has been changed back to 250 characters, which will cut off this content.
I had it set as VARCHAR(19000), perhaps it should be TEXT.

[egbot]

I've submitted a hotfix patch that removes outbound sanitation of the notes field, which should partially resolve the issue above. This should make it into the master branch probably within the week.

[egbot]

Another note in relation to this: The notes field in the Institutions table has been changed back to 250 characters, which will cut off this content. I had it set as VARCHAR(19000), perhaps it should be TEXT.

The new feature you added that grabs institutional data directly from GBIF and IH is awesome. Love it. I wonder if we should add a new field for the additional content you put into the notes field. Maybe a field called dynamicProperties (TEXT) where we store teh data as a JSON object (e.g. key/value pairs), which can be output as html with the core values sanitized. Just throwing out ideas. Thoughts?

[egbot]

Keeping this open until we deal with next step, which I tagged it as an enhancement.

[mickley]

Another note in relation to this: The notes field in the Institutions table has been changed back to 250 characters, which will cut off this content. I had it set as VARCHAR(19000), perhaps it should be TEXT.

The new feature you added that grabs institutional data directly from GBIF and IH is awesome. Love it. I wonder if we should add a new field for the additional content you put into the notes field. Maybe a field called dynamicProperties (TEXT) where we store teh data as a JSON object (e.g. key/value pairs), which can be output as html with the core values sanitized. Just throwing out ideas. Thoughts?

@egbot That sounds fine to me. I think it would be more flexible as well, e.g., if more information is available in the future. From a collections standpoint, it's quite nice to have this data when preparing shipments, and not to have to go looking for it elsewhere.