[foliatextcontent] Implement adding markup information in the text that points to the substrings

[proycon]

This is needed for proycon/flat#92 . There is already an option for this in foliatextcontent but it doesn't seem to work yet in all cases , most specifically, the case where the text content is already present rather than generated by foliatextcontent.

[proycon]

<correction> elements over <str> should be translated to the proper <t-correction> elements.

[pirolen]

Awesome, thanks so much for making this enhancement!

Just wondering (not requesting), would this enable manual correction operations, at least partly, such as:
when a superscript number ("17") was misrecognized as apostrophe:
<t class="OCR" offset="1086">Materialien'</t>

-->

• make a delete operation of this token
• add new token with "Materialien"
• add new token with "17"
• annotate the "17" token as superscript style.

(Once the PAGE-XML to FoLiA converter is there, I could use ucto and generate a test file --please let me know I could do st more.)

[proycon]

(I don't think this relates directly to this issue, which is about substrings (arbitrary references on untokenised text))

I assume you refer to manual annotation in FLAT, and editing corrections in FLAT indeed only works on the token-level. If a tokenised document is available with all the markup information present then the procedure you described would work for the first three steps yes, but the fourth is still an issue as FLAT doesn't support annotating markup (e.g. style) yet, the markup support in FLAT is limited to viewing currently.

The other caveat is preserving all the markup information after tokenisation, ucto currently doesn't do that. You're currently stuck with the markup information on mostly the paragraph level. Neither TICCL nor ucto propagate it to deeper levels, which is what you need if you want to correct it in FLAT. I had already opened a related issue to implement this specific functionality in foliatextcontent: #19 .. The good news is that this should all be automatically resolvable.

[pirolen]

Awesome, thanks! Sorry about commenting at the wrong issue, I meant indeed the functionality of FLAT.