run the keyword script

[laureD19]

on stage to see if there are bugs
based on the mappings tab that includes new values: https://docs.google.com/spreadsheets/d/1-Oh9_SxIhfMAT6KNJrMf4LetCpy5s1fHZEyTL__TUVA/edit#gid=0

[laureD19]

ongoing. 500 errors spotted by Cesare. In contact with Klaus.
Tested on stage and prod data
Script updated to do not work anymore with the 'rejection' tab

[cesareconcordia]

Hi, more info: this morning I'm having the same issue. For this URL at one point I had this error message:

{"timestamp":"2024-04-11 10:54:29","status":500,"error":"Connect to api-stage-sshomp-solr:8983 [api-stage-sshomp-solr/10.43.0.97] failed: Connection refused (Connection refused); nested exception is org.apache.http.conn.HttpHostConnectException: Connect to api-stage-sshomp-solr:8983 [api-stage-sshomp-solr/10.43.0.97] failed: Connection refused (Connection refused)"}

but for the same URL few seconds later the correct result is returned

[mkrzmr]

I had a look at the logs, the service must have been restarting at that time
Starting Solr 2024-04-11 10:55:08.872 INFO (main) [ ] o.e.j.u.log Logging initialized @630ms to org.eclipse.jetty.util.log.Slf4jLog

a few seconds later I see the second request

2024-04-11 10:55:15.537 INFO (qtp1616359099-26) [ x:marketplace-concepts] o.a.s.c.S.Request [marketplace-concepts] webapp=/solr path=/select params={f.{!ex%3Dtag_candidate}candidate.facet.limit=0&facet.field={!ex%3Dtag_types}types&facet.field={!ex%3Dtag_candidate}candidate&fl=code,vocabulary_code,label,notation,definition,uri,types,candidate&start=0&f.{!ex%3Dtag_types}types.facet.mincount=1&fq={!tag%3Dtag_types}types:keyword&sort=score+desc&rows=20&version=2&q=*:*+AND+(code:https\://vocabs.dariah.eu/sshoc\-keyword/POS^10.0+OR+uri:https\://vocabs.dariah.eu/sshoc\-keyword/POS^10.0+OR+notation:https\://vocabs.dariah.eu/sshoc\-keyword/POS^4.0+OR+label_text:*https\://vocabs.dariah.eu/sshoc\-keyword/POS*^2.0+OR+definition_text:*https\://vocabs.dariah.eu/sshoc\-keyword/POS*^1.0+OR+label_text_en:https\://vocabs.dariah.eu/sshoc\-keyword/POS^4.0+OR+definition_text_en:https\://vocabs.dariah.eu/sshoc\-keyword/POS^2.0)&facet.limit=-1&f.{!ex%3Dtag_candidate}candidate.facet.mincount=0&facet.mincount=1&facet=true&wt=javabin&f.{!ex%3Dtag_types}types.facet.limit=0&facet.sort=count} hits=29 status=0 QTime=172

The question is was the service restarting anyway or did it crash?

[cesareconcordia]

Thank you Michael.
The error occurs after a sequence of invocations of the API entry:
/api/concept-search?types=keyword&q=keyword
It seems that it does not depend from a specific keyword, additionally I've made a test on production server and the error occurred too. Maybe there is something that make the solr server restart?

[laureD19]

Cesare: deleting keyword currently delete keyword form the item, but not from the vocabulary.
Open question: what is the workflow to update the vocabulary??

Script to delete keywords from the items worked on stage.
Could be run on production, but need a backup before (action on: @mkrzmr )

Before running the mapping script (and not only the delete one), fix solr upgrade.

[laureD19]

@cesareconcordia - the SOLR upgrade is now done on production instance too. Could you try again to run the keyword script, so that we have clean keyword vocabulary on production?

[cesareconcordia]

@laureD19 ok! will do it tomorrow and will let you know

[cesareconcordia]

Executed the NB 5.1 on production instance:

• the keywords having 'delete' as Map to value in the gsheet have been all removed.
• a subset of the keywords that have to be mapped have been processed, apparently with no errors, will continue to work on this in the next days.

[laureD19]

Some new concepts added to the spreadsheet, either to be deleted, or to be mapped to other vocabularies. Need to run the script again.