index update
CLI command shouldn't take into account dynamic mapping fields
#229
Labels
index update
CLI command shouldn't take into account dynamic mapping fields
#229
The
invenio index update ...
CLI command is meant for performing backwards-compatible updates on existing index mappings. To prevent any mistakes, it also checks for backwards-incompatible changes (i.e. field removals, renames/moves, etc.) by comparing the JSON of the existing mapping on the cluster against the new one.This works for simple cases, but when e.g. there are
dynamic_templates
that match nested paths, the JSON diff will falsely think that there are field removals suggested by the new mapping.As a simple example, imagine the following mapping:
This means that one can index a document like:
Which would generate the
keyword
type mapping fortitle.en
,title.de
,org.title.en
,org.title.de
.If the we would e.g. add a new top-level field to the mapping, like
"year": {"type": "date"}
, when running theinvenio index update ...
command, we would get an error, since the differ thinks that the existingtitle.en
,title.de
, etc. fields were "removed" by the new mapping.The solution would be to parse the
dynamic_templates[].path_match
field and filter out all dictdiffer paths that match them. We should also take into accountunmatch
directives that exclude some of the fields.The text was updated successfully, but these errors were encountered: