Search broken with Plone 5.2rc4 because of operator

[pbauer]

Not all catalog-indexes support the operator option (i.e. providing the option to query using 'and' or 'or'). But when you use the Checkbox- or Multiselect-Widgets you always get that configuration-option even though the selected index may not provide this option.

Since 5.0 of Products.ZCatalog querying fails with a ValueError when a a invalid option is used. See zopefoundation/Products.ZCatalog#67 for the change by @andbag. I guess before 5.0 the option would simply have been ignored in the query.

In Plone 5.2rc4 this leads to a broken search:

2019-06-24 10:29:12,722 ERROR   [eea.facetednavigation:38][waitress] index 'portal_type': option 'operator' is not valid
Traceback (most recent call last):
  File "/Users/pbauer/workspace/step-buildout/src-mrd/eea.facetednavigation/eea/facetednavigation/browser/app/query.py", line 178, in query
    brains = catalog(self.context, **query)
  File "/Users/pbauer/workspace/step-buildout/src-mrd/eea.facetednavigation/eea/facetednavigation/search/catalog.py", line 119, in __call__
    return search(**newquery)
  File "/Users/pbauer/.cache/buildout/eggs/Products.CMFPlone-5.2rc4-py3.7.egg/Products/CMFPlone/CatalogTool.py", line 457, in searchResults
    return ZCatalog.searchResults(self, query, **kw)
  File "/Users/pbauer/.cache/buildout/eggs/Products.ZCatalog-5.0.1-py3.7.egg/Products/ZCatalog/ZCatalog.py", line 611, in searchResults
    return self._catalog.searchResults(query, **kw)
  File "/Users/pbauer/.cache/buildout/eggs/Products.ZCatalog-5.0.1-py3.7.egg/Products/ZCatalog/Catalog.py", line 1091, in searchResults
    return self.search(query, sort_indexes, reverse, sort_limit, _merge)
  File "/Users/pbauer/.cache/buildout/eggs/Products.ZCatalog-5.0.1-py3.7.egg/Products/ZCatalog/Catalog.py", line 634, in search
    rs = self._search_index(cr, index_id, query, rs)
  File "/Users/pbauer/.cache/buildout/eggs/Products.ZCatalog-5.0.1-py3.7.egg/Products/ZCatalog/Catalog.py", line 562, in _search_index
    index.operators, index.useOperator)
  File "/Users/pbauer/.cache/buildout/eggs/Products.ZCatalog-5.0.1-py3.7.egg/Products/ZCatalog/query.py", line 80, in __init__
    self.set(op, param[op])
  File "/Users/pbauer/.cache/buildout/eggs/Products.ZCatalog-5.0.1-py3.7.egg/Products/ZCatalog/query.py", line 126, in set
    ' is not valid').format(self.id, key))
ValueError: index 'portal_type': option 'operator' is not valid

I tried a workaround for that like this in eea.facetednavigation.browser.app.query.FacetedQueryHandler.criteria:

if widget_index:
    ctool = getToolByName(self.context, 'portal_faceted', None)
    if not ctool:
        ctool = getToolByName(self.context, 'portal_catalog')
    index = ctool._catalog.indexes.get(widget_index, None)
    for index_option in list(query.get(widget_index, [])):
        if index_option not in index.query_options:
            del query[widget_index][index_option]

That seemed to do the trick and worked with my updated instance but it would be much better to only offer this option if the index actually supports such a query.

[pbauer]

In collective#54 I added a slightly simpler implementation of my fix.

[jensens]

I have the same issue, and will check your fix now.

[jensens]

collective#54 fixes it for me too.

[jensens]

well no, fulltext search is broken.

[jensens]

collective@2c01dcb fixes this for Text Widget.

I fear we need to check all other widgets too.

[thet]

@pbauer @jensens this could be a fix for this problem: zopefoundation/Products.ZCatalog#79

[jensens]

@thet thanks for pointing us to the issue, but no, I dont think it is related.