TypeError: 'NoneType' object is not iterable while issuing queries to Elastic Search

[lmammino]

How to reproduce:

1. spin up the environment

Use this docker compose setup:

# docker-compose.yml
version: '3'

services:
  splunkenterprise:

    hostname: splunkenterprise
    image: splunk/splunk
    environment:
      SPLUNK_START_ARGS: --accept-license --answer-yes --seed-passwd somepass123456789
      SPLUNK_ENABLE_LISTEN: 9997
      SPLUNK_ADD: tcp 1514
      OPTIMISTIC_ABOUT_FILE_LOCKING: 1
    volumes:
      - ./opt-splunk-etc:/opt/splunk/etc
      - ./opt-splunk-var:/opt/splunk/var
    ports:
      - "8000:8000"
      - "9997:9997"
      - "8088:8088"
      - "1514:1514"

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.3.1
    container_name: elasticsearch
    environment:
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - ./esdata:/usr/share/elasticsearch/data
    ports:
      - 9200:9200

  kibana:
    image: docker.elastic.co/kibana/kibana:6.3.1
    depends_on:
      - elasticsearch
    environment:
      ELASTICSEARCH_URL: http://elasticsearch:9200
    ports:
      - 5601:5601

This will spin up spin up splunk, elastic and kibana

1.2 install the plugin

in your local ./opt-splunk-etc/apps run:

git clone https://github.com/brunotm/elasticsplunk.git

then stop and restart the infrastructure:

docker-compose down && docker-compose up

2. add data to elastic

Use kibana on http://localhost:5601 and through the development tools run the following query:

PUT _bulk
{"index": {"_index": "pear", "_type": "default"}}
{"foo":"bar", "type":"fruit", "subtype":"pear", "ts":"2018-07-19"}
{"index": {"_index": "banana", "_type": "default"}}
{"foo2":"bar2", "type":"fruit", "subtype":"banana", "ts":"2018-07-19"}
{"index": {"_index": "appple", "_type": "default"}}
{"foo3":"bar3", "type":"fruit", "subtype":"apple", "ts":"2018-07-19"}

3. query on splunk

Login in splunk at http://localhost:8000 (admin/somepass123456789) and tun the following query

|ess eaddr=elasticsearch:9200 action=query query="foo:bar" index=pear fields=foo include_es=true include_raw=true stype=doc_type tsfield=ts

The following error will be displayed on screen:

External search command 'ess' returned error code 1. Script output = "None error_message=TypeError at "/opt/splunk/etc/apps/elasticsplunk/bin/splunklib/searchcommands/internals.py", line 520 : 'NoneType' object is not iterable "

while my expectation was to see the following records:

{"foo":"bar", "type":"fruit", "subtype":"pear", "ts":"2018-07-19"}

[brunotm]

Hi @lmammino, thanks for the detailed issue!

Can you please try using the doc_type (stype=default) for the indexed document?

[lmammino]

@brunotm,
thanks for your quick reply.

I just tried it now and I have a very similar result:

External search command 'ess' returned error code 1. Script output = "error_message=TypeError at "/opt/splunk/etc/apps/elasticsplunk/bin/splunklib/searchcommands/internals.py", line 519 : 'NoneType' object is not iterable "

[brunotm]

@lmammino,

This is because of the Elastic api client currently is on 5.x on elasticsplunk, and due to breaking changes on Elastic 6.x (mainly one doc_type per index).

I'll update the API client and adjust the elasticsplunk code in order to support both 5/6 versions and let you know here.

Thanks!

[lmammino]

Thanks for having a better look. I don't expect any commitment being this an open source project, but do you think you will have bandwidth anytime soon to work on supporting elastic 6.x?

[brunotm]

Yes :)
Thanks!

[brunotm]

@lmammino
i just had the time to look at this, and it happens that the current client is still compatible and i overlooked you ess options.

Could you please change the action to action=search with the correct stype as before (you can also omit it for this case.

The following example works fine with your example above:

|ess eaddr="http://localhost:9200" action=search query="foo:bar" fields=* include_es=true include_raw=true tsfield=ts stype=default

Remember to check if the following also works for you:

|ess eaddr="http://localhost:9200" action=indices-list

Change the localhost with your ES address.

Cheers.