elasticsearch.yml is not reflected

[tadashi-oya]

Hi there.
I'm having trouble with changes to docker/elk/elasticsearch/dist/elasticsearch.yml not being reflected.

The reproduction procedure is as follows.

[root@gladmattress:~]# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

[root@gladmattress:~]# cd work
[root@gladmattress:~/work]# git clone https://github.com/telekom-security/tpotce

[root@gladmattress:~/work]# cp tpotce/docker/elk/elasticsearch/dist/elasticsearch.yml tpotce/docker/elk/elasticsearch/dist/elasticsearch.yml.orig

[root@gladmattress:~/work]# vi tpotce/docker/elk/elasticsearch/dist/elasticsearch.yml
(modify "xpack.security.enabled: true" and add "discovery.type: single-node")

[root@gladmattress:~/work]# diff -u tpotce/docker/elk/elasticsearch/dist/elasticsearch.yml.orig tpotce/docker/elk/elasticsearch/dist/elasticsearch.yml
--- tpotce/docker/elk/elasticsearch/dist/elasticsearch.yml.orig 2021-11-29 17:46:06.980000000 +0900
+++ tpotce/docker/elk/elasticsearch/dist/elasticsearch.yml      2021-11-29 16:39:16.440456476 +0900
@@ -1,7 +1,8 @@
 cluster.name: tpotcluster
 node.name: "tpotcluster-node-01"
 xpack.ml.enabled: false
-xpack.security.enabled: false
+discovery.type: single-node
+xpack.security.enabled: true
 #xpack.ilm.enabled: false
 path:
     logs: /data/elk/log

[root@gladmattress:~/work]# cd tpotce/iso/installer/
[root@gladmattress:~/work/tpotce/iso/installer]# cp tpot.conf.dist tpot.conf
[root@gladmattress:~/work/tpotce/iso/installer]# ./install.sh --type=auto --conf=tpot.conf

reboot > start T-pot (watch dps.sh)

[root@gladmattress:~]# docker exec -it elasticsearch sh
/ $ cd /usr/share/elasticsearch/config/
/usr/share/elasticsearch/config $ cat elasticsearch.yml
cluster.name: tpotcluster
node.name: "tpotcluster-node-01"
xpack.ml.enabled: false
xpack.security.enabled: false
#xpack.ilm.enabled: false
path:
    logs: /data/elk/log
    data: /data/elk/data
http.host: 0.0.0.0
http.cors.enabled: true
http.cors.allow-origin: "*"
indices.query.bool.max_clause_count: 2000
cluster.initial_master_nodes:
- "tpotcluster-node-01"
discovery.zen.ping.unicast.hosts:
- localhost

Is there something wrong with the file I've edited?
Thanks.

[t3chn0m4g3]

You only change the source for building the docker images. You either need to work with Volumes (see our Wiki) or you need to build your own images.

[tadashi-oya]

Thank you for your advice.

I have done as follows with reference to Wiki pages "Reconfigure ews.cfg" and "Reconfigure logstash.conf".

# docker exec -it elasticsearch ash

$ id
uid=2000(elasticsearch) gid=2000(elasticsearch)
$ cd /usr/share/elasticsearch/config
$ ls -l elasticsearch.yml
-rw-r--r--    1 elastics elastics       408 Oct 26 14:02 elasticsearch.yml
$ cp elasticsearch.yml /data/elk/elasticsearch.yml
$ exit

# systemctl stop tpot

# vi /data/elk/elasticsearch.yml
cluster.name: tpotcluster
node.name: "tpotcluster-node-01"
xpack.ml.enabled: false
xpack.security.enabled: true <<<MODIFY HERE
#xpack.ilm.enabled: false
path:
    logs: /data/elk/log
    data: /data/elk/data
http.host: 0.0.0.0
http.cors.enabled: true
http.cors.allow-origin: "*"
indices.query.bool.max_clause_count: 2000
cluster.initial_master_nodes:
- "tpotcluster-node-01"
discovery.zen.ping.unicast.hosts:
- localhost

# chmod 644 /data/elk/elasticsearch.yml
# chown tpot:tpot /data/elk/elasticsearch.yml

# vi /opt/tpot/etc/tpot.yml
[...]
#### ELK
## Elasticsearch service
  elasticsearch:
    container_name: elasticsearch
    restart: always
    environment:
     - bootstrap.memory_lock=true
#     - ES_JAVA_OPTS=-Xms2048m -Xmx2048m
     - ES_TMPDIR=/tmp
    cap_add:
     - IPC_LOCK
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
#    mem_limit: 4g
    ports:
     - "127.0.0.1:64298:9200"
    image: "dtagdevsec/elasticsearch:2006"
    volumes:
     - /data:/data
     - /data/elk/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
[...]

# systemctl start tpot

And I got

# telnet localhost 64298
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
GET / HTTP/1.0

HTTP/1.0 401 Unauthorized
WWW-Authenticate: Basic realm="security" charset="UTF-8"
content-type: application/json; charset=UTF-8
content-length: 381

{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":"Basic realm=\"security\" charset=\"UTF-8\""}},"status":401}Connection closed by foreign host.

as I expected.

Thanks again.