Stuck at setup_logstash ca.cert

[axi92]

It get stuck every time here, any ideas on how to fix or debug this?
Client: Docker Engine - Community
Version: 19.03.2

docker-compose version 1.24.1, build 4667896b
docker-py version: 3.7.3
CPython version: 3.6.8
OpenSSL version: OpenSSL 1.1.0j 20 Nov 2018

$ docker-compose -f setup.yml up --remove-orphans
Removing orphan container "setup_logstash"
Removing orphan container "setup_kibana"
Starting stack-docker_setup_1 ... done
Attaching to stack-docker_setup_1
setup_1  | Found orphan containers (stack-docker_setup_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Recreating setup_elasticsearch ... done
Attaching to setup_elasticsearch
setup_1  | setup_elasticsearch    | Determining if x-pack is installed...
setup_1  | setup_elasticsearch    | === CREATE Keystore ===
setup_1  | setup_elasticsearch    | Elastic password is: Uh68Mb/HP5Nk+7FJE4ov9w==
setup_1  | setup_elasticsearch    | Created elasticsearch keystore in /usr/share/elasticsearch/config
setup_1  | setup_elasticsearch    | Setting bootstrap.password...
setup_1  | setup_elasticsearch    | === CREATE SSL CERTS ===
setup_1  | setup_elasticsearch    | Remove old ca zip...
setup_1  | setup_elasticsearch    | Creating docker-cluster-ca.zip...
setup_1  | setup_elasticsearch    | CA directory exists, removing...
setup_1  | setup_elasticsearch    | Unzip ca files...
setup_1  | setup_elasticsearch    | Archive:  /config/ssl/docker-cluster-ca.zip
setup_1  | setup_elasticsearch    |    creating: /config/ssl/ca/
setup_1  | setup_elasticsearch    |   inflating: /config/ssl/ca/ca.crt   
setup_1  | setup_elasticsearch    |   inflating: /config/ssl/ca/ca.key   
setup_1  | setup_elasticsearch    | Remove old docker-cluster.zip zip...
setup_1  | setup_elasticsearch    | Create cluster certs zipfile...
setup_1  | setup_elasticsearch    | Unzipping cluster certs zipfile...
setup_1  | setup_elasticsearch    | Archive:  /config/ssl/docker-cluster.zip
setup_1  | setup_elasticsearch    |    creating: /config/ssl/docker-cluster/elasticsearch/
setup_1  | setup_elasticsearch    |   inflating: /config/ssl/docker-cluster/elasticsearch/elasticsearch.crt  
setup_1  | setup_elasticsearch    |   inflating: /config/ssl/docker-cluster/elasticsearch/elasticsearch.key  
setup_1  | setup_elasticsearch    |    creating: /config/ssl/docker-cluster/kibana/
setup_1  | setup_elasticsearch    |   inflating: /config/ssl/docker-cluster/kibana/kibana.crt  
setup_1  | setup_elasticsearch    |   inflating: /config/ssl/docker-cluster/kibana/kibana.key  
setup_1  | setup_elasticsearch    |    creating: /config/ssl/docker-cluster/logstash/
setup_1  | setup_elasticsearch    |   inflating: /config/ssl/docker-cluster/logstash/logstash.crt  
setup_1  | setup_elasticsearch    |   inflating: /config/ssl/docker-cluster/logstash/logstash.key  
setup_1  | setup_elasticsearch    | Move logstash certs to logstash config dir...
setup_1  | setup_elasticsearch    | Move kibana certs to kibana config dir...
setup_1  | setup_elasticsearch    | Move elasticsearch certs to elasticsearch config dir...
setup_1  | setup_elasticsearch exited with code 0
setup_1  | Found orphan containers (stack-docker_setup_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Starting elasticsearch ... done
Creating setup_logstash ... done
Creating setup_kibana   ... done
Attaching to setup_kibana, setup_logstash
setup_1  | setup_kibana           | -rw-rw-r-- 1 root root 1200 Sep 24 05:45 /usr/share/kibana/config/ca/ca.crt
setup_1  | setup_logstash         | -rw-rw-r-- 1 root root 1200 Sep 24 05:45 /usr/share/logstash/config/ca/ca.crt

[AdityaChaudhary]

I faced the same issue. It was due to the shortage of memory.

1. In the case of virtual machine try increasing the RAM
2. Or set a large swap file in case of Linux

Run sysctl -w vm.max_map_count=262144 and then try running docker-compose -f setup.yml up

[axi92]

I have still 23GB of RAM free, so I don't think thats the Problem?
I have increased the max_map_count like you and still the same problem that it get stuck:
And everytime it finds orphan containers? But I have no containers left...

$ docker-compose -f setup.yml up
Creating network "stack-docker_default" with the default driver
Creating stack-docker_setup_1 ... done
Attaching to stack-docker_setup_1
setup_1  | Creating network "stack-docker_stack" with the default driver
setup_1  | Creating volume "stack-docker_es_data" with default driver
setup_1  | Found orphan containers (stack-docker_setup_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Creating setup_elasticsearch ... done
Attaching to setup_elasticsearch
setup_1  | setup_elasticsearch    | Determining if x-pack is installed...
setup_1  | setup_elasticsearch    | === CREATE Keystore ===
setup_1  | setup_elasticsearch    | Elastic password is: 2Mjbd8Cnhgl7ktM155axlA==
setup_1  | setup_elasticsearch    | Created elasticsearch keystore in /usr/share/elasticsearch/config
setup_1  | setup_elasticsearch    | Setting bootstrap.password...
setup_1  | setup_elasticsearch    | === CREATE SSL CERTS ===
setup_1  | setup_elasticsearch    | Remove old ca zip...
setup_1  | setup_elasticsearch    | Creating docker-cluster-ca.zip...
setup_1  | setup_elasticsearch    | CA directory exists, removing...
setup_1  | setup_elasticsearch    | Unzip ca files...
setup_1  | setup_elasticsearch    | Archive:  /config/ssl/docker-cluster-ca.zip
setup_1  | setup_elasticsearch    |    creating: /config/ssl/ca/
setup_1  | setup_elasticsearch    |   inflating: /config/ssl/ca/ca.crt   
setup_1  | setup_elasticsearch    |   inflating: /config/ssl/ca/ca.key   
setup_1  | setup_elasticsearch    | Remove old docker-cluster.zip zip...
setup_1  | setup_elasticsearch    | Create cluster certs zipfile...
setup_1  | setup_elasticsearch    | Unzipping cluster certs zipfile...
setup_1  | setup_elasticsearch    | Archive:  /config/ssl/docker-cluster.zip
setup_1  | setup_elasticsearch    |    creating: /config/ssl/docker-cluster/elasticsearch/
setup_1  | setup_elasticsearch    |   inflating: /config/ssl/docker-cluster/elasticsearch/elasticsearch.crt  
setup_1  | setup_elasticsearch    |   inflating: /config/ssl/docker-cluster/elasticsearch/elasticsearch.key  
setup_1  | setup_elasticsearch    |    creating: /config/ssl/docker-cluster/kibana/
setup_1  | setup_elasticsearch    |   inflating: /config/ssl/docker-cluster/kibana/kibana.crt  
setup_1  | setup_elasticsearch    |   inflating: /config/ssl/docker-cluster/kibana/kibana.key  
setup_1  | setup_elasticsearch    |    creating: /config/ssl/docker-cluster/logstash/
setup_1  | setup_elasticsearch    |   inflating: /config/ssl/docker-cluster/logstash/logstash.crt  
setup_1  | setup_elasticsearch    |   inflating: /config/ssl/docker-cluster/logstash/logstash.key  
setup_1  | setup_elasticsearch    | Move logstash certs to logstash config dir...
setup_1  | setup_elasticsearch    | Move kibana certs to kibana config dir...
setup_1  | setup_elasticsearch    | Move elasticsearch certs to elasticsearch config dir...
setup_1  | setup_elasticsearch exited with code 0
setup_1  | Found orphan containers (stack-docker_setup_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
Creating elasticsearch ... done
Creating setup_logstash ... done
Creating setup_kibana   ... done
Attaching to setup_logstash, setup_kibana
setup_1  | setup_logstash         | -rw-rw-r-- 1 root root 1200 Sep 26 09:23 /usr/share/logstash/config/ca/ca.crt
setup_1  | setup_kibana           | -rw-rw-r-- 1 root root 1200 Sep 26 09:23 /usr/share/kibana/config/ca/ca.crt

[AdityaChaudhary]

I solved my issue by debugging using custom logs throughout the scripts of this repo. Maybe you can give it a try, if not already done.

[axi92]

I solved my issue by debugging using custom logs throughout the scripts of this repo. Maybe you can give it a try, if not already done.

Thanks for this, I am stuck here: https://github.com/elastic/stack-docker/blob/master/scripts/setup-logstash.sh#L17

And this is my docker ps

CONTAINER ID        IMAGE                                       COMMAND                  CREATED             STATUS              PORTS                NAMES
c217e71c4ef3        docker.elastic.co/kibana/kibana:6.6.0       "/bin/bash -c 'cat /…"   19 seconds ago      Up 18 seconds       5601/tcp             setup_kibana
740cc6641580        docker.elastic.co/logstash/logstash:6.6.0   "/usr/local/bin/dock…"   19 seconds ago      Up 17 seconds       5044/tcp, 9600/tcp   setup_logstash
bdf1df993956        docker/compose:1.21.2                       "/bin/ash -c 'cat ./…"   15 minutes ago      Up 26 seconds                            stack-docker_setup_1

I cant find any elastic container?... But in the log it shows "Starting elasticsearch ... done"

[Mariyo]

Same for me, I got stucked here: docker-compose -p elastic-stack -f setup.yml up --remove-orphans

No elasticsearch container found:

docker ps | grep "elastic"
0f23210e693f        docker.elastic.co/kibana/kibana:6.6.0                "/bin/bash -c 'cat /…"   About a minute ago   Up 57 seconds       5601/tcp                                         setup_kibana
ea8367fe5091        docker.elastic.co/logstash/logstash:6.6.0            "/usr/local/bin/dock…"   About a minute ago   Up 58 seconds       5044/tcp, 9600/tcp                               setup_logstash
f1656bd4d552        docker/compose:1.21.2   

[sangwoo-joh]

I'm stuck here too.
I've just check-outed the old stable branch..