While deploying logsearch-boshrelease I'm consistently getting Error 400007: `ingestor/0' is not running after update.

bosh releases
Acting as user 'admin' on 'Bosh Lite Director'

+-----------+----------+-------------+
| Name | Versions | Commit Hash |
+-----------+----------+-------------+
| cf | 221* | 4d3779e2+ |
| logsearch | 23.0.0* | 871cb8d5 |
+-----------+----------+‹‹‹‹‹‹+

bosh vms
Deployment `logsearch'

Director task 50

Task 50 done

+----------------------------+---------+---------------+-------------+
| Job/index | State | Resource Pool | IPs |
+----------------------------+---------+---------------+-------------+
| api/0 | running | warden | 10.244.2.2 |
| elasticsearch_autoscale/0 | running | warden | 10.244.2.22 |
| elasticsearch_persistent/0 | running | warden | 10.244.2.6 |
| ingestor/0 | failing | warden | 10.244.2.14 |
| parser/0 | running | warden | 10.244.2.18 |
| queue/0 | running | warden | 10.244.2.10 |
+----------------------------+---------+---------------+-------------+

VMs total: 6

On the ingestor VM checked the ingestor_lumberjack.stdout.log shows a lot of the following
The error reported is: uninitialized constant Concurrent::Delay::Executor

Also on the queue VM I don¹t see any redis processes running which makes me wonder if the bosh-lite.yml I used for deployment is either incomplete or not correct.

Any ideas on what should I look next to resolve this?

Thanks
Daya
PS: When I deploy logsearch v22.0 I don't see this problem.

There's a bug in the lumberjack plugin which logsearch/logstash is using in
the v23.0.0 release (some details in
https://github.com/elastic/logstash/issues/3776).

If you don't specifically use the lumberjack logging protocol in your
deployment, just remove the ingestor_lumberjack template from your ingestor
job. Logsearch is planning on dropping the lumberjack ingestor soon anyways
(https://github.com/logsearch/logsearch-boshrelease/issues/182).

If you really do need lumberjack, you can test manually upgrading logstash
by running the following. Then leave a comment on
https://github.com/logsearch/logsearch-boshrelease/issues/182 to try and
request lumberjack gets fixed vs dropped in the next logsearch release.

$ JAVA_HOME=/var/vcap/packages/java8
/var/vcap/packages/logstash/logstash/bin/plugin update
logstash-input-lumberjack
Updated logstash-input-lumberjack 1.0.4 to 1.0.6

For further logsearch-specific help, you might want to create an issue on
logsearch's GitHub project -
https://github.com/logsearch/logsearch-boshrelease

Danny

On Mon, Oct 26, 2015 at 9:53 AM, Daya Shetty <daya.shetty(a)bnymellon.com>
wrote:

While deploying logsearch-boshrelease I'm consistently getting Error
400007: `ingestor/0' is not running after update.

bosh releases
Acting as user 'admin' on 'Bosh Lite Director'

+-----------+----------+-------------+
| Name | Versions | Commit Hash |
+-----------+----------+-------------+
| cf | 221* | 4d3779e2+ |
| logsearch | 23.0.0* | 871cb8d5 |
+-----------+----------+‹‹‹‹‹‹+

bosh vms
Deployment `logsearch'

Director task 50

Task 50 done

+----------------------------+---------+---------------+-------------+
| Job/index | State | Resource Pool | IPs |
+----------------------------+---------+---------------+-------------+
| api/0 | running | warden | 10.244.2.2 |
| elasticsearch_autoscale/0 | running | warden | 10.244.2.22 |
| elasticsearch_persistent/0 | running | warden | 10.244.2.6 |
| ingestor/0 | failing | warden | 10.244.2.14 |
| parser/0 | running | warden | 10.244.2.18 |
| queue/0 | running | warden | 10.244.2.10 |
+----------------------------+---------+---------------+-------------+

VMs total: 6

On the ingestor VM checked the ingestor_lumberjack.stdout.log shows a lot
of the following
The error reported is: uninitialized constant Concurrent::Delay::Executor

Also on the queue VM I don¹t see any redis processes running which makes
me wonder if the bosh-lite.yml I used for deployment is either incomplete
or not correct.

Any ideas on what should I look next to resolve this?

Thanks
Daya
PS: When I deploy logsearch v22.0 I don't see this problem.

--
Danny Berger
http://dpb587.me

Thanks for the response Danny!

I will stick with v22 as I don't use lumberjack.