Deploying cf-170 on Openstack fails


Dimuthu Upeksha
 

I'm deploying cf-170 on Openstack and got following errors when I run bosh deploy. What could be the reason for this?

Failed: Action Failed get_task: Task 51bf20b6-b866-4fe4-645e-b54229a384f1 result: Compiling package rootfs_lucid64: Fetching package rootfs_lucid64: Fetching package blob 025f4797-e59b-4d2f-a282-4f7ec11ab48c: Getting blob from inner blobstore: Getting blob from inner blobstore: Shelling out to bosh-blobstore-dav cli: Running command: 'bosh-blobstore-dav -c /var/vcap/bosh/etc/blobstore-dav.json get 025f4797-e59b-4d2f-a282-4f7ec11ab48c /var/vcap/data/tmp/bosh-blobstore-externalBlobstore-Get132415760', stdout: 'Error running app - flate: read error at offset 138270962: http: unexpected EOF reading trailer', stderr: '': exit status 1 (00:23:06)

Error 450001: Action Failed get_task: Task 51bf20b6-b866-4fe4-645e-b54229a384f1 result: Compiling package rootfs_lucid64: Fetching package rootfs_lucid64: Fetching package blob 025f4797-e59b-4d2f-a282-4f7ec11ab48c: Getting blob from inner blobstore: Getting blob from inner blobstore: Shelling out to bosh-blobstore-dav cli: Running command: 'bosh-blobstore-dav -c /var/vcap/bosh/etc/blobstore-dav.json get 025f4797-e59b-4d2f-a282-4f7ec11ab48c /var/vcap/data/tmp/bosh-blobstore-externalBlobstore-Get132415760', stdout: 'Error running app - flate: read error at offset 138270962: http: unexpected EOF reading trailer', stderr: '': exit status 1

Thanks
Dimuthu


Madireddy Samuel Vijaykumar <mad.vijay@...>
 

Hi DU.

Would be helpful to have more information?
- What version of openstack?
- Which stemcell?
- More log output!

--- Sam

-----

samuel vijaykumar
http://megam.in | this or samof76(a)gmail.com | +91-9940612141
b14 atandra. 25 thirumalai pillai road. t nagar. chennai 600 017


*all hard work brings a profit, but mere talk leads only to poverty. --
from the Bible*

-----

On Wed, Nov 11, 2015 at 1:53 PM, Dimuthu Upeksha <dimuthu(a)adroitlogic.com>
wrote:

I'm deploying cf-170 on Openstack and got following errors when I run
bosh deploy. What could be the reason for this?

Failed: Action Failed get_task: Task 51bf20b6-b866-4fe4-645e-b54229a384f1
result: Compiling package rootfs_lucid64: Fetching package rootfs_lucid64:
Fetching package blob 025f4797-e59b-4d2f-a282-4f7ec11ab48c: Getting blob
from inner blobstore: Getting blob from inner blobstore: Shelling out to
bosh-blobstore-dav cli: Running command: 'bosh-blobstore-dav -c
/var/vcap/bosh/etc/blobstore-dav.json get
025f4797-e59b-4d2f-a282-4f7ec11ab48c
/var/vcap/data/tmp/bosh-blobstore-externalBlobstore-Get132415760', stdout:
'Error running app - flate: read error at offset 138270962: http:
unexpected EOF reading trailer', stderr: '': exit status 1 (00:23:06)

Error 450001: Action Failed get_task: Task
51bf20b6-b866-4fe4-645e-b54229a384f1 result: Compiling package
rootfs_lucid64: Fetching package rootfs_lucid64: Fetching package blob
025f4797-e59b-4d2f-a282-4f7ec11ab48c: Getting blob from inner blobstore:
Getting blob from inner blobstore: Shelling out to bosh-blobstore-dav cli:
Running command: 'bosh-blobstore-dav -c
/var/vcap/bosh/etc/blobstore-dav.json get
025f4797-e59b-4d2f-a282-4f7ec11ab48c
/var/vcap/data/tmp/bosh-blobstore-externalBlobstore-Get132415760', stdout:
'Error running app - flate: read error at offset 138270962: http:
unexpected EOF reading trailer', stderr: '': exit status 1

Thanks
Dimuthu


Dimuthu Upeksha
 

Hi Samuel,

Openstack - Juno installed using Devstack
Stemcell - bosh-openstack-kvm-ubuntu-trusty-go_agent/3120

I stopped the deployment and re-run it then that error was not occurred again but the deployment looks hanged. Following log can be repeatedly seen when I see debug logs.

sk:21-checkpoint] DEBUG -- DirectorJobRunner: (0.006830s) BEGIN
D, [2011-01-01 07:55:13 #20484] [task:21-checkpoint] DEBUG -- DirectorJobRunner: (0.009821s) UPDATE "tasks" SET "checkpoint_time" = '2011-01-01 07:55:13.831474+0000' WHERE ("id" = 21)
D, [2011-01-01 07:55:13 #20484] [task:21-checkpoint] DEBUG -- DirectorJobRunner: (0.047817s) COMMIT
D, [2011-01-01 07:55:14 #20484] [compile_package(ruby/12, bosh-openstack-kvm-ubuntu-trusty-go_agent/3120)] DEBUG -- DirectorJobRunner: SENT: agent.8033542e-f473-417f-85cd-d79575d0f632 {"protocol":2,"method":"ping","arguments":[],"reply_to":"director.e81a5d5e-85e5-4d3b-9a70-61d2527450d2.9fc94192-d859-4b2e-bc47-e3e359066e7a"}
D, [2011-01-01 07:55:15 #20484] [] DEBUG -- DirectorJobRunner: Renewing lock: lock:deployment:cf
D, [2011-01-01 07:55:15 #20484] [compile_package(ruby/12, bosh-openstack-kvm-ubuntu-trusty-go_agent/3120)] DEBUG -- DirectorJobRunner: SENT: agent.8033542e-f473-417f-85cd-d79575d0f632 {"protocol":2,"method":"ping","arguments":[],"reply_to":"director.e81a5d5e-85e5-4d3b-9a70-61d2527450d2.cf68eb0d-e8e6-4deb-8b6f-93ad454decfe"}
D, [2011-01-01 07:55:16 #20484] [compile_package(ruby/12, bosh-openstack-kvm-ubuntu-trusty-go_agent/3120)] DEBUG -- DirectorJobRunner: SENT: agent.8033542e-f473-417f-85cd-d79575d0f632 {"protocol":2,"method":"ping","arguments":[],"reply_to":"director.e81a5d5e-85e5-4d3b-9a70-61d2527450d2.b8471016-7af4-4ea8-9fe1-72ce3f81a69a"}
D, [2011-01-01 07:55:17 #20484] [] DEBUG -- DirectorJobRunner: Renewing lock: lock:compile:70:1
D, [2011-01-01 07:55:17 #20484] [compile_package(ruby/12, bosh-openstack-kvm-ubuntu-trusty-go_agent/3120)] DEBUG -- DirectorJobRunner: SENT: agent.8033542e-f473-417f-85cd-d79575d0f632 {"protocol":2,"method":"ping","arguments":[],"reply_to":"director.e81a5d5e-85e5-4d3b-9a70-61d2527450d2.89625e46-c1be-49ca-a9e0-6006174f90f9"}
D, [2011-01-01 07:55:18 #20484] [compile_package(ruby/12, bosh-openstack-kvm-ubuntu-trusty-go_agent/3120)] DEBUG -- DirectorJobRunner: SENT: agent.8033542e-f473-417f-85cd-d79575d0f632 {"protocol":2,"method":"ping","arguments":[],"reply_to":"director.e81a5d5e-85e5-4d3b-9a70-61d2527450d2.3e1aebca-37d6-41ae-aaf1-3b9e4057f16f"}
D, [2011-01-01 07:55:19 #20484] [compile_package(ruby/12, bosh-openstack-kvm-ubuntu-trusty-go_agent/3120)] DEBUG -- DirectorJobRunner: SENT: agent.8033542e-f473-417f-85cd-d79575d0f632 {"protocol":2,"method":"ping","arguments":[],"reply_to":"director.e81a5d5e-85e5-4d3b-9a70-61d2527450d2.d7336ee8-404c-4eb9-9ace-ab4af6ed251a"}
D, [2011-01-01 07:55:20 #20484] [] DEBUG -- DirectorJobRunner: Renewing lock: lock:deployment:cf
D, [2011-01-01 07:55:20 #20484] [compile_package(ruby/12, bosh-openstack-kvm-ubuntu-trusty-go_agent/3120)] DEBUG -- DirectorJobRunner: SENT: agent.8033542e-f473-417f-85cd-d79575d0f632 {"protocol":2,"method":"ping","arguments":[],"reply_to":"director.e81a5d5e-85e5-4d3b-9a70-61d2527450d2.b4c5b4b6-75cb-446c-b416-253d1467d48a"}
D, [2011-01-01 07:55:21 #20484] [compile_package(ruby/12, bosh-openstack-kvm-ubuntu-trusty-go_agent/3120)] DEBUG -- DirectorJobRunner: SENT: agent.8033542e-f473-417f-85cd-d79575d0f632 {"protocol":2,"method":"ping","arguments":[],"reply_to":"director.e81a5d5e-85e5-4d3b-9a70-61d2527450d2.00243912-dbfc-4996-b041-d22564d2ec76"}
D, [2011-01-01 07:55:22 #20484] [] DEBUG -- DirectorJobRunner: Renewing lock: lock:compile:70:1
D, [2011-01-01 07:55:22 #20484] [compile_package(ruby/12, bosh-openstack-kvm-ubuntu-trusty-go_agent/3120)] DEBUG -- DirectorJobRunner: SENT: agent.8033542e-f473-417f-85cd-d79575d0f632 {"protocol":2,"method":"ping","arguments":[],"reply_to":"director.e81a5d5e-85e5-4d3b-9a70-61d2527450d2.57cf3b12-c8a3-405a-94f0-be6cd1ab698a"}
D, [2011-01-01 07:55:23 #20484] [compile_package(ruby/12, bosh-openstack-kvm-ubuntu-trusty-go_agent/3120)] DEBUG -- DirectorJobRunner: SENT: agent.8033542e-f473-417f-85cd-d79575d0f632 {"protocol":2,"method":"ping","arguments":[],"reply_to":"director.e81a5d5e-85e5-4d3b-9a70-61d2527450d2.ee5df93f-aeb0-4794-975b-7fab73503ed0"}
D, [2011-01-01 07:55:24 #20484] [compile_package(ruby/12, bosh-openstack-kvm-ubuntu-trusty-go_agent/3120)] DEBUG -- DirectorJobRunner: SENT: agent.8033542e-f473-417f-85cd-d79575d0f632 {"protocol":2,"method":"ping","arguments":[],"reply_to":"director.e81a5d5e-85e5-4d3b-9a70-61d2527450d2.61f349ba-2fa0-4222-bc1f-38c08ab1f4c5"}
D, [2011-01-01 07:55:25 #20484] [] DEBUG -- DirectorJobRunner: Renewing lock: lock:deployment:cf
D, [2011-01-01 07:55:25 #20484] [compile_package(ruby/12, bosh-openstack-kvm-ubuntu-trusty-go_agent/3120)] DEBUG -- DirectorJobRunner: SENT: agent.8033542e-f473-417f-85cd-d79575d0f632 {"protocol":2,"method":"ping","arguments":[],"reply_to":"director.e81a5d5e-85e5-4d3b-9a70-61d2527450d2.40431f20-f221-4aba-aa5d-d51a43e4526c"}
D, [2011-01-01 07:55:26 #20484] [compile_package(ruby/12, bosh-openstack-kvm-ubuntu-trusty-go_agent/3120)] DEBUG -- DirectorJobRunner: SENT: agent.8033542e-f473-417f-85cd-d79575d0f632 {"protocol":2,"method":"ping","arguments":[],"reply_to":"director.e81a5d5e-85e5-4d3b-9a70-61d2527450d2.4dd00472-3763-45ed-9fa1-09c3f1452547"}
D, [2011-01-01 07:55:27 #20484] [] DEBUG -- DirectorJobRunner: Renewing lock: lock:compile:70:1
D, [2011-01-01 07:55:27 #20484] [compile_package(ruby/12, bosh-openstack-kvm-ubuntu-trusty-go_agent/3120)] DEBUG -- DirectorJobRunner: SENT: agent.8033542e-f473-417f-85cd-d79575d0f632 {"protocol":2,"method":"ping","arguments":[],"reply_to":"director.e81a5d5e-85e5-4d3b-9a70-61d2527450d2.a9f709e3-28f6-427c-b468-14891d610cf6"}
D, [2011-01-01 07:55:28 #20484] [compile_package(ruby/12, bosh-openstack-kvm-ubuntu-trusty-go_agent/3120)] DEBUG -- DirectorJobRunner: SENT: agent.8033542e-f473-417f-85cd-d79575d0f632 {"protocol":2,"method":"ping","arguments":[],"reply_to":"director.e81a5d5e-85e5-4d3b-9a70-61d2527450d2.3e3c8d74-1ea1-4ff0-ac35-d18da52670ab"}
D, [2011-01-01 07:55:29 #20484] [compile_package(ruby/12, bosh-openstack-kvm-ubuntu-trusty-go_agent/3120)] DEBUG -- DirectorJobRunner: SENT: agent.8033542e-f473-417f-85cd-d79575d0f632 {"protocol":2,"method":"ping","arguments":[],"reply_to":"director.e81a5d5e-85e5-4d3b-9a70-61d2527450d2.bbf29d80-a51c-41c5-85c2-10899a793981"}
D, [2011-01-01 07:55:30 #20484] [] DEBUG -- DirectorJobRunner: Renewing lock: lock:deployment:cf
D, [2011-01-01 07:55:31 #20484] [compile_package(ruby/12, bosh-openstack-kvm-ubuntu-trusty-go_agent/3120)] DEBUG -- DirectorJobRunner: SENT: agent.8033542e-f473-417f-85cd-d79575d0f632 {"protocol":2,"method":"ping","arguments":[],"reply_to":"director.e81a5d5e-85e5-4d3b-9a70-61d2527450d2.e719224d-1264-4e51-b4bf-3e4553fbaaae"}

Thanks
Dimuthu


Dmitriy Kalinin
 

i believe we have seen this error when mtu is set to 1500 when running on openstack. try chnaging your dhcp server to set mtu to 1400.

Sent from my iPhone

On Nov 11, 2015, at 12:23 AM, Dimuthu Upeksha <dimuthu(a)adroitlogic.com> wrote:

I'm deploying cf-170 on Openstack and got following errors when I run bosh deploy. What could be the reason for this?

Failed: Action Failed get_task: Task 51bf20b6-b866-4fe4-645e-b54229a384f1 result: Compiling package rootfs_lucid64: Fetching package rootfs_lucid64: Fetching package blob 025f4797-e59b-4d2f-a282-4f7ec11ab48c: Getting blob from inner blobstore: Getting blob from inner blobstore: Shelling out to bosh-blobstore-dav cli: Running command: 'bosh-blobstore-dav -c /var/vcap/bosh/etc/blobstore-dav.json get 025f4797-e59b-4d2f-a282-4f7ec11ab48c /var/vcap/data/tmp/bosh-blobstore-externalBlobstore-Get132415760', stdout: 'Error running app - flate: read error at offset 138270962: http: unexpected EOF reading trailer', stderr: '': exit status 1 (00:23:06)

Error 450001: Action Failed get_task: Task 51bf20b6-b866-4fe4-645e-b54229a384f1 result: Compiling package rootfs_lucid64: Fetching package rootfs_lucid64: Fetching package blob 025f4797-e59b-4d2f-a282-4f7ec11ab48c: Getting blob from inner blobstore: Getting blob from inner blobstore: Shelling out to bosh-blobstore-dav cli: Running command: 'bosh-blobstore-dav -c /var/vcap/bosh/etc/blobstore-dav.json get 025f4797-e59b-4d2f-a282-4f7ec11ab48c /var/vcap/data/tmp/bosh-blobstore-externalBlobstore-Get132415760', stdout: 'Error running app - flate: read error at offset 138270962: http: unexpected EOF reading trailer', stderr: '': exit status 1

Thanks
Dimuthu


Rubab Syed
 

I am experiencing the same issue. Did you solve it?


Dmitriy Kalinin
 

Rubab: did you try setting the mtu to a lower number as suggested above?

On Mon, Nov 23, 2015 at 2:23 AM, Rubab Syed <rubab.syed21(a)gmail.com> wrote:

I am experiencing the same issue. Did you solve it?