Failed to upgrade cf from 212 to 233 (etcd_z1/1 failed)


王小锋 <zzuwxf at gmail.com...>
 

Many thanks !!

2016-04-07 9:55 GMT+08:00 Amit Gupta <agupta(a)pivotal.io>:

Hi there,

Since you've also asked this question on the cf-bosh mailing list:


https://lists.cloudfoundry.org/archives/list/cf-bosh(a)lists.cloudfoundry.org/thread/KPJNZSVAAOQEFLME4BERHL26DDBTDNXJ/

and have provided additional details there, I will ask the core team
responsible for etcd-release to answer your question on the cf-bosh mailing
list.

Cheers,
Amit

On Tue, Apr 5, 2016 at 12:26 AM, 王小锋 <zzuwxf(a)gmail.com> wrote:

Hi, there

I am trying to upgrade cf from version 212 to latest version 233 using
bosh, I am using spiff to generate the new manifest file. During upgrade,
it failed when updating job etcd_z1/1.

I login to etcd_z1/1 VM and found the following error message in etcd
error log.
Anyone meet the same issue? your help is greatly appreciated !!

2016/04/05 06:36:16 filePurge: successfully removed file
/var/vcap/store/etcd/member/snap/00000000000000d6-0000000003712a07.snap
2016/04/05 06:36:16 filePurge: successfully removed file
/var/vcap/store/etcd/member/wal/000000000000168e-0000000003712a08.wal
2016/04/05 06:46:09 rafthttp: server streaming to d7734b019d58ad5c at
term 214 has been stopped
2016/04/05 06:46:09 etcdserver: removed member d7734b019d58ad5c from
cluster f61148e36337417a
2016/04/05 06:46:10 etcdserver: reject message from removed member
d7734b019d58ad5c
2016/04/05 06:46:18 etcdserver: added member e3682036054bdae0 [
http://10.10.16.20:7001] to cluster f61148e36337417a
2016/04/05 06:46:18 sender: error posting to e3682036054bdae0: dial tcp
10.10.16.20:7001: connection refused
2016/04/05 06:46:18 sender: the connection with e3682036054bdae0 became
inactive
2016/04/05 06:46:20 sender: the connection with e3682036054bdae0 became
active
2016/04/05 06:46:20 rafthttp: path msgapp/e3682036054bdae0 cannot be
parsed
2016/04/05 06:46:20 rafthttp: path message/e3682036054bdae0 cannot be
parsed
2016/04/05 06:46:20 raft: ddea47ef60bbcdca received msgApp
rejection(lastindex: 0) from e3682036054bdae0 for index 57802518
2016/04/05 06:46:20 raft: ddea47ef60bbcdca decreased progress of
e3682036054bdae0 to [next = 1, match = 0, wait = 0]
2016/04/05 06:46:20 raft: ddea47ef60bbcdca [firstindex: 57797981, commit:
57802541] sent snapshot[index: 57797980, term: 214] to e3682036054bdae0
[next = 1, match = 0, wait = 0]
2016/04/05 06:46:20 rafthttp: server streaming to e3682036054bdae0 at
term 0 has been stopped
2016/04/05 06:46:20 rafthttp: path message/e3682036054bdae0 cannot be
parsed
2016/04/05 06:46:20 rafthttp: path msgapp/e3682036054bdae0 cannot be
parsed
2016/04/05 06:46:20 rafthttp: path message/e3682036054bdae0 cannot be
parsed


Amit Kumar Gupta
 

Hi there,

Since you've also asked this question on the cf-bosh mailing list:

https://lists.cloudfoundry.org/archives/list/cf-bosh(a)lists.cloudfoundry.org/thread/KPJNZSVAAOQEFLME4BERHL26DDBTDNXJ/

and have provided additional details there, I will ask the core team
responsible for etcd-release to answer your question on the cf-bosh mailing
list.

Cheers,
Amit

On Tue, Apr 5, 2016 at 12:26 AM, 王小锋 <zzuwxf(a)gmail.com> wrote:

Hi, there

I am trying to upgrade cf from version 212 to latest version 233 using
bosh, I am using spiff to generate the new manifest file. During upgrade,
it failed when updating job etcd_z1/1.

I login to etcd_z1/1 VM and found the following error message in etcd
error log.
Anyone meet the same issue? your help is greatly appreciated !!

2016/04/05 06:36:16 filePurge: successfully removed file
/var/vcap/store/etcd/member/snap/00000000000000d6-0000000003712a07.snap
2016/04/05 06:36:16 filePurge: successfully removed file
/var/vcap/store/etcd/member/wal/000000000000168e-0000000003712a08.wal
2016/04/05 06:46:09 rafthttp: server streaming to d7734b019d58ad5c at term
214 has been stopped
2016/04/05 06:46:09 etcdserver: removed member d7734b019d58ad5c from
cluster f61148e36337417a
2016/04/05 06:46:10 etcdserver: reject message from removed member
d7734b019d58ad5c
2016/04/05 06:46:18 etcdserver: added member e3682036054bdae0 [
http://10.10.16.20:7001] to cluster f61148e36337417a
2016/04/05 06:46:18 sender: error posting to e3682036054bdae0: dial tcp
10.10.16.20:7001: connection refused
2016/04/05 06:46:18 sender: the connection with e3682036054bdae0 became
inactive
2016/04/05 06:46:20 sender: the connection with e3682036054bdae0 became
active
2016/04/05 06:46:20 rafthttp: path msgapp/e3682036054bdae0 cannot be parsed
2016/04/05 06:46:20 rafthttp: path message/e3682036054bdae0 cannot be
parsed
2016/04/05 06:46:20 raft: ddea47ef60bbcdca received msgApp
rejection(lastindex: 0) from e3682036054bdae0 for index 57802518
2016/04/05 06:46:20 raft: ddea47ef60bbcdca decreased progress of
e3682036054bdae0 to [next = 1, match = 0, wait = 0]
2016/04/05 06:46:20 raft: ddea47ef60bbcdca [firstindex: 57797981, commit:
57802541] sent snapshot[index: 57797980, term: 214] to e3682036054bdae0
[next = 1, match = 0, wait = 0]
2016/04/05 06:46:20 rafthttp: server streaming to e3682036054bdae0 at term
0 has been stopped
2016/04/05 06:46:20 rafthttp: path message/e3682036054bdae0 cannot be
parsed
2016/04/05 06:46:20 rafthttp: path msgapp/e3682036054bdae0 cannot be parsed
2016/04/05 06:46:20 rafthttp: path message/e3682036054bdae0 cannot be
parsed


王小锋 <zzuwxf at gmail.com...>
 

Hi, there

I am trying to upgrade cf from version 212 to latest version 233 using
bosh, I am using spiff to generate the new manifest file. During upgrade,
it failed when updating job etcd_z1/1.

I login to etcd_z1/1 VM and found the following error message in etcd error
log.
Anyone meet the same issue? your help is greatly appreciated !!

2016/04/05 06:36:16 filePurge: successfully removed file
/var/vcap/store/etcd/member/snap/00000000000000d6-0000000003712a07.snap
2016/04/05 06:36:16 filePurge: successfully removed file
/var/vcap/store/etcd/member/wal/000000000000168e-0000000003712a08.wal
2016/04/05 06:46:09 rafthttp: server streaming to d7734b019d58ad5c at term
214 has been stopped
2016/04/05 06:46:09 etcdserver: removed member d7734b019d58ad5c from
cluster f61148e36337417a
2016/04/05 06:46:10 etcdserver: reject message from removed member
d7734b019d58ad5c
2016/04/05 06:46:18 etcdserver: added member e3682036054bdae0 [
http://10.10.16.20:7001] to cluster f61148e36337417a
2016/04/05 06:46:18 sender: error posting to e3682036054bdae0: dial tcp
10.10.16.20:7001: connection refused
2016/04/05 06:46:18 sender: the connection with e3682036054bdae0 became
inactive
2016/04/05 06:46:20 sender: the connection with e3682036054bdae0 became
active
2016/04/05 06:46:20 rafthttp: path msgapp/e3682036054bdae0 cannot be parsed
2016/04/05 06:46:20 rafthttp: path message/e3682036054bdae0 cannot be parsed
2016/04/05 06:46:20 raft: ddea47ef60bbcdca received msgApp
rejection(lastindex: 0) from e3682036054bdae0 for index 57802518
2016/04/05 06:46:20 raft: ddea47ef60bbcdca decreased progress of
e3682036054bdae0 to [next = 1, match = 0, wait = 0]
2016/04/05 06:46:20 raft: ddea47ef60bbcdca [firstindex: 57797981, commit:
57802541] sent snapshot[index: 57797980, term: 214] to e3682036054bdae0
[next = 1, match = 0, wait = 0]
2016/04/05 06:46:20 rafthttp: server streaming to e3682036054bdae0 at term
0 has been stopped
2016/04/05 06:46:20 rafthttp: path message/e3682036054bdae0 cannot be parsed
2016/04/05 06:46:20 rafthttp: path msgapp/e3682036054bdae0 cannot be parsed
2016/04/05 06:46:20 rafthttp: path message/e3682036054bdae0 cannot be parsed