App deployment hangs in legacy CF installation
We are running on an extremely old version of CF (we are in the process of
building one based on the latest), so I know there is very little the
community may be able to help.
But regardless... let me give it a try.
In my debug session, I tried to deploy a hello world app, and deployment
stopped with "STARTED" and eventually timeout.
The full log:
I can easily reproduce this when I did two concurrent push. Sometimes they
go through, sometimes they don't.
We have looked at every log in CF and we don't have any lead. I did bosh
restart JOB hoping it was caused by a slow process, but that did not help.
I found ntp was not installed on some of the components (we installed ntp
on all of the DEAs), and i found clock was not synced so I synced the
clocked, and still no help.
Any idea where I should look at? I thought about our EC2 instance health
but all of them seem to be healthy. I am considering relaunching (bosh
recreate) one component at a time.
The one thing I did notice is I am constantly deploying to a couple DEAs. I
will look into them but I am not sure...
Any ideas will be appreciated. Thanks.