Date
1 - 8 of 8
Issue on re-deploy to AWS - static IP already reserved/in use
|
Ed
Hi BOSH users!
We're running into an error when attempting to re-deploy a failing BOSH deployment of CF to AWS: `ha_proxy_z1/0' asked for a static IP 52.28.90.12 but it's already reserved/in use We're essentially doing the following: * Allocate new EIP in AWS * Generate manifest for CF with the EIP assigned to HA Proxy * Run `bosh deploy` - this fails because we're hitting AWS API limits * Run `bosh deploy` again - this now fails due to the above 'already reserved' error Is this the expected behaviour? We've tried manually releasing the EIP in AWS and retrying the deployment, but we still experience the 'already reserved' failure. We're running BOSH version 1.2922.0 and have attached the relevant backtrace from the debug log. Any help appreciated. Thanks, Ed and Gareth aws-static-ip-already-reserved-debug.txt <http://cf-bosh.70367.x6.nabble.com/file/n10/aws-static-ip-already-reserved-debug.txt> -- View this message in context: http://cf-bosh.70367.x6.nabble.com/Issue-on-re-deploy-to-AWS-static-IP-already-reserved-in-use-tp10.html Sent from the CF BOSH mailing list archive at Nabble.com. |
|
|
|
Lev Berman
Have you terminated the previous Amazon HA Proxy instance? If not, this
toggle quoted message
Show quoted text
might be the reason of the reserved IP error. On Thu, May 7, 2015 at 1:11 PM, teddyking <ed(a)cloudcredo.com> wrote:
Hi BOSH users! --
Lev Berman Altoros - Cloud Foundry deployment, training and integration Github *: https://github.com/ldmberman <https://github.com/ldmberman>* |
|
|
|
Dmitriy Kalinin
Was that IP previously assigned to a different VM (check via `bosh vms`)?
toggle quoted message
Show quoted text
Currently BOSH does not allow to pick a static IP if it's already in use by another VM. You could try putting it as a reserved IP, then doing bosh deploy, and then assigning it as a static IP for the haproxy VM. On Thu, May 7, 2015 at 3:32 AM, Lev Berman <lev.berman(a)altoros.com> wrote:
Have you terminated the previous Amazon HA Proxy instance? If not, this |
|
|
|
Alexander Lomov
We had the similar issue with AWS. In AWS pannel we couldn't find any
toggle quoted message
Show quoted text
instance that used some IP address, still there was no ability to create an instance with such IP. We got this bug after we redeployed cluster several times. I think it could be AWS problem, so the solution here will be to put this IP addresses to the list of reserved IPs. ------------------------ Alex Lomov *Altoros* — Cloud Foundry deployment, training and integration *Twitter:* @code1n <https://twitter.com/code1n> *GitHub:* @allomov <https://gist.github.com/allomov> On Thu, May 7, 2015 at 8:10 PM, Dmitriy Kalinin <dkalinin(a)pivotal.io> wrote:
Was that IP previously assigned to a different VM (check via `bosh vms`)? |
|
|
|
Gareth Clay
Hi Dmitriy,
Thanks for the response and sorry for the slow reply! This is the output from 'bosh vms' when we're in the EIP clash situation: So it looks like BOSH knows that the HA proxy instance has already been created and allocated the EIP. When we attempt the same deployment again, I wondered if it would be possible for BOSH to determine 'okay, no work to do here - this HA proxy instance is already in the state we want' and move on, which is what it seems to do with instances which don't have EIPs attached? Thanks, Gareth and Ana -- View this message in context: http://cf-bosh.70367.x6.nabble.com/Re-cf-bosh-Issue-on-re-deploy-to-AWS-static-IP-already-reserved-in-use-tp11p24.html Sent from the CF BOSH mailing list archive at Nabble.com. |
|
|
|
Gareth Clay
Hi,
toggle quoted message
Show quoted text
Sorry, it looks like Nabble stripped out the ‘bosh vms’ output, which was: +-----------------+---------+---------------+---------------+ | Job/index | State | Resource Pool | IPs | +-----------------+---------+---------------+---------------+ | unknown/unknown | running | haproxy | 52.17.205.122 | | | | | 10.250.0.11 | | unknown/unknown | running | large_z1 | 10.250.66.32 | | unknown/unknown | running | large_z1 | 10.250.66.37 | | unknown/unknown | running | medium_z1 | 10.250.66.7 | | unknown/unknown | running | medium_z1 | 10.250.66.30 | | unknown/unknown | running | medium_z1 | 10.250.66.8 | | unknown/unknown | running | medium_z1 | 10.250.66.5 | | unknown/unknown | running | medium_z1 | 10.250.66.6 | | unknown/unknown | running | medium_z1 | 10.250.66.35 | | unknown/unknown | running | medium_z1 | 10.250.66.31 | | unknown/unknown | running | router_z1 | 10.250.66.9 | | unknown/unknown | running | runner_z1 | 10.250.66.36 | | unknown/unknown | running | small_z1 | 10.250.66.34 | | unknown/unknown | running | small_z1 | 10.250.66.38 | | unknown/unknown | running | small_z1 | 10.250.66.29 | +-----------------+---------+---------------+———————+ Gareth On 13 May 2015, at 11:11 am, garethclay <gareth(a)cloudcredo.com> wrote: |
|
|
|
Dmitriy Kalinin
if you have specified your elastic IP in the manifest associated with a
toggle quoted message
Show quoted text
deployment job, BOSH will make sure it's connected and will not do any additional work on the subsequent deploys. Btw I cannot see `bosh vms` output from your previous reply? Gist? On Wed, May 13, 2015 at 3:11 AM, garethclay <gareth(a)cloudcredo.com> wrote:
Hi Dmitriy, |
|
|
|
Gareth Clay
Hi Dmitriy,
toggle quoted message
Show quoted text
Thanks and sorry again for the slow response. Nabble stripped out my bosh vms output unfortunately - I’ve created a gist at https://gist.github.com/spikymonkey/9938dcdcc7a9a6097183 <https://gist.github.com/spikymonkey/9938dcdcc7a9a6097183> It sounds like the behaviour you described is exactly what we want. It seems though that we’ve hit a case where BOSH can’t determine that it has no additional work to do, and so goes off to do it again. This fails because the EIP connection has already been done on the first deployment attempt. Kind regards, Gareth On 13 May 2015, at 6:13 pm, Dmitriy Kalinin <dkalinin(a)pivotal.io> wrote: |
|
|