Problem with missing routes due to recent DHCP -> static change


Aaron Huber
 

We were testing a newer version of the stemcells along with bosh-init in our
lab on OpenStack and ran into an unexpected issue. Our SDN configuration
requires static routes to be added via DHCP to get to the metadata web
service (169.254.169.254). We have BOSH configured to use manual IP
addresses but previously the stemcells were still configured to use DHCP in
that case, so the routes were working fine.

In the new configuration, the BOSH agent is manually configuring the IP
settings (switching from DHCP to static) which a good thing for stability,
but we are losing the route. This results in a failed deployment because
the agent continuously pings the user-data in the metadata web service in
order to get the registry URL to fetch updated configuration, and the agent
is never able to mount a persistent disk because it never gets the updated
configuration.

I don't think having the static routes is an unusual configuration for
OpenStack, so at the moment using "manual" addressing is not going to work
in that case. Can logic be added to also convert any static routes picked
up via DHCP when the agent switches the network config to static?

Aaron Huber
Intel Corporation



--
View this message in context: http://cf-bosh.70367.x6.nabble.com/Problem-with-missing-routes-due-to-recent-DHCP-static-change-tp105.html
Sent from the CF BOSH mailing list archive at Nabble.com.


Dmitriy Kalinin
 

Ah that's interesting. Are you seeing that network manager unsets the
static routes when eth0 or some other interface gets reloaded?

On Mon, Jun 1, 2015 at 10:51 AM, aaron_huber <aaron.m.huber(a)intel.com>
wrote:

We were testing a newer version of the stemcells along with bosh-init in
our
lab on OpenStack and ran into an unexpected issue. Our SDN configuration
requires static routes to be added via DHCP to get to the metadata web
service (169.254.169.254). We have BOSH configured to use manual IP
addresses but previously the stemcells were still configured to use DHCP in
that case, so the routes were working fine.

In the new configuration, the BOSH agent is manually configuring the IP
settings (switching from DHCP to static) which a good thing for stability,
but we are losing the route. This results in a failed deployment because
the agent continuously pings the user-data in the metadata web service in
order to get the registry URL to fetch updated configuration, and the agent
is never able to mount a persistent disk because it never gets the updated
configuration.

I don't think having the static routes is an unusual configuration for
OpenStack, so at the moment using "manual" addressing is not going to work
in that case. Can logic be added to also convert any static routes picked
up via DHCP when the agent switches the network config to static?

Aaron Huber
Intel Corporation



--
View this message in context:
http://cf-bosh.70367.x6.nabble.com/Problem-with-missing-routes-due-to-recent-DHCP-static-change-tp105.html
Sent from the CF BOSH mailing list archive at Nabble.com.
_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh


Aaron Huber
 

Yes, once the /etc/network/interfaces file is converted to "static" and it
does an ifdown/ifup then the route disappears because it is no longer being
added by the DHCP client. Technically I think the best solution would be to
just add any routes that were configured in DHCP to the interfaces file (at
least on Ubuntu, see
http://askubuntu.com/questions/548940/add-static-route-in-ubuntu-14-04).

I was just poking around on the best place to find the info. The
/var/lib/dhcp/dhclient.eth0.leases file will contain an entry like the
following that specifies the route information retrieved from DHCP:

option rfc3442-classless-static-routes 32,169,254,169,254,10,65,25,10;

That would be equivalent to:

post-up route add 169.254.169.254/32 gw 10.65.25.10

Aaron



--
View this message in context: http://cf-bosh.70367.x6.nabble.com/Problem-with-missing-routes-due-to-recent-DHCP-static-change-tp105p107.html
Sent from the CF BOSH mailing list archive at Nabble.com.