Re: App Container IP Address assignment on vSphere


Eric Malm <emalm@...>
 

Hi, Daya,

Based on
https://github.com/cloudfoundry/warden/blob/master/warden/lib/warden/config.rb#L207-L216,
the warden server uses the values of the network.pool_start_address and
network.pool_size properties from the rendered warden.yml config file to
construct a value for the pool_network property. Warden allocates a /30
subnet for each container, to have room for both the host-side and
container-side IP addresses in the veth pair, as well as the broadcast
address on the subnet. With the default values of 10.254.0.0 for the pool
start address and 256 (= 2^8) for the pool size, warden then calculates the
pool network to be 10.254.0.0/22. This /22 subnet includes the 10.254.2.x
and 10.254.3.x addresses you have observed on your DEAs.

In any case, these 10.254.x.y IP addresses are used only internally on each
DEA or Diego cell VM, so there's no conflict between these IP addresses on
other VMs that run warden/garden containers. If you examine the 'nat' table
in the iptables config, you'll see that for each container, warden creates
a NAT rule that directs inbound traffic from a particular port on the host
VM's eth0 interface to that same port on the container's host-side veth
interface (the one with offset 2 in the container's /30 subnet). The DEA
then provides this port as the value of the $PORT environment variable, so
the CF app process running in the container can listen on that port for its
web traffic.

Thanks,
Eric

On Wed, Dec 9, 2015 at 11:25 PM, Will Pragnell <wpragnell(a)pivotal.io> wrote:

Ah, sorry, my bad! I assumed Garden for some reason.

On 9 December 2015 at 21:15, Daya Shetty <daya.shetty(a)bnymellon.com>
wrote:

Will,

We are using warden containers in our deployment and I was referring to
the attributes defined in

./cf-release/jobs/dea_next/templates/warden.yml.erb

network:
pool_start_address: 10.254.0.0
pool_size: 256

and in ./cf-release/src/warden/warden/lib/warden/config.rb

def self.network_defaults
{
"pool_network" => "10.254.0.0/24",
"deny_networks" => [],
"allow_networks" => [],
"allow_host_access" => false,
"mtu" => 1500,
}
end

def self.network_schema
::Membrane::SchemaParser.parse do
{
# Preferred way to specify networks to pool
optional("pool_network") => String,

# Present for Backwards compatibility
optional("pool_start_address") => String,
optional("pool_size") => Integer,
optional("release_delay") => Integer,
optional("mtu") => Integer,

"deny_networks" => [String],
"allow_networks" => [String],
optional("allow_host_access") => bool,
}
end

Thanks
Daya

Join cf-dev@lists.cloudfoundry.org to automatically receive all group messages.