cf-dev@lists.cloudfoundry.org | *.cf.internal on diego windows

Edward Hill

I've been trying out the windows cell by having a bosh lite VM and a Windows 2012 VM locally on my machine and have hit a stumbling block that highlighted I don't know enough

My bosh lite VM is up and running with the windows stack added and my 2012 VM's event log shows it joining the cluster and reporting back details about the box

If I try and start the NET-sample-app in bosh lite it creates the container on the 2012 VM but then fails downloading the buildpack

If I look in the event logs it's failing to get to:

file-server.service.cf.internal

to download the windows_app_lifecycle.tgz file

I can't see how Diego Windows or Garden Windows is going to direct these urls that bosh is providing to the consul agent on the 2012 box or even the consul VM on bosh-lite.

Have I missed something? including misunderstanding consul :-)

Edward Hill

#3497

2 seconds after hitting send I remembered the DNS section in setup.ps1

I expect that 

$newDNS = @("127.0.0.1") + $currentDNS

has something to do with it, need to see where that setting went I think

Edward Hill

#3498

setting had gone

added it back on and app deployed \o/

Matthew Horan

#3511

That's it. localhost needs to be the first DNS server in the list for any
interface on the host. Otherwise, name resolution routines won't be able to
lookup names in Consul.

I'd be interested to hear why the DNS settings got cleared out in your
environment. Or, had you not run the setup.ps1 script in the first case?

Thanks!

Show quoted text

On Sun, Jan 24, 2016 at 4:17 PM, Edward Hill <hill_edd(a)yahoo.co.uk> wrote:

setting had gone

added it back on and app deployed \o/

attachment.html

Edward Hill

#3512

I had run the setup.ps1 but I had a lot of indecision about networking and reduced the number of  adapters later which might have caused it

I notice the setup.ps1 selects the first interface matched:

$ifindex = $routeable_interfaces[0].Index

Edward Hill

#3513

Think the test should be:

TestScript = {
    [array]$routeable_interfaces = Get-WmiObject Win32_NetworkAdapterConfiguration | Where { $_.IpAddress -AND ($_.IpAddress | Where { $addr = [Net.IPAddress] $_; $addr.AddressFamily -eq "InterNetwork" -AND ($addr.address -BAND ([Net.IPAddress] "255.255.0.0").address) -ne ([Net.IPAddress] "169.254.0.0").address }) }

    foreach($routeable_interface in $routeable_interfaces)
    {
        $interface = (Get-WmiObject Win32_NetworkAdapter | Where { $_.DeviceID -eq $routeable_interface.Index }).netconnectionid

        if(-Not (Get-DnsClientServerAddress -InterfaceAlias $interface -AddressFamily ipv4 -ErrorAction Stop).ServerAddresses[0] -eq "127.0.0.1")
        {
            Write-Verbose -Message "DNS Servers not yet correct."
            return $false
        }
    }
    Write-Verbose -Message "DNS Servers are set correctly."
    return $true
}

Amit Kumar Gupta

#3514

Not too sure about how the Windows Cell gets provisioned, but the
consul_agent job in consul-release has been undergoing a lot of changes
recently in terms of how the startup scripts work.  How does the Windows
Cell project keep up to date with changes to
https://github.com/cloudfoundry-incubator/consul-release/blob/master/jobs/consul_agent/templates/agent_ctl.sh.erb?
And is it possible that there's some logic that is now out of sync?

Show quoted text

On Mon, Jan 25, 2016 at 2:28 PM, Edward Hill <hill_edd(a)yahoo.co.uk> wrote:

Think the test should be:

TestScript = {
    [array]$routeable_interfaces = Get-WmiObject
Win32_NetworkAdapterConfiguration | Where { $_.IpAddress -AND ($_.IpAddress
| Where { $addr = [Net.IPAddress] $_; $addr.AddressFamily -eq
"InterNetwork" -AND ($addr.address -BAND ([Net.IPAddress]
"255.255.0.0").address) -ne ([Net.IPAddress] "169.254.0.0").address }) }

    foreach($routeable_interface in $routeable_interfaces)
    {
        $interface = (Get-WmiObject Win32_NetworkAdapter | Where {
$_.DeviceID -eq $routeable_interface.Index }).netconnectionid

        if(-Not (Get-DnsClientServerAddress -InterfaceAlias $interface
-AddressFamily ipv4 -ErrorAction Stop).ServerAddresses[0] -eq "127.0.0.1")
        {
            Write-Verbose -Message "DNS Servers not yet correct."
            return $false
        }
    }
    Write-Verbose -Message "DNS Servers are set correctly."
    return $true
}

attachment.html

Matthew Horan

#3518

Edward -

We used to detect the routable interface as you propose, but ran into
issues with that approach. After testing, we determined that Windows will
query to the first DNS server on every interface, regardless of whether or
not that interface has a "routable" IP. We swapped out our approach as this
was a more reliable way to configure DNS.

In the future, Windows VMs will be managed by bosh for Windows, and this
will be a non issue. For now, operators with more complicated setups can
add the Consul DNS server to the appropriate interface manually, as you
discovered.

Amit -

The Greenhouse team manually syncs the Bosh jobs with our compiled Windows
services. We have a build in place which monitors the job scripts and fails
when there are changes. I'm not sure if we're currently monitoring the
consul_agent job, but perhaps we should be. Regardless, Consul does seem to
be running properly at this time, it's just the DNS resolution interface
config that gets out of sync.

Show quoted text

On Mon, Jan 25, 2016 at 8:11 PM, Amit Gupta <agupta(a)pivotal.io> wrote:

Not too sure about how the Windows Cell gets provisioned, but the
consul_agent job in consul-release has been undergoing a lot of changes
recently in terms of how the startup scripts work.  How does the Windows
Cell project keep up to date with changes to
https://github.com/cloudfoundry-incubator/consul-release/blob/master/jobs/consul_agent/templates/agent_ctl.sh.erb?
And is it possible that there's some logic that is now out of sync?

On Mon, Jan 25, 2016 at 2:28 PM, Edward Hill <hill_edd(a)yahoo.co.uk> wrote:

Think the test should be:

TestScript = {
    [array]$routeable_interfaces = Get-WmiObject
Win32_NetworkAdapterConfiguration | Where { $_.IpAddress -AND ($_.IpAddress
| Where { $addr = [Net.IPAddress] $_; $addr.AddressFamily -eq
"InterNetwork" -AND ($addr.address -BAND ([Net.IPAddress]
"255.255.0.0").address) -ne ([Net.IPAddress] "169.254.0.0").address }) }

    foreach($routeable_interface in $routeable_interfaces)
    {
        $interface = (Get-WmiObject Win32_NetworkAdapter | Where {
$_.DeviceID -eq $routeable_interface.Index }).netconnectionid

        if(-Not (Get-DnsClientServerAddress -InterfaceAlias $interface
-AddressFamily ipv4 -ErrorAction Stop).ServerAddresses[0] -eq "127.0.0.1")
        {
            Write-Verbose -Message "DNS Servers not yet correct."
            return $false
        }
    }
    Write-Verbose -Message "DNS Servers are set correctly."
    return $true
}

attachment.html

Edward Hill

#3551

Thanks Matthew, yeah it's no real hassle to manually add