F5 Load Balancer Configuration for Cloud Foundry Loggregator


Anthony
 

Does any one have any experience configuring F5 load balancers in front of the CF routers? We have configured F5 and app https and cf push requests are working fine. However, the connectivity with loggregator is not working. Taking a look at the documentation, it requires "websocket support" on the load balancer. We've done the configuration specified here:

https://support.f5.com/kb/en-us/solutions/public/14000/800/sol14814.html

With the following irule basically, applying the default TCP profile if it detects websocket traffic:

when HTTP_REQUEST {
if { [string tolower [HTTP::header Upgrade]] contains "websocket" }{
HTTP::disable
}
}

However, we are running into errors. Doing `cf logs myapp1` yields:

Error dialing loggregator server: read tcp <ip redacted>:443: connection reset by peer.
Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.<sys domain redacted>:443).

Does anyone have a clue?

Thanks!
Anthony


Mike Youngstrom
 

What version of F5 software are you running?

Mike

On Tue, Sep 22, 2015 at 5:20 PM, Anthony Lee <lee.apc(a)gmail.com> wrote:

Does any one have any experience configuring F5 load balancers in front of
the CF routers? We have configured F5 and app https and cf push requests
are working fine. However, the connectivity with loggregator is not
working. Taking a look at the documentation, it requires "websocket
support" on the load balancer. We've done the configuration specified here:

https://support.f5.com/kb/en-us/solutions/public/14000/800/sol14814.html

With the following irule basically, applying the default TCP profile if it
detects websocket traffic:

when HTTP_REQUEST {
if { [string tolower [HTTP::header Upgrade]] contains "websocket" }{
HTTP::disable
}
}

However, we are running into errors. Doing `cf logs myapp1` yields:

Error dialing loggregator server: read tcp <ip redacted>:443: connection
reset by peer.
Please ask your Cloud Foundry Operator to check the platform configuration
(loggregator endpoint is wss://loggregator.<sys domain redacted>:443).

Does anyone have a clue?

Thanks!
Anthony


Anthony
 

The version we are testing in is 10.4.

Regards,
Anthony

On Sep 22, 2015, at 6:41 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

What version of F5 software are you running?

Mike

On Tue, Sep 22, 2015 at 5:20 PM, Anthony Lee <lee.apc(a)gmail.com> wrote:
Does any one have any experience configuring F5 load balancers in front of the CF routers? We have configured F5 and app https and cf push requests are working fine. However, the connectivity with loggregator is not working. Taking a look at the documentation, it requires "websocket support" on the load balancer. We've done the configuration specified here:

https://support.f5.com/kb/en-us/solutions/public/14000/800/sol14814.html

With the following irule basically, applying the default TCP profile if it detects websocket traffic:

when HTTP_REQUEST {
if { [string tolower [HTTP::header Upgrade]] contains "websocket" }{
HTTP::disable
}
}

However, we are running into errors. Doing `cf logs myapp1` yields:

Error dialing loggregator server: read tcp <ip redacted>:443: connection reset by peer.
Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.<sys domain redacted>:443).

Does anyone have a clue?

Thanks!
Anthony


Mike Youngstrom
 

We are running 11.4 and 11.6. I'd give an upgrade a try before digging too
much deeper.

Mike

On Sep 22, 2015 6:36 PM, "Anthony" <lee.apc(a)gmail.com> wrote:

The version we are testing in is 10.4.

Regards,
Anthony

On Sep 22, 2015, at 6:41 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

What version of F5 software are you running?

Mike

On Tue, Sep 22, 2015 at 5:20 PM, Anthony Lee <lee.apc(a)gmail.com> wrote:

Does any one have any experience configuring F5 load balancers in front
of the CF routers? We have configured F5 and app https and cf push requests
are working fine. However, the connectivity with loggregator is not
working. Taking a look at the documentation, it requires "websocket
support" on the load balancer. We've done the configuration specified here:

https://support.f5.com/kb/en-us/solutions/public/14000/800/sol14814.html

With the following irule basically, applying the default TCP profile if
it detects websocket traffic:

when HTTP_REQUEST {
if { [string tolower [HTTP::header Upgrade]] contains "websocket" }{
HTTP::disable
}
}

However, we are running into errors. Doing `cf logs myapp1` yields:

Error dialing loggregator server: read tcp <ip redacted>:443: connection
reset by peer.
Please ask your Cloud Foundry Operator to check the platform
configuration (loggregator endpoint is wss://loggregator.<sys domain
redacted>:443).

Does anyone have a clue?

Thanks!
Anthony


Anthony
 

Thanks Mike! Unfortunately, upgrading is not an option since its a really loaded enterprise device. The interesting part is that there is a similarly set up websockets vip (plain old server i think .net) that is working on the same device.

We'll work with our network folks to find other devices with newer software we can use.

Would appreciate if anyone has other ideas?

Regards,
Anthony

On Sep 22, 2015, at 7:49 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

We are running 11.4 and 11.6. I'd give an upgrade a try before digging too much deeper.

Mike

On Sep 22, 2015 6:36 PM, "Anthony" <lee.apc(a)gmail.com> wrote:
The version we are testing in is 10.4.

Regards,
Anthony

On Sep 22, 2015, at 6:41 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

What version of F5 software are you running?

Mike

On Tue, Sep 22, 2015 at 5:20 PM, Anthony Lee <lee.apc(a)gmail.com> wrote:
Does any one have any experience configuring F5 load balancers in front of the CF routers? We have configured F5 and app https and cf push requests are working fine. However, the connectivity with loggregator is not working. Taking a look at the documentation, it requires "websocket support" on the load balancer. We've done the configuration specified here:

https://support.f5.com/kb/en-us/solutions/public/14000/800/sol14814.html

With the following irule basically, applying the default TCP profile if it detects websocket traffic:

when HTTP_REQUEST {
if { [string tolower [HTTP::header Upgrade]] contains "websocket" }{
HTTP::disable
}
}

However, we are running into errors. Doing `cf logs myapp1` yields:

Error dialing loggregator server: read tcp <ip redacted>:443: connection reset by peer.
Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.<sys domain redacted>:443).

Does anyone have a clue?

Thanks!
Anthony


Rohit Kumar
 

Does `cf logs --recent` work for you? The recent logs request goes over
HTTP. If that goes through that means only the websocket requests to
loggregator servers are a problem.

Rohit

On Tue, Sep 22, 2015 at 8:18 PM, Anthony <lee.apc(a)gmail.com> wrote:

Thanks Mike! Unfortunately, upgrading is not an option since its a really
loaded enterprise device. The interesting part is that there is a
similarly set up websockets vip (plain old server i think .net) that is
working on the same device.

We'll work with our network folks to find other devices with newer
software we can use.

Would appreciate if anyone has other ideas?

Regards,
Anthony

On Sep 22, 2015, at 7:49 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

We are running 11.4 and 11.6. I'd give an upgrade a try before digging
too much deeper.

Mike
On Sep 22, 2015 6:36 PM, "Anthony" <lee.apc(a)gmail.com> wrote:

The version we are testing in is 10.4.

Regards,
Anthony

On Sep 22, 2015, at 6:41 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

What version of F5 software are you running?

Mike

On Tue, Sep 22, 2015 at 5:20 PM, Anthony Lee <lee.apc(a)gmail.com> wrote:

Does any one have any experience configuring F5 load balancers in front
of the CF routers? We have configured F5 and app https and cf push requests
are working fine. However, the connectivity with loggregator is not
working. Taking a look at the documentation, it requires "websocket
support" on the load balancer. We've done the configuration specified here:

https://support.f5.com/kb/en-us/solutions/public/14000/800/sol14814.html

With the following irule basically, applying the default TCP profile if
it detects websocket traffic:

when HTTP_REQUEST {
if { [string tolower [HTTP::header Upgrade]] contains "websocket" }{
HTTP::disable
}
}

However, we are running into errors. Doing `cf logs myapp1` yields:

Error dialing loggregator server: read tcp <ip redacted>:443: connection
reset by peer.
Please ask your Cloud Foundry Operator to check the platform
configuration (loggregator endpoint is wss://loggregator.<sys domain
redacted>:443).

Does anyone have a clue?

Thanks!
Anthony


Anthony
 

Yep. --recent works. Other cf commands and cf curl also works.

It definitely is the websockets for loggregator. Just not sure what the right config for F5 (version 10.4) should be.

Regards,
Anthony

On Sep 22, 2015, at 9:53 PM, Rohit Kumar <rokumar(a)pivotal.io> wrote:

Does `cf logs --recent` work for you? The recent logs request goes over HTTP. If that goes through that means only the websocket requests to loggregator servers are a problem.

Rohit

On Tue, Sep 22, 2015 at 8:18 PM, Anthony <lee.apc(a)gmail.com> wrote:
Thanks Mike! Unfortunately, upgrading is not an option since its a really loaded enterprise device. The interesting part is that there is a similarly set up websockets vip (plain old server i think .net) that is working on the same device.

We'll work with our network folks to find other devices with newer software we can use.

Would appreciate if anyone has other ideas?

Regards,
Anthony

On Sep 22, 2015, at 7:49 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

We are running 11.4 and 11.6. I'd give an upgrade a try before digging too much deeper.

Mike

On Sep 22, 2015 6:36 PM, "Anthony" <lee.apc(a)gmail.com> wrote:
The version we are testing in is 10.4.

Regards,
Anthony

On Sep 22, 2015, at 6:41 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

What version of F5 software are you running?

Mike

On Tue, Sep 22, 2015 at 5:20 PM, Anthony Lee <lee.apc(a)gmail.com> wrote:
Does any one have any experience configuring F5 load balancers in front of the CF routers? We have configured F5 and app https and cf push requests are working fine. However, the connectivity with loggregator is not working. Taking a look at the documentation, it requires "websocket support" on the load balancer. We've done the configuration specified here:

https://support.f5.com/kb/en-us/solutions/public/14000/800/sol14814.html

With the following irule basically, applying the default TCP profile if it detects websocket traffic:

when HTTP_REQUEST {
if { [string tolower [HTTP::header Upgrade]] contains "websocket" }{
HTTP::disable
}
}

However, we are running into errors. Doing `cf logs myapp1` yields:

Error dialing loggregator server: read tcp <ip redacted>:443: connection reset by peer.
Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.<sys domain redacted>:443).

Does anyone have a clue?

Thanks!
Anthony


Johannes Hiemer <jvhiemer@...>
 

Are you sure your logregator endpoint is configured on 443 and not 4443?

On 23.09.2015, at 05:26, Anthony <lee.apc(a)gmail.com> wrote:

Yep. --recent works. Other cf commands and cf curl also works.

It definitely is the websockets for loggregator. Just not sure what the right config for F5 (version 10.4) should be.

Regards,
Anthony

On Sep 22, 2015, at 9:53 PM, Rohit Kumar <rokumar(a)pivotal.io> wrote:

Does `cf logs --recent` work for you? The recent logs request goes over HTTP. If that goes through that means only the websocket requests to loggregator servers are a problem.

Rohit

On Tue, Sep 22, 2015 at 8:18 PM, Anthony <lee.apc(a)gmail.com> wrote:
Thanks Mike! Unfortunately, upgrading is not an option since its a really loaded enterprise device. The interesting part is that there is a similarly set up websockets vip (plain old server i think .net) that is working on the same device.

We'll work with our network folks to find other devices with newer software we can use.

Would appreciate if anyone has other ideas?

Regards,
Anthony

On Sep 22, 2015, at 7:49 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

We are running 11.4 and 11.6. I'd give an upgrade a try before digging too much deeper.

Mike

On Sep 22, 2015 6:36 PM, "Anthony" <lee.apc(a)gmail.com> wrote:
The version we are testing in is 10.4.

Regards,
Anthony

On Sep 22, 2015, at 6:41 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

What version of F5 software are you running?

Mike

On Tue, Sep 22, 2015 at 5:20 PM, Anthony Lee <lee.apc(a)gmail.com> wrote:
Does any one have any experience configuring F5 load balancers in front of the CF routers? We have configured F5 and app https and cf push requests are working fine. However, the connectivity with loggregator is not working. Taking a look at the documentation, it requires "websocket support" on the load balancer. We've done the configuration specified here:

https://support.f5.com/kb/en-us/solutions/public/14000/800/sol14814.html

With the following irule basically, applying the default TCP profile if it detects websocket traffic:

when HTTP_REQUEST {
if { [string tolower [HTTP::header Upgrade]] contains "websocket" }{
HTTP::disable
}
}

However, we are running into errors. Doing `cf logs myapp1` yields:

Error dialing loggregator server: read tcp <ip redacted>:443: connection reset by peer.
Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.<sys domain redacted>:443).

Does anyone have a clue?

Thanks!
Anthony


Mike Youngstrom
 

If you are sharing a vip for http and websocket then 443 would be correct.
But Anthony, you can try creating a layer 4 virtual server on 4443 that
goes to the same pool on the back end and configure the CC to use that port
instead for loggregator connections.

Mike

On Tue, Sep 22, 2015 at 10:32 PM, Johannes Hiemer <jvhiemer(a)gmail.com>
wrote:

Are you sure your logregator endpoint is configured on 443 and not 4443?



On 23.09.2015, at 05:26, Anthony <lee.apc(a)gmail.com> wrote:

Yep. --recent works. Other cf commands and cf curl also works.

It definitely is the websockets for loggregator. Just not sure what the
right config for F5 (version 10.4) should be.

Regards,
Anthony

On Sep 22, 2015, at 9:53 PM, Rohit Kumar <rokumar(a)pivotal.io> wrote:

Does `cf logs --recent` work for you? The recent logs request goes over
HTTP. If that goes through that means only the websocket requests to
loggregator servers are a problem.

Rohit

On Tue, Sep 22, 2015 at 8:18 PM, Anthony <lee.apc(a)gmail.com> wrote:

Thanks Mike! Unfortunately, upgrading is not an option since its a really
loaded enterprise device. The interesting part is that there is a
similarly set up websockets vip (plain old server i think .net) that is
working on the same device.

We'll work with our network folks to find other devices with newer
software we can use.

Would appreciate if anyone has other ideas?

Regards,
Anthony

On Sep 22, 2015, at 7:49 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

We are running 11.4 and 11.6. I'd give an upgrade a try before digging
too much deeper.

Mike
On Sep 22, 2015 6:36 PM, "Anthony" <lee.apc(a)gmail.com> wrote:

The version we are testing in is 10.4.

Regards,
Anthony

On Sep 22, 2015, at 6:41 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

What version of F5 software are you running?

Mike

On Tue, Sep 22, 2015 at 5:20 PM, Anthony Lee <lee.apc(a)gmail.com> wrote:

Does any one have any experience configuring F5 load balancers in front
of the CF routers? We have configured F5 and app https and cf push requests
are working fine. However, the connectivity with loggregator is not
working. Taking a look at the documentation, it requires "websocket
support" on the load balancer. We've done the configuration specified here:

https://support.f5.com/kb/en-us/solutions/public/14000/800/sol14814.html

With the following irule basically, applying the default TCP profile if
it detects websocket traffic:

when HTTP_REQUEST {
if { [string tolower [HTTP::header Upgrade]] contains "websocket" }{
HTTP::disable
}
}

However, we are running into errors. Doing `cf logs myapp1` yields:

Error dialing loggregator server: read tcp <ip redacted>:443:
connection reset by peer.
Please ask your Cloud Foundry Operator to check the platform
configuration (loggregator endpoint is wss://loggregator.<sys domain
redacted>:443).

Does anyone have a clue?

Thanks!
Anthony


Anthony
 

Thanks for all the responses. We ended up finding an f5 device that is on 11.5. With that, things worked out of the box with only the irule to insert x-forwarded-proto. Everything is working over 443.

Regards,
Anthony

On Sep 22, 2015, at 11:35 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

If you are sharing a vip for http and websocket then 443 would be correct. But Anthony, you can try creating a layer 4 virtual server on 4443 that goes to the same pool on the back end and configure the CC to use that port instead for loggregator connections.

Mike

On Tue, Sep 22, 2015 at 10:32 PM, Johannes Hiemer <jvhiemer(a)gmail.com> wrote:
Are you sure your logregator endpoint is configured on 443 and not 4443?



On 23.09.2015, at 05:26, Anthony <lee.apc(a)gmail.com> wrote:

Yep. --recent works. Other cf commands and cf curl also works.

It definitely is the websockets for loggregator. Just not sure what the right config for F5 (version 10.4) should be.

Regards,
Anthony

On Sep 22, 2015, at 9:53 PM, Rohit Kumar <rokumar(a)pivotal.io> wrote:

Does `cf logs --recent` work for you? The recent logs request goes over HTTP. If that goes through that means only the websocket requests to loggregator servers are a problem.

Rohit

On Tue, Sep 22, 2015 at 8:18 PM, Anthony <lee.apc(a)gmail.com> wrote:
Thanks Mike! Unfortunately, upgrading is not an option since its a really loaded enterprise device. The interesting part is that there is a similarly set up websockets vip (plain old server i think .net) that is working on the same device.

We'll work with our network folks to find other devices with newer software we can use.

Would appreciate if anyone has other ideas?

Regards,
Anthony

On Sep 22, 2015, at 7:49 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

We are running 11.4 and 11.6. I'd give an upgrade a try before digging too much deeper.

Mike

On Sep 22, 2015 6:36 PM, "Anthony" <lee.apc(a)gmail.com> wrote:
The version we are testing in is 10.4.

Regards,
Anthony

On Sep 22, 2015, at 6:41 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

What version of F5 software are you running?

Mike

On Tue, Sep 22, 2015 at 5:20 PM, Anthony Lee <lee.apc(a)gmail.com> wrote:
Does any one have any experience configuring F5 load balancers in front of the CF routers? We have configured F5 and app https and cf push requests are working fine. However, the connectivity with loggregator is not working. Taking a look at the documentation, it requires "websocket support" on the load balancer. We've done the configuration specified here:

https://support.f5.com/kb/en-us/solutions/public/14000/800/sol14814.html

With the following irule basically, applying the default TCP profile if it detects websocket traffic:

when HTTP_REQUEST {
if { [string tolower [HTTP::header Upgrade]] contains "websocket" }{
HTTP::disable
}
}

However, we are running into errors. Doing `cf logs myapp1` yields:

Error dialing loggregator server: read tcp <ip redacted>:443: connection reset by peer.
Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.<sys domain redacted>:443).

Does anyone have a clue?

Thanks!
Anthony