Considering deprecation of the HAProxy in cf-release


Shannon Coen
 

Over that past year or two, CF teams have been extracting components from
cf-release into their own BOSH releases. This allows teams to innovate more
quickly. These releases are currently submoduled into cf-release, so that
cf-release can be used to deploy the full suite of core CF services. The
current plan is that cf-release will eventually be deprecated in favor of
cf-deployment <https://github.com/cloudfoundry/cf-deployment>, a toolchain
that generates a manifest for CF from these composable releases.

One of the last components in cf-release
<https://github.com/cloudfoundry/cf-release/tree/master/jobs> that has not
been extracted into its own release is HAProxy.

Historically, HAProxy has fulfilled two use cases: load balancing and TLS
termination. Since support for TLS termination was added to Gorouter, load
balancing may be the remaining use case.

In production environments, operators are expected to provide their own
redundant load balancers and configure them to forward requests to the CF
routers. HAProxy may provide a convenience where another load balancer is
not available, but as HAProxy is not inherently highly available in a
production environment it would require another load balancer in front of
it for HA, at which point HAProxy itself would be superfluous. For this
reason we have not recommended use of the HAProxy in cf-release for
production environments.

This leaves the use case of development, or lab environments. Over the past
year, we have begun thinking that the HAProxy provides no additional value
here either. Since HA is not required in these environments, we have begun
recommending
<http://docs.cloudfoundry.org/adminguide/configure-lb-healthcheck.html>
that operators point DNS directly at single instances of the CF Router and
SSH Proxy.

Therefore, we're proposing deprecation of the HAProxy in cf-release. Of
course, we want to hear your feedback. If you are currently leveraging the
HAProxy in cf-release, please share your use cases.

Thank you,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.


Koper, Dies <diesk@...>
 

Hi Shannon,

For a simple bosh-lite user like me, not used to modifying the manifest stubs, is there an easy way on my default (Virtualbox) bosh-lite deployment to try out using CF without HAProxy, to see if it has any impact on what I do?

I tried adding the following to my /etc/hosts file as a test, but `cf login` failed with a “dial tcp 10.244.0.138:443: getsockopt: connection refused” – I
Do I need to manually edit the manifest and copy over the bosh-lite wildcard certificate into the gorouter section and enable its tsl termination, and maybe also remove any restrictions in place on IP addressed connections are accepted from?

10.244.0.22 api.bosh-lite.com
10.244.0.22 uaa.bosh-lite.com
10.244.0.22 login.bosh-lite.com
10.244.0.22 doppler.bosh-lite.com
10.244.0.22 loggregator.bosh-lite.com
10.244.0.22 ssh.bosh-lite.com

Regards,
Dies Koper


From: Shannon Coen [mailto:scoen(a)pivotal.io]
Sent: Saturday, January 14, 2017 7:26 AM
To: Cloud Foundry Dev
Subject: [cf-dev] Considering deprecation of the HAProxy in cf-release

Over that past year or two, CF teams have been extracting components from cf-release into their own BOSH releases. This allows teams to innovate more quickly. These releases are currently submoduled into cf-release, so that cf-release can be used to deploy the full suite of core CF services. The current plan is that cf-release will eventually be deprecated in favor of cf-deployment<https://github.com/cloudfoundry/cf-deployment>, a toolchain that generates a manifest for CF from these composable releases.

One of the last components in cf-release<https://github.com/cloudfoundry/cf-release/tree/master/jobs> that has not been extracted into its own release is HAProxy.

Historically, HAProxy has fulfilled two use cases: load balancing and TLS termination. Since support for TLS termination was added to Gorouter, load balancing may be the remaining use case.

In production environments, operators are expected to provide their own redundant load balancers and configure them to forward requests to the CF routers. HAProxy may provide a convenience where another load balancer is not available, but as HAProxy is not inherently highly available in a production environment it would require another load balancer in front of it for HA, at which point HAProxy itself would be superfluous. For this reason we have not recommended use of the HAProxy in cf-release for production environments.

This leaves the use case of development, or lab environments. Over the past year, we have begun thinking that the HAProxy provides no additional value here either. Since HA is not required in these environments, we have begun recommending<http://docs.cloudfoundry.org/adminguide/configure-lb-healthcheck.html> that operators point DNS directly at single instances of the CF Router and SSH Proxy.

Therefore, we're proposing deprecation of the HAProxy in cf-release. Of course, we want to hear your feedback. If you are currently leveraging the HAProxy in cf-release, please share your use cases.

Thank you,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.


Shannon Coen
 

Hi Dies,

You'd probably need to set the following properties in your cf-release
manifest:

router.enable_ssl: true

router.ssl_cert: <cert from ha_proxy.ssl_pem>
router.ssl_key: <key from ha_proxy.ssl_pem>

Best,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Fri, Jan 13, 2017 at 3:10 PM, Koper, Dies <diesk(a)fast.au.fujitsu.com>
wrote:

Hi Shannon,



For a simple bosh-lite user like me, not used to modifying the manifest
stubs, is there an easy way on my default (Virtualbox) bosh-lite deployment
to try out using CF without HAProxy, to see if it has any impact on what I
do?



I tried adding the following to my /etc/hosts file as a test, but `cf
login` failed with a “dial tcp 10.244.0.138:443: getsockopt: connection
refused” – I

Do I need to manually edit the manifest and copy over the bosh-lite
wildcard certificate into the gorouter section and enable its tsl
termination, and maybe also remove any restrictions in place on IP
addressed connections are accepted from?



10.244.0.22 api.bosh-lite.com

10.244.0.22 uaa.bosh-lite.com

10.244.0.22 login.bosh-lite.com

10.244.0.22 doppler.bosh-lite.com

10.244.0.22 loggregator.bosh-lite.com

10.244.0.22 ssh.bosh-lite.com



Regards,

Dies Koper





*From:* Shannon Coen [mailto:scoen(a)pivotal.io]
*Sent:* Saturday, January 14, 2017 7:26 AM
*To:* Cloud Foundry Dev
*Subject:* [cf-dev] Considering deprecation of the HAProxy in cf-release



Over that past year or two, CF teams have been extracting components from
cf-release into their own BOSH releases. This allows teams to innovate more
quickly. These releases are currently submoduled into cf-release, so that
cf-release can be used to deploy the full suite of core CF services. The
current plan is that cf-release will eventually be deprecated in favor of
cf-deployment <https://github.com/cloudfoundry/cf-deployment>, a
toolchain that generates a manifest for CF from these composable releases.



One of the last components in cf-release
<https://github.com/cloudfoundry/cf-release/tree/master/jobs> that has
not been extracted into its own release is HAProxy.



Historically, HAProxy has fulfilled two use cases: load balancing and TLS
termination. Since support for TLS termination was added to Gorouter, load
balancing may be the remaining use case.



In production environments, operators are expected to provide their own
redundant load balancers and configure them to forward requests to the CF
routers. HAProxy may provide a convenience where another load balancer is
not available, but as HAProxy is not inherently highly available in a
production environment it would require another load balancer in front of
it for HA, at which point HAProxy itself would be superfluous. For this
reason we have not recommended use of the HAProxy in cf-release for
production environments.



This leaves the use case of development, or lab environments. Over the
past year, we have begun thinking that the HAProxy provides no additional
value here either. Since HA is not required in these environments, we
have begun recommending
<http://docs.cloudfoundry.org/adminguide/configure-lb-healthcheck.html>
that operators point DNS directly at single instances of the CF Router and
SSH Proxy.



Therefore, we're proposing deprecation of the HAProxy in cf-release. Of
course, we want to hear your feedback. If you are currently leveraging the
HAProxy in cf-release, please share your use cases.



Thank you,


Shannon Coen

Product Manager, Cloud Foundry

Pivotal, Inc.


Guillaume Berche
 

Hi Shannon,

Sorry for late delay on this.

Historically, HAProxy has fulfilled two use cases: load balancing and TLS
termination. Since support for TLS termination was added to Gorouter, load
balancing may be the remaining use case.

I understand the gorouter still only supports serving a single TLS
certificate [1], which makes it hard to support distinct api and app https
domains, which is often useful in lab environments.

I believe a battery-included load balancer still provides benefits to small
production environments which can't leverage out of the box load balancer
solution. Orange started contributing redundancy/HA support to the
cf-community haproxy bosh release, in the form of an active/passive fail
over leveraging keep-alived through VRRP protocol [2]. While this needs
further work to work in constrained cloud environments (without multicast
and gratuitous ARP support, such as AWS), it is likely these environments
provide out of the box their own production-grade load balancer construct,
or support for them can be added on top of keep-alived using unicast as
Iaas-specific features such as floating IPs.

I wonder whether it could make sense to incubate the
cloudfoundry-community/haproxy-boshrelease as to preserve such portable
battery-included load balancer and keep hard cloud prerequisites low for
newcomers/evaluations.

Regards,

Guillaume.

[1]
http://bosh.io/jobs/gorouter?source=github.com/cloudfoundry/cf-release&version=251#p=router.ssl_cert
[2] https://github.com/cloudfoundry-community/haproxy-boshrelease/pull/39

On Fri, Jan 13, 2017 at 9:25 PM, Shannon Coen <scoen(a)pivotal.io> wrote:

Over that past year or two, CF teams have been extracting components from
cf-release into their own BOSH releases. This allows teams to innovate more
quickly. These releases are currently submoduled into cf-release, so that
cf-release can be used to deploy the full suite of core CF services. The
current plan is that cf-release will eventually be deprecated in favor of
cf-deployment <https://github.com/cloudfoundry/cf-deployment>, a
toolchain that generates a manifest for CF from these composable releases.

One of the last components in cf-release
<https://github.com/cloudfoundry/cf-release/tree/master/jobs> that has
not been extracted into its own release is HAProxy.

Historically, HAProxy has fulfilled two use cases: load balancing and TLS
termination. Since support for TLS termination was added to Gorouter, load
balancing may be the remaining use case.

In production environments, operators are expected to provide their own
redundant load balancers and configure them to forward requests to the CF
routers. HAProxy may provide a convenience where another load balancer is
not available, but as HAProxy is not inherently highly available in a
production environment it would require another load balancer in front of
it for HA, at which point HAProxy itself would be superfluous. For this
reason we have not recommended use of the HAProxy in cf-release for
production environments.

This leaves the use case of development, or lab environments. Over the
past year, we have begun thinking that the HAProxy provides no additional
value here either. Since HA is not required in these environments, we
have begun recommending
<http://docs.cloudfoundry.org/adminguide/configure-lb-healthcheck.html>
that operators point DNS directly at single instances of the CF Router and
SSH Proxy.

Therefore, we're proposing deprecation of the HAProxy in cf-release. Of
course, we want to hear your feedback. If you are currently leveraging the
HAProxy in cf-release, please share your use cases.

Thank you,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.


Shannon Coen
 


I understand the gorouter still only supports serving a single TLS
certificate [1], which makes it hard to support distinct api and app https
domains, which is often useful in lab environments.
HAProxy in cf-release has the same limitation, so removal of it would be no
worse.


I believe a battery-included load balancer still provides benefits to
small production environments which can't leverage out of the box load
balancer solution. Orange started contributing redundancy/HA support to the
cf-community haproxy bosh release, in the form of an active/passive fail
over leveraging keep-alived through VRRP protocol [2]. While this needs
further work to work in constrained cloud environments (without multicast
and gratuitous ARP support, such as AWS), it is likely these environments
provide out of the box their own production-grade load balancer construct,
or support for them can be added on top of keep-alived using unicast as
Iaas-specific features such as floating IPs.
Historically, we've taken the approach that BOSH releases can be deployed
to any infrastructure that BOSH supports, and that all features in a BOSH
release are infrastructure agnostic. As AWS doesn't support the feature
you've mentioned, this would be an infrastructure-specific feature.


I wonder whether it could make sense to incubate the
cloudfoundry-community/haproxy-boshrelease as to preserve such portable
battery-included load balancer and keep hard cloud prerequisites low for
newcomers/evaluations.
That sounds interesting.



Regards,

Guillaume.

[1] http://bosh.io/jobs/gorouter?source=github.com/
cloudfoundry/cf-release&version=251#p=router.ssl_cert
[2] https://github.com/cloudfoundry-community/haproxy-boshrelease/pull/39


On Fri, Jan 13, 2017 at 9:25 PM, Shannon Coen <scoen(a)pivotal.io> wrote:

Over that past year or two, CF teams have been extracting components from
cf-release into their own BOSH releases. This allows teams to innovate more
quickly. These releases are currently submoduled into cf-release, so that
cf-release can be used to deploy the full suite of core CF services. The
current plan is that cf-release will eventually be deprecated in favor of
cf-deployment <https://github.com/cloudfoundry/cf-deployment>, a
toolchain that generates a manifest for CF from these composable releases.

One of the last components in cf-release
<https://github.com/cloudfoundry/cf-release/tree/master/jobs> that has
not been extracted into its own release is HAProxy.

Historically, HAProxy has fulfilled two use cases: load balancing and TLS
termination. Since support for TLS termination was added to Gorouter, load
balancing may be the remaining use case.

In production environments, operators are expected to provide their own
redundant load balancers and configure them to forward requests to the CF
routers. HAProxy may provide a convenience where another load balancer is
not available, but as HAProxy is not inherently highly available in a
production environment it would require another load balancer in front of
it for HA, at which point HAProxy itself would be superfluous. For this
reason we have not recommended use of the HAProxy in cf-release for
production environments.

This leaves the use case of development, or lab environments. Over the
past year, we have begun thinking that the HAProxy provides no additional
value here either. Since HA is not required in these environments, we
have begun recommending
<http://docs.cloudfoundry.org/adminguide/configure-lb-healthcheck.html>
that operators point DNS directly at single instances of the CF Router and
SSH Proxy.

Therefore, we're proposing deprecation of the HAProxy in cf-release. Of
course, we want to hear your feedback. If you are currently leveraging the
HAProxy in cf-release, please share your use cases.

Thank you,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.