Date   

Re: Routing-release 0.144.0: Gorouter performance improvements

Mike Youngstrom
 

Yes, excellent release! We've been anticipating persistent connections
from router to instance for a long time. Great work!

Mike

On Mon, Feb 6, 2017 at 8:17 AM, Marco Nicosia <mnicosia(a)pivotal.io> wrote:

So glad to see this announcement. I know the team has traveled a long way
to get here; congrats!

On Sun, Feb 5, 2017 at 21:30 Onsi Fakhouri <ofakhouri(a)pivotal.io> wrote:

Awesome!! Kudos to the routing team for your patience and
determination. This is a great step forward for CF!

On Sun, Feb 5, 2017 at 9:05 PM, Dieu Cao <dcao(a)pivotal.io> wrote:

Congrats to the Routing team!
Looking forward to the blog post.

-Dieu

On Fri, Feb 3, 2017 at 3:47 PM, Shannon Coen <scoen(a)pivotal.io> wrote:

On behalf of the CF Routing team, I am thrilled to announce
routing-release 0.144.0, that includes the results of many months of effort
to improve throughput and latency performance of Cloud Foundry's L7 HTTP
router, Gorouter.

Attached you'll find a report demonstrating these performance
improvements. In our release notes you'll find a summary of our test
methodology.

https://github.com/cloudfoundry-incubator/routing-release/releases/tag/
0.144.0

We'll be submitting a blog post soon with a deeper review of our efforts
to improve performance.

We expect routing-release 0.144.0 and the updated Gorouter to be included
in the next cf-release, v252.

Best,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.











CF Environment Variable Interpolation

Olga Reva
 

Since migration to Diego, our Spring Boot Applications cannot access properties defined in application.properties files like the following:

gateway.services.dashboard = ${vcap.services.dashboardService.credentials.url:http://localhost:8081}
gateway.credentials.dashboard.username=${vcap.services.dashboardService.credentials.username:username1}
gateway.credentials.dashboard.password=${vcap.services.dashboardService.credentials.password:pwd1}

I.e. app try to connect to http://localhost:8081 instead of using of predefined variables vcap.services.dashboardService.credentials.url...

How to change configs to access variables using vcap.services...
?
Can anybody help me?


Re: Routing-release 0.144.0: Gorouter performance improvements

Marco Nicosia
 

So glad to see this announcement. I know the team has traveled a long way
to get here; congrats!

On Sun, Feb 5, 2017 at 21:30 Onsi Fakhouri <ofakhouri(a)pivotal.io> wrote:

Awesome!! Kudos to the routing team for your patience and determination.
This is a great step forward for CF!

On Sun, Feb 5, 2017 at 9:05 PM, Dieu Cao <dcao(a)pivotal.io> wrote:

Congrats to the Routing team!
Looking forward to the blog post.

-Dieu

On Fri, Feb 3, 2017 at 3:47 PM, Shannon Coen <scoen(a)pivotal.io> wrote:

On behalf of the CF Routing team, I am thrilled to announce
routing-release 0.144.0, that includes the results of many months of effort
to improve throughput and latency performance of Cloud Foundry's L7 HTTP
router, Gorouter.

Attached you'll find a report demonstrating these performance
improvements. In our release notes you'll find a summary of our test
methodology.


https://github.com/cloudfoundry-incubator/routing-release/releases/tag/0.144.0

We'll be submitting a blog post soon with a deeper review of our efforts
to improve performance.

We expect routing-release 0.144.0 and the updated Gorouter to be included
in the next cf-release, v252.

Best,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.











Re: Routing-release 0.144.0: Gorouter performance improvements

Onsi Fakhouri <ofakhouri@...>
 

Awesome!! Kudos to the routing team for your patience and determination.
This is a great step forward for CF!

On Sun, Feb 5, 2017 at 9:05 PM, Dieu Cao <dcao(a)pivotal.io> wrote:

Congrats to the Routing team!
Looking forward to the blog post.

-Dieu

On Fri, Feb 3, 2017 at 3:47 PM, Shannon Coen <scoen(a)pivotal.io> wrote:

On behalf of the CF Routing team, I am thrilled to announce
routing-release 0.144.0, that includes the results of many months of effort
to improve throughput and latency performance of Cloud Foundry's L7 HTTP
router, Gorouter.

Attached you'll find a report demonstrating these performance
improvements. In our release notes you'll find a summary of our test
methodology.

https://github.com/cloudfoundry-incubator/routing-release/
releases/tag/0.144.0

We'll be submitting a blog post soon with a deeper review of our efforts
to improve performance.

We expect routing-release 0.144.0 and the updated Gorouter to be included
in the next cf-release, v252.

Best,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.


Re: Routing-release 0.144.0: Gorouter performance improvements

Dieu Cao <dcao@...>
 

Congrats to the Routing team!
Looking forward to the blog post.

-Dieu

On Fri, Feb 3, 2017 at 3:47 PM, Shannon Coen <scoen(a)pivotal.io> wrote:

On behalf of the CF Routing team, I am thrilled to announce
routing-release 0.144.0, that includes the results of many months of effort
to improve throughput and latency performance of Cloud Foundry's L7 HTTP
router, Gorouter.

Attached you'll find a report demonstrating these performance
improvements. In our release notes you'll find a summary of our test
methodology.

https://github.com/cloudfoundry-incubator/routing-release/releases/tag/
0.144.0

We'll be submitting a blog post soon with a deeper review of our efforts
to improve performance.

We expect routing-release 0.144.0 and the updated Gorouter to be included
in the next cf-release, v252.

Best,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.


Proposal: Instance Identity Credentials in Cloud Foundry

Eric Malm <emalm@...>
 

Hi, all,

The CF Diego team has a proposal[1] for Cloud Foundry to generate and
distribute instance-specific credentials to application instances running
on the platform. These credentials will be in the form of an X.509
certificate-key pair suitable for use in TLS communication, and are
intended to encode the CF identity of each individual application instance
in the subject distinguished name in the certificate.

We envision this functionality to be generally useful as more and more
communication happens directly with containerized application instances.
For example, these platform-native credentials will allow application
instances and tasks to present their identity as a CF application to:

- external services that they access via TLS communication,
- clients that communicate with them via TLS traffic that they terminate
(say, via TCP routing),
- other application instances on an overlay network established by the CF
Container Networking project[2].

The credentials will be relatively short-lived, to limit the impact of
accidental disclosure to untrusted parties, and will be rotated regularly
within the container filesystem. On the host Diego cell, they will be
protected from unauthorized access via filesystem permissions and will be
stored only in system memory via tmpfs or a similar ephemeral filesystem.

We of course welcome input and feedback on the proposal via inline
commentary on the proposal document, especially with regard to potential
use cases for this proposed platform capability. We have already identified
one immediate use case for these credentials: application instances will be
able to use them to authenticate with the recently proposed CredHub
project[3] to retrieve credentials for bound services after service brokers
store them in CredHub, thus maximizing the compartmentalization of
extremely sensitive service credentials within CF.

Some initial stages of work are planned in the Diego backlog in the
"instance-identity:creds" epic[4].

Thanks,
Eric Malm, CF Diego PM

[1]: https://docs.google.com/document/d/1OWrqaNEQkl8VXd8r3W6GgDEXxd3sX
w5C-20dAu76HOk/edit
[2]: https://github.com/cloudfoundry-incubator/cf-networking-release
[3]: https://docs.google.com/document/d/1iG28J2Lm8RY3BXCZqqNWO7v-
G1ppcdK8cizlhbN_o4g/edit
[4]: https://www.pivotaltracker.com/epic/show/3288841


Routing-release 0.144.0: Gorouter performance improvements

Shannon Coen
 

On behalf of the CF Routing team, I am thrilled to announce routing-release
0.144.0, that includes the results of many months of effort to improve
throughput and latency performance of Cloud Foundry's L7 HTTP router,
Gorouter.

Attached you'll find a report demonstrating these performance improvements.
In our release notes you'll find a summary of our test methodology.

https://github.com/cloudfoundry-incubator/routing-release/releases/tag/0.144.0

We'll be submitting a blog post soon with a deeper review of our efforts to
improve performance.

We expect routing-release 0.144.0 and the updated Gorouter to be included
in the next cf-release, v252.

Best,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.


Enabling task creation by default

Zach Robinson
 

Hey all,

Currently task creation is disabled by default, and is enabled using a feature flag. This was to allow billing apps time to update and account for tasks consuming resources. We are now going to enable task creation by default.

Please let me know if there are any concerns.

Thanks
-Zach
CAPI PM


Cf-Extensions projects details - INFO NEEDED

Michael Maximilien
 

Hi,
 
If you are a contributor or owner to a CF-extensions project (such as the one listed below), keep reading. Others can safely skip this message.
 
-----------
I am trying to collect information for the following CF-extensions projects:
 
* Autoscaling
* Brooklyn Broker
* CF-Buildpacks
* Java Buildpack
* Java Tools
* Notifications
* MySQL Broker
 
Information needed are things like Tracker and Github links and short description etc.
 
Please edit this docs which should contain an entry for your project.
 
 
If you own a project that is incubating or part of the CF-extensions and it's not listed, please contact me ASAP.
 
Best,

------
dr.max
ibm cloud labs
silicon valley, ca
maximilien.org


Re: Diego as Default?

Leandro David Cacciagioni
 

I have already deployed several times the diego backend but since now DEAs
are deprecated seems a little awkward to still have them as the default in
the cf-release templates.

2017-02-03 19:06 GMT+01:00 David Sabeti <dsabeti(a)pivotal.io>:

Hi Leandro,

I've been holding off on defaulting to the Diego backend primarily because
we planned to make that change when we shipped cf-deployment
<https://github.com/cloudfoundry/cf-deployment>. That repo is still under
development and won't be ready for a little while longer, so I think it
makes sense to go ahead and update the spiff templates in cf-release to use
Diego by default. I've written a Pivotal Tracker story that you can keep an
eye on here <https://www.pivotaltracker.com/story/show/139030003>. It
likely won't make it into CF 252, but we'll definitely have it by CF 253.

In the meantime, you can still use the `generate_deployment_manifest`
script in cf-release and use Diego; all you need to do to enable Diego is
override this property
<https://github.com/cloudfoundry/cf-release/blob/master/templates/cf.yml#L805> by
setting the value in your spiff stub. It would look something like this:

properties:
...
cc:
default_to_diego_backend: true

Feel free to reach out if you need any other help deploying.

David Sabeti
Product Manager, CF Release Integration

On Fri, Feb 3, 2017 at 1:52 AM Leandro David Cacciagioni <
leandro.21.2008(a)gmail.com> wrote:

guys is there any reason why Diego is not enabled by default in the
`generate_deployment_manifest` templates? AFAIK DEA arch is being
deprecated in a few months so I think it could be good to already have it
as default.

Then since Diego is not in the defaults which guide did I follow for AWS
diego deployment the ones in Diego docs(https://github.com/
cloudfoundry/diego-release/tree/develop/examples/aws), the ones in the
cf-release (https://github.com/cloudfoundry/cf-release/tree/
master/example_manifests) or which ones?

Thanks,
Leandro.-


Re: Diego as Default?

Leandro David Cacciagioni
 

Thanks a lot!!!

2017-02-03 19:06 GMT+01:00 David Sabeti <dsabeti(a)pivotal.io>:

Hi Leandro,

I've been holding off on defaulting to the Diego backend primarily because
we planned to make that change when we shipped cf-deployment
<https://github.com/cloudfoundry/cf-deployment>. That repo is still under
development and won't be ready for a little while longer, so I think it
makes sense to go ahead and update the spiff templates in cf-release to use
Diego by default. I've written a Pivotal Tracker story that you can keep an
eye on here <https://www.pivotaltracker.com/story/show/139030003>. It
likely won't make it into CF 252, but we'll definitely have it by CF 253.

In the meantime, you can still use the `generate_deployment_manifest`
script in cf-release and use Diego; all you need to do to enable Diego is
override this property
<https://github.com/cloudfoundry/cf-release/blob/master/templates/cf.yml#L805> by
setting the value in your spiff stub. It would look something like this:

properties:
...
cc:
default_to_diego_backend: true

Feel free to reach out if you need any other help deploying.

David Sabeti
Product Manager, CF Release Integration

On Fri, Feb 3, 2017 at 1:52 AM Leandro David Cacciagioni <
leandro.21.2008(a)gmail.com> wrote:

guys is there any reason why Diego is not enabled by default in the
`generate_deployment_manifest` templates? AFAIK DEA arch is being
deprecated in a few months so I think it could be good to already have it
as default.

Then since Diego is not in the defaults which guide did I follow for AWS
diego deployment the ones in Diego docs(https://github.com/
cloudfoundry/diego-release/tree/develop/examples/aws), the ones in the
cf-release (https://github.com/cloudfoundry/cf-release/tree/
master/example_manifests) or which ones?

Thanks,
Leandro.-


Re: Diego as Default?

David Sabeti
 

Hi Leandro,

I've been holding off on defaulting to the Diego backend primarily because
we planned to make that change when we shipped cf-deployment
<https://github.com/cloudfoundry/cf-deployment>. That repo is still under
development and won't be ready for a little while longer, so I think it
makes sense to go ahead and update the spiff templates in cf-release to use
Diego by default. I've written a Pivotal Tracker story that you can keep an
eye on here <https://www.pivotaltracker.com/story/show/139030003>. It
likely won't make it into CF 252, but we'll definitely have it by CF 253.

In the meantime, you can still use the `generate_deployment_manifest`
script in cf-release and use Diego; all you need to do to enable Diego is
override this property
<https://github.com/cloudfoundry/cf-release/blob/master/templates/cf.yml#L805>
by
setting the value in your spiff stub. It would look something like this:

properties:
...
cc:
default_to_diego_backend: true

Feel free to reach out if you need any other help deploying.

David Sabeti
Product Manager, CF Release Integration

On Fri, Feb 3, 2017 at 1:52 AM Leandro David Cacciagioni <
leandro.21.2008(a)gmail.com> wrote:

guys is there any reason why Diego is not enabled by default in the
`generate_deployment_manifest` templates? AFAIK DEA arch is being
deprecated in a few months so I think it could be good to already have it
as default.

Then since Diego is not in the defaults which guide did I follow for AWS
diego deployment the ones in Diego docs(
https://github.com/cloudfoundry/diego-release/tree/develop/examples/aws),
the ones in the cf-release (
https://github.com/cloudfoundry/cf-release/tree/master/example_manifests)
or which ones?

Thanks,
Leandro.-


Diego as Default?

Leandro David Cacciagioni
 

guys is there any reason why Diego is not enabled by default in the
`generate_deployment_manifest` templates? AFAIK DEA arch is being
deprecated in a few months so I think it could be good to already have it
as default.

Then since Diego is not in the defaults which guide did I follow for AWS
diego deployment the ones in Diego docs(
https://github.com/cloudfoundry/diego-release/tree/develop/examples/aws),
the ones in the cf-release (
https://github.com/cloudfoundry/cf-release/tree/master/example_manifests)
or which ones?

Thanks,
Leandro.-


Asynchronous route-mapping and blue-green deployment

Anderer, Thomas
 

Hello everyone,

we already had an internal discussion on the topic and since we could not quite come to a solution, I'd like to ask in this list.

We're using a self-made blue-green-deployment-script which pretty much does what's explained on cloudfoundry.org (https://docs.cloudfoundry.org/devguide/deploy-apps/blue-green.html), and additionally checks (HTTP-GET) the application after push, map-route and unmap-route. This script has been working fine on an instance A of CF (DEA-based and relatively small), but has issues on instance B (Diego-based, larger). The aforementioned HTTP-GET sometimes fails with "404 Not Found: Requested route does not exist" directly after push or map-route. We never experienced this issue on instance A. This could of course mean that the instance is small enough that all router operation are handled faster than our script is able to react. On instance B it sometimes takes up to a 1 second or even longer until the route mapping is finally completed.

In our internal discussion with our CF operators, we found a couple of parts of documentation which at least hint to different, maybe even inconsistent inner workings of the route mapping:
1) https://docs.cloudfoundry.org/devguide/deploy-apps/blue-green.html: Step 3: Map-route - The CF Router immediately begins to load balance traffic for demo-time.example.com between Blue and Green.
- In my opinion this implies or at least strongly hints that the map-route call was supposed to be synchronous.
2) PUT to "/v2/apps/#{app_guid}/routes/#{route_guid}" returns 201 CREATED and not 202 ACCEPTED, which also implies that it is a synchronous operation. It also does not return an event or operation which could be pinged to wait for completion of route mapping.
3) Operations which show the state of the route, like for example 'cf routes' already shows the route as being successfully mapped, although 404 is still returned.
4) See https://docs.cloudfoundry.org/devguide/deploy-apps/routes-domains.html#map-route: Applications running on the DEA architecture must be restarted after routes for an app are mapped or unmapped. Applications running on Diego do not need to be restarted.
- This is contrary to what I experienced, since route mapping on our instance A always worked synchronously and without restart of the application. Furthermore, it contradicts what's explained in 1) at least for DEA.
5) https://github.com/cloudfoundry/diego-design-notes#routing-translation-components:
Routing Translation Components: Route-Emitter
a) monitors DesiredLRP state and ActualLRP state via the BBS. When a change is detected, the Route-Emitter emits route registration and unregistration messages to the gorouter via the NATS message bus,
b) periodically emits the entire routing table to the router,
c) maintains a lock in consul to ensure only one route-emitter handles route registration at a time.
- This hints that route mapping is an asynchronous task.

Some of the issues could be fixed by repeatedly pinging the application in order to wait until the route has been mapped. But what about blue-green-deployment, where I map the application to a route on which there is already a running application. Here, I cannot find out in a general way, if the route mapping is completed and when I can start unmapping the route from the old application. If route-mapping is indeed an asynchronous task, in my opinion the description on https://docs.cloudfoundry.org/devguide/deploy-apps/blue-green.html is misleading at best. And all blue-green-deployment-scripts which I've seen so far have this issue.

So, is the route mapping really supposed to be asynchronous? If so, is there any general way to find out when the route-mapping process has finished?

Thank you for your help and clarification,

Best regards,
--
Thomas Anderer
Agile Software Engineer
andrena objects ag
currently working at SAP


UAA 3.10.0 Release Announcement

Sree Tummidi
 

Hi All,

On behalf of the entire UAA team I am pleased to announce the release of *UAA
3.10.0*. This release truly has been community driven effort with major
feature contributions from *SAP* and *Microsoft*.

The release highlights include

Major FeaturesExternal User Claims via UserInfo Endpoint

This feature enables User Attributes (including custom attributes) and
Group Memberships from LDAP, SAML and OpenID Connect providers to be
exposed via the UserInfo endpoint of UAA in addition to propagating them
via OpenID Connect id_token. This is an optional feature per external
identity provider and is turned on by setting the
config.storeCustomAttributes flag in the Identity Provider json. The token
must contain user_attributes and/or roles scopes for retrieving the custom
attributes and roles from the /userinfo
<http://docs.cloudfoundry.org/api/uaa/#user-info> endpoint.

- Ability to retrieve the custom user attributes from the OpenID Connect
userinfo endpoint External OIDC
<https://www.pivotaltracker.com/story/show/130477291>
- Ability to retrieve the roles from the OpenID Connect userinfo
endpoint - All Providers
<https://www.pivotaltracker.com/story/show/137497509>

Force User Password Change for UAA Internal Users

This feature allows the administrator to force all users to change their
password at next login time. This can be enforced on an individual user
basis <http://docs.cloudfoundry.org/api/uaa/#force-user-password-to-expire>.
This feature is multi-tenant and can be enabled per Identity Zone
<http://docs.cloudfoundry.org/api/uaa/index.html#force-pasword-change-for-users>
.

- Add support for User Force Password Change at next login
<https://www.pivotaltracker.com/story/show/131105231>
- Provide ability to force password change for all users in the system
<https://www.pivotaltracker.com/story/show/131113425>
- Update the Login UI to honor Force Password Change
<https://www.pivotaltracker.com/story/show/132023123>

SAML Bearer Token support

This feature enables SAML assertions to be exchanged for access tokens.
This feature has been contributed by *SAP*. The documentation can be found
here. <http://docs.cloudfoundry.org/api/uaa/#saml2-bearer-grant>

- Feature: Add saml2 bearer grant
<https://www.pivotaltracker.com/story/show/134877121>

SQL Server Support

In addition to PostGres and MySQL , UAA now supports SQL Server as a
backend. This feature has been contributed by *Microsoft*.

- Microsoft SQL Server Support as a backend
<https://www.pivotaltracker.com/story/show/136315437>

Detailed release notes are available here
<https://github.com/cloudfoundry/uaa/releases/tag/3.10.0>


Thanks,
Sree Tummidi
Staff Product Manager
Identity - Pivotal Cloud Foundry


Re: Loggregator Architecture Change: Independently Scalable Syslog

Marco Voelz
 

Thanks Adam for the permissions. The linked “Inception notes” document [1] seems to also have problems with permissions. Could you please also take care of that?

Thanks and warm regards
Marco

[1] https://docs.google.com/document/d/1LZCiFSe785BceKg9AM5btrnKucNfL11r1zobRuNy7Bk/edit


From: Adam Hevenor <ahevenor(a)pivotal.io>
Reply-To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev(a)lists.cloudfoundry.org>
Date: Wednesday, 1 February 2017 at 16:39
To: "cf-dev(a)lists.cloudfoundry.org" <cf-dev(a)lists.cloudfoundry.org>
Subject: [cf-dev] Re: Re: Loggregator Architecture Change: Independently Scalable Syslog

Hi All -

I have updated the sharing settings. Everyone should now be able to see and comment.

Thanks
Adam


Dropping support for old versions of SSL and TLS in HAProxy and Gorouter

Shannon Coen
 

When TLS is enabled on Gorouter (router.enable_ssl: true; false by
default), it will currently accept connections using SSLv3, TLSv1.0,
TLSv1.1, or TLSv1.2.

The HAProxy in cf-release always has TLS enabled and will accept
connections using TLSv1.0, TLSv1.1, or TLSv1.2.

For security reasons, we would like to drop support in these components for
all versions except TLSv1.2. Please let me know if you have a compelling
use case for maintaining support for older versions using a manifest
property. I recognize this could be an issue if apps on your deployments of
CF must continue supporting clients that do not use TLSv1.2.

Thank you,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.


Re: Routing for Isolation Segments

Carlo Alberto Ferraris
 

Shannon,
I agree with Mike - we're in a similar situation where we already have implemented all network ACLs and FW rules - so also for us access control would be the priority, and partitioning nice to have.

Thanks,
Carlo


Deprecation Notice: bosh-hm-forwarder is moving

Adam Hevenor
 

Hi All -

In our continued effort for a clean the Loggregator code base we are moving another repo outside of Loggregator. The bosh-hm-forwarder can now be found at the following locations.

New repo: https://github.com/cloudfoundry/bosh-hm-forwarder
Bosh Release: http://bosh.io/releases/github.com/cloudfoundry/bosh-hm-forwarder-release?all=1

If you happen to reference the bosh-hm-forwarder in the loggregator repo you have until March 1st to update your references. I'll bump this thread again on March 1st when we delete this from our repo.

Thanks
Adam


Recurring CAB and CF-extensions meetings

Michael Maximilien
 

fyi...
 
Since I got access to the CF Public Calendar (thanks Chip!) I moved the two recurring meetings (CAB and CF-extensions) to that calendar.
 
 
 
In theory, if you subscribed to the CF Public Calendar you should get these updates ASAP. If you added my calendar entry to yours then it should transfer. If not, delete the old and subscribe to the CF Public Calendar as that should give you everything CF related.
 
Sorry about this mishap. Hopefully, everyone now has the entries in their calendar. Just reminder, next CAB on Wed 02/15 @ 8a and next CF-extensions on Mon 02/27 @ 11a.
 
Talk to you then. Best,

------
dr.max
ibm cloud labs
silicon valley, ca
maximilien.org

3041 - 3060 of 9421