1. cf-dev@lists.cloudfoundry.org
  2. Messages
   Date   
Re: Proposal for Incubation in the Extensions PMC: MultiApps

Michael Maximilien
 

Great feedback Guillaume. 

All, please consider this proposal and including feedback in the docs so that the team can address.

Please ensure your feedback is submitted before next CAB call (scheduled for week of CF summit in Boston).

Thanks,

Max


On Thu, Mar 22, 2018 at 9:18 AM Guillaume Berche <bercheg@...> wrote:
Hi Nikolay,

Thanks for sharing the MultiApp proposal and contributing this work to the community.

It seems MTA shares many objectives with the terraform providers for cloudfoundry [1], credhub [2] and uaa [3]  previous shared in [4], such as
- declarative model
- support for service instance provisionning
- expression dependencies across elements
- integration with secure credentials systems such as credhub or vault

Some objectives seem to differ though:
- terraform isn't designed to orchestrate expressed sequences of operations (and rather relies on external orchestrators for this),
- terraform isn't desired to monitoring state of resources once provisioned (e.g. starting/started/stopped/failed) nor displaying them to end-users

The TF provider seems broader in scope since in addition of the developer persona, it also addresses the cf admin and cf operators personas (i.e. full parity with CF CLI, and plans for handling some bosh resource in the future [5])

Leveraging terraform core and its ecosystem has provided many built-in, features without extra effort, e.g.:
- dry run
- partial update
- concurrent deployments
- templating (through modules)
- UI tooling (e.g. in intellij, atom)
- local or remote state management
- ability to combine multiple providers (e.g CF AR and CF CR)
- CI/CD integration (e.g. in concourse [6])

The TF provider contributors are now focussing towards having the provider join the list of official hashicorp providers [8], and has recently received some great SAP contributions [9].

You may reach out to the team on the terraform slack channel [7], we'd be happy to proceed with further exchanges around common and diverging use-cases.


On Tue, Mar 20, 2018 at 9:38 PM, Zach Robinson <zrobinson@...> wrote:
Excited to see this proposed Nikolay!

Nikolay has been sharing this project with some of the core teams over the last couple of months.  We have even been giving some thought to adopting some of the behaviors and/or code from this extension into core CF.

Some aspects I think are especially interesting:
- declarative manifest behavior
- an opinionated packaging structure
- deployment orchestration

CAPI is especially excited to see whether the community takes to some of this tooling as potential guidance for future roadmap.

-Zach
CAPI Project Lead


On Tue, Mar 20, 2018 at 9:35 AM Nikolay Valchev <nikolay.valchev@...> wrote:

Hi All,

 

SAP is proposing MultiApps (aka CF MTA deploy service) for inclusion in the Extensions PMC as a new incubating project.

MultiApps is envisioned to provide the missing higher-level lifecycle management capabilities for distributed applications in Cloud Foundry. It proposes a new declarative application model and corresponding tools helping developers achieve the following goals:

  1. Describe explicitly the structure of my application, its constituent CF-apps, routes, services and other technical artifacts. Tools, interpreting this model, would automate deployments, achieving unified lifecycle of all application components and ensuring the consistency and completeness of the distributed application.   
  2. Declare the resources (e.g. service instances or APIs from other apps) my application depends on at runtime and/or deployment time. Tools would then check existence of required service instances, possibly allocating the missing ones and making the needed app bindings. Additionally tools would resolve dependent apps API information and inject it in requiring app env.
  3. Declare configuration variables (and their relations) which distinguish different deployments of my application. Tools would bind sub-components, automate deployment based on default settings, or request missing mandatory values interactively.

MultiApps is currently available on GitHub [1].

 

Details:

Project Name: MultiApps

Project Proposal: See [2].

Proposed Project Leads: Nikolay Valchev, SAP

Proposed Scope: See [2]

Development Operating Model: Dedicated committers (local)

Technical Approach: CF CLI plugin and server application automating lifecycle operations

Please let us know if you have any questions or feedback.

 

Best regards,

Nikolay Valchev

Development Architect and Product Owner, SAP

 

[1] https://github.com/SAP/cf-mta-deploy-service

[2] https://docs.google.com/document/d/1TCx_2gv2_n-StcV4lzbbaD4rma9HfaRG7O9wb6-egk4

 


--
dr.max Sent from my iPhone http://maximilien.org
Re: Understanding hard CPU limits

Marco Voelz
 


/cc Eric and Julz: Could you maybe help us understand this? Thanks!

From: cf-dev@... <cf-dev@...> on behalf of Grifalconi, Michael <michael.grifalconi@...>
Sent: Monday, March 26, 2018 10:54:32 AM
To: cf-dev@...
Subject: [CAUTION] [cf-dev] Understanding hard CPU limits
 
Hi,

We were trying out the hard CPU limit as per docs
https://github.com/cloudfoundry/garden-runc-release/releases?after=v1.9.2
github.com
GitHub is where people build software. More than 27 million people use GitHub to discover, fork, and contribute to over 80 million projects.



according to formula for single core machine,

APP_MEM_IN_MB * 100us / 1000 = MILLISECONDS_PER_100_MILLISECOND_PERIOD

In our tests, to get 6.4% CPU usage on a 64MB application and ~100% for a 1GB application we had to set the 'cpu_quota_per_share_in_us' to 3200. (Cell has 4 core and 16GB or ram, overcommit factor of 2).

That changes the formula to:
APP_MEM_IN_MB * 100us / 32000 = MILLISECONDS_PER_100_MILLISECOND_PERIOD

Can you help us to understand from where this 'times 32' comes from? Is the total available RAM of the cell (16GB * overcommit of 2) and number of cpu cores does not matter?

Thanks and regards,
Michael

Proposed BOSH logging interface

Jesse T. Alford
 

Hello! We're the CF Platform Logging team. We maintain `syslog-release` and have been working to improve and regularize platform logging behavior.

This is a proposal intended to establish reasonable expectations about what should be logged and what should be forwarded in bosh-deployed cloud systems.

Historically, it has been up to each release to provide for their log forwarding, if any. We intend `syslog-release` to provide a consistent interface useful enough to replace all other provisions for the forwarding of logs from bosh jobs.

## Proposed Interface
If log forwarding is enabled, some files in `/var/vcap/sys/log` (and its subdirectories, recursively), will have any line written to them forwarded as the MSG portion of an RFC5424 compliant syslog message. Which files are forwarded is governed first by file extension, and secondarily by file permissions.

`syslog-release` attempts to read any file ending in `.log`.
(This allows us to avoid forwarding rotated logs, swapfiles, etc.)
It will forward from such files if either of the following are true:
- it is world-readable
- it is readable to the `vcap` group

In particular, this means that logs will not be forwarded from files where:
- user and group are root:root
- user and group are vcap:root or vcap:none
- user and group are vcap:vcap, but it is not group-readable

…unless they are world-readable.

We think that this interface will allow us to avoid running a log forwarder with elevated permissions, while also allowing jobs to, for instance, write DEBUG or similar logs to a file that is not group-readable, thus improving their security and reducing the load on the logging system while still making them available on the ephemeral disk for debugging purposes.

## Questions
There are a couple of things around this interface we're especially interested in feedback on, in addition to the obvious "will this be a problem for you" overall question.

We may have to have a proviso that the depth of this is not unlimited. This depends somewhat on what is inexpensive to implement and maintain, and is an area we'd appreciate feedback on. Is three levels deep from `/var/vcap/sys/log` (i.e. `/var/vcap/sys/log/jobname/processname/*`) enough? Would four be?

In the old way of doing things, more control over the PRI information and other syslog fields was available to release authors. Logs forwarded from files currently all come out as PRI 14, which translates to Facility: User, Severity: Info. Additionally, the appname/tag field is set to the name of the directory containing the log file. Is this enough/good info? If we were to include the filename, too, would that be useful? Sufficient?

## Testing with the Proposed Interface
We have recently implemented a feature to help release authors evaluate the proposed interface. If you set `syslog.respect_file_permissions: true`, blackbox will not be run with elevated capabilities, and you'll be able to see what is and isn't forwarded under the proposed interface.
Feedback on service instance sharing (experimental)

Matt McNeeney
 

I'm pleased to announce that the ability to share service instances across orgs and spaces has been released!

In order to use this feature, please make sure you are running:
To share a service instance from one space into another, you just need to run cf share-service SERVICE-INSTANCE -s SPACE

Note that this needs to be enabled by both admins (via a feature-flag) and by the author of the service you want to share.

For more information, please check out the following docs:

I'd encourage you all to try this out with a supportive service broker (like this example broker). This feature is currently experimental (beta), so feedback at this time would be extremely valuable.

Best wishes,
The CF Services API team
Re: Measuring cf push times end-to-end

Adam Hevenor
 

This is an interesting idea and related to a couple of feature ideas we (Loggregator) have been doing some research on. Ultimately you would likely want that metric to be available in Loggregator so you can sum, chart, and percentile the metric using existing metric store integrations. This is simple (and may already be present) for diego and CC related actions although it should probably have clearer tags for tracing and a better API for retrieving traces. Also, the client side metric would be valuable but there is not an established OSS Support path for getting that information into Loggregator at the moment. 
Understanding hard CPU limits

Grifalconi, Michael <michael.grifalconi@...>
 

Hi,

We were trying out the hard CPU limit as per docs
https://github.com/cloudfoundry/garden-runc-release/releases?after=v1.9.2

according to formula for single core machine,

APP_MEM_IN_MB * 100us / 1000 = MILLISECONDS_PER_100_MILLISECOND_PERIOD

In our tests, to get 6.4% CPU usage on a 64MB application and ~100% for a 1GB application we had to set the 'cpu_quota_per_share_in_us' to 3200. (Cell has 4 core and 16GB or ram, overcommit factor of 2).

That changes the formula to:
APP_MEM_IN_MB * 100us / 32000 = MILLISECONDS_PER_100_MILLISECOND_PERIOD

Can you help us to understand from where this 'times 32' comes from? Is the total available RAM of the cell (16GB * overcommit of 2) and number of cpu cores does not matter?

Thanks and regards,
Michael
Measuring cf push times end-to-end

Steffen Uhlig
 

As part of developing and improving the bits-service, we would like to better understand what the overall app push times are that developers experience.
 
However, "cf push" is a concept only known to the CLI. The CC and other CF components are not aware of various requests that make up a single "cf push" (e.g. resource matching, building a package, droplet). As a result, we have not been able to identify a straightforward way to measure push times from a developer's perspective.
 
One potential approach is to determine push time at the client side and then (after cf push has completed) have the CLI submit this data to a new endpoint in CF.
 
What do people think about this? Are there better ways to get these numbers? If not, would the community be open for a new "telemetry" endpoint that accepts these measurements from the cli and/or other cf components?
Re: Proposal for Incubation in the Extensions PMC: MultiApps

Guillaume Berche
 

Hi Nikolay,

Thanks for sharing the MultiApp proposal and contributing this work to the community.

It seems MTA shares many objectives with the terraform providers for cloudfoundry [1], credhub [2] and uaa [3]  previous shared in [4], such as
- declarative model
- support for service instance provisionning
- expression dependencies across elements
- integration with secure credentials systems such as credhub or vault

Some objectives seem to differ though:
- terraform isn't designed to orchestrate expressed sequences of operations (and rather relies on external orchestrators for this),
- terraform isn't desired to monitoring state of resources once provisioned (e.g. starting/started/stopped/failed) nor displaying them to end-users

The TF provider seems broader in scope since in addition of the developer persona, it also addresses the cf admin and cf operators personas (i.e. full parity with CF CLI, and plans for handling some bosh resource in the future [5])

Leveraging terraform core and its ecosystem has provided many built-in, features without extra effort, e.g.:
- dry run
- partial update
- concurrent deployments
- templating (through modules)
- UI tooling (e.g. in intellij, atom)
- local or remote state management
- ability to combine multiple providers (e.g CF AR and CF CR)
- CI/CD integration (e.g. in concourse [6])

The TF provider contributors are now focussing towards having the provider join the list of official hashicorp providers [8], and has recently received some great SAP contributions [9].

You may reach out to the team on the terraform slack channel [7], we'd be happy to proceed with further exchanges around common and diverging use-cases.


On Tue, Mar 20, 2018 at 9:38 PM, Zach Robinson <zrobinson@...> wrote:
Excited to see this proposed Nikolay!

Nikolay has been sharing this project with some of the core teams over the last couple of months.  We have even been giving some thought to adopting some of the behaviors and/or code from this extension into core CF.

Some aspects I think are especially interesting:
- declarative manifest behavior
- an opinionated packaging structure
- deployment orchestration

CAPI is especially excited to see whether the community takes to some of this tooling as potential guidance for future roadmap.

-Zach
CAPI Project Lead


On Tue, Mar 20, 2018 at 9:35 AM Nikolay Valchev <nikolay.valchev@...> wrote:

Hi All,

 

SAP is proposing MultiApps (aka CF MTA deploy service) for inclusion in the Extensions PMC as a new incubating project.

MultiApps is envisioned to provide the missing higher-level lifecycle management capabilities for distributed applications in Cloud Foundry. It proposes a new declarative application model and corresponding tools helping developers achieve the following goals:

  1. Describe explicitly the structure of my application, its constituent CF-apps, routes, services and other technical artifacts. Tools, interpreting this model, would automate deployments, achieving unified lifecycle of all application components and ensuring the consistency and completeness of the distributed application.   
  2. Declare the resources (e.g. service instances or APIs from other apps) my application depends on at runtime and/or deployment time. Tools would then check existence of required service instances, possibly allocating the missing ones and making the needed app bindings. Additionally tools would resolve dependent apps API information and inject it in requiring app env.
  3. Declare configuration variables (and their relations) which distinguish different deployments of my application. Tools would bind sub-components, automate deployment based on default settings, or request missing mandatory values interactively.

MultiApps is currently available on GitHub [1].

 

Details:

Project Name: MultiApps

Project Proposal: See [2].

Proposed Project Leads: Nikolay Valchev, SAP

Proposed Scope: See [2]

Development Operating Model: Dedicated committers (local)

Technical Approach: CF CLI plugin and server application automating lifecycle operations

Please let us know if you have any questions or feedback.

 

Best regards,

Nikolay Valchev

Development Architect and Product Owner, SAP

 

[1] https://github.com/SAP/cf-mta-deploy-service

[2] https://docs.google.com/document/d/1TCx_2gv2_n-StcV4lzbbaD4rma9HfaRG7O9wb6-egk4

 
Re: A New Packaging Approach - CLI Tools Pushing Apps and the Story about the #loggregator “space drain” experiment #loggregator

Mike Youngstrom
 

Yeah. Currently the user name / password is simply passed into the app. I expect well at least implement a better hook into CUPS next and for user roles that can create users we could shell out and create a strong user-name password combo. This could be pretty elegant and allow for (automated) cred rotation. 

Sounds good.  Currently in my environment we disable user account creation instead using enterprise LDAP or OAuth.  Supporting internal users and external users isn't something I've considered in my deployments.  Might not be a problem.  Just not something I've considered before.

Thanks,
Mike
 
Re: A New Packaging Approach - CLI Tools Pushing Apps and the Story about the #loggregator “space drain” experiment #loggregator

Adam Hevenor
 

Yeah. Currently the user name / password is simply passed into the app. I expect well at least implement a better hook into CUPS next and for user roles that can create users we could shell out and create a strong user-name password combo. This could be pretty elegant and allow for (automated) cred rotation. 

The service broker approach is definitely an option to consider long term, but I'd like to try and figure out a way to do this within the scope the CC and UAA provide if it all possible. 
CF Networking support for CNI plugin chaining question

Amelia Downs
 

Hi CF,

We on the Container Networking Team are working on an epic to enable CNI plugin chaining.  

If this is of interest to you, we would like your feedback on our following proposal. 

In the spirit of doing the simplest thing, we would like to reduce the complexity of possible configurations. We want to allow users to EITHER provide a conf for a non-chaining CNI plugin OR provide a conflist for CNI plugin chaining.

Does anyone have any use cases that wouldn’t fit this model? 

Thanks,
The CF Networking Team
Re: Proposal for Incubation in the Extensions PMC: MultiApps

Zach Robinson
 

Excited to see this proposed Nikolay!

Nikolay has been sharing this project with some of the core teams over the last couple of months.  We have even been giving some thought to adopting some of the behaviors and/or code from this extension into core CF.

Some aspects I think are especially interesting:
- declarative manifest behavior
- an opinionated packaging structure
- deployment orchestration

CAPI is especially excited to see whether the community takes to some of this tooling as potential guidance for future roadmap.

-Zach
CAPI Project Lead


On Tue, Mar 20, 2018 at 9:35 AM Nikolay Valchev <nikolay.valchev@...> wrote:

Hi All,

 

SAP is proposing MultiApps (aka CF MTA deploy service) for inclusion in the Extensions PMC as a new incubating project.

MultiApps is envisioned to provide the missing higher-level lifecycle management capabilities for distributed applications in Cloud Foundry. It proposes a new declarative application model and corresponding tools helping developers achieve the following goals:

  1. Describe explicitly the structure of my application, its constituent CF-apps, routes, services and other technical artifacts. Tools, interpreting this model, would automate deployments, achieving unified lifecycle of all application components and ensuring the consistency and completeness of the distributed application.   
  2. Declare the resources (e.g. service instances or APIs from other apps) my application depends on at runtime and/or deployment time. Tools would then check existence of required service instances, possibly allocating the missing ones and making the needed app bindings. Additionally tools would resolve dependent apps API information and inject it in requiring app env.
  3. Declare configuration variables (and their relations) which distinguish different deployments of my application. Tools would bind sub-components, automate deployment based on default settings, or request missing mandatory values interactively.

MultiApps is currently available on GitHub [1].

 

Details:

Project Name: MultiApps

Project Proposal: See [2].

Proposed Project Leads: Nikolay Valchev, SAP

Proposed Scope: See [2]

Development Operating Model: Dedicated committers (local)

Technical Approach: CF CLI plugin and server application automating lifecycle operations

Please let us know if you have any questions or feedback.

 

Best regards,

Nikolay Valchev

Development Architect and Product Owner, SAP

 

[1] https://github.com/SAP/cf-mta-deploy-service

[2] https://docs.google.com/document/d/1TCx_2gv2_n-StcV4lzbbaD4rma9HfaRG7O9wb6-egk4

 
Proposal for Incubation in the Extensions PMC: MultiApps

Nikolay Valchev
 

Hi All,

 

SAP is proposing MultiApps (aka CF MTA deploy service) for inclusion in the Extensions PMC as a new incubating project.

MultiApps is envisioned to provide the missing higher-level lifecycle management capabilities for distributed applications in Cloud Foundry. It proposes a new declarative application model and corresponding tools helping developers achieve the following goals:

  1. Describe explicitly the structure of my application, its constituent CF-apps, routes, services and other technical artifacts. Tools, interpreting this model, would automate deployments, achieving unified lifecycle of all application components and ensuring the consistency and completeness of the distributed application.   
  2. Declare the resources (e.g. service instances or APIs from other apps) my application depends on at runtime and/or deployment time. Tools would then check existence of required service instances, possibly allocating the missing ones and making the needed app bindings. Additionally tools would resolve dependent apps API information and inject it in requiring app env.
  3. Declare configuration variables (and their relations) which distinguish different deployments of my application. Tools would bind sub-components, automate deployment based on default settings, or request missing mandatory values interactively.

MultiApps is currently available on GitHub [1].

 

Details:

Project Name: MultiApps

Project Proposal: See [2].

Proposed Project Leads: Nikolay Valchev, SAP

Proposed Scope: See [2]

Development Operating Model: Dedicated committers (local)

Technical Approach: CF CLI plugin and server application automating lifecycle operations

Please let us know if you have any questions or feedback.

 

Best regards,

Nikolay Valchev

Development Architect and Product Owner, SAP

 

[1] https://github.com/SAP/cf-mta-deploy-service

[2] https://docs.google.com/document/d/1TCx_2gv2_n-StcV4lzbbaD4rma9HfaRG7O9wb6-egk4

 
FINAL REMINDER: CAB call for March is Wednesday 03/21 @ 8a PST

Michael Maximilien
 

FYI...

See below. Zoom soon. Best,

dr.max
ibm ☁ 
silicon valley, ca



dr.max
ibm ☁ 
silicon valley, ca


On Mar 14, 2018, at 12:06 PM, Michael Maximilien <maxim@...> wrote:

FYI...
 
Reminder that the CAB call for March is scheduled for next Wednesday 03/21 @ 8a PST.
 
WIP agenda and zoom info in [1] but summary here:
  1. CFF updates.
  2. Highlights from different PMCs and teams.
  3. Community talks:
a. MS SQL Service Broker proposal [3] by Jared Gordon of Pivotal and team (Pivotal and Microsoft)
b. CF-Dev [4] proposal [5] by Stephen Levine of Pivotal
 
I will send one more reminder next week on this list. Zoom soon.
 
Best,

PS: next month the CAB call will be during CF Summit in Boston and we will try to get a room so we can do it live! If you will be at summit, plan to join us.
 
------
dr.max
ibm ☁ 
silicon valley, ca
 
Re: A New Packaging Approach - CLI Tools Pushing Apps and the Story about the #loggregator “space drain” experiment #loggregator

Mike Youngstrom
 

It is an interesting idea/concept.

I'd say the auth model is a significant Con and potential large hurdle.  I'm assuming the username and password passed with the cli is then used by the pushed app to continually query the space for new apps and bind the service, correct?  I'm interested in seeing what UAA and Perm could come up with to improve this.  It seems you'd either need to dynamically generate a service account with developer access to the space.  Or figure out some way to give apps running in a space the ability to create a token to do management operations in the space it exists in.  Neither seems very simple.

Another Con would be lack of operator control over upgrading such a feature.  If new versions this app are released end users would need to know to run the command again to upgrade it.  Might be tricky as CC APIs evolve and such.  Anything that the user doesn't feel like they have ownership over I as the operator would like to be able to upgrade for them.  A CLI plugin pushed app sits in the middle somewhere.

It seems to me a simpler approach to the original problem of sending all logs for apps in a space to a drain could be solved with a service broker.  This broker would need CF_ADMIN client credentials but could simply scan all spaces for instances of itself then binding itself to all apps in spaces where it finds itself.  As a broker it could be upgraded by the operators and would provide less permission complexities.

Thoughts?

Mike

On Mon, Mar 19, 2018 at 4:17 PM, Adam Hevenor <ahevenor@...> wrote:

Back Story
The Loggregator team has recently been seeing a ton of benefit from developing CLI plugins. Specifically they are quick to develop, and distribute and lend themselves well to rapid iteration based on feedback. Developing our noisy neighbor nozzle we worked closely with the Pivotal early access program to meaningfully shape the final product that we eventually posted here. This feedback loop helped us focus on the trying to solve specific packaging UX problems with a plugin and arrived at a new techniques that packaged apps along with a CLI plugin. We have since taken those ideas a step further.

Space Drain
We recently started enhancing the "drain" functionality of Loggregator  with a plugin and it’s been a long standing idea to allow app developers to drain their whole space. If you look at the doc, you’ll see that multiple teams weighed in and the effort was shelved due to a break in conventions and a lack of clarity (at the time) around the precise persona. In my mind I filed this under high value/high complexity and figured we’d come back to it eventually.

But our noisy neighbor nozzle experience got us thinking. What if we we packaged an application with the CLI that was pushed when you ran a command, and that application managed the drain bindings for all the apps in your space. All of a sudden an initiative that involved 3 or more teams, was simplified into a 4 or 5 points in our backlog. Part of the work was figuring out some of the packaging technique so we ended up cutting a version of this in the latest cf-drain release. 

(It’s helpful for context to try this out which you can do so with the following command.)

Feedback & Pro / Cons
I am posting this because I want feedback on this approach before taking it further. The pattern is a liberating one and we have several ideas that we are considering executing on that follow this same approach. 

Pro’s

  • Rapid Iteration with established versioning approach
  • Ability to deliver pushable apps without compile instructions or OS specific scripts
  • Self Service for App Developers

Con’s

  • The auth model seems in-elegant. (UAA Team has been providing some feedback on this and I hope to improve this experience)
  • Self Service for App Developers (I am listing this as a con also because controlling operators may find this problematic)
A New Packaging Approach - CLI Tools Pushing Apps and the Story about the #loggregator “space drain” experiment #loggregator

Adam Hevenor
 

Back Story
The Loggregator team has recently been seeing a ton of benefit from developing CLI plugins. Specifically they are quick to develop, and distribute and lend themselves well to rapid iteration based on feedback. Developing our noisy neighbor nozzle we worked closely with the Pivotal early access program to meaningfully shape the final product that we eventually posted here. This feedback loop helped us focus on the trying to solve specific packaging UX problems with a plugin and arrived at a new techniques that packaged apps along with a CLI plugin. We have since taken those ideas a step further.

Space Drain
We recently started enhancing the "drain" functionality of Loggregator  with a plugin and it’s been a long standing idea to allow app developers to drain their whole space. If you look at the doc, you’ll see that multiple teams weighed in and the effort was shelved due to a break in conventions and a lack of clarity (at the time) around the precise persona. In my mind I filed this under high value/high complexity and figured we’d come back to it eventually.

But our noisy neighbor nozzle experience got us thinking. What if we we packaged an application with the CLI that was pushed when you ran a command, and that application managed the drain bindings for all the apps in your space. All of a sudden an initiative that involved 3 or more teams, was simplified into a 4 or 5 points in our backlog. Part of the work was figuring out some of the packaging technique so we ended up cutting a version of this in the latest cf-drain release. 

(It’s helpful for context to try this out which you can do so with the following command.)

Feedback & Pro / Cons
I am posting this because I want feedback on this approach before taking it further. The pattern is a liberating one and we have several ideas that we are considering executing on that follow this same approach. 

Pro’s

  • Rapid Iteration with established versioning approach
  • Ability to deliver pushable apps without compile instructions or OS specific scripts
  • Self Service for App Developers

Con’s

  • The auth model seems in-elegant. (UAA Team has been providing some feedback on this and I hope to improve this experience)
  • Self Service for App Developers (I am listing this as a con also because controlling operators may find this problematic)
Re: Feedback request: Disable logging Client IP’s in the Gorouter logs for compliance with the EU General Data Protection Regulation (GDPR)

Stefan Mayr
 

Hi,

Am 13.03.2018 um 00:39 schrieb Shubha Anjur Tupil:
Hello everyone, 

In lieu of The EU General Data Protection Regulation (GDPR), the routing
team is investigating adding manifest properties to allow an operator to
disable logging the client IP's in the |X-Forwarded-For| header in the
access and error logs for Gorouter.

Enforcement of The EU General Data Protection Regulation (GDPR)
(https://www.eugdpr.org/) begins May 28th and imposes steep fines. This
law says that companies will be fined if they are capturing PII. The
Gorouter currently captures Client IP addresses that are included in
that definition. We are exploring manifest properties to allow operators
to disable logging the originating client IP.

We need help weighing the options given the use-cases.

*Use-cases*:

#1. L3 Load Balancer e.g. Amazon NLB in front of the Gorouter 
The client IP is logged and the X-Forwarded-For header might have the
originating client IP if an intermediary component or the originating
client is adding the header.

#2. L7 Load Balancer e.g. Amazon NLB in front of the Gorotuer 
The client IP is the IP of the load balancer in front of the Gorouter,
but the X-Forwarded-For header has the originating client IP.

*Option1*: We have one manifest property to disable logging both the
X-Forwarded-For header and the Client IP in the Gorouter logs.

For use-case #1 this works, and there is only one manifest property to
disable all PII being logged by the Gorouter.

For use-case #2 this results in the operator losing the LB IP which
might be helpful for troubleshooting.

*Option2*: We have two separate manifest properties to disable logging
the X-Forwarded-For header and/or the Client IP in the Gorouter logs.
This is generally more flexible at the cost of user experience.

For use-case #1 this would mean that an operator would have to set two
manifest properties instead of one. Both these properties would need to
be exposed in Ops Man for PCF installations. It leads to a more
cumbersome user experience, adding to the already long list of options.

For use-case #2 this results in a better outcome for an operator in that
they still get the information on the LB the request came from, while
the originating client IP will not be logged.

Based on the information we have, we are not sure which experience is
better and which use-case to optimize for. Some questions we have

1. What type of LB is more popular in CF environments; L3 or L7? This
might help us optimize for Use-case #2 and go with Option #2.
Our deployment scenario is #2. The L7 load balancer inserts a
X-Forwarded-For header.

2. Is there a compelling reason from an operator perspective for
installations with a L7 load balancer, making it important to have
the Client IP (i.e. they would absolutely want to have Option #2 or
maybe don’t care and Option #1 would be ok)?
All external request to gorouter are coming through the L7 load
balancer. So for most debugging use cases the load balancer IP in
gorouter access log would not provide any value - but the client IP from
the X-Forwarded-For header does. This brings up another point: it is not
generally forbidden to store ip addresses for specific uses cases, at
least in germany. E.g, as far as I understand the current situation we
are allowed the store IP addresses for defense or debugging purpose for
a limited time if (!) this is documented in the public data privacy
statement. It looks like 1-2 weeks are generally acceptable for this use
case.

3. Does the user experience benefit by having just one option outweigh
the benefit of having the client IP for an operator with an L7 Load
Balancer? (Flexibility over Experience)
For us this would not provide any advantage to have seperate
configuration options. If we cannot limit the timeframe keept by
logrotation we would disable IP logging.

We would love to hear from you if you have thoughts on this.
This topics raises some other questions about the IP addresses being
passed around in multiple components. gorouter creates the RTR log
messages that we can read from loggregator. Does loggregator buffer
those messages or disk or is this only kept in memory? When is this
information purged from loggregator?
Next component we see is the buildpack. Some also show classic access
log information on stdout. Did anybody check if none of the buildpacks
writes this information to disk?


Thanks, 

Shannon Coen & Shubha Anjur Tupil
Product Managers, CF Routing
Regards,

Stefan
Re: Cloud Foundry and Kubernetes Integration Efforts

Krannich, Bernd <bernd.krannich@...>
 

Hello Shannon,

 

I think you are referring to my original mail that was pointing to four documents in total, correct?

 

 

Please let me know if you continue to face issues with commenting in any of the docs.

 

Thanks in advance,

Bernd

 

From: <cf-dev@...> on behalf of Shannon Coen <scoen@...>
Reply-To: "cf-dev@..." <cf-dev@...>
Date: Friday, 16. March 2018 at 18:59
To: "cf-dev@..." <cf-dev@...>
Subject: Re: [cf-dev] Cloud Foundry and Kubernetes Integration Efforts

 

Hello Bernd,

 

Would you please grant anyone with the link permissions to comment on these docs?

 

Thank you.


Shannon Coen

Product Manager, Cloud Foundry

Pivotal, Inc.

 

On Fri, Jan 5, 2018 at 4:03 PM, Adam Hevenor <ahevenor@...> wrote:

Bernd - 

Happy New Year! 

I linked another proposal I have been working on that expands on the ideas of containerizing loggregator and the benefits of "unified logging" (or I like to refer to it as egress). This has benefits beyond PKS, which include stand alone service, multi-foundation and general observability mind share. 

These ideas have really gotten me excited and I think there is a strong case for making unified egress a first class concern of both platforms and we are bubbling with ideas to improve it further. Of course I am now also probably guilty of imagining an attractive end state without thinking through all of the steps along the way - but it feels like we are taking those first steps by sharing these ideas. 

I'll reach out separately to coordinate some offline communications on this subject. 

Adam 

 
Identifying CF Resources

Christopher Brown <cbrown@...>
 

Hi all,

The Permissions component ("Perm") we're building to store and evaluate roles for Cloud Foundry needs to be able to identify resources across component boundaries so that it knows which policies to apply for any given permission check. AWS does this using their ARN scheme.

I'd like to work out if something like this would be the correct approach for Perm. We'd like to find something which can identify resources across foundations and encode the object hierarchy into the resource identifier. For example, an application inside PWS could be identified by:

pws:cc:application/[org guid]/[space guid]/[app guid]

I don't want to get too into the weeds of the exact format yet but if you:
  • ...have already built something like this to help with cross-foundation operations (or any other reason)
  • ...are looking into working on something like this to help with a different problem
then I'd be interested in talking with you about this requirement and what the solution could look like.

All the best,
Christopher
CF Permissions PM
Re: Cloud Foundry and Kubernetes Integration Efforts

Shannon Coen
 

Hello Bernd,

Would you please grant anyone with the link permissions to comment on these docs?

Thank you.

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Fri, Jan 5, 2018 at 4:03 PM, Adam Hevenor <ahevenor@...> wrote:
Bernd - 

Happy New Year! 

I linked another proposal I have been working on that expands on the ideas of containerizing loggregator and the benefits of "unified logging" (or I like to refer to it as egress). This has benefits beyond PKS, which include stand alone service, multi-foundation and general observability mind share. 

These ideas have really gotten me excited and I think there is a strong case for making unified egress a first class concern of both platforms and we are bubbling with ideas to improve it further. Of course I am now also probably guilty of imagining an attractive end state without thinking through all of the steps along the way - but it feels like we are taking those first steps by sharing these ideas. 

I'll reach out separately to coordinate some offline communications on this subject. 

Adam 


1501 - 1520 of 9389