|
Feature Narrative / Proposal: Let's fix* CPU Sharing and Metrics in CF!
Julz Friedman
Hi cf-dev- Here is a feature narrative. The feature narrative is called "Let's Fix CPU Sharing and Metrics in CF" (but actually it's just a proposal to make them quite a lot better). More information about the feature narrative is contained in the feature narrative. Please enjoy the feature narrative. Comments, feedback, suggestions, and questions very welcome and appreciated! Thanks, Julz |
|
|
|
Re: [cf-bosh] Incubation proposal: CF Containerization
Dmitriy Kalinin <dkalinin@...>
Thank you for submitting this proposal. Let's shoot for collecting and resolving most of the comments in the next month by Aug 10th and voting at that time to incubate it in BOSH PMC. On Tue, Jul 3, 2018 at 2:25 AM, Cornelius Schumacher <cschum@...> wrote: Hi all, |
|
|
|
Re: BlockHeads Broker - Proposal for Incubation in the Extensions
Michael Maximilien
Hi, all, It’s been about one month since this proposal has been submitted. I don’t see any pending unresolved issues or comments. Therefore, according to our process [0] and Nima’s request to move a vote, I am giving everyone a last opportunity for comments until EOD Wednesday 07/18. This is also the day of CAB call in case you want to chat with him, myself, or others who attend the call in person. After 07/18, assuming no comments, we will move for a vote. Best, Max On Wed, Jun 20, 2018 at 9:57 AM Nima <nkavian@...> wrote:
--
dr.max
Sent from my iPhone
http://maximilien.org |
|
|
|
Re: cf-deployment 3.0
Marco Voelz
Dear Josh,
You are correct, in the past the RelInt team hasn't provided security releases. Instead, the credo was to go forward with the regular releases to also get the newest security fixes. This, however, was only easily possible because *the newer version did not introduce breaking changes with potentially big impact at the same time*.
I understand your mission of helping other teams increase their velocity. Maintaining multiple branches with fixes is certainly not fun, and I agree that it makes sense to try to avoid this if possible. I'm not sure I get the container networking 2.0 reference, though. Could you elaborate a bit more on this and how it is related to the current discussion?
Thanks and warm regards Marco
From: <cf-dev@...> on behalf of Josh Collins <jcollins@...>
The Release Integration team hasn't provided security releases in the past -- for neither cf-release nor cf-deployment -- and doing so would be burdensome and impede the evolution of cf-deployment. Therefore, we're not currently planning to start providing security patches. But we appreciate the feedback and will keep an eye on the problem.
|
|
|
|
CAB call for July is Wednesday 07/18 @ 8a PST or 11a EST
Michael Maximilien
FYI... Please remember to join the Zoom call [0] Wednesday July 18th at 8a Pacific for QAs, highlights, and two presentations: 1. Project Shield v8 Updates by James Hunt of Stark & Wayne [1] 2. CF-Extensions Project Service Fabrik Updates by Ashish Jain of SAP [2] and [3] Zoom soon. Best, |
|
|
|
Re: Clarification regarding Custom roles in CF
Christopher Brown
Hi there, Thank you for the feedback! Unfortunately it's not currently possible to customize the existing roles or add new roles to the system. However, we (the CF Permissions team) are working on adding these features at the moment. You can read more about the project in our incubation proposal. Progress has been slower than we had hoped but we're advancing and are looking for a better way to expose this progress to the community so that folks can follow along. I'll post an update once we have something to show. Please let me know if you have any other questions. Thanks and all the best, Christopher CF Permissions PM On Thu, Jun 28, 2018 at 12:41 AM Raghav, Prashant <prashant.raghav@...> wrote:
|
|
|
|
Re: cf-deployment 3.0
Josh Collins
The Release Integration team hasn't provided security releases in the past -- for neither cf-release nor cf-deployment -- and doing so would be burdensome and impede the evolution of cf-deployment. Therefore, we're not currently planning to start providing security patches. But we appreciate the feedback and will keep an eye on the problem.
Because the RelInt team's primary goal is to support the CF Foundation engineering teams and their ability to validate their commits in CI, we need to focus more on keeping up-to-date with their changes. We want to set a release cadence that's aligned with, and ideally increases, the velocity of the teams. Take a look at the what happened with container networking when they wanted to ship 2.0... Thanks for reaching out Geoff! |
|
|
|
Feedback requested: More than one service possible with the same name
Jennifer Spinney
Hi cf-dev,
We’d like to know if anyone is using this endpoint in that way and whether they’d be affected by this query now possibly returning more than one service in the returned array. Thanks, Services API (SAPI) Team |
|
|
|
Re: [cf-bosh] Xenial stemcells now available: migration plan
Dr Nic Williams <drnicwilliams@...>
Frédéric, when will Xenial stemcells move from X to X.Y version numbering with less regular updates of the major version X? This will help compiled releases to not have to update as often.
Nic
From: 30461002200n behalf of
Sent: Wednesday, July 11, 2018 7:02 am To: cf-bosh@...; cf-dev@... Subject: [cf-bosh] Xenial stemcells now available: migration plan Hi everyone.
My name is Frédéric. I joined the BOSH team as a product manager recently, and work from Pivotal's Toronto office. Nice to meet you all!
A few weeks ago, the BOSH team introduced a new stemcell line based on Ubuntu 16.04 (Xenial Xerus) on bosh.io. For the time being, this line will be maintained in parallel with the previous ones, based on Ubuntu 14.04 (Trusty Tahr). Canonical will provide security updates for Trusty until April 2019 per their official support lifecycle policy. Because Canonical will no longer provide security updates for Trusty after April 2019, we strongly recommend users start migrating towards the Xenial-based stemcell line now. The BOSH team will continue to support the current 3586.x line of Trusty-based stemcells with upstream security patches until the CFAR migration to Xenial is complete. We do not plan on releasing any new major versions of Trusty-based stemcells, unless consumers have a specific request for a new major, and instead will focus on Xenial going forward. We are currently evaluating when we will retire the Trusty-based stemcells, and are looking for feedback from the community about technical blockers that could impede the adoption of Xenial stemcells. If you are a release author, please take the time to verify your software on Xenial-based stemcells at the earliest opportunity. As a reminder, operators using cf-deployment must keep in mind that the repository will switch to Xenial-based stemcells well before April 2019 and should plan accordingly (more details on this can be found here). Frédéric Desbiens
Product Manager | Pivotal Cloud Foundry BOSH
|
|
|
|
Xenial stemcells now available: migration plan
Hi everyone. My name is Frédéric. I joined the BOSH team as a product manager recently, and work from Pivotal's Toronto office. Nice to meet you all! A few weeks ago, the BOSH team introduced a new stemcell line based on Ubuntu 16.04 (Xenial Xerus) on bosh.io. For the time being, this line will be maintained in parallel with the previous ones, based on Ubuntu 14.04 (Trusty Tahr). Canonical will provide security updates for Trusty until April 2019 per their official support lifecycle policy. Because Canonical will no longer provide security updates for Trusty after April 2019, we strongly recommend users start migrating towards the Xenial-based stemcell line now. The BOSH team will continue to support the current 3586.x line of Trusty-based stemcells with upstream security patches until the CFAR migration to Xenial is complete. We do not plan on releasing any new major versions of Trusty-based stemcells, unless consumers have a specific request for a new major, and instead will focus on Xenial going forward. We are currently evaluating when we will retire the Trusty-based stemcells, and are looking for feedback from the community about technical blockers that could impede the adoption of Xenial stemcells. If you are a release author, please take the time to verify your software on Xenial-based stemcells at the earliest opportunity. As a reminder, operators using cf-deployment must keep in mind that the repository will switch to Xenial-based stemcells well before April 2019 and should plan accordingly (more details on this can be found here). Frédéric Desbiens Product Manager | Pivotal Cloud Foundry BOSH |
|
|
|
Re: CF Application Runtime PMC - CF Networking Project Lead Call for Nominations
Dieu Cao <dcao@...>
Hello all, Pivotal is nominating Preethi Varambally for the CF Networking Project Lead in the Application Runtime PMC. Any other nominations should be sent to me/in reply by end of day July 3rd, 2018. Preethi previously worked as a technical product owner and business analyst at a GIS company on a customer facing web application. Here, along with managing the product, she also worked on designing application layer data flow architecture for efficiently getting data from various source systems. Prior to this, she worked as an engineer, building data tracking and reporting tools using MVC framework and gradually moved to a product owner role managing a team of onshore and offshore engineers. Preethi holds a Masters degree in Computer Science from University of Texas, Dallas. -Dieu Cao On Mon, Jul 2, 2018 at 11:45 AM Dieu Cao <dcao@...> wrote:
|
|
|
|
cf-deployment transition to Xenial stemcell
Josh Collins
CF-Deployment default to Ubuntu 16.04 (Xenial Xerus) BOSH stemcells in late-Fall/early-Winter 2018.Canonical will stop providing security updates for the current Ubuntu 14.04 (Trusty Tahr), the current cf-deployment default, in April 2019 per their official support lifecycle policy. CF-Deployment plans to transition to the fully-supported Xenial stemcell lines well before April 2019 and we’ll need each component team to validate the Xenial stemcell against their components before making the switch. If each component's features are confirmed compatible before the end of September 2018, we should be able to hit our intended target for the transition. CF-D/Xenial Status to-date:
This message is a gentle invitation to begin/continue validating your component against Xenial. Note that RelInt validates releases via CATs. The scope of CATs is purposely limited to validating core cf workflows. The stability of cf-deployment releases assumes each component team executes a much deeper level of component-specific testing before integrating into cf-deployment. For this reason, we don't/won't recommend using Xenial for production installs of cf-deployment nor make Xenial default in CF-D until each component team has validated that all its supported features are compatible with the Xenial stemcell. We've proven Xenial can be run in a pipeline and CATs will pass so now's the time. Please do due diligence and let us know when you've completed your validation. Thanks so much! Josh Collins Release Integration PM |
|
|
|
Service Plan Descriptions
Alexander Blease <ablease@...>
Hi Everyone, The Services API team is thinking about limiting the size of service plan descriptions to 10,000 characters, upon registering a service broker in Cloud Foundry. We would be keen to hear from people who have specific use cases for descriptions that are longer than 10,000 chars. If we imposed this limit, would you strongly object to this change, and why? If you have thoughts, queries, concerns, please contact us on the #sapi channel on the Cloud Foundry Slack, or respond directly to this email. Cheers, Alex + Niki |
|
|
|
Re: New PM for CF AutoScaler project [Re: PM Vacancy for Auto-Scaler]
Marco Nicosia
Welcome to the project, Bo! On Thu, Jul 5, 2018 at 07:38 Michael Maximilien <maxim@...> wrote:
--
|
|
|
|
Re: cf-deployment 3.0
Franks, Geoff
How long will 1.x, and 2.x cf-deployments be maintained with security patches? Without that, it sounds like there’s potential for a lot of organizations to be faced with breaking changes and instability every time they upgrade (if upgrade cycles internally take a month or two, and major versions are coming out as often or more), not to mention the difficulties of jumping multiple major versions at once.
From:
<cf-dev@...> on behalf of Josh Collins <jcollins@...>
Hey Y'all,
Cf-deployment 3.0 is around the corner. We're going to go 3.0 in 2-3 weeks.
We released cf-deployment 2.0 on June 18th and included 'breaking' changes.
Breaking changes in the context of cf-d are changes which would require special attention from operators for the deployment to succeed. Executing the same bosh deploy command/args run used in the previous deployment may fail depending on which ops files and features operators had deployed with in the past.
Going forward, we'd like to introduce a more regular (~monthly) cadence to major point releases of cf-deployment.
The goal is two-fold and in-order-of-importance:
As of today, we've got one PR that includes breaking changes and I'm putting out a call to y'all. If you've got what you'd consider to be a breaking change that warrants going out in a major point release of cf-deployment, please submit your PRs and reach out to the RelInt team as soon as you're able to so we can come up to speed and support you!
Cheers,
Josh
|
|
|
|
New PM for CF AutoScaler project [Re: PM Vacancy for Auto-Scaler]
Michael Maximilien
Hi, all,
toggle quoted message
Show quoted text
Please welcome Bo Yang (on cc:) of IBM as PM for CF App AutoScaler Extensions project [0]. You can keep up with the project’s monthly drumbeat as well as other CF Extensions updates here [1]. Welcome Bo, I look forward to working with you. Best, dr.max ibm ☁ silicon valley, ca On Jun 25, 2018, at 1:02 PM, Michael Maximilien <maxim@...> wrote:
|
|
|
|
Re: [CAUTION] Re: [cf-dev] Proposed BOSH logging interface
Marco Voelz
Dear Jesse,
did anything come out of this proposal? Did you end up picking up this track of work?
Warm regards Marco
From: <cf-dev@...> on behalf of Marco Voelz <marco.voelz@...>
Dear Jesse,
Thanks for putting this proposal out there. We would be happy to see an automated logfile forwarding mechanism. Here's a couple of comments on your initial points: * Including the filename in the syslog metadata is very useful and something we'd really like to have. Currently it is something we're working around a bit. * The appname/tag field should probably contain the release's name as well as a prefix. My proposal here is `<deployment name>.<instance group name>.<job name>`. wdyt? * We haven't made any particular use of the priority field, so losing control over this field wouldn't matter for out use-cases. Severity is usually something that the actual log message needs to contain, as the logger's severity can only be set on its initial creation, afaik. * Restricting the depth of recursion seems reasonable. So far, I don't think we're using bosh releases which have more than 1 folder below their /var/vcap/sys/log/<job name>/ folder.
Concerning the requirements about permissions on the logfiles you'd want to forward: Did you talk to Dmitriy/the BOSH team about this? With stemcell series 3541.x the permissions on the standard folders below /var/vcap were tightened a bit, so just wanted to make sure that your assumptions are in line with the upcoming changes in the stemcells.
Warm regards Marco From: cf-dev@... <cf-dev@...> on behalf of Jesse T. Alford <jalford@...>
Sent: Tuesday, April 3, 2018 12:55:38 AM To: cf-dev@... Subject: [cf-dev] Proposed BOSH logging interface
Hello! We're the CF Platform Logging team. We maintain `syslog-release` and have been working to improve and regularize platform logging behavior.
This is a proposal intended to establish reasonable expectations about what should be logged and what should be forwarded in bosh-deployed cloud systems.
Historically, it has been up to each release to provide for their log forwarding, if any. We intend `syslog-release` to provide a consistent interface useful enough to replace all other provisions for the forwarding of logs from bosh jobs.
## Proposed Interface If log forwarding is enabled, some files in `/var/vcap/sys/log` (and its subdirectories, recursively), will have any line written to them forwarded as the MSG portion of an RFC5424 compliant syslog message. Which files are forwarded is governed first by file extension, and secondarily by file permissions.
`syslog-release` attempts to read any file ending in `.log`. (This allows us to avoid forwarding rotated logs, swapfiles, etc.) It will forward from such files if either of the following are true: - it is world-readable - it is readable to the `vcap` group
In particular, this means that logs will not be forwarded from files where: - user and group are root:root - user and group are vcap:root or vcap:none - user and group are vcap:vcap, but it is not group-readable
…unless they are world-readable.
We think that this interface will allow us to avoid running a log forwarder with elevated permissions, while also allowing jobs to, for instance, write DEBUG or similar logs to a file that is not group-readable, thus improving their security and reducing the load on the logging system while still making them available on the ephemeral disk for debugging purposes.
## Questions There are a couple of things around this interface we're especially interested in feedback on, in addition to the obvious "will this be a problem for you" overall question.
We may have to have a proviso that the depth of this is not unlimited. This depends somewhat on what is inexpensive to implement and maintain, and is an area we'd appreciate feedback on. Is three levels deep from `/var/vcap/sys/log` (i.e. `/var/vcap/sys/log/jobname/processname/*`) enough? Would four be?
In the old way of doing things, more control over the PRI information and other syslog fields was available to release authors. Logs forwarded from files currently all come out as PRI 14, which translates to Facility: User, Severity: Info. Additionally, the appname/tag field is set to the name of the directory containing the log file. Is this enough/good info? If we were to include the filename, too, would that be useful? Sufficient?
## Testing with the Proposed Interface We have recently implemented a feature to help release authors evaluate the proposed interface. If you set `syslog.respect_file_permissions: true`, blackbox will not be run with elevated capabilities, and you'll be able to see what is and isn't forwarded under the proposed interface. |
|
|
|
Re: Free to Migrate to BPM
Krannich, Bernd
Hi Josh,
This is great news, thanks for sharing!
So essentially, cf-deployment at some point in time will run containerized processes, even on “plain VMs”, excellent.
This coincides with the mail from Cornelius (https://lists.cloudfoundry.org/g/cf-dev/message/8120) around Containerizing CF. When discussing the Containerizing proposal, our hope was always that BPM adoption would pick up so that converting into Kubernetes artifacts could benefit from the BPM metadata.
Seems like all of this is now nicely falling in place.
Thanks, Bernd
Bernd Krannich SAP Cloud Platform SAP SE Dietmar-Hopp-Allee 16, 69190 Walldorf, Germany
Pflichtangaben/Mandatory Disclosure Statement: www.sap.com/impressum
Diese E-Mail kann Betriebs- oder Geschäftsgeheimnisse oder sonstige vertrauliche Informationen enthalten. Sollten Sie diese E-Mail irrtümlich erhalten haben, ist Ihnen eine Kenntnisnahme des Inhalts, eine Vervielfältigung oder Weitergabe der E-Mail ausdrücklich untersagt. Bitte benachrichtigen Sie uns und vernichten Sie die empfangene E-Mail. Vielen Dank.
This e-mail may contain trade secrets or privileged, undisclosed, or otherwise confidential information. If you have received this e-mail in error, you are hereby notified that any review, copying, or distribution of it is strictly prohibited. Please inform us immediately and destroy the original transmittal. Thank you for your cooperation.
From: <cf-dev@...> on behalf of Josh Collins <jcollins@...>
Hey Y'all,
You may know this already, but I wanted to send out a broad communication to make sure you're in the know and to build a shared understanding about the process for migrating to BPM going forward.
As of cf-deployment v1.40.0, BPM is colocated on all instance groups in cf-d.
It's available for every component team to adopt when validated against their component's features.
To-date, teams that have completed validation have enabled BPM by including logic within their jobs to rely on BPM based on whether the `bpm.enabled` property is set to true for their jobs. And adding an entry to `operations/experimental/enable-bpm.yml` to set that property/value to true in the deployment manifest.
The following jobs currently have entries in enable-bpm.yml:
Now that BPM is on each VM by default you can validate your components in your pipelines by enabling BPM directly within your job(s).
And when ready to publish your bpm-enablement changes in a release, please...
For each team that has a component/job that's already validated against BPM, it would be great if you could enable it directly as per above and, if appropriate, submit a PR which removes the entry from `enable-bpm.yml.`
At some point in the relatively near future, I'll delete `enable-bpm.yml,` but only after all components listed have enabled it and I've heard folk's pipelines no longer rely on it's existence.
Thank you for reading this through and thanks very much in advance for replying with any questions, feedback, or suggestions and issues related to this communique.
Cheers,
Josh
|
|
|
|
cf-deployment 3.0
Josh Collins
Hey Y'all, Cf-deployment 3.0 is around the corner. We're going to go 3.0 in 2-3 weeks. We released cf-deployment 2.0 on June 18th and included 'breaking' changes. Breaking changes in the context of cf-d are changes which would require special attention from operators for the deployment to succeed. Executing the same bosh deploy command/args run used in the previous deployment may fail depending on which ops files and features operators had deployed with in the past. Going forward, we'd like to introduce a more regular (~monthly) cadence to major point releases of cf-deployment. The goal is two-fold and in-order-of-importance:
As of today, we've got one PR that includes breaking changes and I'm putting out a call to y'all. If you've got what you'd consider to be a breaking change that warrants going out in a major point release of cf-deployment, please submit your PRs and reach out to the RelInt team as soon as you're able to so we can come up to speed and support you! Cheers, Josh |
|
|
|
Free to Migrate to BPM
Josh Collins
Hey Y'all, You may know this already, but I wanted to send out a broad communication to make sure you're in the know and to build a shared understanding about the process for migrating to BPM going forward. As of cf-deployment v1.40.0, BPM is colocated on all instance groups in cf-d. It's available for every component team to adopt when validated against their component's features. To-date, teams that have completed validation have enabled BPM by including logic within their jobs to rely on BPM based on whether the `bpm.enabled` property is set to true for their jobs. And adding an entry to `operations/experimental/enable-bpm.yml` to set that property/value to true in the deployment manifest. The following jobs currently have entries in enable-bpm.yml:
Now that BPM is on each VM by default you can validate your components in your pipelines by enabling BPM directly within your job(s). And when ready to publish your bpm-enablement changes in a release, please...
For each team that has a component/job that's already validated against BPM, it would be great if you could enable it directly as per above and, if appropriate, submit a PR which removes the entry from `enable-bpm.yml.` At some point in the relatively near future, I'll delete `enable-bpm.yml,` but only after all components listed have enabled it and I've heard folk's pipelines no longer rely on it's existence. Thank you for reading this through and thanks very much in advance for replying with any questions, feedback, or suggestions and issues related to this communique. Cheers, Josh |
|
|