Date   
Re: CF Application Runtime PMC: CF Infrastructure Project Lead Call for Nominations

Eric Malm
 

Hi, everyone,

Pivotal is nominating Preethi Varambally for the CF Infrastructure Project Lead in the Application Runtime PMC.

Preethi has been working as a product manager on the CF Infrastructure team for the past two months, and prior to that had been the Project Lead for the Container Networking team since July 2018.

Preethi previously worked as a technical product owner and business analyst at a GIS company on a customer-facing web application. Here, along with managing the product, she also worked on designing application layer data flow architecture for efficiently getting data from various source systems. Prior to this, she worked as an engineer, building data tracking and reporting tools using MVC framework and gradually moved to a product owner role managing a team of onshore and offshore engineers.

Preethi holds a Masters degree in Computer Science from University of Texas, Dallas.

Please send any other nominations directly to me or in reply to this message no later than 11:59 PM PST on Friday, February 22, 2018.

Thanks,
Eric Malm

On Wed, Feb 13, 2019 at 12:04 PM Eric Malm <emalm@...> wrote:
Hi, everyone,

Evan Farrar, the Project Lead for the CF Infrastructure team within the Application Runtime PMC, is stepping down from the project. We thank him for his years of service as the CF Infrastructure Project Lead.

The CF Infrastructure team, based in Santa Monica, now has an opening for its project lead. Project leads must be nominated by a Cloud Foundry Foundation member. Please send nominations directly to me or in reply to this message no later than 11:59 PM PST on Friday, February 22, 2018.

Also, if you have any questions about the role or the nomination process, as described in the CFF governance documents (https://www.cloudfoundry.org/governance/cff_development_operations_policy/), please let me know.

Thanks,
Eric Malm, CF Application Runtime PMC Lead

CF Application Runtime PMC: CF Infrastructure Project Lead Call for Nominations

Eric Malm
 

Hi, everyone,

Evan Farrar, the Project Lead for the CF Infrastructure team within the Application Runtime PMC, is stepping down from the project. We thank him for his years of service as the CF Infrastructure Project Lead.

The CF Infrastructure team, based in Santa Monica, now has an opening for its project lead. Project leads must be nominated by a Cloud Foundry Foundation member. Please send nominations directly to me or in reply to this message no later than 11:59 PM PST on Friday, February 22, 2018.

Also, if you have any questions about the role or the nomination process, as described in the CFF governance documents (https://www.cloudfoundry.org/governance/cff_development_operations_policy/), please let me know.

Thanks,
Eric Malm, CF Application Runtime PMC Lead

Re: [Proposal] Deprecation of the firehose endpoint

Neil MacDougall
 

Johannes,

Thank your for the notification.

We use the firehose in Stratos to stream logs for applications and the firehose itself for users/admins to view in the UI.

We also use the CF Firehose exporter (https://github.com/bosh-prometheus/firehose_exporter) as does https://github.com/bosh-prometheus/prometheus-boshrelease) to retrieve metrics from the firehose and store those in Prometheus for display within Stratos. Do you know what the plan is for these to be updated?

It is a large amount of work for us to make the changes for the removal of the firehose and I think 3 months is too aggressive a time frame for us to be able to complete those - this is not something we’ve factored into our planning, so I think we’d need a 12 month window on the firehose removal to be safe.

We have users using Stratos on a variety of Cloud Foundry distributions, which we won’t be able to guarantee will be updated in your proposed three month window, so we will most likely have to make updates to allow Stratos to work with systems that use the current firehose and those that use the newer APIs.

I’m happy to discuss this further with you directly if that would help.

Regards,

Neil


On 11 Feb 2019, at 23:54, Johannes Tuchscherer <jtuchscherer@...> wrote:

Hi there,

Following the last thread about the deprecation of the /containermetrics endpoint, the Loggregator team would like to continue on its path of deprecations. Next on the chopping block is the /firehose endpoint. We understand that this endpoint is used by a few integrations, so we will provide reference implementations and documents for migrating away from the firehose to the newer loggregator endpoints (namely, the RLP Gateway and LogCache). Depending on the feedback we receive to this email, we would like to proceed with the removal of the firehose endpoint in three months - meaning that the loggregator release being cut in three month’s time won't have support for that endpoint anymore.

You can find some example consumers and the documentation of the RLP here:

Here are some example consumers and the documentation of the Log Cache:


Our desire to remove endpoints on the trafficcontroller is based on our plan to get rid of the trafficcontroller - and then everything in loggregator that still references the V1 api and protocol. This will help us to significantly reduce code complexity, but it will also allow us to operate much more efficiently. This should result in reduced overhead for the logging pipeline and ultimately lower infrastructure costs for the logging components in a CF deployment.

Please let us know if you have any comments or concerns.

Johannes

[Proposal] Deprecation of the firehose endpoint

Johannes Tuchscherer
 

Hi there,


Following the last thread about the deprecation of the /containermetrics endpoint, the Loggregator team would like to continue on its path of deprecations. Next on the chopping block is the /firehose endpoint. We understand that this endpoint is used by a few integrations, so we will provide reference implementations and documents for migrating away from the firehose to the newer loggregator endpoints (namely, the RLP Gateway and LogCache). Depending on the feedback we receive to this email, we would like to proceed with the removal of the firehose endpoint in three months - meaning that the loggregator release being cut in three month’s time won't have support for that endpoint anymore.


You can find some example consumers and the documentation of the RLP here:

Go: https://github.com/cloudfoundry/log-stream-cli

Go Client: https://github.com/cloudfoundry/go-loggregator/blob/master/rlp_gateway_client.go

Java: https://github.com/cloudfoundry-community/jmx-consumer-release/tree/develop/src/jmxconsumer

Docs: https://github.com/cloudfoundry/loggregator/blob/master/docs/rlp_gateway.md


Here are some example consumers and the documentation of the Log Cache:

Go: https://github.com/cloudfoundry/log-cache-cli

Go Client: https://github.com/cloudfoundry/log-cache/tree/master/pkg/client

Docs: https://github.com/cloudfoundry/log-cache/blob/master/README.md



Our desire to remove endpoints on the trafficcontroller is based on our plan to get rid of the trafficcontroller - and then everything in loggregator that still references the V1 api and protocol. This will help us to significantly reduce code complexity, but it will also allow us to operate much more efficiently. This should result in reduced overhead for the logging pipeline and ultimately lower infrastructure costs for the logging components in a CF deployment.


Please let us know if you have any comments or concerns.


Johannes

PM of #loggregator

CF CAB call for February is Wednesday February 20th @ 8a PST

Michael Maximilien
 

Hi, all,
 
First thing is that the CAB call survey will close this Monday 2/11. Link again here, two minutes of time max: 


Second, reminder that the CAB call for February is Wednesday February 20th @ 8a PST (in two weeks).
 
We will have our regular PMCs highlights and two talks:
 
1. Summary of CAB 2019 survey results by me
2. Silk for CFCR prototype project by Konstantin Kiess from Stark and Wayne [1]
 
All other info in agenda here [0].
 
Zoom soon. Best,
 
dr.max
ibm ☁ 
silicon valley, ca
 

Reminder about our CoC and a change to the slack configuration

Chip Childers
 

All,

Over the weekend, there were several reports of inappropriate content being shared within the CF community's slack team in various channels. These messages have been removed and the accounts that posted them have been disabled.

We'd like to take this opportunity to remind everyone that we take breaches of our code of conduct very seriously. Our code of conduct can be found here: https://www.cloudfoundry.org/code-of-conduct/ 

While these occurrences were likely caused by people outside of our community taking advantage of the fact that our slack instance is open to everyone, it's important for community members to help ensure that we have a welcoming and inclusive environment.

If you see any inappropriate content posted to slack or have any code of conduct concerns, we ask that you reach out to the CFF via conduct@...We also ask that those that have administrative access to the slack team help with policing when / if they see something. If you do have administrative access and see inappropriate content, please help by (1) deleting the messages, (2) capturing a screenshot of the offending content and (3) notifying the CFF staff at conduct@....

Also note: due to reports of similar activity in other community slack teams having included tagging @channel and @here in postings of inappropriate content, we have disabled the use of those for non-admin users. This is to reduce the impact of any bad actor's actions.

Please reach out to me directly (cchilders@...) or the conduct@ email address if you have any questions or concerns.

Chip Childers, CTO
Cloud Foundry Foundation

NOTICE: Issue with recent BOSH release

Elliott Shanks
 

Hi there,

We wanted to let you know of an issue related to the rootfs that has been introduced in recent BOSH releases that could result in your applications crashing if you are pulling the rootfs releases from bosh.io. The permissions on certs used in the cflinuxfs2-release 1.261.0 - 1.262.0 and cflinuxfs3-release versions 0.52.0 - 0.55.0  were changed causing issues with apps using system certs. 

These releases have not been included in cf-deployment and would only affect your environment if they were pulled in directly from bosh.io

A fix has been identified and is being released shortly in 1.263.0 for cflinuxfs2-release and 0.56.0 for cflinuxfs3-release. If you are using one of the broken releases stated above, please update to one of these upcoming releases.

Please feel free to reach out with further questions or concerns. 

--
Respectfully, 

Elliott Shanks
PM Buildpacks

Re: Proposal to restructure Routing and Container Networking teams

Eric Malm
 

Thanks, Shannon. To give the PMC members sufficient notice to consider the proposed project restructuring and project lead change, let's plan to discuss both matters at the Feb 19 Application Runtime PMC meeting. Any alternative project restructuring plans or project lead nominations should be proposed on the cf-dev list no later than Fri, Feb 15.

For context, Shubha is a Product Manager at Pivotal who has been working on the Routing team since Nov 2017. Prior to coming to Pivotal, she worked with Huawei, SAP, McAfee and several startups spanning SaaS, consumer tech and the enterprise technology space.

Best,
Eric Malm, Application Runtime PMC lead


On Mon, Feb 4, 2019 at 12:36 PM Shannon Coen <scoen@...> wrote:
Members of the Routing and Container Networking teams have for some time expressed pain from broad domain surface area and challenges in context switching, overlapping objectives and challenges with cross-team collaboration and communication.

The teams collocated last week for alignment exercises which resulted in a plan to merge into one Networking team, with project lead Shubha Anjur Tupil.

We’ve consolidated Slack channels; you can find us in #networking.

We’ll add this proposal to the agenda for the next PMC call.

Best,
Shannon Coen
Product Lead, CF Networking
Pivotal


Supporting container metrics for multi-process apps

Chris Selzo
 

Hello CF folks,

Historically, Cloud Foundry apps consisted of a single process. Container metrics (memory, disk, and cpu usage) for these apps were emitted with a source_id corresponding to the app’s guid. When multi-process support was introduced, there was a need to distinguish container metrics on a per-process basis, so the source_id was set to the guid of each process. To give container metrics consumers the ability to continue to aggregate these metrics by app, changes were required and additional metadata is now included on the metrics envelopes. If you are a consumer of these metrics envelopes, these changes will be relevant to you.

Please refer to this document for details.

Thanks,
The CAPI team

Proposal to restructure Routing and Container Networking teams

Shannon Coen
 

Members of the Routing and Container Networking teams have for some time expressed pain from broad domain surface area and challenges in context switching, overlapping objectives and challenges with cross-team collaboration and communication.

The teams collocated last week for alignment exercises which resulted in a plan to merge into one Networking team, with project lead Shubha Anjur Tupil.

We’ve consolidated Slack channels; you can find us in #networking.

We’ll add this proposal to the agenda for the next PMC call.

Best,
Shannon Coen
Product Lead, CF Networking
Pivotal

[Proposal] Deprecation /containermetrics endpoint on the loggregator_trafficcontroller

Johannes Tuchscherer
 

Hi there,

the loggregator team would like to remove the /containermetrics endpoint on the loggregator_trafficcontroller (see here: https://github.com/cloudfoundry/loggregator-release/blob/master/docs/trafficcontroller.md#endpoints). This endpoint originally was introduced as an endpoint for the Cloud Controller to fetch metrics about the running container.
As of cf-deployment v7.2 (https://github.com/cloudfoundry/cf-deployment/releases/tag/v7.2.0), Cloud Controller now reaches out to log-cache to fetch the container metrics. 
Therefore, we would like to remove this endpoint to keep our API and code base lean and clean. If there are consumers of this endpoint besides the Cloud Controller, please let us know. 

Thanks,
Johannes

Re: cflinuxfs2 --> cflinuxfs3 transition

Krannich, Bernd
 

Hi Elliott,

 

Yes, sure. Feel free to schedule a meeting with me.

 

Thanks,

Bernd

 

From: <cf-dev@...> on behalf of Elliott Shanks <eshanks@...>
Reply-To: "cf-dev@..." <cf-dev@...>
Date: Monday, 4. February 2019 at 16:19
To: "cf-dev@..." <cf-dev@...>
Subject: Re: [cf-dev] cflinuxfs2 --> cflinuxfs3 transition

 

Bernd,

 

Thank you so much for using the stack-auditor plugin and providing feedback. I've got a story in our tracker for your pull request: https://www.pivotaltracker.com/story/show/163698676.

 

We have been aware of some potential performance limitations for the audit-stack command on large orgs and intend to see how we can reduce that response time. I am a fan of your export idea as well. I'd love to understand your use case a little further in order to make stack-auditor a one-and-done tool for cflinuxfs3 migration purposes. Would you be open to a 30 minute conversation at some point in the near future? 

 

Thanks again!

 

Elliott Shanks

 

On Fri, Feb 1, 2019 at 10:38 AM Krannich, Bernd <bernd.krannich@...> wrote:

Hi Elliott,

 

Nice to e-meet you and thank you very much for the pointer!

 

Finding out how to build the plugin, I created a PR adding this part to the readme: https://github.com/cloudfoundry/stack-auditor/pull/2

 

Additionally, I have tried the plugin on our biggest Cloud Foundry landscape and here is my feedback:

  • The plugin took a bit longer than 30 mins to collect the overall status of the system using `cf audit-stack` and an admin user. This is more a result of the size of our Cloud Foundry landscape. So, I guess even knowing about the plugin I’d still wish for a SQL join that I hope would give me the same overview with what I’d hope to be a much shorter runtime.
  • I think the plugin is still a nice self-service for individual org owners planning and executing a switch, so thank you very much for providing it!
  • Regarding the output, one use case I keep having for any such tool is to be able to dump things into a CSV so that I can analyze the data in Excel later on. Here, it would help if either the plugin would offer a special output format or offer an option to quote “nasty” things like orgs and spaces containing blanks.

 

I hope this helps as an additional input.

 

Thanks & Regards,

Bernd

 

From: <cf-dev@...> on behalf of Elliott Shanks <eshanks@...>
Reply-To: "cf-dev@..." <cf-dev@...>
Date: Thursday, 31. January 2019 at 16:08
To: "cf-dev@..." <cf-dev@...>
Subject: Re: [cf-dev] cflinuxfs2 --> cflinuxfs3 transition

 

Bernd,

 

My name is Elliott Shanks and I'm a new PM on the CF Core Buildpacks Team. We have recently introduced a cf-cli plugin called stack-auditor which would accomplish exactly what you are looking to do. It includes an audit command (audit-stack) to show all applications and their respective stacks as well as a command to change the stack (change-stack) of any particular app. You can download the plugin at:

 

 

Not only would I recommend this approach for your current needs, but would love to gather some feedback from you about the stack-auditor plugin after you have had a chance to use it. We are actively contributing to the plugin and would welcome any feedback in an attempt to make this a perfect tool for everyone who still needs to migrate applications over to cflinuxfs3.

 

Feel free to let me know if you have any questions while using the tool. I'd be happy to help.

 

Thanks a bunch,

Elliott Shanks

 

 

On Thu, Jan 31, 2019 at 7:04 AM Krannich, Bernd <bernd.krannich@...> wrote:

Hi Josh, all,

 

I assume many people are currently in the process of moving off of cflinuxfs2 to cflinuxfs3.

 

I was trying to figure out a good (and quick) way to see which apps haven’t done the move yet. Looking into the Cloud Controller tables, I was hoping that the apps table would contain a reference to the stack or at least the buildpack table (which has a reference to the stack), but it seems like this is not how things work.

 

What are the additional Cloud Controller tables to consider when formulating a join to answer the question: Which apps are still on cflinuxfs2?

 

Thanks in advance,

Bernd

 

From: <cf-dev@...> on behalf of Josh Collins <jcollins@...>
Reply-To: "cf-dev@..." <cf-dev@...>
Date: Saturday, 3. November 2018 at 04:40
To: "cf-dev@..." <cf-dev@...>
Subject: [cf-dev] cflinuxfs2 --> cflinuxfs3 transition

 

Hi CF Community,

 

As of cf-deployment v6.0 (Nov 5, 2018) cf-d will deploy with support for both cflinuxfs2 and cflinuxfs3 stacks (with cflinuxfs2 continuing to be the default stack).

 

In cf-deployment v7.0, to be published early in December, cflinuxfs3 will become the default stack (cflinuxfs2 will continue to be supported, but will no longer be default).

 

Once cflinuxfs3 is the default, new apps that are pushed will be staged with cflinuxfs3 unless new apps that are pushed set the stack explicitly to cflinuxfs2. The stack property is “sticky,” so all existing apps will remain associated with cflinuxfs2 unless they are manually migrated to cflinuxfs3 (or deleted and re-created).

 

Both stacks will be supported for several months to allow app developers and operators time to migrate all apps from cflinuxfs2 to cflinuxfs3.

 

Canonical will cease support for Ubuntu 14.04 LTS (Trusty) in April 2019. Therefore, at the end of March 2019, cflinuxfs2 will be removed from cf-deployment and all apps that haven’t been migrated will crash when the foundation is upgraded.

 

Thanks!

Josh Collins & Stephen Levine

 


 

--

Respectfully, 

 

Elliott Shanks

Manager, Engineering


 

--

Respectfully, 

 

Elliott Shanks

Manager, Engineering

Re: cflinuxfs2 --> cflinuxfs3 transition

Elliott Shanks
 

Bernd,

Thank you so much for using the stack-auditor plugin and providing feedback. I've got a story in our tracker for your pull request: https://www.pivotaltracker.com/story/show/163698676.

We have been aware of some potential performance limitations for the audit-stack command on large orgs and intend to see how we can reduce that response time. I am a fan of your export idea as well. I'd love to understand your use case a little further in order to make stack-auditor a one-and-done tool for cflinuxfs3 migration purposes. Would you be open to a 30 minute conversation at some point in the near future? 

Thanks again!

Elliott Shanks


On Fri, Feb 1, 2019 at 10:38 AM Krannich, Bernd <bernd.krannich@...> wrote:

Hi Elliott,

 

Nice to e-meet you and thank you very much for the pointer!

 

Finding out how to build the plugin, I created a PR adding this part to the readme: https://github.com/cloudfoundry/stack-auditor/pull/2

 

Additionally, I have tried the plugin on our biggest Cloud Foundry landscape and here is my feedback:

  • The plugin took a bit longer than 30 mins to collect the overall status of the system using `cf audit-stack` and an admin user. This is more a result of the size of our Cloud Foundry landscape. So, I guess even knowing about the plugin I’d still wish for a SQL join that I hope would give me the same overview with what I’d hope to be a much shorter runtime.
  • I think the plugin is still a nice self-service for individual org owners planning and executing a switch, so thank you very much for providing it!
  • Regarding the output, one use case I keep having for any such tool is to be able to dump things into a CSV so that I can analyze the data in Excel later on. Here, it would help if either the plugin would offer a special output format or offer an option to quote “nasty” things like orgs and spaces containing blanks.

 

I hope this helps as an additional input.

 

Thanks & Regards,

Bernd

 

From: <cf-dev@...> on behalf of Elliott Shanks <eshanks@...>
Reply-To: "cf-dev@..." <cf-dev@...>
Date: Thursday, 31. January 2019 at 16:08
To: "cf-dev@..." <cf-dev@...>
Subject: Re: [cf-dev] cflinuxfs2 --> cflinuxfs3 transition

 

Bernd,

 

My name is Elliott Shanks and I'm a new PM on the CF Core Buildpacks Team. We have recently introduced a cf-cli plugin called stack-auditor which would accomplish exactly what you are looking to do. It includes an audit command (audit-stack) to show all applications and their respective stacks as well as a command to change the stack (change-stack) of any particular app. You can download the plugin at:

 

 

Not only would I recommend this approach for your current needs, but would love to gather some feedback from you about the stack-auditor plugin after you have had a chance to use it. We are actively contributing to the plugin and would welcome any feedback in an attempt to make this a perfect tool for everyone who still needs to migrate applications over to cflinuxfs3.

 

Feel free to let me know if you have any questions while using the tool. I'd be happy to help.

 

Thanks a bunch,

Elliott Shanks

 

 

On Thu, Jan 31, 2019 at 7:04 AM Krannich, Bernd <bernd.krannich@...> wrote:

Hi Josh, all,

 

I assume many people are currently in the process of moving off of cflinuxfs2 to cflinuxfs3.

 

I was trying to figure out a good (and quick) way to see which apps haven’t done the move yet. Looking into the Cloud Controller tables, I was hoping that the apps table would contain a reference to the stack or at least the buildpack table (which has a reference to the stack), but it seems like this is not how things work.

 

What are the additional Cloud Controller tables to consider when formulating a join to answer the question: Which apps are still on cflinuxfs2?

 

Thanks in advance,

Bernd

 

From: <cf-dev@...> on behalf of Josh Collins <jcollins@...>
Reply-To: "cf-dev@..." <cf-dev@...>
Date: Saturday, 3. November 2018 at 04:40
To: "cf-dev@..." <cf-dev@...>
Subject: [cf-dev] cflinuxfs2 --> cflinuxfs3 transition

 

Hi CF Community,

 

As of cf-deployment v6.0 (Nov 5, 2018) cf-d will deploy with support for both cflinuxfs2 and cflinuxfs3 stacks (with cflinuxfs2 continuing to be the default stack).

 

In cf-deployment v7.0, to be published early in December, cflinuxfs3 will become the default stack (cflinuxfs2 will continue to be supported, but will no longer be default).

 

Once cflinuxfs3 is the default, new apps that are pushed will be staged with cflinuxfs3 unless new apps that are pushed set the stack explicitly to cflinuxfs2. The stack property is “sticky,” so all existing apps will remain associated with cflinuxfs2 unless they are manually migrated to cflinuxfs3 (or deleted and re-created).

 

Both stacks will be supported for several months to allow app developers and operators time to migrate all apps from cflinuxfs2 to cflinuxfs3.

 

Canonical will cease support for Ubuntu 14.04 LTS (Trusty) in April 2019. Therefore, at the end of March 2019, cflinuxfs2 will be removed from cf-deployment and all apps that haven’t been migrated will crash when the foundation is upgraded.

 

Thanks!

Josh Collins & Stephen Levine

 


 

--

Respectfully, 

 

Elliott Shanks

Manager, Engineering



--
Respectfully, 

Elliott Shanks
Manager, Engineering

Join our CF Services API beta testing user group!

Laurel Gray
 

Hi cf-dev,

We – the Cloud Foundry Services API team – are looking for people who would like to be part of our beta-testing group

This is an opportunity for you to actively shape our roadmap, and for us to directly hear from you about what's working well, and what isn't. 

At this point we are mainly looking for CF admins who manage multiple CF foundations. But feel free to leave your details anyways to be contacted in the future.

Please fill out this form so we can better understand your role and use of CF: https://goo.gl/forms/5dPrhdB4LdFvV9i72 
  • For every 45 - 60 min session, we will compensate your time with an Amazon gift card for £50 (or localized to your currency)
  • Any session would be remote and only requires an internet connection
  • We'll be in contact and will reach out to schedule sessions

Feel free to pass this on to others. Thank you!  

Laurel Gray, Product Manager
Annamaria Boheim, Product Designer
Alex Blease, Product Developer

Re: cflinuxfs2 --> cflinuxfs3 transition

Krannich, Bernd
 

Hi Connor, Hi Stanislav,

 

Thank you very much for your suggestions!

 

I tried out both of them now and can confirm that both are much quicker (sub-second for the SQL, few seconds for cfdot) than going via the Cloud Controller (either self-scripted or via the stack-auditor plugin).

 

And, I think I can confirm the general recommendations given in this thread:

  • Use the existing means via Cloud Controller if you are responsible for apps in one or few CF orgs
  • Use cfdot and the Cloud Controller tables to get an overview on the overall state of the system if you have the corresponding admin access

 

Thank you very much again to everybody for your suggestions!

 

Regards,

Bernd

 

From: <cf-dev@...> on behalf of Connor Braa <cbraa@...>
Reply-To: "cf-dev@..." <cf-dev@...>
Date: Friday, 1. February 2019 at 21:01
To: "cf-dev@..." <cf-dev@...>
Subject: Re: [cf-dev] cflinuxfs2 --> cflinuxfs3 transition

 

Hi Bernd!

 

If you need to use SQL to figure out which apps still use the old stack, you'll want to join apps across the buildpack_lifecycle_data table's app_guid column. but there are lots of possible complications in terms of treating that as a source of truth. 

 

For one, entries in that table where `stack` is nil indicate that the default stack should be used, but when upgrading your stack, you may have changed the default from cflinuxfs2 to 3, and in doing so made it very difficult to tell which stack is actually running for a given app (ie whether you'll need to restage). The ruby code handling that table is also pretty nontrivial, so there _might_ be some edge cases around droplet copying that mean the result of a straightforward join won't get you exactly what you need.

 

Generally, the SQL approach is going to be a bit error prone, but if your deployment is very big, I understand why it might be a necessary first step. Stanislav's approach using cfdot is going to get you much closer to the absolute truth of what is running cflinuxfs2, and a good audit will probably use both at different stages. 

 

Best,

Connor

 

On Thu, Jan 31, 2019 at 5:04 AM Krannich, Bernd <bernd.krannich@...> wrote:

Hi Josh, all,

 

I assume many people are currently in the process of moving off of cflinuxfs2 to cflinuxfs3.

 

I was trying to figure out a good (and quick) way to see which apps haven’t done the move yet. Looking into the Cloud Controller tables, I was hoping that the apps table would contain a reference to the stack or at least the buildpack table (which has a reference to the stack), but it seems like this is not how things work.

 

What are the additional Cloud Controller tables to consider when formulating a join to answer the question: Which apps are still on cflinuxfs2?

 

Thanks in advance,

Bernd

 

From: <cf-dev@...> on behalf of Josh Collins <jcollins@...>
Reply-To: "cf-dev@..." <cf-dev@...>
Date: Saturday, 3. November 2018 at 04:40
To: "cf-dev@..." <cf-dev@...>
Subject: [cf-dev] cflinuxfs2 --> cflinuxfs3 transition

 

Hi CF Community,

 

As of cf-deployment v6.0 (Nov 5, 2018) cf-d will deploy with support for both cflinuxfs2 and cflinuxfs3 stacks (with cflinuxfs2 continuing to be the default stack).

 

In cf-deployment v7.0, to be published early in December, cflinuxfs3 will become the default stack (cflinuxfs2 will continue to be supported, but will no longer be default).

 

Once cflinuxfs3 is the default, new apps that are pushed will be staged with cflinuxfs3 unless new apps that are pushed set the stack explicitly to cflinuxfs2. The stack property is “sticky,” so all existing apps will remain associated with cflinuxfs2 unless they are manually migrated to cflinuxfs3 (or deleted and re-created).

 

Both stacks will be supported for several months to allow app developers and operators time to migrate all apps from cflinuxfs2 to cflinuxfs3.

 

Canonical will cease support for Ubuntu 14.04 LTS (Trusty) in April 2019. Therefore, at the end of March 2019, cflinuxfs2 will be removed from cf-deployment and all apps that haven’t been migrated will crash when the foundation is upgraded.

 

Thanks!

Josh Collins & Stephen Levine

 

Re: cflinuxfs2 --> cflinuxfs3 transition

Connor Braa
 

Hi Bernd!

If you need to use SQL to figure out which apps still use the old stack, you'll want to join apps across the buildpack_lifecycle_data table's app_guid column. but there are lots of possible complications in terms of treating that as a source of truth. 

For one, entries in that table where `stack` is nil indicate that the default stack should be used, but when upgrading your stack, you may have changed the default from cflinuxfs2 to 3, and in doing so made it very difficult to tell which stack is actually running for a given app (ie whether you'll need to restage). The ruby code handling that table is also pretty nontrivial, so there _might_ be some edge cases around droplet copying that mean the result of a straightforward join won't get you exactly what you need.

Generally, the SQL approach is going to be a bit error prone, but if your deployment is very big, I understand why it might be a necessary first step. Stanislav's approach using cfdot is going to get you much closer to the absolute truth of what is running cflinuxfs2, and a good audit will probably use both at different stages. 

Best,
Connor


On Thu, Jan 31, 2019 at 5:04 AM Krannich, Bernd <bernd.krannich@...> wrote:

Hi Josh, all,

 

I assume many people are currently in the process of moving off of cflinuxfs2 to cflinuxfs3.

 

I was trying to figure out a good (and quick) way to see which apps haven’t done the move yet. Looking into the Cloud Controller tables, I was hoping that the apps table would contain a reference to the stack or at least the buildpack table (which has a reference to the stack), but it seems like this is not how things work.

 

What are the additional Cloud Controller tables to consider when formulating a join to answer the question: Which apps are still on cflinuxfs2?

 

Thanks in advance,

Bernd

 

From: <cf-dev@...> on behalf of Josh Collins <jcollins@...>
Reply-To: "cf-dev@..." <cf-dev@...>
Date: Saturday, 3. November 2018 at 04:40
To: "cf-dev@..." <cf-dev@...>
Subject: [cf-dev] cflinuxfs2 --> cflinuxfs3 transition

 

Hi CF Community,

 

As of cf-deployment v6.0 (Nov 5, 2018) cf-d will deploy with support for both cflinuxfs2 and cflinuxfs3 stacks (with cflinuxfs2 continuing to be the default stack).

 

In cf-deployment v7.0, to be published early in December, cflinuxfs3 will become the default stack (cflinuxfs2 will continue to be supported, but will no longer be default).

 

Once cflinuxfs3 is the default, new apps that are pushed will be staged with cflinuxfs3 unless new apps that are pushed set the stack explicitly to cflinuxfs2. The stack property is “sticky,” so all existing apps will remain associated with cflinuxfs2 unless they are manually migrated to cflinuxfs3 (or deleted and re-created).

 

Both stacks will be supported for several months to allow app developers and operators time to migrate all apps from cflinuxfs2 to cflinuxfs3.

 

Canonical will cease support for Ubuntu 14.04 LTS (Trusty) in April 2019. Therefore, at the end of March 2019, cflinuxfs2 will be removed from cf-deployment and all apps that haven’t been migrated will crash when the foundation is upgraded.

 

Thanks!

Josh Collins & Stephen Levine

 

Re: cflinuxfs2 --> cflinuxfs3 transition

Stanislav German-Evtushenko
 

Hi Bernd,

If performance is a concern how about using cfdot?
`cfdot desired-lrps | jq -r '[ .log_guid, .rootfs ] | @tsv`
will give you (instantly):
```
7bc7eef4-121d-4879-bac1-d738c54faa3a    preloaded:cflinuxfs2
2d04940f-ceb1-4884-8cf7-47fd70a0a39c    preloaded:cflinuxfs2
c3c18f43-58df-429d-bb2e-895e621c1a6d    preloaded:cflinuxfs3
5b070c85-85cd-4717-bb59-ce3fd889efad    docker:///xxx/yyy#latest
```
Where the first field is application guid.

Regards,
Stanislav

Re: cflinuxfs2 --> cflinuxfs3 transition

Krannich, Bernd
 

Hi Elliott,

 

Nice to e-meet you and thank you very much for the pointer!

 

Finding out how to build the plugin, I created a PR adding this part to the readme: https://github.com/cloudfoundry/stack-auditor/pull/2

 

Additionally, I have tried the plugin on our biggest Cloud Foundry landscape and here is my feedback:

  • The plugin took a bit longer than 30 mins to collect the overall status of the system using `cf audit-stack` and an admin user. This is more a result of the size of our Cloud Foundry landscape. So, I guess even knowing about the plugin I’d still wish for a SQL join that I hope would give me the same overview with what I’d hope to be a much shorter runtime.
  • I think the plugin is still a nice self-service for individual org owners planning and executing a switch, so thank you very much for providing it!
  • Regarding the output, one use case I keep having for any such tool is to be able to dump things into a CSV so that I can analyze the data in Excel later on. Here, it would help if either the plugin would offer a special output format or offer an option to quote “nasty” things like orgs and spaces containing blanks.

 

I hope this helps as an additional input.

 

Thanks & Regards,

Bernd

 

From: <cf-dev@...> on behalf of Elliott Shanks <eshanks@...>
Reply-To: "cf-dev@..." <cf-dev@...>
Date: Thursday, 31. January 2019 at 16:08
To: "cf-dev@..." <cf-dev@...>
Subject: Re: [cf-dev] cflinuxfs2 --> cflinuxfs3 transition

 

Bernd,

 

My name is Elliott Shanks and I'm a new PM on the CF Core Buildpacks Team. We have recently introduced a cf-cli plugin called stack-auditor which would accomplish exactly what you are looking to do. It includes an audit command (audit-stack) to show all applications and their respective stacks as well as a command to change the stack (change-stack) of any particular app. You can download the plugin at:

 

 

Not only would I recommend this approach for your current needs, but would love to gather some feedback from you about the stack-auditor plugin after you have had a chance to use it. We are actively contributing to the plugin and would welcome any feedback in an attempt to make this a perfect tool for everyone who still needs to migrate applications over to cflinuxfs3.

 

Feel free to let me know if you have any questions while using the tool. I'd be happy to help.

 

Thanks a bunch,

Elliott Shanks

 

 

On Thu, Jan 31, 2019 at 7:04 AM Krannich, Bernd <bernd.krannich@...> wrote:

Hi Josh, all,

 

I assume many people are currently in the process of moving off of cflinuxfs2 to cflinuxfs3.

 

I was trying to figure out a good (and quick) way to see which apps haven’t done the move yet. Looking into the Cloud Controller tables, I was hoping that the apps table would contain a reference to the stack or at least the buildpack table (which has a reference to the stack), but it seems like this is not how things work.

 

What are the additional Cloud Controller tables to consider when formulating a join to answer the question: Which apps are still on cflinuxfs2?

 

Thanks in advance,

Bernd

 

From: <cf-dev@...> on behalf of Josh Collins <jcollins@...>
Reply-To: "cf-dev@..." <cf-dev@...>
Date: Saturday, 3. November 2018 at 04:40
To: "cf-dev@..." <cf-dev@...>
Subject: [cf-dev] cflinuxfs2 --> cflinuxfs3 transition

 

Hi CF Community,

 

As of cf-deployment v6.0 (Nov 5, 2018) cf-d will deploy with support for both cflinuxfs2 and cflinuxfs3 stacks (with cflinuxfs2 continuing to be the default stack).

 

In cf-deployment v7.0, to be published early in December, cflinuxfs3 will become the default stack (cflinuxfs2 will continue to be supported, but will no longer be default).

 

Once cflinuxfs3 is the default, new apps that are pushed will be staged with cflinuxfs3 unless new apps that are pushed set the stack explicitly to cflinuxfs2. The stack property is “sticky,” so all existing apps will remain associated with cflinuxfs2 unless they are manually migrated to cflinuxfs3 (or deleted and re-created).

 

Both stacks will be supported for several months to allow app developers and operators time to migrate all apps from cflinuxfs2 to cflinuxfs3.

 

Canonical will cease support for Ubuntu 14.04 LTS (Trusty) in April 2019. Therefore, at the end of March 2019, cflinuxfs2 will be removed from cf-deployment and all apps that haven’t been migrated will crash when the foundation is upgraded.

 

Thanks!

Josh Collins & Stephen Levine

 


 

--

Respectfully, 

 

Elliott Shanks

Manager, Engineering

Re: cflinuxfs2 --> cflinuxfs3 transition

Elliott Shanks
 

Bernd,

My name is Elliott Shanks and I'm a new PM on the CF Core Buildpacks Team. We have recently introduced a cf-cli plugin called stack-auditor which would accomplish exactly what you are looking to do. It includes an audit command (audit-stack) to show all applications and their respective stacks as well as a command to change the stack (change-stack) of any particular app. You can download the plugin at:


Not only would I recommend this approach for your current needs, but would love to gather some feedback from you about the stack-auditor plugin after you have had a chance to use it. We are actively contributing to the plugin and would welcome any feedback in an attempt to make this a perfect tool for everyone who still needs to migrate applications over to cflinuxfs3.

Feel free to let me know if you have any questions while using the tool. I'd be happy to help.

Thanks a bunch,
Elliott Shanks


On Thu, Jan 31, 2019 at 7:04 AM Krannich, Bernd <bernd.krannich@...> wrote:

Hi Josh, all,

 

I assume many people are currently in the process of moving off of cflinuxfs2 to cflinuxfs3.

 

I was trying to figure out a good (and quick) way to see which apps haven’t done the move yet. Looking into the Cloud Controller tables, I was hoping that the apps table would contain a reference to the stack or at least the buildpack table (which has a reference to the stack), but it seems like this is not how things work.

 

What are the additional Cloud Controller tables to consider when formulating a join to answer the question: Which apps are still on cflinuxfs2?

 

Thanks in advance,

Bernd

 

From: <cf-dev@...> on behalf of Josh Collins <jcollins@...>
Reply-To: "cf-dev@..." <cf-dev@...>
Date: Saturday, 3. November 2018 at 04:40
To: "cf-dev@..." <cf-dev@...>
Subject: [cf-dev] cflinuxfs2 --> cflinuxfs3 transition

 

Hi CF Community,

 

As of cf-deployment v6.0 (Nov 5, 2018) cf-d will deploy with support for both cflinuxfs2 and cflinuxfs3 stacks (with cflinuxfs2 continuing to be the default stack).

 

In cf-deployment v7.0, to be published early in December, cflinuxfs3 will become the default stack (cflinuxfs2 will continue to be supported, but will no longer be default).

 

Once cflinuxfs3 is the default, new apps that are pushed will be staged with cflinuxfs3 unless new apps that are pushed set the stack explicitly to cflinuxfs2. The stack property is “sticky,” so all existing apps will remain associated with cflinuxfs2 unless they are manually migrated to cflinuxfs3 (or deleted and re-created).

 

Both stacks will be supported for several months to allow app developers and operators time to migrate all apps from cflinuxfs2 to cflinuxfs3.

 

Canonical will cease support for Ubuntu 14.04 LTS (Trusty) in April 2019. Therefore, at the end of March 2019, cflinuxfs2 will be removed from cf-deployment and all apps that haven’t been migrated will crash when the foundation is upgraded.

 

Thanks!

Josh Collins & Stephen Levine

 



--
Respectfully, 

Elliott Shanks
Manager, Engineering

Re: cflinuxfs2 --> cflinuxfs3 transition

Krannich, Bernd
 

Hi Stanislav,

 

Thank you very much for the pointer. I forgot to mention my reason to look into the Cloud Controller tables directly: It might take me quite some time to pull the data via the Cloud Controller API. So if possible, I’d like to take a more direct route to collect the data.

 

Thanks,

Bernd

 

From: <cf-dev@...> on behalf of Stanislav German-Evtushenko <s.germanevtushenko@...>
Reply-To: "cf-dev@..." <cf-dev@...>
Date: Thursday, 31. January 2019 at 15:32
To: "cf-dev@..." <cf-dev@...>
Subject: Re: [cf-dev] cflinuxfs2 --> cflinuxfs3 transition

 

Hi Bernd,

You may want to try this https://github.com/rakutentech/cf-tools/blob/master/cf-applist.sh

Best regards,
Stanislav