Date   

FYI: Welcome “CF Backup & Restore” project to CF-Extensions

Michael Maximilien
 

Hi, all,

The CF Extensions PMC is glad to announce the acceptance of the “CF Backup & Restore” plugin as the newest extension to Cloud Foundry.

With CF Backup & Restore, users of Cloud Foundry can easily backup and restore their orgs, spaces, applications, and service bindings. The restoration can be into the same foundry or a different one.

The CF Backup & Restore team has transitioned the project to the cf-incubator organization:

https://github.com/cloudfoundry-incubator/cf-plugin-backup

We invite you to engage with the team with questions in their Slack channel: #cf-backup-restore on the CloudFoundry slack.

In behalf of the project leaders:

Vlad Iovanov (SUSE)
Enrique Encalada (IBM)

--
dr.max Sent from my iPhone http://maximilien.org


Re: Announcing Cloud Foundry CLI v7 Release Plan

Stefan Mayr
 

Hi Josh

Am 27.04.2020 um 18:13 schrieb Josh Collins:
Dear Cloud Foundry Community,


We are nearing the completion of the v7 CF CLI and we’re excited to
announce that we’ll be cutting our GA release soon!
This is fantastic news. Our developers are waiting for some of the
experimental v3-commands to be replaced with stable CLI v7 commands.

The exact target date for the launch hasn’t been finalized but because
transitioning from v6 to v7 will require coordination and planning we’re
sending an initial heads up. We'll send follow ups to this communication
once the launch date and associated details become concrete.

 

In case you haven’t been following the development of the v7 CLI
closely, here’s three of many new capabilities that will become
available when v7 GAs:

*

Rolling Deploys
<https://docs.cloudfoundry.org/devguide/deploy-apps/rolling-deploy.html>using
the "--strategy" flag for "cf push" and other commands

*

Metadata
<https://docs.cloudfoundry.org/adminguide/metadata.html>apply labels
and annotations to apps, spaces, organizations, and other resources

*

Sidecar Processes
<https://docs.cloudfoundry.org/devguide/sidecars.html>for
applications using application manifests

...
How about manifest files? Do we have to expect any manifest syntax or
naming changes? Will some of the new CLI features also be available in
the manifest?

Thanks,

Stefan


Re: Is anyone working on a CF CRD & Controller?

Zach Robinson
 

Hey Daniel,

Wanted to follow up. There's now a doc to start building out our understanding of CRDs here https://docs.google.com/document/d/1mIMH4uEtIJkZjVa5mKWJoIAIeVyM4gVqO-Cx5JtYgtE/edit?usp=sharing

Shared with the mailing list under topic "Exploring CRDs in CF".  Would love to get your thoughts and feedback.

-Zach

On Thu, Apr 16, 2020 at 4:09 PM Zach Robinson <zrobinson@...> wrote:
Zach, am I correct in thinking that the current approach is the have clients interact with CloudController, have CloudController persist to CCDB as per usual, and then CloudController post stuff to CRDs in the Kube API?

Yes, this is the current approach for the CRDs that Cloud Controller interacts with now. There are also reconciliation loops that we run for these resources analogous to how we've always reconciled CCDB state with Diego.

I think this method has developed organically from our goal of preserving in-place workflows via having the same API surface, allowing cli to function without changes, while trying to consume existing functionality in k8s. I think the longer term goal here is to review that usage and be more intentional about how we expect CRDs to fit into the system as a whole.

-Zach

On Thu, Apr 16, 2020 at 4:10 AM Daniel Jones <daniel.jones@...> wrote:
Thanks for that folks - especially for such a long and detailed response, Zach. Much appreciated.

It's great that folks are already thinking about this.

Zach, am I correct in thinking that the current approach is the have clients interact with CloudController, have CloudController persist to CCDB as per usual, and then CloudController post stuff to CRDs in the Kube API?

I'd kinda imagined things going the other way around, introducing the CRDs as a new user interface, and have the controllers do the diff and then post/put to CloudController. The intention here would be saving Kubes-native folks (and CI servers!) from having to deal with the imperative interface. I was working on the assumption that all CF concepts would be exposed as CRDs.

From a CAPI perspective, I've been thinking more about representing apps and push as CRDs, and I believe the Eirini team has some thoughts in this area as well.

I'd always dodged this area in cf-converger as it was the most complicated, and the biggest break from the current CF experience :)

For example when we look at orgs/spaces - we have to wonder if those are even resources we want to represent as CRDs, when k8s already has a namespace construct.

I can imagine that the logical concept of orgs may one day map onto distinct Kubernetes namespaces, but I hadn't imagined that orgs would in any way disappear, or not be configurable via the Kubernetes approach.

It'd be great to hear what others think on all of this.

Thanks again for sharing, Keshav and Zach.

Regards,
Daniel 'Deejay' Jones - CEO
+44 (0)79 8000 9153
EngineerBetter Ltd - More than cloud platform specialists


On Wed, 15 Apr 2020 at 23:45, Zach Robinson <zrobinson@...> wrote:
Hey Daniel,

Thanks for starting a convo. The timing couldn't be better. I can share some of the thinking going on in both CAPI and VMware land. 

But first and foremost I want to address the question: "Is there any point in the community working on such a solution, or are one of the bigger companies secretly working on something that would supersede any community effort?" - yes, yes, a thousand times yes to community collaboration!

The topic of CRDs is kind of huge. It's really great to hear your focus is on things like orgs/spaces. From a CAPI perspective, I've been thinking more about representing apps and push as CRDs, and I believe the Eirini team has some thoughts in this area as well. There's a lot of ground to cover clearly.

In terms of things that are in flight now.
  • Keshav kindly shared that networking is looking to provide Route CRDs.
  • Networking is also exploring providing Security Groups as CRDs.
  • CAPI is integrating kpack into CF to provide buildpack staging. This is done by generating kpack CRDs.
Currently these CRDs are being created and owned by the Cloud Controller, which enables us to present backward compatible APIs to our CLI and UI like stratos, while moving the implementation down into controllers fronted by CRDs.  There's an obvious potential to interact with k8s directly in the future, but the current use case is to be used by CC. Except for kpack which is an excellent stand alone tool already :).  

These efforts are great and are allowing us to begin a path towards using existing k8s tooling and projects to provide Cloud Foundry outcomes, but obviously there's a lot further to go.

From VMware's side, over the last couple weeks, there has been an effort to explore what it might look like to use CRDs entirely as a source of truth for *all* CF data, in place of a SQL database, as a learning exercise, not necessarily a desired outcome. The primary goal of that effort was to start the convo that we're having now, and to be able to contribute in a meaningful way.  Note there are no VMware-based plans to dump a fully fledged proposal. We need to start from a premise as a community of understanding how/why/and which resources are valuable as CRDs.

For example when we look at orgs/spaces - we have to wonder if those are even resources we want to represent as CRDs, when k8s already has a namespace construct.

I think we'll need to start some dedicated working document to collaborate in. I had planned on facilitating that after some of the investigation that VMware has wraps up, but I don't think that's a reason to hold off if you or somebody else in the community has thoughts they want to start getting down. CAPI and other teams can contribute to an existing doc if that makes sense. Any other thoughts you'd prefer for next steps?

Looking forward to a continued conversation.

-Zach


On Wed, Apr 15, 2020 at 9:56 AM Keshav Sharma <ksharma@...> wrote:
Hi Daniel, 
The CF-K8s Networking team is currently working on a solution using Route CRDto introduce a “Route” custom resource and have Cloud Controller make/update these directly as part of the `cf map-route` and `cf unmap-route` workflows. Happy to discuss this further in our Cloud-Foundry Slack- #networking group. Regards, Keshav Sharma Product Manager | CF-K8s-Networking VMware

On Wed, Apr 15, 2020 at 3:01 AM Daniel Jones <daniel.jones@...> wrote:
Hi all,

In the spirit of community, I'm going to ask this question outright and in the open - is anyone (*looks at VMware MAPBU*) working on a CRD and accompanying controller for CF on Kubernetes?

Anyone with a non-trivial amount of Cloud Foundry experience knows that whilst the imperative interface of the CF CLI is great for exploratory work, it's far from ideal for production environments. I make a point of telling folks in CF training courses that no human should be using the CLI in production, other than for debugging.

A declarative interface for CF has always been desirable, and its omission is going to become even more stark once running CF on Kubernetes is the norm. We've had cf-mgmt which is close, but involves more steps than is ideal.

It'd be great to have one or more CRDs that represent the state of a Cloud Foundry, and accompanying controllers that converge upon that state. When I talk about state, I mean things like orgs, spaces, service broker registrations, service instances, roles and so on.

Four years ago I started work on a project called cf-converger to diff and converge a Cloud Foundry instance on a state declared in YAML. I got distracted with running a business and doing billable work, so whilst the concept was proven, it never really got to a state of usefulness. Plus, I seemed to spend half my coding time just writing excessively-verbose chained method calls on the official Java CF client :D 

I see such a solution as an inevitability in the Kubernetes-based future. Whilst I'm personally tempted to resurrect the project, it makes very little sense for a company like EngineerBetter to divert resources from billable work to a project that is needed, but can't be monetised.

Are any of the big players working on something similar? Is it something we should all be thinking and talking about? Is there any point in the community working on such a solution, or are one of the bigger companies secretly working on something that would supersede any community effort?

Regards,
Daniel 'Deejay' Jones - CEO
+44 (0)79 8000 9153
EngineerBetter Ltd - More than cloud platform specialists


Exploring CRDs in CF

Zach Robinson
 

Hey all,

As we move towards a kubernetes-based Cloud Foundry, the inclusion of CRDs into CF has been a recurring topic. At this point we already see CRDs being incorporated into CF in a number of ways, as well as questions about them such as a thread on this list titled "Is anyone working on a CF CRD & Controller?".  

We would like to use this space to start gathering an understanding of how folks are using CRDs, the reasons folks are using CRDs, and what CRDs folks would like to see in the future. 

This will help us build a better understanding as a community towards the outcomes of
  • Include CRDs in CF with well-defined architectural patterns
  • Understand how CRDs should be exposed as a user interface

This document has been created to help house the discussion asynchronously. It has been built with the understanding of cf contributors from a couple of teams, but we need everybody's help to flesh out the full picture.

Looking forward to discussion


Re: Cloud Foundry content migration document

Vlad Iovanov
 

Hi @Krannich, Bernd

 

We’ve just moved a new project in the Foundation, Extensions PMC: https://github.com/cloudfoundry-incubator/cf-plugin-backup

Its purpose is to do what you describe, migrate foundries.

 

Right now, it’s a CLI plugin that works exclusively with the API (by design).

It has a few missing features, but with help from our CAPI friends, I think we could close the gap.

 

The team is currently formed of SUSE and IBM – more help would be greatly appreciated.

 

Cheers,

Vlad

 

From: Krannich, Bernd
Sent: Tuesday, April 28, 2020 2:56 PM
To: cf-dev@...
Subject: [cf-dev] Cloud Foundry content migration document

 

Hello all,

 

As a follow-up of a discussion in our Cloud Foundry for Kubernetes Special Interest Group meeting [1], I took an action to write up my thoughts around the topic of Cloud Foundry content migration (which is not only relevant in the context of CF@K8s).

 

Here is my initial write-up:

https://docs.google.com/document/d/1R54vlw16kOxiSQpcMF0fJUuE-cBsqxYAU-8z87CIHyc/edit?usp=sharing

 

More than happy to receive feedback, shape the topic in the CF community, and ideally in a follow-up step, gather a group of companies/individuals that take the topic further in Cloud Foundry open source.

 

Thanks,

Bernd

 

[1] https://www.cloudfoundry.org/community-calendar/ à SIG Meeting: Cloud Foundry for Kubernetes

 

Bernd Krannich

SAP Cloud Platform

SAP SE

Dietmar-Hopp-Allee 16, 69190 Walldorf, Germany

 

bernd.krannich@...

 

Pflichtangaben/Mandatory Disclosure Statement: www.sap.com/impressum

 

Diese E-Mail kann Betriebs- oder Geschäftsgeheimnisse oder sonstige vertrauliche Informationen enthalten. Sollten Sie diese E-Mail irrtümlich erhalten haben, ist Ihnen eine Kenntnisnahme des Inhalts, eine Vervielfältigung oder Weitergabe der E-Mail ausdrücklich untersagt. Bitte benachrichtigen Sie uns und vernichten Sie die empfangene E-Mail. Vielen Dank.

 

This e-mail may contain trade secrets or privileged, undisclosed, or otherwise confidential information. If you have received this e-mail in error, you are hereby notified that any review, copying, or distribution of it is strictly prohibited. Please inform us immediately and destroy the original transmittal. Thank you for your cooperation.

 


Cloud Foundry content migration document

Krannich, Bernd
 

Hello all,

 

As a follow-up of a discussion in our Cloud Foundry for Kubernetes Special Interest Group meeting [1], I took an action to write up my thoughts around the topic of Cloud Foundry content migration (which is not only relevant in the context of CF@K8s).

 

Here is my initial write-up:

https://docs.google.com/document/d/1R54vlw16kOxiSQpcMF0fJUuE-cBsqxYAU-8z87CIHyc/edit?usp=sharing

 

More than happy to receive feedback, shape the topic in the CF community, and ideally in a follow-up step, gather a group of companies/individuals that take the topic further in Cloud Foundry open source.

 

Thanks,

Bernd

 

[1] https://www.cloudfoundry.org/community-calendar/ SIG Meeting: Cloud Foundry for Kubernetes

 

Bernd Krannich

SAP Cloud Platform

SAP SE

Dietmar-Hopp-Allee 16, 69190 Walldorf, Germany

 

bernd.krannich@...

 

Pflichtangaben/Mandatory Disclosure Statement: www.sap.com/impressum

 

Diese E-Mail kann Betriebs- oder Geschäftsgeheimnisse oder sonstige vertrauliche Informationen enthalten. Sollten Sie diese E-Mail irrtümlich erhalten haben, ist Ihnen eine Kenntnisnahme des Inhalts, eine Vervielfältigung oder Weitergabe der E-Mail ausdrücklich untersagt. Bitte benachrichtigen Sie uns und vernichten Sie die empfangene E-Mail. Vielen Dank.

 

This e-mail may contain trade secrets or privileged, undisclosed, or otherwise confidential information. If you have received this e-mail in error, you are hereby notified that any review, copying, or distribution of it is strictly prohibited. Please inform us immediately and destroy the original transmittal. Thank you for your cooperation.


Announcing Cloud Foundry CLI v7 Release Plan

Josh Collins
 

Dear Cloud Foundry Community,


We are nearing the completion of the v7 CF CLI and we’re excited to announce that we’ll be cutting our GA release soon!

 

The exact target date for the launch hasn’t been finalized but because transitioning from v6 to v7 will require coordination and planning we’re sending an initial heads up. We'll send follow ups to this communication once the launch date and associated details become concrete.

 

In case you haven’t been following the development of the v7 CLI closely, here’s three of many new capabilities that will become available when v7 GAs:

  • Rolling Deploys using the "--strategy" flag for "cf push" and other commands

  • Metadata apply labels and annotations to apps, spaces, organizations, and other resources

  • Sidecar Processes for applications using application manifests


Current Plan:

  1. We’ll continue working through and finalizing the finer-grained details regarding the transition from v6 to v7 (these details will be sent as a follow up to this announcement in advance of the v7 GA)

  2. In the coming weeks, we’ll cut a feature complete v7 CLI release candidate (7.0.0-rc.1)

  3. We’ll coordinate with Release Integration (RelInt) team to integrate the RC v7 CLI into their CF-Deployment CI pipelines and iterate on fixes as necessary ‘till they run green 

  4. Once we’ve finalized and published the v6 to v7 transition plan & the RC v7 CLI has been passing successfully through RelInt’s CF-Deployment CI Pipelines, we’ll GA the v7 CLI

  5. Once the v7 CLI is GA, the active development and release of new features and bug fixes will take place on the v7 CLI, and as per Phase 2 of the v6 CLI deprecation plan, the v6 CLI will no longer be under active development and only be updated to fix severe bugs and or CVEs


While our goal is for the v6 to v7 CLI transition to be fairly easy, it is a major version bump containing breaking changes and a certain amount of impact is unavoidable. 


The previously published “Upgrading to cf cli v7” describes many of the breaking changes that will be included at launch. Although the doc isn’t final, it’s the best resource currently for those needing to understand the change required in day to day manual and/or automated workflows.

 

At the time of this communication, although the list of breaking changes included in the document linked above is nearly complete, it should not be considered comprehensive.

Once we complete our final review/audit we’ll publish an exhaustive list of the changes from v6 to v7 and we’ll send an announcement to the community.   


Automated scripts may break when v7 of the CLI GAs. To minimize disruption and allow for teams to migrate when ready, we will support pinning to the 6.x major version. Although this isn’t possible today, the CLI team is actively working on this capability and will send a follow-up communication with instructions once that work is complete.


Lastly, with the GA release of v7, we plan to update the CF-CLI Minimum Supported Version policy. A separate communication describing the intended changes will be sent to this distribution list soon. 


For more information about the v7 CLI, please visit Cloud Foundry docs, message us on slack, or visit our github.


Thanks,

Cloud Foundry CLI Contributors


Cloud Foundry Summit Goes Cloud Native

Chip Childers <cchilders@...>
 

It’s official, folks: Cloud Foundry Summit is going virtual. We’ve moved the Austin, Texas, event online to keep you — our fantastic community — safe and healthy during this unforeseen time. What was initially a one-day Summit will now be two half-days devoted to our developer and contributor communities so you can comfortably engage with presentations without spending the entire day in front of your computer.

We know this is a sea change for a community so fiercely devoted to quality in-person discussion. I look forward to our Summits every year, and I am also disappointed that I won’t get to see you all in person. But we remain committed to providing you with the same opportunities for collaboration and education that we’ve offered at all of our in-person Summits, even if the setting is your home office (or kitchen table).

We are working with our inimitable events team to design a virtual Summit that caters not only to the specific interests of our community, but to the abnormal circumstances in which we find ourselves. Cloud Foundry Virtual Summit will take place on Wednesday, June 24th and Thursday, June 25th for several hours each day, with sessions abbreviated to keep you engaged and networking opportunities meant to mimic the beloved “hallway track” at Summit. You’ll find special events like the Diversity Luncheon, Hands-on labs and Community Awards scheduled as usual — just in a different format than you’ve previously experienced. 

Contributors can register here for free using this code: CFNA20CON 

You’ll also notice that we’ve reopened the CFP now that Summit is going virtual, as the format will be a bit different. Check out the CFP submission process here and note that the new deadline is Friday, May 1st. You can submit a talk to the Developer Experience track, the Contributor track or the Diversity track. Feel free to reach out for CFP guidance by tagging @cfp-help in #summit on Cloud Foundry slack.

I imagine this news may not come as a surprise given how many events have moved online for the next few months, but I know it’s still a disappointment for our tight-knit community. I want to make sure that you get to shape this event to be exactly what you need it to be as we venture together into unknown territory. What features do you want to see in a virtual event? What pieces from Cloud Foundry Summit are a must-have? What tools have you used at online events that you loved, and what types of sessions most engaged you? This is feedback we need and want to make sure we make this new virtual summit the best possible experience for you — our community. Please email me or send me a DM on Cloud Foundry slack @chipchilders to share your learnings from other events and how we can serve you.

In the meantime, I hope you are staying safe at home and doing what you need to feel mentally, emotionally and physically healthy during this time. I appreciate your flexibility and understanding around this decision, and I look forward to seeing you from a safe distance at Summit!


Chip Childers
Executive Director
Cloud Foundry Foundation


Stratos 3.1.0

Richard Cox
 

Hi Everybody!

It gives me great pleasure to announce the release of Stratos 3.1.0.

Highlights include...

- CF user roles can be added and removed via username
- The way Stratos handles large lists has received may improvements
- Jetstream logging of API requests can now be disabled
- Deploy CF Application now supports all v3 manifest features such as multiple buildpacks
- Administrators can now edit the details of existing endpoints
- The Stratos user profile icon now supports Gravatar

Full release notes are available here - https://github.com/cloudfoundry/stratos/releases/tag/3.1.0.

We welcome your feedback, comments and bug reports. Please feel free to raise them in github (https://github.com/cloudfoundry/stratos) or reach out directly to us in slack (#stratos)

Regards,

Richard Cox
on behalf of the Stratos team


Routing Release 0.200.0

Clay Kauzlaric <ckauzlaric@...>
 

Hi cf-dev!

A new Routing Release has been cut.

Routing Release Highlights

Regards,
CloudFoundry Networking Program


CF-Networking and Silk Release 2.29.0

Clay Kauzlaric <ckauzlaric@...>
 

Hi cf-dev!

New CF-Networking and Silk Releases have been cut.

CF-Networking Release Highlights
Silk Release Highlights

Regards,
CloudFoundry Networking Program


Re: REQUEST for REVIEW - Scope for CF-Deployment v13.0

Marco Voelz
 

Thanks for taking the time to getting this resolved, Sai! Highly appreciated!

 

Warm regards

Marco

 

From: <cf-dev@...> on behalf of Saikiran Yerram <syerram@...>
Reply to: "cf-dev@..." <cf-dev@...>
Date: Monday, 20. April 2020 at 16:32
To: "cf-dev@..." <cf-dev@...>
Subject: Re: [cf-dev] REQUEST for REVIEW - Scope for CF-Deployment v13.0

 

Hello everyone,

We are moving the cf-deployment v13.0 release date from 04/27/2020 to 05/04/2020. We are running behind schedule and there are still open questions on the scope.

We are working on resolving questions in the release note, so please check-in for updates later this week.

https://docs.google.com/document/d/1s07x8ONnaCqY8pY5dXK94wwjVWT1jFE0RppbdvL4kyg/edit?usp=sharing

_._,_._,_


Re: REQUEST for REVIEW - Scope for CF-Deployment v13.0

Saikiran Yerram
 

Hello everyone,

We are moving the cf-deployment v13.0 release date from 04/27/2020 to 05/04/2020. We are running behind schedule and there are still open questions on the scope.

We are working on resolving questions in the release note, so please check-in for updates later this week.

https://docs.google.com/document/d/1s07x8ONnaCqY8pY5dXK94wwjVWT1jFE0RppbdvL4kyg/edit?usp=sharing


KubeCF 2.0.0 is out!

Jaime Gomes
 

After some CI difficulties…KubeCF 2.0 is finally here! https://github.com/cloudfoundry-incubator/kubecf/releases/tag/v2.0.0
I welcome everyone to check the release notes to know what is coming on this major version.
Big applause to the team for code contributions and the community for the support and discussions on Slack and Github issues. See u all next week ;)

warning: this major version introduces some changes required to consume Eirini as a native component (scheduled for next week's release) by forcing the usage of kubecf name throughout the deployment resources (pod, namespace, ...) and so, upgrading from 1.x to 2.x will not be possible!


Re: Is anyone working on a CF CRD & Controller?

Zach Robinson
 

Zach, am I correct in thinking that the current approach is the have clients interact with CloudController, have CloudController persist to CCDB as per usual, and then CloudController post stuff to CRDs in the Kube API?

Yes, this is the current approach for the CRDs that Cloud Controller interacts with now. There are also reconciliation loops that we run for these resources analogous to how we've always reconciled CCDB state with Diego.

I think this method has developed organically from our goal of preserving in-place workflows via having the same API surface, allowing cli to function without changes, while trying to consume existing functionality in k8s. I think the longer term goal here is to review that usage and be more intentional about how we expect CRDs to fit into the system as a whole.

-Zach

On Thu, Apr 16, 2020 at 4:10 AM Daniel Jones <daniel.jones@...> wrote:
Thanks for that folks - especially for such a long and detailed response, Zach. Much appreciated.

It's great that folks are already thinking about this.

Zach, am I correct in thinking that the current approach is the have clients interact with CloudController, have CloudController persist to CCDB as per usual, and then CloudController post stuff to CRDs in the Kube API?

I'd kinda imagined things going the other way around, introducing the CRDs as a new user interface, and have the controllers do the diff and then post/put to CloudController. The intention here would be saving Kubes-native folks (and CI servers!) from having to deal with the imperative interface. I was working on the assumption that all CF concepts would be exposed as CRDs.

From a CAPI perspective, I've been thinking more about representing apps and push as CRDs, and I believe the Eirini team has some thoughts in this area as well.

I'd always dodged this area in cf-converger as it was the most complicated, and the biggest break from the current CF experience :)

For example when we look at orgs/spaces - we have to wonder if those are even resources we want to represent as CRDs, when k8s already has a namespace construct.

I can imagine that the logical concept of orgs may one day map onto distinct Kubernetes namespaces, but I hadn't imagined that orgs would in any way disappear, or not be configurable via the Kubernetes approach.

It'd be great to hear what others think on all of this.

Thanks again for sharing, Keshav and Zach.

Regards,
Daniel 'Deejay' Jones - CEO
+44 (0)79 8000 9153
EngineerBetter Ltd - More than cloud platform specialists


On Wed, 15 Apr 2020 at 23:45, Zach Robinson <zrobinson@...> wrote:
Hey Daniel,

Thanks for starting a convo. The timing couldn't be better. I can share some of the thinking going on in both CAPI and VMware land. 

But first and foremost I want to address the question: "Is there any point in the community working on such a solution, or are one of the bigger companies secretly working on something that would supersede any community effort?" - yes, yes, a thousand times yes to community collaboration!

The topic of CRDs is kind of huge. It's really great to hear your focus is on things like orgs/spaces. From a CAPI perspective, I've been thinking more about representing apps and push as CRDs, and I believe the Eirini team has some thoughts in this area as well. There's a lot of ground to cover clearly.

In terms of things that are in flight now.
  • Keshav kindly shared that networking is looking to provide Route CRDs.
  • Networking is also exploring providing Security Groups as CRDs.
  • CAPI is integrating kpack into CF to provide buildpack staging. This is done by generating kpack CRDs.
Currently these CRDs are being created and owned by the Cloud Controller, which enables us to present backward compatible APIs to our CLI and UI like stratos, while moving the implementation down into controllers fronted by CRDs.  There's an obvious potential to interact with k8s directly in the future, but the current use case is to be used by CC. Except for kpack which is an excellent stand alone tool already :).  

These efforts are great and are allowing us to begin a path towards using existing k8s tooling and projects to provide Cloud Foundry outcomes, but obviously there's a lot further to go.

From VMware's side, over the last couple weeks, there has been an effort to explore what it might look like to use CRDs entirely as a source of truth for *all* CF data, in place of a SQL database, as a learning exercise, not necessarily a desired outcome. The primary goal of that effort was to start the convo that we're having now, and to be able to contribute in a meaningful way.  Note there are no VMware-based plans to dump a fully fledged proposal. We need to start from a premise as a community of understanding how/why/and which resources are valuable as CRDs.

For example when we look at orgs/spaces - we have to wonder if those are even resources we want to represent as CRDs, when k8s already has a namespace construct.

I think we'll need to start some dedicated working document to collaborate in. I had planned on facilitating that after some of the investigation that VMware has wraps up, but I don't think that's a reason to hold off if you or somebody else in the community has thoughts they want to start getting down. CAPI and other teams can contribute to an existing doc if that makes sense. Any other thoughts you'd prefer for next steps?

Looking forward to a continued conversation.

-Zach


On Wed, Apr 15, 2020 at 9:56 AM Keshav Sharma <ksharma@...> wrote:
Hi Daniel, 
The CF-K8s Networking team is currently working on a solution using Route CRDto introduce a “Route” custom resource and have Cloud Controller make/update these directly as part of the `cf map-route` and `cf unmap-route` workflows. Happy to discuss this further in our Cloud-Foundry Slack- #networking group. Regards, Keshav Sharma Product Manager | CF-K8s-Networking VMware

On Wed, Apr 15, 2020 at 3:01 AM Daniel Jones <daniel.jones@...> wrote:
Hi all,

In the spirit of community, I'm going to ask this question outright and in the open - is anyone (*looks at VMware MAPBU*) working on a CRD and accompanying controller for CF on Kubernetes?

Anyone with a non-trivial amount of Cloud Foundry experience knows that whilst the imperative interface of the CF CLI is great for exploratory work, it's far from ideal for production environments. I make a point of telling folks in CF training courses that no human should be using the CLI in production, other than for debugging.

A declarative interface for CF has always been desirable, and its omission is going to become even more stark once running CF on Kubernetes is the norm. We've had cf-mgmt which is close, but involves more steps than is ideal.

It'd be great to have one or more CRDs that represent the state of a Cloud Foundry, and accompanying controllers that converge upon that state. When I talk about state, I mean things like orgs, spaces, service broker registrations, service instances, roles and so on.

Four years ago I started work on a project called cf-converger to diff and converge a Cloud Foundry instance on a state declared in YAML. I got distracted with running a business and doing billable work, so whilst the concept was proven, it never really got to a state of usefulness. Plus, I seemed to spend half my coding time just writing excessively-verbose chained method calls on the official Java CF client :D 

I see such a solution as an inevitability in the Kubernetes-based future. Whilst I'm personally tempted to resurrect the project, it makes very little sense for a company like EngineerBetter to divert resources from billable work to a project that is needed, but can't be monetised.

Are any of the big players working on something similar? Is it something we should all be thinking and talking about? Is there any point in the community working on such a solution, or are one of the bigger companies secretly working on something that would supersede any community effort?

Regards,
Daniel 'Deejay' Jones - CEO
+44 (0)79 8000 9153
EngineerBetter Ltd - More than cloud platform specialists


Re: Is anyone working on a CF CRD & Controller?

Daniel Jones
 

Thanks for that folks - especially for such a long and detailed response, Zach. Much appreciated.

It's great that folks are already thinking about this.

Zach, am I correct in thinking that the current approach is the have clients interact with CloudController, have CloudController persist to CCDB as per usual, and then CloudController post stuff to CRDs in the Kube API?

I'd kinda imagined things going the other way around, introducing the CRDs as a new user interface, and have the controllers do the diff and then post/put to CloudController. The intention here would be saving Kubes-native folks (and CI servers!) from having to deal with the imperative interface. I was working on the assumption that all CF concepts would be exposed as CRDs.

From a CAPI perspective, I've been thinking more about representing apps and push as CRDs, and I believe the Eirini team has some thoughts in this area as well.

I'd always dodged this area in cf-converger as it was the most complicated, and the biggest break from the current CF experience :)

For example when we look at orgs/spaces - we have to wonder if those are even resources we want to represent as CRDs, when k8s already has a namespace construct.

I can imagine that the logical concept of orgs may one day map onto distinct Kubernetes namespaces, but I hadn't imagined that orgs would in any way disappear, or not be configurable via the Kubernetes approach.

It'd be great to hear what others think on all of this.

Thanks again for sharing, Keshav and Zach.

Regards,
Daniel 'Deejay' Jones - CEO
+44 (0)79 8000 9153
EngineerBetter Ltd - More than cloud platform specialists


On Wed, 15 Apr 2020 at 23:45, Zach Robinson <zrobinson@...> wrote:
Hey Daniel,

Thanks for starting a convo. The timing couldn't be better. I can share some of the thinking going on in both CAPI and VMware land. 

But first and foremost I want to address the question: "Is there any point in the community working on such a solution, or are one of the bigger companies secretly working on something that would supersede any community effort?" - yes, yes, a thousand times yes to community collaboration!

The topic of CRDs is kind of huge. It's really great to hear your focus is on things like orgs/spaces. From a CAPI perspective, I've been thinking more about representing apps and push as CRDs, and I believe the Eirini team has some thoughts in this area as well. There's a lot of ground to cover clearly.

In terms of things that are in flight now.
  • Keshav kindly shared that networking is looking to provide Route CRDs.
  • Networking is also exploring providing Security Groups as CRDs.
  • CAPI is integrating kpack into CF to provide buildpack staging. This is done by generating kpack CRDs.
Currently these CRDs are being created and owned by the Cloud Controller, which enables us to present backward compatible APIs to our CLI and UI like stratos, while moving the implementation down into controllers fronted by CRDs.  There's an obvious potential to interact with k8s directly in the future, but the current use case is to be used by CC. Except for kpack which is an excellent stand alone tool already :).  

These efforts are great and are allowing us to begin a path towards using existing k8s tooling and projects to provide Cloud Foundry outcomes, but obviously there's a lot further to go.

From VMware's side, over the last couple weeks, there has been an effort to explore what it might look like to use CRDs entirely as a source of truth for *all* CF data, in place of a SQL database, as a learning exercise, not necessarily a desired outcome. The primary goal of that effort was to start the convo that we're having now, and to be able to contribute in a meaningful way.  Note there are no VMware-based plans to dump a fully fledged proposal. We need to start from a premise as a community of understanding how/why/and which resources are valuable as CRDs.

For example when we look at orgs/spaces - we have to wonder if those are even resources we want to represent as CRDs, when k8s already has a namespace construct.

I think we'll need to start some dedicated working document to collaborate in. I had planned on facilitating that after some of the investigation that VMware has wraps up, but I don't think that's a reason to hold off if you or somebody else in the community has thoughts they want to start getting down. CAPI and other teams can contribute to an existing doc if that makes sense. Any other thoughts you'd prefer for next steps?

Looking forward to a continued conversation.

-Zach


On Wed, Apr 15, 2020 at 9:56 AM Keshav Sharma <ksharma@...> wrote:
Hi Daniel, 
The CF-K8s Networking team is currently working on a solution using Route CRDto introduce a “Route” custom resource and have Cloud Controller make/update these directly as part of the `cf map-route` and `cf unmap-route` workflows. Happy to discuss this further in our Cloud-Foundry Slack- #networking group. Regards, Keshav Sharma Product Manager | CF-K8s-Networking VMware

On Wed, Apr 15, 2020 at 3:01 AM Daniel Jones <daniel.jones@...> wrote:
Hi all,

In the spirit of community, I'm going to ask this question outright and in the open - is anyone (*looks at VMware MAPBU*) working on a CRD and accompanying controller for CF on Kubernetes?

Anyone with a non-trivial amount of Cloud Foundry experience knows that whilst the imperative interface of the CF CLI is great for exploratory work, it's far from ideal for production environments. I make a point of telling folks in CF training courses that no human should be using the CLI in production, other than for debugging.

A declarative interface for CF has always been desirable, and its omission is going to become even more stark once running CF on Kubernetes is the norm. We've had cf-mgmt which is close, but involves more steps than is ideal.

It'd be great to have one or more CRDs that represent the state of a Cloud Foundry, and accompanying controllers that converge upon that state. When I talk about state, I mean things like orgs, spaces, service broker registrations, service instances, roles and so on.

Four years ago I started work on a project called cf-converger to diff and converge a Cloud Foundry instance on a state declared in YAML. I got distracted with running a business and doing billable work, so whilst the concept was proven, it never really got to a state of usefulness. Plus, I seemed to spend half my coding time just writing excessively-verbose chained method calls on the official Java CF client :D 

I see such a solution as an inevitability in the Kubernetes-based future. Whilst I'm personally tempted to resurrect the project, it makes very little sense for a company like EngineerBetter to divert resources from billable work to a project that is needed, but can't be monetised.

Are any of the big players working on something similar? Is it something we should all be thinking and talking about? Is there any point in the community working on such a solution, or are one of the bigger companies secretly working on something that would supersede any community effort?

Regards,
Daniel 'Deejay' Jones - CEO
+44 (0)79 8000 9153
EngineerBetter Ltd - More than cloud platform specialists


Re: CF Application Runtime PMC: UAA Project Lead Call for Nominations

Eric Malm
 

Hi, everyone,

VMware is nominating Pablo Schuhmacher for the UAA Project Lead in the Application Runtime PMC.

Pablo is a senior product manager at VMware working with the UAA team. Over the course of his 4 years at Pivotal and VMware, Pablo has worked across several Pivotal R&D teams and on many Pivotal Labs engagements. He has also worked with the Innovation Strategy team to provide consulting on product strategy and enterprise transformation for clients such as The Home Depot, Boeing, and Dick’s Sporting Goods.

Please send any other nominations directly to me or in reply to this message no later than 11:59 PM PDT on Wednesday, April 29, 2020.

Thanks,
Eric Malm


From: cf-dev@... <cf-dev@...> on behalf of Eric Malm via lists.cloudfoundry.org <emalm=vmware.com@...>
Sent: Wednesday, April 15, 2020 5:32 PM
To: Discussions about Cloud Foundry projects and the system overall. <cf-dev@...>
Subject: [cf-dev] CF Application Runtime PMC: UAA Project Lead Call for Nominations
 
Hi, everyone,

Dan Beneke, the Project Lead for the UAA team within the Application Runtime PMC, has stepped down from the project, as he has recently left VMware. We thank him for his service.

The UAA team, based in San Francisco, now has an opening for its project lead. Project leads must be nominated by a Cloud Foundry Foundation member. Please send nominations directly to me or in reply to this message no later than 11:59 PM PDT on Wednesday, April 29, 2020.

Also, if you have any questions about the role or the nomination process, as described in the CFF governance documents (https://www.cloudfoundry.org/governance/cff_development_operations_policy/), please let me know.

Thanks,
Eric Malm, CF Application Runtime PMC Lead


CF Application Runtime PMC: UAA Project Lead Call for Nominations

Eric Malm
 

Hi, everyone,

Dan Beneke, the Project Lead for the UAA team within the Application Runtime PMC, has stepped down from the project, as he has recently left VMware. We thank him for his service.

The UAA team, based in San Francisco, now has an opening for its project lead. Project leads must be nominated by a Cloud Foundry Foundation member. Please send nominations directly to me or in reply to this message no later than 11:59 PM PDT on Wednesday, April 29, 2020.

Also, if you have any questions about the role or the nomination process, as described in the CFF governance documents (https://www.cloudfoundry.org/governance/cff_development_operations_policy/), please let me know.

Thanks,
Eric Malm, CF Application Runtime PMC Lead


Re: Is anyone working on a CF CRD & Controller?

Zach Robinson
 

Hey Daniel,

Thanks for starting a convo. The timing couldn't be better. I can share some of the thinking going on in both CAPI and VMware land. 

But first and foremost I want to address the question: "Is there any point in the community working on such a solution, or are one of the bigger companies secretly working on something that would supersede any community effort?" - yes, yes, a thousand times yes to community collaboration!

The topic of CRDs is kind of huge. It's really great to hear your focus is on things like orgs/spaces. From a CAPI perspective, I've been thinking more about representing apps and push as CRDs, and I believe the Eirini team has some thoughts in this area as well. There's a lot of ground to cover clearly.

In terms of things that are in flight now.
  • Keshav kindly shared that networking is looking to provide Route CRDs.
  • Networking is also exploring providing Security Groups as CRDs.
  • CAPI is integrating kpack into CF to provide buildpack staging. This is done by generating kpack CRDs.
Currently these CRDs are being created and owned by the Cloud Controller, which enables us to present backward compatible APIs to our CLI and UI like stratos, while moving the implementation down into controllers fronted by CRDs.  There's an obvious potential to interact with k8s directly in the future, but the current use case is to be used by CC. Except for kpack which is an excellent stand alone tool already :).  

These efforts are great and are allowing us to begin a path towards using existing k8s tooling and projects to provide Cloud Foundry outcomes, but obviously there's a lot further to go.

From VMware's side, over the last couple weeks, there has been an effort to explore what it might look like to use CRDs entirely as a source of truth for *all* CF data, in place of a SQL database, as a learning exercise, not necessarily a desired outcome. The primary goal of that effort was to start the convo that we're having now, and to be able to contribute in a meaningful way.  Note there are no VMware-based plans to dump a fully fledged proposal. We need to start from a premise as a community of understanding how/why/and which resources are valuable as CRDs.

For example when we look at orgs/spaces - we have to wonder if those are even resources we want to represent as CRDs, when k8s already has a namespace construct.

I think we'll need to start some dedicated working document to collaborate in. I had planned on facilitating that after some of the investigation that VMware has wraps up, but I don't think that's a reason to hold off if you or somebody else in the community has thoughts they want to start getting down. CAPI and other teams can contribute to an existing doc if that makes sense. Any other thoughts you'd prefer for next steps?

Looking forward to a continued conversation.

-Zach


On Wed, Apr 15, 2020 at 9:56 AM Keshav Sharma <ksharma@...> wrote:
Hi Daniel, 
The CF-K8s Networking team is currently working on a solution using Route CRDto introduce a “Route” custom resource and have Cloud Controller make/update these directly as part of the `cf map-route` and `cf unmap-route` workflows. Happy to discuss this further in our Cloud-Foundry Slack- #networking group. Regards, Keshav Sharma Product Manager | CF-K8s-Networking VMware

On Wed, Apr 15, 2020 at 3:01 AM Daniel Jones <daniel.jones@...> wrote:
Hi all,

In the spirit of community, I'm going to ask this question outright and in the open - is anyone (*looks at VMware MAPBU*) working on a CRD and accompanying controller for CF on Kubernetes?

Anyone with a non-trivial amount of Cloud Foundry experience knows that whilst the imperative interface of the CF CLI is great for exploratory work, it's far from ideal for production environments. I make a point of telling folks in CF training courses that no human should be using the CLI in production, other than for debugging.

A declarative interface for CF has always been desirable, and its omission is going to become even more stark once running CF on Kubernetes is the norm. We've had cf-mgmt which is close, but involves more steps than is ideal.

It'd be great to have one or more CRDs that represent the state of a Cloud Foundry, and accompanying controllers that converge upon that state. When I talk about state, I mean things like orgs, spaces, service broker registrations, service instances, roles and so on.

Four years ago I started work on a project called cf-converger to diff and converge a Cloud Foundry instance on a state declared in YAML. I got distracted with running a business and doing billable work, so whilst the concept was proven, it never really got to a state of usefulness. Plus, I seemed to spend half my coding time just writing excessively-verbose chained method calls on the official Java CF client :D 

I see such a solution as an inevitability in the Kubernetes-based future. Whilst I'm personally tempted to resurrect the project, it makes very little sense for a company like EngineerBetter to divert resources from billable work to a project that is needed, but can't be monetised.

Are any of the big players working on something similar? Is it something we should all be thinking and talking about? Is there any point in the community working on such a solution, or are one of the bigger companies secretly working on something that would supersede any community effort?

Regards,
Daniel 'Deejay' Jones - CEO
+44 (0)79 8000 9153
EngineerBetter Ltd - More than cloud platform specialists


Re: Is anyone working on a CF CRD & Controller?

Keshav Sharma
 

Hi Daniel, 
The CF-K8s Networking team is currently working on a solution using Route CRDto introduce a “Route” custom resource and have Cloud Controller make/update these directly as part of the `cf map-route` and `cf unmap-route` workflows. Happy to discuss this further in our Cloud-Foundry Slack- #networking group. Regards, Keshav Sharma Product Manager | CF-K8s-Networking VMware


On Wed, Apr 15, 2020 at 3:01 AM Daniel Jones <daniel.jones@...> wrote:
Hi all,

In the spirit of community, I'm going to ask this question outright and in the open - is anyone (*looks at VMware MAPBU*) working on a CRD and accompanying controller for CF on Kubernetes?

Anyone with a non-trivial amount of Cloud Foundry experience knows that whilst the imperative interface of the CF CLI is great for exploratory work, it's far from ideal for production environments. I make a point of telling folks in CF training courses that no human should be using the CLI in production, other than for debugging.

A declarative interface for CF has always been desirable, and its omission is going to become even more stark once running CF on Kubernetes is the norm. We've had cf-mgmt which is close, but involves more steps than is ideal.

It'd be great to have one or more CRDs that represent the state of a Cloud Foundry, and accompanying controllers that converge upon that state. When I talk about state, I mean things like orgs, spaces, service broker registrations, service instances, roles and so on.

Four years ago I started work on a project called cf-converger to diff and converge a Cloud Foundry instance on a state declared in YAML. I got distracted with running a business and doing billable work, so whilst the concept was proven, it never really got to a state of usefulness. Plus, I seemed to spend half my coding time just writing excessively-verbose chained method calls on the official Java CF client :D 

I see such a solution as an inevitability in the Kubernetes-based future. Whilst I'm personally tempted to resurrect the project, it makes very little sense for a company like EngineerBetter to divert resources from billable work to a project that is needed, but can't be monetised.

Are any of the big players working on something similar? Is it something we should all be thinking and talking about? Is there any point in the community working on such a solution, or are one of the bigger companies secretly working on something that would supersede any community effort?

Regards,
Daniel 'Deejay' Jones - CEO
+44 (0)79 8000 9153
EngineerBetter Ltd - More than cloud platform specialists

401 - 420 of 9377