Food for thought: One of the challenges here is that maintaining patches for past coordinated releases is expensive (both in time and CI costs). In the CF ecosystem, this has traditionally been the responsibility of the downstream commercial distributions.

This isn't to say that there isn't a solution that can help all downstream users (including non-commercial users AND the distros), yet not burden the Rel Int team too much. I'm not sure what that solution is though...

On Mon, Jul 16, 2018 at 9:47 AM Franks, Geoff <geoff.franks@...> wrote:

I’m going to agree with Marco’s concerns here. Making life harder and less stable for the end users of CF has a real potential to alienate and push away the CF userbase altogether, even if it’s just in appearance (seeing monthly major releases of a product may cause new organizations to hesitate to migrate, until the release process appears more stable.

 

 

From: <cf-dev@...> on behalf of Marco Voelz <marco.voelz@...>
Reply-To: "cf-dev@..." <cf-dev@...>
Date: Monday, July 16, 2018 at 1:34 AM
To: "cf-dev@..." <cf-dev@...>
Subject: [External] Re: [cf-dev] cf-deployment 3.0

 

Dear Josh,

 

Thanks for the context, I wasn't aware of what happened before the release of networking 2.0. To stick with your example, though: From what you are saying I have understood that you would rather have done it this way – please correct me here if I'm wrong:

• integrate networking release 2.0 into cf-deployment, 
• integrate other PRs with breaking changes
• bumping cf-deployment to a new major version, given above changes
• merging the CVE fixes only into the new major version of cf-deployment

 

With this process, you would have achieved the following:

• the development teams are happy, because they shipped as soon as they were ready to
• operators are grumpy, because they have to bump networking to a new major version and adopt to other breaking changes in order to fix CVEs

 

I'm not saying you have to turn this tradeoff the other way around, but in my opinion this doesn't seem very consumer friendly. 

 

In your team's mission, you have clearly stated that your goal is to enable development teams to maintain a high velocity. I'd like to stress that we shouldn't leave the operators and users out of the picture here. In the end, you're developing for them, not for yourself. 

 

I'm not sure if the consumer/operator persona is a thing for RelInt, but if that's the case, here's something I'd like to hold true for whatever change RelInt makes to their process:

"As an operator of CF, I'd like to consume CVE fixes with as little changes to my existing installation as possible, such that I close known vulnerabilities as soon as possible"

 

Does that sound reasonable?

 

Warm regards

Marco

---

From: cf-dev@... <cf-dev@...> on behalf of Josh Collins <jcollins@...>
Sent: Friday, July 13, 2018 11:39:30 PM
To: cf-dev@...
Subject: Re: [cf-dev] cf-deployment 3.0

 

Hi Marco,

I'm happy to provide more context on the container networking 2.0 reference.
The container networking team submitted a PR to cf-deployment with changes required for them to ship v2.0. 
RelInt deferred the container networking team's PR for a few weeks due to competing priorities including multiple CVE's fixes.
During the deferral time, a few other PRs were submitted which included breaking changes.
These additional changes took much more time to integrate and validate than anticipated and in the end, the container networking team's 2.0 release was published in cf-d about 5 weeks after it was ready to go.
The introduction of a regular cadence aims to mitigate this type of delay in the future. Had we had one at the time, the networking team would have timed it's PR to align and we would have been poised to accept and publish it quickly.
We believe this will help teams confidently plan for, communicate about, and negotiate integrating their releases into cf-deployment.
And hopefully enable the RelInt team to integrate and ship major releases more seamlessly.

This is an evolving process so we'll see how things roll in the coming months and make adjustments where it makes sense to do so. 
I appreciate and welcome any and all feedback along the way.

Thanks very much,

Josh

--

Chip Childers
CTO, Cloud Foundry Foundation
1.267.250.0815

I’m going to agree with Marco’s concerns here. Making life harder and less stable for the end users of CF has a real potential to alienate and push away the CF userbase altogether, even if it’s just in appearance (seeing monthly major releases of a product may cause new organizations to hesitate to migrate, until the release process appears more stable.

 

 

From: <cf-dev@...> on behalf of Marco Voelz <marco.voelz@...>
Reply-To: "cf-dev@..." <cf-dev@...>
Date: Monday, July 16, 2018 at 1:34 AM
To: "cf-dev@..." <cf-dev@...>
Subject: [External] Re: [cf-dev] cf-deployment 3.0

 

Dear Josh,

 

Thanks for the context, I wasn't aware of what happened before the release of networking 2.0. To stick with your example, though: From what you are saying I have understood that you would rather have done it this way – please correct me here if I'm wrong:

• integrate networking release 2.0 into cf-deployment, 
• integrate other PRs with breaking changes
• bumping cf-deployment to a new major version, given above changes
• merging the CVE fixes only into the new major version of cf-deployment

 

With this process, you would have achieved the following:

• the development teams are happy, because they shipped as soon as they were ready to
• operators are grumpy, because they have to bump networking to a new major version and adopt to other breaking changes in order to fix CVEs

 

I'm not saying you have to turn this tradeoff the other way around, but in my opinion this doesn't seem very consumer friendly. 

 

In your team's mission, you have clearly stated that your goal is to enable development teams to maintain a high velocity. I'd like to stress that we shouldn't leave the operators and users out of the picture here. In the end, you're developing for them, not for yourself. 

 

I'm not sure if the consumer/operator persona is a thing for RelInt, but if that's the case, here's something I'd like to hold true for whatever change RelInt makes to their process:

"As an operator of CF, I'd like to consume CVE fixes with as little changes to my existing installation as possible, such that I close known vulnerabilities as soon as possible"

 

Does that sound reasonable?

 

Warm regards

Marco

---

From: cf-dev@... <cf-dev@...> on behalf of Josh Collins <jcollins@...>
Sent: Friday, July 13, 2018 11:39:30 PM
To: cf-dev@...
Subject: Re: [cf-dev] cf-deployment 3.0

 

Hi Marco,

I'm happy to provide more context on the container networking 2.0 reference.
The container networking team submitted a PR to cf-deployment with changes required for them to ship v2.0. 
RelInt deferred the container networking team's PR for a few weeks due to competing priorities including multiple CVE's fixes.
During the deferral time, a few other PRs were submitted which included breaking changes.
These additional changes took much more time to integrate and validate than anticipated and in the end, the container networking team's 2.0 release was published in cf-d about 5 weeks after it was ready to go.
The introduction of a regular cadence aims to mitigate this type of delay in the future. Had we had one at the time, the networking team would have timed it's PR to align and we would have been poised to accept and publish it quickly.
We believe this will help teams confidently plan for, communicate about, and negotiate integrating their releases into cf-deployment.
And hopefully enable the RelInt team to integrate and ship major releases more seamlessly.

This is an evolving process so we'll see how things roll in the coming months and make adjustments where it makes sense to do so. 
I appreciate and welcome any and all feedback along the way.

Thanks very much,

Josh

aaarrgh, there is a bug in my psuedo code

clusterWeight = [v1,v1,v1,v1,v1,v1,v2,v2,v2,v3,v3] should be

clusterWeight = [v1,v1,v1,v1,v1,v1,v2,v2,v2,v3,v4]

Full solution:

Implementation: "Randomized Round Robin" is also super simple [pseudo code follows]

clusterWeight = [v1,v1,v1,v1,v1,v1,v2,v2,v2,v3,v4] //very easy to create based on base1 solution

randomCluster = random(clusterWeight) 

int atomicPointer = 0;

for each request:

  next = atomicPointer.getAndIncrease();

  application = randomCluster[atomicPointer];

On Fri, Jul 13, 2018 at 8:09 PM Filip Hanik <fhanik@...> wrote:

I put a long comment in the doc, maybe comments are good for short notes. here is the spiel

"The sum of weights must add to 100"

I would say this is where being user friendly ends. If I add reviews-v4 I have to go in and rebalance the whole thing just to figure out how to get to 100.

an alternate solution can be much simpler:

What if you just used a single integer that is relative to the whole cluster. let's call it "base1-lb"

reviews-v1: 6

reviews-v2: 3

reviews-v3: 1

there are two ways to think of this

relative to each other:

In this scenario, v1 gets twice as many requests as v2, and six times as many requests as v3

or in consideration of X requests: (and this is most likely how the code implements it so that it doesn't have to do a lot of math)

This is saying is that for every (total) 10 requests, this is how they distributed.

to add v4

reviews-v1: 6

reviews-v2: 3

reviews-v3: 1

reviews-v4: 1

this is still super simple to look at. v1 gets 6x more than v3/v4, still gets 2x more than v2. I don't have to figure out how to "add up to a 100"

and it's not complicated to calculate either. for every 11 requests:

v1 gets 6

v2 gets 3

v3 gets 1

v4 gets 1

Implementation: "Randomized Round Robin" is also super simple [pseudo code follows]

clusterWeight = [v1,v1,v1,v1,v1,v1,v2,v2,v2,v3,v3] //very easy to create based on base1 solution

randomCluster = random(clusterWeight) 

int atomicPointer = 0;

for each request:

  next = atomicPointer.getAndIncrease();

  application = randomCluster[atomicPointer];

and that's it. the router doesn't have to figure out where the next request goes. This is a simple, elegant and easy to understand solution.

Filip

On Fri, Jul 13, 2018 at 3:09 PM Shubha Anjur Tupil <sanjurtupil@...> wrote:

The CF Routing team has received feedback from many users that support for weighted routing would make it easier to accomplish their goals. We have a proposal on the preferred user experience for weighted routing and the considerations we have taken into account.

If you have thoughts on this or have experience working with traffic splitting on other platforms, please share your feedback with us. Feel free to comment on the doc or reply here.

Regards,

CF Routing Team

I put a long comment in the doc, maybe comments are good for short notes. here is the spiel

"The sum of weights must add to 100"

I would say this is where being user friendly ends. If I add reviews-v4 I have to go in and rebalance the whole thing just to figure out how to get to 100.

an alternate solution can be much simpler:

What if you just used a single integer that is relative to the whole cluster. let's call it "base1-lb"

reviews-v1: 6

reviews-v2: 3

reviews-v3: 1

there are two ways to think of this

relative to each other:

In this scenario, v1 gets twice as many requests as v2, and six times as many requests as v3

or in consideration of X requests: (and this is most likely how the code implements it so that it doesn't have to do a lot of math)

This is saying is that for every (total) 10 requests, this is how they distributed.

to add v4

reviews-v1: 6

reviews-v2: 3

reviews-v3: 1

reviews-v4: 1

this is still super simple to look at. v1 gets 6x more than v3/v4, still gets 2x more than v2. I don't have to figure out how to "add up to a 100"

and it's not complicated to calculate either. for every 11 requests:

v1 gets 6

v2 gets 3

v3 gets 1

v4 gets 1

Implementation: "Randomized Round Robin" is also super simple [pseudo code follows]

clusterWeight = [v1,v1,v1,v1,v1,v1,v2,v2,v2,v3,v3] //very easy to create based on base1 solution

randomCluster = random(clusterWeight) 

int atomicPointer = 0;

for each request:

  next = atomicPointer.getAndIncrease();

  application = randomCluster[atomicPointer];

and that's it. the router doesn't have to figure out where the next request goes. This is a simple, elegant and easy to understand solution.

Filip

On Fri, Jul 13, 2018 at 3:09 PM Shubha Anjur Tupil <sanjurtupil@...> wrote:

The CF Routing team has received feedback from many users that support for weighted routing would make it easier to accomplish their goals. We have a proposal on the preferred user experience for weighted routing and the considerations we have taken into account.

If you have thoughts on this or have experience working with traffic splitting on other platforms, please share your feedback with us. Feel free to comment on the doc or reply here.

Regards,

CF Routing Team

The CF Routing team has received feedback from many users that support for weighted routing would make it easier to accomplish their goals. We have a proposal on the preferred user experience for weighted routing and the considerations we have taken into account.

If you have thoughts on this or have experience working with traffic splitting on other platforms, please share your feedback with us. Feel free to comment on the doc or reply here.

Regards,

CF Routing Team

Also: I suspect that the CFCR route-sync feature has a dangerous interaction with CFAR Cloud Controller, if both CFCR and CFAR are sharing a TCP Routing API.  CFAR Cloud Controller creates and uses a TCP Router Group for itself, and expects to completely own that router group.  My reading of the CFCR code is that route-sync will happily discover and use that Router Group as-is.  The CFAR Routing API has no mechanism to prevent this collision, or to prevent the two clients from reserving the same TCP port for different backends.  I think that the result will be that ingress to that TCP Router Port will get load balanced to both the CFAR App and the CFCR Service.  This is likely not what the user intends.

On Fri, Jul 13, 2018 at 4:14 AM, arghya sadhu <arghya88@...> wrote:

Hi Oleksandr,

What alternative do we have if we want to use kubectl with tls

Thanks,

Arghya

On Fri, Jul 13, 2018, 3:01 PM <oslynko@...> wrote:

Hi, cf-dev

Almost one year ago CFCR has added the ability to expose applications using CFAR gorouter. This was an experiment.
We haven't added any changes to this feature for one year and plan to remove it in next release. It will greatly reduce the burden on the team.

If someone uses it, please contact us via email or Slack (#cfcr).

Thanks,
Oleksandr

Hi Oleksandr,

What alternative do we have if we want to use kubectl with tls

Thanks,

Arghya

On Fri, Jul 13, 2018, 3:01 PM <oslynko@...> wrote:

Hi, cf-dev

Almost one year ago CFCR has added the ability to expose applications using CFAR gorouter. This was an experiment.
We haven't added any changes to this feature for one year and plan to remove it in next release. It will greatly reduce the burden on the team.

If someone uses it, please contact us via email or Slack (#cfcr).

Thanks,
Oleksandr

Hi, cf-dev

Almost one year ago CFCR has added the ability to expose applications using CFAR gorouter. This was an experiment.
We haven't added any changes to this feature for one year and plan to remove it in next release. It will greatly reduce the burden on the team.

If someone uses it, please contact us via email or Slack (#cfcr).

Thanks,
Oleksandr