BPM Incubation Proposal


Christopher Brown
 

Hi all,

We’ve been working internally on experiments to isolate BOSH jobs from one
another and providing a friendlier and less error-prone interface than the
current boilerplate bash control scripts. We call this project BPM
(initially BOSH Process Manager).

We’ve successfully written an internal proof-of-concept of this project. It
is able to run many of the Cloud Foundry runtime jobs (CC, Diego,
Loggregator, UAA) inside containers. BPM manages the lifecycle of the jobs,
isolates the jobs from one another, and restricts the job’s capabilities
such that they are only allowed to perform actions crucial to their
function. At the same time we were able to remove the vast majority of the
surrounding duplicative, error prone bash control scripts. We are now in
the process of rebuilding this proof of concept into the production-ready
end product.

You can follow the development here:
https://github.com/pivotal-cf/bpm-release

I’d like to propose the incubation of this project under the BOSH PMC. We’d
like to start by adding this functionality behind a feature flag to the
Diego release. Once this has been successfully completed then we’d be
interested in expanding the deployment if there is demand.

Thanks!

Christopher Brown
PCF Security


Dr Nic Williams
 

This seems super cool. Is there an example release that uses bpm rather than monit etc?

________________________________
From: Christopher Brown <cbrown(a)pivotal.io>
Sent: Wednesday, July 12, 2017 8:24:20 AM
To: cf-bosh(a)lists.cloudfoundry.org
Subject: [cf-bosh] BPM Incubation Proposal

Hi all,

We’ve been working internally on experiments to isolate BOSH jobs from one another and providing a friendlier and less error-prone interface than the current boilerplate bash control scripts. We call this project BPM (initially BOSH Process Manager).

We’ve successfully written an internal proof-of-concept of this project. It is able to run many of the Cloud Foundry runtime jobs (CC, Diego, Loggregator, UAA) inside containers. BPM manages the lifecycle of the jobs, isolates the jobs from one another, and restricts the job’s capabilities such that they are only allowed to perform actions crucial to their function. At the same time we were able to remove the vast majority of the surrounding duplicative, error prone bash control scripts. We are now in the process of rebuilding this proof of concept into the production-ready end product.

You can follow the development here: https://github.com/pivotal-cf/bpm-release

I’d like to propose the incubation of this project under the BOSH PMC. We’d like to start by adding this functionality behind a feature flag to the Diego release. Once this has been successfully completed then we’d be interested in expanding the deployment if there is demand.

Thanks!

Christopher Brown
PCF Security


Dmitriy Kalinin
 

Is there an example release that uses bpm rather than monit etc?
im sure there is going to be one very soon but here is a commit on
diego-release that makes it optional configuration for a trial run (
https://github.com/cloudfoundry/diego-release/commit/27ee06ecdfccb3d19026a3d7f7d36ad480def84e
).

note that bpm is working under monit, not as a replacement.

On Thu, Jul 13, 2017 at 4:33 PM, Dr Nic Williams <drnicwilliams(a)gmail.com>
wrote:

This seems super cool. Is there an example release that uses bpm rather
than monit etc?

------------------------------
*From:* Christopher Brown <cbrown(a)pivotal.io>
*Sent:* Wednesday, July 12, 2017 8:24:20 AM
*To:* cf-bosh(a)lists.cloudfoundry.org
*Subject:* [cf-bosh] BPM Incubation Proposal

Hi all,

We’ve been working internally on experiments to isolate BOSH jobs from one
another and providing a friendlier and less error-prone interface than the
current boilerplate bash control scripts. We call this project BPM
(initially BOSH Process Manager).

We’ve successfully written an internal proof-of-concept of this project.
It is able to run many of the Cloud Foundry runtime jobs (CC, Diego,
Loggregator, UAA) inside containers. BPM manages the lifecycle of the jobs,
isolates the jobs from one another, and restricts the job’s capabilities
such that they are only allowed to perform actions crucial to their
function. At the same time we were able to remove the vast majority of the
surrounding duplicative, error prone bash control scripts. We are now in
the process of rebuilding this proof of concept into the production-ready
end product.

You can follow the development here: https://github.com/pivotal-cf/
bpm-release

I’d like to propose the incubation of this project under the BOSH PMC.
We’d like to start by adding this functionality behind a feature flag to
the Diego release. Once this has been successfully completed then we’d be
interested in expanding the deployment if there is demand.

Thanks!

Christopher Brown
PCF Security


Alex Ley
 

Hi Chris,

Very cool!

What is the thinking of having this as an additional release and not part
core BOSH as an experimental feature? Do you see this being moved into core
BOSH at some point?

On 14 July 2017 at 01:40, Dmitriy Kalinin <dkalinin(a)pivotal.io> wrote:

Is there an example release that uses bpm rather than monit etc?
im sure there is going to be one very soon but here is a commit on
diego-release that makes it optional configuration for a trial run (
https://github.com/cloudfoundry/diego-release/commit/
27ee06ecdfccb3d19026a3d7f7d36ad480def84e).

note that bpm is working under monit, not as a replacement.

On Thu, Jul 13, 2017 at 4:33 PM, Dr Nic Williams <drnicwilliams(a)gmail.com>
wrote:

This seems super cool. Is there an example release that uses bpm rather
than monit etc?

------------------------------
*From:* Christopher Brown <cbrown(a)pivotal.io>
*Sent:* Wednesday, July 12, 2017 8:24:20 AM
*To:* cf-bosh(a)lists.cloudfoundry.org
*Subject:* [cf-bosh] BPM Incubation Proposal

Hi all,

We’ve been working internally on experiments to isolate BOSH jobs from
one another and providing a friendlier and less error-prone interface than
the current boilerplate bash control scripts. We call this project BPM
(initially BOSH Process Manager).

We’ve successfully written an internal proof-of-concept of this project.
It is able to run many of the Cloud Foundry runtime jobs (CC, Diego,
Loggregator, UAA) inside containers. BPM manages the lifecycle of the jobs,
isolates the jobs from one another, and restricts the job’s capabilities
such that they are only allowed to perform actions crucial to their
function. At the same time we were able to remove the vast majority of the
surrounding duplicative, error prone bash control scripts. We are now in
the process of rebuilding this proof of concept into the production-ready
end product.

You can follow the development here: https://github.com/pivotal-cf/
bpm-release

I’d like to propose the incubation of this project under the BOSH PMC.
We’d like to start by adding this functionality behind a feature flag to
the Diego release. Once this has been successfully completed then we’d be
interested in expanding the deployment if there is demand.

Thanks!

Christopher Brown
PCF Security


Christopher Brown
 

Thanks, Alex.

The current design doesn't require any BOSH changes in order for it to
work. Keeping it separate from BOSH in the meantime has the advantages that
we can make experimental changes and try them out without waiting for the
BOSH release cycle.

It may eventually find its way into BOSH but I'm not responsible for making
that decision. We haven't discussed it in detail. It's still early days and
that decision would rely heavily on whether or not people find it useful.

On Fri, Jul 14, 2017 at 3:44 AM, Alex Ley <aley(a)pivotal.io> wrote:

Hi Chris,

Very cool!

What is the thinking of having this as an additional release and not part
core BOSH as an experimental feature? Do you see this being moved into core
BOSH at some point?

On 14 July 2017 at 01:40, Dmitriy Kalinin <dkalinin(a)pivotal.io> wrote:

Is there an example release that uses bpm rather than monit etc?
im sure there is going to be one very soon but here is a commit on
diego-release that makes it optional configuration for a trial run (
https://github.com/cloudfoundry/diego-release/commit/27ee06
ecdfccb3d19026a3d7f7d36ad480def84e).

note that bpm is working under monit, not as a replacement.

On Thu, Jul 13, 2017 at 4:33 PM, Dr Nic Williams <drnicwilliams(a)gmail.com
wrote:
This seems super cool. Is there an example release that uses bpm rather
than monit etc?

------------------------------
*From:* Christopher Brown <cbrown(a)pivotal.io>
*Sent:* Wednesday, July 12, 2017 8:24:20 AM
*To:* cf-bosh(a)lists.cloudfoundry.org
*Subject:* [cf-bosh] BPM Incubation Proposal

Hi all,

We’ve been working internally on experiments to isolate BOSH jobs from
one another and providing a friendlier and less error-prone interface than
the current boilerplate bash control scripts. We call this project BPM
(initially BOSH Process Manager).

We’ve successfully written an internal proof-of-concept of this project.
It is able to run many of the Cloud Foundry runtime jobs (CC, Diego,
Loggregator, UAA) inside containers. BPM manages the lifecycle of the jobs,
isolates the jobs from one another, and restricts the job’s capabilities
such that they are only allowed to perform actions crucial to their
function. At the same time we were able to remove the vast majority of the
surrounding duplicative, error prone bash control scripts. We are now in
the process of rebuilding this proof of concept into the production-ready
end product.

You can follow the development here: https://github.com/pivotal-cf/
bpm-release

I’d like to propose the incubation of this project under the BOSH PMC.
We’d like to start by adding this functionality behind a feature flag to
the Diego release. Once this has been successfully completed then we’d be
interested in expanding the deployment if there is demand.

Thanks!

Christopher Brown
PCF Security