Topics

Removing Support for v1 Style Manifests

Morgan Fine
 

Friends of BOSH & CF,

In the olden days of BOSH, all IaaS configuration was done in individual deployment manifests, a la "v1 manifests". This became tedious and difficult for operators to mange. 

In order to abstract IaaS configuration, and simplify the operator workflow, BOSH added support for "v2 manifests". This introduced a concept called Cloud Config in order to consolidate IaaS configuration in a single file. The new "v2 manifests" reference IaaS concepts/objects from the Cloud Config and were more reusable across infrastructures.

Support for "v2 manifests" was added over 2 years ago. We believe this has given ample time for operators and release authors to migrate and stop relying on v1 syntax. Thus, the time has come where the BOSH team would like to simplify the code base and formally remove support for "v1 manifests". 

If you have any feedback about this change, please let us know.

For reference:
v1 manifest documentation: https://bosh.io/docs/deployment-manifest/
v2 manifest documentation: https://bosh.io/docs/manifest-v2/

Best,
Morgan Fine
CF BOSH

Ronak Banka
 

Great job BOSH team 👏🏻

On Tue, 19 Mar 2019 at 12:36 AM, Morgan Fine <mfine@...> wrote:
Friends of BOSH & CF,

In the olden days of BOSH, all IaaS configuration was done in individual deployment manifests, a la "v1 manifests". This became tedious and difficult for operators to mange. 

In order to abstract IaaS configuration, and simplify the operator workflow, BOSH added support for "v2 manifests". This introduced a concept called Cloud Config in order to consolidate IaaS configuration in a single file. The new "v2 manifests" reference IaaS concepts/objects from the Cloud Config and were more reusable across infrastructures.

Support for "v2 manifests" was added over 2 years ago. We believe this has given ample time for operators and release authors to migrate and stop relying on v1 syntax. Thus, the time has come where the BOSH team would like to simplify the code base and formally remove support for "v1 manifests". 

If you have any feedback about this change, please let us know.

For reference:
v1 manifest documentation: https://bosh.io/docs/deployment-manifest/
v2 manifest documentation: https://bosh.io/docs/manifest-v2/

Best,
Morgan Fine
CF BOSH

Corey Innis
 

+100 ... big proponent of making v1 manifests obsolete.

Cheers,
Corey

On Mon, Mar 18, 2019 at 6:29 PM Ronak Banka <ronakbanka.cse@...> wrote:
Great job BOSH team 👏🏻

On Tue, 19 Mar 2019 at 12:36 AM, Morgan Fine <mfine@...> wrote:
Friends of BOSH & CF,

In the olden days of BOSH, all IaaS configuration was done in individual deployment manifests, a la "v1 manifests". This became tedious and difficult for operators to mange. 

In order to abstract IaaS configuration, and simplify the operator workflow, BOSH added support for "v2 manifests". This introduced a concept called Cloud Config in order to consolidate IaaS configuration in a single file. The new "v2 manifests" reference IaaS concepts/objects from the Cloud Config and were more reusable across infrastructures.

Support for "v2 manifests" was added over 2 years ago. We believe this has given ample time for operators and release authors to migrate and stop relying on v1 syntax. Thus, the time has come where the BOSH team would like to simplify the code base and formally remove support for "v1 manifests". 

If you have any feedback about this change, please let us know.

For reference:
v1 manifest documentation: https://bosh.io/docs/deployment-manifest/
v2 manifest documentation: https://bosh.io/docs/manifest-v2/

Best,
Morgan Fine
CF BOSH

Stanislav German-Evtushenko
 

How to know what exactly is planned to be removed? I remember some overlaps between v1 and v2 as well as not explicitly documented features which worked.

What happens to bosh create-env manifest, will it be documented separately? As of now, it's a mix of both (v1 and v2) because it should support cloud properties.

Good time to update bosh-deployment btw.

Morgan Fine
 

Hi all,

A few clarifications and questions that have come up. While the create-env flow is still using "v1 manifest" structure, that code path does not involve the BOSH director itself as create-env is done in the CLI. To that end, we are not intending to change the create-env manifest structure at this time, only the manifest structure used in `bosh deploy` commands. 

In regards to what are we intending to remove, at a high level, we are intending to remove the sections from the v1 manifest page that have migrated to v2 syntax:
  • Deployment Identification i.e. "director_uuid"
  • Networks
  • Resource Pools
  • Disk Pools
  • Compilation
  • Jobs becomes Instance Groups
Best,
Morgan

On Thu, Mar 21, 2019 at 8:10 AM Stanislav German-Evtushenko <s.germanevtushenko@...> wrote:
How to know what exactly is planned to be removed? I remember some overlaps between v1 and v2 as well as not explicitly documented features which worked.

What happens to bosh create-env manifest, will it be documented separately? As of now, it's a mix of both (v1 and v2) because it should support cloud properties.

Good time to update bosh-deployment btw.

Jeanyhwh Desulme
 

Hey Morgan -

This is unfortunately a breaking change for our implementation of Bosh. We looked at fully implementing V2 some time ago but found that there was an issue in how we wanted to use it. For example our pipelines combine both a CFT and a deployment manifest. So that when we run a deployment of a given application we are passing down physical ids of native aws resources directly into the manifest itself. The following open Github issue has some more specifics in what were looking to do and what we currently have in place today.

Ideally we would love to move but only if we can override some of the deployment configuration at deploy time. 

https://github.com/cloudfoundry/bosh/issues/2094

Regards,
Jean 
Liberty Mutual

Morgan Fine
 

Hey Jean,

Apologies for the delay. Thank you for raising your concerns around the removal of v1 manifest support. Looking at the Github issue you raised, it seems like while not ideal from your perspective, the workflow that Maya proposed would work. It's also worth mentioning that we're thinking about what it might look like to have BOSH be able to manage more and more of the IaaS resources that your issue references so that could be alternative way to support your use case.

That being said, we are still planning to proceed with removing v1 manifests. It would be great if you could give some of the proposals in that Github issue a try and give us feedback there, that way we can work to ensure your use case is still covered in a v2 world. 

Thank you again for the feedback.

Best,
Morgan

On Mon, Apr 1, 2019 at 8:38 AM Jeanyhwh Desulme via Lists.Cloudfoundry.Org <jeanyhwh.desulme=libertymutual.com@...> wrote:
Hey Morgan -

This is unfortunately a breaking change for our implementation of Bosh. We looked at fully implementing V2 some time ago but found that there was an issue in how we wanted to use it. For example our pipelines combine both a CFT and a deployment manifest. So that when we run a deployment of a given application we are passing down physical ids of native aws resources directly into the manifest itself. The following open Github issue has some more specifics in what were looking to do and what we currently have in place today.

Ideally we would love to move but only if we can override some of the deployment configuration at deploy time. 

https://github.com/cloudfoundry/bosh/issues/2094

Regards,
Jean 
Liberty Mutual