Upcoming extraction of cflinuxfs2 rootfs release from diego-release


Eric Malm <emalm@...>
 

Dear CF Community,

Over the next few weeks, the Diego and Buildpacks teams will be working
together to extract a new BOSH release for the cflinuxfs2 rootfs/stack out
of the Diego BOSH release. The Buildpacks team has been doing an amazing
job of publishing new rootfs images in response to CVEs, and this
separation will make it easier for all Diego deployment operators to update
to those latest rootfs images without having to update their other
releases. We've already taken advantage of the same kind of separation
between Garden-Linux and Diego when addressing some recent Garden CVEs, and
we're looking forward to having that flexibility with the rootfs image as
well.

Once completed, the release extraction will mean a couple of minor changes
for Diego deployment operators:

- You'll have one more release to upload alongside the Diego, Garden-Linux,
and etcd BOSH releases to deploy your Diego cluster. The Diego release
tarball itself will be much smaller, as it will no longer include the
rootfs image that accounts for about 70% of its current size.
- If you use the spiff-based manifest-generation script in the
diego-release repo to produce your manifest, that's all you'll have to do!
If you're hand-rolling your manifests, you will have one or two BOSH
properties to add or move, and an entry to change in the list of job
templates on the Diego Cell VMs.

We'll call out these changes explicitly in the Diego release notes on
GitHub when the time comes.

Just as we do with the Garden-Linux and etcd releases today, the Diego team
will also attach a recent final cflinuxfs2-rootfs release tarball to each
final Diego release we publish on GitHub, so it will be easy for consumers
to get a validated set of default 'batteries' to plug into Diego. We'll
also work with the CF Release Integration team to make sure that when the
most recent rootfs image passes tests against Diego in their integration
environments, its release version is recorded in the Diego/CF compatibility
record, at
https://github.com/cloudfoundry-incubator/diego-cf-compatibility/blob/master/compatibility-v2.csv
.

If you would like to track our progress, please follow the
'cflinuxfs2-release-extraction' epic in the Diego tracker (
https://www.pivotaltracker.com/epic/show/2395419) and the 'bosh-release'
label in the Buildpacks tracker (
https://www.pivotaltracker.com/n/projects/1042066/search?q=label%3A%22bosh-release%22
).

Thanks,
Eric Malm, CF Runtime Diego PM


Gwenn Etourneau
 

That's nice.
Thanks Eric.

On Fri, Mar 18, 2016 at 8:01 AM, Eric Malm <emalm(a)pivotal.io> wrote:

Dear CF Community,

Over the next few weeks, the Diego and Buildpacks teams will be working
together to extract a new BOSH release for the cflinuxfs2 rootfs/stack out
of the Diego BOSH release. The Buildpacks team has been doing an amazing
job of publishing new rootfs images in response to CVEs, and this
separation will make it easier for all Diego deployment operators to update
to those latest rootfs images without having to update their other
releases. We've already taken advantage of the same kind of separation
between Garden-Linux and Diego when addressing some recent Garden CVEs, and
we're looking forward to having that flexibility with the rootfs image as
well.

Once completed, the release extraction will mean a couple of minor changes
for Diego deployment operators:

- You'll have one more release to upload alongside the Diego,
Garden-Linux, and etcd BOSH releases to deploy your Diego cluster. The
Diego release tarball itself will be much smaller, as it will no longer
include the rootfs image that accounts for about 70% of its current size.
- If you use the spiff-based manifest-generation script in the
diego-release repo to produce your manifest, that's all you'll have to do!
If you're hand-rolling your manifests, you will have one or two BOSH
properties to add or move, and an entry to change in the list of job
templates on the Diego Cell VMs.

We'll call out these changes explicitly in the Diego release notes on
GitHub when the time comes.

Just as we do with the Garden-Linux and etcd releases today, the Diego
team will also attach a recent final cflinuxfs2-rootfs release tarball to
each final Diego release we publish on GitHub, so it will be easy for
consumers to get a validated set of default 'batteries' to plug into Diego.
We'll also work with the CF Release Integration team to make sure that when
the most recent rootfs image passes tests against Diego in their
integration environments, its release version is recorded in the Diego/CF
compatibility record, at
https://github.com/cloudfoundry-incubator/diego-cf-compatibility/blob/master/compatibility-v2.csv
.

If you would like to track our progress, please follow the
'cflinuxfs2-release-extraction' epic in the Diego tracker (
https://www.pivotaltracker.com/epic/show/2395419) and the 'bosh-release'
label in the Buildpacks tracker (
https://www.pivotaltracker.com/n/projects/1042066/search?q=label%3A%22bosh-release%22
).

Thanks,
Eric Malm, CF Runtime Diego PM


Benjamin Gandon
 

Hi Eric,

Facing the pace of cflinuxfs2 updates, I've done and successfully deployed such a BOSH release. It also ships with example manifests. I'll share it with you this afternoon.

I've been using a "cheap" local blobstore but it would be interesting that you publish the blobs on a public bucket.

/Benjamin

Le 18 mars 2016 à 00:01, Eric Malm <emalm(a)pivotal.io> a écrit :

Dear CF Community,

Over the next few weeks, the Diego and Buildpacks teams will be working together to extract a new BOSH release for the cflinuxfs2 rootfs/stack out of the Diego BOSH release. The Buildpacks team has been doing an amazing job of publishing new rootfs images in response to CVEs, and this separation will make it easier for all Diego deployment operators to update to those latest rootfs images without having to update their other releases. We've already taken advantage of the same kind of separation between Garden-Linux and Diego when addressing some recent Garden CVEs, and we're looking forward to having that flexibility with the rootfs image as well.

Once completed, the release extraction will mean a couple of minor changes for Diego deployment operators:

- You'll have one more release to upload alongside the Diego, Garden-Linux, and etcd BOSH releases to deploy your Diego cluster. The Diego release tarball itself will be much smaller, as it will no longer include the rootfs image that accounts for about 70% of its current size.
- If you use the spiff-based manifest-generation script in the diego-release repo to produce your manifest, that's all you'll have to do! If you're hand-rolling your manifests, you will have one or two BOSH properties to add or move, and an entry to change in the list of job templates on the Diego Cell VMs.

We'll call out these changes explicitly in the Diego release notes on GitHub when the time comes.

Just as we do with the Garden-Linux and etcd releases today, the Diego team will also attach a recent final cflinuxfs2-rootfs release tarball to each final Diego release we publish on GitHub, so it will be easy for consumers to get a validated set of default 'batteries' to plug into Diego. We'll also work with the CF Release Integration team to make sure that when the most recent rootfs image passes tests against Diego in their integration environments, its release version is recorded in the Diego/CF compatibility record, at https://github.com/cloudfoundry-incubator/diego-cf-compatibility/blob/master/compatibility-v2.csv.

If you would like to track our progress, please follow the 'cflinuxfs2-release-extraction' epic in the Diego tracker (https://www.pivotaltracker.com/epic/show/2395419) and the 'bosh-release' label in the Buildpacks tracker (https://www.pivotaltracker.com/n/projects/1042066/search?q=label%3A%22bosh-release%22).

Thanks,
Eric Malm, CF Runtime Diego PM


Eric Malm <emalm@...>
 

Thanks, Benjamin! The Buildpacks team just started the extraction story (
https://www.pivotaltracker.com/story/show/115888335) yesterday and is
continuing it today, so now would be an ideal time for you to weigh in with
your extraction efforts.

Best,
Eric

On Thu, Mar 24, 2016 at 3:29 AM, Benjamin Gandon <benjamin(a)gandon.org>
wrote:

Hi Eric,

Facing the pace of cflinuxfs2 updates, I've done and successfully deployed
such a BOSH release. It also ships with example manifests. I'll share it
with you this afternoon.

I've been using a "cheap" local blobstore but it would be interesting that
you publish the blobs on a public bucket.

/Benjamin

Le 18 mars 2016 à 00:01, Eric Malm <emalm(a)pivotal.io> a écrit :

Dear CF Community,

Over the next few weeks, the Diego and Buildpacks teams will be working
together to extract a new BOSH release for the cflinuxfs2 rootfs/stack out
of the Diego BOSH release. The Buildpacks team has been doing an amazing
job of publishing new rootfs images in response to CVEs, and this
separation will make it easier for all Diego deployment operators to update
to those latest rootfs images without having to update their other
releases. We've already taken advantage of the same kind of separation
between Garden-Linux and Diego when addressing some recent Garden CVEs, and
we're looking forward to having that flexibility with the rootfs image as
well.

Once completed, the release extraction will mean a couple of minor changes
for Diego deployment operators:

- You'll have one more release to upload alongside the Diego,
Garden-Linux, and etcd BOSH releases to deploy your Diego cluster. The
Diego release tarball itself will be much smaller, as it will no longer
include the rootfs image that accounts for about 70% of its current size.
- If you use the spiff-based manifest-generation script in the
diego-release repo to produce your manifest, that's all you'll have to do!
If you're hand-rolling your manifests, you will have one or two BOSH
properties to add or move, and an entry to change in the list of job
templates on the Diego Cell VMs.

We'll call out these changes explicitly in the Diego release notes on
GitHub when the time comes.

Just as we do with the Garden-Linux and etcd releases today, the Diego
team will also attach a recent final cflinuxfs2-rootfs release tarball to
each final Diego release we publish on GitHub, so it will be easy for
consumers to get a validated set of default 'batteries' to plug into Diego.
We'll also work with the CF Release Integration team to make sure that when
the most recent rootfs image passes tests against Diego in their
integration environments, its release version is recorded in the Diego/CF
compatibility record, at
https://github.com/cloudfoundry-incubator/diego-cf-compatibility/blob/master/compatibility-v2.csv
.

If you would like to track our progress, please follow the
'cflinuxfs2-release-extraction' epic in the Diego tracker (
https://www.pivotaltracker.com/epic/show/2395419) and the 'bosh-release'
label in the Buildpacks tracker (
https://www.pivotaltracker.com/n/projects/1042066/search?q=label%3A%22bosh-release%22
).

Thanks,
Eric Malm, CF Runtime Diego PM


Benjamin Gandon
 

Ok sorry for the delay, I was listening and chatting with Josh here at Paris Spring Meetup. That was really cool.

Anyway, here it is : <https://github.com/bgandon/rootfses-boshrelease <https://github.com/bgandon/rootfses-boshrelease>>

Here is what I did :

1. Patch Diego deployment manifests with the deployment-samples/diego-manifests.yml.patch <https://github.com/bgandon/rootfses-boshrelease/blob/master/deployment-samples/diego-manifests.yml.patch>

2. Add the deployment-samples/property-overrides.yml <https://github.com/bgandon/rootfses-boshrelease/blob/master/deployment-samples/property-overrides.yml> to the Diego deployment

3. If needed, customize the deployment-samples/rootfses-properties.yml <https://github.com/bgandon/rootfses-boshrelease/blob/master/deployment-samples/rootfses-properties.yml>

4. Create the release tarball and upload it to the director

bosh create release --final --name rootfses --version 1.43.0
bosh upload release

5. Deploy

And it worked like a charm.

I shall add a README soon with those instructions.
Have fun!

/Benjamin

Le 24 mars 2016 à 19:28, Eric Malm <emalm(a)pivotal.io> a écrit :

Thanks, Benjamin! The Buildpacks team just started the extraction story (https://www.pivotaltracker.com/story/show/115888335 <https://www.pivotaltracker.com/story/show/115888335>) yesterday and is continuing it today, so now would be an ideal time for you to weigh in with your extraction efforts.

Best,
Eric

On Thu, Mar 24, 2016 at 3:29 AM, Benjamin Gandon <benjamin(a)gandon.org <mailto:benjamin(a)gandon.org>> wrote:
Hi Eric,

Facing the pace of cflinuxfs2 updates, I've done and successfully deployed such a BOSH release. It also ships with example manifests. I'll share it with you this afternoon.

I've been using a "cheap" local blobstore but it would be interesting that you publish the blobs on a public bucket.

/Benjamin

Le 18 mars 2016 à 00:01, Eric Malm <emalm(a)pivotal.io <mailto:emalm(a)pivotal.io>> a écrit :

Dear CF Community,

Over the next few weeks, the Diego and Buildpacks teams will be working together to extract a new BOSH release for the cflinuxfs2 rootfs/stack out of the Diego BOSH release. The Buildpacks team has been doing an amazing job of publishing new rootfs images in response to CVEs, and this separation will make it easier for all Diego deployment operators to update to those latest rootfs images without having to update their other releases. We've already taken advantage of the same kind of separation between Garden-Linux and Diego when addressing some recent Garden CVEs, and we're looking forward to having that flexibility with the rootfs image as well.

Once completed, the release extraction will mean a couple of minor changes for Diego deployment operators:

- You'll have one more release to upload alongside the Diego, Garden-Linux, and etcd BOSH releases to deploy your Diego cluster. The Diego release tarball itself will be much smaller, as it will no longer include the rootfs image that accounts for about 70% of its current size.
- If you use the spiff-based manifest-generation script in the diego-release repo to produce your manifest, that's all you'll have to do! If you're hand-rolling your manifests, you will have one or two BOSH properties to add or move, and an entry to change in the list of job templates on the Diego Cell VMs.

We'll call out these changes explicitly in the Diego release notes on GitHub when the time comes.

Just as we do with the Garden-Linux and etcd releases today, the Diego team will also attach a recent final cflinuxfs2-rootfs release tarball to each final Diego release we publish on GitHub, so it will be easy for consumers to get a validated set of default 'batteries' to plug into Diego. We'll also work with the CF Release Integration team to make sure that when the most recent rootfs image passes tests against Diego in their integration environments, its release version is recorded in the Diego/CF compatibility record, at https://github.com/cloudfoundry-incubator/diego-cf-compatibility/blob/master/compatibility-v2.csv <https://github.com/cloudfoundry-incubator/diego-cf-compatibility/blob/master/compatibility-v2.csv>.

If you would like to track our progress, please follow the 'cflinuxfs2-release-extraction' epic in the Diego tracker (https://www.pivotaltracker.com/epic/show/2395419 <https://www.pivotaltracker.com/epic/show/2395419>) and the 'bosh-release' label in the Buildpacks tracker (https://www.pivotaltracker.com/n/projects/1042066/search?q=label%3A%22bosh-release%22 <https://www.pivotaltracker.com/n/projects/1042066/search?q=label%3A%22bosh-release%22>).

Thanks,
Eric Malm, CF Runtime Diego PM


Benjamin Gandon
 

And here is the README!

https://github.com/bgandon/rootfses-boshrelease/blob/master/README.md <https://github.com/bgandon/rootfses-boshrelease/blob/master/README.md>

I took the chance of upgrading to cflinuxfs2 v1.48.0 to document how people can do that themselves.
That’s a pretty comprehensive 15-steps workflow!

/Benjamin

Le 25 mars 2016 à 00:47, Benjamin Gandon <benjamin(a)gandon.org> a écrit :

Ok sorry for the delay, I was listening and chatting with Josh here at Paris Spring Meetup. That was really cool.

Anyway, here it is : <https://github.com/bgandon/rootfses-boshrelease <https://github.com/bgandon/rootfses-boshrelease>>

Here is what I did :

1. Patch Diego deployment manifests with the deployment-samples/diego-manifests.yml.patch <https://github.com/bgandon/rootfses-boshrelease/blob/master/deployment-samples/diego-manifests.yml.patch>

2. Add the deployment-samples/property-overrides.yml <https://github.com/bgandon/rootfses-boshrelease/blob/master/deployment-samples/property-overrides.yml> to the Diego deployment

3. If needed, customize the deployment-samples/rootfses-properties.yml <https://github.com/bgandon/rootfses-boshrelease/blob/master/deployment-samples/rootfses-properties.yml>

4. Create the release tarball and upload it to the director

bosh create release --final --name rootfses --version 1.43.0
bosh upload release

5. Deploy

And it worked like a charm.

I shall add a README soon with those instructions.
Have fun!

/Benjamin


Le 24 mars 2016 à 19:28, Eric Malm <emalm(a)pivotal.io <mailto:emalm(a)pivotal.io>> a écrit :

Thanks, Benjamin! The Buildpacks team just started the extraction story (https://www.pivotaltracker.com/story/show/115888335 <https://www.pivotaltracker.com/story/show/115888335>) yesterday and is continuing it today, so now would be an ideal time for you to weigh in with your extraction efforts.

Best,
Eric

On Thu, Mar 24, 2016 at 3:29 AM, Benjamin Gandon <benjamin(a)gandon.org <mailto:benjamin(a)gandon.org>> wrote:
Hi Eric,

Facing the pace of cflinuxfs2 updates, I've done and successfully deployed such a BOSH release. It also ships with example manifests. I'll share it with you this afternoon.

I've been using a "cheap" local blobstore but it would be interesting that you publish the blobs on a public bucket.

/Benjamin

Le 18 mars 2016 à 00:01, Eric Malm <emalm(a)pivotal.io <mailto:emalm(a)pivotal.io>> a écrit :

Dear CF Community,

Over the next few weeks, the Diego and Buildpacks teams will be working together to extract a new BOSH release for the cflinuxfs2 rootfs/stack out of the Diego BOSH release. The Buildpacks team has been doing an amazing job of publishing new rootfs images in response to CVEs, and this separation will make it easier for all Diego deployment operators to update to those latest rootfs images without having to update their other releases. We've already taken advantage of the same kind of separation between Garden-Linux and Diego when addressing some recent Garden CVEs, and we're looking forward to having that flexibility with the rootfs image as well.

Once completed, the release extraction will mean a couple of minor changes for Diego deployment operators:

- You'll have one more release to upload alongside the Diego, Garden-Linux, and etcd BOSH releases to deploy your Diego cluster. The Diego release tarball itself will be much smaller, as it will no longer include the rootfs image that accounts for about 70% of its current size.
- If you use the spiff-based manifest-generation script in the diego-release repo to produce your manifest, that's all you'll have to do! If you're hand-rolling your manifests, you will have one or two BOSH properties to add or move, and an entry to change in the list of job templates on the Diego Cell VMs.

We'll call out these changes explicitly in the Diego release notes on GitHub when the time comes.

Just as we do with the Garden-Linux and etcd releases today, the Diego team will also attach a recent final cflinuxfs2-rootfs release tarball to each final Diego release we publish on GitHub, so it will be easy for consumers to get a validated set of default 'batteries' to plug into Diego. We'll also work with the CF Release Integration team to make sure that when the most recent rootfs image passes tests against Diego in their integration environments, its release version is recorded in the Diego/CF compatibility record, at https://github.com/cloudfoundry-incubator/diego-cf-compatibility/blob/master/compatibility-v2.csv <https://github.com/cloudfoundry-incubator/diego-cf-compatibility/blob/master/compatibility-v2.csv>.

If you would like to track our progress, please follow the 'cflinuxfs2-release-extraction' epic in the Diego tracker (https://www.pivotaltracker.com/epic/show/2395419 <https://www.pivotaltracker.com/epic/show/2395419>) and the 'bosh-release' label in the Buildpacks tracker (https://www.pivotaltracker.com/n/projects/1042066/search?q=label%3A%22bosh-release%22 <https://www.pivotaltracker.com/n/projects/1042066/search?q=label%3A%22bosh-release%22>).

Thanks,
Eric Malm, CF Runtime Diego PM