Re: Required manifest changes for Cloud Foundry


Benjamin Gandon
 

Indeed I scripted a couple of « bosh create release » / « bosh upload release » and then bosh-workspace is happy with it.

It worked like a charm. Buildpacks just end up being there, automatically updated. That’s Great!
I really look forward to being able to update the java-buildpack in the same way!

I’m not familiar to the inner workflow of bosh-workspace either. I use it because what you type just makes sense.
I suppose bosh-workspace uploads the manifest release and tell the director to recreate the release tarball from that. It’s time effective when your director has a much wider bandwidth than your Bosh CLI.

By the way, all config/final.yml for all buildpacks contain the same settings:

blobstore:
file_name: stacks
provider: s3
options:
bucket_name: pivotal-buildpacks
folder: tmp/builpacks-release-blobs

Is it normal that the filenames are all « stacks » for all buildpacks? I’m afraid these settings might not have been properly set. This would explain the whole thing.

/Benjamin

Le 11 mars 2016 à 00:04, Amit Gupta <agupta(a)pivotal.io> a écrit :

I did

cd cf-release
git fetch origin
git checkout develop
git pull --ff-only
./scripts/update
bosh create release --with-tarball

And also

cd src/buildpacks/binary-buildpack-release/
git fetch origin
(confirmed HEAD was pointed at origin/master)
bosh create release

Everything worked fine for me, meaning it was able to sync blobs down from the remote blobstores. I'm not familiar with bosh-workspace, and it's not clear to me why it's trying to upload anything (e.g. Uploading 'ruby-buildpack-release/1.6.14').

On Thu, Mar 10, 2016 at 2:48 PM, Benjamin Gandon <benjamin(a)gandon.org <mailto:benjamin(a)gandon.org>> wrote:
And btw Amit, it looks like the java-buildpack v3.6 is here with its fellows:
https://github.com/cloudfoundry/buildpack-releases/blob/master/java-buildpack-release/releases/java-buildpack-release/java-buildpack-release-3.6.yml <https://github.com/cloudfoundry/buildpack-releases/blob/master/java-buildpack-release/releases/java-buildpack-release/java-buildpack-release-3.6.yml>
For example the « e6ff7d79e50f0aaafa92f100f346e648c503ab17 » SHA in the error below (when recreating the java-buildpack-release) is the one of the first blob in the release manifest above.


Le 10 mars 2016 à 23:24, Benjamin Gandon <benjamin(a)gandon.org <mailto:benjamin(a)gandon.org>> a écrit :

No no no, these are not SHAs of cf-release, but those of all the buildpack-releases indeed.
Looks like no blobs of these releases are actually available online, are they?

I'm running the standard middle step "bosh prepare deployment" provided by bosh-workspace.
(See <https://github.com/cloudfoundry-incubator/bosh-workspace <https://github.com/cloudfoundry-incubator/bosh-workspace>>)

/Benjamin


Le 10 mars 2016 à 21:14, Amit Gupta <agupta(a)pivotal.io <mailto:agupta(a)pivotal.io>> a écrit :

At the time of the email, the java buildpack hadn't been extracted into a separate release yet. I believe it has now, and that will be reflected in CF v232.

What command did you run?
What SHA of cf-release have you checked out?

On Thu, Mar 10, 2016 at 12:10 PM, Benjamin Gandon <benjamin(a)gandon.org <mailto:benjamin(a)gandon.org>> wrote:
Amit, just for me to be sure, why didn’t you list the java-buildpack?

Also, have the blobs properly been uploaded?
I copy below the BOSH errors I get:

With binary-buildpack:

Uploading 'binary-buildpack-release/1.0.1'
Recreating release from the manifest
MISSING
Cannot find package with checksum `413ce11236f87273ba8a9249b6e3bebb3d0db92b'

With go-buildpack:

Uploading 'go-buildpack-release/1.7.3'
Recreating release from the manifest
MISSING
Cannot find package with checksum `300760637ee0babd5fddd474101dfa634116d9c4'

With java-buildpack:

Uploading 'java-buildpack-release/3.6'
Recreating release from the manifest
MISSING
Cannot find package with checksum `e6ff7d79e50f0aaafa92f100f346e648c503ab17'

With nodejs-buildpack:

Uploading 'nodejs-buildpack-release/1.5.7'
Recreating release from the manifest
MISSING
Cannot find package with checksum `b3edbcfb9435892749dffcb99f06d00fb4c59c5b'

With php-buildpack:

Uploading 'php-buildpack-release/4.3.6'
Recreating release from the manifest
MISSING
Cannot find package with checksum `fbc784608ffa3ceafed1810b69c12a7277c86ee0'

With python-buildpack:

Uploading 'python-buildpack-release/1.5.4'
Recreating release from the manifest
MISSING
Cannot find package with checksum `7e2377ccd9df10b21aba49c8e95338a0b1b3b92e'

With ruby-buildpack:

Uploading 'ruby-buildpack-release/1.6.14'
Recreating release from the manifest
MISSING
Cannot find package with checksum `362282d45873634db888a609cd64d7d70e9f4be2'

With staticfile-buildpack:

Uploading 'staticfile-buildpack-release/1.3.2'
Recreating release from the manifest
MISSING
Cannot find package with checksum `06382f7c804cc7f01a8dc78ca9c91e9b7f4712cc'

Are these on a specific blobstore I should point my deployment manifest at?

/Benjamin


Le 18 févr. 2016 à 19:31, Amit Gupta <agupta(a)pivotal.io <mailto:agupta(a)pivotal.io>> a écrit :

Hey developers,

The buildpacks team has recently extracted the buildpacks as separate releases. As we transition to deploying CF via a bunch of composed releases, for now we're making the change more transparent, by submoduling and symlinking the buildpacks releases back into cf-release. This requires some manifest changes: buildpacks are now colocated with cloud controller, rather than package dependencies of cloud controller.

If you are using spiff to generate manifests, and are not overriding the templates/jobs colocated on the api_zN jobs, you can ignore this email. If you are overriding the api_zN templates in your stub, or if you are not using spiff, you will need to add the following:

templates:
- name: consul_agent
release: cf
+ - name: go-buildpack
+ release: cf
+ - name: binary-buildpack
+ release: cf
+ - name: nodejs-buildpack
+ release: cf
+ - name: ruby-buildpack
+ release: cf
+ - name: php-buildpack
+ release: cf
+ - name: python-buildpack
+ release: cf
+ - name: staticfile-buildpack
+ release: cf
- name: cloud_controller_ng
release: cf

Please see this commit (https://github.com/cloudfoundry/cf-release/commit/549e5a8271bbf0d30efdb84f381f38c8bf22099d <https://github.com/cloudfoundry/cf-release/commit/549e5a8271bbf0d30efdb84f381f38c8bf22099d>) for more details.

Best,
Amit

Join cf-dev@lists.cloudfoundry.org to automatically receive all group messages.