Re: [buildpacks] CF-built binaries: release timeline and beta testing

Shawn Nielsen


Thanks for the quick reply.

My question is more around using an 'online' buildpack vs. 'offline'.
When I say 'online', I'm referring to the binaries being pulled from the s3
repo at staging time.

If we use the offline buildpack binaries, we are restricted to node
versions explicitly defined in the manifest files. This creates some
issues when projects want to do validation testing between one version and
the next or need to roll back for whatever reason. It also requires us to
manually update the buildpack every time there is a new release. We've
historically used the online buildpacks to prevent these types of issues,
but we'd like to better understand Pivotal's long term strategy here.

Going forward, what is Pivotal's strategy for online binaries that would be
used at staging time?


On Mon, Nov 9, 2015 at 4:03 PM, Mike Dalessio <mdalessio(a)> wrote:

Hi Shawn,

Thanks for asking.

The actual URL used is going to depend on what version of the buildpack
you're using. Currently master (v1.5.1) is only referncing CF-generated

We removed references to in February, for v1.2.0 of
the buildpack.

Let me know if you need more context around this. Thanks!


On Mon, Nov 9, 2015 at 4:40 PM, Shawn Nielsen <sknielse(a)> wrote:


It looks like the online NodeJS buildpack is currently pointing to
Heroku's repo:

Whereas the offline NodeJS buildpack manifest files seems to be pointing
to pivotal's repo:

Is there a reason for this discrepency?

I would think the online NodeJS buildpack should be pointing to the same
pivotal repo as the offline:

Let me know if you have any input here. Thanks,


On Fri, Jul 17, 2015 at 11:46 PM, James Bayer <jbayer(a)> wrote:

mike d is the best to answer, but i'll take a crack at it

On Fri, Jul 17, 2015 at 3:24 PM, Shawn Nielsen <sknielse(a)>

Two questions on these cf-built binaries:

1. From my understanding, the purpose of this is to compile the
binaries so they are optimized for the cf specific stacks (e.g.
cflinuxfs2 ) as opposed to something that was generically compiled
(e.g. to 64 bit trusty). Can you confirm this or expound upon this if
there are other reasons?
for some buildpacks, we have been relying on heroku binaries which is an
external dependency. we want the cf team to be in complete control of how
and when the binaries are built. this ensures cf can be in control of our
own destiny when for patching security issues or bugs. additionally it
means cf can take responsibility for how the binaries are compiled to
increase trust of the binary contents.

2. Are these binaries available in offline mode only or is there also
intent that they will be hosted, allowing us to consume them in an online
most/all cf buildpacks are available in online and offline mode as i
understand it. see:



On Mon, Jul 13, 2015 at 2:08 PM, Mike Dalessio <mdalessio(a)>

On Mon, Jul 13, 2015 at 1:08 PM, Daniel Mikusa <dmikusa(a)>

On Wed, Jul 8, 2015 at 1:55 PM, Mike Dalessio <mdalessio(a)>

Hi all,

In the June CAB call, it was mentioned that the Buildpacks team was
working on generating CF-specific binaries to be packaged in the
buildpacks. I'm pleased to announce that the team is ready to start
shipping these binaries in the golang, nodejs, php, python, and ruby
buildpacks for the cflinuxfs2 stack.

*__We're planning to cut releases of these buildpacks with the CF*
*binaries on Monday, 20 July.__*

In the meantime, I'd like to ask the community to beta these
buildpacks and give us feedback.

We're successfully running [BRATs][1] (the buildpack runtime
acceptance tests) on these binaries, so we don't expect any issues; but
we'd love to hear if anyone experiences any issues deploying their apps.

Obviously, until we cut official releases, you should use judgement
when deciding where to use these beta buildpacks.


*## Timeline, Versioning and Proposed Changes*

Unless we hear of any blocking issues, we'll cut official releases
of the go, node, php, python and ruby buildpacks on July 20th.

When we cut these releases, we'll be bumping the major version
number, and removing the `manifest-including-unsupported.yml` file from the
repository HEAD. I'd love to hear anyone's opinion on these changes as well.

*## How to deploy with the "beta" binary buildpacks*

Until we cut official releases, we are maintaining a git branch in
each buildpack repository, named `cf-binary-beta`, in which the manifest
points at our CF-specific binaries.

If your CF deployment has access to the public internet, you can
push your app and specify the github repo and branch with the `-b` option.


`cf push <appname> -b`


`cf push <appname> -b`


`cf push <appname> -b`
Looks good mostly. One minor issue. It seems like the snmp
extension is not loading correctly. Looks to be missing a required shared

2015-07-13T12:29:53.09-0400 [App/0] OUT 16:29:53 php-fpm |
[13-Jul-2015 16:29:53] NOTICE: PHP message: PHP Warning: PHP Startup:
Unable to load dynamic library
'/home/vcap/app/php/lib/php/extensions/no-debug-non-zts-20100525/' - cannot open shared object file: No such file or directory
in Unknown on line 0

That's with PHP 5.4.42, 5.5.26 and 5.6.10.
Good catch, I've created a story for this:

One other thing, which is not really an issue, but perhaps an
optimization. I noticed for the PHP extensions the bundle has both ".a"
and ".so" files for many of the extensions. The ".a" static libraries
should not be necessary, just the ".so" shared libraries. Seems like
removing them could save 8 to 9M. You could save another 25M by deleting
the bin/php-cgi file. You really only need bin/php for cmd line stuff and
sbin/php-fpm for web apps. Can't think of any reason you'd need / want to
do cgi. It's not a ton of space, 35M X 3 (each current version) + 35M X 3
(each of previous versions) - whatever compression will save. Just thought
I'd throw it out there though.

Another good catch, I've created a story to look into it:



`cf push <appname> -b`


`cf push <appname> -b`

If your CF deployment doesn't have access to the public internet and
you'd like to try these buildpacks, please reach out directly and we'll
figure out the best way to accommodate.

*## Tooling Details*

If you'd like to take a look at how we're currently building these
binaries, our tooling is open-sourced at:
Note that `binary-builder` uses the rootfs docker image to compile
each binary, which means that this tool can easily be extended to
essentially cross-compile **any** binary for a CF rootfs. We'd love
to hear your feedback on this, as well.


cf-dev mailing list

cf-dev mailing list

cf-dev mailing list

cf-dev mailing list

Thank you,

James Bayer

cf-dev mailing list

Join to automatically receive all group messages.