Re: [buildpacks] CF-built binaries: release timeline and beta testing


Shawn Nielsen
 

James,

It looks like the online NodeJS buildpack is currently pointing to Heroku's
repo:
http://s3pository.heroku.com/node/...

Whereas the offline NodeJS buildpack manifest files seems to be pointing to
pivotal's repo:
https://pivotal-buildpacks.s3.amazonaws.com/concourse-binaries/node/...

Is there a reason for this discrepency?

I would think the online NodeJS buildpack should be pointing to the same
pivotal repo as the offline:
https://pivotal-buildpacks.s3.amazonaws.com/concourse-binaries/node/...


Let me know if you have any input here. Thanks,

-Shawn

On Fri, Jul 17, 2015 at 11:46 PM, James Bayer <jbayer(a)pivotal.io> wrote:

mike d is the best to answer, but i'll take a crack at it

On Fri, Jul 17, 2015 at 3:24 PM, Shawn Nielsen <sknielse(a)gmail.com> wrote:

Two questions on these cf-built binaries:

1. From my understanding, the purpose of this is to compile the binaries
so they are optimized for the cf specific stacks (e.g. cflinuxfs2 ) as
opposed to something that was generically compiled (e.g. to 64 bit
trusty). Can you confirm this or expound upon this if there are other
reasons?
for some buildpacks, we have been relying on heroku binaries which is an
external dependency. we want the cf team to be in complete control of how
and when the binaries are built. this ensures cf can be in control of our
own destiny when for patching security issues or bugs. additionally it
means cf can take responsibility for how the binaries are compiled to
increase trust of the binary contents.



2. Are these binaries available in offline mode only or is there also
intent that they will be hosted, allowing us to consume them in an online
mode?
most/all cf buildpacks are available in online and offline mode as i
understand it. see:
https://github.com/cloudfoundry-incubator/buildpack-packager/blob/master/doc/disconnected_environments.md



Thanks,

-Shawn

On Mon, Jul 13, 2015 at 2:08 PM, Mike Dalessio <mdalessio(a)pivotal.io>
wrote:



On Mon, Jul 13, 2015 at 1:08 PM, Daniel Mikusa <dmikusa(a)pivotal.io>
wrote:

On Wed, Jul 8, 2015 at 1:55 PM, Mike Dalessio <mdalessio(a)pivotal.io>
wrote:

Hi all,

In the June CAB call, it was mentioned that the Buildpacks team was
working on generating CF-specific binaries to be packaged in the
buildpacks. I'm pleased to announce that the team is ready to start
shipping these binaries in the golang, nodejs, php, python, and ruby
buildpacks for the cflinuxfs2 stack.

*__We're planning to cut releases of these buildpacks with the CF*
*binaries on Monday, 20 July.__*

In the meantime, I'd like to ask the community to beta these
buildpacks and give us feedback.

We're successfully running [BRATs][1] (the buildpack runtime
acceptance tests) on these binaries, so we don't expect any issues; but
we'd love to hear if anyone experiences any issues deploying their apps.

Obviously, until we cut official releases, you should use judgement
when deciding where to use these beta buildpacks.

[1]: https://github.com/cloudfoundry/brats


*## Timeline, Versioning and Proposed Changes*

Unless we hear of any blocking issues, we'll cut official releases of
the go, node, php, python and ruby buildpacks on July 20th.

When we cut these releases, we'll be bumping the major version number,
and removing the `manifest-including-unsupported.yml` file from the
repository HEAD. I'd love to hear anyone's opinion on these changes as well.


*## How to deploy with the "beta" binary buildpacks*

Until we cut official releases, we are maintaining a git branch in
each buildpack repository, named `cf-binary-beta`, in which the manifest
points at our CF-specific binaries.

If your CF deployment has access to the public internet, you can push
your app and specify the github repo and branch with the `-b` option.

*__Go:__*

`cf push <appname> -b
https://github.com/cloudfoundry/go-buildpack#cf-binary-beta`
<https://github.com/cloudfoundry/go-buildpack#cf-binary-beta>

*__Node:__*

`cf push <appname> -b
https://github.com/cloudfoundry/nodejs-buildpack#cf-binary-beta`
<https://github.com/cloudfoundry/nodejs-buildpack#cf-binary-beta>

*__PHP:__*

`cf push <appname> -b
https://github.com/cloudfoundry/php-buildpack#cf-binary-beta`
<https://github.com/cloudfoundry/php-buildpack#cf-binary-beta>
Looks good mostly. One minor issue. It seems like the snmp extension
is not loading correctly. Looks to be missing a required shared library.

2015-07-13T12:29:53.09-0400 [App/0] OUT 16:29:53 php-fpm |
[13-Jul-2015 16:29:53] NOTICE: PHP message: PHP Warning: PHP Startup:
Unable to load dynamic library
'/home/vcap/app/php/lib/php/extensions/no-debug-non-zts-20100525/snmp.so' -
libnetsnmp.so.30: cannot open shared object file: No such file or directory
in Unknown on line 0

That's with PHP 5.4.42, 5.5.26 and 5.6.10.
Good catch, I've created a story for this:
https://www.pivotaltracker.com/story/show/98980384





One other thing, which is not really an issue, but perhaps an
optimization. I noticed for the PHP extensions the bundle has both ".a"
and ".so" files for many of the extensions. The ".a" static libraries
should not be necessary, just the ".so" shared libraries. Seems like
removing them could save 8 to 9M. You could save another 25M by deleting
the bin/php-cgi file. You really only need bin/php for cmd line stuff and
sbin/php-fpm for web apps. Can't think of any reason you'd need / want to
do cgi. It's not a ton of space, 35M X 3 (each current version) + 35M X 3
(each of previous versions) - whatever compression will save. Just thought
I'd throw it out there though.

Another good catch, I've created a story to look into it:
https://www.pivotaltracker.com/story/show/98980692




Dan




*__Python:__*

`cf push <appname> -b
https://github.com/cloudfoundry/python-buildpack#cf-binary-beta`
<https://github.com/cloudfoundry/python-buildpack#cf-binary-beta>

*__Ruby:__*

`cf push <appname> -b
https://github.com/cloudfoundry/ruby-buildpack#cf-binary-beta`
<https://github.com/cloudfoundry/ruby-buildpack#cf-binary-beta>


If your CF deployment doesn't have access to the public internet and
you'd like to try these buildpacks, please reach out directly and we'll
figure out the best way to accommodate.


*## Tooling Details*

If you'd like to take a look at how we're currently building these
binaries, our tooling is open-sourced at:

https://github.com/cloudfoundry/binary-builder
Note that `binary-builder` uses the rootfs docker image to compile
each binary, which means that this tool can easily be extended to
essentially cross-compile **any** binary for a CF rootfs. We'd love
to hear your feedback on this, as well.


Thanks,
-mike



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Thank you,

James Bayer

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

Join cf-dev@lists.cloudfoundry.org to automatically receive all group messages.