Re: Exposing more meta-data for admin buildpacks


Jack Cai
 

Mike, thanks for sharing your thoughts.

I agree that it's also valuable to provide the runtime information for a
pushed application.

For the use case of providing supported runtime by a buildpack, letting the
user look at the doc to find out involves several steps:
1. do "cf buildpacks" to find out what version of the buildpack is
installed in the cloud environment;
2. do a google search for that buildpack version
3. click the best hit, browse through the web page and find the information

As opposed to one command: cf buildpack <xyz-buildpack>. So to me obviously
the command makes things so much easier.

The other potential use case is for a dev tool to query that information
programmatically and then set up local dev environment accordingly. For
example, the Eclipse tool can query the default JRE version used by the
Java buildpack, and then use that JRE level to compile a local Java project.

Jack

On Wed, Dec 16, 2015 at 4:37 PM, Mike Dalessio <mdalessio(a)pivotal.io> wrote:

Hi, Jack! Thanks for bringing this up.

I agree we could and should be thinking about how to provide more metadata
about the buildpacks. Some thoughts inline below.


On Mon, Dec 14, 2015 at 5:29 PM, Jack Cai <greensight(a)gmail.com> wrote:

Now that most buildpacks only support a small set of runtime versions (as
defined in their manifest), it makes good sense to make this information
easily accessible by users. They can of course go to the github project and
open that manifest file, or read the doc, but I think the better way might
be making that information accessible through API/CLI. We already have a
set of API/CLI-commands for managing admin buildpacks. It seems pretty
straightforward to add that support, e.g., allowing users to issue:

cf buildpack xyz_buildpack

which returns some descriptive information on supported runtime versions
(and the default) in addition to existing information like index, enabled,
etc. These descriptive information can be registered when
"create-buildpack" is issued, either provided explicitly, or reading from a
certain file in the buildpack package.
The information you're asking for, e.g. runtime versions, is pretty easily
findable and readable from the release notes (e.g.,
https://github.com/cloudfoundry/ruby-buildpack/releases/tag/v1.6.10). Is
there a particular reason that you'd like to be able to access this
information dynamically?

I can imagine a situation where a developer starts an app and wonders
"What version of {ruby,php,python,node} should I use to make sure my app
runs on my CF?"

We could address this situation by making the manifest information more
easily available; but I think we're most of the way down an alternative
path that addresses this situation in a way that Just Works:

Ideally, there won't be substantive functional differences between two
patches on a major/minor branch (e.g., ruby 2.2.2 and 2.2.3 behave the same
way, modulo bug fixes). I'd much prefer if we allowed developers to say,
"Just use the most recent Ruby 2.2.x version you have".

Currently developers can do this with some of the buildpacks, notably:

* php-buildpack supports setting PHP_VERSION to variables like
PHP_56_LATEST which are evaluated at staging time based on the buildpack's
manifest.
* nodejs-buildpack supports npm's version ranges, such as "~>0.12.0" or
"4.2.*"

We have a story in motion to add this support to the Ruby buildpack
<https://www.pivotaltracker.com/story/show/106008092>, which notably
required us to send a PR into Bundler
<https://github.com/bundler/bundler/pull/4064> to support version
operators in Gemfiles. This PR was just accepted, and so the feature is
unblocked.

We have another story in motion to add this support to the Go buildpack
<https://www.pivotaltracker.com/story/show/106117500>, which notably
required us to send a PR into Godep
<https://github.com/tools/godep/pull/318> to support version operators in
Godeps.json files. This PR hasn't been accepted yet, as the conversation is
still ongoing over the best way to solve this issue.

We don't have any plans to add support for this to the Python buildpack,
as `pip` doesn't support version operators, and we haven't heard anyone
request it to justify the effort to submit the necessary changes to `pip`.



Does anybody like this idea? Can we add it to the backlog?
I have a hunch that it would be higher-value to provide buildpack
information about how a specific app was staged, but would like to validate
that hypothesis.

Many older and forked and non-core buildpacks don't emit identifying
information from `bin/detect` and so it's impossible to tell what version
of the buildpack was used once the app has been staged. Additionally, if I
push with the `-b` option, even with an up-to-date buildpack, right now I
still have no version information available.

How can I tell if I need to restage my app to address a vulnerability in
one of the vendored binaries? Today, you can't.

It's possible that we could simply store the buildpack manifest contents,
if available, as a record of what staged the application. I haven't thought
deeply about what the simplest thing might be here, but would love to
continue the conversation and bounce some ideas off you!




Jack

Join cf-dev@lists.cloudfoundry.org to automatically receive all group messages.