Request for Multibuildpack Use Cases


Danny Rosen
 

Hi there,

The CF Buildpacks team is considering taking on a line of work to provide
more formal support for multibuildpacks. Before we start, we would be
interested in learning if any community users have compelling use cases
they could share with us.

For more information on multibuildpacks, see Heroku's documentation [1]

[1] -
https://devcenter.heroku.com/articles/using-multiple-buildpacks-for-an-app


Mike Youngstrom
 

This seems to be yet another way to extend buildpacks with out forking to
go along with [0] and [1]. My only hope is that all these newly proposed
extension mechanisms come together in a simple, coherent, and extensible
way.

Mike

[0]
https://github.com/cloudfoundry-incubator/buildpack_app_lifecycle/pull/13
[1]
https://docs.google.com/document/d/145aOpNoq7BpuB3VOzUIDh-HBx0l3v4NHLYfW8xt2zK0/edit#

On Sun, Apr 10, 2016 at 6:15 PM, Danny Rosen <drosen(a)pivotal.io> wrote:

Hi there,

The CF Buildpacks team is considering taking on a line of work to provide
more formal support for multibuildpacks. Before we start, we would be
interested in learning if any community users have compelling use cases
they could share with us.

For more information on multibuildpacks, see Heroku's documentation [1]

[1] -
https://devcenter.heroku.com/articles/using-multiple-buildpacks-for-an-app


JT Archie <jarchie@...>
 

Mike, this is so buildpacks can be composed together.

For example, if a user want to use NodeJS to compile Javascript assets for
a Java app (yes, there always Maven support, but some have different work
flows).

Multi-buildpack doesn't alleviate forking of buildpacks. In my opinion, the
above proposals for extensions to the buildpack app life cycle
are orthogonal.

On Mon, Apr 11, 2016 at 10:44 AM, Mike Youngstrom <youngm(a)gmail.com> wrote:

This seems to be yet another way to extend buildpacks with out forking to
go along with [0] and [1]. My only hope is that all these newly proposed
extension mechanisms come together in a simple, coherent, and extensible
way.

Mike

[0]
https://github.com/cloudfoundry-incubator/buildpack_app_lifecycle/pull/13
[1]
https://docs.google.com/document/d/145aOpNoq7BpuB3VOzUIDh-HBx0l3v4NHLYfW8xt2zK0/edit#



On Sun, Apr 10, 2016 at 6:15 PM, Danny Rosen <drosen(a)pivotal.io> wrote:

Hi there,

The CF Buildpacks team is considering taking on a line of work to provide
more formal support for multibuildpacks. Before we start, we would be
interested in learning if any community users have compelling use cases
they could share with us.

For more information on multibuildpacks, see Heroku's documentation [1]

[1] -
https://devcenter.heroku.com/articles/using-multiple-buildpacks-for-an-app


Mike Youngstrom
 

Although, correct me if I'm wrong, this feature could be used as a solution
for some of those cases. For example, if my application depends upon an
oracle-service my node application will require the oracle binary libraries
to be configured in my environment before the nodejs buildpack runs. If I
create an "oracle-library" buildpack that installs the oracle libraries in
my staging container before npm install then this is potentially another
way to cover some of the use cases of the other extension points right?

Mike

On Mon, Apr 11, 2016 at 3:38 PM, JT Archie <jarchie(a)pivotal.io> wrote:

Mike, this is so buildpacks can be composed together.

For example, if a user want to use NodeJS to compile Javascript assets for
a Java app (yes, there always Maven support, but some have different work
flows).

Multi-buildpack doesn't alleviate forking of buildpacks. In my opinion,
the above proposals for extensions to the buildpack app life cycle
are orthogonal.


On Mon, Apr 11, 2016 at 10:44 AM, Mike Youngstrom <youngm(a)gmail.com>
wrote:

This seems to be yet another way to extend buildpacks with out forking to
go along with [0] and [1]. My only hope is that all these newly proposed
extension mechanisms come together in a simple, coherent, and extensible
way.

Mike

[0]
https://github.com/cloudfoundry-incubator/buildpack_app_lifecycle/pull/13
[1]
https://docs.google.com/document/d/145aOpNoq7BpuB3VOzUIDh-HBx0l3v4NHLYfW8xt2zK0/edit#



On Sun, Apr 10, 2016 at 6:15 PM, Danny Rosen <drosen(a)pivotal.io> wrote:

Hi there,

The CF Buildpacks team is considering taking on a line of work to
provide more formal support for multibuildpacks. Before we start, we would
be interested in learning if any community users have compelling use cases
they could share with us.

For more information on multibuildpacks, see Heroku's documentation [1]

[1] -
https://devcenter.heroku.com/articles/using-multiple-buildpacks-for-an-app


Danny Rosen
 

The Cloud Foundry Buildpacks project does not contain an "oracle-library"
buildpack.

For as list of Cloud Foundry buildpacks please see the "System Buildpacks"
section of the official Cloud Foundry documentation [1].

If your use case includes multiple buildpacks from this list, we would be
interested in hearing about it as we recognize that this type of
functionality may be useful to those in the community.

I apologize if that was not clear in my original post.

[1] - http://docs.cloudfoundry.org/buildpacks/

On Apr 11, 2016 6:57 PM, "Mike Youngstrom" <youngm(a)gmail.com> wrote:

Although, correct me if I'm wrong, this feature could be used as a
solution for some of those cases. For example, if my application depends
upon an oracle-service my node application will require the oracle binary
libraries to be configured in my environment before the nodejs buildpack
runs. If I create an "oracle-library" buildpack that installs the oracle
libraries in my staging container before npm install then this is
potentially another way to cover some of the use cases of the other
extension points right?

Mike

On Mon, Apr 11, 2016 at 3:38 PM, JT Archie <jarchie(a)pivotal.io> wrote:

Mike, this is so buildpacks can be composed together.

For example, if a user want to use NodeJS to compile Javascript assets
for a Java app (yes, there always Maven support, but some have different
work flows).

Multi-buildpack doesn't alleviate forking of buildpacks. In my opinion,
the above proposals for extensions to the buildpack app life cycle
are orthogonal.


On Mon, Apr 11, 2016 at 10:44 AM, Mike Youngstrom <youngm(a)gmail.com>
wrote:

This seems to be yet another way to extend buildpacks with out forking
to go along with [0] and [1]. My only hope is that all these newly
proposed extension mechanisms come together in a simple, coherent, and
extensible way.

Mike

[0]
https://github.com/cloudfoundry-incubator/buildpack_app_lifecycle/pull/13
[1]
https://docs.google.com/document/d/145aOpNoq7BpuB3VOzUIDh-HBx0l3v4NHLYfW8xt2zK0/edit#



On Sun, Apr 10, 2016 at 6:15 PM, Danny Rosen <drosen(a)pivotal.io> wrote:

Hi there,

The CF Buildpacks team is considering taking on a line of work to
provide more formal support for multibuildpacks. Before we start, we would
be interested in learning if any community users have compelling use cases
they could share with us.

For more information on multibuildpacks, see Heroku's documentation [1]

[1] -
https://devcenter.heroku.com/articles/using-multiple-buildpacks-for-an-app


Mike Youngstrom
 

Ah, I think that helps a lot. So to restate this would be a feature of the
buildpacks themselves and the current buildpack contract (perhaps
triggering off of detect or something) not a generic feature of the cloud
controller feature would allows you to apply and order multiple buildpacks
against a single app, correct?

Thanks,
Mike

On Mon, Apr 11, 2016 at 8:12 PM, Danny Rosen <danny.rosen(a)gmail.com> wrote:

The Cloud Foundry Buildpacks project does not contain an "oracle-library"
buildpack.

For as list of Cloud Foundry buildpacks please see the "System Buildpacks"
section of the official Cloud Foundry documentation [1].

If your use case includes multiple buildpacks from this list, we would be
interested in hearing about it as we recognize that this type of
functionality may be useful to those in the community.

I apologize if that was not clear in my original post.

[1] - http://docs.cloudfoundry.org/buildpacks/
On Apr 11, 2016 6:57 PM, "Mike Youngstrom" <youngm(a)gmail.com> wrote:

Although, correct me if I'm wrong, this feature could be used as a
solution for some of those cases. For example, if my application depends
upon an oracle-service my node application will require the oracle binary
libraries to be configured in my environment before the nodejs buildpack
runs. If I create an "oracle-library" buildpack that installs the oracle
libraries in my staging container before npm install then this is
potentially another way to cover some of the use cases of the other
extension points right?

Mike

On Mon, Apr 11, 2016 at 3:38 PM, JT Archie <jarchie(a)pivotal.io> wrote:

Mike, this is so buildpacks can be composed together.

For example, if a user want to use NodeJS to compile Javascript assets
for a Java app (yes, there always Maven support, but some have different
work flows).

Multi-buildpack doesn't alleviate forking of buildpacks. In my opinion,
the above proposals for extensions to the buildpack app life cycle
are orthogonal.


On Mon, Apr 11, 2016 at 10:44 AM, Mike Youngstrom <youngm(a)gmail.com>
wrote:

This seems to be yet another way to extend buildpacks with out forking
to go along with [0] and [1]. My only hope is that all these newly
proposed extension mechanisms come together in a simple, coherent, and
extensible way.

Mike

[0]
https://github.com/cloudfoundry-incubator/buildpack_app_lifecycle/pull/13
[1]
https://docs.google.com/document/d/145aOpNoq7BpuB3VOzUIDh-HBx0l3v4NHLYfW8xt2zK0/edit#



On Sun, Apr 10, 2016 at 6:15 PM, Danny Rosen <drosen(a)pivotal.io> wrote:

Hi there,

The CF Buildpacks team is considering taking on a line of work to
provide more formal support for multibuildpacks. Before we start, we would
be interested in learning if any community users have compelling use cases
they could share with us.

For more information on multibuildpacks, see Heroku's documentation [1]

[1] -
https://devcenter.heroku.com/articles/using-multiple-buildpacks-for-an-app


David Illsley <davidillsley@...>
 

In the past we've used the multi-buildpack to be able to use ruby sass to
compile SCSS for non-ruby projects (node and Java). In that case we used
the multi-buildpack and a .buildpacks file which worked reasonably well
(and was very clear).

On Mon, Apr 11, 2016 at 1:15 AM, Danny Rosen <drosen(a)pivotal.io> wrote:

Hi there,

The CF Buildpacks team is considering taking on a line of work to provide
more formal support for multibuildpacks. Before we start, we would be
interested in learning if any community users have compelling use cases
they could share with us.

For more information on multibuildpacks, see Heroku's documentation [1]

[1] -
https://devcenter.heroku.com/articles/using-multiple-buildpacks-for-an-app


Jack Cai
 

It would be more useful if the multi-buildpack can reference an admin
buildpack in addition to a remote git-hosted buildpack. :-)

Jack


On Tue, Apr 12, 2016 at 6:38 AM, David Illsley <davidillsley(a)gmail.com>
wrote:

In the past we've used the multi-buildpack to be able to use ruby sass to
compile SCSS for non-ruby projects (node and Java). In that case we used
the multi-buildpack and a .buildpacks file which worked reasonably well
(and was very clear).

On Mon, Apr 11, 2016 at 1:15 AM, Danny Rosen <drosen(a)pivotal.io> wrote:

Hi there,

The CF Buildpacks team is considering taking on a line of work to provide
more formal support for multibuildpacks. Before we start, we would be
interested in learning if any community users have compelling use cases
they could share with us.

For more information on multibuildpacks, see Heroku's documentation [1]

[1] -
https://devcenter.heroku.com/articles/using-multiple-buildpacks-for-an-app


John Feminella <jxf@...>
 

Multibuildpack is absolutely useful and I'm excited for this proposal.

I encounter a lot of use cases for this. The most common is that an
application wants to pull in private dependencies during a future
dependency-resolution step of a later buildpack, but the dependency
resolver needs to be primed in some specific way. If you wait until
buildpack time it's too late.

On Heroku, for example, this is accomplished by having something like the
netrc buildpack (https://github.com/timshadel/heroku-buildpack-github-netrc),
adding a GITHUB_TOKEN environment variable, and then running your "real"
buildpack. The netrc BP runs first, allowing Bundler to see the private
dependencies.

best,
~ jf

On Tue, Apr 12, 2016 at 12:36 PM Jack Cai <greensight(a)gmail.com> wrote:

It would be more useful if the multi-buildpack can reference an admin
buildpack in addition to a remote git-hosted buildpack. :-)

Jack


On Tue, Apr 12, 2016 at 6:38 AM, David Illsley <davidillsley(a)gmail.com>
wrote:

In the past we've used the multi-buildpack to be able to use ruby sass to
compile SCSS for non-ruby projects (node and Java). In that case we used
the multi-buildpack and a .buildpacks file which worked reasonably well
(and was very clear).

On Mon, Apr 11, 2016 at 1:15 AM, Danny Rosen <drosen(a)pivotal.io> wrote:

Hi there,

The CF Buildpacks team is considering taking on a line of work to
provide more formal support for multibuildpacks. Before we start, we would
be interested in learning if any community users have compelling use cases
they could share with us.

For more information on multibuildpacks, see Heroku's documentation [1]

[1] -
https://devcenter.heroku.com/articles/using-multiple-buildpacks-for-an-app


John Feminella <jxf@...>
 

Another common use case for me is that an application is polyglot in some
way. For example, you have a blog you'd like to deploy that uses a static
site generator written in (say) Go. You would have the Go buildpack run to
bootstrap and generate the resulting files in some output directory, and
then the staticfile buildpack runs and serves the files from the output
directory.

best,
~ jf

On Tue, Apr 12, 2016 at 12:45 PM John Feminella <jxf(a)pivotal.io> wrote:

Multibuildpack is absolutely useful and I'm excited for this proposal.

I encounter a lot of use cases for this. The most common is that an
application wants to pull in private dependencies during a future
dependency-resolution step of a later buildpack, but the dependency
resolver needs to be primed in some specific way. If you wait until
buildpack time it's too late.

On Heroku, for example, this is accomplished by having something like the
netrc buildpack (
https://github.com/timshadel/heroku-buildpack-github-netrc), adding a
GITHUB_TOKEN environment variable, and then running your "real" buildpack.
The netrc BP runs first, allowing Bundler to see the private dependencies.

best,
~ jf

On Tue, Apr 12, 2016 at 12:36 PM Jack Cai <greensight(a)gmail.com> wrote:

It would be more useful if the multi-buildpack can reference an admin
buildpack in addition to a remote git-hosted buildpack. :-)

Jack


On Tue, Apr 12, 2016 at 6:38 AM, David Illsley <davidillsley(a)gmail.com>
wrote:

In the past we've used the multi-buildpack to be able to use ruby sass
to compile SCSS for non-ruby projects (node and Java). In that case we used
the multi-buildpack and a .buildpacks file which worked reasonably well
(and was very clear).

On Mon, Apr 11, 2016 at 1:15 AM, Danny Rosen <drosen(a)pivotal.io> wrote:

Hi there,

The CF Buildpacks team is considering taking on a line of work to
provide more formal support for multibuildpacks. Before we start, we would
be interested in learning if any community users have compelling use cases
they could share with us.

For more information on multibuildpacks, see Heroku's documentation [1]

[1] -
https://devcenter.heroku.com/articles/using-multiple-buildpacks-for-an-app


Mike Youngstrom
 

John,

It sounds like the buildpack team is thinking the multi buildpack feature
would only work for buildpacks they provide not a custom
"dependency-resolution" buildpack. Or at least that is how I understood
the message from Danny Rosen earlier in the thread.

Mike

On Tue, Apr 12, 2016 at 10:45 AM, John Feminella <jxf(a)pivotal.io> wrote:

Multibuildpack is absolutely useful and I'm excited for this proposal.

I encounter a lot of use cases for this. The most common is that an
application wants to pull in private dependencies during a future
dependency-resolution step of a later buildpack, but the dependency
resolver needs to be primed in some specific way. If you wait until
buildpack time it's too late.

On Heroku, for example, this is accomplished by having something like the
netrc buildpack (
https://github.com/timshadel/heroku-buildpack-github-netrc), adding a
GITHUB_TOKEN environment variable, and then running your "real" buildpack.
The netrc BP runs first, allowing Bundler to see the private dependencies.

best,
~ jf

On Tue, Apr 12, 2016 at 12:36 PM Jack Cai <greensight(a)gmail.com> wrote:

It would be more useful if the multi-buildpack can reference an admin
buildpack in addition to a remote git-hosted buildpack. :-)

Jack


On Tue, Apr 12, 2016 at 6:38 AM, David Illsley <davidillsley(a)gmail.com>
wrote:

In the past we've used the multi-buildpack to be able to use ruby sass
to compile SCSS for non-ruby projects (node and Java). In that case we used
the multi-buildpack and a .buildpacks file which worked reasonably well
(and was very clear).

On Mon, Apr 11, 2016 at 1:15 AM, Danny Rosen <drosen(a)pivotal.io> wrote:

Hi there,

The CF Buildpacks team is considering taking on a line of work to
provide more formal support for multibuildpacks. Before we start, we would
be interested in learning if any community users have compelling use cases
they could share with us.

For more information on multibuildpacks, see Heroku's documentation [1]

[1] -
https://devcenter.heroku.com/articles/using-multiple-buildpacks-for-an-app


Danny Rosen
 

As a starting point we have decided to work with officially provided
buildpacks as their behavior is known and controlled by the buildpacks
team. By discovering use cases (thank you John, Jack and David for your
examples) we can start work towards implementing multibuildpack solutions
that would be open to the community to consume and iterate on.

On Apr 12, 2016 1:06 PM, "Mike Youngstrom" <youngm(a)gmail.com> wrote:

John,

It sounds like the buildpack team is thinking the multi buildpack feature
would only work for buildpacks they provide not a custom
"dependency-resolution" buildpack. Or at least that is how I understood
the message from Danny Rosen earlier in the thread.

Mike

On Tue, Apr 12, 2016 at 10:45 AM, John Feminella <jxf(a)pivotal.io> wrote:

Multibuildpack is absolutely useful and I'm excited for this proposal.

I encounter a lot of use cases for this. The most common is that an
application wants to pull in private dependencies during a future
dependency-resolution step of a later buildpack, but the dependency
resolver needs to be primed in some specific way. If you wait until
buildpack time it's too late.

On Heroku, for example, this is accomplished by having something like the
netrc buildpack (
https://github.com/timshadel/heroku-buildpack-github-netrc), adding a
GITHUB_TOKEN environment variable, and then running your "real" buildpack.
The netrc BP runs first, allowing Bundler to see the private dependencies.

best,
~ jf

On Tue, Apr 12, 2016 at 12:36 PM Jack Cai <greensight(a)gmail.com> wrote:

It would be more useful if the multi-buildpack can reference an admin
buildpack in addition to a remote git-hosted buildpack. :-)

Jack


On Tue, Apr 12, 2016 at 6:38 AM, David Illsley <davidillsley(a)gmail.com>
wrote:

In the past we've used the multi-buildpack to be able to use ruby sass
to compile SCSS for non-ruby projects (node and Java). In that case we used
the multi-buildpack and a .buildpacks file which worked reasonably well
(and was very clear).

On Mon, Apr 11, 2016 at 1:15 AM, Danny Rosen <drosen(a)pivotal.io> wrote:

Hi there,

The CF Buildpacks team is considering taking on a line of work to
provide more formal support for multibuildpacks. Before we start, we would
be interested in learning if any community users have compelling use cases
they could share with us.

For more information on multibuildpacks, see Heroku's documentation [1]

[1] -
https://devcenter.heroku.com/articles/using-multiple-buildpacks-for-an-app


Mike Youngstrom
 

Thanks for the clarification Danny. I guess the point I was trying to make
earlier with the "oracle_library" buildpack use case, is that this feature
has some potential functional commonality with the proposed binary service
broker feature and the pre-post hook proposed features. So, my hope is
that all these features can be considered together to help ensure a coherent
solution across this entire buildpack extensiblity space that is being
explored right now.

Thanks,
Mike

On Tue, Apr 12, 2016 at 4:16 PM, Danny Rosen <drosen(a)pivotal.io> wrote:

As a starting point we have decided to work with officially provided
buildpacks as their behavior is known and controlled by the buildpacks
team. By discovering use cases (thank you John, Jack and David for your
examples) we can start work towards implementing multibuildpack solutions
that would be open to the community to consume and iterate on.
On Apr 12, 2016 1:06 PM, "Mike Youngstrom" <youngm(a)gmail.com> wrote:

John,

It sounds like the buildpack team is thinking the multi buildpack feature
would only work for buildpacks they provide not a custom
"dependency-resolution" buildpack. Or at least that is how I understood
the message from Danny Rosen earlier in the thread.

Mike

On Tue, Apr 12, 2016 at 10:45 AM, John Feminella <jxf(a)pivotal.io> wrote:

Multibuildpack is absolutely useful and I'm excited for this proposal.

I encounter a lot of use cases for this. The most common is that an
application wants to pull in private dependencies during a future
dependency-resolution step of a later buildpack, but the dependency
resolver needs to be primed in some specific way. If you wait until
buildpack time it's too late.

On Heroku, for example, this is accomplished by having something like
the netrc buildpack (
https://github.com/timshadel/heroku-buildpack-github-netrc), adding a
GITHUB_TOKEN environment variable, and then running your "real" buildpack.
The netrc BP runs first, allowing Bundler to see the private dependencies.

best,
~ jf

On Tue, Apr 12, 2016 at 12:36 PM Jack Cai <greensight(a)gmail.com> wrote:

It would be more useful if the multi-buildpack can reference an admin
buildpack in addition to a remote git-hosted buildpack. :-)

Jack


On Tue, Apr 12, 2016 at 6:38 AM, David Illsley <davidillsley(a)gmail.com>
wrote:

In the past we've used the multi-buildpack to be able to use ruby sass
to compile SCSS for non-ruby projects (node and Java). In that case we used
the multi-buildpack and a .buildpacks file which worked reasonably well
(and was very clear).

On Mon, Apr 11, 2016 at 1:15 AM, Danny Rosen <drosen(a)pivotal.io>
wrote:

Hi there,

The CF Buildpacks team is considering taking on a line of work to
provide more formal support for multibuildpacks. Before we start, we would
be interested in learning if any community users have compelling use cases
they could share with us.

For more information on multibuildpacks, see Heroku's documentation
[1]

[1] -
https://devcenter.heroku.com/articles/using-multiple-buildpacks-for-an-app


Troy Topnik
 

We used heroku-buildpack-multi to deploy OpenProject (NodeJS and Ruby).

https://github.com/Stackato-Apps/openproject

The manifest.yml in that repo has some Stackato-isms, but you get the idea.

Deploying this kind of Ruby+Node app (which I'm starting to see more of) without multibuildpacks would otherwise a customized buildpack:

https://github.com/qnyp/heroku-buildpack-ruby-bower/tree/run-bower

It's easier for a developer to combine existing buildpacks than fork one to add functionality.


Dubois, Jan <jan.dubois@...>
 

The problem right now is that the buildpack names are no longer available during staging. All the code is available (because the stager needs to call `bin/detect` for each buildpack in turn), but it uses a hash as the directory name.

I made a quick experiment that grabs the mapping from buildpack names to filesystem location directly out of the CCDB and stuffs it into a CF_BUILDPACKS environment variable. With that in place, it is a trivial change to extend the multi-buildpack to support admin buildpacks as well. I've put some more information into the README.md:

https://github.com/jandubois/cf-buildpack-multi

I haven't tested this with the legacy DEA; I only checked it with Diego.

I would be happy to work on patches for CC and Stager to implement support for CF_BUILDPACKS if this approach is considered useful by the community, and acceptable to buildpack team.

Please let me know what you think about this approach!

Cheers,
-Jan


From: Jack Cai <greensight(a)gmail.com<mailto:greensight(a)gmail.com>>
Reply-To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>>
Date: Tuesday, April 12, 2016 at 9:36
To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>>
Subject: [cf-dev] Re: Re: Request for Multibuildpack Use Cases

It would be more useful if the multi-buildpack can reference an admin buildpack in addition to a remote git-hosted buildpack. :-)

Jack

On Tue, Apr 12, 2016 at 6:38 AM, David Illsley <davidillsley(a)gmail.com<mailto:davidillsley(a)gmail.com>> wrote:
In the past we've used the multi-buildpack to be able to use ruby sass to compile SCSS for non-ruby projects (node and Java). In that case we used the multi-buildpack and a .buildpacks file which worked reasonably well (and was very clear).

On Mon, Apr 11, 2016 at 1:15 AM, Danny Rosen <drosen(a)pivotal.io<mailto:drosen(a)pivotal.io>> wrote:
Hi there,

The CF Buildpacks team is considering taking on a line of work to provide more formal support for multibuildpacks. Before we start, we would be interested in learning if any community users have compelling use cases they could share with us.

For more information on multibuildpacks, see Heroku's documentation [1]

[1] - https://devcenter.heroku.com/articles/using-multiple-buildpacks-for-an-app


Guillaume Berche
 

Danny,

Some additional related use-cases:

1- our corporate CF platform provider is willing to scan content all apps
pre staging and post-staging. For instance, in order to black list of
known-to-be-vulnerable php library pulled by the production apps, or
perform static code analysis. We're currently considering a php buildpack
fork to insert an extension to do this. Being able to do such scanning
without forking the buildpack would be useful.

This seemed well match https://www.pivotaltracker.com/story/show/100758730

2- add some features that are language-independent to the set of supported
buildpacks without forking them (e.g. an agent collocated with the app, or
a command prexecuted prior to app starting).

A concrete example of this is securely fetching secrets from an hashicorp
vault before app startup and expose them as transient env vars (i.e. env
vars not stored in CC db). More details into
https://github.com/Orange-OpenSource/static-creds-broker/issues/11

As a starting point we have decided to work with officially provided
buildpacks as their behavior is known and controlled by the buildpacks team.

I worry that only supporting a combination of the officially provided
buildpacks will drastically limit the number of use-cases addressed such a
"multi-buildpack" 1st step

Guillaume.



Guillaume.

On Wed, Apr 13, 2016 at 12:16 AM, Danny Rosen <drosen(a)pivotal.io> wrote:

As a starting point we have decided to work with officially provided
buildpacks as their behavior is known and controlled by the buildpacks
team. By discovering use cases (thank you John, Jack and David for your
examples) we can start work towards implementing multibuildpack solutions
that would be open to the community to consume and iterate on.
On Apr 12, 2016 1:06 PM, "Mike Youngstrom" <youngm(a)gmail.com> wrote:

John,

It sounds like the buildpack team is thinking the multi buildpack feature
would only work for buildpacks they provide not a custom
"dependency-resolution" buildpack. Or at least that is how I understood
the message from Danny Rosen earlier in the thread.

Mike

On Tue, Apr 12, 2016 at 10:45 AM, John Feminella <jxf(a)pivotal.io> wrote:

Multibuildpack is absolutely useful and I'm excited for this proposal.

I encounter a lot of use cases for this. The most common is that an
application wants to pull in private dependencies during a future
dependency-resolution step of a later buildpack, but the dependency
resolver needs to be primed in some specific way. If you wait until
buildpack time it's too late.

On Heroku, for example, this is accomplished by having something like
the netrc buildpack (
https://github.com/timshadel/heroku-buildpack-github-netrc), adding a
GITHUB_TOKEN environment variable, and then running your "real" buildpack.
The netrc BP runs first, allowing Bundler to see the private dependencies.

best,
~ jf

On Tue, Apr 12, 2016 at 12:36 PM Jack Cai <greensight(a)gmail.com> wrote:

It would be more useful if the multi-buildpack can reference an admin
buildpack in addition to a remote git-hosted buildpack. :-)

Jack


On Tue, Apr 12, 2016 at 6:38 AM, David Illsley <davidillsley(a)gmail.com>
wrote:

In the past we've used the multi-buildpack to be able to use ruby sass
to compile SCSS for non-ruby projects (node and Java). In that case we used
the multi-buildpack and a .buildpacks file which worked reasonably well
(and was very clear).

On Mon, Apr 11, 2016 at 1:15 AM, Danny Rosen <drosen(a)pivotal.io>
wrote:

Hi there,

The CF Buildpacks team is considering taking on a line of work to
provide more formal support for multibuildpacks. Before we start, we would
be interested in learning if any community users have compelling use cases
they could share with us.

For more information on multibuildpacks, see Heroku's documentation
[1]

[1] -
https://devcenter.heroku.com/articles/using-multiple-buildpacks-for-an-app


Noburou TANIGUCHI
 

Hi Danny,

We (Japan Cloud Foundry Group) tried to deploy six applications with
ddollar's heroku-buildpack-multi (
https://github.com/ddollar/heroku-buildpack-multi (deprecated now)).

Here are our blog posts (sorry that all the posts are in Japanese, but I
think you may see the actual deployment procedures in "pre" sections (in
black background)):

* Yabitz http://blog.cloudfoundry.gr.jp/2015/06/cf100apps-014-yabitz.html
* jqplay http://blog.cloudfoundry.gr.jp/2015/07/jqplay-cloud-foundry.html
* Mattermost
http://blog.cloudfoundry.gr.jp/2015/09/cf100apps-066-mattermost.html
* Wizard Warz
http://blog.cloudfoundry.gr.jp/2015/10/cf100apps-079-wizardwarz.html
* Jekyll http://blog.cloudfoundry.gr.jp/2015/10/cf100apps-087-jekyll.html
* Mconf-Web
http://blog.cloudfoundry.gr.jp/2015/11/cf100apps-098-mconf-web.html

I also have some experience using James Bayer's scm-buildpack with
heroku-buildpack-multi. But it seems not the cases that you require.

Regards,


gberche wrote
Danny,

Some additional related use-cases:

1- our corporate CF platform provider is willing to scan content all apps
pre staging and post-staging. For instance, in order to black list of
known-to-be-vulnerable php library pulled by the production apps, or
perform static code analysis. We're currently considering a php buildpack
fork to insert an extension to do this. Being able to do such scanning
without forking the buildpack would be useful.

This seemed well match https://www.pivotaltracker.com/story/show/100758730

2- add some features that are language-independent to the set of supported
buildpacks without forking them (e.g. an agent collocated with the app, or
a command prexecuted prior to app starting).

A concrete example of this is securely fetching secrets from an hashicorp
vault before app startup and expose them as transient env vars (i.e. env
vars not stored in CC db). More details into
https://github.com/Orange-OpenSource/static-creds-broker/issues/11

As a starting point we have decided to work with officially provided
buildpacks as their behavior is known and controlled by the buildpacks
team.

I worry that only supporting a combination of the officially provided
buildpacks will drastically limit the number of use-cases addressed such a
"multi-buildpack" 1st step

Guillaume.



Guillaume.

On Wed, Apr 13, 2016 at 12:16 AM, Danny Rosen &lt;
drosen@
&gt; wrote:

As a starting point we have decided to work with officially provided
buildpacks as their behavior is known and controlled by the buildpacks
team. By discovering use cases (thank you John, Jack and David for your
examples) we can start work towards implementing multibuildpack solutions
that would be open to the community to consume and iterate on.
On Apr 12, 2016 1:06 PM, "Mike Youngstrom" &lt;
youngm@
&gt; wrote:

John,

It sounds like the buildpack team is thinking the multi buildpack
feature
would only work for buildpacks they provide not a custom
"dependency-resolution" buildpack. Or at least that is how I understood
the message from Danny Rosen earlier in the thread.

Mike

On Tue, Apr 12, 2016 at 10:45 AM, John Feminella &lt;
jxf@
&gt; wrote:

Multibuildpack is absolutely useful and I'm excited for this proposal.

I encounter a lot of use cases for this. The most common is that an
application wants to pull in private dependencies during a future
dependency-resolution step of a later buildpack, but the dependency
resolver needs to be primed in some specific way. If you wait until
buildpack time it's too late.

On Heroku, for example, this is accomplished by having something like
the netrc buildpack (
https://github.com/timshadel/heroku-buildpack-github-netrc), adding a
GITHUB_TOKEN environment variable, and then running your "real"
buildpack.
The netrc BP runs first, allowing Bundler to see the private
dependencies.

best,
~ jf

On Tue, Apr 12, 2016 at 12:36 PM Jack Cai &lt;
greensight@
&gt; wrote:

It would be more useful if the multi-buildpack can reference an admin
buildpack in addition to a remote git-hosted buildpack. :-)

Jack


On Tue, Apr 12, 2016 at 6:38 AM, David Illsley &lt;
davidillsley@
&gt;
wrote:

In the past we've used the multi-buildpack to be able to use ruby
sass
to compile SCSS for non-ruby projects (node and Java). In that case
we used
the multi-buildpack and a .buildpacks file which worked reasonably
well
(and was very clear).

On Mon, Apr 11, 2016 at 1:15 AM, Danny Rosen &lt;
drosen@
&gt;
wrote:

Hi there,

The CF Buildpacks team is considering taking on a line of work to
provide more formal support for multibuildpacks. Before we start, we
would
be interested in learning if any community users have compelling use
cases
they could share with us.

For more information on multibuildpacks, see Heroku's documentation
[1]

[1] -
https://devcenter.heroku.com/articles/using-multiple-buildpacks-for-an-app




-----
I'm not a ...
noburou taniguchi
--
View this message in context: http://cf-dev.70369.x6.nabble.com/cf-dev-Request-for-Multibuildpack-Use-Cases-tp4540p4859.html
Sent from the CF Dev mailing list archive at Nabble.com.