Re: Ubuntu Xenial stemcell and rootfs plans
On Thu, May 12, 2016 at 12:52 PM, Danny Rosen <drosen(a)pivotal.io> wrote:
Fair enough. Though if you divorce the binary from the buildpack youI respectfully disagree. Build pack binaries are nothing more than
packages (just a different format than rpm or deb). Any competent Linux
administrator should be familiar with package management for their distro
of choice, and that includes the ability to have an internal repo (i.e.
mirror) for the distro's packages. Essentially the idea here would be to
do the same thing but for build pack binaries. Because having internal
repos / mirrors is something a lot of large companies do, I suspect many
administrators will be familiar with these concepts.
I think this change could actually simplify build packs. Currently most of
the build packs have ugly hacks in them to intercept requests for external
files, translate URLs and load local copies of files instead . The idea
that I'm proposing would simply require the build packs to pull files from
a repo. It doesn't matter if you are behind a firewall or on the public
Internet. You just download the files from a configured repository.
Simple and straightforward.
I think we can simplify this conversation a bit though using our *current*Again, I disagree. I don't think you can address them using the current
architecture because it's the architecture that's the problem. Bundling
binaries with the build packs is at it's core a bad idea. Mike D listed
some of these earlier in this email thread. Summarizing the ones that come
to mind below.
- large build packs are hard to distribute
- large build packs increase staging time and in some cases cause staging
- build packs are tied to the stack of the binaries they include
- build packs are tied to specific versions of the binaries they include
- supporting multiple sets of binaries requires multiple build packs or
really large build packs
- customizing build packs becomes more difficult as you now have to wrap
up the binaries in your custom build pack
- build packs are forced to release to keep up with their binaries, not
because the build packs need to change at that pace
Separating the binaries and build packs would seem to address these
issues. It would also turn binary management into a task that is more
similar to what Linux admins do today for their distro's packages. Perhaps
we could even piggy back on existing tools in this space to manage the
binaries like Artifactory.
bundling binaries into the build pack. The original issue on this thread
is being able to support the addition of a new stack. Mike D made the
point that supporting an additional stack would be difficult because it
would cause the size of the build pack to spike. He offered one possible
solution, but that looked like it would require work to the cloud
controller. I offered the idea of splitting the binaries out of the build
pack. It doesn't require any cloud controller work and it would scale
nicely as additional stacks are added (assuming you have an HTTP server
with a large enough disk).
One idea we're throwing around is being able to use a url containing a zipI think that could be helpful. I remember back to the early days of Diego
when it could pull in a zip archive and it was nice in certain situations.
Having said that, I'm not seeing how this would help with the other issues
caused by having build packs and binaries bundled together. In particular,
the one of supporting multiple stacks.
If you're interested in working with us on this solution, let's talk! We're
happy to work with the community.