Re: Droplets and Stacks


Guillaume Berche
 

Thanks Onsi for these precisions.

Similar to Colin's blog question, I'm wondering how much versionning is
currently provided to rootfs and buildpacks. In other words, when a rootfs
(say cflinuxfs2) gets patched (e.g. following CVE such as in [1]), what
support is provided in the platform to identify apps that are still running
an old version of cflinuxfs2 rootfs and require restaging ?

Am I right to assume that there will be multiple distinct stack instances
returned by cc api calls such as [3] (with distinct guid but same entity
names) and that stacks are indeed immutable (i.e. the field "updated_at"
will remain null in [4] ) ? Writing a cli plugin to identify apps that need
restaging following a rootfs patch (similar to [2] but for a minor version
of a given stack), would therefore browse all stacks using [5], and order
those with the same name to understand minor patch version ?

I recall similar discussion related to buildpack versionning, where it was
mentionned that the buildpacks were mutable, and the only strategy to
support multiple versions of a buildpack is to append version numbers to
the buildpack names (and rely on priority ordering to have most recent
version be picked first). This has the drawback that an app specifying a
buildpack (e.g. to deambiguate or fix an invalid detect phase) will then
need to manually update the buildpack reference to benefit from new version
(a restaging won't be sufficient).

Is this correct ? On CF instances that don't version buildpack names, how
can users know whether apps were stage a vulrenable version of an offline
buildpack, and require restaging ? Is it by comparing staging date, and
known rootfs patch date ?

Is there some improvements planned around this stack and buildpack
versionning by the CAPI team (in tracker [6] seems related, is there
something more specific planned) ?

Thanks,

Guillaume.

[1] https://www.pivotaltracker.com/n/projects/966314/stories/90428236
[2] https://github.com/simonleung8/cli-stack-changer
[3] http://apidocs.cloudfoundry.org/214/apps/get_app_summary.html
[4]
http://apidocs.cloudfoundry.org/214/stacks/retrieve_a_particular_stack.html
[5] http://apidocs.cloudfoundry.org/214/stacks/list_all_stacks.html
[6] https://www.pivotaltracker.com/story/show/91553650

On Wed, Jul 29, 2015 at 7:16 PM, Onsi Fakhouri <ofakhouri(a)pivotal.io> wrote:

Hey Colin,

Good stuff. I like to draw a circle around the rootfs, the buildpacks,
the generated droplet, the Task/LRP recipes, and the lifecycle binaries
that run inside containers to stage and launch droplets. You could label
that circle an application lifecycle. Diego currently supports three
application lifecycles and is loosely coupled to those lifecycles:

1. The Linux-Buildpack App lifecycle: includes the cflinuxfs2 rootfs, the
various buildpacks (including a known interface for building custom
buildpacks), the droplets (compiled artifacts guaranteed to run with
cflinuxfs2), two binaries: the builder (performs staging) and the launcher
(runs applications), and code that can convert CC requests for staging and
running instances to Diego Task/LRP recipes.

2. The Windows App lifecycle: includes the notion of a correctly
configured windows environment, a windows-compatible droplet, a builder, a
launcher, and code that can generate Tasks/LRPs. In this context we do not
yet have/need the notion of a buildpack though we are free to add one
later. The builder simply prepares the droplet from source and the
launcher knows how to invoke it.

3. The Docker App lifecycle: has no rootfs as the docker image provides
the entire rootfs, includes a builder to extract docker-metadata and send
it back to CC for safe-keeping, and a launcher to launch the requested
process *and* present it with a standard CF environment. Again, there's
also code that knows how to translate CC requests for a docker-based
application into Tasks and LRPs.

The cool thing is that Diego doesn't care about any of these details and
you are free to construct your own lifecycles and have your own contracts
within each lifecycle. You are spot on in noting that there is an implicit
contract between the buildpacks and the rootfs. I'd go further and say
that that implicit contract covers everything in the lifecycle circle (e.g.
the builder has a contract with the buildpacks, it expects `detect`,
`compile` and `release` to work a certain way, the recipes have a contract
with the builder/launcher, they expect particular command line arguments,
etc...)

This is one reason why we've recently transitioned the ownership of the
rootfs from the runtime team to the buildpack team - as the buildpack team
is best suited to define and maintain the contract between the buildpacks
and the rootfs. Would love to explore ways to make all these contracts
more explicit.

One last point. I didn't use the word "stack" in this e-mail until just
now. I agree that it's an overloaded concept that is easily and often
misunderstood ;)

Onsi

On Wed, Jul 29, 2015 at 9:51 AM, Colin Humphreys <colin(a)cloudcredo.com>
wrote:

Hi All,

I wrote a couple of articles about droplets and stacks.

http://www.cloudcredo.com/a-droplet-of-value/

http://www.cloudcredo.com/stacks-of-problems/

The droplet post is fairly self-explanatory, and enabling the choice of
shipping droplets or source is well on the way in Cloud Foundry development.

I feel our story around stacks is far less complete. It seems to be an
overloaded concept inherited from Heroku and the contract with the stack
seems to cause issues for both app and buildpack developers.

I'd like to open the discussion on what the future should be for stacks,
or if you think they're perfect as they are.

Cheers,
Colin

CloudCredo Cheerleader

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

Join cf-dev@lists.cloudfoundry.org to automatically receive all group messages.