FYI: CAB call for July 2015 - moved to 7/15 @ 8a PDT
FYI...toggle quoted messageShow quoted text
Final reminder of the CAB call tomorrow at 8a PDT. See all info from last week's email below.
Also reminder to all PMs to include their highlights and to others to list any proposals or discussion items.
Talk to you all tomorrow morning. Best,
ibm cloud labs
silicon valley, ca
Sent from my iPhone
On Jul 7, 2015, at 9:54 AM, Michael Maximilien <maxim(a)us.ibm.com> wrote:
Thanks again to Phil Whelan for crafting the greatly detailed cab minutes.
The MEGA section says the following:
*Updates from this MEGA team for the past week include having extracted
etcd-release and now they are working on extracting consul-release, setting
up pipelines for them, testing them and publishing them to bosh.io
*Longer-term they want to generalize the patterns that have been introduced
for Diego around manifest generation.*
*Cornelia Davis from Pivotal asked Amit to confirm whether manifest
generation was moving away from Spiff, to which Amit replied "yes"."*
Looking at diego repo, I still see the spiff installation instructions, and
spiff used for manifest generation . Where can I learn about the
manifest generation pattern for diego, to better understand this CAB
Thanks in advance,
On Tue, Jul 14, 2015 at 4:56 PM, Michael Maximilien <maxim(a)us.ibm.com>
Amit Kumar Gupta
Thanks for your question. The best place to start understanding how diego
generates manifests is to look at the manifest generation script in the
It primarily merges data into the main diego template:
There are several principles operating at the heart of this manifest
1. The inputs to the script should be a fixed list of stubs with a specific
data structure (not an arbitrary list of stub files which can have all sorts
of crazy structures in their YAML).
2. The YAML structure of the stub should make sense for the purpose of the
stub, i.e. clearly represent the data it contains, it should not represent
the data in some strange contorted format that needs to know where in the
upstream manifest template it will be merged into.
3. Avoid (( merge )) within the body of the template, places that need to
have values filled in explicitly reference the value from the stub that they
4. No nested templates, or "stubs" that have spiff double-parens syntax in
5. Each stub should have a single responsibility (e.g. instance count
overrides, persistent disk overrides, etc.) which should be small and easy
to understand; any two stubs should have orthogonal responsibilities, e.g.
you don't specify some credentials in this stub, and some other credentials
in another stub.
6. The responsibilities of the stubs should be generalizable to any BOSH
release (e.g. "instance count overrides" is a generic enough purpose that
would make sense for any BOSH release.)
7. Have an explicit stub for IaaS settings (basically anything that goes
into a cloud_properties stanza in a BOSH manifest); the main template should
be IaaS agnostic (rather than a template for each IaaS).
8. Cleanliness; do not produce a manifest in the end that has a bunch of
unnecessary junk that BOSH doesn't even use (e.g. "meta"). People tend to
develop the bad habit of depending on this random data.
9. No default property values in the templates; defaults should live in the
job specs, the only properties exposed in the template should be the ones it
makes sense to override.
10. Don't expose every property for override. spiff has hard limitations
that make it impossible to use if you want to construct any arbitrary
deployment manifest for a given release (or set of releases). Given that it
already can't solve every problem, understand that the manifest generation
tooling also doesn't need to solve every possible problem. Advanced users
who want to override obscure properties can use their own advanced tooling.
11. Dovetail with upcoming bosh features:
The long-term goal is to make BOSH smarter and ideally obviate the need for
complex tooling to generate these unwieldy, several-thousand-line YAML
manifests. On the surface, both diego-release and cf-release suggest how to
use spiff to generate manifests, but the way the two use spiff is
drastically different. With diego, we've started by simplifying how
manifests are generated so that the full power of spiff isn't needed, and
identifying better abstractions that make sense for BOSH to know about so
that eventually none of the power of spiff is needed.
Amit, CF OSS Release Integration PM
Pivotal Software, Inc.
View this message in context: http://cf-dev.70369.x6.nabble.com/cf-dev-CAB-call-for-July-2015-moved-to-7-15-8a-PDT-tp645p986.html
Sent from the CF Dev mailing list archive at Nabble.com.
Thanks a lot Amit for this detailed write up. This is much clearer now.toggle quoted messageShow quoted text
On Fri, Jul 31, 2015 at 6:01 AM, Amit Gupta <agupta(a)pivotal.io> wrote: