Re: [IMPORTANT] 2017 PaaS Certification Requirements
Thanks Dieu and Chip for explicitly asking feedback on the 2017
certification process, and leaving time for the CF community to provide
such feedback. I see seldom comments in the google docs document itself, so
I flesh out for now some more comments on cf-dev(a). It might make sense to
track/follow up some of them as comments/suggested edits in google docs,
which I plan to do in the coming days.
Following is some more feedback from the perspective of CF users such as
Orange (as a complement to previous useful feedback from CF
- as a user considering adopting CF as a portable platform backed by
multiple certified vendors, in order to trust the certification label, I
expect the certification process to be transparent, the certification
audits/results/details to be publicly available (including URL and version
of the product or service instance PF URL was assessed), and documentation
to help me distinguish the CF portable core from proprietary added-value
features that may come with vendor lock-in.
- as a CF user picking a certified CF platform, in order to have confidence
that the minimum quality enforced by the CFF teams is guaranteed, I expect
the smoke and acceptance tests to be passing on the certified platform.
- as a CF user picking a certified CF platform, in order to securely run
production workload, I expect a certified platform to provide up-to-date
Here are some other examples of risks I would like to mitigate when picking
a certified platform:
- a certified vendor might choose to provide CF on a different OS distro
than what the CFF release engineering team is running in the pipeline
qualifying CF releases  (which is leveraging bosh stemcells  to
provision OS), therefore exposing me to unexpected bugs/quality issues in
the "CF kernel"
- a certified vendor might not yet publicly provide offerings that can be
purchased, in effect reducing the list of candidate vendors I can pick from
when considering CF as a portable multi-vendor platform
- a certified vendor might not provide timely CVE patches to the "CF
kernel", exposing me to security vulnerabilities that CF I expect CF to
protect me from. The current requirements of components not older than 6
months does not seem strong enough from this perspective.
- a certified vendor leveraging by default proprietary assets others than
the CFF "batteries included" default from the CFF distribution. If
undocumented from in the certification audits, this creates vendor lock-in
for users adopting this certified platform.
- a certified vendor might choose to not enable/support a range of cf core
features (e.g. "cf ssh", "TCP routing", or "route services"), reducing the
scope of the certification.
Some more suggestions to the cert requirements:
- require the outcome/details of the certification audit to be published
for certified platforms (detailing source of distros, version tested,
- require the passing smoke tests and acceptance tests test to become
requirements for certified platforms, and configuration of the tests and
reports to be published. Ideally encourage/require public pipelines from
the certified providers as to provide trust that the tests are still
passing after the certification is granted. As a side effect, this would
also encourage certified vendors in contributing to the release engineering
effort (which is currently mostly sponsored by Pivotal as Amit reminded)
and ensure that the pipelines are portable and can be run by the community
without hitting/missing pivotal-specifics. See  for related effort
orange went through with the help of the buildpacks team to replicate the
buildpacks-ci within Orange.
- require to document how the certified provider ensures the certified
platform remains secured w.r.t. upcoming CVEs
- make more explicit the CF extension points (either in the document or as
an external reference): services implementing the service broker API,
volume driver provider, network plugin provider, external CC/diego DB or
blobstore, CLI plugins, etc.... This would help clarify the scope of the
certified core and distinguish it from added value, proprietary extensions.
On Tue, Nov 1, 2016 at 7:04 PM, Chip Childers <cchilders(a)cloudfoundry.org>
Certainly! I didn't mean to close it down on anyone... keep them coming.