Re: Eirini in 2019 CFAR certification requirements
|
Eric Malm <emalm@...>
Hi, all, Thanks for raising the topic of whether to include Eirini as an option in the 2019 CFAR certification requirements. Although the Eirini team has made tremendous progress over the past 6 months or so, it seems premature for the CFF to commit to including Eirini as a certified option for 2019. The team is close to passing a minimal set of CF Acceptance Tests, with the latest update in the #eirini-dev channel (https://cloudfoundry.slack.com/archives/C8RU3BZ26/p1543575973001900) reporting 66 passing, 1 failing, and 155 skipped tests, but according to https://github.com/cloudfoundry-incubator/eirini-ci/blob/master/pipelines/ci/cats-conf.yml that set of CATs does not yet even include all of the default suites in the CATs config (as per https://github.com/cloudfoundry/cf-acceptance-tests#test-configuration). In either case, those particular suites of CATs do not validate functionality such as application security groups, CF SSH, and app tasks that platform operators can enable or should configure, even if some CFAR environments may happen to disable them for their app developers. In addition to parity of feature functionality, I'm concerned that we would be granting certification to CFAR distributions that have not yet demonstrated a reasonable level of parity with current expectations of operational reliability and scale. For comparison, at the start of 2016, Diego had already been running the entirety of PWS for three months and had committed to backwards compatibility of its APIs and BOSH properties, and Pivotal had shipped PCF with it as the default container backend. I'm not aware of an Eirini-backed CFAR having the same level of practical operational validation at scale. I understand that the community members that are most committed to Eirini are concerned about missing all of 2019 for certification, so I'd like to propose the following course of action. Although the CFAR certifications have been on an annual cadence, the "Updates" section in the provider requirements (https://www.cloudfoundry.org/provider-requirements/) seem to me to state that the CFF has the authority to update them at any time. This annual cadence for certification seems intended primarily to give vendors enough lead time to react to proposed breaking changes in the certification, such as the removal of the DEAs and the transition from cf-release to cf-deployment in terms of integration expectations. Adding Eirini as a certification option once it is sufficiently robust in both feature parity and operational maturity would be an additive change, however, and therefore one that would not require advance notice or review lest it render existing distributions incompatible. Should Eirini achieve that parity during the 2019 calendar, that amendment could be released in, say, a "2019.1" certification revision during that year, in advance of the expected certification requirements for 2020. Thanks again, Eric On Mon, Nov 26, 2018 at 10:53 AM Julz Friedman <julz.friedman@...> wrote: > That phrasing doesn't read as a component I would want to run in production (or in a certified distribution). |
|
|