Re: Proposal to Change CATs Ownership


Michael Fraenkel <michael.fraenkel@...>
 

All the teams have to deal with a workflow and bumping. I don't believe
there is a team that doesn't have to also deploy CF with Diego (except
mine) and run CATs.

The issue I believe you are highlighting is something different. CAPI
needs a CAPI acceptance for its new features which have no CLI
counterpart. In the past CATs contained that. I think now that CATs now
includes DATs and RATs and xATs, its not part of one team. If CAPI needs
a place to test out new functions using cf curl, that can either live in
CATs or it goes some where else. CATs should be what an end user will
attempt which is hopefully not driving CF via 'cf curl'.

Having CATs live under any one release is a problem for any other team
that uses it and provides code. Any changes are gated by someone merging
a commit and making it through a pipeline that includes other changes
that maybe should not be consumed by others since its not part of the
"release" because we will get additional components from the CAPI
release that may not be ready.

- Michael

On 3/23/16 2:43 PM, Utako Ueda wrote:
We had a series of meetings to figure out a good path for CATs that
ranged from multiple teams to smaller groups. We met most recently to
address concerns that include the following:

• Workflow and Release Bumping Issues: CATs relies on calls to
specific versions of CC API to run successfully. The CAPI CI currently
bumps cloud_controller_ng within capi-release, and on a successful run
of CATs (which lives on cf-release develop), capi-release then gets
bumped in cf-release develop. This leads to a "chicken and egg"
scenario, in that when changes are made to both CATs and CC, both
cf-release and capi pipelines are broken until a manual fix is made.
• In a world where cf-release no longer exists, how do we ensure CATs
uses the correct versions of the CF CLI and CC API?

tl;dr: Though not all of the CATs test the CC API explicitly, they all
make use of the CC API and therefore make CC a choke point. Having
CATs as a separate release may be a good idea, but doesn't necessarily
address the strong coupling issues.

We'd like to know if there are specific cases in which this will be
detrimental to other projects' workflows before moving forward.


On Wed, Mar 23, 2016 at 4:08 AM, Michael Fraenkel
<michael.fraenkel(a)gmail.com <mailto:michael.fraenkel(a)gmail.com>> wrote:

Can you explain why CATs should live under CAPI?

CATs now represents acceptance test for all of the Cloud Foundry
function driven mainly via the CLI.

Tieing this to any one release seems a bit artificial.
CATs is about testing the aggregation of all the various releases
not just CAPI.

I am more surprised that it wasn't suggested to be its own release
which dictated the versions for all releases that had passed.

- Michael


On 3/22/16 3:05 AM, Utako Ueda wrote:
Hi all,

The CAPI team would like to take ownership of the CF acceptance
tests <https://github.com/cloudfoundry/cf-acceptance-tests/> and
include them as part of CAPI release
<http://github.com/cloudfoundry/capi-release>.

This solves several pain points we've experienced over the last
few months, mainly due to the strong coupling between CATs and
the Cloud Controller API.

Our plan is to submodule the CATs repo in to CAPI release, and
bump CATs on a successful run through the CAPI ci pipeline. At
this point, CAPI release will be bumped in cf-release develop,
which other teams will consume for their own testing purposes.

We hope to make these changes in the near future as we wrap up
our release extraction from cf-release. We'd like to know if any
teams have any concerns about this before we proceed, so do let
us know as soon as possible so we can address them.

Thanks,
Utako, CF CAPI Team

Join cf-dev@lists.cloudfoundry.org to automatically receive all group messages.