We had a series of meetings to figure out a good path for CATs that
ranged from multiple teams to smaller groups. We met most recently to
address concerns that include the following:

• Workflow and Release Bumping Issues: CATs relies on calls to
specific versions of CC API to run successfully. The CAPI CI currently
bumps cloud_controller_ng within capi-release, and on a successful run
of CATs (which lives on cf-release develop), capi-release then gets
bumped in cf-release develop. This leads to a "chicken and egg"
scenario, in that when changes are made to both CATs and CC, both
cf-release and capi pipelines are broken until a manual fix is made.
• In a world where cf-release no longer exists, how do we ensure CATs
uses the correct versions of the CF CLI and CC API?

tl;dr: Though not all of the CATs test the CC API explicitly, they all
make use of the CC API and therefore make CC a choke point. Having
CATs as a separate release may be a good idea, but doesn't necessarily
address the strong coupling issues.

We'd like to know if there are specific cases in which this will be
detrimental to other projects' workflows before moving forward.


On Wed, Mar 23, 2016 at 4:08 AM, Michael Fraenkel
<michael.fraenkel(a)gmail.com <mailto:michael.fraenkel(a)gmail.com>> wrote:

Can you explain why CATs should live under CAPI?

CATs now represents acceptance test for all of the Cloud Foundry
function driven mainly via the CLI.

Tieing this to any one release seems a bit artificial.
CATs is about testing the aggregation of all the various releases
not just CAPI.

I am more surprised that it wasn't suggested to be its own release
which dictated the versions for all releases that had passed.

- Michael


On 3/22/16 3:05 AM, Utako Ueda wrote:

Hi all,

The CAPI team would like to take ownership of the CF acceptance
tests <https://github.com/cloudfoundry/cf-acceptance-tests/> and
include them as part of CAPI release
<http://github.com/cloudfoundry/capi-release>.

This solves several pain points we've experienced over the last
few months, mainly due to the strong coupling between CATs and
the Cloud Controller API.

Our plan is to submodule the CATs repo in to CAPI release, and
bump CATs on a successful run through the CAPI ci pipeline. At
this point, CAPI release will be bumped in cf-release develop,
which other teams will consume for their own testing purposes.

We hope to make these changes in the near future as we wrap up
our release extraction from cf-release. We'd like to know if any
teams have any concerns about this before we proceed, so do let
us know as soon as possible so we can address them.

Thanks,
Utako, CF CAPI Team