We had a series of meetings to figure out a good path for CATs that ranged
from multiple teams to smaller groups. We met most recently to address
concerns that include the following:

• Workflow and Release Bumping Issues: CATs relies on calls to specific
versions of CC API to run successfully. The CAPI CI currently bumps
cloud_controller_ng within capi-release, and on a successful run of CATs
(which lives on cf-release develop), capi-release then gets bumped in
cf-release develop. This leads to a "chicken and egg" scenario, in that
when changes are made to both CATs and CC, both cf-release and capi
pipelines are broken until a manual fix is made.
• In a world where cf-release no longer exists, how do we ensure CATs uses
the correct versions of the CF CLI and CC API?

tl;dr: Though not all of the CATs test the CC API explicitly, they all
make use of the CC API and therefore make CC a choke point. Having CATs as
a separate release may be a good idea, but doesn't necessarily address the
strong coupling issues.

We'd like to know if there are specific cases in which this will be
detrimental to other projects' workflows before moving forward.


On Wed, Mar 23, 2016 at 4:08 AM, Michael Fraenkel <
michael.fraenkel(a)gmail.com> wrote:

Can you explain why CATs should live under CAPI?

CATs now represents acceptance test for all of the Cloud Foundry function

driven mainly via the CLI.

Tieing this to any one release seems a bit artificial.

CATs is about testing the aggregation of all the various releases not
just CAPI.

I am more surprised that it wasn't suggested to be its own release which
dictated the versions for all releases that had passed.

- Michael


On 3/22/16 3:05 AM, Utako Ueda wrote:

Hi all,

The CAPI team would like to take ownership of the CF acceptance tests
<https://github.com/cloudfoundry/cf-acceptance-tests/> and include them
as part of CAPI release <http://github.com/cloudfoundry/capi-release>.

This solves several pain points we've experienced over the last few
months, mainly due to the strong coupling between CATs and the Cloud
Controller API.

Our plan is to submodule the CATs repo in to CAPI release, and bump CATs
on a successful run through the CAPI ci pipeline. At this point, CAPI
release will be bumped in cf-release develop, which other teams will
consume for their own testing purposes.

We hope to make these changes in the near future as we wrap up our
release extraction from cf-release. We'd like to know if any teams have any
concerns about this before we proceed, so do let us know as soon as
possible so we can address them.

Thanks,
Utako, CF CAPI Team