Re: Testing behaviour of a production CF environment


Graham Bleach
 

Hi Amit,

Thanks for your reply.

On 1 June 2016 at 04:06, Amit Gupta <agupta(a)pivotal.io> wrote:

The README attempts to give some idea of whether test suites are unsafe to
run in certain contexts:


https://github.com/cloudfoundry/cf-acceptance-tests#explanation-of-test-suites

And the section on Test Execution explains how you can skip test suites,
tests matching a certain regex, etc:

https://github.com/cloudfoundry/cf-acceptance-tests#test-execution
We've been looking through the test suites and can't see a straightforward
way to run only the "safe" tests that won't affect normal users / don't
require an admin user.

For instance, the apps suite includes both tests we'd like to run, testing
core user-facing behaviour and the admin buildpack lifecycle test with the
issue you linked to. I don't think skipping based on regexes on test names
works well, both because the regex will become long quite quickly and
because cf-acceptance-tests is a moving target - each time we upgraded to a
new release we'd need to review which tests were added and update our
regexes.

I wondered if other people would be interested in having a way to only run
the "non-admin" tests? If so, perhaps re-organising the suites to enable
that would be a welcome change?

Regards,
Graham

Join cf-dev@lists.cloudfoundry.org to automatically receive all group messages.