Performance of cf-acceptance-tests


Graham Bleach
 

Hello,

Related to my earlier question, we're running the cf-acceptance-tests
often. It takes us between 50 and 70 minutes to run them (with some
exclusions [1]) in our environment.

Is this faster or slower than in other environments? Are we missing any
techniques to reduce their runtime?

Graham

[1]
https://github.com/alphagov/paas-cf/blob/master/manifests/cf-manifest/deployments/900-cf-tests.yml#L12
https://github.com/alphagov/paas-cf/blob/master/manifests/cf-manifest/deployments/900-cf-tests.yml#L13
https://github.com/alphagov/paas-cf/blob/master/manifests/cf-manifest/deployments/900-cf-tests.yml#L16
https://github.com/alphagov/paas-cf/blob/master/manifests/cf-manifest/deployments/900-cf-tests.yml#L21


Sunil Babu <cloudgrp.assist@...>
 

Hi

Can you check on the loop calls been made and dependency on waiting for for
international loop to complete

It's design issue check it

Rgds
Sunil
On 31-May-2016 10:59 pm, "Graham Bleach" <
graham.bleach(a)digital.cabinet-office.gov.uk> wrote:

Hello,

Related to my earlier question, we're running the cf-acceptance-tests
often. It takes us between 50 and 70 minutes to run them (with some
exclusions [1]) in our environment.

Is this faster or slower than in other environments? Are we missing any
techniques to reduce their runtime?

Graham

[1]

https://github.com/alphagov/paas-cf/blob/master/manifests/cf-manifest/deployments/900-cf-tests.yml#L12

https://github.com/alphagov/paas-cf/blob/master/manifests/cf-manifest/deployments/900-cf-tests.yml#L13

https://github.com/alphagov/paas-cf/blob/master/manifests/cf-manifest/deployments/900-cf-tests.yml#L16

https://github.com/alphagov/paas-cf/blob/master/manifests/cf-manifest/deployments/900-cf-tests.yml#L21


Graham Bleach
 

On 31 May 2016 at 18:33, Sunil Babu <cloudgrp.assist(a)gmail.com> wrote:

Can you check on the loop calls been made and dependency on waiting for
for international loop to complete

It's design issue check it
Can you be a bit more specific about what you mean and link to the code
please?

Graham


CF Runtime
 

Hi Graham,

cf-acceptance-tests (cats) offers parallelization. The readme docs (https://github.com/cloudfoundry/cf-acceptance-tests#test-execution) explain how to configure this. (./bin/test -r -nodes=4)

Are you running cats in parallel? If so how many nodes are you running cf-acceptance-tests with?

The performance gain by running cats with multiple nodes is dependent on the resource capacity of the cf backend.

In our environments it typically takes ~45 minutes. We have successfully run cats in as little as ~20 minutes by running on multiple nodes.

Thanks
Rob & Dennis
cf-release-integration
Pivotal