We are new to CF. After some hacking, we found ourselves with hundreds of
jobs in Jenkins and a variety of shell scripts to promote our apps from
space to space with a blue/green strategy. We also run smoke tests before
we switch the routes and sometimes need to retry a deploy due to network
issues, etc. We would like to specify what needs to be deployed in Github,
not Jenkins.

Has anyone heard of a tool that does this well?

Thanks,
David

--
http://www.linkedin.com/in/dvydra
http://www.testdriven.com
http://twitter.com/vydra

Nicely put. Let’s see what we get.

Azad

From: David Vydra <david(a)vydra.net<mailto:david(a)vydra.net>>
Reply-To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>>
Date: Saturday, July 11, 2015 at 12:57 PM
To: "cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>" <cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>>
Subject: [cf-dev] Composite Application Deployer

We are new to CF. After some hacking, we found ourselves with hundreds of jobs in Jenkins and a variety of shell scripts to promote our apps from space to space with a blue/green strategy. We also run smoke tests before we switch the routes and sometimes need to retry a deploy due to network issues, etc. We would like to specify what needs to be deployed in Github, not Jenkins.

Has anyone heard of a tool that does this well?

Thanks,
David

--
http://www.linkedin.com/in/dvydra<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.linkedin.com_in_dvydra&d=AwMFAw&c=IV_clAzoPDE253xZdHuilRgztyh_RiV3wUrLrDQYWSI&r=mKQyOqxpP3Pxe4bGZxxgMJpUdWb5sjU245GYtlUzGHQ&m=WtunChoVNxq-qxs8N37bT3qZXiLppWHyL7j0xp59u30&s=R9tcTiMZ9qFlKxgcInGRgfK4Q31ySJu7TUaHY6g_wbU&e=>
http://www.testdriven.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.testdriven.com&d=AwMFAw&c=IV_clAzoPDE253xZdHuilRgztyh_RiV3wUrLrDQYWSI&r=mKQyOqxpP3Pxe4bGZxxgMJpUdWb5sjU245GYtlUzGHQ&m=WtunChoVNxq-qxs8N37bT3qZXiLppWHyL7j0xp59u30&s=BiSKIAR3Zs5D5k3_UNX6xzs4WEUKHp3cRrD-iTs_rzs&e=>
http://twitter.com/vydra<https://urldefense.proofpoint.com/v2/url?u=http-3A__twitter.com_vydra&d=AwMFAw&c=IV_clAzoPDE253xZdHuilRgztyh_RiV3wUrLrDQYWSI&r=mKQyOqxpP3Pxe4bGZxxgMJpUdWb5sjU245GYtlUzGHQ&m=WtunChoVNxq-qxs8N37bT3qZXiLppWHyL7j0xp59u30&s=uJH9TrCsdV8Asywf0NLIIDwYhmAp44Myn7yEv8Qpmxw&e=>

david,

it sounds like you have a fair amount of applications. any numbers you
would be willing to share that talks about the jenkins jobs, or at least
the ratio of:
app : spaces : jenkins jobs

to develop cloud foundry, we use concourse [1] for most of the ci.
previously we had relied on jenkins and gocd from thoughtworks [2].
however, concourse was designed with container images in-mind and it has
great support for pipelines like you're describing that involve multiple
environments with tests gating the automated promotion.

one of the features of concourse is that it works with the cf cli plugin
named autopilot [3] which does zero-downtime deploys and relies on a cf app
manifest which can be checked into SCM. the concourse files should also be
checked into SCM. this way your SCM drives your CI and your deployments.

the runtime team has been working on the so-called v3 cloud controller api
for awhile when enables having multiple versions of the same cf application
to exist and be running at the same time. when that work finishes, you
won't need 2 separate apps that share the same route as the current
blue/green processes require. then the platform can support a standard
zero-downtime application update, but you still may want to have some sort
of mechanism to health-check the new version of the app before cutting over
traffic to the new version completely.

[1] http://concourse.ci/
[2] http://www.thoughtworks.com/products/go-continuous-delivery
[3] https://github.com/concourse/autopilot

--
Thank you,

James Bayer

James,

Thanks for the detailed answer. We started treating Jenkins views as the
representation of a particular space and each view contains many of jobs
for that push one app to that space.

autopilot plugin is definitely a move in the right direction, but we have a
richer deployment story --- we run smoke tests before we switch the routes.
We are implementing lower-level primitives that support this level of
granularity.

We are currently using Java, but will probably switch to Go if our approach
works (and we find time to learn Go)

Thanks,
David

--
http://www.linkedin.com/in/dvydra
http://www.testdriven.com
http://twitter.com/vydra