Date   

Application Releases and Rollback Feature Narrative

Troy Topnik
 

This narrative proposes adding Heroku-style "releases" for applications:

https://docs.google.com/document/d/1sIViZyeCri2URHAt57YW4n482cBav3b5dOgP6D_vr4A/

The feature allows developers to see a list of previous deployments of their applications (code revisions stored as a droplet, plus settings related to a particular push), and to roll back applications to a previous release.

The basic functionality can already be accomplished in CF using client plugins, client-side tools, and/or manual workflows. This proposal is for a more automatic, server-side feature which could be enabled by an operator or manager.

The narrative doesn't (yet) take into account some of the new Droplet APIs in v3. Suggestions on how this feature could utilize these new APIs would be very welcome. If there is overlap with other work in progress, please make a note of it here or in a comment in the doc.

Thanks!

TT


Re: How to install things on specfic APP container

Daniel Mikusa
 

You don't have root access in application containers. You could bundle the
software into a custom build pack. Again, you can't install stuff with
root so you have to manually extract the files somewhere writeable by the
vcap user and run them from that location. A log of build packs use
`/home/vcap/app/vendor`. Docker might be another option at least to more
easily install the software. You'd still need a command to run it as the
vcap user. I don't think you could run it as a service.

Dan

On Thu, May 5, 2016 at 10:00 PM, Stanley Shen <meteorping(a)gmail.com> wrote:

Hello, all

For some purpose, I would like to install things on specific APP container.
For example, I have a web application, and I want to do virus scan for
uploading attachments, originally on ubuntu VM we can do "sudo apt-get
install scan-virus" and make it a service process.

How can we do such things now?


Re: offline java-buildpack is not loading new gem from DEA during cf push

Lingesh Mouleeshwaran
 

Hi cloudfoundry team ,

we have modified java-buildpack validate input yaml against schema using
kwalify gem. we have created offline buidpack for the same. now when we do
cf push using this new buildpack we are getting below error

*2016-05-06T11:44:28.85+0530 [STG/9] OUT -----> Downloaded app package
(4.0K)*
*2016-05-06T11:44:29.07+0530 [STG/0] ERR
/usr/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require': cannot load
such file -- kwalify (LoadError*

these require kwalify gem included in Gemfile and Gemfile.lock. Please
could you suggest whether we need to run bundle install each runner ruby or
is there any other way available to load gem file ??


Regards
Lingesh M

On Fri, May 6, 2016 at 1:32 PM, Lingesh Mouleeshwaran <
lingeshmouleeshwaran(a)gmail.com> wrote:

Hi cloudfoundry team ,

we have modified java-buildpack validate input yaml against schema using
kwalify gem. we have created offline buidpack for the same. now when we do
cf push using this new buildpack we are getting below error

*2016-05-06T11:44:28.85+0530 [STG/9] OUT -----> Downloaded app
package (4.0K)*
*2016-05-06T11:44:29.07+0530 [STG/0] ERR
/usr/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require': cannot load
such file -- kwalify (LoadError*

these require qualify included in Gemfile and Gemfile.lock. Please could
you suggest whether we need to run bundle install each runner ruby or is
there any other way available to load gem file ??


Regards
Lingesh M


offline java-buildpack is not loading new gem from DEA during cf push

Lingesh Mouleeshwaran
 

Hi cloudfoundry team ,

we have modified java-buildpack validate input yaml against schema using
kwalify gem. we have created offline buidpack for the same. now when we do
cf push using this new buildpack we are getting below error

*2016-05-06T11:44:28.85+0530 [STG/9] OUT -----> Downloaded app package
(4.0K)*
*2016-05-06T11:44:29.07+0530 [STG/0] ERR
/usr/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require': cannot load
such file -- kwalify (LoadError*

these require qualify included in Gemfile and Gemfile.lock. Please could
you suggest whether we need to run bundle install each runner ruby or is
there any other way available to load gem file ??


Regards
Lingesh M


Re: push APP using docker image

Stanley Shen <meteorping@...>
 

Thanks for information.

I tried it today and it works well, I didn't see such logs, not sure why I saw that error yesterday :-)


How to install things on specfic APP container

Stanley Shen <meteorping@...>
 

Hello, all

For some purpose, I would like to install things on specific APP container.
For example, I have a web application, and I want to do virus scan for uploading attachments, originally on ubuntu VM we can do "sudo apt-get install scan-virus" and make it a service process.

How can we do such things now?


Re: My nodejs application can't work if the version nodejs buildpack >= v1.5.9

Sam Dai
 

I have checked the pull request of nodejs buildpack git project, but haven't found any clue, so I guess the version of my npm might be old, upgrade to new version, and it works


Re: cf scale command sequence

Siva Balan <mailsiva@...>
 

Thanks. What I am concerned about are the inflight transactions on the
container when it is being destroyed. What happens to them? If the routes
are removed from the GoRouter before the container is destroyed, I believe,
it has some lead time to process inflight transactions. But if the route is
removed after the container is destroyed, the probability of inflight
transactions getting killed is more. Also, I dont understand how it can
happen either way. The code path should do one or the other but not both.

On Thu, May 5, 2016 at 2:16 PM, Michael Fraenkel <michael.fraenkel(a)gmail.com
wrote:
It is pretty much a race. It can happen either way. GoRouter will retry on
failed connects so if the route remains when the container is removed, the
next instance will be tried.

- Michael


On 5/5/16 3:34 PM, Siva Balan wrote:

When I issue a cf scale command to bring down the number of app instances
from, say 10 to 5, what is the sequence of events that take place to make
this happen? I am particularly interested to know if the containers are
removed before their routes are removed from the gorouter or is it the
other way round? This question if for both diego and DEA versions.

Thanks
Siva Balan

--
http://www.twitter.com/sivabalans



Re: VCAP_APPLICATION in the V3 Cloud Foundry API

Nicholas Calugar
 

Thanks for all your input. Upon seeing how the buildpacks use VCAP_APPLICATION, I’m thinking we want to leave it as is and potentially add a few new top level environment variables.

CF_APP_GUID

CF_APP_NAME

CF_PROCESS_GUID

CF_PROCESS_TYPE

With the already present

CF_INSTANCE_INDEX, we could encourage buildpacks to build the identifier using all ENV variables:

CF_APP_NAME:CF_PROCESS_TYPE:CF_INSTANCE_INDEX

Nicholas Calugar

Cloud Foundry API Product Manager

Pivotal Software, Inc.

On Mon, May 02, 2016 at 2:36 PM Sabha Parameswaran

<
mailto:Sabha Parameswaran <sabhap(a)pivotal.io>
wrote:
Agree with Dan.

Currently most APM vendors use the VCAP_APPLICATION env variable for mapping and keeping track of a specific instance using the instance guid, instance index, app guid fields. Changing it would affect the buildpacks and extensions.

Sample for AppD APM:

default_node_name:

$(ruby -e "require 'json' ; a = JSON.parse(ENV['VCAP_APPLICATION']);

puts \"#{a['application_name']}

:#{a[

'

instance_index

'

]}\

"

"

)

Some of these complex expressions might become simpler, as long as the values still come through with different names. 

So, as long as the current set of values continue to exist and either in flat or easy to grab from ENV, I am okay with that.

thanks,

Sabha

On Mon, May 2, 2016 at 2:19 PM, Mike Youngstrom

<
mailto:youngm(a)gmail.com
wrote:

I have a few customers that grab that instance index to aid in monitoring and logging and to self identify in certain situations.  Is there an issue with continuing to expose the instance index?

I don't know anything about v2 vs v3 apis.  Would VCAP_APPLICATION continue to work for apps pushed using the v2 api and only have this new value if pushed using the v3 api?

Mike

On Mon, May 2, 2016 at 2:49 PM, Daniel Mikusa

<
mailto:dmikusa(a)pivotal.io
wrote:

I know some of the build packs will look at application name and use that for configuration of some of the APM agents (NewRelic, AppDynamics, etc..).  

Ex:  
https://github.com/cloudfoundry/java-buildpack/blob/master/config/app_dynamics_agent.yml#L21

The PHP build pack does this too.

As long as that info is exposed somewhere, it would be OK.

Dan

On Mon, May 2, 2016 at 4:34 PM, john mcteague

<
mailto:john.mcteague(a)gmail.com
wrote:

While I dont use this directly, my first reaction is some libraries such as Spring Cloud Connectors depend on this to identify that an app is running in CF, and we rely heavily on Spring Cloud Connectors.

That being said, assuming there are alternative variable that uniquely identify CF, the library authors could key off that instead.

On Mon, May 2, 2016 at 9:28 PM, Nicholas Calugar

<
mailto:ncalugar(a)pivotal.io
wrote:

Hi CF-Dev,

The CAPI team is working hard to get to MVP for V3 of the Cloud Foundry API. You can follow along by visiting our documentation and tracker:

http://v3-apidocs.cloudfoundry.org/
https://www.pivotaltracker.com/n/projects/966314

We'd like to get some feedback from the community regarding VCAP_APPLICATION, an environment variable provided to applications running on Cloud Foundry. In V3, an application contains 1 or more processes that can be scaled independently, so VCAP_APPLICATION doesn't quite fit.

Do you have any use cases around VCAP_APPLICATION that you could share? Are there Cloud Foundry operators / users that would like to get rid of this all together? Our current plan was to do something like below, but wanted to hear from the community first.

"CF_APPLICATION_PROCESS": {

  "space": {

    "guid": "682e10a0-6f3a-4a9f-b52c-f508b2bd99c6",

    "name": "ncalugar-a1"

  },

  "application": {

    "guid": "<v3-app-guid>",

    "name": "nickdora"

    "version": "<app-version-guid>"

  },

  "process": {

    "guid": "<v3-proc-guid>",

    "type": "web",

    "ports": [8080,8081]

  },

  "route_mappings": [

    {

      "guid": "<v3-route-mapping-guid>",

      "uri": "
http://nickdora.a1-app.cf-app.com
",

      "port": 8080

    },

    {

      "guid": "<v3-route-mapping-guid>",

      "uri": "
http://nickdora.a1-app.cf-app.com/admin
",

      "port":"8081"

    }

  ],

  "limits": {

    "fds": 16384,

    "mem": 1024,

    "disk": 4096

  }

}

Thanks,

Nick

--

Sabha Parameswaran

Platform Engineering, Cloud Foundry

Pivotal, Inc.

m: 
tel:%2B1%20%28415%29%20244-5554


Re: cf scale command sequence

Michael Fraenkel <michael.fraenkel@...>
 

It is pretty much a race. It can happen either way. GoRouter will retry
on failed connects so if the route remains when the container is
removed, the next instance will be tried.

- Michael

On 5/5/16 3:34 PM, Siva Balan wrote:
When I issue a cf scale command to bring down the number of app
instances from, say 10 to 5, what is the sequence of events that take
place to make this happen? I am particularly interested to know if the
containers are removed before their routes are removed from the
gorouter or is it the other way round? This question if for both diego
and DEA versions.

Thanks
Siva Balan

--
http://www.twitter.com/sivabalans


cf scale command sequence

Siva Balan <mailsiva@...>
 

When I issue a cf scale command to bring down the number of app instances
from, say 10 to 5, what is the sequence of events that take place to make
this happen? I am particularly interested to know if the containers are
removed before their routes are removed from the gorouter or is it the
other way round? This question if for both diego and DEA versions.

Thanks
Siva Balan

--
http://www.twitter.com/sivabalans


Cloud Controller jobs no longer using properties.domains

Utako Ueda
 

Heads up, y'all:

To reduce confusion about manifest properties `domain` and `system_domain`,
CAPI is working with Release Integration to consolidate the two under the
property `system_domain`. As part of this effort, the cloud_controller_ng,
cloud_controller_worker, cloud_controller_clock, and blobstore jobs no
longer use `domain`, and are using `system_domain` in its stead.

This should be a backwards compatible change. As mentioned in previous
emails, Release Integration team will be coordinating the removal of
`domain` from manifest generation templates and documentation sometime in
the near future.

Thanks,
Utako


In Diego (maybe DEA also) why is $HOME /home/vcap/app and not /home/vcap?

Mike Youngstrom
 

This burned me today because I was attempting to put fonts into the home
directory identified by the java system property "user.home" (which is
/home/vcap) instead of the environment variable $HOME which is
/var/vcap/app.

This is especially confusing because SSH home is "/home/vcap".

Anyone have any context into why $HOME for the app is /var/vcap/app and not
/var/vcap?

Thanks,
Mike


Looking to do user interviews with larger OSS installations, focusing on logging and metrics

Jim CF Campbell
 

Hi cf-dev,

I'm the product manager for Loggregator. I'm looking for users on the
larger side to interview about your experiences, needs and wants for
logging and metrics in Cloud Foundry. I can also inform you of what we're
planning on doing in the future
<https://docs.google.com/spreadsheets/d/1QOCUIlTkhGzVwfRji7Q14vczqkBbFGkiDWrJSKdRLRg/edit#gid=0>
.

When you reply, please also indicate where you are, and if you would be
open to an onsite visit.

It's potentially a great benefit to get your input in - based on user
interviews with LDS and SAP, they both got features they want in the
roadmap. (but no promises :-)


--
Jim Campbell | Product Manager | Cloud Foundry | Pivotal.io | 303.618.0963


Re: My nodejs application can't work if the version nodejs buildpack >= v1.5.9

John Shahid
 

Glad to here you got your app to deploy. Any idea why you had to update the
version of npm ? Did you see anything in the changelog that is interesting ?

On Sat, Apr 30, 2016 at 9:45 PM Sam Dai <sam.dai(a)servicemax.com> wrote:

Hello,
I have a nodejs application, it can be deployed and started in CF if the
version nodejs buildpack <v 1.5.9, the version of node and npm in
package.json of my nodejs application is as below:

, "engines" : {
"node" : "~0.12.7",
"npm" : "~2.11.3"
}

When I deployed my nodejs application to nodejs v1.5.9, there is the
following error, do I need change something?

Downloading nodejs_buildpack...

Downloaded nodejs_buildpack (60.7M)

Creating container

Successfully created container

Downloading app package...

Downloaded app package (4.8M)

Staging...

-------> Buildpack version 1.5.11

-----> Creating runtime environment



NPM_CONFIG_LOGLEVEL=error

NPM_CONFIG_PRODUCTION=true

NODE_ENV=production

NODE_MODULES_CACHE=true

-----> Installing binaries

engines.node (package.json): ~0.12.7

engines.npm (package.json): ~2.11.3



Downloading and installing node 0.12.13...

Downloaded
[file:///tmp/buildpacks/d9e57be809c47eef0c67cf7e2b84bbbe/dependencies/https___pivotal-buildpacks.s3.amazonaws.com_concourse-binaries_node_node-0.12.13-linux-x64.tgz]

Resolving npm version ~2.11.3 via semver.io...

Downloading and installing npm 2.11.3 (replacing version 2.15.0)...

-----> Restoring cache

Skipping cache restore (new runtime signature)

-----> Building dependencies

Prebuild detected (node_modules already exists)

Rebuilding any native modules



> bson(a)0.2.2 install
/tmp/app/node_modules/mongoskin/node_modules/mongodb/node_modules/bson

> (node-gyp rebuild 2> builderror.log) || (exit 0)





> kerberos(a)0.0.3 install
/tmp/app/node_modules/mongoskin/node_modules/mongodb/node_modules/kerberos

> (node-gyp rebuild 2> builderror.log) || (exit 0)





> dtrace-provider(a)0.2.8 install
/tmp/app/node_modules/ldapjs/node_modules/dtrace-provider

> node-gyp rebuild



gyp: /tmp/app/.heroku/node/common.gypi not found (cwd:
/tmp/app/node_modules/ldapjs/node_modules/dtrace-provider) while reading
includes of binding.gyp while trying to load binding.gyp

gyp ERR! configure error

gyp ERR! stack Error: `gyp` failed with exit code: 1

gyp ERR! stack at ChildProcess.onCpExit
(/tmp/app/.heroku/node/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:355:16)

gyp ERR! stack at ChildProcess.emit (events.js:110:17)

gyp ERR! stack at Process.ChildProcess._handle.onexit
(child_process.js:1078:12)

gyp ERR! System Linux 3.19.0-28-generic

gyp ERR! command "node"
"/tmp/app/.heroku/node/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js"
"rebuild"

gyp ERR! cwd
/tmp/app/node_modules/ldapjs/node_modules/dtrace-provider

gyp ERR! node -v v0.12.13

gyp ERR! node-gyp -v v2.0.1

gyp ERR! not ok



npm ERR! Linux 3.19.0-28-generic

npm ERR! argv "node" "/tmp/app/.heroku/node/bin/npm" "rebuild"
"--nodedir=/tmp/app/.heroku/node"

npm ERR! node v0.12.13

npm ERR! npm v2.11.3

npm ERR! code ELIFECYCLE

npm ERR! dtrace-provider(a)0.2.8 install: `node-gyp rebuild`

npm ERR! Exit status 1

npm ERR!

npm ERR! Failed at the dtrace-provider(a)0.2.8 install script
'node-gyp rebuild'.

npm ERR! This is most likely a problem with the dtrace-provider
package,

npm ERR! not with npm itself.

npm ERR! Tell the author that this fails on your system:

npm ERR! node-gyp rebuild

npm ERR! You can get their info via:

npm ERR! npm owner ls dtrace-provider

npm ERR! There is likely additional logging output above.



npm ERR! Please include the following file with any support request:

npm ERR! /tmp/app/npm-debug.log

-----> Build failed



We're sorry this build is failing! You can troubleshoot common
issues here:

https://devcenter.heroku.com/articles/troubleshooting-node-deploys



Some possible problems:



- node_modules checked into source control


https://docs.npmjs.com/misc/faq#should-i-check-my-node-modules-folder-into-git



Love,

Heroku


Re: My nodejs application can't work if the version nodejs buildpack >= v1.5.9

Sam Dai
 

Have resolved this issue, the solution is change the npm version of package.json from ~2.11.3 to ~2.15.0 .
Change
```
, "engines" : {
"node" : "~0.12.7",
"npm" : "~2.11.3"
}
```
into:
```
, "engines" : {
"node" : "~0.12.7",
"npm" : "~2.15.0"
}
```


Re: How to create/start an APP in quickest way

Daniel Mikusa
 

Correct. There's a couple ways you can create the target app. Push a
dummy app or use the API directly (i.e. `cf curl` [1]). Once you create
the target app, copy source will override whatever is in the target.

Dan

[1] - http://apidocs.cloudfoundry.org/236/apps/creating_an_app.html

On Thu, May 5, 2016 at 9:27 AM, Stanley Shen <meteorping(a)gmail.com> wrote:

yes, I looked at "copy-source".

copy-source - Make a copy of app source code from one application to
another. Unless overridden, the copy-source command will restart the
application.

cf copy-source test1 cc
FAILED
App cc not found

I tried it too, is seems it needs the target APP is an existed APP, or I
did something wrong?


Re: How to create/start an APP in quickest way

Stanley Shen <meteorping@...>
 

yes, I looked at "copy-source".

copy-source - Make a copy of app source code from one application to another. Unless overridden, the copy-source command will restart the application.

cf copy-source test1 cc
FAILED
App cc not found

I tried it too, is seems it needs the target APP is an existed APP, or I did something wrong?


Re: How to create/start an APP in quickest way

Daniel Mikusa
 

Have you looked at `cf copy-source`? You can use that to copy one app to
another. You could then change service bindings, environment variables and
scale the copy of the app separately. I believe you will still need to
stage the app though, at least if you change env variables or service
bindings it is recommended.

Dan

On Thu, May 5, 2016 at 6:40 AM, Stanley Shen <meteorping(a)gmail.com> wrote:

Hello, all

As we know, right now if we want to create an APP, we need push it to CF,
which will cause us some time.
It needs to download buildpack, upload APP bits etc, it may takes several
minutes.

We have scenario that we want to launch this APP for serveral instances,
the only difference is that they are binding to different services, routes,
right now we only push them separately.

But we do want to have a quick way to do that, do we have something like
cloning an APP, and then just change something binding as we needed?

We can scale APP instance number by running "cf scale APP_NAME -i 3" for
example, and I can see the other 2 instances are started soon after running
this "scale" command, how we did that?


How to create/start an APP in quickest way

Stanley Shen <meteorping@...>
 

Hello, all

As we know, right now if we want to create an APP, we need push it to CF, which will cause us some time.
It needs to download buildpack, upload APP bits etc, it may takes several minutes.

We have scenario that we want to launch this APP for serveral instances, the only difference is that they are binding to different services, routes, right now we only push them separately.

But we do want to have a quick way to do that, do we have something like cloning an APP, and then just change something binding as we needed?

We can scale APP instance number by running "cf scale APP_NAME -i 3" for example, and I can see the other 2 instances are started soon after running this "scale" command, how we did that?

4621 - 4640 of 9426