cf-dev@lists.cloudfoundry.org

Re: diego questions: default memory and passing env variables

Amit Kumar Gupta

#1469

This is not a environment variable you can set.  To scale down the memory
allocated to your instances, do

cf scale <app-name> -m 1024M

This will tell Warden/Garden to allocate that much memory to your
containers and kill them if they exceed it.  The documentation is saying
that the processes running inside the container can learn about the memory
limit applied to them via the environment variable MEMORY_LIMIT.  If you
try to set this variable via set-env, it will get overwritten by the value
set by "cf scale".  Again, the MEMORY_LIMIT environment variable doesn't do
anything, it's just information passed into the container via the process's
environment.  To actually constrain the memory, use "cf scale" or set it in
your manifest file:

https://docs.cloudfoundry.org/devguide/deploy-apps/manifest.html#memory

Show quoted text

On Wed, Sep 2, 2015 at 3:08 PM, Siva Balan <mailsiva(a)gmail.com> wrote:

We are not using Diego and we are on CF version 211.
I had my application started with 2GB of memory set in the manifest file.
I then ran the following command and restaged the app:
cf set-env <app-name> MEMORY_LIMIT 1024m
Then when I ran "cf app <app-name>", it was still showing the instance
memory of 2GB but the heap size allocated to the application was cut in
half and I confirmed this through New Relic app. Also, the first full GC of
the app occurred when the memory of the instance was at 1.3GB. So I am not
quite sure of the description of the MEMORY_LIMIT env variable as described
in
http://docs.run.pivotal.io/devguide/deploy-apps/environment-variable.html#memory.
I did not see my application restart when the instance memory went over
1GB.
Any ideas about this behavior?

Thanks.
Siva

On Tue, Sep 1, 2015 at 7:40 PM, Amit Gupta <agupta(a)pivotal.io> wrote:

Responses inline.

On Tue, Sep 1, 2015 at 7:13 PM, Tom Sherrod <tom.sherrod(a)gmail.com>
wrote:

What is the default memory size of an app deployed in diego? cf apps
indicate 1GB.
Yes.  This isn't a diego thing, this is a CC thing.


cf set-env doesn't appear to be getting the environment variable
information to the running container. Is cf set-env the correct method to
use?
You likely need to restage the application for ENV var changes to take
effect.  Unless I'm mistaken, this too is not diego-specific, and applies
to DEA containers as well.


Thanks!
Tom
Best,
Amit

attachment.html

Re: diego questions: default memory and passing env variables

Matthew Sykes <matthew.sykes@...>

#1468

Memory limits are set via cf scale.

`cf scale <app-name> -m 512M`

will scale your app to 512MB per instance. The cli will then tell you to
restage your application so the buildpacks can observe the new limit and
make the necessary changes. At one time, this was especially true of the
java buildpack.

Show quoted text

On Wed, Sep 2, 2015 at 6:08 PM, Siva Balan <mailsiva(a)gmail.com> wrote:

We are not using Diego and we are on CF version 211.
I had my application started with 2GB of memory set in the manifest file.
I then ran the following command and restaged the app:
cf set-env <app-name> MEMORY_LIMIT 1024m
Then when I ran "cf app <app-name>", it was still showing the instance
memory of 2GB but the heap size allocated to the application was cut in
half and I confirmed this through New Relic app. Also, the first full GC of
the app occurred when the memory of the instance was at 1.3GB. So I am not
quite sure of the description of the MEMORY_LIMIT env variable as described
in
http://docs.run.pivotal.io/devguide/deploy-apps/environment-variable.html#memory.
I did not see my application restart when the instance memory went over
1GB.
Any ideas about this behavior?

Thanks.
Siva

On Tue, Sep 1, 2015 at 7:40 PM, Amit Gupta <agupta(a)pivotal.io> wrote:

Responses inline.

On Tue, Sep 1, 2015 at 7:13 PM, Tom Sherrod <tom.sherrod(a)gmail.com>
wrote:

What is the default memory size of an app deployed in diego? cf apps
indicate 1GB.
Yes.  This isn't a diego thing, this is a CC thing.


cf set-env doesn't appear to be getting the environment variable
information to the running container. Is cf set-env the correct method to
use?
You likely need to restage the application for ENV var changes to take
effect.  Unless I'm mistaken, this too is not diego-specific, and applies
to DEA containers as well.


Thanks!
Tom
Best,
Amit

-- 
Matthew Sykes
matthew.sykes(a)gmail.com

attachment.html

Re: diego questions: default memory and passing env variables

Siva Balan <mailsiva@...>

#1467

We are not using Diego and we are on CF version 211.
I had my application started with 2GB of memory set in the manifest file. I
then ran the following command and restaged the app:
cf set-env <app-name> MEMORY_LIMIT 1024m
Then when I ran "cf app <app-name>", it was still showing the instance
memory of 2GB but the heap size allocated to the application was cut in
half and I confirmed this through New Relic app. Also, the first full GC of
the app occurred when the memory of the instance was at 1.3GB. So I am not
quite sure of the description of the MEMORY_LIMIT env variable as described
in
http://docs.run.pivotal.io/devguide/deploy-apps/environment-variable.html#memory.
I did not see my application restart when the instance memory went over
1GB.
Any ideas about this behavior?

Thanks.
Siva

Show quoted text

On Tue, Sep 1, 2015 at 7:40 PM, Amit Gupta <agupta(a)pivotal.io> wrote:

Responses inline.

On Tue, Sep 1, 2015 at 7:13 PM, Tom Sherrod <tom.sherrod(a)gmail.com> wrote:

What is the default memory size of an app deployed in diego? cf apps
indicate 1GB.
Yes.  This isn't a diego thing, this is a CC thing.


cf set-env doesn't appear to be getting the environment variable
information to the running container. Is cf set-env the correct method to
use?
You likely need to restage the application for ENV var changes to take
effect.  Unless I'm mistaken, this too is not diego-specific, and applies
to DEA containers as well.


Thanks!
Tom
Best,
Amit

attachment.html

Re: Generic data points for dropsonde

Johannes Tuchscherer

#1466

The current way of sending metrics as either Values or Counters through the
pipeline makes the development of a downstream consumer (=nozzle) pretty
easy. If you look at the datadog nozzle[0], it just takes all ValueMetrics
and Counters and sends them off to datadog. The nozzle does not have to
know anything about these metrics (e.g. their origin, name, or layout).

Adding a new way to send metrics as a nested object would make the
downstream implementation certainly more complicated. In that case, the
nozzle developer has to know what metrics are included inside the generic
point (basically the schema of the metric) and treat each point
accordingly. For example, if I were to write a nozzle that emits metrics to
Graphite with a StatsD client (like it is done here[1]), I need to know if
my int64 value is a Gauge or a Counter. Also, my consumer is under constant
risk of breaking when the upstream schema changes.

We are already facing this problem with the container metrics. But at least
the container metrics are in a defined format that is well documented and
not likely to change.

I agree with you, though, the the dropsonde protocol could use a mechanism
for easier extension. Having a GenericPoint and/or GenericEvent seems like
a good idea that I whole-heartedly support. I would just like to stay away
from nested metrics. I think the cost of adding more logic into the
downstream consumer (and making it harder to maintain) is not worth the
benefit of a more concise metric transport.


[0] https://github.com/cloudfoundry-incubator/datadog-firehose-nozzle
[1] https://github.com/CloudCredo/graphite-nozzle

Show quoted text

On Tue, Sep 1, 2015 at 5:52 PM, Benjamin Black <bblack(a)pivotal.io> wrote:

great questions, dwayne.

1) the partition key is intended to be used in a similar manner to
partitioners in distributed systems like cassandra or kafka. the specific
behavior i would like to make part of the contract is two-fold: that all
data with the same key is routed to the same partition and that all data in
a partition is FIFO (meaning no ordering guarantees beyond arrival time).

this could help with the multi-line log problem by ensuring a single
consumer will receive all lines for a given log entry in order, allowing
simple reassembly. however, the lines might be interleaved with other lines
with the same key or even other keys that happen to map to the same
partition, so the consumer does require a bit of intelligence. this was
actually one of the driving scenarios for adding the key.

2) i expect typical points to be in the hundreds of bytes to a few KB. if
we find ourselves regularly needing much larger points, especially near
that 64KB limit, i'd look to the JSON representation as the hierarchical
structure is more efficiently managed there.


b




On Tue, Sep 1, 2015 at 4:42 PM, <dschultz(a)pivotal.io> wrote:

Hi Ben,

I was wondering if you could give a concrete use case for the partition
key functionality.

In particular I am interested in how we solve multi line log entries. I
think it would be better to solve it by keeping all the data (the multiple
lines) together throughout the logging/metrics pipeline, but could see how
something like a partition key might help keep the data together as well.

Second question: how large do you see these point messages getting
(average and max)? There are still several stages of the logging/metrics
pipeline that use UDP with a standard 64K size limit.

Thanks,
Dwayne

On Aug 28, 2015, at 4:54 PM, Benjamin Black <bblack(a)pivotal.io> wrote:

All,

The existing dropsonde protocol uses a different message type for each
event type. HttpStart, HttpStop, ContainerMetrics, and so on are all
distinct types in the protocol definition. This requires protocol changes
to introduce any new event type, making such changes very expensive. We've
been working for the past few weeks on an addition to the dropsonde
protocol to support easier future extension to new types of events and to
make it easier for users to define their own events.

The document linked below [1] describes a generic data point message
capable of carrying multi-dimensional, multi-metric points as sets of
name/value pairs. This new message is expected to be added as an additional
entry in the existing dropsonde protocol metric type enum. Things are now
at a point where we'd like to get feedback from the community before moving
forward with implementation.

Please contribute your thoughts on the document in whichever way you are
most comfortable: comments on the document, email here, or email directly
to me. If you comment on the document, please make sure you are logged in
so we can keep track of who is asking for what. Your views are not just
appreciated, but critical to the continued health and success of the Cloud
Foundry community. Thank you!


b

[1]
https://docs.google.com/document/d/1SzvT1BjrBPqUw6zfSYYFfaW9vX_dTZZjn5sl2nxB6Bc/edit?usp=sharing

attachment.html

Re: Correcting CF Docs

Joaquin Zalazar <joaquin.zalazar@...>

#1465

Hi Daniel!

What I've done while testing is to set the variable BOSH_AWS_REGION to the
region I needed.

export BOSH_AWS_REGION=<your region here>

Regards!!

Joaquin Zalazar

Altoros


On Wed, Sep 2, 2015 at 9:43 AM, Daniel Jones <
daniel.jones(a)engineerbetter.com> wrote:

Hi all,

The CF docs for deploying CF to an AWS VPC
<https://docs.cloudfoundry.org/deploying/ec2/bootstrap-aws-vpc.html> have
a broken link to a Gist of Dan Highham's, that presumably talks about
deploying to regions other than us-east-1.

Does anyone know where the instructions for non-US regions are now?

Regards,

Daniel Jones
EngineerBetter.com


-- 

*Joaquin Zalazar*
*ALTOROS* — Cloud Foundry deployment, training and integration
www.altoros.com

attachment.html

Re: docker-push failed with crashing and got the following stack trace in grarden-linux

Shaozhen Ding

#1464

Thanks for the help

I had some permission issues on several files. Since diego run docker
containers with user vcap. I created a user named as vcap and gave the file
permission to vcap in advanced. Then i got the error above, even though I
created the home folder.

I remove the user, then the error is gone. Definitely curious of what diego
is trying to do with vcap user.

Show quoted text

On Wed, Sep 2, 2015 at 11:40 AM, Will Pragnell <wpragnell(a)pivotal.io> wrote:

The error message suggests that in `/etc/passwd` in the Docker image, the
entry for `vcap` lists a path for the home dir that doesn't exist. I'm
afraid I don't understand Diego well enough to know why it's trying to
stream into the container as `vcap` though.

On 2 September 2015 at 17:25, Shaozhen Ding <dsz0111(a)gmail.com> wrote:

Anyone can help with interpreting the error log here
garden-linux.pool.vl5hraecvg1.stream-in.command.failed

cf-213 and diego/0.1353.0


{"timestamp":"1441209896.800271988","source":"garden-linux","message":"garden-linux.garden-server.create.created","log_level":1,"data":{"request":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","rootfs":"docker:///datianshi/cf-docker-music#latest","properties":{"executor:allocated-at":"1441209893185279557","executor:cpu-weight":"4","executor:disk-mb":"1024","executor:log-config":"{\"guid\":\"cb6ac15a-c3a0-4b5d-9716-e99934c35953\",\"index\":0,\"source_name\":\"CELL\"}","executor:memory-mb":"512","executor:metrics-config":"{\"guid\":\"cb6ac15a-c3a0-4b5d-9716-e99934c35953\",\"index\":0}","executor:owner":"executor","executor:result":"{\"failed\":false,\"failure_reason\":\"\",\"stopped\":false}","executor:rootfs":"docker:///datianshi/cf-docker-music#latest","executor:start-timeout":"60","executor:state":"created","tag:domain":"cf-apps","tag:instance-guid":"397eef75-bf73-46d3-6ba2-6a3737ea3d5e","tag:lifecyc
 le":"lrp

 ","tag:process-guid":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0","tag:process-index":"0"},"env":["INSTANCE_GUID=397eef75-bf73-46d3-6ba2-6a3737ea3d5e","INSTANCE_INDEX=0"]},"session":"4.247804"}}

{"timestamp":"1441209896.811697721","source":"garden-linux","message":"garden-linux.garden-server.net-in.port-mapped","log_level":1,"data":{"container-port":8080,"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","host-port":61055,"session":"4.247809"}}

{"timestamp":"1441209896.818696976","source":"garden-linux","message":"garden-linux.garden-server.net-in.port-mapped","log_level":1,"data":{"container-port":2222,"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","host-port":61056,"session":"4.247810"}}

{"timestamp":"1441209896.842704535","source":"garden-linux","message":"garden-linux.garden-server.limit-memory.limited","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","resulting-limits":{"limit_in_bytes":536870912},"session":"4.247818"}}

{"timestamp":"1441209896.843993425","source":"garden-linux","message":"garden-linux.garden-server.limit-disk.limited","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","resulting-limits":{},"session":"4.247819"}}

{"timestamp":"1441209896.844884396","source":"garden-linux","message":"garden-linux.garden-server.limit-cpu.limited","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","resulting-limits":{"limit_in_shares":40},"session":"4.247820"}}

{"timestamp":"1441209896.845991373","source":"garden-linux","message":"garden-linux.garden-server.info.got-info","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","session":"4.247821"}}
{"timestamp":"1441209896.857844830","source":"garden-linux","message":"garden-linux.pool.vl5hraecvg1.stream-in.command.failed","log_level":2,"data":{"argv":["/var/vcap/data/garden-linux/depot/vl5hraecvg1/bin/nstar","11856","vcap","/tmp/lifecycle"],"error":"exit
status 1","exit-status":1,"session":"2.67.7.1","stderr":"chdir to user
home: No such file or directory\n","stdout":"","took":"2.959157ms"}}
{"timestamp":"1441209896.858107328","source":"garden-linux","message":"garden-linux.garden-server.stream-in.failed","log_level":2,"data":{"destination":"/tmp/lifecycle","error":"exit
status
1","handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","session":"4.247823"}}

attachment.html

Re: v3 cc api style guide feedback requested

Zach Robinson

#1463

Thanks James, I've just corrected the three issues you've noted so far

Re: docker-push failed with crashing and got the following stack trace in grarden-linux

Will Pragnell <wpragnell@...>

#1462

The error message suggests that in `/etc/passwd` in the Docker image, the
entry for `vcap` lists a path for the home dir that doesn't exist. I'm
afraid I don't understand Diego well enough to know why it's trying to
stream into the container as `vcap` though.

Show quoted text

On 2 September 2015 at 17:25, Shaozhen Ding <dsz0111(a)gmail.com> wrote:

Anyone can help with interpreting the error log here
garden-linux.pool.vl5hraecvg1.stream-in.command.failed

cf-213 and diego/0.1353.0


{"timestamp":"1441209896.800271988","source":"garden-linux","message":"garden-linux.garden-server.create.created","log_level":1,"data":{"request":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","rootfs":"docker:///datianshi/cf-docker-music#latest","properties":{"executor:allocated-at":"1441209893185279557","executor:cpu-weight":"4","executor:disk-mb":"1024","executor:log-config":"{\"guid\":\"cb6ac15a-c3a0-4b5d-9716-e99934c35953\",\"index\":0,\"source_name\":\"CELL\"}","executor:memory-mb":"512","executor:metrics-config":"{\"guid\":\"cb6ac15a-c3a0-4b5d-9716-e99934c35953\",\"index\":0}","executor:owner":"executor","executor:result":"{\"failed\":false,\"failure_reason\":\"\",\"stopped\":false}","executor:rootfs":"docker:///datianshi/cf-docker-music#latest","executor:start-timeout":"60","executor:state":"created","tag:domain":"cf-apps","tag:instance-guid":"397eef75-bf73-46d3-6ba2-6a3737ea3d5e","tag:lifecyc
 le":"lrp

 ","tag:process-guid":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0","tag:process-index":"0"},"env":["INSTANCE_GUID=397eef75-bf73-46d3-6ba2-6a3737ea3d5e","INSTANCE_INDEX=0"]},"session":"4.247804"}}

{"timestamp":"1441209896.811697721","source":"garden-linux","message":"garden-linux.garden-server.net-in.port-mapped","log_level":1,"data":{"container-port":8080,"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","host-port":61055,"session":"4.247809"}}

{"timestamp":"1441209896.818696976","source":"garden-linux","message":"garden-linux.garden-server.net-in.port-mapped","log_level":1,"data":{"container-port":2222,"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","host-port":61056,"session":"4.247810"}}

{"timestamp":"1441209896.842704535","source":"garden-linux","message":"garden-linux.garden-server.limit-memory.limited","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","resulting-limits":{"limit_in_bytes":536870912},"session":"4.247818"}}

{"timestamp":"1441209896.843993425","source":"garden-linux","message":"garden-linux.garden-server.limit-disk.limited","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","resulting-limits":{},"session":"4.247819"}}

{"timestamp":"1441209896.844884396","source":"garden-linux","message":"garden-linux.garden-server.limit-cpu.limited","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","resulting-limits":{"limit_in_shares":40},"session":"4.247820"}}

{"timestamp":"1441209896.845991373","source":"garden-linux","message":"garden-linux.garden-server.info.got-info","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","session":"4.247821"}}
{"timestamp":"1441209896.857844830","source":"garden-linux","message":"garden-linux.pool.vl5hraecvg1.stream-in.command.failed","log_level":2,"data":{"argv":["/var/vcap/data/garden-linux/depot/vl5hraecvg1/bin/nstar","11856","vcap","/tmp/lifecycle"],"error":"exit
status 1","exit-status":1,"session":"2.67.7.1","stderr":"chdir to user
home: No such file or directory\n","stdout":"","took":"2.959157ms"}}
{"timestamp":"1441209896.858107328","source":"garden-linux","message":"garden-linux.garden-server.stream-in.failed","log_level":2,"data":{"destination":"/tmp/lifecycle","error":"exit
status
1","handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","session":"4.247823"}}

attachment.html

Re: v3 cc api style guide feedback requested

James Bayer

#1461

the example used for actions uses the v2 api instead of v3:
https://github.com/cloudfoundry/cc-api-v3-style-guide#example-4

i like the idea of a unique error code. ideally the CAPI team maintains
public documentation of an error code dictionary
https://github.com/cloudfoundry/cc-api-v3-style-guide#issues-with-v2-error-format

the async proposal seems like a big improvement.

Show quoted text

On Wed, Sep 2, 2015 at 8:53 AM, James Bayer <jbayer(a)pivotal.io> wrote:

the collections example [1] does not actually include the required
*resources* field

[1] https://github.com/cloudfoundry/cc-api-v3-style-guide#example-2

On Wed, Sep 2, 2015 at 8:16 AM, James Bayer <jbayer(a)pivotal.io> wrote:

should the PUT example that updates the app name and space guid actually
be a PATCH since it updates the resource?
https://github.com/cloudfoundry/cc-api-v3-style-guide#put

On Wed, Sep 2, 2015 at 1:40 AM, Dieu Cao <dcao(a)pivotal.io> wrote:

Hi All,

The CAPI team has created a style guide for v3 of the cloud controller
api and would like to share this again with the wider cf community for
feedback after initial review within the team. [1]
Issues/PRs are welcome and appreciated!

Thanks,
Dieu
CF CAPI PM

[1] https://github.com/cloudfoundry/cc-api-v3-style-guide

--
Thank you,

James Bayer

--
Thank you,

James Bayer

-- 
Thank you,

James Bayer

attachment.html

docker-push failed with crashing and got the following stack trace in grarden-linux

Shaozhen Ding

#1460

Anyone can help with interpreting the error log here
garden-linux.pool.vl5hraecvg1.stream-in.command.failed

cf-213 and diego/0.1353.0

{"timestamp":"1441209896.800271988","source":"garden-linux","message":"garden-linux.garden-server.create.created","log_level":1,"data":{"request":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","rootfs":"docker:///datianshi/cf-docker-music#latest","properties":{"executor:allocated-at":"1441209893185279557","executor:cpu-weight":"4","executor:disk-mb":"1024","executor:log-config":"{\"guid\":\"cb6ac15a-c3a0-4b5d-9716-e99934c35953\",\"index\":0,\"source_name\":\"CELL\"}","executor:memory-mb":"512","executor:metrics-config":"{\"guid\":\"cb6ac15a-c3a0-4b5d-9716-e99934c35953\",\"index\":0}","executor:owner":"executor","executor:result":"{\"failed\":false,\"failure_reason\":\"\",\"stopped\":false}","executor:rootfs":"docker:///datianshi/cf-docker-music#latest","executor:start-timeout":"60","executor:state":"created","tag:domain":"cf-apps","tag:instance-guid":"397eef75-bf73-46d3-6ba2-6a3737ea3d5e","tag:lifecycle":"lrp
 ","tag:process-guid":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0","tag:process-index":"0"},"env":["INSTANCE_GUID=397eef75-bf73-46d3-6ba2-6a3737ea3d5e","INSTANCE_INDEX=0"]},"session":"4.247804"}}
{"timestamp":"1441209896.811697721","source":"garden-linux","message":"garden-linux.garden-server.net-in.port-mapped","log_level":1,"data":{"container-port":8080,"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","host-port":61055,"session":"4.247809"}}
{"timestamp":"1441209896.818696976","source":"garden-linux","message":"garden-linux.garden-server.net-in.port-mapped","log_level":1,"data":{"container-port":2222,"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","host-port":61056,"session":"4.247810"}}
{"timestamp":"1441209896.842704535","source":"garden-linux","message":"garden-linux.garden-server.limit-memory.limited","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","resulting-limits":{"limit_in_bytes":536870912},"session":"4.247818"}}
{"timestamp":"1441209896.843993425","source":"garden-linux","message":"garden-linux.garden-server.limit-disk.limited","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","resulting-limits":{},"session":"4.247819"}}
{"timestamp":"1441209896.844884396","source":"garden-linux","message":"garden-linux.garden-server.limit-cpu.limited","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","resulting-limits":{"limit_in_shares":40},"session":"4.247820"}}
{"timestamp":"1441209896.845991373","source":"garden-linux","message":"garden-linux.garden-server.info.got-info","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","session":"4.247821"}}
{"timestamp":"1441209896.857844830","source":"garden-linux","message":"garden-linux.pool.vl5hraecvg1.stream-in.command.failed","log_level":2,"data":{"argv":["/var/vcap/data/garden-linux/depot/vl5hraecvg1/bin/nstar","11856","vcap","/tmp/lifecycle"],"error":"exit status 1","exit-status":1,"session":"2.67.7.1","stderr":"chdir to user home: No such file or directory\n","stdout":"","took":"2.959157ms"}}
{"timestamp":"1441209896.858107328","source":"garden-linux","message":"garden-linux.garden-server.stream-in.failed","log_level":2,"data":{"destination":"/tmp/lifecycle","error":"exit status 1","handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","session":"4.247823"}}

Re: v3 cc api style guide feedback requested

James Bayer

#1459

the collections example [1] does not actually include the required
*resources* field

[1] https://github.com/cloudfoundry/cc-api-v3-style-guide#example-2

Show quoted text

On Wed, Sep 2, 2015 at 8:16 AM, James Bayer <jbayer(a)pivotal.io> wrote:

should the PUT example that updates the app name and space guid actually
be a PATCH since it updates the resource?
https://github.com/cloudfoundry/cc-api-v3-style-guide#put


On Wed, Sep 2, 2015 at 1:40 AM, Dieu Cao <dcao(a)pivotal.io> wrote:

Hi All,

The CAPI team has created a style guide for v3 of the cloud controller
api and would like to share this again with the wider cf community for
feedback after initial review within the team. [1]
Issues/PRs are welcome and appreciated!

Thanks,
Dieu
CF CAPI PM

[1] https://github.com/cloudfoundry/cc-api-v3-style-guide


--
Thank you,

James Bayer

-- 
Thank you,

James Bayer

attachment.html

CF CLI Release v 6.12.3

Greg Oehmen

#1458

The CF CLI team just cut 6.12.3.  Release notes and binaries are available
at:

https://github.com/cloudfoundry/cli/releases


Highlights of this release include:

Bug Fix:  runtime error: index out of range

originally reported here https://github.com/cloudfoundry/cli/issues/509 and
in many subsequent Github issues

   -

   https://www.pivotaltracker.com/story/show/98672332
   -

   The root problem for this particular crash is because
   v2/apps/GUID/instances reports 0 instance for a newly stopped app, but
   GetContainerMetrics() from noaa library will continue to report instances
   metrics for a while, and eventually be consistent with
   v2/apps/GUID/instance. This causes the CLI’s 'index out of range error'.
   The bug is fixed by not gathering metrics unless instance>0, but
   GetContainerMetrics() should be consistent with what CC endpoint reports.




Pull Requests


   -

   Merge pull request #544 <https://github.com/cloudfoundry/cli/pull/544>
   from cloudfoundry/code-tidy Code tidy:  Code cleanup: remove unused
   variables, remove orphan functions, shadowing reserved word
   -

   Merge pull request #523 <https://github.com/cloudfoundry/cli/pull/523>
   from zachgersh/master Unmarshal the extra field, get documentation url
   -

   Merge pull request #540 <https://github.com/cloudfoundry/cli/pull/540>
   from cloudfoundry/use_go_yaml Support yaml '<<' merge type
   -

   Merge pull request #534 <https://github.com/cloudfoundry/cli/pull/534>
   from cloudfoundry/feature/commands-restart-and-create Move commands to new
   command pattern.
   -

   Merge pull request #505 <https://github.com/cloudfoundry/cli/pull/505>
   from zhang-hua/bug-93578300 Reduce API calls when CRU operations of service
   keys
   -

   Merge branch 'story-87481016' of https://github.com/zhang-hua/cli into
   zhang-hua-story-87481016
   -

   Merge pull request #514 <https://github.com/cloudfoundry/cli/pull/514>
   Fix create-app-manifest only includes one host [92530254]




Also notable:

We deprecated the use of CodeGangsta within the CF CLI.  Previously,
CodeGangsta had been used for managing help and flags within the CLI.  We
now manage those functions natively in the CLI.  This is a precursor to the
work we will begin soon of refactoring help, restructuring command syntax
and abstracting admin-only commands out into some separate manifestation;
perhaps a plugin.



Greg Oehmen
Cloud Foundry Product Manager

attachment.html

Re: v3 cc api style guide feedback requested

James Bayer

#1457

should the PUT example that updates the app name and space guid actually be
a PATCH since it updates the resource?
https://github.com/cloudfoundry/cc-api-v3-style-guide#put

Show quoted text

On Wed, Sep 2, 2015 at 1:40 AM, Dieu Cao <dcao(a)pivotal.io> wrote:

Hi All,

The CAPI team has created a style guide for v3 of the cloud controller api
and would like to share this again with the wider cf community for feedback
after initial review within the team. [1]
Issues/PRs are welcome and appreciated!

Thanks,
Dieu
CF CAPI PM

[1] https://github.com/cloudfoundry/cc-api-v3-style-guide

-- 
Thank you,

James Bayer

attachment.html

Re: stdout.log and stderr.log not show in CF197 with loggregator enabled

Erik Jasiak

#1456

Dan is correct - "cf file" for getting stdout/stderr files directly was 
deprecated some time ago. I'll try to locate when that happened (it was 
before my time.)

 From the cli, "cf logs" for streaming, and "cf logs --recent" for a 
quick dump are the way to go.

Erik
PM - loggregator

Daniel Mikusa wrote:

Show quoted text

Have you tried running `cf logs` instead? Anything written to STDOUT
or STDERR should be visible there for some period of time. For long
term storage, you should look at setting up a log drain on your
application.

http://docs.cloudfoundry.org/devguide/services/log-management.html

Dan

On Wed, Sep 2, 2015 at 9:08 AM, Shruthi Ravindra
<Shruthi.ravindra(a)ge.com <mailto:Shruthi.ravindra(a)ge.com>> wrote:

Hi,

we are also facing the same issue when we check the "cf files
pfh-cos-dicomobjectstore-dev /logs" we are able to see only the
staging logs and not the stdout.log abd stderror.log.
we need to access these logs to debug the error.
how do we enable the stdout and stderror logs

attachment.html

Re: stdout.log and stderr.log not show in CF197 with loggregator enabled

Daniel Mikusa

#1454

Have you tried running `cf logs` instead?  Anything written to STDOUT or
STDERR should be visible there for some period of time.  For long term
storage, you should look at setting up a log drain on your application.

http://docs.cloudfoundry.org/devguide/services/log-management.html

Dan

On Wed, Sep 2, 2015 at 9:08 AM, Shruthi Ravindra <Shruthi.ravindra(a)ge.com>
wrote:

Hi,

we are also facing the same issue when we check the "cf files
pfh-cos-dicomobjectstore-dev /logs" we are able to see only the staging
logs and not the stdout.log abd stderror.log.
we need to access these logs to debug the error.
how do we enable the stdout and stderror logs

attachment.html

Re: stdout.log and stderr.log not show in CF197 with loggregator enabled

Shruthi Ravindra

#1453

Hi,

we are also facing the same issue when we check the "cf files pfh-cos-dicomobjectstore-dev /logs" we are able to see only the staging logs and not the stdout.log abd stderror.log.
we need to access these logs to debug the error.
how do we enable the stdout and stderror logs

Re: stdout.log and stderr.log not show in CF197 with loggregator enabled

Shruthi Ravindra

#1455

Hi ,

We are having the same problem where we are not able to see the stdout.log and stderror.log for all the application deployed in the cloud.
Could you please let me know how do we access the logs other than cf logs "appname" command which is not giving us the full logs.

Thanks,
Shruthi

attachment.html

Correcting CF Docs

Daniel Jones

#1452

Hi all,

The CF docs for deploying CF to an AWS VPC
<https://docs.cloudfoundry.org/deploying/ec2/bootstrap-aws-vpc.html> have a
broken link to a Gist of Dan Highham's, that presumably talks about
deploying to regions other than us-east-1.

Does anyone know where the instructions for non-US regions are now?

Regards,

Daniel Jones
EngineerBetter.com

attachment.html

Re: Announcing Experimental support for Asynchronous Service Operations

Guillaume Berche

#1451

Thanks Dieu for your response.

We have successfully implemented an async service broker for out internal
dbaas [1], [2] which db farms provisionning sometimes takes longer than
30s.

Concerning improvement suggestions on the documentation, I could only spot
an inconsistency on the sequence diagram, where the last operation
description is mismatching between broker response (1/3 nodes") and CC api
response ("40% complete") whereas the 'description' is a string which
should be returned as is. Hard to propose a PR on that since the source
file for this diagram does not seem available under docs-services [4]. BTW,
It would be great if bookbinder would support rendering diagrams from a
text format such as plantuml [5], or rely on plantuml public service [6]
for such rendering.

Wouldn't is make sense to exit the experimental status of the async api,
even if it's not feature complete ? i.e. considering the api is stable
enough and won't change in an incompatible maneer soon, thereby enabling
clients such as cf-java-client to rely on them.

One other potential improvement I see to the API would be to support
localized user-facing operation status description. Having the CLI being
I18Ned but brokers enable to return user facing messages in the user
language seems limiting. Maybe it could be the case of having the CLI add a
usual 'Accept-Language: zh_Hans" matching the current CLI locale, and the
CC propagating this header in the last_operation broker endpoint [7] so
that they can return the status message in the smae language when
translations are available.

Thanks,

Guillaume.

[1]
https://github.com/Orange-OpenSource/elpaaso-brokers/blob/master/cf-service-broker-dbaas/src/main/java/com/orange/clara/cloud/cf/servicebroker/dbaas/service/DbaasServiceInstanceService.java#L60
[2]
https://github.com/Orange-OpenSource/elpaaso-brokers/blob/a3f150fc7d3bb889875aac202496a4f63efc3595/cf-service-broker-dbaas/src/main/java/com/orange/clara/cloud/cf/servicebroker/dbaas/service/DbaasServiceInstanceService.java#L67-L73
[3]
http://docs.cloudfoundry.org/services/images/async-service-broker-flow.png
[4]
https://github.com/cloudfoundry/docs-services/blob/master/images/async-service-broker-flow.png
[5] http://plantuml.com/sequence.html
[6] http://plantuml.com/plantuml/
[7]
http://docs.cloudfoundry.org/services/asynchronous-operations.html#polling

Show quoted text

On Tue, Sep 1, 2015 at 5:50 PM, Dieu Cao <dcao(a)pivotal.io> wrote:

Hi Guillaume,

We are looking for a couple more pieces of feedback.  All feedback I've
received so far has been positive.

There was one report recently from Robert Moss about being able to provide
the dashboard_url asynchronously instead of part of the original response
[1].
I'm also considering whether that needs to be addressed prior to marking
the feature complete, or if it should be a separate/additive change to the
service broker api.

If you've had success with the current implementation, I would be greatly
interested in hearing that.

-Dieu

[1]
https://lists.cloudfoundry.org/archives/list/cf-dev(a)lists.cloudfoundry.org/thread/LJ26K22WG7OJCAIXWYPTJQKELVAUBUZC/

On Tue, Sep 1, 2015 at 5:20 AM, Guillaume Berche <bercheg(a)gmail.com>
wrote:

Hi,

I'm wondering whether the experimental async service broker api is now
ready to become official, or whether the CAPI team and PMs are still
lacking feedback or have some outstanding issues that need to be addressed
before calling the spec official? I was not able to spot any async-related
stories in the CAPI backlog [1]

We have a pending PR in cf-java-client [2] which is waiting for the async
broker spec to be made official.

Thanks in advance,

Guillaume.

[1] https://www.pivotaltracker.com/n/projects/966314
[2]
https://github.com/cloudfoundry/cf-java-client/pull/282#issuecomment-115874984

On Fri, Jun 5, 2015 at 9:21 AM, James Bayer <jbayer(a)pivotal.io> wrote:

i'm very happy to see this work delivered as the 60 second limit has
come up so often as a pain point in the past. great job to all who
contributed!

On Wed, Jun 3, 2015 at 5:05 PM, Onsi Fakhouri <ofakhouri(a)pivotal.io>
wrote:

Well done Services API!  This is an awesome milestone!

On Wed, Jun 3, 2015 at 5:04 PM, Chip Childers <
cchilders(a)cloudfoundry.org> wrote:

Awesome news! Long time coming, and it opens up a whole world of
additional capabilities for users.

Nice work everyone!



On Jun 4, 2015, at 9:00 AM, Shannon Coen <scoen(a)pivotal.io> wrote:

On behalf of the Services API team, including Dojo participants from
IBM and SAP, I'm pleased to announce experimental availability and
published documentation for this much-anticipated feature.

As of cf-release v208 and CLI v6.11.1, Cloud Foundry now supports an
enhanced service broker integration in support of long-running
provisioning, update, and delete operations. This significantly broadens
the supported use cases for Cloud Foundry Marketplace Services, and I can't
wait to hear what creative things the ecosystem does with it. Provision
VMs, orchestrate clusters, install software, move data... yes, your broker
can even open support tickets to have those things done manually!

This feature is currently considered experimental, as we'd like you
all to review our docs, try out the feature, and give us feedback. We very
interested to hear about any confusion in the docs or the UX, and any
sticky issues you encounter in implementation. Our goal is for our docs
enable a painless, intuitive (can we hope for joyful?) implementation
experience.

We have not bumped the broker API yet for this feature. You'll notice
that our documentation for the feature is separate from the stable API docs
at this point. Once we're confident in the design (we're relying on your
feedback!), we'll bump the broker API version, move the docs for
asynchronous operations into the stable docs, AND implement support for
asynchronous bind/create-key and unbind/delete-key.

Documentation:
- http://docs.cloudfoundry.org/services/asynchronous-operations.html
- http://docs.cloudfoundry.org/services/api.html
Example broker for AWS (contributed by IBM):
- http://docs.cloudfoundry.org/services/examples.html
- https://github.com/cloudfoundry-samples/go_service_broker
Demo of the feature presented at CF Summit 2015:
- https://youtu.be/Ij5KSKrAq9Q

tl;dr

Cloud Foundry expects broker responses within 60 seconds. Now a broker
can return an immediate response indicating that a provision, update, or
delete operation is in progress. Cloud Foundry then returns a similar
response to the client, and begins polling the broker for the status of the
operation. Users, via API clients, can discover the status of the operation
("in progress", "succeeded", or "failed"), and brokers can provide
user-facing messages in response to each poll which are exposed to users
(e.g. "VMs provisioned, installing software, 30% complete").

Thank you,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Thank you,

James Bayer

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

attachment.html

v3 cc api style guide feedback requested

Dieu Cao <dcao@...>

#1450

Hi All,

The CAPI team has created a style guide for v3 of the cloud controller api
and would like to share this again with the wider cf community for feedback
after initial review within the team. [1]
Issues/PRs are welcome and appreciated!

Thanks,
Dieu
CF CAPI PM

[1] https://github.com/cloudfoundry/cc-api-v3-style-guide

attachment.html