Date   

Re: Generic data points for dropsonde

Johannes Tuchscherer
 

The current way of sending metrics as either Values or Counters through the
pipeline makes the development of a downstream consumer (=nozzle) pretty
easy. If you look at the datadog nozzle[0], it just takes all ValueMetrics
and Counters and sends them off to datadog. The nozzle does not have to
know anything about these metrics (e.g. their origin, name, or layout).

Adding a new way to send metrics as a nested object would make the
downstream implementation certainly more complicated. In that case, the
nozzle developer has to know what metrics are included inside the generic
point (basically the schema of the metric) and treat each point
accordingly. For example, if I were to write a nozzle that emits metrics to
Graphite with a StatsD client (like it is done here[1]), I need to know if
my int64 value is a Gauge or a Counter. Also, my consumer is under constant
risk of breaking when the upstream schema changes.

We are already facing this problem with the container metrics. But at least
the container metrics are in a defined format that is well documented and
not likely to change.

I agree with you, though, the the dropsonde protocol could use a mechanism
for easier extension. Having a GenericPoint and/or GenericEvent seems like
a good idea that I whole-heartedly support. I would just like to stay away
from nested metrics. I think the cost of adding more logic into the
downstream consumer (and making it harder to maintain) is not worth the
benefit of a more concise metric transport.


[0] https://github.com/cloudfoundry-incubator/datadog-firehose-nozzle
[1] https://github.com/CloudCredo/graphite-nozzle

On Tue, Sep 1, 2015 at 5:52 PM, Benjamin Black <bblack(a)pivotal.io> wrote:

great questions, dwayne.

1) the partition key is intended to be used in a similar manner to
partitioners in distributed systems like cassandra or kafka. the specific
behavior i would like to make part of the contract is two-fold: that all
data with the same key is routed to the same partition and that all data in
a partition is FIFO (meaning no ordering guarantees beyond arrival time).

this could help with the multi-line log problem by ensuring a single
consumer will receive all lines for a given log entry in order, allowing
simple reassembly. however, the lines might be interleaved with other lines
with the same key or even other keys that happen to map to the same
partition, so the consumer does require a bit of intelligence. this was
actually one of the driving scenarios for adding the key.

2) i expect typical points to be in the hundreds of bytes to a few KB. if
we find ourselves regularly needing much larger points, especially near
that 64KB limit, i'd look to the JSON representation as the hierarchical
structure is more efficiently managed there.


b




On Tue, Sep 1, 2015 at 4:42 PM, <dschultz(a)pivotal.io> wrote:

Hi Ben,

I was wondering if you could give a concrete use case for the partition
key functionality.

In particular I am interested in how we solve multi line log entries. I
think it would be better to solve it by keeping all the data (the multiple
lines) together throughout the logging/metrics pipeline, but could see how
something like a partition key might help keep the data together as well.

Second question: how large do you see these point messages getting
(average and max)? There are still several stages of the logging/metrics
pipeline that use UDP with a standard 64K size limit.

Thanks,
Dwayne

On Aug 28, 2015, at 4:54 PM, Benjamin Black <bblack(a)pivotal.io> wrote:

All,

The existing dropsonde protocol uses a different message type for each
event type. HttpStart, HttpStop, ContainerMetrics, and so on are all
distinct types in the protocol definition. This requires protocol changes
to introduce any new event type, making such changes very expensive. We've
been working for the past few weeks on an addition to the dropsonde
protocol to support easier future extension to new types of events and to
make it easier for users to define their own events.

The document linked below [1] describes a generic data point message
capable of carrying multi-dimensional, multi-metric points as sets of
name/value pairs. This new message is expected to be added as an additional
entry in the existing dropsonde protocol metric type enum. Things are now
at a point where we'd like to get feedback from the community before moving
forward with implementation.

Please contribute your thoughts on the document in whichever way you are
most comfortable: comments on the document, email here, or email directly
to me. If you comment on the document, please make sure you are logged in
so we can keep track of who is asking for what. Your views are not just
appreciated, but critical to the continued health and success of the Cloud
Foundry community. Thank you!


b

[1]
https://docs.google.com/document/d/1SzvT1BjrBPqUw6zfSYYFfaW9vX_dTZZjn5sl2nxB6Bc/edit?usp=sharing





Re: Correcting CF Docs

Joaquin Zalazar <joaquin.zalazar@...>
 

Hi Daniel!

What I've done while testing is to set the variable BOSH_AWS_REGION to the
region I needed.

export BOSH_AWS_REGION=<your region here>

Regards!!

Joaquin Zalazar

Altoros


On Wed, Sep 2, 2015 at 9:43 AM, Daniel Jones <
daniel.jones(a)engineerbetter.com> wrote:

Hi all,

The CF docs for deploying CF to an AWS VPC
<https://docs.cloudfoundry.org/deploying/ec2/bootstrap-aws-vpc.html> have
a broken link to a Gist of Dan Highham's, that presumably talks about
deploying to regions other than us-east-1.

Does anyone know where the instructions for non-US regions are now?

Regards,

Daniel Jones
EngineerBetter.com


--

*Joaquin Zalazar*
*ALTOROS* — Cloud Foundry deployment, training and integration
www.altoros.com


Re: docker-push failed with crashing and got the following stack trace in grarden-linux

Shaozhen Ding
 

Thanks for the help

I had some permission issues on several files. Since diego run docker
containers with user vcap. I created a user named as vcap and gave the file
permission to vcap in advanced. Then i got the error above, even though I
created the home folder.

I remove the user, then the error is gone. Definitely curious of what diego
is trying to do with vcap user.

On Wed, Sep 2, 2015 at 11:40 AM, Will Pragnell <wpragnell(a)pivotal.io> wrote:

The error message suggests that in `/etc/passwd` in the Docker image, the
entry for `vcap` lists a path for the home dir that doesn't exist. I'm
afraid I don't understand Diego well enough to know why it's trying to
stream into the container as `vcap` though.

On 2 September 2015 at 17:25, Shaozhen Ding <dsz0111(a)gmail.com> wrote:

Anyone can help with interpreting the error log here
garden-linux.pool.vl5hraecvg1.stream-in.command.failed

cf-213 and diego/0.1353.0


{"timestamp":"1441209896.800271988","source":"garden-linux","message":"garden-linux.garden-server.create.created","log_level":1,"data":{"request":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","rootfs":"docker:///datianshi/cf-docker-music#latest","properties":{"executor:allocated-at":"1441209893185279557","executor:cpu-weight":"4","executor:disk-mb":"1024","executor:log-config":"{\"guid\":\"cb6ac15a-c3a0-4b5d-9716-e99934c35953\",\"index\":0,\"source_name\":\"CELL\"}","executor:memory-mb":"512","executor:metrics-config":"{\"guid\":\"cb6ac15a-c3a0-4b5d-9716-e99934c35953\",\"index\":0}","executor:owner":"executor","executor:result":"{\"failed\":false,\"failure_reason\":\"\",\"stopped\":false}","executor:rootfs":"docker:///datianshi/cf-docker-music#latest","executor:start-timeout":"60","executor:state":"created","tag:domain":"cf-apps","tag:instance-guid":"397eef75-bf73-46d3-6ba2-6a3737ea3d5e","tag:lifecyc
le":"lrp

","tag:process-guid":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0","tag:process-index":"0"},"env":["INSTANCE_GUID=397eef75-bf73-46d3-6ba2-6a3737ea3d5e","INSTANCE_INDEX=0"]},"session":"4.247804"}}

{"timestamp":"1441209896.811697721","source":"garden-linux","message":"garden-linux.garden-server.net-in.port-mapped","log_level":1,"data":{"container-port":8080,"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","host-port":61055,"session":"4.247809"}}

{"timestamp":"1441209896.818696976","source":"garden-linux","message":"garden-linux.garden-server.net-in.port-mapped","log_level":1,"data":{"container-port":2222,"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","host-port":61056,"session":"4.247810"}}

{"timestamp":"1441209896.842704535","source":"garden-linux","message":"garden-linux.garden-server.limit-memory.limited","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","resulting-limits":{"limit_in_bytes":536870912},"session":"4.247818"}}

{"timestamp":"1441209896.843993425","source":"garden-linux","message":"garden-linux.garden-server.limit-disk.limited","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","resulting-limits":{},"session":"4.247819"}}

{"timestamp":"1441209896.844884396","source":"garden-linux","message":"garden-linux.garden-server.limit-cpu.limited","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","resulting-limits":{"limit_in_shares":40},"session":"4.247820"}}

{"timestamp":"1441209896.845991373","source":"garden-linux","message":"garden-linux.garden-server.info.got-info","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","session":"4.247821"}}
{"timestamp":"1441209896.857844830","source":"garden-linux","message":"garden-linux.pool.vl5hraecvg1.stream-in.command.failed","log_level":2,"data":{"argv":["/var/vcap/data/garden-linux/depot/vl5hraecvg1/bin/nstar","11856","vcap","/tmp/lifecycle"],"error":"exit
status 1","exit-status":1,"session":"2.67.7.1","stderr":"chdir to user
home: No such file or directory\n","stdout":"","took":"2.959157ms"}}
{"timestamp":"1441209896.858107328","source":"garden-linux","message":"garden-linux.garden-server.stream-in.failed","log_level":2,"data":{"destination":"/tmp/lifecycle","error":"exit
status
1","handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","session":"4.247823"}}


Re: v3 cc api style guide feedback requested

Zach Robinson
 

Thanks James, I've just corrected the three issues you've noted so far


Re: docker-push failed with crashing and got the following stack trace in grarden-linux

Will Pragnell <wpragnell@...>
 

The error message suggests that in `/etc/passwd` in the Docker image, the
entry for `vcap` lists a path for the home dir that doesn't exist. I'm
afraid I don't understand Diego well enough to know why it's trying to
stream into the container as `vcap` though.

On 2 September 2015 at 17:25, Shaozhen Ding <dsz0111(a)gmail.com> wrote:

Anyone can help with interpreting the error log here
garden-linux.pool.vl5hraecvg1.stream-in.command.failed

cf-213 and diego/0.1353.0


{"timestamp":"1441209896.800271988","source":"garden-linux","message":"garden-linux.garden-server.create.created","log_level":1,"data":{"request":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","rootfs":"docker:///datianshi/cf-docker-music#latest","properties":{"executor:allocated-at":"1441209893185279557","executor:cpu-weight":"4","executor:disk-mb":"1024","executor:log-config":"{\"guid\":\"cb6ac15a-c3a0-4b5d-9716-e99934c35953\",\"index\":0,\"source_name\":\"CELL\"}","executor:memory-mb":"512","executor:metrics-config":"{\"guid\":\"cb6ac15a-c3a0-4b5d-9716-e99934c35953\",\"index\":0}","executor:owner":"executor","executor:result":"{\"failed\":false,\"failure_reason\":\"\",\"stopped\":false}","executor:rootfs":"docker:///datianshi/cf-docker-music#latest","executor:start-timeout":"60","executor:state":"created","tag:domain":"cf-apps","tag:instance-guid":"397eef75-bf73-46d3-6ba2-6a3737ea3d5e","tag:lifecyc
le":"lrp

","tag:process-guid":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0","tag:process-index":"0"},"env":["INSTANCE_GUID=397eef75-bf73-46d3-6ba2-6a3737ea3d5e","INSTANCE_INDEX=0"]},"session":"4.247804"}}

{"timestamp":"1441209896.811697721","source":"garden-linux","message":"garden-linux.garden-server.net-in.port-mapped","log_level":1,"data":{"container-port":8080,"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","host-port":61055,"session":"4.247809"}}

{"timestamp":"1441209896.818696976","source":"garden-linux","message":"garden-linux.garden-server.net-in.port-mapped","log_level":1,"data":{"container-port":2222,"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","host-port":61056,"session":"4.247810"}}

{"timestamp":"1441209896.842704535","source":"garden-linux","message":"garden-linux.garden-server.limit-memory.limited","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","resulting-limits":{"limit_in_bytes":536870912},"session":"4.247818"}}

{"timestamp":"1441209896.843993425","source":"garden-linux","message":"garden-linux.garden-server.limit-disk.limited","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","resulting-limits":{},"session":"4.247819"}}

{"timestamp":"1441209896.844884396","source":"garden-linux","message":"garden-linux.garden-server.limit-cpu.limited","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","resulting-limits":{"limit_in_shares":40},"session":"4.247820"}}

{"timestamp":"1441209896.845991373","source":"garden-linux","message":"garden-linux.garden-server.info.got-info","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","session":"4.247821"}}
{"timestamp":"1441209896.857844830","source":"garden-linux","message":"garden-linux.pool.vl5hraecvg1.stream-in.command.failed","log_level":2,"data":{"argv":["/var/vcap/data/garden-linux/depot/vl5hraecvg1/bin/nstar","11856","vcap","/tmp/lifecycle"],"error":"exit
status 1","exit-status":1,"session":"2.67.7.1","stderr":"chdir to user
home: No such file or directory\n","stdout":"","took":"2.959157ms"}}
{"timestamp":"1441209896.858107328","source":"garden-linux","message":"garden-linux.garden-server.stream-in.failed","log_level":2,"data":{"destination":"/tmp/lifecycle","error":"exit
status
1","handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","session":"4.247823"}}


Re: v3 cc api style guide feedback requested

James Bayer
 

the example used for actions uses the v2 api instead of v3:
https://github.com/cloudfoundry/cc-api-v3-style-guide#example-4

i like the idea of a unique error code. ideally the CAPI team maintains
public documentation of an error code dictionary
https://github.com/cloudfoundry/cc-api-v3-style-guide#issues-with-v2-error-format

the async proposal seems like a big improvement.

On Wed, Sep 2, 2015 at 8:53 AM, James Bayer <jbayer(a)pivotal.io> wrote:

the collections example [1] does not actually include the required
*resources* field

[1] https://github.com/cloudfoundry/cc-api-v3-style-guide#example-2


On Wed, Sep 2, 2015 at 8:16 AM, James Bayer <jbayer(a)pivotal.io> wrote:

should the PUT example that updates the app name and space guid actually
be a PATCH since it updates the resource?
https://github.com/cloudfoundry/cc-api-v3-style-guide#put


On Wed, Sep 2, 2015 at 1:40 AM, Dieu Cao <dcao(a)pivotal.io> wrote:

Hi All,

The CAPI team has created a style guide for v3 of the cloud controller
api and would like to share this again with the wider cf community for
feedback after initial review within the team. [1]
Issues/PRs are welcome and appreciated!

Thanks,
Dieu
CF CAPI PM

[1] https://github.com/cloudfoundry/cc-api-v3-style-guide


--
Thank you,

James Bayer


--
Thank you,

James Bayer
--
Thank you,

James Bayer


docker-push failed with crashing and got the following stack trace in grarden-linux

Shaozhen Ding
 

Anyone can help with interpreting the error log here
garden-linux.pool.vl5hraecvg1.stream-in.command.failed

cf-213 and diego/0.1353.0

{"timestamp":"1441209896.800271988","source":"garden-linux","message":"garden-linux.garden-server.create.created","log_level":1,"data":{"request":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","rootfs":"docker:///datianshi/cf-docker-music#latest","properties":{"executor:allocated-at":"1441209893185279557","executor:cpu-weight":"4","executor:disk-mb":"1024","executor:log-config":"{\"guid\":\"cb6ac15a-c3a0-4b5d-9716-e99934c35953\",\"index\":0,\"source_name\":\"CELL\"}","executor:memory-mb":"512","executor:metrics-config":"{\"guid\":\"cb6ac15a-c3a0-4b5d-9716-e99934c35953\",\"index\":0}","executor:owner":"executor","executor:result":"{\"failed\":false,\"failure_reason\":\"\",\"stopped\":false}","executor:rootfs":"docker:///datianshi/cf-docker-music#latest","executor:start-timeout":"60","executor:state":"created","tag:domain":"cf-apps","tag:instance-guid":"397eef75-bf73-46d3-6ba2-6a3737ea3d5e","tag:lifecycle":"lrp
","tag:process-guid":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0","tag:process-index":"0"},"env":["INSTANCE_GUID=397eef75-bf73-46d3-6ba2-6a3737ea3d5e","INSTANCE_INDEX=0"]},"session":"4.247804"}}
{"timestamp":"1441209896.811697721","source":"garden-linux","message":"garden-linux.garden-server.net-in.port-mapped","log_level":1,"data":{"container-port":8080,"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","host-port":61055,"session":"4.247809"}}
{"timestamp":"1441209896.818696976","source":"garden-linux","message":"garden-linux.garden-server.net-in.port-mapped","log_level":1,"data":{"container-port":2222,"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","host-port":61056,"session":"4.247810"}}
{"timestamp":"1441209896.842704535","source":"garden-linux","message":"garden-linux.garden-server.limit-memory.limited","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","resulting-limits":{"limit_in_bytes":536870912},"session":"4.247818"}}
{"timestamp":"1441209896.843993425","source":"garden-linux","message":"garden-linux.garden-server.limit-disk.limited","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","resulting-limits":{},"session":"4.247819"}}
{"timestamp":"1441209896.844884396","source":"garden-linux","message":"garden-linux.garden-server.limit-cpu.limited","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","resulting-limits":{"limit_in_shares":40},"session":"4.247820"}}
{"timestamp":"1441209896.845991373","source":"garden-linux","message":"garden-linux.garden-server.info.got-info","log_level":1,"data":{"handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","session":"4.247821"}}
{"timestamp":"1441209896.857844830","source":"garden-linux","message":"garden-linux.pool.vl5hraecvg1.stream-in.command.failed","log_level":2,"data":{"argv":["/var/vcap/data/garden-linux/depot/vl5hraecvg1/bin/nstar","11856","vcap","/tmp/lifecycle"],"error":"exit status 1","exit-status":1,"session":"2.67.7.1","stderr":"chdir to user home: No such file or directory\n","stdout":"","took":"2.959157ms"}}
{"timestamp":"1441209896.858107328","source":"garden-linux","message":"garden-linux.garden-server.stream-in.failed","log_level":2,"data":{"destination":"/tmp/lifecycle","error":"exit status 1","handle":"cb6ac15a-c3a0-4b5d-9716-e99934c35953-2669bf25-c226-4908-bbba-63e00a6a4fa0-397eef75-bf73-46d3-6ba2-6a3737ea3d5e","session":"4.247823"}}


Re: v3 cc api style guide feedback requested

James Bayer
 

the collections example [1] does not actually include the required
*resources* field

[1] https://github.com/cloudfoundry/cc-api-v3-style-guide#example-2

On Wed, Sep 2, 2015 at 8:16 AM, James Bayer <jbayer(a)pivotal.io> wrote:

should the PUT example that updates the app name and space guid actually
be a PATCH since it updates the resource?
https://github.com/cloudfoundry/cc-api-v3-style-guide#put


On Wed, Sep 2, 2015 at 1:40 AM, Dieu Cao <dcao(a)pivotal.io> wrote:

Hi All,

The CAPI team has created a style guide for v3 of the cloud controller
api and would like to share this again with the wider cf community for
feedback after initial review within the team. [1]
Issues/PRs are welcome and appreciated!

Thanks,
Dieu
CF CAPI PM

[1] https://github.com/cloudfoundry/cc-api-v3-style-guide


--
Thank you,

James Bayer
--
Thank you,

James Bayer


CF CLI Release v 6.12.3

Greg Oehmen
 

The CF CLI team just cut 6.12.3. Release notes and binaries are available
at:

https://github.com/cloudfoundry/cli/releases


Highlights of this release include:

Bug Fix: runtime error: index out of range

originally reported here https://github.com/cloudfoundry/cli/issues/509 and
in many subsequent Github issues

-

https://www.pivotaltracker.com/story/show/98672332
-

The root problem for this particular crash is because
v2/apps/GUID/instances reports 0 instance for a newly stopped app, but
GetContainerMetrics() from noaa library will continue to report instances
metrics for a while, and eventually be consistent with
v2/apps/GUID/instance. This causes the CLI’s 'index out of range error'.
The bug is fixed by not gathering metrics unless instance>0, but
GetContainerMetrics() should be consistent with what CC endpoint reports.




Pull Requests


-

Merge pull request #544 <https://github.com/cloudfoundry/cli/pull/544>
from cloudfoundry/code-tidy Code tidy: Code cleanup: remove unused
variables, remove orphan functions, shadowing reserved word
-

Merge pull request #523 <https://github.com/cloudfoundry/cli/pull/523>
from zachgersh/master Unmarshal the extra field, get documentation url
-

Merge pull request #540 <https://github.com/cloudfoundry/cli/pull/540>
from cloudfoundry/use_go_yaml Support yaml '<<' merge type
-

Merge pull request #534 <https://github.com/cloudfoundry/cli/pull/534>
from cloudfoundry/feature/commands-restart-and-create Move commands to new
command pattern.
-

Merge pull request #505 <https://github.com/cloudfoundry/cli/pull/505>
from zhang-hua/bug-93578300 Reduce API calls when CRU operations of service
keys
-

Merge branch 'story-87481016' of https://github.com/zhang-hua/cli into
zhang-hua-story-87481016
-

Merge pull request #514 <https://github.com/cloudfoundry/cli/pull/514>
Fix create-app-manifest only includes one host [92530254]




Also notable:

We deprecated the use of CodeGangsta within the CF CLI. Previously,
CodeGangsta had been used for managing help and flags within the CLI. We
now manage those functions natively in the CLI. This is a precursor to the
work we will begin soon of refactoring help, restructuring command syntax
and abstracting admin-only commands out into some separate manifestation;
perhaps a plugin.



Greg Oehmen
Cloud Foundry Product Manager


Re: v3 cc api style guide feedback requested

James Bayer
 

should the PUT example that updates the app name and space guid actually be
a PATCH since it updates the resource?
https://github.com/cloudfoundry/cc-api-v3-style-guide#put

On Wed, Sep 2, 2015 at 1:40 AM, Dieu Cao <dcao(a)pivotal.io> wrote:

Hi All,

The CAPI team has created a style guide for v3 of the cloud controller api
and would like to share this again with the wider cf community for feedback
after initial review within the team. [1]
Issues/PRs are welcome and appreciated!

Thanks,
Dieu
CF CAPI PM

[1] https://github.com/cloudfoundry/cc-api-v3-style-guide
--
Thank you,

James Bayer


Re: stdout.log and stderr.log not show in CF197 with loggregator enabled

Erik Jasiak
 

Dan is correct - "cf file" for getting stdout/stderr files directly was
deprecated some time ago. I'll try to locate when that happened (it was
before my time.)

From the cli, "cf logs" for streaming, and "cf logs --recent" for a
quick dump are the way to go.

Erik
PM - loggregator

Daniel Mikusa wrote:


Have you tried running `cf logs` instead? Anything written to STDOUT
or STDERR should be visible there for some period of time. For long
term storage, you should look at setting up a log drain on your
application.

http://docs.cloudfoundry.org/devguide/services/log-management.html

Dan

On Wed, Sep 2, 2015 at 9:08 AM, Shruthi Ravindra
<Shruthi.ravindra(a)ge.com <mailto:Shruthi.ravindra(a)ge.com>> wrote:

Hi,

we are also facing the same issue when we check the "cf files
pfh-cos-dicomobjectstore-dev /logs" we are able to see only the
staging logs and not the stdout.log abd stderror.log.
we need to access these logs to debug the error.
how do we enable the stdout and stderror logs


Re: stdout.log and stderr.log not show in CF197 with loggregator enabled

Daniel Mikusa
 

Have you tried running `cf logs` instead? Anything written to STDOUT or
STDERR should be visible there for some period of time. For long term
storage, you should look at setting up a log drain on your application.

http://docs.cloudfoundry.org/devguide/services/log-management.html

Dan

On Wed, Sep 2, 2015 at 9:08 AM, Shruthi Ravindra <Shruthi.ravindra(a)ge.com>
wrote:

Hi,

we are also facing the same issue when we check the "cf files
pfh-cos-dicomobjectstore-dev /logs" we are able to see only the staging
logs and not the stdout.log abd stderror.log.
we need to access these logs to debug the error.
how do we enable the stdout and stderror logs


Re: stdout.log and stderr.log not show in CF197 with loggregator enabled

Shruthi Ravindra
 

Hi,

we are also facing the same issue when we check the "cf files pfh-cos-dicomobjectstore-dev /logs" we are able to see only the staging logs and not the stdout.log abd stderror.log.
we need to access these logs to debug the error.
how do we enable the stdout and stderror logs


Re: stdout.log and stderr.log not show in CF197 with loggregator enabled

Shruthi Ravindra
 

Hi ,

We are having the same problem where we are not able to see the stdout.log and stderror.log for all the application deployed in the cloud.
Could you please let me know how do we access the logs other than cf logs "appname" command which is not giving us the full logs.

Thanks,
Shruthi


Correcting CF Docs

Daniel Jones
 

Hi all,

The CF docs for deploying CF to an AWS VPC
<https://docs.cloudfoundry.org/deploying/ec2/bootstrap-aws-vpc.html> have a
broken link to a Gist of Dan Highham's, that presumably talks about
deploying to regions other than us-east-1.

Does anyone know where the instructions for non-US regions are now?

Regards,

Daniel Jones
EngineerBetter.com


Re: Announcing Experimental support for Asynchronous Service Operations

Guillaume Berche
 

Thanks Dieu for your response.

We have successfully implemented an async service broker for out internal
dbaas [1], [2] which db farms provisionning sometimes takes longer than
30s.

Concerning improvement suggestions on the documentation, I could only spot
an inconsistency on the sequence diagram, where the last operation
description is mismatching between broker response (1/3 nodes") and CC api
response ("40% complete") whereas the 'description' is a string which
should be returned as is. Hard to propose a PR on that since the source
file for this diagram does not seem available under docs-services [4]. BTW,
It would be great if bookbinder would support rendering diagrams from a
text format such as plantuml [5], or rely on plantuml public service [6]
for such rendering.

Wouldn't is make sense to exit the experimental status of the async api,
even if it's not feature complete ? i.e. considering the api is stable
enough and won't change in an incompatible maneer soon, thereby enabling
clients such as cf-java-client to rely on them.

One other potential improvement I see to the API would be to support
localized user-facing operation status description. Having the CLI being
I18Ned but brokers enable to return user facing messages in the user
language seems limiting. Maybe it could be the case of having the CLI add a
usual 'Accept-Language: zh_Hans" matching the current CLI locale, and the
CC propagating this header in the last_operation broker endpoint [7] so
that they can return the status message in the smae language when
translations are available.

Thanks,

Guillaume.

[1]
https://github.com/Orange-OpenSource/elpaaso-brokers/blob/master/cf-service-broker-dbaas/src/main/java/com/orange/clara/cloud/cf/servicebroker/dbaas/service/DbaasServiceInstanceService.java#L60
[2]
https://github.com/Orange-OpenSource/elpaaso-brokers/blob/a3f150fc7d3bb889875aac202496a4f63efc3595/cf-service-broker-dbaas/src/main/java/com/orange/clara/cloud/cf/servicebroker/dbaas/service/DbaasServiceInstanceService.java#L67-L73
[3]
http://docs.cloudfoundry.org/services/images/async-service-broker-flow.png
[4]
https://github.com/cloudfoundry/docs-services/blob/master/images/async-service-broker-flow.png
[5] http://plantuml.com/sequence.html
[6] http://plantuml.com/plantuml/
[7]
http://docs.cloudfoundry.org/services/asynchronous-operations.html#polling

On Tue, Sep 1, 2015 at 5:50 PM, Dieu Cao <dcao(a)pivotal.io> wrote:

Hi Guillaume,

We are looking for a couple more pieces of feedback. All feedback I've
received so far has been positive.

There was one report recently from Robert Moss about being able to provide
the dashboard_url asynchronously instead of part of the original response
[1].
I'm also considering whether that needs to be addressed prior to marking
the feature complete, or if it should be a separate/additive change to the
service broker api.

If you've had success with the current implementation, I would be greatly
interested in hearing that.

-Dieu

[1]
https://lists.cloudfoundry.org/archives/list/cf-dev(a)lists.cloudfoundry.org/thread/LJ26K22WG7OJCAIXWYPTJQKELVAUBUZC/

On Tue, Sep 1, 2015 at 5:20 AM, Guillaume Berche <bercheg(a)gmail.com>
wrote:

Hi,

I'm wondering whether the experimental async service broker api is now
ready to become official, or whether the CAPI team and PMs are still
lacking feedback or have some outstanding issues that need to be addressed
before calling the spec official? I was not able to spot any async-related
stories in the CAPI backlog [1]

We have a pending PR in cf-java-client [2] which is waiting for the async
broker spec to be made official.

Thanks in advance,

Guillaume.

[1] https://www.pivotaltracker.com/n/projects/966314
[2]
https://github.com/cloudfoundry/cf-java-client/pull/282#issuecomment-115874984

On Fri, Jun 5, 2015 at 9:21 AM, James Bayer <jbayer(a)pivotal.io> wrote:

i'm very happy to see this work delivered as the 60 second limit has
come up so often as a pain point in the past. great job to all who
contributed!

On Wed, Jun 3, 2015 at 5:05 PM, Onsi Fakhouri <ofakhouri(a)pivotal.io>
wrote:

Well done Services API! This is an awesome milestone!

On Wed, Jun 3, 2015 at 5:04 PM, Chip Childers <
cchilders(a)cloudfoundry.org> wrote:

Awesome news! Long time coming, and it opens up a whole world of
additional capabilities for users.

Nice work everyone!



On Jun 4, 2015, at 9:00 AM, Shannon Coen <scoen(a)pivotal.io> wrote:

On behalf of the Services API team, including Dojo participants from
IBM and SAP, I'm pleased to announce experimental availability and
published documentation for this much-anticipated feature.

As of cf-release v208 and CLI v6.11.1, Cloud Foundry now supports an
enhanced service broker integration in support of long-running
provisioning, update, and delete operations. This significantly broadens
the supported use cases for Cloud Foundry Marketplace Services, and I can't
wait to hear what creative things the ecosystem does with it. Provision
VMs, orchestrate clusters, install software, move data... yes, your broker
can even open support tickets to have those things done manually!

This feature is currently considered experimental, as we'd like you
all to review our docs, try out the feature, and give us feedback. We very
interested to hear about any confusion in the docs or the UX, and any
sticky issues you encounter in implementation. Our goal is for our docs
enable a painless, intuitive (can we hope for joyful?) implementation
experience.

We have not bumped the broker API yet for this feature. You'll notice
that our documentation for the feature is separate from the stable API docs
at this point. Once we're confident in the design (we're relying on your
feedback!), we'll bump the broker API version, move the docs for
asynchronous operations into the stable docs, AND implement support for
asynchronous bind/create-key and unbind/delete-key.

Documentation:
- http://docs.cloudfoundry.org/services/asynchronous-operations.html
- http://docs.cloudfoundry.org/services/api.html
Example broker for AWS (contributed by IBM):
- http://docs.cloudfoundry.org/services/examples.html
- https://github.com/cloudfoundry-samples/go_service_broker
Demo of the feature presented at CF Summit 2015:
- https://youtu.be/Ij5KSKrAq9Q

tl;dr

Cloud Foundry expects broker responses within 60 seconds. Now a broker
can return an immediate response indicating that a provision, update, or
delete operation is in progress. Cloud Foundry then returns a similar
response to the client, and begins polling the broker for the status of the
operation. Users, via API clients, can discover the status of the operation
("in progress", "succeeded", or "failed"), and brokers can provide
user-facing messages in response to each poll which are exposed to users
(e.g. "VMs provisioned, installing software, 30% complete").

Thank you,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Thank you,

James Bayer

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


v3 cc api style guide feedback requested

Dieu Cao <dcao@...>
 

Hi All,

The CAPI team has created a style guide for v3 of the cloud controller api
and would like to share this again with the wider cf community for feedback
after initial review within the team. [1]
Issues/PRs are welcome and appreciated!

Thanks,
Dieu
CF CAPI PM

[1] https://github.com/cloudfoundry/cc-api-v3-style-guide


Re: How to set cc.info with cf-release spiff templates ?

Guillaume Berche
 

Thanks Joseph for your response. Any chance that the OSS build pipeline
would update the spiff template with the current version number of the cf
release being built, as to avoid each consummers of the release to manually
update version number when updating cc.info.version ? Ideally by having the
spiff templates override the default value (yet-to-be-specified in the
release's spec file, see [3]) by the current cf release number ?

I was trying to manually assign this value with meta.releases[0].version,
but I see that meta.releases[0].version seems to always specify 'latest'
[2]. One possibility would have to specify the current release number (e.g.
'215')

Thanks in advance,

Guillaume.

[2]
https://github.com/cloudfoundry/cf-release/blob/90d730a2d13d9e065a7f348e7fd31a1522074d02/templates/cf-deployment.yml#L10
[3] https://github.com/cloudfoundry/cf-release/issues/774

On Tue, Sep 1, 2015 at 11:29 PM, CF Runtime <cfruntime(a)gmail.com> wrote:

Unfortunately those properties don't actually do what they say. A story
has been added to the CAPI backlog to clean those up, and possibly change
them: https://www.pivotaltracker.com/story/show/102523218

In the meantime, you can override `version` and `description`, they are
currently used to populate the info endpoint, but they may change in the
process of doing the above story.

After the story is done, we'll add the attributes to the template so they
are easy to override, but for now you'll need to manually edit your
manifest to add the `version` and `description` properties.

Joseph
OSS Release Integration Team

On Thu, Aug 27, 2015 at 1:07 PM, Guillaume Berche <bercheg(a)gmail.com>
wrote:

Hi,

I'm trying to assign cc.info.description and cc.info.version [1] (to
merge meta.releases[0].version if cf release version), as to be able to
describe to end users which cf version is currently deployed from a "cf
curl v2/info" command.

I'm however struggling with spiff syntax to do so. I trial-and-errored
around modifying cf-release/templates as to allow for the merge without
luck so far.

Anyone succeeded in doing so without forking standard spiff templates ?

Thanks in advance,

Guillaume.

[1]
https://github.com/cloudfoundry/cf-release/blob/9334295133435fa77767651030500d2b0de62611/jobs/cloud_controller_ng/spec#L140-L145


Re: Which kind of resources stored in cc-resources container?

Matt Reider
 

You can refer to this discussion for some understanding of what is stored
in the cc-resources container:

https://groups.google.com/a/cloudfoundry.org/forum/#!topic/vcap-dev/nTQSn8spBPE

It's basically stores larger files to speed up subsequent app push. An
initial push will be slow, as all files are uploaded. A second push will be
faster, as the larger files (over 64k by default, but can be adjusted up),
are stored in the cc-resources container for re-use.

On Tue, Sep 1, 2015 at 6:40 PM Guangcai Wang <guangcai.wang(a)gmail.com>
wrote:

Hi,

I am using Swift as Cloud Foundry blobstore. I noticed there are four
containers:
cc-buildpacks
cc-droplets
cc-packages
cc-resources

For the first three containers, I can find the relationship between the
object in containers and the records in postgres database. I also noticed
that some objects will be stored in cc-resources when I push applications.
They seem related to the type of application (ruby, java, etc). However, I
cannot find what should be stored in cc-resources container. Are they
uploaded automatically by buildpack? Is it decided by buildpack or
application? How to configure them if it is decided by application?


Thanks.


Re: diego questions: default memory and passing env variables

Amit Kumar Gupta
 

Responses inline.

On Tue, Sep 1, 2015 at 7:13 PM, Tom Sherrod <tom.sherrod(a)gmail.com> wrote:

What is the default memory size of an app deployed in diego? cf apps
indicate 1GB.
Yes. This isn't a diego thing, this is a CC thing.


cf set-env doesn't appear to be getting the environment variable
information to the running container. Is cf set-env the correct method to
use?
You likely need to restage the application for ENV var changes to take
effect. Unless I'm mistaken, this too is not diego-specific, and applies
to DEA containers as well.


Thanks!
Tom
Best,
Amit

7961 - 7980 of 9426