Date   

Re: How to update blobs in blob.cfblob.com ?

Matthew Sykes <matthew.sykes@...>
 

Hi. This isn't what I expected. You've basically changed the bucket
configuration to point to your own and uploaded the blobs to that bucket.
That's why bosh is telling you there's nothing to upload.

There's also a general expectation that blobs are pristine artifacts from a
well known distribution point. In this case, that's not true. You've used a
script to modify some content and there's no way for someone to know that
without looking in detail so it's going to be tough to trust what's there.

Regardless, given what your scripts are doing, you should be able to
generate patches for each of the packages, add them as source artifacts,
and update the packaging scripts to apply the patches before the build.
That's probably how this should be done to keep things sane.

On Fri, Jul 24, 2015 at 4:37 AM, Lomov Alexander <
alexander.lomov(a)altoros.com> wrote:

Hi, Matthew.

After I made `bosh upload blobs`, `bosh blobs` command tells me `No
blobs to upload`.

Still I created a pull request where you can figure out what blobs are
changed from `config/blobs.yml` file [1].

Could you tell if it is okey?

[1] https://github.com/cloudfoundry/bosh/pull/892

On Jul 1, 2015, at 5:38 PM, Matthew Sykes <matthew.sykes(a)gmail.com>
wrote:

Since you won't be able to upload the blobs to the cf-release bucket,
I'd suggest you capture the output of `bosh blobs` in your pull request.
That command should enumerate all of the new blobs and their sizes.

For each entry that's there, point to a publicly available URL and a
hash that can be used to verify it.

When the PR is reviewed, if things look good, the pair will likely pull
the blobs down to evaluate them and test the overall function.

On Wed, Jul 1, 2015 at 6:57 AM, Alexander Lomov <
alexander.lomov(a)altoros.com> wrote:

Hi, all.

I work on adding support of Power architecture to CF. During the work I
needed to update not only cf-release, but existing blobs (postgresql, mysql
client and etc.). I wonder how I can make PR to cf-release project with
updated blobs.

I couldn't find any clue in the contributing guild [1] , so I've decided
to write here.

[1]
https://github.com/cloudfoundry/cf-release/blob/master/CONTRIBUTING.md

Thank you,
Alex L.

------------------------
Alex Lomov
*Altoros* — Cloud Foundry deployment, training and integration
*Twitter:* @code1n <https://twitter.com/code1n> *GitHub:* @allomov
<https://gist.github.com/allomov>

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Matthew Sykes
matthew.sykes(a)gmail.com
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Matthew Sykes
matthew.sykes(a)gmail.com


How cloud foundry deployed by bosh-lite manage DEA

GuopingZhang
 

I read that bosh-lite use warden as container for cloud foundry components.I deployed two java spring apps into the local cloud foundry.
Below is my VMs (warden container), which (two) warden container is running my 2 apps?
Or the apps are actually running as java process in my virtualbox of unbuntu (which contains all below containers), without running inside a warden container?
+------------------------------------+---------+---------------+--------------+| Job/index | State | Resource Pool | IPs |+------------------------------------+---------+---------------+--------------+| api_z1/0 | running | large_z1 | 10.244.0.134 || doppler_z1/0 | running | medium_z1 | 10.244.0.142 || etcd_z1/0 | running | medium_z1 | 10.244.0.42 || ha_proxy_z1/0 | running | router_z1 | 10.244.0.34 || hm9000_z1/0 | running | medium_z1 | 10.244.0.138 || loggregator_trafficcontroller_z1/0 | running | small_z1 | 10.244.0.146 || nats_z1/0 | running | medium_z1 | 10.244.0.6 || postgres_z1/0 | running | medium_z1 | 10.244.0.30 || router_z1/0 | running | router_z1 | 10.244.0.22 || runner_z1/0 | running | runner_z1 | 10.244.0.26 || uaa_z1/0 | running | medium_z1 | 10.244.0.130 |+------------------------------------+---------+---------------+--------------+


Re: Allow gorouter to log random headers.

David Laing
 

Shannon, Simon,

As the lead of the logsearch.io (ELK) project; I'm also interested in having
GoRouter log additional headers. Specifically Trace-Id and Span-Id
generated by the spring-cloud-sleuth project
(https://github.com/spring-cloud-incubator/spring-cloud-sleuth ).

If GoRouter logged whatever headers were included in the request, wouldn't
this satisfy your requirements?
This would certainly satisfy my requirements, and I think Simon's too.

However, not having a whitelist of headers to log opens a possible DDOS
attack vector on the GoRouter, so I think having a operator configureable
whitelist (with some sensible defaults like Trace-Id and Span-Id) is the
right approach.

Doesn't GoRouter do this already?
I don't think so.

Specifically, sending the following curl to an app hosted on PWS:

curl --header "Trace-Id: 1c884728-466c-4ba3-8caa-a02a780c6d56"
http://www.birdsangola.org/

Gives the following [RTR] log from loggregator:

Fri Jul 24 2015 13:10:52 GMT+0100 (BST) [RTR] OUT www.birdsangola.org -
[24/07/2015:12:10:52 +0000] "GET / HTTP/1.1" 200 0 7772 "-" "curl/7.30.0"
10.10.2.185:46765 x_forwarded_for:"92.40.249.226"
vcap_request_id:3a33d5f6-dc11-42c4-61c7-32a1a2557200
response_time:0.001380276 app_id:0c34cc9f-45a8-440e-b876-b0cde564fbe3

It doesn't look like the extra Trace-Id header has been passed through to
the loggregator [RTR] log.

I'd be happy to work with Simon to contribute to a PR that implements the
"above whitelist of headers to log" feature.

Thoughts?

:D



--
View this message in context: http://cf-dev.70369.x6.nabble.com/cf-dev-Allow-gorouter-to-log-random-headers-tp800p877.html
Sent from the CF Dev mailing list archive at Nabble.com.


Re: How to update blobs in blob.cfblob.com ?

Alexander Lomov <alexander.lomov@...>
 

Hi, Matthew.

After I made `bosh upload blobs`, `bosh blobs` command tells me `No blobs to upload`.

Still I created a pull request where you can figure out what blobs are changed from `config/blobs.yml` file [1].

Could you tell if it is okey?

[1] https://github.com/cloudfoundry/bosh/pull/892

On Jul 1, 2015, at 5:38 PM, Matthew Sykes <matthew.sykes(a)gmail.com<mailto:matthew.sykes(a)gmail.com>> wrote:

Since you won't be able to upload the blobs to the cf-release bucket, I'd suggest you capture the output of `bosh blobs` in your pull request. That command should enumerate all of the new blobs and their sizes.

For each entry that's there, point to a publicly available URL and a hash that can be used to verify it.

When the PR is reviewed, if things look good, the pair will likely pull the blobs down to evaluate them and test the overall function.

On Wed, Jul 1, 2015 at 6:57 AM, Alexander Lomov <alexander.lomov(a)altoros.com<mailto:alexander.lomov(a)altoros.com>> wrote:
Hi, all.

I work on adding support of Power architecture to CF. During the work I needed to update not only cf-release, but existing blobs (postgresql, mysql client and etc.). I wonder how I can make PR to cf-release project with updated blobs.

I couldn't find any clue in the contributing guild [1] , so I've decided to write here.

[1] https://github.com/cloudfoundry/cf-release/blob/master/CONTRIBUTING.md

Thank you,
Alex L.

------------------------
Alex Lomov
Altoros — Cloud Foundry deployment, training and integration
Twitter: @code1n<https://twitter.com/code1n> GitHub: @allomov<https://gist.github.com/allomov>

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




--
Matthew Sykes
matthew.sykes(a)gmail.com<mailto:matthew.sykes(a)gmail.com>
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Notifications on ORG, SPACE and USER modifications

Matthias X Hub
 

Hi, happy to participate, we already have some initial ideas! Regards,
Matthias



From: Juan Pablo Genovese <juanpgenovese(a)gmail.com>
To: "Discussions about Cloud Foundry projects and the system overall."
<cf-dev(a)lists.cloudfoundry.org>
Date: 23.07.2015 15:41
Subject: Re: [cf-dev] Notifications on ORG, SPACE and USER
modifications
Sent by: cf-dev-bounces(a)lists.cloudfoundry.org



Anyone wiling to do a Hangout and start talking this? I really want to
push this forward.

Thanks!!

JP

2015-07-23 4:39 GMT-03:00 Benjamin Black <bblack(a)pivotal.io>:
ETags and a 304 response are specifically intended for that purpose. I'd
recommend that over relying on Last-Modified.


b

On Thu, Jul 23, 2015 at 12:34 AM, Koper, Dies <diesk(a)fast.au.fujitsu.com>
wrote:
Or setting the Last-Modified HTTP response header accordingly, and allow
clients to use HTTP caching mechanisms (Last-Modified, etc.) to get quick
empty responses with the current APIs if no changes have been made? (Or
maybe this is already working so ? haven?t checked).

Regards,
Dies Koper

From: cf-dev-bounces(a)lists.cloudfoundry.org [mailto:
cf-dev-bounces(a)lists.cloudfoundry.org] On Behalf Of Matt Cowger
Sent: Thursday, July 23, 2015 4:45 PM
To: Discussions about Cloud Foundry projects and the system overall.
Subject: Re: [cf-dev] Notifications on ORG, SPACE and USER modifications

I've wanted something similar as well.

On a related note, having a CC API 'serial' number (for each object in CC
- apps, spaces, etc) that increments on every change relevant to that
object would be of value for detecting if something has changed.

On Thu, Jul 23, 2015 at 3:27 PM, Dieu Cao <dcao(a)pivotal.io> wrote:
There are a few different approaches to this and different concerns that
are possible.
The requests I've seen have been around wanting to be able to subscribe to
and filter the various events that cc currently generates so that other
behavior could be triggered.
We currently have events, app usage events, and service usage events.
Is it acceptable for the notifications to be lossy? Depends on the use
case but If so, then the firehose may be an acceptable approach.

The CAPI team is currently focusing on other work in the near term, such
as the v3 API and private brokers, but would be happy to collaborate on a
proposal.


On Wed, Jul 22, 2015 at 2:05 PM, Juan Pablo Genovese <
juanpgenovese(a)gmail.com> wrote:
My take:

CC should have callbacks on for each model create, update and delete
methods. Those callbacks will send a message to an MQ, which you can
subscribe to consume those messages.
This can be expanded to pretty much every event we need to track.
What do you think?

JP

2015-07-22 17:30 GMT-03:00 Matthias X Hub <matthias.hub(a)de.ibm.com>:
Hi,

we (=IBM) are also having the need and are currently investigating how to
solve this. We plan to work on a proposal to discuss this further with the
cf community. I'll keep you updated on that.

Regards,
Matthias



From: Mike Youngstrom <youngm(a)gmail.com>
To: "Discussions about Cloud Foundry projects and the system
overall." <cf-dev(a)lists.cloudfoundry.org>
Date: 22.07.2015 20:57
Subject: Re: [cf-dev] Notifications on ORG, SPACE and USER
modifications
Sent by: cf-dev-bounces(a)lists.cloudfoundry.org




We have the same need. Today we are polling the CC.

It would be nice for us also if we could get CC event notifications via
something like the firehose.

Mike

On Wed, Jul 22, 2015 at 10:23 AM, Juan Pablo Genovese <
juanpgenovese(a)gmail.com> wrote:
I mean, I know you can list those events thru the API, but I want
something that will react on an event instead of having to be constantly
polling for them.

2015-07-22 13:18 GMT-03:00 Juan Pablo Genovese <juanpgenovese(a)gmail.com>:
Sree,

thanks! Any pointers on how can I hook up to these audit events?

Thank you!

2015-07-22 13:12 GMT-03:00 Sree Tummidi <stummidi(a)pivotal.io>:
I believe there are audit events generated for all these actions which can
be captured and forwarded to an SIEM solution like splunk

Thanks,
Sree

Sent from my iPhone

On Jul 22, 2015, at 8:54 AM, Juan Pablo Genovese <juanpgenovese(a)gmail.com>
wrote:

Guys,

I need to somehow hook up into the Cloud Controller (CC) to capture ORG,
SPACE and USER deletion, insertion and update.

So far, I considered some approaches, such as forking the CC (the least
favorite) and modifying the code with some hooks, tapping into Nginx to
capture the requests, and using triggers in the database to capture each
event and send the necessary info to a service.

What do you think?
Any other idea you might have?

Thanks!

--
Mis mejores deseos,
Best wishes,
Meilleurs v?ux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




--
Mis mejores deseos,
Best wishes,
Meilleurs v?ux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com



--
Mis mejores deseos,
Best wishes,
Meilleurs v?ux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



--
Mis mejores deseos,
Best wishes,
Meilleurs v?ux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




--
Mis mejores deseos,
Best wishes,
Meilleurs v?ux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Announcing support for Arbitrary Parameters and Service Instance Tags

Noburou TANIGUCHI
 

Great news!
I should tell it to my colleagues.


Dieu Cao wrote
On behalf of the CAPI team I'm pleased to announce support for Arbitrary
Parameters and Service Instance Tags, two features which extend the
possibilities for Cloud Foundry Marketplace service offerings, and provide
developers with increased flexibility in how services are consumed by
applications.

Support for Arbitrary Parameters is introduced with cf-release v209 and
CLI
v6.12.1. This features enables service providers to support user-provided
configuration options with the create, update, and bind service instance
operations, and the create service key operation (stay tuned for a
forthcoming announcement of support for Service Keys). Previously, this
could only be achieved by providing many plans to cover various
combinations of configuration options, or to provide a service instance
dashboard that users can SSO into and adjust configuration options after
creation. Although the platform and CLI now support the feature, service
broker authors must implement support for the feature as described in
the Service
Broker API v2.5 specification
&lt;http://docs.cloudfoundry.org/services/api.html>;.

Support for Instance Tags is introduced with cf-release v211 and CLI
v6.12.1. Since v2.0 of the Service Broker API, broker authors have be
able
to provide tags for a service offering in the /v2/catalog endpoint that
Cloud Foundry delivers to applications in the VCAP_SERVICES Environment
Variable
&lt;http://docs.cloudfoundry.org/devguide/deploy-apps/environment-variable.html#VCAP-SERVICES>;.
These tags provide developers with a more generic way for applications to
parse VCAP_SERVICES for credentials. Developers may now provide their own
tags when creating or updating a service instance by including a
comma-separated list of tags with the -t flag.

Documentation:

- http://docs.cloudfoundry.org/services/api.html

- http://docs.cloudfoundry.org/devguide/services/managing-services.html

Special thanks to the former CF Services API team and Shannon for their
hard work on these features.

-Dieu
CF CAPI PM

_______________________________________________
cf-dev mailing list
cf-dev(a).cloudfoundry
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




--
View this message in context: http://cf-dev.70369.x6.nabble.com/cf-dev-Announcing-support-for-Arbitrary-Parameters-and-Service-Instance-Tags-tp860p874.html
Sent from the CF Dev mailing list archive at Nabble.com.


Re: Did anybody deploy a wiki as app to CF?

Noburou TANIGUCHI
 

Thank you, all.

I am very glad to see such many positive reactions.
We should consider about English version of the articles.




--
View this message in context: http://cf-dev.70369.x6.nabble.com/cf-dev-Did-anybody-deploy-a-wiki-as-app-to-CF-tp643p873.html
Sent from the CF Dev mailing list archive at Nabble.com.


Re: 3 etcd nodes don't work well in single zone

Amit Kumar Gupta
 

Looks like there are a lot of log lines in the hm.listener component where
it takes to long to save state to etcd. It updates state based on
heartbeats from the DEAs. When the etcd request takes too long, the
listener doesn't mark the data as fresh (it lets a key with a TTL expire).
Then when another component tries to get the state of actual running
instances (this value populates the number of running instances you see
changing in the CLI output), it bails early because it detects the data is
stale. CC can't determine the number of running instances, so it reports
-1 as a sentinel to indicate unknown, which the CLI renders as ?.

The question is why are saves sometimes taking too long, causing the data
to be marked stale so frequently?


Re: "running_instances": -1

Matthew Sykes <matthew.sykes@...>
 

As Maggie and Dieu have said, the `running_instances` field from the
"summary" endpoint the cli is using reflects hm9k view of the world and the
`state` field represents the desired state of the app. The API is basically
stitching what the CC knows about the app with what the health manager
knows into a view.

While the '-1' does imply that hm9k doesn't know about the app, a stale
view is not the only cause.

The problem that you're hitting is that the summary endpoint isn't doing
the right thing when the app has not been correctly staged.

In particular, hm9k throws away information it receives about applications
that are not staged [1]. That means that when the CC asks hm9k for
information about the app [2], it has none and returns '-1' for instances
(and all other metrics).

Basically, it looks like the summary endpoint or `instances_reporter.rb`
[3] should be evaluating the package state a little more carefully so it
can present the correct information as hm9k will not have info on apps that
did not successfully stage.

[1]:
https://github.com/cloudfoundry/hm9000/blob/17b3830948ded967bd4052d4975683e107a43030/desiredstatefetcher/desired_state_fetcher.go#L162
[2]:
https://github.com/cloudfoundry/cloud_controller_ng/blob/4b23dd32b39174d5bf1c11170851c2ea99704272/app/controllers/runtime/app_summaries_controller.rb#L18
[3]:
https://github.com/cloudfoundry/cloud_controller_ng/blob/4b23dd32b39174d5bf1c11170851c2ea99704272/lib/cloud_controller/dea/instances_reporter.rb#L14-L17

On Thu, Jul 23, 2015 at 10:03 PM, Meng, Xiangyi <xiangyi.meng(a)emc.com>
wrote:

Hi, Kris



I think it may be a CLI 6.0 issue to show the “requested_state” not the
real state. I remembered in CLI 5.0 it would show stopped if application
failed to start.



But the “-1” you get has nothing with the status of application. I got
this value even when my application starts and runs successfully. The “-1”
just indicates that CC could not get your application’s status. If your
hm900 server works well, you will see 0/0 if your application fails at
deployment or staging stage.



Thanks,

Maggie



*From:* cf-dev-bounces(a)lists.cloudfoundry.org [mailto:
cf-dev-bounces(a)lists.cloudfoundry.org] *On Behalf Of *Kris Kobylinski
*Sent:* 2015年7月24日 9:53

*To:* Discussions about Cloud Foundry projects and the system overall.
*Subject:* Re: [cf-dev] "running_instances": -1



@Maggie -- you are probably right that -1 does not mean that the
deployment failed. But for my application the staging failed and I am
getting the following:

*"running_instances":-1,*

*"state": "STARTED"*,

*"package_state": "FAILED", "staging_failed_reason":
"BuildpackCompileFailed",*

It is very confusing. Why the app state is STARTED when the staging FAILED
? Also if the staging FAILED shouldn't the running_instances be 0 ?

What are all the scenarios where running_instances = -1 ?

Thank you,

Kris



On Thu, Jul 23, 2015 at 9:29 PM, Meng, Xiangyi <xiangyi.meng(a)emc.com>
wrote:

We had encountered the same issue before. The “?” or “-1” doesn’t mean the
deployment fails. It only means CC could not get the status of running
instance. Most likely there is something wrong with your ETCD server or
HM9000 server. You should check the log files from those servers.



Thanks,

Maggie



*From:* cf-dev-bounces(a)lists.cloudfoundry.org [mailto:
cf-dev-bounces(a)lists.cloudfoundry.org] *On Behalf Of *Kris Kobylinski
*Sent:* 2015年7月23日 20:42
*To:* Discussions about Cloud Foundry projects and the system overall.
*Subject:* Re: [cf-dev] "running_instances": -1



If the deployment failed wouldn't that mean that no instances are running
? Shouldn't the running_instances be 0 in that case ?


Thank you,

Kris



On Thu, Jul 23, 2015 at 2:06 AM, Dieu Cao <dcao(a)pivotal.io> wrote:

-1 indicates that the state of the instances is unknown.

The CLI is purposefully converting the -1 to a ? to indicate that the
state is unknown.



On Wed, Jul 22, 2015 at 4:09 PM, Kris Kobylinski <kriskobylinski(a)gmail.com>
wrote:

After pushing an app which fails at buildpack support, the following
parameters are observed:
"state": "STARTED"

"running_instances":-1
"package_state": "FAILED"

It seems that the -1 for running instances is problematic for the CF CLI
which shows something like the following :
name requested state instances memory disk urls
app started ?/1 1G 1G app
URL

Shouldn't the running_instances be 0 ? What is the meaning of -1 ?

Thank you,

Kris




--

________________________________________
http://kriskobylinski.mybluemix.net/ <http://koby.acndirect.com>



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




--

________________________________________
http://kriskobylinski.mybluemix.net/ <http://koby.acndirect.com>


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




--

________________________________________
http://kriskobylinski.mybluemix.net/ <http://koby.acndirect.com>

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Matthew Sykes
matthew.sykes(a)gmail.com


Re: "running_instances": -1

Kris Kobylinski <kriskobylinski@...>
 

Maggie,

I don't think the app state is a CLI problem - the state = STARTED for
failed staging comes from CC as a response to GET /v2/apps/app_guid/summary

Thanx for the tip with hm900 - I will investigate the number of instances
there.

Thank you,
Kris

On Thu, Jul 23, 2015 at 10:03 PM, Meng, Xiangyi <xiangyi.meng(a)emc.com>
wrote:

Hi, Kris



I think it may be a CLI 6.0 issue to show the “requested_state” not the
real state. I remembered in CLI 5.0 it would show stopped if application
failed to start.



But the “-1” you get has nothing with the status of application. I got
this value even when my application starts and runs successfully. The “-1”
just indicates that CC could not get your application’s status. If your
hm900 server works well, you will see 0/0 if your application fails at
deployment or staging stage.



Thanks,

Maggie



*From:* cf-dev-bounces(a)lists.cloudfoundry.org [mailto:
cf-dev-bounces(a)lists.cloudfoundry.org] *On Behalf Of *Kris Kobylinski
*Sent:* 2015年7月24日 9:53

*To:* Discussions about Cloud Foundry projects and the system overall.
*Subject:* Re: [cf-dev] "running_instances": -1



@Maggie -- you are probably right that -1 does not mean that the
deployment failed. But for my application the staging failed and I am
getting the following:

*"running_instances":-1,*

*"state": "STARTED"*,

*"package_state": "FAILED", "staging_failed_reason":
"BuildpackCompileFailed",*

It is very confusing. Why the app state is STARTED when the staging FAILED
? Also if the staging FAILED shouldn't the running_instances be 0 ?

What are all the scenarios where running_instances = -1 ?

Thank you,

Kris



On Thu, Jul 23, 2015 at 9:29 PM, Meng, Xiangyi <xiangyi.meng(a)emc.com>
wrote:

We had encountered the same issue before. The “?” or “-1” doesn’t mean the
deployment fails. It only means CC could not get the status of running
instance. Most likely there is something wrong with your ETCD server or
HM9000 server. You should check the log files from those servers.



Thanks,

Maggie



*From:* cf-dev-bounces(a)lists.cloudfoundry.org [mailto:
cf-dev-bounces(a)lists.cloudfoundry.org] *On Behalf Of *Kris Kobylinski
*Sent:* 2015年7月23日 20:42
*To:* Discussions about Cloud Foundry projects and the system overall.
*Subject:* Re: [cf-dev] "running_instances": -1



If the deployment failed wouldn't that mean that no instances are running
? Shouldn't the running_instances be 0 in that case ?


Thank you,

Kris



On Thu, Jul 23, 2015 at 2:06 AM, Dieu Cao <dcao(a)pivotal.io> wrote:

-1 indicates that the state of the instances is unknown.

The CLI is purposefully converting the -1 to a ? to indicate that the
state is unknown.



On Wed, Jul 22, 2015 at 4:09 PM, Kris Kobylinski <kriskobylinski(a)gmail.com>
wrote:

After pushing an app which fails at buildpack support, the following
parameters are observed:
"state": "STARTED"

"running_instances":-1
"package_state": "FAILED"

It seems that the -1 for running instances is problematic for the CF CLI
which shows something like the following :
name requested state instances memory disk urls
app started ?/1 1G 1G app
URL

Shouldn't the running_instances be 0 ? What is the meaning of -1 ?

Thank you,

Kris




--

________________________________________
http://kriskobylinski.mybluemix.net/ <http://koby.acndirect.com>



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




--

________________________________________
http://kriskobylinski.mybluemix.net/ <http://koby.acndirect.com>


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




--

________________________________________
http://kriskobylinski.mybluemix.net/ <http://koby.acndirect.com>

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
________________________________________
http://kriskobylinski.mybluemix.net/ <http://koby.acndirect.com>


Re: "running_instances": -1

MaggieMeng
 

Hi, Kris

I think it may be a CLI 6.0 issue to show the “requested_state” not the real state. I remembered in CLI 5.0 it would show stopped if application failed to start.

But the “-1” you get has nothing with the status of application. I got this value even when my application starts and runs successfully. The “-1” just indicates that CC could not get your application’s status. If your hm900 server works well, you will see 0/0 if your application fails at deployment or staging stage.

Thanks,
Maggie

From: cf-dev-bounces(a)lists.cloudfoundry.org [mailto:cf-dev-bounces(a)lists.cloudfoundry.org] On Behalf Of Kris Kobylinski
Sent: 2015年7月24日 9:53
To: Discussions about Cloud Foundry projects and the system overall.
Subject: Re: [cf-dev] "running_instances": -1

@Maggie -- you are probably right that -1 does not mean that the deployment failed. But for my application the staging failed and I am getting the following:
"running_instances":-1,
"state": "STARTED",
"package_state": "FAILED",
"staging_failed_reason": "BuildpackCompileFailed",
It is very confusing. Why the app state is STARTED when the staging FAILED ? Also if the staging FAILED shouldn't the running_instances be 0 ?
What are all the scenarios where running_instances = -1 ?
Thank you,
Kris

On Thu, Jul 23, 2015 at 9:29 PM, Meng, Xiangyi <xiangyi.meng(a)emc.com<mailto:xiangyi.meng(a)emc.com>> wrote:
We had encountered the same issue before. The “?” or “-1” doesn’t mean the deployment fails. It only means CC could not get the status of running instance. Most likely there is something wrong with your ETCD server or HM9000 server. You should check the log files from those servers.

Thanks,
Maggie

From: cf-dev-bounces(a)lists.cloudfoundry.org<mailto:cf-dev-bounces(a)lists.cloudfoundry.org> [mailto:cf-dev-bounces(a)lists.cloudfoundry.org<mailto:cf-dev-bounces(a)lists.cloudfoundry.org>] On Behalf Of Kris Kobylinski
Sent: 2015年7月23日 20:42
To: Discussions about Cloud Foundry projects and the system overall.
Subject: Re: [cf-dev] "running_instances": -1

If the deployment failed wouldn't that mean that no instances are running ? Shouldn't the running_instances be 0 in that case ?

Thank you,
Kris

On Thu, Jul 23, 2015 at 2:06 AM, Dieu Cao <dcao(a)pivotal.io<mailto:dcao(a)pivotal.io>> wrote:
-1 indicates that the state of the instances is unknown.
The CLI is purposefully converting the -1 to a ? to indicate that the state is unknown.

On Wed, Jul 22, 2015 at 4:09 PM, Kris Kobylinski <kriskobylinski(a)gmail.com<mailto:kriskobylinski(a)gmail.com>> wrote:
After pushing an app which fails at buildpack support, the following parameters are observed:
"state": "STARTED"
"running_instances":-1
"package_state": "FAILED"
It seems that the -1 for running instances is problematic for the CF CLI which shows something like the following :
name requested state instances memory disk urls
app started ?/1 1G 1G app URL
Shouldn't the running_instances be 0 ? What is the meaning of -1 ?
Thank you,
Kris


--
________________________________________
http://kriskobylinski.mybluemix.net/<http://koby.acndirect.com>

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



--
________________________________________
http://kriskobylinski.mybluemix.net/<http://koby.acndirect.com>

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



--
________________________________________
http://kriskobylinski.mybluemix.net/<http://koby.acndirect.com>


Re: "running_instances": -1

Kris Kobylinski <kriskobylinski@...>
 

@Maggie -- you are probably right that -1 does not mean that the deployment
failed. But for my application the staging failed and I am getting the
following:
*"running_instances":-1,*
* "state": "STARTED"*,

*"package_state": "FAILED","staging_failed_reason":
"BuildpackCompileFailed",*

It is very confusing. Why the app state is STARTED when the staging FAILED
? Also if the staging FAILED shouldn't the running_instances be 0 ?
What are all the scenarios where running_instances = -1 ?

Thank you,
Kris

On Thu, Jul 23, 2015 at 9:29 PM, Meng, Xiangyi <xiangyi.meng(a)emc.com> wrote:

We had encountered the same issue before. The “?” or “-1” doesn’t mean
the deployment fails. It only means CC could not get the status of running
instance. Most likely there is something wrong with your ETCD server or
HM9000 server. You should check the log files from those servers.



Thanks,

Maggie



*From:* cf-dev-bounces(a)lists.cloudfoundry.org [mailto:
cf-dev-bounces(a)lists.cloudfoundry.org] *On Behalf Of *Kris Kobylinski
*Sent:* 2015年7月23日 20:42
*To:* Discussions about Cloud Foundry projects and the system overall.
*Subject:* Re: [cf-dev] "running_instances": -1



If the deployment failed wouldn't that mean that no instances are running
? Shouldn't the running_instances be 0 in that case ?


Thank you,

Kris



On Thu, Jul 23, 2015 at 2:06 AM, Dieu Cao <dcao(a)pivotal.io> wrote:

-1 indicates that the state of the instances is unknown.

The CLI is purposefully converting the -1 to a ? to indicate that the
state is unknown.



On Wed, Jul 22, 2015 at 4:09 PM, Kris Kobylinski <kriskobylinski(a)gmail.com>
wrote:

After pushing an app which fails at buildpack support, the following
parameters are observed:
"state": "STARTED"

"running_instances":-1
"package_state": "FAILED"

It seems that the -1 for running instances is problematic for the CF CLI
which shows something like the following :
name requested state instances memory disk urls
app started ?/1 1G 1G app
URL

Shouldn't the running_instances be 0 ? What is the meaning of -1 ?

Thank you,

Kris




--

________________________________________
http://kriskobylinski.mybluemix.net/ <http://koby.acndirect.com>



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




--

________________________________________
http://kriskobylinski.mybluemix.net/ <http://koby.acndirect.com>

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

--
________________________________________
http://kriskobylinski.mybluemix.net/ <http://koby.acndirect.com>


Re: "running_instances": -1

MaggieMeng
 

We had encountered the same issue before. The “?” or “-1” doesn’t mean the deployment fails. It only means CC could not get the status of running instance. Most likely there is something wrong with your ETCD server or HM9000 server. You should check the log files from those servers.

Thanks,
Maggie

From: cf-dev-bounces(a)lists.cloudfoundry.org [mailto:cf-dev-bounces(a)lists.cloudfoundry.org] On Behalf Of Kris Kobylinski
Sent: 2015年7月23日 20:42
To: Discussions about Cloud Foundry projects and the system overall.
Subject: Re: [cf-dev] "running_instances": -1

If the deployment failed wouldn't that mean that no instances are running ? Shouldn't the running_instances be 0 in that case ?

Thank you,
Kris

On Thu, Jul 23, 2015 at 2:06 AM, Dieu Cao <dcao(a)pivotal.io<mailto:dcao(a)pivotal.io>> wrote:
-1 indicates that the state of the instances is unknown.
The CLI is purposefully converting the -1 to a ? to indicate that the state is unknown.

On Wed, Jul 22, 2015 at 4:09 PM, Kris Kobylinski <kriskobylinski(a)gmail.com<mailto:kriskobylinski(a)gmail.com>> wrote:
After pushing an app which fails at buildpack support, the following parameters are observed:
"state": "STARTED"
"running_instances":-1
"package_state": "FAILED"
It seems that the -1 for running instances is problematic for the CF CLI which shows something like the following :
name requested state instances memory disk urls
app started ?/1 1G 1G app URL
Shouldn't the running_instances be 0 ? What is the meaning of -1 ?
Thank you,
Kris


--
________________________________________
http://kriskobylinski.mybluemix.net/<http://koby.acndirect.com>

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



--
________________________________________
http://kriskobylinski.mybluemix.net/<http://koby.acndirect.com>


Re: Incubation Proposal: Project CF-Abacus

Onsi Fakhouri <ofakhouri@...>
 

Hooray! Looking forward to what Abacus will bring to the platform!

Onsi

On Thu, Jul 23, 2015 at 5:16 PM, Shannon Coen <scoen(a)pivotal.io> wrote:

This morning on the Services PMC call, Abacus was accepted into incubation.

Thank you Max, and your colleagues at IBM, for your months of effort to
open source this project.

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Wed, Jul 15, 2015 at 8:40 AM, Chris Sterling <chris.sterling(a)gmail.com>
wrote:

This looks like a useful service. Thanks for the contribution!

Chris Sterling
chris.sterling(a)gmail.com
twitter: @csterwa
linkedin: http://www.linkedin.com/in/chrissterling

On Wed, Jul 15, 2015 at 8:37 AM, Duncan Johnston Watt <
duncan.johnstonwatt(a)cloudsoftcorp.com> wrote:

+1 this looks incredibly useful.

Best

Duncan Johnston-Watt
CEO | Cloudsoft Corporation

Twitter | @duncanjw
Mobile | +44 777 190 2653
Skype | duncan_johnstonwatt
Linkedin | www.linkedin.com/in/duncanjohnstonwatt

On 15 July 2015 at 15:07, James Bayer <jbayer(a)pivotal.io> wrote:

thanks to ibm for this proposal! we're excited to review this in more
depth.

what this means for service authors is that they would have a standard
system component and format for to track custom billing metrics for service
usage other than "how long was this service deployed". consider an email
service, you could track usage for every email sent or for a data service
you could track the amount of disk usage for data storage. it opens up a
lot of flexibility for how to do chargeback and showback.

for operators, in addition to apps, if the services provided on the
platform use this approach or are translated to this, it provides a
standard way to roll-up usage for a tenant across all of their platform
usage and present it nicely aggregated.

looking forward to the feedback,

james

On Tue, Jul 14, 2015 at 8:30 PM, Michael Maximilien <maxim(a)us.ibm.com>
wrote:

Hi,

This email is to introduce and propose project Abacus to the CF
community for incubation.

*What?*

The IBM Bluemix PaaS (based on CloudFoundry) includes a billing and
metering component such that users can project and drill down on the costs
they have incurred or will incur as they use the system.

Part of this component includes not only APIs that service brokers use
to provide usage data, but also, how to aggregate that information, as well
as a component to expose the aggregated data into a format (JSON) that can
be subsequently consumed by users — this is done via a drill down UI in
Bluemix.

Project Abacus is an extracted Open Source Software version (under
Apache 2.0 license) of the APIs, usage metering, and aggregation components
from the Bluemix feature.

Finally, it’s important to state that project Abacus uses a
micro-services architecture and is written in Node.js with CouchDB
compatible DBs as datastore and for map/reduce aggregations. Also, all
components can actually run as CF apps or be externalized. In Bluemix they
run as apps.

In other words, this is a bit different from other CF components,
however, we have reviewed Abacus with our colleagues from Pivotal and they
also believe it warrants incubation.

*When?*

Now.

Find the first self-contained release here:
https://github.com/jsdelfino/cf-abacus/archive/v0.0.1.tar.gz

There is a comprehensive README
<https://github.com/jsdelfino/cf-abacus/blob/master/README.md>that
can help you setup and try this first release. The team has included a fake
broker that can be used to generate usage data so you can try the whole
thing out of the box.

*How?*

We plan to introduce this as part of the CAB call discussion tomorrow
07/15 and field any questions.

The next step will be to get approval and become an incubation
project. Once this is done we will be staffing the project with a team
(many from the Bluemix engineers that created it) as well as a PM, and
follow the CFF guidelines for new projects.

We are ready to move on this with no delays and we welcome all
interested parties to try out the first release and to help us refine the
APIs and subsequent releases.

------
Best,

IBM CF team

dr.max
ibm cloud labs
silicon valley, ca

Sent from my iPhone

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Thank you,

James Bayer

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

Cloudsoft Corporation Limited, Registered in Scotland No: SC349230.
Registered Office: 13 Dryden Place, Edinburgh, EH9 1RP

This e-mail message is confidential and for use by the addressee only.
If the message is received by anyone other than the addressee, please
return the message to the sender by replying to it and then delete the
message from your computer. Internet e-mails are not necessarily secure.
Cloudsoft Corporation Limited does not accept responsibility for changes
made to this message after it was sent.

Whilst all reasonable care has been taken to avoid the transmission of
viruses, it is the responsibility of the recipient to ensure that the
onward transmission, opening or use of this message and any attachments
will not adversely affect its systems or data. No responsibility is
accepted by Cloudsoft Corporation Limited in this regard and the recipient
should carry out such virus and other checks as it considers appropriate.
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Incubation Proposal: Project CF-Abacus

Shannon Coen
 

This morning on the Services PMC call, Abacus was accepted into incubation.

Thank you Max, and your colleagues at IBM, for your months of effort to
open source this project.

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Wed, Jul 15, 2015 at 8:40 AM, Chris Sterling <chris.sterling(a)gmail.com>
wrote:

This looks like a useful service. Thanks for the contribution!

Chris Sterling
chris.sterling(a)gmail.com
twitter: @csterwa
linkedin: http://www.linkedin.com/in/chrissterling

On Wed, Jul 15, 2015 at 8:37 AM, Duncan Johnston Watt <
duncan.johnstonwatt(a)cloudsoftcorp.com> wrote:

+1 this looks incredibly useful.

Best

Duncan Johnston-Watt
CEO | Cloudsoft Corporation

Twitter | @duncanjw
Mobile | +44 777 190 2653
Skype | duncan_johnstonwatt
Linkedin | www.linkedin.com/in/duncanjohnstonwatt

On 15 July 2015 at 15:07, James Bayer <jbayer(a)pivotal.io> wrote:

thanks to ibm for this proposal! we're excited to review this in more
depth.

what this means for service authors is that they would have a standard
system component and format for to track custom billing metrics for service
usage other than "how long was this service deployed". consider an email
service, you could track usage for every email sent or for a data service
you could track the amount of disk usage for data storage. it opens up a
lot of flexibility for how to do chargeback and showback.

for operators, in addition to apps, if the services provided on the
platform use this approach or are translated to this, it provides a
standard way to roll-up usage for a tenant across all of their platform
usage and present it nicely aggregated.

looking forward to the feedback,

james

On Tue, Jul 14, 2015 at 8:30 PM, Michael Maximilien <maxim(a)us.ibm.com>
wrote:

Hi,

This email is to introduce and propose project Abacus to the CF
community for incubation.

*What?*

The IBM Bluemix PaaS (based on CloudFoundry) includes a billing and
metering component such that users can project and drill down on the costs
they have incurred or will incur as they use the system.

Part of this component includes not only APIs that service brokers use
to provide usage data, but also, how to aggregate that information, as well
as a component to expose the aggregated data into a format (JSON) that can
be subsequently consumed by users — this is done via a drill down UI in
Bluemix.

Project Abacus is an extracted Open Source Software version (under
Apache 2.0 license) of the APIs, usage metering, and aggregation components
from the Bluemix feature.

Finally, it’s important to state that project Abacus uses a
micro-services architecture and is written in Node.js with CouchDB
compatible DBs as datastore and for map/reduce aggregations. Also, all
components can actually run as CF apps or be externalized. In Bluemix they
run as apps.

In other words, this is a bit different from other CF components,
however, we have reviewed Abacus with our colleagues from Pivotal and they
also believe it warrants incubation.

*When?*

Now.

Find the first self-contained release here:
https://github.com/jsdelfino/cf-abacus/archive/v0.0.1.tar.gz

There is a comprehensive README
<https://github.com/jsdelfino/cf-abacus/blob/master/README.md>that can
help you setup and try this first release. The team has included a fake
broker that can be used to generate usage data so you can try the whole
thing out of the box.

*How?*

We plan to introduce this as part of the CAB call discussion tomorrow
07/15 and field any questions.

The next step will be to get approval and become an incubation project.
Once this is done we will be staffing the project with a team (many from
the Bluemix engineers that created it) as well as a PM, and follow the CFF
guidelines for new projects.

We are ready to move on this with no delays and we welcome all
interested parties to try out the first release and to help us refine the
APIs and subsequent releases.

------
Best,

IBM CF team

dr.max
ibm cloud labs
silicon valley, ca

Sent from my iPhone

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Thank you,

James Bayer

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

Cloudsoft Corporation Limited, Registered in Scotland No: SC349230.
Registered Office: 13 Dryden Place, Edinburgh, EH9 1RP

This e-mail message is confidential and for use by the addressee only. If
the message is received by anyone other than the addressee, please return
the message to the sender by replying to it and then delete the message
from your computer. Internet e-mails are not necessarily secure. Cloudsoft
Corporation Limited does not accept responsibility for changes made to this
message after it was sent.

Whilst all reasonable care has been taken to avoid the transmission of
viruses, it is the responsibility of the recipient to ensure that the
onward transmission, opening or use of this message and any attachments
will not adversely affect its systems or data. No responsibility is
accepted by Cloudsoft Corporation Limited in this regard and the recipient
should carry out such virus and other checks as it considers appropriate.
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Allow gorouter to log random headers.

Shannon Coen
 

This is not something that would be merged, as originally proposed, without
additional investigation and discussion.

If I understand correctly, you're proposing that an operator of
CloudFoundry configure GoRouter, which is a multi-tenant, shared service,
with knowledge specific to one or a few applications. This should not be an
operator responsibility, nor should the solution be specific to one or a
few applications.

The goal is "the flexibility of being able to annotate our logs with what
we consider to be important for our debugging purposes." More specifically
you're requesting logging of headers. Do you have a preference?

If GoRouter logged whatever headers were included in the request, wouldn't
this satisfy your requirements? Doesn't GoRouter do this already?

I'm interested in solving your requirement generically for all
applications, and focussing the user experience on the correct persona.
Based on what you've described, the persona is the app developer, so
control of what is logged should be in their hands. I'm also not convinced
GoRouter should have any knowledge of headers specific to one application
or another.





Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Wed, Jul 22, 2015 at 3:05 AM, Alex Lomov <alexander.lomov(a)altoros.com>
wrote:

Some time ago routing services were discussed on a CAB [1]. Here is a
description of this proposal.

Do you think that using such service will allow your developers to cover
this requirements?

[1]
http://www.activestate.com/blog/2015/02/cloud-foundry-advisory-board-meeting-2015-february
[2]
https://docs.google.com/document/d/1bGOQxiKkmaw6uaRWGd-sXpxL0Y28d3QihcluI15FiIA/edit#heading=h.8djffzes9pnb

On Jul 21, 2015, at 4:06 PM, Simon Johansson <simon(a)simonjohansson.com>
wrote:

Howdie!

We have some devs who want to be able to trace a request troughout their
applications.

user -> a -> b -> c
|
|-> d -> e

When a user makes a request to "a" uuid is generated inside the app, and
the request to "b" from "a" will set a header(call it WAKAWAKA to uuid),
"b"'s call will passthrough WAKAWAKA to "c" and "d", "d" will passthrough
WAKAWAKA to "e". Etc.

We aggregate all RTR logs into ELK so it would be super helpful to them to
be able to filter on WAKAWAKA and get all the access logs(and app logs
aswell, they mostly use GELF so its easy for them to add whatewher field
they want) from the services involved.

I had a quick peek at the gorouter(
https://github.com/cloudfoundry/gorouter/blob/76668f5818ea8c089ff52a14fcdfbf703c8e8767/access_log/access_log_record.go#L40)
and it seems like this should be a simple PR.

1. To gorouter.yml add
passthrough_headers:
- WAKAWAKA
- X-Random-Header

2. In makeRecord at the bottom add something like(in psuedo)

data = {}
for header in passthrough_headers:
header_val = r.FormatRequestHeader("X-Forwarded-For")
if header_val:
passthrough_headers[header] = header_val

if data:
fmt.Fprintf(b, data.to_stringified_json())

That would yield a log line like
blurgh.dev.cf.private.domain.com - [21/07/2015:10:17:05 +0000] "GET
/statements?ascending=true&since=2015-06-30T14%3A10%3A03.078Z&skipStatementId=30a88204-0779-4385-9859-e4aabd30baf0
HTTP/1.1" 200 0 17 "-" "NING/1.0" 10.230.31.2:46204 x_forwarded_for:"-"
vcap_request_id:1e58195a-cde6-4afd-7f03-43061c9ea91c
response_time:0.004927106 app_id:9784cd03-050d-4b74-9e90-5f17134a3f08
{"WAKAWAKA": "Space is the place", "X-Random-Header": "Once upon a midnight
dreary, while I pondered weak and weary"}

The reason for a stringified JSON is to make it easy to parse with
logstash or other loganalysis tools.

Before I spend time implementing, is this something you would merge?
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Allow gorouter to log random headers.

Guillaume Berche
 

Simon,

Some buildpacks such as java-buildpack and php, offer the ability to log
incoming http request and control format of the logs inc custom headers.
Would that help in your case ?

https://github.com/cloudfoundry/java-buildpack/issues/57
https://github.com/cloudfoundry/java-buildpack/blob/master/docs/container-tomcat.md#configuration

Guillaume.
Le 22 juil. 2015 11:50, "Simon Johansson" <simon(a)simonjohansson.com> a
écrit :

Well, from my point of view the customization is just adding the
flexibility of being able to annotate our logs with what we consider to be
important for our debugging purposes, a feature that is surely interesting
for other parties aswell. We are not interested in adding extra headers
into CF, just pulling headers from incoming requests(where the headers have
ben set elsewhere) to Gorouter into a doppler event. :)

On Wed, Jul 22, 2015 at 11:43 AM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

Simon I know what you want todo :) I am shinji62 lol
I am jsut afraid that you want to customize the platform and you will be
stuck with because other don't support custom header.

On Wed, Jul 22, 2015 at 6:36 PM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

I feel that we are stearing this conversation into the wrong direction.

Gist it, I would like to with cf logs app / firehose-to-syslog see my
RTR-logs as

blurgh.dev.cf.private.domain.com - blablablab
vcap_request_id:1e58195a-cde6-4afd-7f03-43061c9ea91c
response_time:0.004927106 app_id:9784cd03-050d-4b74-9e90-5f17134a3f08
*{"X-Random-Header": "Once upon a midnight dreary, while I pondered weak
and weary", "Another-Header": "wryyyy"}*

By being able to tell Gorouter what headers Im interested in via the
manifest
logging:
add_extra_headers_if_available:
- X-Random-Header
- Another-Header
- This-header-will-only-be-in-1%-of-the-requests

Is this something you would be interested in mergin, I dont want to
implement it if there is no chance of it being merged.


On Wed, Jul 22, 2015 at 11:30 AM, Gwenn Etourneau <getourneau(a)pivotal.io
wrote:
So if you move to Heroku you will be able to change the platform ??? I
don't think so..
So you can put the new header from your app or at least the runtime
buildpack or docker.

Your application should be platform dependent that's why you need to
implement this header into your application to avoid any lock-in or forking
issue.

On Wed, Jul 22, 2015 at 6:27 PM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

WAKAWAKA is just an example
But the difference is that WAKAWAKA is not platform specific whereas
X-Cf-Requestid is. If we want to move our app to Heroku, or a VM, or
whatewher we have platform specific implementation details in our
app(namely we rely on a header that is not there anymore). But that is not
the point of this thread.

The point is,

say we have an app that is fronted by a CDN, and the CDN sets the
X-Im-a-shark header with some value that we are interested to see in our
logs. The easiest way to achivie this without having to implement it into
our own apps is just to tell the Gorouter that it should append the value
of that header into the string that it logs so the event that flow via
Doppler and ultimately into cf logs/ELK will contain that value.

The reason why this would be such a valuable feature for us is that we
dont have to do anything. CF already provide the out of the box facility to
give us routing logs, so if we can piggy back on that for what we are
interested in we dont have to add libraries to our apps to log interesting
headers on the side.



On Wed, Jul 22, 2015 at 12:45 AM, Shannon Coen <scoen(a)pivotal.io>
wrote:

I don't see the difference between WAKAWAKA and X-Cf-Requestid.
Gorouter would have to add some header with a uuid for the request. Your
apps have to have logic to pass this header on, so that a log search
returns the original request as well as subsequent requests between apps.
Could you please clarify?

Thank you,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Tue, Jul 21, 2015 at 11:09 AM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

Sure that header can be used. But then we are adding CF specific
stuff into the implementation of our services, which is something we want
to avoid at all costs.
Also Im not entirely sure if all the libraries we use for Zipkin
supports using custom headers.

All our public apps are fronted by different CDNs, which sets
headers that we might want to store for debugging, so we still need a way
to pass those trough into the log.

On Tuesday, 21 July 2015, Shannon Coen <scoen(a)pivotal.io> wrote:

Hello Simon,

The X-Cf-Requestid header already provides a uuid. Couldn't app "a"
add the value of X-Cf-Requestid to a header of your choosing? Call it
WAKAWAKA or X-Random-Header, but it doesn't need to be a platform standard.
Wouldn't a search for the value of X-Cf-Requestid then provide the desired
results?

Thank you,
Shannon

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Tue, Jul 21, 2015 at 6:06 AM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

Howdie!

We have some devs who want to be able to trace a request troughout
their applications.

user -> a -> b -> c
|
|-> d -> e

When a user makes a request to "a" uuid is generated inside the
app, and the request to "b" from "a" will set a header(call it WAKAWAKA to
uuid), "b"'s call will passthrough WAKAWAKA to "c" and "d", "d" will
passthrough WAKAWAKA to "e". Etc.

We aggregate all RTR logs into ELK so it would be super helpful to
them to be able to filter on WAKAWAKA and get all the access logs(and app
logs aswell, they mostly use GELF so its easy for them to add whatewher
field they want) from the services involved.

I had a quick peek at the gorouter(
https://github.com/cloudfoundry/gorouter/blob/76668f5818ea8c089ff52a14fcdfbf703c8e8767/access_log/access_log_record.go#L40)
and it seems like this should be a simple PR.

1. To gorouter.yml add
passthrough_headers:
- WAKAWAKA
- X-Random-Header

2. In makeRecord at the bottom add something like(in psuedo)

data = {}
for header in passthrough_headers:
header_val = r.FormatRequestHeader("X-Forwarded-For")
if header_val:
passthrough_headers[header] = header_val

if data:
fmt.Fprintf(b, data.to_stringified_json())

That would yield a log line like
blurgh.dev.cf.private.domain.com - [21/07/2015:10:17:05 +0000]
"GET
/statements?ascending=true&since=2015-06-30T14%3A10%3A03.078Z&skipStatementId=30a88204-0779-4385-9859-e4aabd30baf0
HTTP/1.1" 200 0 17 "-" "NING/1.0" 10.230.31.2:46204
x_forwarded_for:"-" vcap_request_id:1e58195a-cde6-4afd-7f03-43061c9ea91c
response_time:0.004927106 app_id:9784cd03-050d-4b74-9e90-5f17134a3f08
{"WAKAWAKA": "Space is the place", "X-Random-Header": "Once upon a midnight
dreary, while I pondered weak and weary"}

The reason for a stringified JSON is to make it easy to parse with
logstash or other loganalysis tools.

Before I spend time implementing, is this something you would
merge?

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Announcing support for Arbitrary Parameters and Service Instance Tags

john mcteague <john.mcteague@...>
 

Great news, we have been eagerly awaiting this feature.

John

On Thu, Jul 23, 2015 at 9:31 PM, Dieu Cao <dcao(a)pivotal.io> wrote:

On behalf of the CAPI team I'm pleased to announce support for Arbitrary
Parameters and Service Instance Tags, two features which extend the
possibilities for Cloud Foundry Marketplace service offerings, and provide
developers with increased flexibility in how services are consumed by
applications.

Support for Arbitrary Parameters is introduced with cf-release v209 and
CLI v6.12.1. This features enables service providers to support
user-provided configuration options with the create, update, and bind
service instance operations, and the create service key operation (stay
tuned for a forthcoming announcement of support for Service Keys).
Previously, this could only be achieved by providing many plans to cover
various combinations of configuration options, or to provide a service
instance dashboard that users can SSO into and adjust configuration options
after creation. Although the platform and CLI now support the feature,
service broker authors must implement support for the feature as described
in the Service Broker API v2.5 specification
<http://docs.cloudfoundry.org/services/api.html>.

Support for Instance Tags is introduced with cf-release v211 and CLI
v6.12.1. Since v2.0 of the Service Broker API, broker authors have be able
to provide tags for a service offering in the /v2/catalog endpoint that
Cloud Foundry delivers to applications in the VCAP_SERVICES Environment
Variable
<http://docs.cloudfoundry.org/devguide/deploy-apps/environment-variable.html#VCAP-SERVICES>.
These tags provide developers with a more generic way for applications to
parse VCAP_SERVICES for credentials. Developers may now provide their own
tags when creating or updating a service instance by including a
comma-separated list of tags with the -t flag.

Documentation:

- http://docs.cloudfoundry.org/services/api.html

- http://docs.cloudfoundry.org/devguide/services/managing-services.html

Special thanks to the former CF Services API team and Shannon for their
hard work on these features.

-Dieu
CF CAPI PM

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Announcing support for Arbitrary Parameters and Service Instance Tags

Juan Pablo Genovese
 

Nice! Thanks for the hard work!

2015-07-23 17:31 GMT-03:00 Dieu Cao <dcao(a)pivotal.io>:

On behalf of the CAPI team I'm pleased to announce support for Arbitrary
Parameters and Service Instance Tags, two features which extend the
possibilities for Cloud Foundry Marketplace service offerings, and provide
developers with increased flexibility in how services are consumed by
applications.

Support for Arbitrary Parameters is introduced with cf-release v209 and
CLI v6.12.1. This features enables service providers to support
user-provided configuration options with the create, update, and bind
service instance operations, and the create service key operation (stay
tuned for a forthcoming announcement of support for Service Keys).
Previously, this could only be achieved by providing many plans to cover
various combinations of configuration options, or to provide a service
instance dashboard that users can SSO into and adjust configuration options
after creation. Although the platform and CLI now support the feature,
service broker authors must implement support for the feature as described
in the Service Broker API v2.5 specification
<http://docs.cloudfoundry.org/services/api.html>.

Support for Instance Tags is introduced with cf-release v211 and CLI
v6.12.1. Since v2.0 of the Service Broker API, broker authors have be able
to provide tags for a service offering in the /v2/catalog endpoint that
Cloud Foundry delivers to applications in the VCAP_SERVICES Environment
Variable
<http://docs.cloudfoundry.org/devguide/deploy-apps/environment-variable.html#VCAP-SERVICES>.
These tags provide developers with a more generic way for applications to
parse VCAP_SERVICES for credentials. Developers may now provide their own
tags when creating or updating a service instance by including a
comma-separated list of tags with the -t flag.

Documentation:

- http://docs.cloudfoundry.org/services/api.html

- http://docs.cloudfoundry.org/devguide/services/managing-services.html

Special thanks to the former CF Services API team and Shannon for their
hard work on these features.

-Dieu
CF CAPI PM

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com


Announcing support for Arbitrary Parameters and Service Instance Tags

Dieu Cao <dcao@...>
 

On behalf of the CAPI team I'm pleased to announce support for Arbitrary
Parameters and Service Instance Tags, two features which extend the
possibilities for Cloud Foundry Marketplace service offerings, and provide
developers with increased flexibility in how services are consumed by
applications.

Support for Arbitrary Parameters is introduced with cf-release v209 and CLI
v6.12.1. This features enables service providers to support user-provided
configuration options with the create, update, and bind service instance
operations, and the create service key operation (stay tuned for a
forthcoming announcement of support for Service Keys). Previously, this
could only be achieved by providing many plans to cover various
combinations of configuration options, or to provide a service instance
dashboard that users can SSO into and adjust configuration options after
creation. Although the platform and CLI now support the feature, service
broker authors must implement support for the feature as described in
the Service
Broker API v2.5 specification
<http://docs.cloudfoundry.org/services/api.html>.

Support for Instance Tags is introduced with cf-release v211 and CLI
v6.12.1. Since v2.0 of the Service Broker API, broker authors have be able
to provide tags for a service offering in the /v2/catalog endpoint that
Cloud Foundry delivers to applications in the VCAP_SERVICES Environment
Variable
<http://docs.cloudfoundry.org/devguide/deploy-apps/environment-variable.html#VCAP-SERVICES>.
These tags provide developers with a more generic way for applications to
parse VCAP_SERVICES for credentials. Developers may now provide their own
tags when creating or updating a service instance by including a
comma-separated list of tags with the -t flag.

Documentation:

- http://docs.cloudfoundry.org/services/api.html

- http://docs.cloudfoundry.org/devguide/services/managing-services.html

Special thanks to the former CF Services API team and Shannon for their
hard work on these features.

-Dieu
CF CAPI PM