Date   

Re: Invalid password change request

Gwenn Etourneau
 

I think "schemas":["urn:scim:schemas:core:1.0"] is missing into your body
request no ?

On Wed, Jul 29, 2015 at 6:16 PM, Kayode Odeyemi <dreyemi(a)gmail.com> wrote:

Hi,

I have a valid token with the right scopes and authorities as seen below
when decoded:


{"jti":"06ef4e8d-2dc9-4458-9aca-ef89384861c6","sub":"85d99b6e-eaeb-4171-98ba-3e6843f577d7","scope":["password.write","openid","oauth.approvals"],"client_id":"useraccount","cid":"useraccount","azp":"useraccount","grant_type":"password","user_id":"85d99b6e-eaeb-4171-98ba-3e6843f577d7","user_name":"johndoeyAgB4wxoe","email":"
kay1(a)email.com","iat":1438159365,"exp":1438202565,"iss":"
http://localhost:8080/uaa/oauth/token
","aud":["useraccount","password","openid","oauth"]}

However, when I make a request to /Users/[userid]/password, I get the
following error:

"authentication":null,"extraInformation":null,"message":"Invalid password
change request","localizedMessage":"Invalid password change
request","suppressed":[]

The full error log is attached.

To replicate, this is the sample request:

curl -v -H 'Authorization: Bearer
eyJhbGciOiJIUzI1NiJ9.eyJqdGkiOiIwNmVmNGU4ZC0yZGM5LTQ0NTgtOWFjYS1lZjg5Mzg0ODYxYzYiLCJzdWIiOiI4NWQ5OWI2ZS1lYWViLTQxNzEtOThiYS0zZTY4NDNmNTc3ZDciLCJzY29wZSI6WyJwYXNzd29yZC53cml0ZSIsIm9wZW5pZCIsIm9hdXRoLmFwcHJvdmFscyJdLCJjbGllbnRfaWQiOiJ1c2VyYWNjb3VudCIsImNpZCI6InVzZXJhY2NvdW50IiwiYXpwIjoidXNlcmFjY291bnQiLCJncmFudF90eXBlIjoicGFzc3dvcmQiLCJ1c2VyX2lkIjoiODVkOTliNmUtZWFlYi00MTcxLTk4YmEtM2U2ODQzZjU3N2Q3IiwidXNlcl9uYW1lIjoiam9obmRvZXlBZ0I0d3hvZSIsImVtYWlsIjoia2F5MUB5b29rb3MuY29tIiwiaWF0IjoxNDM4MTU5MzY1LCJleHAiOjE0MzgyMDI1NjUsImlzcyI6Imh0dHA6Ly9sb2NhbGhvc3Q6ODA4MC91YWEvb2F1dGgvdG9rZW4iLCJhdWQiOlsidXNlcmFjY291bnQiLCJwYXNzd29yZCIsIm9wZW5pZCIsIm9hdXRoIl19.dz1ysSNt-TYvVspIqxXizBJp6ZahIv7Q5INwvYmJ430'
-H 'Content-Type: application/json' -d '{"password":"newpass",
"oldPassword":"oldpass03"}' -X PUT
http://localhost:8080/uaa/Users/e39919f6-6f47-45c5-915d-734b9b2f1387/password


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Invalid password change request

Paul Bakare
 

Hi,

I have a valid token with the right scopes and authorities as seen below
when decoded:

{"jti":"06ef4e8d-2dc9-4458-9aca-ef89384861c6","sub":"85d99b6e-eaeb-4171-98ba-3e6843f577d7","scope":["password.write","openid","oauth.approvals"],"client_id":"useraccount","cid":"useraccount","azp":"useraccount","grant_type":"password","user_id":"85d99b6e-eaeb-4171-98ba-3e6843f577d7","user_name":"johndoeyAgB4wxoe","email":"
kay1(a)email.com","iat":1438159365,"exp":1438202565,"iss":"
http://localhost:8080/uaa/oauth/token
","aud":["useraccount","password","openid","oauth"]}

However, when I make a request to /Users/[userid]/password, I get the
following error:

"authentication":null,"extraInformation":null,"message":"Invalid password
change request","localizedMessage":"Invalid password change
request","suppressed":[]

The full error log is attached.

To replicate, this is the sample request:

curl -v -H 'Authorization: Bearer
eyJhbGciOiJIUzI1NiJ9.eyJqdGkiOiIwNmVmNGU4ZC0yZGM5LTQ0NTgtOWFjYS1lZjg5Mzg0ODYxYzYiLCJzdWIiOiI4NWQ5OWI2ZS1lYWViLTQxNzEtOThiYS0zZTY4NDNmNTc3ZDciLCJzY29wZSI6WyJwYXNzd29yZC53cml0ZSIsIm9wZW5pZCIsIm9hdXRoLmFwcHJvdmFscyJdLCJjbGllbnRfaWQiOiJ1c2VyYWNjb3VudCIsImNpZCI6InVzZXJhY2NvdW50IiwiYXpwIjoidXNlcmFjY291bnQiLCJncmFudF90eXBlIjoicGFzc3dvcmQiLCJ1c2VyX2lkIjoiODVkOTliNmUtZWFlYi00MTcxLTk4YmEtM2U2ODQzZjU3N2Q3IiwidXNlcl9uYW1lIjoiam9obmRvZXlBZ0I0d3hvZSIsImVtYWlsIjoia2F5MUB5b29rb3MuY29tIiwiaWF0IjoxNDM4MTU5MzY1LCJleHAiOjE0MzgyMDI1NjUsImlzcyI6Imh0dHA6Ly9sb2NhbGhvc3Q6ODA4MC91YWEvb2F1dGgvdG9rZW4iLCJhdWQiOlsidXNlcmFjY291bnQiLCJwYXNzd29yZCIsIm9wZW5pZCIsIm9hdXRoIl19.dz1ysSNt-TYvVspIqxXizBJp6ZahIv7Q5INwvYmJ430'
-H 'Content-Type: application/json' -d '{"password":"newpass",
"oldPassword":"oldpass03"}' -X PUT
http://localhost:8080/uaa/Users/e39919f6-6f47-45c5-915d-734b9b2f1387/password


Re: Allow gorouter to log random headers.

James Bayer
 

awesome! thanks for the contribution you two!

On Mon, Jul 27, 2015 at 11:55 PM, Simon Johansson <simon(a)simonjohansson.com>
wrote:

Here is a PR David and I worked on this morning to implement the above

https://github.com/cloudfoundry/gorouter/pull/92

Comments are appreciated.

On Tue, Jul 28, 2015 at 6:46 AM, David Laing <david(a)davidlaing.com> wrote:

Shannon,

Understood. PR for "as an operator of Cloud Foundry, I want to enable
logging of specified headers for all applications." coming up.

Experience will tell whether there are any sensible defaults; but we can
leave that for a future enhancement of this functionality.

Regards

David

On 28 July 2015 at 03:54, Shannon Coen <scoen(a)pivotal.io> wrote:

Thank you, James. That helps.

What bothered me about this thread was that the use case was presented
as, "as an unspecified persona, I want to customize the multi-tenant router
to log specified headers for a few apps". This is troubling.

Instead it sounds like the use case is "as an operator of Cloud Foundry,
I want to enable logging of specified headers for all applications." This
is more compelling.

I'm skeptical that this manifest property should have a default. Who's
to say what the minimum list of "sensible" defaults is? Does WAKAWAKA make
the cut? j/k.

We'll take a PR.

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Fri, Jul 24, 2015 at 8:46 AM, David Laing <david(a)davidlaing.com>
wrote:

James,

Yep - we're pretty much on the same page.

Only addition I would ask for is that the whitelist contain some
"sensible" defaults (eg, Trace-Id, Span-Id) that are switched on by
default; since then tight integration with tools like spring-cloud /
buildpacks would work out the box.

D

On 24 July 2015 at 16:26, James Bayer <jbayer(a)pivotal.io> wrote:

shannon,

from what i'm reading here about the use case, the main interest is
that a CF operator knows that their cf installation is more deeply
integrated with a specific log parsing solution for all/many apps on that
platform that choose to use it (whether than is ELK, Zipkin, etc). it does
not sound like it is a special case with lots of variation by many
different app teams on the cf installation. rather, it sounds like this is
most likely to be used as an installation-wide option to enhance the app
developer / app operations experience.

it seems like an operator configured whitelist set of headers that get
logged with the RTR message satisfies the current needs well and is
reasonable.

if we were to find that in the future lots of variation and different
app teams on the same CF wanted to have the RTR tier log many different
headers in the access log, then we could enhance the "log the whitelist of
headers to the access log" capability to be exposed to a limited number of
headers that each route could be configured by developers to specify that
would apply in addition to the operator configured logged headers. but it
sounds like that isn't needed right now.

simon/david, did i summarize this correctly?

On Fri, Jul 24, 2015 at 7:23 AM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

If I understand correctly, you're proposing that an operator of
CloudFoundry configure GoRouter, which is a multi-tenant, shared service,
with knowledge specific to one or a few applications

That is indeed the proposal. In my org we work closely with the
different development streams to provide a good out of the box experience
by creating certain opt-in features that makes their life easier. This
would be one of those features, if you pass certain headers in your
requests thats standardised across the entire online organisation you will
be able to query for them in Kibana.
I understand that this is not the case in all environments.

If GoRouter logged whatever headers were included in the request,
wouldn't this satisfy your requirements?

This would indeed satisfy my requests, but as David points out("However,
not having a whitelist of headers to log opens a possible DDOS attack
vector on the GoRouter") it might now be appropriate.

Based on what you've described, the persona is the app developer.
Not necessarily, we operators are also very interested in certain
headers.

so control of what is logged should be in their hands.
If one via the API could set headers that should be logged for an
org, space or application that would be magical. The need for a list of
"must-log-all-these-headers" would still be there I think as not to have to
maintain a list of these standardised headers across different objects.


On Thu, Jul 23, 2015 at 10:51 PM, Shannon Coen <scoen(a)pivotal.io>
wrote:

This is not something that would be merged, as originally proposed,
without additional investigation and discussion.

If I understand correctly, you're proposing that an operator of
CloudFoundry configure GoRouter, which is a multi-tenant, shared service,
with knowledge specific to one or a few applications. This should not be an
operator responsibility, nor should the solution be specific to one or a
few applications.

The goal is "the flexibility of being able to annotate our logs
with what we consider to be important for our debugging purposes." More
specifically you're requesting logging of headers. Do you have a preference?

If GoRouter logged whatever headers were included in the request,
wouldn't this satisfy your requirements? Doesn't GoRouter do this already?

I'm interested in solving your requirement generically for all
applications, and focussing the user experience on the correct persona.
Based on what you've described, the persona is the app developer, so
control of what is logged should be in their hands. I'm also not convinced
GoRouter should have any knowledge of headers specific to one application
or another.





Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Wed, Jul 22, 2015 at 3:05 AM, Alex Lomov <
alexander.lomov(a)altoros.com> wrote:

Some time ago routing services were discussed on a CAB [1]. Here is
a description of this proposal.

Do you think that using such service will allow your developers to
cover this requirements?

[1]
http://www.activestate.com/blog/2015/02/cloud-foundry-advisory-board-meeting-2015-february
[2]
https://docs.google.com/document/d/1bGOQxiKkmaw6uaRWGd-sXpxL0Y28d3QihcluI15FiIA/edit#heading=h.8djffzes9pnb

On Jul 21, 2015, at 4:06 PM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

Howdie!

We have some devs who want to be able to trace a request troughout
their applications.

user -> a -> b -> c
|
|-> d -> e

When a user makes a request to "a" uuid is generated inside the
app, and the request to "b" from "a" will set a header(call it WAKAWAKA to
uuid), "b"'s call will passthrough WAKAWAKA to "c" and "d", "d" will
passthrough WAKAWAKA to "e". Etc.

We aggregate all RTR logs into ELK so it would be super helpful to
them to be able to filter on WAKAWAKA and get all the access logs(and app
logs aswell, they mostly use GELF so its easy for them to add whatewher
field they want) from the services involved.

I had a quick peek at the gorouter(
https://github.com/cloudfoundry/gorouter/blob/76668f5818ea8c089ff52a14fcdfbf703c8e8767/access_log/access_log_record.go#L40)
and it seems like this should be a simple PR.

1. To gorouter.yml add
passthrough_headers:
- WAKAWAKA
- X-Random-Header

2. In makeRecord at the bottom add something like(in psuedo)

data = {}
for header in passthrough_headers:
header_val = r.FormatRequestHeader("X-Forwarded-For")
if header_val:
passthrough_headers[header] = header_val

if data:
fmt.Fprintf(b, data.to_stringified_json())

That would yield a log line like
blurgh.dev.cf.private.domain.com - [21/07/2015:10:17:05 +0000]
"GET
/statements?ascending=true&since=2015-06-30T14%3A10%3A03.078Z&skipStatementId=30a88204-0779-4385-9859-e4aabd30baf0
HTTP/1.1" 200 0 17 "-" "NING/1.0" 10.230.31.2:46204
x_forwarded_for:"-" vcap_request_id:1e58195a-cde6-4afd-7f03-43061c9ea91c
response_time:0.004927106 app_id:9784cd03-050d-4b74-9e90-5f17134a3f08
{"WAKAWAKA": "Space is the place", "X-Random-Header": "Once upon a midnight
dreary, while I pondered weak and weary"}

The reason for a stringified JSON is to make it easy to parse with
logstash or other loganalysis tools.

Before I spend time implementing, is this something you would merge?
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Thank you,

James Bayer

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
David Laing
logsearch.io - build your own open source cloud logging cluster
http://davidlaing.com

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
David Laing
logsearch.io - build your own open source cloud logging cluster
http://davidlaing.com

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Thank you,

James Bayer


Re: How to use /v2/app_usage_events for billing purpose

Noburou TANIGUCHI
 

Hi,

Is it able to change the default "one month" in CF deployment manifest?
I think this is the entry:

https://github.com/cloudfoundry/cf-release/blob/30c387facb7a9f618b1a6d5d25a5cd5ee3a469c9/jobs/cloud_controller_ng/spec#L131
<pre>
cc.app_usage_events.cutoff_age_in_days:
</pre>

Off course you shouldn't change the default in cf-release. You should change
the correspond entry in your CF deployment manifest.

Regards,


王小锋 wrote
Hi, there

I have deployed CF env in AWS, and there are some applications running on
it. I found one useful article talking about billing CF users :
https://blog.starkandwayne.com/2015/01/22/billing-your-cloud-foundry-users/.

I tried cf curl /v2/app_usage_events in my CF env, I found that I could
only retrieve app events for one month, is this exepcted? Is it able to
change the default "one month" in CF deployment manifest?

If I want to track the app usage for one org, for example org1, and org1
have one app running for more than one month, then there will be no app
usage for this long running app, then how to meter such app? Should I list
all apps available in the org currenlty, if the app is not in the app
usage
event, I could image the app has been created more than one month?

Another thing, if I want to bill app usage by month, should I call
/v2/app_usage_events/destructively_purge_all_and_reseed_started_apps at
the
end of each month?

Is there anyone have done similar things in the past and share your
experience? Many thanks.

_______________________________________________
cf-dev mailing list
cf-dev(a).cloudfoundry
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




--
View this message in context: http://cf-dev.70369.x6.nabble.com/cf-dev-How-to-use-v2-app-usage-events-for-billing-purpose-tp919p922.html
Sent from the CF Dev mailing list archive at Nabble.com.


Re: Notifications on ORG, SPACE and USER modifications

Dieu Cao <dcao@...>
 

Hi all interested in notifications on modification of resources,

It would be helpful for me in framing the "what" and the "why" of this
feature if you could also describe your specific use cases and pain points
on why you would want notifications on modifications and also which
resources you particularly care about.
Is it for real time updates on a dashboard? For consumption for billing
purposes? For triggering provisioning/deprovisioning of resources?

-Dieu

On Tue, Jul 28, 2015 at 11:05 AM, Jean-Sebastien Delfino <
jsdelfino(a)gmail.com> wrote:

I’m going to need something like this too for the CF Abacus service
metering project, as I’d like to track the lifecycle of orgs, users, etc to
match their history with the usage data reported for them.


Here’s a straw man description of what I had in mind:


- For Abacus, I’d need a Lossless API. Usage metering eventually
translates to billing and money, you don’t want to lose that :)


- An extension or variant of the current CF /v2/events API supporting
users, orgs, app usage etc, as even with a notification API I’ll still need
to do GETs sometimes.


- 304 responses with etags on these GETs (as suggested earlier in the
thread [1]) would be good.


- A Webhook style notification API where I could register interest in a
selection of events with a callback URL, and get these events POSTed back
to me at that URL, similar to what Github and many others do with Webhooks.


- On top of Webhooks, it’d be nice to have a form of streaming (either
down to the client like the Firehose does or in the other direction up to
the Webhook callback URL), but I'm not sure if we’ll need that in the
project right away.


- We’d obviously need some form of security, maybe use my user token to
register for events on entities that I have access to?


- I’m also curious about the group’s thoughts on queueing and
back-pressure when events get generated faster that they can be consumed
for example. There was a mention of some message queuing earlier [2]. That
would make sense to me (although IMO it’d be good if the underlying MQ
didn’t shine through the API). What did you have in mind for this?


I guess there are quite a few things to figure out here! I’ll be happy to
collaborate with the community on these discussions.


Thoughts?


[1]
http://cf-dev.70369.x6.nabble.com/cf-dev-Notifications-on-ORG-SPACE-and-USER-modifications-tp827p842.html

[2]
http://cf-dev.70369.x6.nabble.com/cf-dev-Notifications-on-ORG-SPACE-and-USER-modifications-tp827p834.html


- Jean-Sebastien

On Fri, Jul 24, 2015 at 9:59 PM, Matt Cowger <matt(a)cowger.us> wrote:

I think ETags is reasonable thought as well.

On Thu, Jul 23, 2015 at 4:39 PM, Benjamin Black <bblack(a)pivotal.io>
wrote:

ETags and a 304 response are specifically intended for that purpose. I'd
recommend that over relying on Last-Modified.


b

On Thu, Jul 23, 2015 at 12:34 AM, Koper, Dies <diesk(a)fast.au.fujitsu.com
wrote:
Or setting the Last-Modified HTTP response header accordingly, and
allow clients to use HTTP caching mechanisms (Last-Modified, etc.) to get
quick empty responses with the current APIs if no changes have been made?
(Or maybe this is already working so – haven’t checked).



Regards,

Dies Koper



*From:* cf-dev-bounces(a)lists.cloudfoundry.org [mailto:
cf-dev-bounces(a)lists.cloudfoundry.org] *On Behalf Of *Matt Cowger
*Sent:* Thursday, July 23, 2015 4:45 PM
*To:* Discussions about Cloud Foundry projects and the system overall.
*Subject:* Re: [cf-dev] Notifications on ORG, SPACE and USER
modifications



I've wanted something similar as well.



On a related note, having a CC API 'serial' number (for each object in
CC - apps, spaces, etc) that increments on every change relevant to that
object would be of value for detecting if something has changed.



On Thu, Jul 23, 2015 at 3:27 PM, Dieu Cao <dcao(a)pivotal.io> wrote:

There are a few different approaches to this and different concerns
that are possible.

The requests I've seen have been around wanting to be able to subscribe
to and filter the various events that cc currently generates so that other
behavior could be triggered.

We currently have events, app usage events, and service usage events.

Is it acceptable for the notifications to be lossy? Depends on the use
case but If so, then the firehose may be an acceptable approach.



The CAPI team is currently focusing on other work in the near term,
such as the v3 API and private brokers, but would be happy to collaborate
on a proposal.





On Wed, Jul 22, 2015 at 2:05 PM, Juan Pablo Genovese <
juanpgenovese(a)gmail.com> wrote:

My take:



CC should have callbacks on for each model create, update and delete
methods. Those callbacks will send a message to an MQ, which you can
subscribe to consume those messages.

This can be expanded to pretty much every event we need to track.

What do you think?



JP



2015-07-22 17:30 GMT-03:00 Matthias X Hub <matthias.hub(a)de.ibm.com>:

Hi,

we (=IBM) are also having the need and are currently investigating how
to solve this. We plan to work on a proposal to discuss this further with
the cf community. I'll keep you updated on that.

Regards,
Matthias



From: Mike Youngstrom <youngm(a)gmail.com>
To: "Discussions about Cloud Foundry projects and the system
overall." <cf-dev(a)lists.cloudfoundry.org>
Date: 22.07.2015 20:57
Subject: Re: [cf-dev] Notifications on ORG, SPACE and USER
modifications
Sent by: cf-dev-bounces(a)lists.cloudfoundry.org
------------------------------




We have the same need. Today we are polling the CC.

It would be nice for us also if we could get CC event notifications via
something like the firehose.

Mike

On Wed, Jul 22, 2015 at 10:23 AM, Juan Pablo Genovese <
juanpgenovese(a)gmail.com> wrote:
I mean, I know you can list those events thru the API, but I want
something that will react on an event instead of having to be constantly
polling for them.

2015-07-22 13:18 GMT-03:00 Juan Pablo Genovese <juanpgenovese(a)gmail.com>:

Sree,

thanks! Any pointers on how can I hook up to these audit events?

Thank you!

2015-07-22 13:12 GMT-03:00 Sree Tummidi <stummidi(a)pivotal.io>:
I believe there are audit events generated for all these actions which
can be captured and forwarded to an SIEM solution like splunk

Thanks,
Sree

Sent from my iPhone

On Jul 22, 2015, at 8:54 AM, Juan Pablo Genovese <
juanpgenovese(a)gmail.com> wrote:

Guys,

I need to somehow hook up into the Cloud Controller (CC) to capture
ORG, SPACE and USER deletion, insertion and update.

So far, I considered some approaches, such as forking the CC (the least
favorite) and modifying the code with some hooks, tapping into Nginx to
capture the requests, and using triggers in the database to capture each
event and send the necessary info to a service.

What do you think?
Any other idea you might have?

Thanks!

--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------

http://www.jpgenovese.com
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com



--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev





--

Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------

http://www.jpgenovese.com


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev





--

-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Jean-Sebastien

Sent from my DynaTAC 8000x

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


CF-Abacus: incubation and inception meeting coming soon

Michael Maximilien
 

Hi, all,

Now that CF-Abacus is officially an incubator under the guidance of the
CFF, here are some quick updates:

1. The project official github moved to:

https://github.com/cloudfoundry-incubator/cf-abacus

2. We are planning an inception next week Wednesday from 2p to 5p in SF.

We invite everyone interested to take a look at the repo, provide feedback,
or better, join us at the inception meeting. The location will be either
CFF, Pivotal, or IBM. All within a few blocks in downtown SF.

We will also have Google hangout and conference call for remote
participants.

If interested, then respond to me directly so I add you to the invite list.

Thanks and talk next week. Best,

CF-Abacus team

dr.max
ibm cloud labs
silicon valley, ca

Sent from my iPhone


How to use /v2/app_usage_events for billing purpose

王小锋 <zzuwxf at gmail.com...>
 

Hi, there

I have deployed CF env in AWS, and there are some applications running on
it. I found one useful article talking about billing CF users :
https://blog.starkandwayne.com/2015/01/22/billing-your-cloud-foundry-users/.

I tried cf curl /v2/app_usage_events in my CF env, I found that I could
only retrieve app events for one month, is this exepcted? Is it able to
change the default "one month" in CF deployment manifest?

If I want to track the app usage for one org, for example org1, and org1
have one app running for more than one month, then there will be no app
usage for this long running app, then how to meter such app? Should I list
all apps available in the org currenlty, if the app is not in the app usage
event, I could image the app has been created more than one month?

Another thing, if I want to bill app usage by month, should I call
/v2/app_usage_events/destructively_purge_all_and_reseed_started_apps at the
end of each month?

Is there anyone have done similar things in the past and share your
experience? Many thanks.


Re: JMX MBean Endpoint on Pivotal Web Services

Amit Kumar Gupta
 

Hi Nikhil,

This mailing list is for the open source Cloud Foundry work. You can
search the PWS support forums: http://support.run.pivotal.io/home, or shoot
them an email: support(a)run.pivotal.io.

Best,
Amit, Project Manger for CF OSS Release Integration team
Pivotal

On Tue, Jul 28, 2015 at 5:44 PM, Nikhil Katre <nikhil.katre(a)appdynamics.com>
wrote:

Hello,

I am trying to get some JMX Mbean stats related to an application which is
deployed on Pivotal Web Services.

I do know that in the case of private PCF installation I can get the JMX
service end point easily but how can I get the JMX endpoint in the case of
app deployed on PWS.

Looking for something like this:
jmxServiceUrl :
service:jmx:rmi:///jndi/rmi://<hostname>:<post_number>/jmxrmi

I don't even know whether this is possible in case of PWS. Any help or
pointers specific to PWS use case would be appreciated.

Thanks !

--
Thanks,

*Nikhil Katre* | Software Engineer
Mobile: (919) 633 3940 <%28303%29%20946%209911>

AppDynamics
The Application Intelligence Company
Watch <http://appdynamics.wistia.com/medias/56gnkuk6mv>our Video | Try
<https://portal.appdynamics.com/account/signup/signupForm>our FREE Trial
| Twitter <http://www.twitter.com/appdynamics>| Facebook
<http://www.facebook.com/pages/AppDynamics/193264136815?ref=nf>|
appdynamics.com <http://www.appdynamics.com/>


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


JMX MBean Endpoint on Pivotal Web Services

Nikhil Katre <nikhil.katre@...>
 

Hello,

I am trying to get some JMX Mbean stats related to an application which is
deployed on Pivotal Web Services.

I do know that in the case of private PCF installation I can get the JMX
service end point easily but how can I get the JMX endpoint in the case of
app deployed on PWS.

Looking for something like this:
jmxServiceUrl :
service:jmx:rmi:///jndi/rmi://<hostname>:<post_number>/jmxrmi

I don't even know whether this is possible in case of PWS. Any help or
pointers specific to PWS use case would be appreciated.

Thanks !

--
Thanks,

*Nikhil Katre* | Software Engineer
Mobile: (919) 633 3940 <%28303%29%20946%209911>

AppDynamics
The Application Intelligence Company
Watch <http://appdynamics.wistia.com/medias/56gnkuk6mv>our Video | Try
<https://portal.appdynamics.com/account/signup/signupForm>our FREE Trial |
Twitter <http://www.twitter.com/appdynamics>| Facebook
<http://www.facebook.com/pages/AppDynamics/193264136815?ref=nf>|
appdynamics.com <http://www.appdynamics.com/>


Default cgroup CPU share

John Wong
 

I am reading
https://docs.cloudfoundry.org/concepts/architecture/warden.html#cpu and it
said:

If B is idle, A may receive up to all the CPU. Shares per cgroup range
from 2 to 1024, with 1024 the default. Both Diego apps and DEA apps scale
the number of allocated shares linearly with the amount of memory, with an
app instance requesting 8G of memory getting the upper limit of 1024
shares. Diego also guarantees a minimum of 10 shares per app instance.

So 1024 is the default share every app get by default?

Say I start with an empty DEA.


APP #1: 1G shared = 1024?

APP #2 added. 1G shared =? what happen to APP #1?

APP #2 added: 512MB shared =? What happened to APP #1 & APP 2?

APP #3 added: 8GB, now what happened?


I am all assuming their usage is nearly idle. What is the total number of
share for a N-core DEA? Also are the shares dynamic? In the mean time I
will try to understand how CPU usage is shared in cgroup from other
resource.


Thanks.


John


Re: Notifications on ORG, SPACE and USER modifications

Jean-Sebastien Delfino
 

I’m going to need something like this too for the CF Abacus service
metering project, as I’d like to track the lifecycle of orgs, users, etc to
match their history with the usage data reported for them.


Here’s a straw man description of what I had in mind:


- For Abacus, I’d need a Lossless API. Usage metering eventually translates
to billing and money, you don’t want to lose that :)


- An extension or variant of the current CF /v2/events API supporting
users, orgs, app usage etc, as even with a notification API I’ll still need
to do GETs sometimes.


- 304 responses with etags on these GETs (as suggested earlier in the
thread [1]) would be good.


- A Webhook style notification API where I could register interest in a
selection of events with a callback URL, and get these events POSTed back
to me at that URL, similar to what Github and many others do with Webhooks.


- On top of Webhooks, it’d be nice to have a form of streaming (either down
to the client like the Firehose does or in the other direction up to the
Webhook callback URL), but I'm not sure if we’ll need that in the project
right away.


- We’d obviously need some form of security, maybe use my user token to
register for events on entities that I have access to?


- I’m also curious about the group’s thoughts on queueing and back-pressure
when events get generated faster that they can be consumed for example.
There was a mention of some message queuing earlier [2]. That would make
sense to me (although IMO it’d be good if the underlying MQ didn’t shine
through the API). What did you have in mind for this?


I guess there are quite a few things to figure out here! I’ll be happy to
collaborate with the community on these discussions.


Thoughts?


[1]
http://cf-dev.70369.x6.nabble.com/cf-dev-Notifications-on-ORG-SPACE-and-USER-modifications-tp827p842.html

[2]
http://cf-dev.70369.x6.nabble.com/cf-dev-Notifications-on-ORG-SPACE-and-USER-modifications-tp827p834.html


- Jean-Sebastien

On Fri, Jul 24, 2015 at 9:59 PM, Matt Cowger <matt(a)cowger.us> wrote:

I think ETags is reasonable thought as well.

On Thu, Jul 23, 2015 at 4:39 PM, Benjamin Black <bblack(a)pivotal.io> wrote:

ETags and a 304 response are specifically intended for that purpose. I'd
recommend that over relying on Last-Modified.


b

On Thu, Jul 23, 2015 at 12:34 AM, Koper, Dies <diesk(a)fast.au.fujitsu.com>
wrote:

Or setting the Last-Modified HTTP response header accordingly, and
allow clients to use HTTP caching mechanisms (Last-Modified, etc.) to get
quick empty responses with the current APIs if no changes have been made?
(Or maybe this is already working so – haven’t checked).



Regards,

Dies Koper



*From:* cf-dev-bounces(a)lists.cloudfoundry.org [mailto:
cf-dev-bounces(a)lists.cloudfoundry.org] *On Behalf Of *Matt Cowger
*Sent:* Thursday, July 23, 2015 4:45 PM
*To:* Discussions about Cloud Foundry projects and the system overall.
*Subject:* Re: [cf-dev] Notifications on ORG, SPACE and USER
modifications



I've wanted something similar as well.



On a related note, having a CC API 'serial' number (for each object in
CC - apps, spaces, etc) that increments on every change relevant to that
object would be of value for detecting if something has changed.



On Thu, Jul 23, 2015 at 3:27 PM, Dieu Cao <dcao(a)pivotal.io> wrote:

There are a few different approaches to this and different concerns that
are possible.

The requests I've seen have been around wanting to be able to subscribe
to and filter the various events that cc currently generates so that other
behavior could be triggered.

We currently have events, app usage events, and service usage events.

Is it acceptable for the notifications to be lossy? Depends on the use
case but If so, then the firehose may be an acceptable approach.



The CAPI team is currently focusing on other work in the near term, such
as the v3 API and private brokers, but would be happy to collaborate on a
proposal.





On Wed, Jul 22, 2015 at 2:05 PM, Juan Pablo Genovese <
juanpgenovese(a)gmail.com> wrote:

My take:



CC should have callbacks on for each model create, update and delete
methods. Those callbacks will send a message to an MQ, which you can
subscribe to consume those messages.

This can be expanded to pretty much every event we need to track.

What do you think?



JP



2015-07-22 17:30 GMT-03:00 Matthias X Hub <matthias.hub(a)de.ibm.com>:

Hi,

we (=IBM) are also having the need and are currently investigating how
to solve this. We plan to work on a proposal to discuss this further with
the cf community. I'll keep you updated on that.

Regards,
Matthias



From: Mike Youngstrom <youngm(a)gmail.com>
To: "Discussions about Cloud Foundry projects and the system
overall." <cf-dev(a)lists.cloudfoundry.org>
Date: 22.07.2015 20:57
Subject: Re: [cf-dev] Notifications on ORG, SPACE and USER
modifications
Sent by: cf-dev-bounces(a)lists.cloudfoundry.org
------------------------------




We have the same need. Today we are polling the CC.

It would be nice for us also if we could get CC event notifications via
something like the firehose.

Mike

On Wed, Jul 22, 2015 at 10:23 AM, Juan Pablo Genovese <
juanpgenovese(a)gmail.com> wrote:
I mean, I know you can list those events thru the API, but I want
something that will react on an event instead of having to be constantly
polling for them.

2015-07-22 13:18 GMT-03:00 Juan Pablo Genovese <juanpgenovese(a)gmail.com>:

Sree,

thanks! Any pointers on how can I hook up to these audit events?

Thank you!

2015-07-22 13:12 GMT-03:00 Sree Tummidi <stummidi(a)pivotal.io>:
I believe there are audit events generated for all these actions which
can be captured and forwarded to an SIEM solution like splunk

Thanks,
Sree

Sent from my iPhone

On Jul 22, 2015, at 8:54 AM, Juan Pablo Genovese <
juanpgenovese(a)gmail.com> wrote:

Guys,

I need to somehow hook up into the Cloud Controller (CC) to capture ORG,
SPACE and USER deletion, insertion and update.

So far, I considered some approaches, such as forking the CC (the least
favorite) and modifying the code with some hooks, tapping into Nginx to
capture the requests, and using triggers in the database to capture each
event and send the necessary info to a service.

What do you think?
Any other idea you might have?

Thanks!

--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------

http://www.jpgenovese.com
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com



--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev





--

Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------

http://www.jpgenovese.com


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev





--

-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

--
Jean-Sebastien

Sent from my DynaTAC 8000x


Re: Running Jira into Cloudfoundry

Marco Voelz
 

Dear Gwenn,

I guess Jira is a typical example of an application not really fit for a platform like CF, given the way of configuring the DB and that it stores attachments on the local filesystem. However, at the same time it would make an interesting example case, as many people ask about it. Please let us know how it works out for you! :)

Warm regards
Marco

Date: Fri, 24 Jul 2015 01:24:28 +0900
From: Gwenn Etourneau <getourneau(a)pivotal.io>
To: "Discussions about Cloud Foundry projects and the system overall."
<cf-dev(a)lists.cloudfoundry.org>
Subject: Re: [cf-dev] Running Jira into Cloudfoundry
Message-ID:
<CAPJeHf=gUX7FjpGZ92_U+kOjVL=7PUo+ztg0RX_KCJSFxo79kg(a)mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Not sure yet .. I am still thinking about that ..
But anyway if ERS is too hard I will use the bosh way :)

On Thu, Jul 23, 2015 at 6:24 PM, john mcteague <john.mcteague(a)gmail.com>
wrote:

Out of curiosity how do you plan on solving the storage problem for
attachments, assuming you get the DB working.

John.


Re: Allow gorouter to log random headers.

Simon Johansson <simon@...>
 

Here is a PR David and I worked on this morning to implement the above

https://github.com/cloudfoundry/gorouter/pull/92

Comments are appreciated.

On Tue, Jul 28, 2015 at 6:46 AM, David Laing <david(a)davidlaing.com> wrote:

Shannon,

Understood. PR for "as an operator of Cloud Foundry, I want to enable
logging of specified headers for all applications." coming up.

Experience will tell whether there are any sensible defaults; but we can
leave that for a future enhancement of this functionality.

Regards

David

On 28 July 2015 at 03:54, Shannon Coen <scoen(a)pivotal.io> wrote:

Thank you, James. That helps.

What bothered me about this thread was that the use case was presented
as, "as an unspecified persona, I want to customize the multi-tenant router
to log specified headers for a few apps". This is troubling.

Instead it sounds like the use case is "as an operator of Cloud Foundry,
I want to enable logging of specified headers for all applications." This
is more compelling.

I'm skeptical that this manifest property should have a default. Who's to
say what the minimum list of "sensible" defaults is? Does WAKAWAKA make the
cut? j/k.

We'll take a PR.

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Fri, Jul 24, 2015 at 8:46 AM, David Laing <david(a)davidlaing.com>
wrote:

James,

Yep - we're pretty much on the same page.

Only addition I would ask for is that the whitelist contain some
"sensible" defaults (eg, Trace-Id, Span-Id) that are switched on by
default; since then tight integration with tools like spring-cloud /
buildpacks would work out the box.

D

On 24 July 2015 at 16:26, James Bayer <jbayer(a)pivotal.io> wrote:

shannon,

from what i'm reading here about the use case, the main interest is
that a CF operator knows that their cf installation is more deeply
integrated with a specific log parsing solution for all/many apps on that
platform that choose to use it (whether than is ELK, Zipkin, etc). it does
not sound like it is a special case with lots of variation by many
different app teams on the cf installation. rather, it sounds like this is
most likely to be used as an installation-wide option to enhance the app
developer / app operations experience.

it seems like an operator configured whitelist set of headers that get
logged with the RTR message satisfies the current needs well and is
reasonable.

if we were to find that in the future lots of variation and different
app teams on the same CF wanted to have the RTR tier log many different
headers in the access log, then we could enhance the "log the whitelist of
headers to the access log" capability to be exposed to a limited number of
headers that each route could be configured by developers to specify that
would apply in addition to the operator configured logged headers. but it
sounds like that isn't needed right now.

simon/david, did i summarize this correctly?

On Fri, Jul 24, 2015 at 7:23 AM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

If I understand correctly, you're proposing that an operator of
CloudFoundry configure GoRouter, which is a multi-tenant, shared service,
with knowledge specific to one or a few applications

That is indeed the proposal. In my org we work closely with the
different development streams to provide a good out of the box experience
by creating certain opt-in features that makes their life easier. This
would be one of those features, if you pass certain headers in your
requests thats standardised across the entire online organisation you will
be able to query for them in Kibana.
I understand that this is not the case in all environments.

If GoRouter logged whatever headers were included in the request,
wouldn't this satisfy your requirements?

This would indeed satisfy my requests, but as David points out("However,
not having a whitelist of headers to log opens a possible DDOS attack
vector on the GoRouter") it might now be appropriate.

Based on what you've described, the persona is the app developer.
Not necessarily, we operators are also very interested in certain
headers.

so control of what is logged should be in their hands.
If one via the API could set headers that should be logged for an org,
space or application that would be magical. The need for a list of
"must-log-all-these-headers" would still be there I think as not to have to
maintain a list of these standardised headers across different objects.


On Thu, Jul 23, 2015 at 10:51 PM, Shannon Coen <scoen(a)pivotal.io>
wrote:

This is not something that would be merged, as originally proposed,
without additional investigation and discussion.

If I understand correctly, you're proposing that an operator of
CloudFoundry configure GoRouter, which is a multi-tenant, shared service,
with knowledge specific to one or a few applications. This should not be an
operator responsibility, nor should the solution be specific to one or a
few applications.

The goal is "the flexibility of being able to annotate our logs with
what we consider to be important for our debugging purposes." More
specifically you're requesting logging of headers. Do you have a preference?

If GoRouter logged whatever headers were included in the request,
wouldn't this satisfy your requirements? Doesn't GoRouter do this already?

I'm interested in solving your requirement generically for all
applications, and focussing the user experience on the correct persona.
Based on what you've described, the persona is the app developer, so
control of what is logged should be in their hands. I'm also not convinced
GoRouter should have any knowledge of headers specific to one application
or another.





Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Wed, Jul 22, 2015 at 3:05 AM, Alex Lomov <
alexander.lomov(a)altoros.com> wrote:

Some time ago routing services were discussed on a CAB [1]. Here is
a description of this proposal.

Do you think that using such service will allow your developers to
cover this requirements?

[1]
http://www.activestate.com/blog/2015/02/cloud-foundry-advisory-board-meeting-2015-february
[2]
https://docs.google.com/document/d/1bGOQxiKkmaw6uaRWGd-sXpxL0Y28d3QihcluI15FiIA/edit#heading=h.8djffzes9pnb

On Jul 21, 2015, at 4:06 PM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

Howdie!

We have some devs who want to be able to trace a request troughout
their applications.

user -> a -> b -> c
|
|-> d -> e

When a user makes a request to "a" uuid is generated inside the app,
and the request to "b" from "a" will set a header(call it WAKAWAKA to
uuid), "b"'s call will passthrough WAKAWAKA to "c" and "d", "d" will
passthrough WAKAWAKA to "e". Etc.

We aggregate all RTR logs into ELK so it would be super helpful to
them to be able to filter on WAKAWAKA and get all the access logs(and app
logs aswell, they mostly use GELF so its easy for them to add whatewher
field they want) from the services involved.

I had a quick peek at the gorouter(
https://github.com/cloudfoundry/gorouter/blob/76668f5818ea8c089ff52a14fcdfbf703c8e8767/access_log/access_log_record.go#L40)
and it seems like this should be a simple PR.

1. To gorouter.yml add
passthrough_headers:
- WAKAWAKA
- X-Random-Header

2. In makeRecord at the bottom add something like(in psuedo)

data = {}
for header in passthrough_headers:
header_val = r.FormatRequestHeader("X-Forwarded-For")
if header_val:
passthrough_headers[header] = header_val

if data:
fmt.Fprintf(b, data.to_stringified_json())

That would yield a log line like
blurgh.dev.cf.private.domain.com - [21/07/2015:10:17:05 +0000] "GET
/statements?ascending=true&since=2015-06-30T14%3A10%3A03.078Z&skipStatementId=30a88204-0779-4385-9859-e4aabd30baf0
HTTP/1.1" 200 0 17 "-" "NING/1.0" 10.230.31.2:46204
x_forwarded_for:"-" vcap_request_id:1e58195a-cde6-4afd-7f03-43061c9ea91c
response_time:0.004927106 app_id:9784cd03-050d-4b74-9e90-5f17134a3f08
{"WAKAWAKA": "Space is the place", "X-Random-Header": "Once upon a midnight
dreary, while I pondered weak and weary"}

The reason for a stringified JSON is to make it easy to parse with
logstash or other loganalysis tools.

Before I spend time implementing, is this something you would merge?
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Thank you,

James Bayer

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
David Laing
logsearch.io - build your own open source cloud logging cluster
http://davidlaing.com

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
David Laing
logsearch.io - build your own open source cloud logging cluster
http://davidlaing.com

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Allow gorouter to log random headers.

David Laing
 

Shannon,

Understood. PR for "as an operator of Cloud Foundry, I want to enable
logging of specified headers for all applications." coming up.

Experience will tell whether there are any sensible defaults; but we can
leave that for a future enhancement of this functionality.

Regards

David

On 28 July 2015 at 03:54, Shannon Coen <scoen(a)pivotal.io> wrote:

Thank you, James. That helps.

What bothered me about this thread was that the use case was presented as,
"as an unspecified persona, I want to customize the multi-tenant router to
log specified headers for a few apps". This is troubling.

Instead it sounds like the use case is "as an operator of Cloud Foundry, I
want to enable logging of specified headers for all applications." This is
more compelling.

I'm skeptical that this manifest property should have a default. Who's to
say what the minimum list of "sensible" defaults is? Does WAKAWAKA make the
cut? j/k.

We'll take a PR.

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Fri, Jul 24, 2015 at 8:46 AM, David Laing <david(a)davidlaing.com> wrote:

James,

Yep - we're pretty much on the same page.

Only addition I would ask for is that the whitelist contain some
"sensible" defaults (eg, Trace-Id, Span-Id) that are switched on by
default; since then tight integration with tools like spring-cloud /
buildpacks would work out the box.

D

On 24 July 2015 at 16:26, James Bayer <jbayer(a)pivotal.io> wrote:

shannon,

from what i'm reading here about the use case, the main interest is that
a CF operator knows that their cf installation is more deeply integrated
with a specific log parsing solution for all/many apps on that platform
that choose to use it (whether than is ELK, Zipkin, etc). it does not sound
like it is a special case with lots of variation by many different app
teams on the cf installation. rather, it sounds like this is most likely to
be used as an installation-wide option to enhance the app developer / app
operations experience.

it seems like an operator configured whitelist set of headers that get
logged with the RTR message satisfies the current needs well and is
reasonable.

if we were to find that in the future lots of variation and different
app teams on the same CF wanted to have the RTR tier log many different
headers in the access log, then we could enhance the "log the whitelist of
headers to the access log" capability to be exposed to a limited number of
headers that each route could be configured by developers to specify that
would apply in addition to the operator configured logged headers. but it
sounds like that isn't needed right now.

simon/david, did i summarize this correctly?

On Fri, Jul 24, 2015 at 7:23 AM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

If I understand correctly, you're proposing that an operator of
CloudFoundry configure GoRouter, which is a multi-tenant, shared service,
with knowledge specific to one or a few applications

That is indeed the proposal. In my org we work closely with the
different development streams to provide a good out of the box experience
by creating certain opt-in features that makes their life easier. This
would be one of those features, if you pass certain headers in your
requests thats standardised across the entire online organisation you will
be able to query for them in Kibana.
I understand that this is not the case in all environments.

If GoRouter logged whatever headers were included in the request,
wouldn't this satisfy your requirements?

This would indeed satisfy my requests, but as David points out("However,
not having a whitelist of headers to log opens a possible DDOS attack
vector on the GoRouter") it might now be appropriate.

Based on what you've described, the persona is the app developer.
Not necessarily, we operators are also very interested in certain
headers.

so control of what is logged should be in their hands.
If one via the API could set headers that should be logged for an org,
space or application that would be magical. The need for a list of
"must-log-all-these-headers" would still be there I think as not to have to
maintain a list of these standardised headers across different objects.


On Thu, Jul 23, 2015 at 10:51 PM, Shannon Coen <scoen(a)pivotal.io>
wrote:

This is not something that would be merged, as originally proposed,
without additional investigation and discussion.

If I understand correctly, you're proposing that an operator of
CloudFoundry configure GoRouter, which is a multi-tenant, shared service,
with knowledge specific to one or a few applications. This should not be an
operator responsibility, nor should the solution be specific to one or a
few applications.

The goal is "the flexibility of being able to annotate our logs with
what we consider to be important for our debugging purposes." More
specifically you're requesting logging of headers. Do you have a preference?

If GoRouter logged whatever headers were included in the request,
wouldn't this satisfy your requirements? Doesn't GoRouter do this already?

I'm interested in solving your requirement generically for all
applications, and focussing the user experience on the correct persona.
Based on what you've described, the persona is the app developer, so
control of what is logged should be in their hands. I'm also not convinced
GoRouter should have any knowledge of headers specific to one application
or another.





Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Wed, Jul 22, 2015 at 3:05 AM, Alex Lomov <
alexander.lomov(a)altoros.com> wrote:

Some time ago routing services were discussed on a CAB [1]. Here is a
description of this proposal.

Do you think that using such service will allow your developers to
cover this requirements?

[1]
http://www.activestate.com/blog/2015/02/cloud-foundry-advisory-board-meeting-2015-february
[2]
https://docs.google.com/document/d/1bGOQxiKkmaw6uaRWGd-sXpxL0Y28d3QihcluI15FiIA/edit#heading=h.8djffzes9pnb

On Jul 21, 2015, at 4:06 PM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

Howdie!

We have some devs who want to be able to trace a request troughout
their applications.

user -> a -> b -> c
|
|-> d -> e

When a user makes a request to "a" uuid is generated inside the app,
and the request to "b" from "a" will set a header(call it WAKAWAKA to
uuid), "b"'s call will passthrough WAKAWAKA to "c" and "d", "d" will
passthrough WAKAWAKA to "e". Etc.

We aggregate all RTR logs into ELK so it would be super helpful to
them to be able to filter on WAKAWAKA and get all the access logs(and app
logs aswell, they mostly use GELF so its easy for them to add whatewher
field they want) from the services involved.

I had a quick peek at the gorouter(
https://github.com/cloudfoundry/gorouter/blob/76668f5818ea8c089ff52a14fcdfbf703c8e8767/access_log/access_log_record.go#L40)
and it seems like this should be a simple PR.

1. To gorouter.yml add
passthrough_headers:
- WAKAWAKA
- X-Random-Header

2. In makeRecord at the bottom add something like(in psuedo)

data = {}
for header in passthrough_headers:
header_val = r.FormatRequestHeader("X-Forwarded-For")
if header_val:
passthrough_headers[header] = header_val

if data:
fmt.Fprintf(b, data.to_stringified_json())

That would yield a log line like
blurgh.dev.cf.private.domain.com - [21/07/2015:10:17:05 +0000] "GET
/statements?ascending=true&since=2015-06-30T14%3A10%3A03.078Z&skipStatementId=30a88204-0779-4385-9859-e4aabd30baf0
HTTP/1.1" 200 0 17 "-" "NING/1.0" 10.230.31.2:46204
x_forwarded_for:"-" vcap_request_id:1e58195a-cde6-4afd-7f03-43061c9ea91c
response_time:0.004927106 app_id:9784cd03-050d-4b74-9e90-5f17134a3f08
{"WAKAWAKA": "Space is the place", "X-Random-Header": "Once upon a midnight
dreary, while I pondered weak and weary"}

The reason for a stringified JSON is to make it easy to parse with
logstash or other loganalysis tools.

Before I spend time implementing, is this something you would merge?
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Thank you,

James Bayer

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
David Laing
logsearch.io - build your own open source cloud logging cluster
http://davidlaing.com

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

--
David Laing
logsearch.io - build your own open source cloud logging cluster
http://davidlaing.com


Re: cflinuxfs2 stack not fount under v210.2 release

Yitao Jiang
 

What weird is that it happens occasionally, and i had pushed java, nodejs,
pyton application with community buildpacks.
Yes, it's at staging level. I'll paste logs here when failed again.
Thank you


On Tue, Jul 28, 2015 at 11:12 AM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

Btw when the error "*cflinuxfs2 stack not found" come ? At the staging
level ? do you have some log?*
*Which buildpack are you using and so on ..*

On Tue, Jul 28, 2015 at 12:10 PM, Yitao Jiang <jiangyt.cn(a)gmail.com>
wrote:

This is the output:


Getting stacks in org org / space ace as ***@*****...
OK
name description
cflinuxfs2 Ubuntu 14.04.2 trusty


On Tue, Jul 28, 2015 at 11:05 AM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

What cf stacks command give you ?

On Tue, Jul 28, 2015 at 11:55 AM, Yitao Jiang <jiangyt.cn(a)gmail.com>
wrote:

​We built our own release using community source code, and he just told
me that using cf-210002.
I know that the stack are install during the deployment and i just
verify that all deas have installed cflinusfs2 stack.

On Tue, Jul 28, 2015 at 10:07 AM, Gwenn Etourneau <
getourneau(a)pivotal.io> wrote:

I mean which version are the DEA ? the stack do not live in the
stemcell ... The stack are install during the deployment

On Tue, Jul 28, 2015 at 11:04 AM, Yitao Jiang <jiangyt.cn(a)gmail.com>
wrote:

Yes,all deas are built from same stemcell, all with fresh
installation


+-------------------------------------------+---------+--------------------------------------+
| Name | Version | CID
|

+-------------------------------------------+---------+--------------------------------------+
| bosh-openstack-kvm-ubuntu-trusty-go_agent | 2978* |
bc3592a8-64fd-44b1-a9c0-797953f86e13 |

+-------------------------------------------+---------+--------------------------------------+

(*) Currently in-use


On Tue, Jul 28, 2015 at 9:42 AM, Gwenn Etourneau <
getourneau(a)pivotal.io> wrote:

Are you sure all you DEA are at the same version ?
I think so but I don't know maybe rare case.



On Tue, Jul 28, 2015 at 8:35 AM, Yitao Jiang <jiangyt.cn(a)gmail.com>
wrote:

Hi,

Does any body ever have the problem of deploying application failed
in *cflinuxfs2 stack not found?*
we deployed v210.2 release and occasionally it failed in*
cflinuxfs2 not found* when deploying. It's weird that the nats
with 2 nodes all works fine, i can manually received the msgs distributed
among the cluster.

Regards,

Yitao
jiangyt.github.io

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--

Regards,

Yitao
jiangyt.github.io

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--

Regards,

Yitao
jiangyt.github.io

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--

Regards,

Yitao
jiangyt.github.io

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--

Regards,

Yitao
jiangyt.github.io


Re: cflinuxfs2 stack not fount under v210.2 release

Gwenn Etourneau
 

Btw when the error "*cflinuxfs2 stack not found" come ? At the staging
level ? do you have some log?*
*Which buildpack are you using and so on ..*

On Tue, Jul 28, 2015 at 12:10 PM, Yitao Jiang <jiangyt.cn(a)gmail.com> wrote:

This is the output:


Getting stacks in org org / space ace as ***@*****...
OK
name description
cflinuxfs2 Ubuntu 14.04.2 trusty


On Tue, Jul 28, 2015 at 11:05 AM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

What cf stacks command give you ?

On Tue, Jul 28, 2015 at 11:55 AM, Yitao Jiang <jiangyt.cn(a)gmail.com>
wrote:

​We built our own release using community source code, and he just told
me that using cf-210002.
I know that the stack are install during the deployment and i just
verify that all deas have installed cflinusfs2 stack.

On Tue, Jul 28, 2015 at 10:07 AM, Gwenn Etourneau <getourneau(a)pivotal.io
wrote:
I mean which version are the DEA ? the stack do not live in the
stemcell ... The stack are install during the deployment

On Tue, Jul 28, 2015 at 11:04 AM, Yitao Jiang <jiangyt.cn(a)gmail.com>
wrote:

Yes,all deas are built from same stemcell, all with fresh installation


+-------------------------------------------+---------+--------------------------------------+
| Name | Version | CID
|

+-------------------------------------------+---------+--------------------------------------+
| bosh-openstack-kvm-ubuntu-trusty-go_agent | 2978* |
bc3592a8-64fd-44b1-a9c0-797953f86e13 |

+-------------------------------------------+---------+--------------------------------------+

(*) Currently in-use


On Tue, Jul 28, 2015 at 9:42 AM, Gwenn Etourneau <
getourneau(a)pivotal.io> wrote:

Are you sure all you DEA are at the same version ?
I think so but I don't know maybe rare case.



On Tue, Jul 28, 2015 at 8:35 AM, Yitao Jiang <jiangyt.cn(a)gmail.com>
wrote:

Hi,

Does any body ever have the problem of deploying application failed
in *cflinuxfs2 stack not found?*
we deployed v210.2 release and occasionally it failed in*
cflinuxfs2 not found* when deploying. It's weird that the nats with
2 nodes all works fine, i can manually received the msgs distributed among
the cluster.

Regards,

Yitao
jiangyt.github.io

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--

Regards,

Yitao
jiangyt.github.io

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--

Regards,

Yitao
jiangyt.github.io

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--

Regards,

Yitao
jiangyt.github.io

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: cflinuxfs2 stack not fount under v210.2 release

Yitao Jiang
 

This is the output:


Getting stacks in org org / space ace as ***@*****...
OK
name description
cflinuxfs2 Ubuntu 14.04.2 trusty


On Tue, Jul 28, 2015 at 11:05 AM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

What cf stacks command give you ?

On Tue, Jul 28, 2015 at 11:55 AM, Yitao Jiang <jiangyt.cn(a)gmail.com>
wrote:

​We built our own release using community source code, and he just told
me that using cf-210002.
I know that the stack are install during the deployment and i just verify
that all deas have installed cflinusfs2 stack.

On Tue, Jul 28, 2015 at 10:07 AM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

I mean which version are the DEA ? the stack do not live in the stemcell
... The stack are install during the deployment

On Tue, Jul 28, 2015 at 11:04 AM, Yitao Jiang <jiangyt.cn(a)gmail.com>
wrote:

Yes,all deas are built from same stemcell, all with fresh installation


+-------------------------------------------+---------+--------------------------------------+
| Name | Version | CID
|

+-------------------------------------------+---------+--------------------------------------+
| bosh-openstack-kvm-ubuntu-trusty-go_agent | 2978* |
bc3592a8-64fd-44b1-a9c0-797953f86e13 |

+-------------------------------------------+---------+--------------------------------------+

(*) Currently in-use


On Tue, Jul 28, 2015 at 9:42 AM, Gwenn Etourneau <getourneau(a)pivotal.io
wrote:
Are you sure all you DEA are at the same version ?
I think so but I don't know maybe rare case.



On Tue, Jul 28, 2015 at 8:35 AM, Yitao Jiang <jiangyt.cn(a)gmail.com>
wrote:

Hi,

Does any body ever have the problem of deploying application failed
in *cflinuxfs2 stack not found?*
we deployed v210.2 release and occasionally it failed in* cflinuxfs2
not found* when deploying. It's weird that the nats with 2 nodes all
works fine, i can manually received the msgs distributed among the cluster.

Regards,

Yitao
jiangyt.github.io

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--

Regards,

Yitao
jiangyt.github.io

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--

Regards,

Yitao
jiangyt.github.io

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--

Regards,

Yitao
jiangyt.github.io


Re: cflinuxfs2 stack not fount under v210.2 release

Gwenn Etourneau
 

What cf stacks command give you ?

On Tue, Jul 28, 2015 at 11:55 AM, Yitao Jiang <jiangyt.cn(a)gmail.com> wrote:

​We built our own release using community source code, and he just told me
that using cf-210002.
I know that the stack are install during the deployment and i just verify
that all deas have installed cflinusfs2 stack.

On Tue, Jul 28, 2015 at 10:07 AM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

I mean which version are the DEA ? the stack do not live in the stemcell
... The stack are install during the deployment

On Tue, Jul 28, 2015 at 11:04 AM, Yitao Jiang <jiangyt.cn(a)gmail.com>
wrote:

Yes,all deas are built from same stemcell, all with fresh installation


+-------------------------------------------+---------+--------------------------------------+
| Name | Version | CID
|

+-------------------------------------------+---------+--------------------------------------+
| bosh-openstack-kvm-ubuntu-trusty-go_agent | 2978* |
bc3592a8-64fd-44b1-a9c0-797953f86e13 |

+-------------------------------------------+---------+--------------------------------------+

(*) Currently in-use


On Tue, Jul 28, 2015 at 9:42 AM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

Are you sure all you DEA are at the same version ?
I think so but I don't know maybe rare case.



On Tue, Jul 28, 2015 at 8:35 AM, Yitao Jiang <jiangyt.cn(a)gmail.com>
wrote:

Hi,

Does any body ever have the problem of deploying application failed in *cflinuxfs2
stack not found?*
we deployed v210.2 release and occasionally it failed in* cflinuxfs2
not found* when deploying. It's weird that the nats with 2 nodes all
works fine, i can manually received the msgs distributed among the cluster.

Regards,

Yitao
jiangyt.github.io

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--

Regards,

Yitao
jiangyt.github.io

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--

Regards,

Yitao
jiangyt.github.io

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: cflinuxfs2 stack not fount under v210.2 release

Yitao Jiang
 

​We built our own release using community source code, and he just told me
that using cf-210002.
I know that the stack are install during the deployment and i just verify
that all deas have installed cflinusfs2 stack.

On Tue, Jul 28, 2015 at 10:07 AM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

I mean which version are the DEA ? the stack do not live in the stemcell
... The stack are install during the deployment

On Tue, Jul 28, 2015 at 11:04 AM, Yitao Jiang <jiangyt.cn(a)gmail.com>
wrote:

Yes,all deas are built from same stemcell, all with fresh installation


+-------------------------------------------+---------+--------------------------------------+
| Name | Version | CID
|

+-------------------------------------------+---------+--------------------------------------+
| bosh-openstack-kvm-ubuntu-trusty-go_agent | 2978* |
bc3592a8-64fd-44b1-a9c0-797953f86e13 |

+-------------------------------------------+---------+--------------------------------------+

(*) Currently in-use


On Tue, Jul 28, 2015 at 9:42 AM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

Are you sure all you DEA are at the same version ?
I think so but I don't know maybe rare case.



On Tue, Jul 28, 2015 at 8:35 AM, Yitao Jiang <jiangyt.cn(a)gmail.com>
wrote:

Hi,

Does any body ever have the problem of deploying application failed in *cflinuxfs2
stack not found?*
we deployed v210.2 release and occasionally it failed in* cflinuxfs2
not found* when deploying. It's weird that the nats with 2 nodes all
works fine, i can manually received the msgs distributed among the cluster.

Regards,

Yitao
jiangyt.github.io

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--

Regards,

Yitao
jiangyt.github.io

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--

Regards,

Yitao
jiangyt.github.io


Re: Allow gorouter to log random headers.

Shannon Coen
 

Thank you, James. That helps.

What bothered me about this thread was that the use case was presented as,
"as an unspecified persona, I want to customize the multi-tenant router to
log specified headers for a few apps". This is troubling.

Instead it sounds like the use case is "as an operator of Cloud Foundry, I
want to enable logging of specified headers for all applications." This is
more compelling.

I'm skeptical that this manifest property should have a default. Who's to
say what the minimum list of "sensible" defaults is? Does WAKAWAKA make the
cut? j/k.

We'll take a PR.

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Fri, Jul 24, 2015 at 8:46 AM, David Laing <david(a)davidlaing.com> wrote:

James,

Yep - we're pretty much on the same page.

Only addition I would ask for is that the whitelist contain some
"sensible" defaults (eg, Trace-Id, Span-Id) that are switched on by
default; since then tight integration with tools like spring-cloud /
buildpacks would work out the box.

D

On 24 July 2015 at 16:26, James Bayer <jbayer(a)pivotal.io> wrote:

shannon,

from what i'm reading here about the use case, the main interest is that
a CF operator knows that their cf installation is more deeply integrated
with a specific log parsing solution for all/many apps on that platform
that choose to use it (whether than is ELK, Zipkin, etc). it does not sound
like it is a special case with lots of variation by many different app
teams on the cf installation. rather, it sounds like this is most likely to
be used as an installation-wide option to enhance the app developer / app
operations experience.

it seems like an operator configured whitelist set of headers that get
logged with the RTR message satisfies the current needs well and is
reasonable.

if we were to find that in the future lots of variation and different app
teams on the same CF wanted to have the RTR tier log many different headers
in the access log, then we could enhance the "log the whitelist of headers
to the access log" capability to be exposed to a limited number of headers
that each route could be configured by developers to specify that would
apply in addition to the operator configured logged headers. but it sounds
like that isn't needed right now.

simon/david, did i summarize this correctly?

On Fri, Jul 24, 2015 at 7:23 AM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

If I understand correctly, you're proposing that an operator of
CloudFoundry configure GoRouter, which is a multi-tenant, shared service,
with knowledge specific to one or a few applications

That is indeed the proposal. In my org we work closely with the
different development streams to provide a good out of the box experience
by creating certain opt-in features that makes their life easier. This
would be one of those features, if you pass certain headers in your
requests thats standardised across the entire online organisation you will
be able to query for them in Kibana.
I understand that this is not the case in all environments.

If GoRouter logged whatever headers were included in the request,
wouldn't this satisfy your requirements?

This would indeed satisfy my requests, but as David points out("However,
not having a whitelist of headers to log opens a possible DDOS attack
vector on the GoRouter") it might now be appropriate.

Based on what you've described, the persona is the app developer.
Not necessarily, we operators are also very interested in certain
headers.

so control of what is logged should be in their hands.
If one via the API could set headers that should be logged for an org,
space or application that would be magical. The need for a list of
"must-log-all-these-headers" would still be there I think as not to have to
maintain a list of these standardised headers across different objects.


On Thu, Jul 23, 2015 at 10:51 PM, Shannon Coen <scoen(a)pivotal.io> wrote:

This is not something that would be merged, as originally proposed,
without additional investigation and discussion.

If I understand correctly, you're proposing that an operator of
CloudFoundry configure GoRouter, which is a multi-tenant, shared service,
with knowledge specific to one or a few applications. This should not be an
operator responsibility, nor should the solution be specific to one or a
few applications.

The goal is "the flexibility of being able to annotate our logs with
what we consider to be important for our debugging purposes." More
specifically you're requesting logging of headers. Do you have a preference?

If GoRouter logged whatever headers were included in the request,
wouldn't this satisfy your requirements? Doesn't GoRouter do this already?

I'm interested in solving your requirement generically for all
applications, and focussing the user experience on the correct persona.
Based on what you've described, the persona is the app developer, so
control of what is logged should be in their hands. I'm also not convinced
GoRouter should have any knowledge of headers specific to one application
or another.





Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Wed, Jul 22, 2015 at 3:05 AM, Alex Lomov <
alexander.lomov(a)altoros.com> wrote:

Some time ago routing services were discussed on a CAB [1]. Here is a
description of this proposal.

Do you think that using such service will allow your developers to
cover this requirements?

[1]
http://www.activestate.com/blog/2015/02/cloud-foundry-advisory-board-meeting-2015-february
[2]
https://docs.google.com/document/d/1bGOQxiKkmaw6uaRWGd-sXpxL0Y28d3QihcluI15FiIA/edit#heading=h.8djffzes9pnb

On Jul 21, 2015, at 4:06 PM, Simon Johansson <simon(a)simonjohansson.com>
wrote:

Howdie!

We have some devs who want to be able to trace a request troughout
their applications.

user -> a -> b -> c
|
|-> d -> e

When a user makes a request to "a" uuid is generated inside the app,
and the request to "b" from "a" will set a header(call it WAKAWAKA to
uuid), "b"'s call will passthrough WAKAWAKA to "c" and "d", "d" will
passthrough WAKAWAKA to "e". Etc.

We aggregate all RTR logs into ELK so it would be super helpful to
them to be able to filter on WAKAWAKA and get all the access logs(and app
logs aswell, they mostly use GELF so its easy for them to add whatewher
field they want) from the services involved.

I had a quick peek at the gorouter(
https://github.com/cloudfoundry/gorouter/blob/76668f5818ea8c089ff52a14fcdfbf703c8e8767/access_log/access_log_record.go#L40)
and it seems like this should be a simple PR.

1. To gorouter.yml add
passthrough_headers:
- WAKAWAKA
- X-Random-Header

2. In makeRecord at the bottom add something like(in psuedo)

data = {}
for header in passthrough_headers:
header_val = r.FormatRequestHeader("X-Forwarded-For")
if header_val:
passthrough_headers[header] = header_val

if data:
fmt.Fprintf(b, data.to_stringified_json())

That would yield a log line like
blurgh.dev.cf.private.domain.com - [21/07/2015:10:17:05 +0000] "GET
/statements?ascending=true&since=2015-06-30T14%3A10%3A03.078Z&skipStatementId=30a88204-0779-4385-9859-e4aabd30baf0
HTTP/1.1" 200 0 17 "-" "NING/1.0" 10.230.31.2:46204
x_forwarded_for:"-" vcap_request_id:1e58195a-cde6-4afd-7f03-43061c9ea91c
response_time:0.004927106 app_id:9784cd03-050d-4b74-9e90-5f17134a3f08
{"WAKAWAKA": "Space is the place", "X-Random-Header": "Once upon a midnight
dreary, while I pondered weak and weary"}

The reason for a stringified JSON is to make it easy to parse with
logstash or other loganalysis tools.

Before I spend time implementing, is this something you would merge?
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Thank you,

James Bayer

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
David Laing
logsearch.io - build your own open source cloud logging cluster
http://davidlaing.com

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

8441 - 8460 of 9398