Date   

Re: Notifications on ORG, SPACE and USER modifications

Mike Youngstrom
 

We have the same need. Today we are polling the CC.

It would be nice for us also if we could get CC event notifications via
something like the firehose.

Mike

On Wed, Jul 22, 2015 at 10:23 AM, Juan Pablo Genovese <
juanpgenovese(a)gmail.com> wrote:

I mean, I know you can list those events thru the API, but I want
something that will react on an event instead of having to be constantly
polling for them.

2015-07-22 13:18 GMT-03:00 Juan Pablo Genovese <juanpgenovese(a)gmail.com>:

Sree,

thanks! Any pointers on how can I hook up to these audit events?

Thank you!

2015-07-22 13:12 GMT-03:00 Sree Tummidi <stummidi(a)pivotal.io>:

I believe there are audit events generated for all these actions which
can be captured and forwarded to an SIEM solution like splunk

Thanks,
Sree

Sent from my iPhone

On Jul 22, 2015, at 8:54 AM, Juan Pablo Genovese <
juanpgenovese(a)gmail.com> wrote:

Guys,

I need to somehow hook up into the Cloud Controller (CC) to capture ORG,
SPACE and USER deletion, insertion and update.

So far, I considered some approaches, such as forking the CC (the least
favorite) and modifying the code with some hooks, tapping into Nginx to
capture the requests, and using triggers in the database to capture each
event and send the necessary info to a service.

What do you think?
Any other idea you might have?

Thanks!

--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com


--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Merge Request for

Jeff Sloyer
 

Hey Gwenn,

Could you merge one more?

https://github.com/cloudfoundry-community/cf-meteor-buildpack/pull/7

Also, could you maybe make me a maintainer on this project as well?

On Mon, Jul 20, 2015 at 10:44 PM Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

I merge it.
This repos since unmaintained since some month.

On Tue, Jul 21, 2015 at 1:32 AM, Jeff Sloyer <jsloyer(a)gmail.com> wrote:

Hello,

I have submitted a PR (
https://github.com/cloudfoundry-community/cf-meteor-buildpack/pull/3)
for the Meteor buildpack, what is the process for it getting reviewed and
merged?



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Notifications on ORG, SPACE and USER modifications

Juan Pablo Genovese
 

I mean, I know you can list those events thru the API, but I want something
that will react on an event instead of having to be constantly polling for
them.

2015-07-22 13:18 GMT-03:00 Juan Pablo Genovese <juanpgenovese(a)gmail.com>:

Sree,

thanks! Any pointers on how can I hook up to these audit events?

Thank you!

2015-07-22 13:12 GMT-03:00 Sree Tummidi <stummidi(a)pivotal.io>:

I believe there are audit events generated for all these actions which
can be captured and forwarded to an SIEM solution like splunk

Thanks,
Sree

Sent from my iPhone

On Jul 22, 2015, at 8:54 AM, Juan Pablo Genovese <juanpgenovese(a)gmail.com>
wrote:

Guys,

I need to somehow hook up into the Cloud Controller (CC) to capture ORG,
SPACE and USER deletion, insertion and update.

So far, I considered some approaches, such as forking the CC (the least
favorite) and modifying the code with some hooks, tapping into Nginx to
capture the requests, and using triggers in the database to capture each
event and send the necessary info to a service.

What do you think?
Any other idea you might have?

Thanks!

--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com
--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com


Re: Notifications on ORG, SPACE and USER modifications

Juan Pablo Genovese
 

Sree,

thanks! Any pointers on how can I hook up to these audit events?

Thank you!

2015-07-22 13:12 GMT-03:00 Sree Tummidi <stummidi(a)pivotal.io>:

I believe there are audit events generated for all these actions which can
be captured and forwarded to an SIEM solution like splunk

Thanks,
Sree

Sent from my iPhone

On Jul 22, 2015, at 8:54 AM, Juan Pablo Genovese <juanpgenovese(a)gmail.com>
wrote:

Guys,

I need to somehow hook up into the Cloud Controller (CC) to capture ORG,
SPACE and USER deletion, insertion and update.

So far, I considered some approaches, such as forking the CC (the least
favorite) and modifying the code with some hooks, tapping into Nginx to
capture the requests, and using triggers in the database to capture each
event and send the necessary info to a service.

What do you think?
Any other idea you might have?

Thanks!

--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com


Re: Notifications on ORG, SPACE and USER modifications

Sree Tummidi
 

I believe there are audit events generated for all these actions which can be captured and forwarded to an SIEM solution like splunk

Thanks,
Sree

Sent from my iPhone

On Jul 22, 2015, at 8:54 AM, Juan Pablo Genovese <juanpgenovese(a)gmail.com> wrote:

Guys,

I need to somehow hook up into the Cloud Controller (CC) to capture ORG, SPACE and USER deletion, insertion and update.

So far, I considered some approaches, such as forking the CC (the least favorite) and modifying the code with some hooks, tapping into Nginx to capture the requests, and using triggers in the database to capture each event and send the necessary info to a service.

What do you think?
Any other idea you might have?

Thanks!

--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Notifications on ORG, SPACE and USER modifications

Juan Pablo Genovese
 

Guys,

I need to somehow hook up into the Cloud Controller (CC) to capture ORG,
SPACE and USER deletion, insertion and update.

So far, I considered some approaches, such as forking the CC (the least
favorite) and modifying the code with some hooks, tapping into Nginx to
capture the requests, and using triggers in the database to capture each
event and send the necessary info to a service.

What do you think?
Any other idea you might have?

Thanks!

--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com


Re: Assigning Role to Group

Filip Hanik
 

To elaborate a bit more, at this time the cloud controller maintains its
own roles and ACLs in the CC database.

Filip

On Wednesday, July 22, 2015, Sree Tummidi <stummidi(a)pivotal.io> wrote:

This support is not yet available

Thanks,
Sree

Sent from my iPad

On Jul 22, 2015, at 4:35 AM, Daniel Mikusa <dmikusa(a)pivotal.io
<javascript:_e(%7B%7D,'cvml','dmikusa(a)pivotal.io');>> wrote:

On Wed, Jul 22, 2015 at 3:27 AM, Zakharov Alexey <
alexey.zakharov(a)altoros.com
<javascript:_e(%7B%7D,'cvml','alexey.zakharov(a)altoros.com');>> wrote:

>* Hi guys!
*>* Sorry if my question is newbie or it was discussed before.
*>* I want to use LDAP for users authentication/authorisation. And I’ve
*>* successfully bound CF to LDAP, and managed to configure uaac group mappings.
*>* But then I realised, that there are no way to assign a Role to that group.
*>* 'cf set-org-role’ accepts only usernames as parameter, but not groups. I
*>* think assigning Developer role to group is more flexible than assigning is
*>* to every particular user.
*>* Are you going to add this feature later? Or maybe there is an another way
*>* to do group binding?
*>
Have you looked at the `uaac` tool? I'm not quite sure I understand what
you're trying to do, but you can map an LDAP group DN to a UAA group with
`uaac`. Then if a user in that LDAP group logs in, they'll have that uaa
group. Is that what you're looking to do?

Ex:

uaac group map --name cloud_controller.admin "GROUP-DISTINGUISHED-NAME"

Or are you asking about mapping LDAP groups to CF org & space roles? i.e.
user in ldap group X is automatically given the OrgManager role in org Y.

Dan

Hi Dan!

Yes, as I’ve stated before, I’ve already managed to configure group mappings using ‘uaac group map’.

And now I want to bind group members to Organizations and Spaces. Is it possible to do?

Sorry, missed that in your original post. Last I heard no you couldn't do
this mapping, but that was a while ago though. Maybe someone on the
Identity team could confirm.

Dan

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
<javascript:_e(%7B%7D,'cvml','cf-dev(a)lists.cloudfoundry.org');>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Assigning Role to Group

Sree Tummidi
 

This support is not yet available

Thanks,
Sree

Sent from my iPad

On Jul 22, 2015, at 4:35 AM, Daniel Mikusa <dmikusa(a)pivotal.io> wrote:

On Wed, Jul 22, 2015 at 3:27 AM, Zakharov Alexey <alexey.zakharov(a)altoros.com> wrote:
Hi guys!
Sorry if my question is newbie or it was discussed before.
I want to use LDAP for users authentication/authorisation. And I’ve
successfully bound CF to LDAP, and managed to configure uaac group mappings.
But then I realised, that there are no way to assign a Role to that group.
'cf set-org-role’ accepts only usernames as parameter, but not groups. I
think assigning Developer role to group is more flexible than assigning is
to every particular user.
Are you going to add this feature later? Or maybe there is an another way
to do group binding?
Have you looked at the `uaac` tool? I'm not quite sure I understand what
you're trying to do, but you can map an LDAP group DN to a UAA group with
`uaac`. Then if a user in that LDAP group logs in, they'll have that uaa
group. Is that what you're looking to do?

Ex:

uaac group map --name cloud_controller.admin "GROUP-DISTINGUISHED-NAME"

Or are you asking about mapping LDAP groups to CF org & space roles? i.e.
user in ldap group X is automatically given the OrgManager role in org Y.

Dan
Hi Dan!
Yes, as I’ve stated before, I’ve already managed to configure group mappings using ‘uaac group map’.
And now I want to bind group members to Organizations and Spaces. Is it possible to do?
Sorry, missed that in your original post. Last I heard no you couldn't do this mapping, but that was a while ago though. Maybe someone on the Identity team could confirm.

Dan

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Did anybody deploy a wiki as app to CF?

Daniel Mikusa
 

That's awesome!

On Wed, Jul 22, 2015 at 3:14 AM, Dieu Cao <dcao(a)pivotal.io> wrote:

Very cool!

On Tue, Jul 21, 2015 at 8:16 PM, Cornelia Davis <cdavis(a)pivotal.io> wrote:

What Josh said!! WOW!

On Tue, Jul 21, 2015 at 6:53 PM, Josh Long <starbuxman(a)gmail.com> wrote:

WOW!



--


Thanks,
Josh Long
the Spring Developer Advocate, Pivotal
http://joshlong.com || http://twitter.com/starbuxman ||
http://spring.io/team/jlong

On July 21, 2015 at 5:44:50 PM, Noburou TANIGUCHI (dev(a)nota.m001.jp)
wrote:

cloudfoundry.gr.jp, a Cloud Foundry user group in Japan, is currently
running
a project (or a campaign, more properly) called "Cloud Foundry 100-day
challenge" ("Cloud Foundry 100-nichi Gyou" in Japanese. "gyou"
originally
means a kind of discipline or training in Buddhism). It is an activity
to
make 100 open source apps run on Cloud Foundry, one app per day.

http://blog.cloudfoundry.gr.jp/search/label/%E7%99%BE%E6%97%A5%E8%A1%8C

We have done 33 apps until now. We are sorry the all articles are
presented
only in Japanese. Do they help your project, James?




--
View this message in context:
http://cf-dev.70369.x6.nabble.com/cf-dev-Did-anybody-deploy-a-wiki-as-app-to-CF-tp643p808.html
Sent from the CF Dev mailing list archive at Nabble.com.
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Assigning Role to Group

Daniel Mikusa
 

On Wed, Jul 22, 2015 at 3:27 AM, Zakharov Alexey <
alexey.zakharov(a)altoros.com> wrote:

>* Hi guys!
*>* Sorry if my question is newbie or it was discussed before.
*>* I want to use LDAP for users authentication/authorisation. And I’ve
*>* successfully bound CF to LDAP, and managed to configure uaac group mappings.
*>* But then I realised, that there are no way to assign a Role to that group.
*>* 'cf set-org-role’ accepts only usernames as parameter, but not groups. I
*>* think assigning Developer role to group is more flexible than assigning is
*>* to every particular user.
*>* Are you going to add this feature later? Or maybe there is an another way
*>* to do group binding?
*>
Have you looked at the `uaac` tool? I'm not quite sure I understand what
you're trying to do, but you can map an LDAP group DN to a UAA group with
`uaac`. Then if a user in that LDAP group logs in, they'll have that uaa
group. Is that what you're looking to do?

Ex:

uaac group map --name cloud_controller.admin "GROUP-DISTINGUISHED-NAME"

Or are you asking about mapping LDAP groups to CF org & space roles? i.e.
user in ldap group X is automatically given the OrgManager role in org Y.

Dan

Hi Dan!

Yes, as I’ve stated before, I’ve already managed to configure group mappings using ‘uaac group map’.

And now I want to bind group members to Organizations and Spaces. Is it possible to do?

Sorry, missed that in your original post. Last I heard no you couldn't do
this mapping, but that was a while ago though. Maybe someone on the
Identity team could confirm.

Dan


[vcap-dev] Loggregator dropping application logs and warning 'message output too high'

Daniel Jones
 

Hi all,

Enacting a bit of thread necromancy in the hope of aiding a few Java CF
users.

1. Logback 1.1.3 has new functionality that allows selective truncation
of Java stack traces: https://github.com/qos-ch/logback/pull/244. It's
not documented, but I've used it in tested code.
2. There may or may not be a flaw in the logic of Doppler which is
throttling messages sent to syslog to one per millisecond:
https://github.com/cloudfoundry/loggregator/issues/66
3. I've also raised issues to discuss the merits of dropping the whole
buffer as opposed to refusing new messages and/or allowing buffers to grow
in size: https://github.com/cloudfoundry/loggregator/issues/67
4. There's also this issue discussing the parameterisation of buffer
sizes: https://github.com/cloudfoundry/loggregator/issues/64

Hope the above is of help to someone.

On Thu, Mar 5, 2015 at 6:42 PM, Mike Heath <elcapo(a)gmail.com> wrote:

I have to disagree with regard to putting back pressure on the
application. If someone is tailing the logs of an app over a slow VPN
connection, that should not slow the application down. I do agree that 100
retained log messages is too few. We retain 500 log messages on our Cloud
Foundry instances and that seems to work well for us.

-Mike



On Thursday, March 5, 2015 at 9:17:54 AM UTC-7, Yann ROBERT wrote:

Hi James,

Just to clarify on the logging rate :
The main issue that is occuring pretty often is when an application is
logging let's say 120 lines of logs at a time and then keep quiet. This is
a pretty common scenario when a Java application is printing a stacktrace
after a failure. For instance if you have 2 or 3 threads accessing a
backend and the connection to the backend fails then all of the running
thread will get a failure, which may result in more than 100 lines of logs
at the same time.
This simple scenario is enough to get the LGR buffer overflowed because
it cannot send to the syslog before having it's buffer full.

NOTE : The number of lines required to be sent at once to get the LGR
overflow differs upon the public cloud provider I tested this.

In the light of the previous explaination, because the buffer size is
only 100, it's simply impossible to support that scenario.

I am not talking about a scenario when the application is constantly
logging 200 lines per seconds for several minutes or hours.

On the syslog server capability to keep up, and the available bandwidth
between CF and the syslog server :
My tests show that when the application is directly sending logs to the
syslog server, no log is losts, and the syslog server is able to keep up
with the flow, be it a temporary peak in the flow or a constant flow of
100s of lines per seconds.
So I am pretty confident that the hosted highly available syslog service
is able to keep up with a peak of 100 lines.

The question is just how much time it take for the LGR buffer to be full.
It you send 120 lines at the same time to STDOUT, the buffer is full
instantly and overflows immediately.
It ends up being FIRST a problem with the buffer size.

Java application are very chatty when it comes to failures because a
common practice is to print the stack trace.
Supporting very temporary peak in logs rate would be great. The default
buffer size should be 1000, not 100.

Also the strategy of what is done when the buffer is full is questionable.

Putting backpressure on the application is fine to me.
I guess some application developers would rather have the thread that is
logging be blocked while the buffer is full, than to loose some logs.
It would end up being the developers responsibility to either leave it
that way if it's rare enough, or optimize the application so that it logs
less. The developer would be in control.

An other strategy to discard to logs would be to discard the new messages
instead of discarding the content of the buffer. This would allow to at
least get 100 first lines of logs instead of none or just the latest lines
if your lucky.
Alternatively, using a circular buffer may also be interesting to avoid
dropping chunks; and allows to only drop the oldest overflow.

A dynamic strategy that would discard the logs based on the level (INFO,
WARN, ERROR, ...) like what is done by logback's AsyncAppender [
http://logback.qos.ch/apidocs/ch/qos/logback/classic/AsyncAppender.html
] would be great but we don't have a level on plain STDOUT flow. Also, when
all logs are important because it's WARNING or ERROR level, you don't want
to filter them.

A dynamic strategy that would arbitrarily discard 1 line out of 2 may be
better than dropping a full chunk of 100 messages. Because when you look at
the logs it's easier to 'read between the lines'.

We can think of many more alternative strategies to discard messages. It
all depends on the developers need.
Putting backpressure on the application to provide control to the
developer is what will work best.

Yann

--
You received this message because you are subscribed to the Google Groups
"Cloud Foundry Developers" group.
To view this discussion on the web visit
https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/3b9036ed-787c-4b12-b4da-6e317cabac52%40cloudfoundry.org
<https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/3b9036ed-787c-4b12-b4da-6e317cabac52%40cloudfoundry.org?utm_medium=email&utm_source=footer>
.

To unsubscribe from this group and stop receiving emails from it, send an
email to vcap-dev+unsubscribe(a)cloudfoundry.org.


--
Regards,

Daniel Jones
EngineerBetter.com
--
Regards,

Daniel Jones
EngineerBetter.com


Re: Allow gorouter to log random headers.

Alexander Lomov <alexander.lomov@...>
 

On Jul 21, 2015, at 4:06 PM, Simon Johansson <simon(a)simonjohansson.com> wrote:

Howdie!

We have some devs who want to be able to trace a request troughout their applications.

user -> a -> b -> c
|
|-> d -> e

When a user makes a request to "a" uuid is generated inside the app, and the request to "b" from "a" will set a header(call it WAKAWAKA to uuid), "b"'s call will passthrough WAKAWAKA to "c" and "d", "d" will passthrough WAKAWAKA to "e". Etc.

We aggregate all RTR logs into ELK so it would be super helpful to them to be able to filter on WAKAWAKA and get all the access logs(and app logs aswell, they mostly use GELF so its easy for them to add whatewher field they want) from the services involved.

I had a quick peek at the gorouter(https://github.com/cloudfoundry/gorouter/blob/76668f5818ea8c089ff52a14fcdfbf703c8e8767/access_log/access_log_record.go#L40 <https://github.com/cloudfoundry/gorouter/blob/76668f5818ea8c089ff52a14fcdfbf703c8e8767/access_log/access_log_record.go#L40>) and it seems like this should be a simple PR.

1. To gorouter.yml add
passthrough_headers:
- WAKAWAKA
- X-Random-Header

2. In makeRecord at the bottom add something like(in psuedo)

data = {}
for header in passthrough_headers:
header_val = r.FormatRequestHeader("X-Forwarded-For")
if header_val:
passthrough_headers[header] = header_val

if data:
fmt.Fprintf(b, data.to_stringified_json())

That would yield a log line like
blurgh.dev.cf.private.domain.com <http://blurgh.dev.cf.private.domain.com/> - [21/07/2015:10:17:05 +0000] "GET /statements?ascending=true&since=2015-06-30T14%3A10%3A03.078Z&skipStatementId=30a88204-0779-4385-9859-e4aabd30baf0 HTTP/1.1" 200 0 17 "-" "NING/1.0" 10.230.31.2:46204 <http://10.230.31.2:46204/> x_forwarded_for:"-" vcap_request_id:1e58195a-cde6-4afd-7f03-43061c9ea91c response_time:0.004927106 app_id:9784cd03-050d-4b74-9e90-5f17134a3f08 {"WAKAWAKA": "Space is the place", "X-Random-Header": "Once upon a midnight dreary, while I pondered weak and weary"}

The reason for a stringified JSON is to make it easy to parse with logstash or other loganalysis tools.

Before I spend time implementing, is this something you would merge?
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Allow gorouter to log random headers.

Simon Johansson <simon@...>
 

Well, from my point of view the customization is just adding the
flexibility of being able to annotate our logs with what we consider to be
important for our debugging purposes, a feature that is surely interesting
for other parties aswell. We are not interested in adding extra headers
into CF, just pulling headers from incoming requests(where the headers have
ben set elsewhere) to Gorouter into a doppler event. :)

On Wed, Jul 22, 2015 at 11:43 AM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

Simon I know what you want todo :) I am shinji62 lol
I am jsut afraid that you want to customize the platform and you will be
stuck with because other don't support custom header.

On Wed, Jul 22, 2015 at 6:36 PM, Simon Johansson <simon(a)simonjohansson.com
wrote:
I feel that we are stearing this conversation into the wrong direction.

Gist it, I would like to with cf logs app / firehose-to-syslog see my
RTR-logs as

blurgh.dev.cf.private.domain.com - blablablab
vcap_request_id:1e58195a-cde6-4afd-7f03-43061c9ea91c
response_time:0.004927106 app_id:9784cd03-050d-4b74-9e90-5f17134a3f08
*{"X-Random-Header": "Once upon a midnight dreary, while I pondered weak
and weary", "Another-Header": "wryyyy"}*

By being able to tell Gorouter what headers Im interested in via the
manifest
logging:
add_extra_headers_if_available:
- X-Random-Header
- Another-Header
- This-header-will-only-be-in-1%-of-the-requests

Is this something you would be interested in mergin, I dont want to
implement it if there is no chance of it being merged.


On Wed, Jul 22, 2015 at 11:30 AM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

So if you move to Heroku you will be able to change the platform ??? I
don't think so..
So you can put the new header from your app or at least the runtime
buildpack or docker.

Your application should be platform dependent that's why you need to
implement this header into your application to avoid any lock-in or forking
issue.

On Wed, Jul 22, 2015 at 6:27 PM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

WAKAWAKA is just an example
But the difference is that WAKAWAKA is not platform specific whereas
X-Cf-Requestid is. If we want to move our app to Heroku, or a VM, or
whatewher we have platform specific implementation details in our
app(namely we rely on a header that is not there anymore). But that is not
the point of this thread.

The point is,

say we have an app that is fronted by a CDN, and the CDN sets the
X-Im-a-shark header with some value that we are interested to see in our
logs. The easiest way to achivie this without having to implement it into
our own apps is just to tell the Gorouter that it should append the value
of that header into the string that it logs so the event that flow via
Doppler and ultimately into cf logs/ELK will contain that value.

The reason why this would be such a valuable feature for us is that we
dont have to do anything. CF already provide the out of the box facility to
give us routing logs, so if we can piggy back on that for what we are
interested in we dont have to add libraries to our apps to log interesting
headers on the side.



On Wed, Jul 22, 2015 at 12:45 AM, Shannon Coen <scoen(a)pivotal.io>
wrote:

I don't see the difference between WAKAWAKA and X-Cf-Requestid.
Gorouter would have to add some header with a uuid for the request. Your
apps have to have logic to pass this header on, so that a log search
returns the original request as well as subsequent requests between apps.
Could you please clarify?

Thank you,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Tue, Jul 21, 2015 at 11:09 AM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

Sure that header can be used. But then we are adding CF specific
stuff into the implementation of our services, which is something we want
to avoid at all costs.
Also Im not entirely sure if all the libraries we use for Zipkin
supports using custom headers.

All our public apps are fronted by different CDNs, which sets headers
that we might want to store for debugging, so we still need a way to pass
those trough into the log.

On Tuesday, 21 July 2015, Shannon Coen <scoen(a)pivotal.io> wrote:

Hello Simon,

The X-Cf-Requestid header already provides a uuid. Couldn't app "a"
add the value of X-Cf-Requestid to a header of your choosing? Call it
WAKAWAKA or X-Random-Header, but it doesn't need to be a platform standard.
Wouldn't a search for the value of X-Cf-Requestid then provide the desired
results?

Thank you,
Shannon

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Tue, Jul 21, 2015 at 6:06 AM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

Howdie!

We have some devs who want to be able to trace a request troughout
their applications.

user -> a -> b -> c
|
|-> d -> e

When a user makes a request to "a" uuid is generated inside the
app, and the request to "b" from "a" will set a header(call it WAKAWAKA to
uuid), "b"'s call will passthrough WAKAWAKA to "c" and "d", "d" will
passthrough WAKAWAKA to "e". Etc.

We aggregate all RTR logs into ELK so it would be super helpful to
them to be able to filter on WAKAWAKA and get all the access logs(and app
logs aswell, they mostly use GELF so its easy for them to add whatewher
field they want) from the services involved.

I had a quick peek at the gorouter(
https://github.com/cloudfoundry/gorouter/blob/76668f5818ea8c089ff52a14fcdfbf703c8e8767/access_log/access_log_record.go#L40)
and it seems like this should be a simple PR.

1. To gorouter.yml add
passthrough_headers:
- WAKAWAKA
- X-Random-Header

2. In makeRecord at the bottom add something like(in psuedo)

data = {}
for header in passthrough_headers:
header_val = r.FormatRequestHeader("X-Forwarded-For")
if header_val:
passthrough_headers[header] = header_val

if data:
fmt.Fprintf(b, data.to_stringified_json())

That would yield a log line like
blurgh.dev.cf.private.domain.com - [21/07/2015:10:17:05 +0000]
"GET
/statements?ascending=true&since=2015-06-30T14%3A10%3A03.078Z&skipStatementId=30a88204-0779-4385-9859-e4aabd30baf0
HTTP/1.1" 200 0 17 "-" "NING/1.0" 10.230.31.2:46204
x_forwarded_for:"-" vcap_request_id:1e58195a-cde6-4afd-7f03-43061c9ea91c
response_time:0.004927106 app_id:9784cd03-050d-4b74-9e90-5f17134a3f08
{"WAKAWAKA": "Space is the place", "X-Random-Header": "Once upon a midnight
dreary, while I pondered weak and weary"}

The reason for a stringified JSON is to make it easy to parse with
logstash or other loganalysis tools.

Before I spend time implementing, is this something you would merge?

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Allow gorouter to log random headers.

Gwenn Etourneau
 

Simon I know what you want todo :) I am shinji62 lol
I am jsut afraid that you want to customize the platform and you will be
stuck with because other don't support custom header.

On Wed, Jul 22, 2015 at 6:36 PM, Simon Johansson <simon(a)simonjohansson.com>
wrote:

I feel that we are stearing this conversation into the wrong direction.

Gist it, I would like to with cf logs app / firehose-to-syslog see my
RTR-logs as

blurgh.dev.cf.private.domain.com - blablablab
vcap_request_id:1e58195a-cde6-4afd-7f03-43061c9ea91c
response_time:0.004927106 app_id:9784cd03-050d-4b74-9e90-5f17134a3f08
*{"X-Random-Header": "Once upon a midnight dreary, while I pondered weak
and weary", "Another-Header": "wryyyy"}*

By being able to tell Gorouter what headers Im interested in via the
manifest
logging:
add_extra_headers_if_available:
- X-Random-Header
- Another-Header
- This-header-will-only-be-in-1%-of-the-requests

Is this something you would be interested in mergin, I dont want to
implement it if there is no chance of it being merged.


On Wed, Jul 22, 2015 at 11:30 AM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

So if you move to Heroku you will be able to change the platform ??? I
don't think so..
So you can put the new header from your app or at least the runtime
buildpack or docker.

Your application should be platform dependent that's why you need to
implement this header into your application to avoid any lock-in or forking
issue.

On Wed, Jul 22, 2015 at 6:27 PM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

WAKAWAKA is just an example
But the difference is that WAKAWAKA is not platform specific whereas
X-Cf-Requestid is. If we want to move our app to Heroku, or a VM, or
whatewher we have platform specific implementation details in our
app(namely we rely on a header that is not there anymore). But that is not
the point of this thread.

The point is,

say we have an app that is fronted by a CDN, and the CDN sets the
X-Im-a-shark header with some value that we are interested to see in our
logs. The easiest way to achivie this without having to implement it into
our own apps is just to tell the Gorouter that it should append the value
of that header into the string that it logs so the event that flow via
Doppler and ultimately into cf logs/ELK will contain that value.

The reason why this would be such a valuable feature for us is that we
dont have to do anything. CF already provide the out of the box facility to
give us routing logs, so if we can piggy back on that for what we are
interested in we dont have to add libraries to our apps to log interesting
headers on the side.



On Wed, Jul 22, 2015 at 12:45 AM, Shannon Coen <scoen(a)pivotal.io> wrote:

I don't see the difference between WAKAWAKA and X-Cf-Requestid.
Gorouter would have to add some header with a uuid for the request. Your
apps have to have logic to pass this header on, so that a log search
returns the original request as well as subsequent requests between apps.
Could you please clarify?

Thank you,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Tue, Jul 21, 2015 at 11:09 AM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

Sure that header can be used. But then we are adding CF specific stuff
into the implementation of our services, which is something we want to
avoid at all costs.
Also Im not entirely sure if all the libraries we use for Zipkin
supports using custom headers.

All our public apps are fronted by different CDNs, which sets headers
that we might want to store for debugging, so we still need a way to pass
those trough into the log.

On Tuesday, 21 July 2015, Shannon Coen <scoen(a)pivotal.io> wrote:

Hello Simon,

The X-Cf-Requestid header already provides a uuid. Couldn't app "a"
add the value of X-Cf-Requestid to a header of your choosing? Call it
WAKAWAKA or X-Random-Header, but it doesn't need to be a platform standard.
Wouldn't a search for the value of X-Cf-Requestid then provide the desired
results?

Thank you,
Shannon

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Tue, Jul 21, 2015 at 6:06 AM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

Howdie!

We have some devs who want to be able to trace a request troughout
their applications.

user -> a -> b -> c
|
|-> d -> e

When a user makes a request to "a" uuid is generated inside the app,
and the request to "b" from "a" will set a header(call it WAKAWAKA to
uuid), "b"'s call will passthrough WAKAWAKA to "c" and "d", "d" will
passthrough WAKAWAKA to "e". Etc.

We aggregate all RTR logs into ELK so it would be super helpful to
them to be able to filter on WAKAWAKA and get all the access logs(and app
logs aswell, they mostly use GELF so its easy for them to add whatewher
field they want) from the services involved.

I had a quick peek at the gorouter(
https://github.com/cloudfoundry/gorouter/blob/76668f5818ea8c089ff52a14fcdfbf703c8e8767/access_log/access_log_record.go#L40)
and it seems like this should be a simple PR.

1. To gorouter.yml add
passthrough_headers:
- WAKAWAKA
- X-Random-Header

2. In makeRecord at the bottom add something like(in psuedo)

data = {}
for header in passthrough_headers:
header_val = r.FormatRequestHeader("X-Forwarded-For")
if header_val:
passthrough_headers[header] = header_val

if data:
fmt.Fprintf(b, data.to_stringified_json())

That would yield a log line like
blurgh.dev.cf.private.domain.com - [21/07/2015:10:17:05 +0000] "GET
/statements?ascending=true&since=2015-06-30T14%3A10%3A03.078Z&skipStatementId=30a88204-0779-4385-9859-e4aabd30baf0
HTTP/1.1" 200 0 17 "-" "NING/1.0" 10.230.31.2:46204
x_forwarded_for:"-" vcap_request_id:1e58195a-cde6-4afd-7f03-43061c9ea91c
response_time:0.004927106 app_id:9784cd03-050d-4b74-9e90-5f17134a3f08
{"WAKAWAKA": "Space is the place", "X-Random-Header": "Once upon a midnight
dreary, while I pondered weak and weary"}

The reason for a stringified JSON is to make it easy to parse with
logstash or other loganalysis tools.

Before I spend time implementing, is this something you would merge?

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Allow gorouter to log random headers.

Simon Johansson <simon@...>
 

I feel that we are stearing this conversation into the wrong direction.

Gist it, I would like to with cf logs app / firehose-to-syslog see my
RTR-logs as

blurgh.dev.cf.private.domain.com - blablablab
vcap_request_id:1e58195a-cde6-4afd-7f03-43061c9ea91c
response_time:0.004927106 app_id:9784cd03-050d-4b74-9e90-5f17134a3f08
*{"X-Random-Header": "Once upon a midnight dreary, while I pondered weak
and weary", "Another-Header": "wryyyy"}*

By being able to tell Gorouter what headers Im interested in via the
manifest
logging:
add_extra_headers_if_available:
- X-Random-Header
- Another-Header
- This-header-will-only-be-in-1%-of-the-requests

Is this something you would be interested in mergin, I dont want to
implement it if there is no chance of it being merged.


On Wed, Jul 22, 2015 at 11:30 AM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

So if you move to Heroku you will be able to change the platform ??? I
don't think so..
So you can put the new header from your app or at least the runtime
buildpack or docker.

Your application should be platform dependent that's why you need to
implement this header into your application to avoid any lock-in or forking
issue.

On Wed, Jul 22, 2015 at 6:27 PM, Simon Johansson <simon(a)simonjohansson.com
wrote:
WAKAWAKA is just an example
But the difference is that WAKAWAKA is not platform specific whereas
X-Cf-Requestid is. If we want to move our app to Heroku, or a VM, or
whatewher we have platform specific implementation details in our
app(namely we rely on a header that is not there anymore). But that is not
the point of this thread.

The point is,

say we have an app that is fronted by a CDN, and the CDN sets the
X-Im-a-shark header with some value that we are interested to see in our
logs. The easiest way to achivie this without having to implement it into
our own apps is just to tell the Gorouter that it should append the value
of that header into the string that it logs so the event that flow via
Doppler and ultimately into cf logs/ELK will contain that value.

The reason why this would be such a valuable feature for us is that we
dont have to do anything. CF already provide the out of the box facility to
give us routing logs, so if we can piggy back on that for what we are
interested in we dont have to add libraries to our apps to log interesting
headers on the side.



On Wed, Jul 22, 2015 at 12:45 AM, Shannon Coen <scoen(a)pivotal.io> wrote:

I don't see the difference between WAKAWAKA and X-Cf-Requestid. Gorouter
would have to add some header with a uuid for the request. Your apps have
to have logic to pass this header on, so that a log search returns the
original request as well as subsequent requests between apps. Could you
please clarify?

Thank you,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Tue, Jul 21, 2015 at 11:09 AM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

Sure that header can be used. But then we are adding CF specific stuff
into the implementation of our services, which is something we want to
avoid at all costs.
Also Im not entirely sure if all the libraries we use for Zipkin
supports using custom headers.

All our public apps are fronted by different CDNs, which sets headers
that we might want to store for debugging, so we still need a way to pass
those trough into the log.

On Tuesday, 21 July 2015, Shannon Coen <scoen(a)pivotal.io> wrote:

Hello Simon,

The X-Cf-Requestid header already provides a uuid. Couldn't app "a"
add the value of X-Cf-Requestid to a header of your choosing? Call it
WAKAWAKA or X-Random-Header, but it doesn't need to be a platform standard.
Wouldn't a search for the value of X-Cf-Requestid then provide the desired
results?

Thank you,
Shannon

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Tue, Jul 21, 2015 at 6:06 AM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

Howdie!

We have some devs who want to be able to trace a request troughout
their applications.

user -> a -> b -> c
|
|-> d -> e

When a user makes a request to "a" uuid is generated inside the app,
and the request to "b" from "a" will set a header(call it WAKAWAKA to
uuid), "b"'s call will passthrough WAKAWAKA to "c" and "d", "d" will
passthrough WAKAWAKA to "e". Etc.

We aggregate all RTR logs into ELK so it would be super helpful to
them to be able to filter on WAKAWAKA and get all the access logs(and app
logs aswell, they mostly use GELF so its easy for them to add whatewher
field they want) from the services involved.

I had a quick peek at the gorouter(
https://github.com/cloudfoundry/gorouter/blob/76668f5818ea8c089ff52a14fcdfbf703c8e8767/access_log/access_log_record.go#L40)
and it seems like this should be a simple PR.

1. To gorouter.yml add
passthrough_headers:
- WAKAWAKA
- X-Random-Header

2. In makeRecord at the bottom add something like(in psuedo)

data = {}
for header in passthrough_headers:
header_val = r.FormatRequestHeader("X-Forwarded-For")
if header_val:
passthrough_headers[header] = header_val

if data:
fmt.Fprintf(b, data.to_stringified_json())

That would yield a log line like
blurgh.dev.cf.private.domain.com - [21/07/2015:10:17:05 +0000] "GET
/statements?ascending=true&since=2015-06-30T14%3A10%3A03.078Z&skipStatementId=30a88204-0779-4385-9859-e4aabd30baf0
HTTP/1.1" 200 0 17 "-" "NING/1.0" 10.230.31.2:46204
x_forwarded_for:"-" vcap_request_id:1e58195a-cde6-4afd-7f03-43061c9ea91c
response_time:0.004927106 app_id:9784cd03-050d-4b74-9e90-5f17134a3f08
{"WAKAWAKA": "Space is the place", "X-Random-Header": "Once upon a midnight
dreary, while I pondered weak and weary"}

The reason for a stringified JSON is to make it easy to parse with
logstash or other loganalysis tools.

Before I spend time implementing, is this something you would merge?

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Allow gorouter to log random headers.

Gwenn Etourneau
 

So if you move to Heroku you will be able to change the platform ??? I
don't think so..
So you can put the new header from your app or at least the runtime
buildpack or docker.

Your application should be platform dependent that's why you need to
implement this header into your application to avoid any lock-in or forking
issue.

On Wed, Jul 22, 2015 at 6:27 PM, Simon Johansson <simon(a)simonjohansson.com>
wrote:

WAKAWAKA is just an example
But the difference is that WAKAWAKA is not platform specific whereas
X-Cf-Requestid is. If we want to move our app to Heroku, or a VM, or
whatewher we have platform specific implementation details in our
app(namely we rely on a header that is not there anymore). But that is not
the point of this thread.

The point is,

say we have an app that is fronted by a CDN, and the CDN sets the
X-Im-a-shark header with some value that we are interested to see in our
logs. The easiest way to achivie this without having to implement it into
our own apps is just to tell the Gorouter that it should append the value
of that header into the string that it logs so the event that flow via
Doppler and ultimately into cf logs/ELK will contain that value.

The reason why this would be such a valuable feature for us is that we
dont have to do anything. CF already provide the out of the box facility to
give us routing logs, so if we can piggy back on that for what we are
interested in we dont have to add libraries to our apps to log interesting
headers on the side.



On Wed, Jul 22, 2015 at 12:45 AM, Shannon Coen <scoen(a)pivotal.io> wrote:

I don't see the difference between WAKAWAKA and X-Cf-Requestid. Gorouter
would have to add some header with a uuid for the request. Your apps have
to have logic to pass this header on, so that a log search returns the
original request as well as subsequent requests between apps. Could you
please clarify?

Thank you,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Tue, Jul 21, 2015 at 11:09 AM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

Sure that header can be used. But then we are adding CF specific stuff
into the implementation of our services, which is something we want to
avoid at all costs.
Also Im not entirely sure if all the libraries we use for Zipkin
supports using custom headers.

All our public apps are fronted by different CDNs, which sets headers
that we might want to store for debugging, so we still need a way to pass
those trough into the log.

On Tuesday, 21 July 2015, Shannon Coen <scoen(a)pivotal.io> wrote:

Hello Simon,

The X-Cf-Requestid header already provides a uuid. Couldn't app "a" add
the value of X-Cf-Requestid to a header of your choosing? Call it WAKAWAKA
or X-Random-Header, but it doesn't need to be a platform standard. Wouldn't
a search for the value of X-Cf-Requestid then provide the desired results?

Thank you,
Shannon

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Tue, Jul 21, 2015 at 6:06 AM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

Howdie!

We have some devs who want to be able to trace a request troughout
their applications.

user -> a -> b -> c
|
|-> d -> e

When a user makes a request to "a" uuid is generated inside the app,
and the request to "b" from "a" will set a header(call it WAKAWAKA to
uuid), "b"'s call will passthrough WAKAWAKA to "c" and "d", "d" will
passthrough WAKAWAKA to "e". Etc.

We aggregate all RTR logs into ELK so it would be super helpful to
them to be able to filter on WAKAWAKA and get all the access logs(and app
logs aswell, they mostly use GELF so its easy for them to add whatewher
field they want) from the services involved.

I had a quick peek at the gorouter(
https://github.com/cloudfoundry/gorouter/blob/76668f5818ea8c089ff52a14fcdfbf703c8e8767/access_log/access_log_record.go#L40)
and it seems like this should be a simple PR.

1. To gorouter.yml add
passthrough_headers:
- WAKAWAKA
- X-Random-Header

2. In makeRecord at the bottom add something like(in psuedo)

data = {}
for header in passthrough_headers:
header_val = r.FormatRequestHeader("X-Forwarded-For")
if header_val:
passthrough_headers[header] = header_val

if data:
fmt.Fprintf(b, data.to_stringified_json())

That would yield a log line like
blurgh.dev.cf.private.domain.com - [21/07/2015:10:17:05 +0000] "GET
/statements?ascending=true&since=2015-06-30T14%3A10%3A03.078Z&skipStatementId=30a88204-0779-4385-9859-e4aabd30baf0
HTTP/1.1" 200 0 17 "-" "NING/1.0" 10.230.31.2:46204
x_forwarded_for:"-" vcap_request_id:1e58195a-cde6-4afd-7f03-43061c9ea91c
response_time:0.004927106 app_id:9784cd03-050d-4b74-9e90-5f17134a3f08
{"WAKAWAKA": "Space is the place", "X-Random-Header": "Once upon a midnight
dreary, while I pondered weak and weary"}

The reason for a stringified JSON is to make it easy to parse with
logstash or other loganalysis tools.

Before I spend time implementing, is this something you would merge?

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Allow gorouter to log random headers.

Simon Johansson <simon@...>
 

WAKAWAKA is just an example
But the difference is that WAKAWAKA is not platform specific whereas
X-Cf-Requestid is. If we want to move our app to Heroku, or a VM, or
whatewher we have platform specific implementation details in our
app(namely we rely on a header that is not there anymore). But that is not
the point of this thread.

The point is,

say we have an app that is fronted by a CDN, and the CDN sets the
X-Im-a-shark header with some value that we are interested to see in our
logs. The easiest way to achivie this without having to implement it into
our own apps is just to tell the Gorouter that it should append the value
of that header into the string that it logs so the event that flow via
Doppler and ultimately into cf logs/ELK will contain that value.

The reason why this would be such a valuable feature for us is that we dont
have to do anything. CF already provide the out of the box facility to give
us routing logs, so if we can piggy back on that for what we are interested
in we dont have to add libraries to our apps to log interesting headers on
the side.

On Wed, Jul 22, 2015 at 12:45 AM, Shannon Coen <scoen(a)pivotal.io> wrote:

I don't see the difference between WAKAWAKA and X-Cf-Requestid. Gorouter
would have to add some header with a uuid for the request. Your apps have
to have logic to pass this header on, so that a log search returns the
original request as well as subsequent requests between apps. Could you
please clarify?

Thank you,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Tue, Jul 21, 2015 at 11:09 AM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

Sure that header can be used. But then we are adding CF specific stuff
into the implementation of our services, which is something we want to
avoid at all costs.
Also Im not entirely sure if all the libraries we use for Zipkin supports
using custom headers.

All our public apps are fronted by different CDNs, which sets headers
that we might want to store for debugging, so we still need a way to pass
those trough into the log.

On Tuesday, 21 July 2015, Shannon Coen <scoen(a)pivotal.io> wrote:

Hello Simon,

The X-Cf-Requestid header already provides a uuid. Couldn't app "a" add
the value of X-Cf-Requestid to a header of your choosing? Call it WAKAWAKA
or X-Random-Header, but it doesn't need to be a platform standard. Wouldn't
a search for the value of X-Cf-Requestid then provide the desired results?

Thank you,
Shannon

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Tue, Jul 21, 2015 at 6:06 AM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

Howdie!

We have some devs who want to be able to trace a request troughout
their applications.

user -> a -> b -> c
|
|-> d -> e

When a user makes a request to "a" uuid is generated inside the app,
and the request to "b" from "a" will set a header(call it WAKAWAKA to
uuid), "b"'s call will passthrough WAKAWAKA to "c" and "d", "d" will
passthrough WAKAWAKA to "e". Etc.

We aggregate all RTR logs into ELK so it would be super helpful to them
to be able to filter on WAKAWAKA and get all the access logs(and app logs
aswell, they mostly use GELF so its easy for them to add whatewher field
they want) from the services involved.

I had a quick peek at the gorouter(
https://github.com/cloudfoundry/gorouter/blob/76668f5818ea8c089ff52a14fcdfbf703c8e8767/access_log/access_log_record.go#L40)
and it seems like this should be a simple PR.

1. To gorouter.yml add
passthrough_headers:
- WAKAWAKA
- X-Random-Header

2. In makeRecord at the bottom add something like(in psuedo)

data = {}
for header in passthrough_headers:
header_val = r.FormatRequestHeader("X-Forwarded-For")
if header_val:
passthrough_headers[header] = header_val

if data:
fmt.Fprintf(b, data.to_stringified_json())

That would yield a log line like
blurgh.dev.cf.private.domain.com - [21/07/2015:10:17:05 +0000] "GET
/statements?ascending=true&since=2015-06-30T14%3A10%3A03.078Z&skipStatementId=30a88204-0779-4385-9859-e4aabd30baf0
HTTP/1.1" 200 0 17 "-" "NING/1.0" 10.230.31.2:46204
x_forwarded_for:"-" vcap_request_id:1e58195a-cde6-4afd-7f03-43061c9ea91c
response_time:0.004927106 app_id:9784cd03-050d-4b74-9e90-5f17134a3f08
{"WAKAWAKA": "Space is the place", "X-Random-Header": "Once upon a midnight
dreary, while I pondered weak and weary"}

The reason for a stringified JSON is to make it easy to parse with
logstash or other loganalysis tools.

Before I spend time implementing, is this something you would merge?

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Assigning Role to Group

Zakharov Alexey <alexey.zakharov@...>
 

Hi guys!
Sorry if my question is newbie or it was discussed before.
I want to use LDAP for users authentication/authorisation. And I’ve
successfully bound CF to LDAP, and managed to configure uaac group mappings.
But then I realised, that there are no way to assign a Role to that group.
'cf set-org-role’ accepts only usernames as parameter, but not groups. I
think assigning Developer role to group is more flexible than assigning is
to every particular user.
Are you going to add this feature later? Or maybe there is an another way
to do group binding?
Have you looked at the `uaac` tool? I'm not quite sure I understand what
you're trying to do, but you can map an LDAP group DN to a UAA group with
`uaac`. Then if a user in that LDAP group logs in, they'll have that uaa
group. Is that what you're looking to do?

Ex:

uaac group map --name cloud_controller.admin "GROUP-DISTINGUISHED-NAME"

Or are you asking about mapping LDAP groups to CF org & space roles? i.e.
user in ldap group X is automatically given the OrgManager role in org Y.

Dan


Hi Dan!

Yes, as I’ve stated before, I’ve already managed to configure group mappings using ‘uaac group map’.

And now I want to bind group members to Organizations and Spaces. Is it possible to do?

---
Alexey Zakharov | CloudFoundry Team | Altoros
Tel: (617) 841-2121 ext. 5704 | Toll free: 855-ALTOROS
Fax: (866) 201-3646 | Skype: alexey.zakharov.a
www.altoros.com<http://www.altoros.com> | blog.altoros.com<http://blog.altoros.com> | twitter.com/altoros<http://twitter.com/altoros>


Re: Did anybody deploy a wiki as app to CF?

Dieu Cao <dcao@...>
 

Very cool!

On Tue, Jul 21, 2015 at 8:16 PM, Cornelia Davis <cdavis(a)pivotal.io> wrote:

What Josh said!! WOW!

On Tue, Jul 21, 2015 at 6:53 PM, Josh Long <starbuxman(a)gmail.com> wrote:

WOW!



--


Thanks,
Josh Long
the Spring Developer Advocate, Pivotal
http://joshlong.com || http://twitter.com/starbuxman ||
http://spring.io/team/jlong

On July 21, 2015 at 5:44:50 PM, Noburou TANIGUCHI (dev(a)nota.m001.jp)
wrote:

cloudfoundry.gr.jp, a Cloud Foundry user group in Japan, is currently
running
a project (or a campaign, more properly) called "Cloud Foundry 100-day
challenge" ("Cloud Foundry 100-nichi Gyou" in Japanese. "gyou" originally
means a kind of discipline or training in Buddhism). It is an activity to
make 100 open source apps run on Cloud Foundry, one app per day.

http://blog.cloudfoundry.gr.jp/search/label/%E7%99%BE%E6%97%A5%E8%A1%8C

We have done 33 apps until now. We are sorry the all articles are
presented
only in Japanese. Do they help your project, James?




--
View this message in context:
http://cf-dev.70369.x6.nabble.com/cf-dev-Did-anybody-deploy-a-wiki-as-app-to-CF-tp643p808.html
Sent from the CF Dev mailing list archive at Nabble.com.
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Did anybody deploy a wiki as app to CF?

Cornelia Davis <cdavis@...>
 

What Josh said!! WOW!

On Tue, Jul 21, 2015 at 6:53 PM, Josh Long <starbuxman(a)gmail.com> wrote:

WOW!



--


Thanks,
Josh Long
the Spring Developer Advocate, Pivotal
http://joshlong.com || http://twitter.com/starbuxman ||
http://spring.io/team/jlong

On July 21, 2015 at 5:44:50 PM, Noburou TANIGUCHI (dev(a)nota.m001.jp)
wrote:

cloudfoundry.gr.jp, a Cloud Foundry user group in Japan, is currently
running
a project (or a campaign, more properly) called "Cloud Foundry 100-day
challenge" ("Cloud Foundry 100-nichi Gyou" in Japanese. "gyou" originally
means a kind of discipline or training in Buddhism). It is an activity to
make 100 open source apps run on Cloud Foundry, one app per day.

http://blog.cloudfoundry.gr.jp/search/label/%E7%99%BE%E6%97%A5%E8%A1%8C

We have done 33 apps until now. We are sorry the all articles are
presented
only in Japanese. Do they help your project, James?




--
View this message in context:
http://cf-dev.70369.x6.nabble.com/cf-dev-Did-anybody-deploy-a-wiki-as-app-to-CF-tp643p808.html
Sent from the CF Dev mailing list archive at Nabble.com.
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

8561 - 8580 of 9425