Date   

Re: Notifications on ORG, SPACE and USER modifications

Matt Cowger
 

I think ETags is reasonable thought as well.

On Thu, Jul 23, 2015 at 4:39 PM, Benjamin Black <bblack(a)pivotal.io> wrote:

ETags and a 304 response are specifically intended for that purpose. I'd
recommend that over relying on Last-Modified.


b

On Thu, Jul 23, 2015 at 12:34 AM, Koper, Dies <diesk(a)fast.au.fujitsu.com>
wrote:

Or setting the Last-Modified HTTP response header accordingly, and
allow clients to use HTTP caching mechanisms (Last-Modified, etc.) to get
quick empty responses with the current APIs if no changes have been made?
(Or maybe this is already working so – haven’t checked).



Regards,

Dies Koper



*From:* cf-dev-bounces(a)lists.cloudfoundry.org [mailto:
cf-dev-bounces(a)lists.cloudfoundry.org] *On Behalf Of *Matt Cowger
*Sent:* Thursday, July 23, 2015 4:45 PM
*To:* Discussions about Cloud Foundry projects and the system overall.
*Subject:* Re: [cf-dev] Notifications on ORG, SPACE and USER
modifications



I've wanted something similar as well.



On a related note, having a CC API 'serial' number (for each object in CC
- apps, spaces, etc) that increments on every change relevant to that
object would be of value for detecting if something has changed.



On Thu, Jul 23, 2015 at 3:27 PM, Dieu Cao <dcao(a)pivotal.io> wrote:

There are a few different approaches to this and different concerns that
are possible.

The requests I've seen have been around wanting to be able to subscribe
to and filter the various events that cc currently generates so that other
behavior could be triggered.

We currently have events, app usage events, and service usage events.

Is it acceptable for the notifications to be lossy? Depends on the use
case but If so, then the firehose may be an acceptable approach.



The CAPI team is currently focusing on other work in the near term, such
as the v3 API and private brokers, but would be happy to collaborate on a
proposal.





On Wed, Jul 22, 2015 at 2:05 PM, Juan Pablo Genovese <
juanpgenovese(a)gmail.com> wrote:

My take:



CC should have callbacks on for each model create, update and delete
methods. Those callbacks will send a message to an MQ, which you can
subscribe to consume those messages.

This can be expanded to pretty much every event we need to track.

What do you think?



JP



2015-07-22 17:30 GMT-03:00 Matthias X Hub <matthias.hub(a)de.ibm.com>:

Hi,

we (=IBM) are also having the need and are currently investigating how to
solve this. We plan to work on a proposal to discuss this further with the
cf community. I'll keep you updated on that.

Regards,
Matthias



From: Mike Youngstrom <youngm(a)gmail.com>
To: "Discussions about Cloud Foundry projects and the system
overall." <cf-dev(a)lists.cloudfoundry.org>
Date: 22.07.2015 20:57
Subject: Re: [cf-dev] Notifications on ORG, SPACE and USER
modifications
Sent by: cf-dev-bounces(a)lists.cloudfoundry.org
------------------------------




We have the same need. Today we are polling the CC.

It would be nice for us also if we could get CC event notifications via
something like the firehose.

Mike

On Wed, Jul 22, 2015 at 10:23 AM, Juan Pablo Genovese <
juanpgenovese(a)gmail.com> wrote:
I mean, I know you can list those events thru the API, but I want
something that will react on an event instead of having to be constantly
polling for them.

2015-07-22 13:18 GMT-03:00 Juan Pablo Genovese <juanpgenovese(a)gmail.com>:

Sree,

thanks! Any pointers on how can I hook up to these audit events?

Thank you!

2015-07-22 13:12 GMT-03:00 Sree Tummidi <stummidi(a)pivotal.io>:
I believe there are audit events generated for all these actions which
can be captured and forwarded to an SIEM solution like splunk

Thanks,
Sree

Sent from my iPhone

On Jul 22, 2015, at 8:54 AM, Juan Pablo Genovese <juanpgenovese(a)gmail.com>
wrote:

Guys,

I need to somehow hook up into the Cloud Controller (CC) to capture ORG,
SPACE and USER deletion, insertion and update.

So far, I considered some approaches, such as forking the CC (the least
favorite) and modifying the code with some hooks, tapping into Nginx to
capture the requests, and using triggers in the database to capture each
event and send the necessary info to a service.

What do you think?
Any other idea you might have?

Thanks!

--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------

http://www.jpgenovese.com
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com



--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev





--

Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------

http://www.jpgenovese.com


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev





--

-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

--
-- Matt


Hello, just a test ,pls ignore

Xiao
 



Re: Allow gorouter to log random headers.

David Laing
 

James,

Yep - we're pretty much on the same page.

Only addition I would ask for is that the whitelist contain some "sensible"
defaults (eg, Trace-Id, Span-Id) that are switched on by default; since
then tight integration with tools like spring-cloud / buildpacks would work
out the box.

D

On 24 July 2015 at 16:26, James Bayer <jbayer(a)pivotal.io> wrote:

shannon,

from what i'm reading here about the use case, the main interest is that a
CF operator knows that their cf installation is more deeply integrated with
a specific log parsing solution for all/many apps on that platform that
choose to use it (whether than is ELK, Zipkin, etc). it does not sound like
it is a special case with lots of variation by many different app teams on
the cf installation. rather, it sounds like this is most likely to be used
as an installation-wide option to enhance the app developer / app
operations experience.

it seems like an operator configured whitelist set of headers that get
logged with the RTR message satisfies the current needs well and is
reasonable.

if we were to find that in the future lots of variation and different app
teams on the same CF wanted to have the RTR tier log many different headers
in the access log, then we could enhance the "log the whitelist of headers
to the access log" capability to be exposed to a limited number of headers
that each route could be configured by developers to specify that would
apply in addition to the operator configured logged headers. but it sounds
like that isn't needed right now.

simon/david, did i summarize this correctly?

On Fri, Jul 24, 2015 at 7:23 AM, Simon Johansson <simon(a)simonjohansson.com
wrote:
If I understand correctly, you're proposing that an operator of
CloudFoundry configure GoRouter, which is a multi-tenant, shared service,
with knowledge specific to one or a few applications

That is indeed the proposal. In my org we work closely with the different
development streams to provide a good out of the box experience by creating
certain opt-in features that makes their life easier. This would be one of
those features, if you pass certain headers in your requests thats
standardised across the entire online organisation you will be able to
query for them in Kibana.
I understand that this is not the case in all environments.

If GoRouter logged whatever headers were included in the request,
wouldn't this satisfy your requirements?

This would indeed satisfy my requests, but as David points out("However,
not having a whitelist of headers to log opens a possible DDOS attack
vector on the GoRouter") it might now be appropriate.

Based on what you've described, the persona is the app developer.
Not necessarily, we operators are also very interested in certain headers.

so control of what is logged should be in their hands.
If one via the API could set headers that should be logged for an org,
space or application that would be magical. The need for a list of
"must-log-all-these-headers" would still be there I think as not to have to
maintain a list of these standardised headers across different objects.


On Thu, Jul 23, 2015 at 10:51 PM, Shannon Coen <scoen(a)pivotal.io> wrote:

This is not something that would be merged, as originally proposed,
without additional investigation and discussion.

If I understand correctly, you're proposing that an operator of
CloudFoundry configure GoRouter, which is a multi-tenant, shared service,
with knowledge specific to one or a few applications. This should not be an
operator responsibility, nor should the solution be specific to one or a
few applications.

The goal is "the flexibility of being able to annotate our logs with
what we consider to be important for our debugging purposes." More
specifically you're requesting logging of headers. Do you have a preference?

If GoRouter logged whatever headers were included in the request,
wouldn't this satisfy your requirements? Doesn't GoRouter do this already?

I'm interested in solving your requirement generically for all
applications, and focussing the user experience on the correct persona.
Based on what you've described, the persona is the app developer, so
control of what is logged should be in their hands. I'm also not convinced
GoRouter should have any knowledge of headers specific to one application
or another.





Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Wed, Jul 22, 2015 at 3:05 AM, Alex Lomov <alexander.lomov(a)altoros.com
wrote:
Some time ago routing services were discussed on a CAB [1]. Here is a
description of this proposal.

Do you think that using such service will allow your developers to
cover this requirements?

[1]
http://www.activestate.com/blog/2015/02/cloud-foundry-advisory-board-meeting-2015-february
[2]
https://docs.google.com/document/d/1bGOQxiKkmaw6uaRWGd-sXpxL0Y28d3QihcluI15FiIA/edit#heading=h.8djffzes9pnb

On Jul 21, 2015, at 4:06 PM, Simon Johansson <simon(a)simonjohansson.com>
wrote:

Howdie!

We have some devs who want to be able to trace a request troughout
their applications.

user -> a -> b -> c
|
|-> d -> e

When a user makes a request to "a" uuid is generated inside the app,
and the request to "b" from "a" will set a header(call it WAKAWAKA to
uuid), "b"'s call will passthrough WAKAWAKA to "c" and "d", "d" will
passthrough WAKAWAKA to "e". Etc.

We aggregate all RTR logs into ELK so it would be super helpful to them
to be able to filter on WAKAWAKA and get all the access logs(and app logs
aswell, they mostly use GELF so its easy for them to add whatewher field
they want) from the services involved.

I had a quick peek at the gorouter(
https://github.com/cloudfoundry/gorouter/blob/76668f5818ea8c089ff52a14fcdfbf703c8e8767/access_log/access_log_record.go#L40)
and it seems like this should be a simple PR.

1. To gorouter.yml add
passthrough_headers:
- WAKAWAKA
- X-Random-Header

2. In makeRecord at the bottom add something like(in psuedo)

data = {}
for header in passthrough_headers:
header_val = r.FormatRequestHeader("X-Forwarded-For")
if header_val:
passthrough_headers[header] = header_val

if data:
fmt.Fprintf(b, data.to_stringified_json())

That would yield a log line like
blurgh.dev.cf.private.domain.com - [21/07/2015:10:17:05 +0000] "GET
/statements?ascending=true&since=2015-06-30T14%3A10%3A03.078Z&skipStatementId=30a88204-0779-4385-9859-e4aabd30baf0
HTTP/1.1" 200 0 17 "-" "NING/1.0" 10.230.31.2:46204
x_forwarded_for:"-" vcap_request_id:1e58195a-cde6-4afd-7f03-43061c9ea91c
response_time:0.004927106 app_id:9784cd03-050d-4b74-9e90-5f17134a3f08
{"WAKAWAKA": "Space is the place", "X-Random-Header": "Once upon a midnight
dreary, while I pondered weak and weary"}

The reason for a stringified JSON is to make it easy to parse with
logstash or other loganalysis tools.

Before I spend time implementing, is this something you would merge?
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Thank you,

James Bayer

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

--
David Laing
logsearch.io - build your own open source cloud logging cluster
http://davidlaing.com


Re: Allow gorouter to log random headers.

Simon Johansson <simon@...>
 

simon/david, did i summarize this correctly?
Spot on summarisation. :)

On Fri, Jul 24, 2015 at 5:26 PM, James Bayer <jbayer(a)pivotal.io> wrote:

shannon,

from what i'm reading here about the use case, the main interest is that a
CF operator knows that their cf installation is more deeply integrated with
a specific log parsing solution for all/many apps on that platform that
choose to use it (whether than is ELK, Zipkin, etc). it does not sound like
it is a special case with lots of variation by many different app teams on
the cf installation. rather, it sounds like this is most likely to be used
as an installation-wide option to enhance the app developer / app
operations experience.

it seems like an operator configured whitelist set of headers that get
logged with the RTR message satisfies the current needs well and is
reasonable.

if we were to find that in the future lots of variation and different app
teams on the same CF wanted to have the RTR tier log many different headers
in the access log, then we could enhance the "log the whitelist of headers
to the access log" capability to be exposed to a limited number of headers
that each route could be configured by developers to specify that would
apply in addition to the operator configured logged headers. but it sounds
like that isn't needed right now.

simon/david, did i summarize this correctly?

On Fri, Jul 24, 2015 at 7:23 AM, Simon Johansson <simon(a)simonjohansson.com
wrote:
If I understand correctly, you're proposing that an operator of
CloudFoundry configure GoRouter, which is a multi-tenant, shared service,
with knowledge specific to one or a few applications

That is indeed the proposal. In my org we work closely with the different
development streams to provide a good out of the box experience by creating
certain opt-in features that makes their life easier. This would be one of
those features, if you pass certain headers in your requests thats
standardised across the entire online organisation you will be able to
query for them in Kibana.
I understand that this is not the case in all environments.

If GoRouter logged whatever headers were included in the request,
wouldn't this satisfy your requirements?

This would indeed satisfy my requests, but as David points out("However,
not having a whitelist of headers to log opens a possible DDOS attack
vector on the GoRouter") it might now be appropriate.

Based on what you've described, the persona is the app developer.
Not necessarily, we operators are also very interested in certain headers.

so control of what is logged should be in their hands.
If one via the API could set headers that should be logged for an org,
space or application that would be magical. The need for a list of
"must-log-all-these-headers" would still be there I think as not to have to
maintain a list of these standardised headers across different objects.


On Thu, Jul 23, 2015 at 10:51 PM, Shannon Coen <scoen(a)pivotal.io> wrote:

This is not something that would be merged, as originally proposed,
without additional investigation and discussion.

If I understand correctly, you're proposing that an operator of
CloudFoundry configure GoRouter, which is a multi-tenant, shared service,
with knowledge specific to one or a few applications. This should not be an
operator responsibility, nor should the solution be specific to one or a
few applications.

The goal is "the flexibility of being able to annotate our logs with
what we consider to be important for our debugging purposes." More
specifically you're requesting logging of headers. Do you have a preference?

If GoRouter logged whatever headers were included in the request,
wouldn't this satisfy your requirements? Doesn't GoRouter do this already?

I'm interested in solving your requirement generically for all
applications, and focussing the user experience on the correct persona.
Based on what you've described, the persona is the app developer, so
control of what is logged should be in their hands. I'm also not convinced
GoRouter should have any knowledge of headers specific to one application
or another.





Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Wed, Jul 22, 2015 at 3:05 AM, Alex Lomov <alexander.lomov(a)altoros.com
wrote:
Some time ago routing services were discussed on a CAB [1]. Here is a
description of this proposal.

Do you think that using such service will allow your developers to
cover this requirements?

[1]
http://www.activestate.com/blog/2015/02/cloud-foundry-advisory-board-meeting-2015-february
[2]
https://docs.google.com/document/d/1bGOQxiKkmaw6uaRWGd-sXpxL0Y28d3QihcluI15FiIA/edit#heading=h.8djffzes9pnb

On Jul 21, 2015, at 4:06 PM, Simon Johansson <simon(a)simonjohansson.com>
wrote:

Howdie!

We have some devs who want to be able to trace a request troughout
their applications.

user -> a -> b -> c
|
|-> d -> e

When a user makes a request to "a" uuid is generated inside the app,
and the request to "b" from "a" will set a header(call it WAKAWAKA to
uuid), "b"'s call will passthrough WAKAWAKA to "c" and "d", "d" will
passthrough WAKAWAKA to "e". Etc.

We aggregate all RTR logs into ELK so it would be super helpful to them
to be able to filter on WAKAWAKA and get all the access logs(and app logs
aswell, they mostly use GELF so its easy for them to add whatewher field
they want) from the services involved.

I had a quick peek at the gorouter(
https://github.com/cloudfoundry/gorouter/blob/76668f5818ea8c089ff52a14fcdfbf703c8e8767/access_log/access_log_record.go#L40)
and it seems like this should be a simple PR.

1. To gorouter.yml add
passthrough_headers:
- WAKAWAKA
- X-Random-Header

2. In makeRecord at the bottom add something like(in psuedo)

data = {}
for header in passthrough_headers:
header_val = r.FormatRequestHeader("X-Forwarded-For")
if header_val:
passthrough_headers[header] = header_val

if data:
fmt.Fprintf(b, data.to_stringified_json())

That would yield a log line like
blurgh.dev.cf.private.domain.com - [21/07/2015:10:17:05 +0000] "GET
/statements?ascending=true&since=2015-06-30T14%3A10%3A03.078Z&skipStatementId=30a88204-0779-4385-9859-e4aabd30baf0
HTTP/1.1" 200 0 17 "-" "NING/1.0" 10.230.31.2:46204
x_forwarded_for:"-" vcap_request_id:1e58195a-cde6-4afd-7f03-43061c9ea91c
response_time:0.004927106 app_id:9784cd03-050d-4b74-9e90-5f17134a3f08
{"WAKAWAKA": "Space is the place", "X-Random-Header": "Once upon a midnight
dreary, while I pondered weak and weary"}

The reason for a stringified JSON is to make it easy to parse with
logstash or other loganalysis tools.

Before I spend time implementing, is this something you would merge?
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Thank you,

James Bayer

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Allow gorouter to log random headers.

James Bayer
 

shannon,

from what i'm reading here about the use case, the main interest is that a
CF operator knows that their cf installation is more deeply integrated with
a specific log parsing solution for all/many apps on that platform that
choose to use it (whether than is ELK, Zipkin, etc). it does not sound like
it is a special case with lots of variation by many different app teams on
the cf installation. rather, it sounds like this is most likely to be used
as an installation-wide option to enhance the app developer / app
operations experience.

it seems like an operator configured whitelist set of headers that get
logged with the RTR message satisfies the current needs well and is
reasonable.

if we were to find that in the future lots of variation and different app
teams on the same CF wanted to have the RTR tier log many different headers
in the access log, then we could enhance the "log the whitelist of headers
to the access log" capability to be exposed to a limited number of headers
that each route could be configured by developers to specify that would
apply in addition to the operator configured logged headers. but it sounds
like that isn't needed right now.

simon/david, did i summarize this correctly?

On Fri, Jul 24, 2015 at 7:23 AM, Simon Johansson <simon(a)simonjohansson.com>
wrote:

If I understand correctly, you're proposing that an operator of
CloudFoundry configure GoRouter, which is a multi-tenant, shared service,
with knowledge specific to one or a few applications

That is indeed the proposal. In my org we work closely with the different
development streams to provide a good out of the box experience by creating
certain opt-in features that makes their life easier. This would be one of
those features, if you pass certain headers in your requests thats
standardised across the entire online organisation you will be able to
query for them in Kibana.
I understand that this is not the case in all environments.

If GoRouter logged whatever headers were included in the request,
wouldn't this satisfy your requirements?

This would indeed satisfy my requests, but as David points out("However,
not having a whitelist of headers to log opens a possible DDOS attack
vector on the GoRouter") it might now be appropriate.

Based on what you've described, the persona is the app developer.
Not necessarily, we operators are also very interested in certain headers.

so control of what is logged should be in their hands.
If one via the API could set headers that should be logged for an org,
space or application that would be magical. The need for a list of
"must-log-all-these-headers" would still be there I think as not to have to
maintain a list of these standardised headers across different objects.


On Thu, Jul 23, 2015 at 10:51 PM, Shannon Coen <scoen(a)pivotal.io> wrote:

This is not something that would be merged, as originally proposed,
without additional investigation and discussion.

If I understand correctly, you're proposing that an operator of
CloudFoundry configure GoRouter, which is a multi-tenant, shared service,
with knowledge specific to one or a few applications. This should not be an
operator responsibility, nor should the solution be specific to one or a
few applications.

The goal is "the flexibility of being able to annotate our logs with
what we consider to be important for our debugging purposes." More
specifically you're requesting logging of headers. Do you have a preference?

If GoRouter logged whatever headers were included in the request,
wouldn't this satisfy your requirements? Doesn't GoRouter do this already?

I'm interested in solving your requirement generically for all
applications, and focussing the user experience on the correct persona.
Based on what you've described, the persona is the app developer, so
control of what is logged should be in their hands. I'm also not convinced
GoRouter should have any knowledge of headers specific to one application
or another.





Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Wed, Jul 22, 2015 at 3:05 AM, Alex Lomov <alexander.lomov(a)altoros.com>
wrote:

Some time ago routing services were discussed on a CAB [1]. Here is a
description of this proposal.

Do you think that using such service will allow your developers to cover
this requirements?

[1]
http://www.activestate.com/blog/2015/02/cloud-foundry-advisory-board-meeting-2015-february
[2]
https://docs.google.com/document/d/1bGOQxiKkmaw6uaRWGd-sXpxL0Y28d3QihcluI15FiIA/edit#heading=h.8djffzes9pnb

On Jul 21, 2015, at 4:06 PM, Simon Johansson <simon(a)simonjohansson.com>
wrote:

Howdie!

We have some devs who want to be able to trace a request troughout their
applications.

user -> a -> b -> c
|
|-> d -> e

When a user makes a request to "a" uuid is generated inside the app, and
the request to "b" from "a" will set a header(call it WAKAWAKA to uuid),
"b"'s call will passthrough WAKAWAKA to "c" and "d", "d" will passthrough
WAKAWAKA to "e". Etc.

We aggregate all RTR logs into ELK so it would be super helpful to them
to be able to filter on WAKAWAKA and get all the access logs(and app logs
aswell, they mostly use GELF so its easy for them to add whatewher field
they want) from the services involved.

I had a quick peek at the gorouter(
https://github.com/cloudfoundry/gorouter/blob/76668f5818ea8c089ff52a14fcdfbf703c8e8767/access_log/access_log_record.go#L40)
and it seems like this should be a simple PR.

1. To gorouter.yml add
passthrough_headers:
- WAKAWAKA
- X-Random-Header

2. In makeRecord at the bottom add something like(in psuedo)

data = {}
for header in passthrough_headers:
header_val = r.FormatRequestHeader("X-Forwarded-For")
if header_val:
passthrough_headers[header] = header_val

if data:
fmt.Fprintf(b, data.to_stringified_json())

That would yield a log line like
blurgh.dev.cf.private.domain.com - [21/07/2015:10:17:05 +0000] "GET
/statements?ascending=true&since=2015-06-30T14%3A10%3A03.078Z&skipStatementId=30a88204-0779-4385-9859-e4aabd30baf0
HTTP/1.1" 200 0 17 "-" "NING/1.0" 10.230.31.2:46204 x_forwarded_for:"-"
vcap_request_id:1e58195a-cde6-4afd-7f03-43061c9ea91c
response_time:0.004927106 app_id:9784cd03-050d-4b74-9e90-5f17134a3f08
{"WAKAWAKA": "Space is the place", "X-Random-Header": "Once upon a midnight
dreary, while I pondered weak and weary"}

The reason for a stringified JSON is to make it easy to parse with
logstash or other loganalysis tools.

Before I spend time implementing, is this something you would merge?
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Thank you,

James Bayer


Re: How cloud foundry deployed by bosh-lite manage DEA

Daniel Mikusa
 

On Fri, Jul 24, 2015 at 11:17 AM, GuopingZhang <goodier_cn(a)hotmail.com>
wrote:

Dan, thanks a lot for your explanation!
So you mean, warden container can have its own sub-warden container ?
Yes. That's my understanding of it.

Dan



Guoping
------------------------------
Date: Fri, 24 Jul 2015 09:13:13 -0400
From: dmikusa(a)pivotal.io
To: cf-dev(a)lists.cloudfoundry.org
Subject: Re: [cf-dev] How cloud foundry deployed by bosh-lite manage DEA

On Fri, Jul 24, 2015 at 8:35 AM, GuopingZhang <goodier_cn(a)hotmail.com>
wrote:


I read that bosh-lite use warden as container for cloud foundry components.
I deployed two java spring apps into the local cloud foundry.

Below is my VMs (warden container), which (two) warden container is
running my 2 apps?

Or the apps are actually running as java process in my virtualbox of
unbuntu (which contains all below containers), without running inside a
warden container?


It's a little different because you're using bosh-lite. In that case, you
have one VM (probably running in Virtualbox) and that VM has some
containers on it. Those containers run the various components of CF.
Inside the DEA container (labeled "runner_z1" below), you'll find your apps
running each in it's own sub-container.

Dan





+------------------------------------+---------+---------------+--------------+
| Job/index | State | Resource Pool | IPs
|

+------------------------------------+---------+---------------+--------------+
| api_z1/0 | running | large_z1 |
10.244.0.134 |
| doppler_z1/0 | running | medium_z1 |
10.244.0.142 |
| etcd_z1/0 | running | medium_z1 |
10.244.0.42 |
| ha_proxy_z1/0 | running | router_z1 |
10.244.0.34 |
| hm9000_z1/0 | running | medium_z1 |
10.244.0.138 |
| loggregator_trafficcontroller_z1/0 | running | small_z1 |
10.244.0.146 |
| nats_z1/0 | running | medium_z1 |
10.244.0.6 |
| postgres_z1/0 | running | medium_z1 |
10.244.0.30 |
| router_z1/0 | running | router_z1 |
10.244.0.22 |
| runner_z1/0 | running | runner_z1 |
10.244.0.26 |

| uaa_z1/0 | running | medium_z1 |
10.244.0.130 |

+------------------------------------+---------+---------------+--------------+


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________ cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: How cloud foundry deployed by bosh-lite manage DEA

GuopingZhang
 

Dan, thanks a lot for your explanation!So you mean, warden container can have its own sub-warden container ?

Guoping
Date: Fri, 24 Jul 2015 09:13:13 -0400
From: dmikusa(a)pivotal.io
To: cf-dev(a)lists.cloudfoundry.org
Subject: Re: [cf-dev] How cloud foundry deployed by bosh-lite manage DEA

On Fri, Jul 24, 2015 at 8:35 AM, GuopingZhang <goodier_cn(a)hotmail.com> wrote:




I read that bosh-lite use warden as container for cloud foundry components.I deployed two java spring apps into the local cloud foundry.
Below is my VMs (warden container), which (two) warden container is running my 2 apps?
Or the apps are actually running as java process in my virtualbox of unbuntu (which contains all below containers), without running inside a warden container?
It's a little different because you're using bosh-lite. In that case, you have one VM (probably running in Virtualbox) and that VM has some containers on it. Those containers run the various components of CF. Inside the DEA container (labeled "runner_z1" below), you'll find your apps running each in it's own sub-container.
Dan

+------------------------------------+---------+---------------+--------------+| Job/index | State | Resource Pool | IPs |+------------------------------------+---------+---------------+--------------+| api_z1/0 | running | large_z1 | 10.244.0.134 || doppler_z1/0 | running | medium_z1 | 10.244.0.142 || etcd_z1/0 | running | medium_z1 | 10.244.0.42 || ha_proxy_z1/0 | running | router_z1 | 10.244.0.34 || hm9000_z1/0 | running | medium_z1 | 10.244.0.138 || loggregator_trafficcontroller_z1/0 | running | small_z1 | 10.244.0.146 || nats_z1/0 | running | medium_z1 | 10.244.0.6 || postgres_z1/0 | running | medium_z1 | 10.244.0.30 || router_z1/0 | running | router_z1 | 10.244.0.22 || runner_z1/0 | running | runner_z1 | 10.244.0.26 | | uaa_z1/0 | running | medium_z1 | 10.244.0.130 |+------------------------------------+---------+---------------+--------------+


_______________________________________________

cf-dev mailing list

cf-dev(a)lists.cloudfoundry.org

https://lists.cloudfoundry.org/mailman/listinfo/cf-dev





_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: UAA: How to set client_credentials token grant type to not expire

Filip Hanik
 

public static final intMAX_VALUE
<http://docs.oracle.com/javase/7/docs/api/java/lang/Integer.html#MAX_VALUE>
2147483647
that's the largest value you can set. However, I just realized, that if
you set this value, the UAA will get an overflow and the int will become
negative.


so set the value to

315 569 260


that's a 10 year expiration. the computer that retrieved that token won't
be around in 10 years so it's a safe number.

On Friday, July 24, 2015, Kayode Odeyemi <dreyemi(a)gmail.com> wrote:

Thanks Filip.

However, everytime I set that (for example, I set it to 2542838400), I get
this:

"The request sent by the client was syntactically incorrect."

That's December 31st 2050 when decoded. Is it possible that, that number
is too large?

On Thu, Jul 23, 2015 at 7:09 PM, Filip Hanik <fhanik(a)pivotal.io
<javascript:_e(%7B%7D,'cvml','fhanik(a)pivotal.io');>> wrote:



https://github.com/cloudfoundry/uaa/blob/master/docs/UAA-APIs.rst#register-client-post-oauthclients

access_token_validity - int Optional Value in seconds for how long an
access token is valid for

Set this field to a very large value, like
http://docs.oracle.com/javase/7/docs/api/constant-values.html#java.lang.Integer.MAX_VALUE

On Thu, Jul 23, 2015 at 11:05 AM, Kayode Odeyemi <dreyemi(a)gmail.com
<javascript:_e(%7B%7D,'cvml','dreyemi(a)gmail.com');>> wrote:

Hi,

I have some trusted clients set up to use client_credentials token
grant. I'll like to set their tokens not to expire.

How do I achieve this?


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
<javascript:_e(%7B%7D,'cvml','cf-dev(a)lists.cloudfoundry.org');>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
<javascript:_e(%7B%7D,'cvml','cf-dev(a)lists.cloudfoundry.org');>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Allow gorouter to log random headers.

Simon Johansson <simon@...>
 

If I understand correctly, you're proposing that an operator of
CloudFoundry configure GoRouter, which is a multi-tenant, shared service,
with knowledge specific to one or a few applications

That is indeed the proposal. In my org we work closely with the different
development streams to provide a good out of the box experience by creating
certain opt-in features that makes their life easier. This would be one of
those features, if you pass certain headers in your requests thats
standardised across the entire online organisation you will be able to
query for them in Kibana.
I understand that this is not the case in all environments.

If GoRouter logged whatever headers were included in the request,
wouldn't this satisfy your requirements?

This would indeed satisfy my requests, but as David points out("However,
not having a whitelist of headers to log opens a possible DDOS attack
vector on the GoRouter") it might now be appropriate.

Based on what you've described, the persona is the app developer.
Not necessarily, we operators are also very interested in certain headers.

so control of what is logged should be in their hands.
If one via the API could set headers that should be logged for an org,
space or application that would be magical. The need for a list of
"must-log-all-these-headers" would still be there I think as not to have to
maintain a list of these standardised headers across different objects.


On Thu, Jul 23, 2015 at 10:51 PM, Shannon Coen <scoen(a)pivotal.io> wrote:

This is not something that would be merged, as originally proposed,
without additional investigation and discussion.

If I understand correctly, you're proposing that an operator of
CloudFoundry configure GoRouter, which is a multi-tenant, shared service,
with knowledge specific to one or a few applications. This should not be an
operator responsibility, nor should the solution be specific to one or a
few applications.

The goal is "the flexibility of being able to annotate our logs with what
we consider to be important for our debugging purposes." More specifically
you're requesting logging of headers. Do you have a preference?

If GoRouter logged whatever headers were included in the request, wouldn't
this satisfy your requirements? Doesn't GoRouter do this already?

I'm interested in solving your requirement generically for all
applications, and focussing the user experience on the correct persona.
Based on what you've described, the persona is the app developer, so
control of what is logged should be in their hands. I'm also not convinced
GoRouter should have any knowledge of headers specific to one application
or another.





Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Wed, Jul 22, 2015 at 3:05 AM, Alex Lomov <alexander.lomov(a)altoros.com>
wrote:

Some time ago routing services were discussed on a CAB [1]. Here is a
description of this proposal.

Do you think that using such service will allow your developers to cover
this requirements?

[1]
http://www.activestate.com/blog/2015/02/cloud-foundry-advisory-board-meeting-2015-february
[2]
https://docs.google.com/document/d/1bGOQxiKkmaw6uaRWGd-sXpxL0Y28d3QihcluI15FiIA/edit#heading=h.8djffzes9pnb

On Jul 21, 2015, at 4:06 PM, Simon Johansson <simon(a)simonjohansson.com>
wrote:

Howdie!

We have some devs who want to be able to trace a request troughout their
applications.

user -> a -> b -> c
|
|-> d -> e

When a user makes a request to "a" uuid is generated inside the app, and
the request to "b" from "a" will set a header(call it WAKAWAKA to uuid),
"b"'s call will passthrough WAKAWAKA to "c" and "d", "d" will passthrough
WAKAWAKA to "e". Etc.

We aggregate all RTR logs into ELK so it would be super helpful to them
to be able to filter on WAKAWAKA and get all the access logs(and app logs
aswell, they mostly use GELF so its easy for them to add whatewher field
they want) from the services involved.

I had a quick peek at the gorouter(
https://github.com/cloudfoundry/gorouter/blob/76668f5818ea8c089ff52a14fcdfbf703c8e8767/access_log/access_log_record.go#L40)
and it seems like this should be a simple PR.

1. To gorouter.yml add
passthrough_headers:
- WAKAWAKA
- X-Random-Header

2. In makeRecord at the bottom add something like(in psuedo)

data = {}
for header in passthrough_headers:
header_val = r.FormatRequestHeader("X-Forwarded-For")
if header_val:
passthrough_headers[header] = header_val

if data:
fmt.Fprintf(b, data.to_stringified_json())

That would yield a log line like
blurgh.dev.cf.private.domain.com - [21/07/2015:10:17:05 +0000] "GET
/statements?ascending=true&since=2015-06-30T14%3A10%3A03.078Z&skipStatementId=30a88204-0779-4385-9859-e4aabd30baf0
HTTP/1.1" 200 0 17 "-" "NING/1.0" 10.230.31.2:46204 x_forwarded_for:"-"
vcap_request_id:1e58195a-cde6-4afd-7f03-43061c9ea91c
response_time:0.004927106 app_id:9784cd03-050d-4b74-9e90-5f17134a3f08
{"WAKAWAKA": "Space is the place", "X-Random-Header": "Once upon a midnight
dreary, while I pondered weak and weary"}

The reason for a stringified JSON is to make it easy to parse with
logstash or other loganalysis tools.

Before I spend time implementing, is this something you would merge?
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Did anybody deploy a wiki as app to CF?

James Bayer
 

what an exciting effort!

lots of things work with CF, it's just a matter of curating some good
information.

maybe i can get Yudai to pair with me on some translations!

On Thu, Jul 23, 2015 at 10:41 PM, Noburou TANIGUCHI <dev(a)nota.m001.jp>
wrote:

Thank you, all.

I am very glad to see such many positive reactions.
We should consider about English version of the articles.




--
View this message in context:
http://cf-dev.70369.x6.nabble.com/cf-dev-Did-anybody-deploy-a-wiki-as-app-to-CF-tp643p873.html
Sent from the CF Dev mailing list archive at Nabble.com.
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Thank you,

James Bayer


Re: UAA: How to set client_credentials token grant type to not expire

Paul Bakare
 

Thanks Filip.

However, everytime I set that (for example, I set it to 2542838400), I get
this:

"The request sent by the client was syntactically incorrect."

That's December 31st 2050 when decoded. Is it possible that, that number is
too large?

On Thu, Jul 23, 2015 at 7:09 PM, Filip Hanik <fhanik(a)pivotal.io> wrote:



https://github.com/cloudfoundry/uaa/blob/master/docs/UAA-APIs.rst#register-client-post-oauthclients

access_token_validity - int Optional Value in seconds for how long an
access token is valid for

Set this field to a very large value, like
http://docs.oracle.com/javase/7/docs/api/constant-values.html#java.lang.Integer.MAX_VALUE

On Thu, Jul 23, 2015 at 11:05 AM, Kayode Odeyemi <dreyemi(a)gmail.com>
wrote:

Hi,

I have some trusted clients set up to use client_credentials token grant.
I'll like to set their tokens not to expire.

How do I achieve this?


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: How cloud foundry deployed by bosh-lite manage DEA

Daniel Mikusa
 

On Fri, Jul 24, 2015 at 8:35 AM, GuopingZhang <goodier_cn(a)hotmail.com>
wrote:


I read that bosh-lite use warden as container for cloud foundry components.
I deployed two java spring apps into the local cloud foundry.

Below is my VMs (warden container), which (two) warden container is
running my 2 apps?

Or the apps are actually running as java process in my virtualbox of
unbuntu (which contains all below containers), without running inside a
warden container?
It's a little different because you're using bosh-lite. In that case, you
have one VM (probably running in Virtualbox) and that VM has some
containers on it. Those containers run the various components of CF.
Inside the DEA container (labeled "runner_z1" below), you'll find your apps
running each in it's own sub-container.

Dan





+------------------------------------+---------+---------------+--------------+
| Job/index | State | Resource Pool | IPs
|

+------------------------------------+---------+---------------+--------------+
| api_z1/0 | running | large_z1 |
10.244.0.134 |
| doppler_z1/0 | running | medium_z1 |
10.244.0.142 |
| etcd_z1/0 | running | medium_z1 |
10.244.0.42 |
| ha_proxy_z1/0 | running | router_z1 |
10.244.0.34 |
| hm9000_z1/0 | running | medium_z1 |
10.244.0.138 |
| loggregator_trafficcontroller_z1/0 | running | small_z1 |
10.244.0.146 |
| nats_z1/0 | running | medium_z1 |
10.244.0.6 |
| postgres_z1/0 | running | medium_z1 |
10.244.0.30 |
| router_z1/0 | running | router_z1 |
10.244.0.22 |
| runner_z1/0 | running | runner_z1 |
10.244.0.26 |
| uaa_z1/0 | running | medium_z1 |
10.244.0.130 |

+------------------------------------+---------+---------------+--------------+


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: How to update blobs in blob.cfblob.com ?

Alexander Lomov <alexander.lomov@...>
 

Thank you for the answer.

I like the idea with pathing existing sources. Going to make another PR soon.

Have a good day,
Alex L.

On Jul 24, 2015, at 3:36 PM, Matthew Sykes <matthew.sykes(a)gmail.com<mailto:matthew.sykes(a)gmail.com>> wrote:

Hi. This isn't what I expected. You've basically changed the bucket configuration to point to your own and uploaded the blobs to that bucket. That's why bosh is telling you there's nothing to upload.

There's also a general expectation that blobs are pristine artifacts from a well known distribution point. In this case, that's not true. You've used a script to modify some content and there's no way for someone to know that without looking in detail so it's going to be tough to trust what's there.

Regardless, given what your scripts are doing, you should be able to generate patches for each of the packages, add them as source artifacts, and update the packaging scripts to apply the patches before the build. That's probably how this should be done to keep things sane.

On Fri, Jul 24, 2015 at 4:37 AM, Lomov Alexander <alexander.lomov(a)altoros.com<mailto:alexander.lomov(a)altoros.com>> wrote:
Hi, Matthew.

After I made `bosh upload blobs`, `bosh blobs` command tells me `No blobs to upload`.

Still I created a pull request where you can figure out what blobs are changed from `config/blobs.yml` file [1].

Could you tell if it is okey?

[1] https://github.com/cloudfoundry/bosh/pull/892

On Jul 1, 2015, at 5:38 PM, Matthew Sykes <matthew.sykes(a)gmail.com<mailto:matthew.sykes(a)gmail.com>> wrote:

Since you won't be able to upload the blobs to the cf-release bucket, I'd suggest you capture the output of `bosh blobs` in your pull request. That command should enumerate all of the new blobs and their sizes.

For each entry that's there, point to a publicly available URL and a hash that can be used to verify it.

When the PR is reviewed, if things look good, the pair will likely pull the blobs down to evaluate them and test the overall function.

On Wed, Jul 1, 2015 at 6:57 AM, Alexander Lomov <alexander.lomov(a)altoros.com<mailto:alexander.lomov(a)altoros.com>> wrote:
Hi, all.

I work on adding support of Power architecture to CF. During the work I needed to update not only cf-release, but existing blobs (postgresql, mysql client and etc.). I wonder how I can make PR to cf-release project with updated blobs.

I couldn't find any clue in the contributing guild [1] , so I've decided to write here.

[1] https://github.com/cloudfoundry/cf-release/blob/master/CONTRIBUTING.md

Thank you,
Alex L.

------------------------
Alex Lomov
Altoros — Cloud Foundry deployment, training and integration
Twitter: @code1n<https://twitter.com/code1n> GitHub: @allomov<https://gist.github.com/allomov>

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




--
Matthew Sykes
matthew.sykes(a)gmail.com<mailto:matthew.sykes(a)gmail.com>
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




--
Matthew Sykes
matthew.sykes(a)gmail.com<mailto:matthew.sykes(a)gmail.com>
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: How to update blobs in blob.cfblob.com ?

Matthew Sykes <matthew.sykes@...>
 

Hi. This isn't what I expected. You've basically changed the bucket
configuration to point to your own and uploaded the blobs to that bucket.
That's why bosh is telling you there's nothing to upload.

There's also a general expectation that blobs are pristine artifacts from a
well known distribution point. In this case, that's not true. You've used a
script to modify some content and there's no way for someone to know that
without looking in detail so it's going to be tough to trust what's there.

Regardless, given what your scripts are doing, you should be able to
generate patches for each of the packages, add them as source artifacts,
and update the packaging scripts to apply the patches before the build.
That's probably how this should be done to keep things sane.

On Fri, Jul 24, 2015 at 4:37 AM, Lomov Alexander <
alexander.lomov(a)altoros.com> wrote:

Hi, Matthew.

After I made `bosh upload blobs`, `bosh blobs` command tells me `No
blobs to upload`.

Still I created a pull request where you can figure out what blobs are
changed from `config/blobs.yml` file [1].

Could you tell if it is okey?

[1] https://github.com/cloudfoundry/bosh/pull/892

On Jul 1, 2015, at 5:38 PM, Matthew Sykes <matthew.sykes(a)gmail.com>
wrote:

Since you won't be able to upload the blobs to the cf-release bucket,
I'd suggest you capture the output of `bosh blobs` in your pull request.
That command should enumerate all of the new blobs and their sizes.

For each entry that's there, point to a publicly available URL and a
hash that can be used to verify it.

When the PR is reviewed, if things look good, the pair will likely pull
the blobs down to evaluate them and test the overall function.

On Wed, Jul 1, 2015 at 6:57 AM, Alexander Lomov <
alexander.lomov(a)altoros.com> wrote:

Hi, all.

I work on adding support of Power architecture to CF. During the work I
needed to update not only cf-release, but existing blobs (postgresql, mysql
client and etc.). I wonder how I can make PR to cf-release project with
updated blobs.

I couldn't find any clue in the contributing guild [1] , so I've decided
to write here.

[1]
https://github.com/cloudfoundry/cf-release/blob/master/CONTRIBUTING.md

Thank you,
Alex L.

------------------------
Alex Lomov
*Altoros* — Cloud Foundry deployment, training and integration
*Twitter:* @code1n <https://twitter.com/code1n> *GitHub:* @allomov
<https://gist.github.com/allomov>

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Matthew Sykes
matthew.sykes(a)gmail.com
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Matthew Sykes
matthew.sykes(a)gmail.com


How cloud foundry deployed by bosh-lite manage DEA

GuopingZhang
 

I read that bosh-lite use warden as container for cloud foundry components.I deployed two java spring apps into the local cloud foundry.
Below is my VMs (warden container), which (two) warden container is running my 2 apps?
Or the apps are actually running as java process in my virtualbox of unbuntu (which contains all below containers), without running inside a warden container?
+------------------------------------+---------+---------------+--------------+| Job/index | State | Resource Pool | IPs |+------------------------------------+---------+---------------+--------------+| api_z1/0 | running | large_z1 | 10.244.0.134 || doppler_z1/0 | running | medium_z1 | 10.244.0.142 || etcd_z1/0 | running | medium_z1 | 10.244.0.42 || ha_proxy_z1/0 | running | router_z1 | 10.244.0.34 || hm9000_z1/0 | running | medium_z1 | 10.244.0.138 || loggregator_trafficcontroller_z1/0 | running | small_z1 | 10.244.0.146 || nats_z1/0 | running | medium_z1 | 10.244.0.6 || postgres_z1/0 | running | medium_z1 | 10.244.0.30 || router_z1/0 | running | router_z1 | 10.244.0.22 || runner_z1/0 | running | runner_z1 | 10.244.0.26 || uaa_z1/0 | running | medium_z1 | 10.244.0.130 |+------------------------------------+---------+---------------+--------------+


Re: Allow gorouter to log random headers.

David Laing
 

Shannon, Simon,

As the lead of the logsearch.io (ELK) project; I'm also interested in having
GoRouter log additional headers. Specifically Trace-Id and Span-Id
generated by the spring-cloud-sleuth project
(https://github.com/spring-cloud-incubator/spring-cloud-sleuth ).

If GoRouter logged whatever headers were included in the request, wouldn't
this satisfy your requirements?
This would certainly satisfy my requirements, and I think Simon's too.

However, not having a whitelist of headers to log opens a possible DDOS
attack vector on the GoRouter, so I think having a operator configureable
whitelist (with some sensible defaults like Trace-Id and Span-Id) is the
right approach.

Doesn't GoRouter do this already?
I don't think so.

Specifically, sending the following curl to an app hosted on PWS:

curl --header "Trace-Id: 1c884728-466c-4ba3-8caa-a02a780c6d56"
http://www.birdsangola.org/

Gives the following [RTR] log from loggregator:

Fri Jul 24 2015 13:10:52 GMT+0100 (BST) [RTR] OUT www.birdsangola.org -
[24/07/2015:12:10:52 +0000] "GET / HTTP/1.1" 200 0 7772 "-" "curl/7.30.0"
10.10.2.185:46765 x_forwarded_for:"92.40.249.226"
vcap_request_id:3a33d5f6-dc11-42c4-61c7-32a1a2557200
response_time:0.001380276 app_id:0c34cc9f-45a8-440e-b876-b0cde564fbe3

It doesn't look like the extra Trace-Id header has been passed through to
the loggregator [RTR] log.

I'd be happy to work with Simon to contribute to a PR that implements the
"above whitelist of headers to log" feature.

Thoughts?

:D



--
View this message in context: http://cf-dev.70369.x6.nabble.com/cf-dev-Allow-gorouter-to-log-random-headers-tp800p877.html
Sent from the CF Dev mailing list archive at Nabble.com.


Re: How to update blobs in blob.cfblob.com ?

Alexander Lomov <alexander.lomov@...>
 

Hi, Matthew.

After I made `bosh upload blobs`, `bosh blobs` command tells me `No blobs to upload`.

Still I created a pull request where you can figure out what blobs are changed from `config/blobs.yml` file [1].

Could you tell if it is okey?

[1] https://github.com/cloudfoundry/bosh/pull/892

On Jul 1, 2015, at 5:38 PM, Matthew Sykes <matthew.sykes(a)gmail.com<mailto:matthew.sykes(a)gmail.com>> wrote:

Since you won't be able to upload the blobs to the cf-release bucket, I'd suggest you capture the output of `bosh blobs` in your pull request. That command should enumerate all of the new blobs and their sizes.

For each entry that's there, point to a publicly available URL and a hash that can be used to verify it.

When the PR is reviewed, if things look good, the pair will likely pull the blobs down to evaluate them and test the overall function.

On Wed, Jul 1, 2015 at 6:57 AM, Alexander Lomov <alexander.lomov(a)altoros.com<mailto:alexander.lomov(a)altoros.com>> wrote:
Hi, all.

I work on adding support of Power architecture to CF. During the work I needed to update not only cf-release, but existing blobs (postgresql, mysql client and etc.). I wonder how I can make PR to cf-release project with updated blobs.

I couldn't find any clue in the contributing guild [1] , so I've decided to write here.

[1] https://github.com/cloudfoundry/cf-release/blob/master/CONTRIBUTING.md

Thank you,
Alex L.

------------------------
Alex Lomov
Altoros — Cloud Foundry deployment, training and integration
Twitter: @code1n<https://twitter.com/code1n> GitHub: @allomov<https://gist.github.com/allomov>

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




--
Matthew Sykes
matthew.sykes(a)gmail.com<mailto:matthew.sykes(a)gmail.com>
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Notifications on ORG, SPACE and USER modifications

Matthias X Hub
 

Hi, happy to participate, we already have some initial ideas! Regards,
Matthias



From: Juan Pablo Genovese <juanpgenovese(a)gmail.com>
To: "Discussions about Cloud Foundry projects and the system overall."
<cf-dev(a)lists.cloudfoundry.org>
Date: 23.07.2015 15:41
Subject: Re: [cf-dev] Notifications on ORG, SPACE and USER
modifications
Sent by: cf-dev-bounces(a)lists.cloudfoundry.org



Anyone wiling to do a Hangout and start talking this? I really want to
push this forward.

Thanks!!

JP

2015-07-23 4:39 GMT-03:00 Benjamin Black <bblack(a)pivotal.io>:
ETags and a 304 response are specifically intended for that purpose. I'd
recommend that over relying on Last-Modified.


b

On Thu, Jul 23, 2015 at 12:34 AM, Koper, Dies <diesk(a)fast.au.fujitsu.com>
wrote:
Or setting the Last-Modified HTTP response header accordingly, and allow
clients to use HTTP caching mechanisms (Last-Modified, etc.) to get quick
empty responses with the current APIs if no changes have been made? (Or
maybe this is already working so ? haven?t checked).

Regards,
Dies Koper

From: cf-dev-bounces(a)lists.cloudfoundry.org [mailto:
cf-dev-bounces(a)lists.cloudfoundry.org] On Behalf Of Matt Cowger
Sent: Thursday, July 23, 2015 4:45 PM
To: Discussions about Cloud Foundry projects and the system overall.
Subject: Re: [cf-dev] Notifications on ORG, SPACE and USER modifications

I've wanted something similar as well.

On a related note, having a CC API 'serial' number (for each object in CC
- apps, spaces, etc) that increments on every change relevant to that
object would be of value for detecting if something has changed.

On Thu, Jul 23, 2015 at 3:27 PM, Dieu Cao <dcao(a)pivotal.io> wrote:
There are a few different approaches to this and different concerns that
are possible.
The requests I've seen have been around wanting to be able to subscribe to
and filter the various events that cc currently generates so that other
behavior could be triggered.
We currently have events, app usage events, and service usage events.
Is it acceptable for the notifications to be lossy? Depends on the use
case but If so, then the firehose may be an acceptable approach.

The CAPI team is currently focusing on other work in the near term, such
as the v3 API and private brokers, but would be happy to collaborate on a
proposal.


On Wed, Jul 22, 2015 at 2:05 PM, Juan Pablo Genovese <
juanpgenovese(a)gmail.com> wrote:
My take:

CC should have callbacks on for each model create, update and delete
methods. Those callbacks will send a message to an MQ, which you can
subscribe to consume those messages.
This can be expanded to pretty much every event we need to track.
What do you think?

JP

2015-07-22 17:30 GMT-03:00 Matthias X Hub <matthias.hub(a)de.ibm.com>:
Hi,

we (=IBM) are also having the need and are currently investigating how to
solve this. We plan to work on a proposal to discuss this further with the
cf community. I'll keep you updated on that.

Regards,
Matthias



From: Mike Youngstrom <youngm(a)gmail.com>
To: "Discussions about Cloud Foundry projects and the system
overall." <cf-dev(a)lists.cloudfoundry.org>
Date: 22.07.2015 20:57
Subject: Re: [cf-dev] Notifications on ORG, SPACE and USER
modifications
Sent by: cf-dev-bounces(a)lists.cloudfoundry.org




We have the same need. Today we are polling the CC.

It would be nice for us also if we could get CC event notifications via
something like the firehose.

Mike

On Wed, Jul 22, 2015 at 10:23 AM, Juan Pablo Genovese <
juanpgenovese(a)gmail.com> wrote:
I mean, I know you can list those events thru the API, but I want
something that will react on an event instead of having to be constantly
polling for them.

2015-07-22 13:18 GMT-03:00 Juan Pablo Genovese <juanpgenovese(a)gmail.com>:
Sree,

thanks! Any pointers on how can I hook up to these audit events?

Thank you!

2015-07-22 13:12 GMT-03:00 Sree Tummidi <stummidi(a)pivotal.io>:
I believe there are audit events generated for all these actions which can
be captured and forwarded to an SIEM solution like splunk

Thanks,
Sree

Sent from my iPhone

On Jul 22, 2015, at 8:54 AM, Juan Pablo Genovese <juanpgenovese(a)gmail.com>
wrote:

Guys,

I need to somehow hook up into the Cloud Controller (CC) to capture ORG,
SPACE and USER deletion, insertion and update.

So far, I considered some approaches, such as forking the CC (the least
favorite) and modifying the code with some hooks, tapping into Nginx to
capture the requests, and using triggers in the database to capture each
event and send the necessary info to a service.

What do you think?
Any other idea you might have?

Thanks!

--
Mis mejores deseos,
Best wishes,
Meilleurs v?ux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




--
Mis mejores deseos,
Best wishes,
Meilleurs v?ux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com



--
Mis mejores deseos,
Best wishes,
Meilleurs v?ux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



--
Mis mejores deseos,
Best wishes,
Meilleurs v?ux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




--
Mis mejores deseos,
Best wishes,
Meilleurs v?ux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Announcing support for Arbitrary Parameters and Service Instance Tags

Noburou TANIGUCHI
 

Great news!
I should tell it to my colleagues.


Dieu Cao wrote
On behalf of the CAPI team I'm pleased to announce support for Arbitrary
Parameters and Service Instance Tags, two features which extend the
possibilities for Cloud Foundry Marketplace service offerings, and provide
developers with increased flexibility in how services are consumed by
applications.

Support for Arbitrary Parameters is introduced with cf-release v209 and
CLI
v6.12.1. This features enables service providers to support user-provided
configuration options with the create, update, and bind service instance
operations, and the create service key operation (stay tuned for a
forthcoming announcement of support for Service Keys). Previously, this
could only be achieved by providing many plans to cover various
combinations of configuration options, or to provide a service instance
dashboard that users can SSO into and adjust configuration options after
creation. Although the platform and CLI now support the feature, service
broker authors must implement support for the feature as described in
the Service
Broker API v2.5 specification
&lt;http://docs.cloudfoundry.org/services/api.html>;.

Support for Instance Tags is introduced with cf-release v211 and CLI
v6.12.1. Since v2.0 of the Service Broker API, broker authors have be
able
to provide tags for a service offering in the /v2/catalog endpoint that
Cloud Foundry delivers to applications in the VCAP_SERVICES Environment
Variable
&lt;http://docs.cloudfoundry.org/devguide/deploy-apps/environment-variable.html#VCAP-SERVICES>;.
These tags provide developers with a more generic way for applications to
parse VCAP_SERVICES for credentials. Developers may now provide their own
tags when creating or updating a service instance by including a
comma-separated list of tags with the -t flag.

Documentation:

- http://docs.cloudfoundry.org/services/api.html

- http://docs.cloudfoundry.org/devguide/services/managing-services.html

Special thanks to the former CF Services API team and Shannon for their
hard work on these features.

-Dieu
CF CAPI PM

_______________________________________________
cf-dev mailing list
cf-dev(a).cloudfoundry
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




--
View this message in context: http://cf-dev.70369.x6.nabble.com/cf-dev-Announcing-support-for-Arbitrary-Parameters-and-Service-Instance-Tags-tp860p874.html
Sent from the CF Dev mailing list archive at Nabble.com.


Re: Did anybody deploy a wiki as app to CF?

Noburou TANIGUCHI
 

Thank you, all.

I am very glad to see such many positive reactions.
We should consider about English version of the articles.




--
View this message in context: http://cf-dev.70369.x6.nabble.com/cf-dev-Did-anybody-deploy-a-wiki-as-app-to-CF-tp643p873.html
Sent from the CF Dev mailing list archive at Nabble.com.

8501 - 8520 of 9425