Hi all interested in notifications on modification of resources,

It would be helpful for me in framing the "what" and the "why" of this
feature if you could also describe your specific use cases and pain points
on why you would want notifications on modifications and also which
resources you particularly care about.
Is it for real time updates on a dashboard? For consumption for billing
purposes? For triggering provisioning/deprovisioning of resources?

-Dieu

On Tue, Jul 28, 2015 at 11:05 AM, Jean-Sebastien Delfino <
jsdelfino(a)gmail.com> wrote:

I’m going to need something like this too for the CF Abacus service
metering project, as I’d like to track the lifecycle of orgs, users, etc to
match their history with the usage data reported for them.


Here’s a straw man description of what I had in mind:


- For Abacus, I’d need a Lossless API. Usage metering eventually
translates to billing and money, you don’t want to lose that :)


- An extension or variant of the current CF /v2/events API supporting
users, orgs, app usage etc, as even with a notification API I’ll still need
to do GETs sometimes.


- 304 responses with etags on these GETs (as suggested earlier in the
thread [1]) would be good.


- A Webhook style notification API where I could register interest in a
selection of events with a callback URL, and get these events POSTed back
to me at that URL, similar to what Github and many others do with Webhooks.


- On top of Webhooks, it’d be nice to have a form of streaming (either
down to the client like the Firehose does or in the other direction up to
the Webhook callback URL), but I'm not sure if we’ll need that in the
project right away.


- We’d obviously need some form of security, maybe use my user token to
register for events on entities that I have access to?


- I’m also curious about the group’s thoughts on queueing and
back-pressure when events get generated faster that they can be consumed
for example. There was a mention of some message queuing earlier [2]. That
would make sense to me (although IMO it’d be good if the underlying MQ
didn’t shine through the API). What did you have in mind for this?


I guess there are quite a few things to figure out here! I’ll be happy to
collaborate with the community on these discussions.


Thoughts?


[1]
http://cf-dev.70369.x6.nabble.com/cf-dev-Notifications-on-ORG-SPACE-and-USER-modifications-tp827p842.html

[2]
http://cf-dev.70369.x6.nabble.com/cf-dev-Notifications-on-ORG-SPACE-and-USER-modifications-tp827p834.html


- Jean-Sebastien

On Fri, Jul 24, 2015 at 9:59 PM, Matt Cowger <matt(a)cowger.us> wrote:

I think ETags is reasonable thought as well.

On Thu, Jul 23, 2015 at 4:39 PM, Benjamin Black <bblack(a)pivotal.io>
wrote:

ETags and a 304 response are specifically intended for that purpose.
I'd recommend that over relying on Last-Modified.


b

On Thu, Jul 23, 2015 at 12:34 AM, Koper, Dies <
diesk(a)fast.au.fujitsu.com> wrote:

Or setting the Last-Modified HTTP response header accordingly, and
allow clients to use HTTP caching mechanisms (Last-Modified, etc.) to get
quick empty responses with the current APIs if no changes have been made?
(Or maybe this is already working so – haven’t checked).



Regards,

Dies Koper



*From:* cf-dev-bounces(a)lists.cloudfoundry.org [mailto:
cf-dev-bounces(a)lists.cloudfoundry.org] *On Behalf Of *Matt Cowger
*Sent:* Thursday, July 23, 2015 4:45 PM
*To:* Discussions about Cloud Foundry projects and the system overall.
*Subject:* Re: [cf-dev] Notifications on ORG, SPACE and USER
modifications



I've wanted something similar as well.



On a related note, having a CC API 'serial' number (for each object in
CC - apps, spaces, etc) that increments on every change relevant to that
object would be of value for detecting if something has changed.



On Thu, Jul 23, 2015 at 3:27 PM, Dieu Cao <dcao(a)pivotal.io> wrote:

There are a few different approaches to this and different concerns
that are possible.

The requests I've seen have been around wanting to be able to
subscribe to and filter the various events that cc currently generates so
that other behavior could be triggered.

We currently have events, app usage events, and service usage events.

Is it acceptable for the notifications to be lossy? Depends on the
use case but If so, then the firehose may be an acceptable approach.



The CAPI team is currently focusing on other work in the near term,
such as the v3 API and private brokers, but would be happy to collaborate
on a proposal.





On Wed, Jul 22, 2015 at 2:05 PM, Juan Pablo Genovese <
juanpgenovese(a)gmail.com> wrote:

My take:



CC should have callbacks on for each model create, update and delete
methods. Those callbacks will send a message to an MQ, which you can
subscribe to consume those messages.

This can be expanded to pretty much every event we need to track.

What do you think?



JP



2015-07-22 17:30 GMT-03:00 Matthias X Hub <matthias.hub(a)de.ibm.com>:

Hi,

we (=IBM) are also having the need and are currently investigating how
to solve this. We plan to work on a proposal to discuss this further with
the cf community. I'll keep you updated on that.

Regards,
Matthias



From: Mike Youngstrom <youngm(a)gmail.com>
To: "Discussions about Cloud Foundry projects and the system
overall." <cf-dev(a)lists.cloudfoundry.org>
Date: 22.07.2015 20:57
Subject: Re: [cf-dev] Notifications on ORG, SPACE and USER
modifications
Sent by: cf-dev-bounces(a)lists.cloudfoundry.org
------------------------------




We have the same need. Today we are polling the CC.

It would be nice for us also if we could get CC event notifications
via something like the firehose.

Mike

On Wed, Jul 22, 2015 at 10:23 AM, Juan Pablo Genovese <
juanpgenovese(a)gmail.com> wrote:
I mean, I know you can list those events thru the API, but I want
something that will react on an event instead of having to be constantly
polling for them.

2015-07-22 13:18 GMT-03:00 Juan Pablo Genovese <
juanpgenovese(a)gmail.com>:
Sree,

thanks! Any pointers on how can I hook up to these audit events?

Thank you!

2015-07-22 13:12 GMT-03:00 Sree Tummidi <stummidi(a)pivotal.io>:
I believe there are audit events generated for all these actions which
can be captured and forwarded to an SIEM solution like splunk

Thanks,
Sree

Sent from my iPhone

On Jul 22, 2015, at 8:54 AM, Juan Pablo Genovese <
juanpgenovese(a)gmail.com> wrote:

Guys,

I need to somehow hook up into the Cloud Controller (CC) to capture
ORG, SPACE and USER deletion, insertion and update.

So far, I considered some approaches, such as forking the CC (the
least favorite) and modifying the code with some hooks, tapping into Nginx
to capture the requests, and using triggers in the database to capture each
event and send the necessary info to a service.

What do you think?
Any other idea you might have?

Thanks!

--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------

http://www.jpgenovese.com
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com



--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev





--

Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------

http://www.jpgenovese.com


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev





--

-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

--
Jean-Sebastien

Sent from my DynaTAC 8000x

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev