Re: Notifications on ORG, SPACE and USER modifications

Home Messages Hashtags Subgroups

#948

Re: Notifications on ORG, SPACE and USER modifications

Jean-Sebastien Delfino #948 I’m going to need something like this too for the CF Abacus service metering project, as I’d like to track the lifecycle of orgs, users, etc to match their history with the usage data reported for them. Here’s a straw man description of what I had in mind: - For Abacus, I’d need a Lossless API. Usage metering eventually translates to billing and money, you don’t want to lose that :) - An extension or variant of the current CF /v2/events API supporting users, orgs, app usage etc, as even with a notification API I’ll still need to do GETs sometimes. - 304 responses with etags on these GETs (as suggested earlier in the thread [1]) would be good. - A Webhook style notification API where I could register interest in a selection of events with a callback URL, and get these events POSTed back to me at that URL, similar to what Github and many others do with Webhooks. - On top of Webhooks, it’d be nice to have a form of streaming (either down to the client like the Firehose does or in the other direction up to the Webhook callback URL), but I'm not sure if we’ll need that in the project right away. - We’d obviously need some form of security, maybe use my user token to register for events on entities that I have access to? - I’m also curious about the group’s thoughts on queueing and back-pressure when events get generated faster that they can be consumed for example. There was a mention of some message queuing earlier [2]. That would make sense to me (although IMO it’d be good if the underlying MQ didn’t shine through the API). What did you have in mind for this? I guess there are quite a few things to figure out here! I’ll be happy to collaborate with the community on these discussions. Thoughts? [1] http://cf-dev.70369.x6.nabble.com/cf-dev-Notifications-on-ORG-SPACE-and-USER-modifications-tp827p842.html [2] http://cf-dev.70369.x6.nabble.com/cf-dev-Notifications-on-ORG-SPACE-and-USER-modifications-tp827p834.html - Jean-Sebastien toggle quoted messageShow quoted text On Fri, Jul 24, 2015 at 9:59 PM, Matt Cowger <matt(a)cowger.us> wrote: I think ETags is reasonable thought as well. On Thu, Jul 23, 2015 at 4:39 PM, Benjamin Black <bblack(a)pivotal.io> wrote: ETags and a 304 response are specifically intended for that purpose. I'd recommend that over relying on Last-Modified. b On Thu, Jul 23, 2015 at 12:34 AM, Koper, Dies <diesk(a)fast.au.fujitsu.com> wrote: Or setting the Last-Modified HTTP response header accordingly, and allow clients to use HTTP caching mechanisms (Last-Modified, etc.) to get quick empty responses with the current APIs if no changes have been made? (Or maybe this is already working so – haven’t checked). Regards, Dies Koper From: cf-dev-bounces(a)lists.cloudfoundry.org [mailto: cf-dev-bounces(a)lists.cloudfoundry.org] On Behalf Of Matt Cowger Sent: Thursday, July 23, 2015 4:45 PM To: Discussions about Cloud Foundry projects and the system overall. Subject: Re: [cf-dev] Notifications on ORG, SPACE and USER modifications I've wanted something similar as well. On a related note, having a CC API 'serial' number (for each object in CC - apps, spaces, etc) that increments on every change relevant to that object would be of value for detecting if something has changed. On Thu, Jul 23, 2015 at 3:27 PM, Dieu Cao <dcao(a)pivotal.io> wrote: There are a few different approaches to this and different concerns that are possible. The requests I've seen have been around wanting to be able to subscribe to and filter the various events that cc currently generates so that other behavior could be triggered. We currently have events, app usage events, and service usage events. Is it acceptable for the notifications to be lossy? Depends on the use case but If so, then the firehose may be an acceptable approach. The CAPI team is currently focusing on other work in the near term, such as the v3 API and private brokers, but would be happy to collaborate on a proposal. On Wed, Jul 22, 2015 at 2:05 PM, Juan Pablo Genovese < juanpgenovese(a)gmail.com> wrote: My take: CC should have callbacks on for each model create, update and delete methods. Those callbacks will send a message to an MQ, which you can subscribe to consume those messages. This can be expanded to pretty much every event we need to track. What do you think? JP 2015-07-22 17:30 GMT-03:00 Matthias X Hub <matthias.hub(a)de.ibm.com>: Hi, we (=IBM) are also having the need and are currently investigating how to solve this. We plan to work on a proposal to discuss this further with the cf community. I'll keep you updated on that. Regards, Matthias From: Mike Youngstrom <youngm(a)gmail.com> To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev(a)lists.cloudfoundry.org> Date: 22.07.2015 20:57 Subject: Re: [cf-dev] Notifications on ORG, SPACE and USER modifications Sent by: cf-dev-bounces(a)lists.cloudfoundry.org ------------------------------ We have the same need. Today we are polling the CC. It would be nice for us also if we could get CC event notifications via something like the firehose. Mike On Wed, Jul 22, 2015 at 10:23 AM, Juan Pablo Genovese < juanpgenovese(a)gmail.com> wrote: I mean, I know you can list those events thru the API, but I want something that will react on an event instead of having to be constantly polling for them. 2015-07-22 13:18 GMT-03:00 Juan Pablo Genovese <juanpgenovese(a)gmail.com>: Sree, thanks! Any pointers on how can I hook up to these audit events? Thank you! 2015-07-22 13:12 GMT-03:00 Sree Tummidi <stummidi(a)pivotal.io>: I believe there are audit events generated for all these actions which can be captured and forwarded to an SIEM solution like splunk Thanks, Sree Sent from my iPhone On Jul 22, 2015, at 8:54 AM, Juan Pablo Genovese < juanpgenovese(a)gmail.com> wrote: Guys, I need to somehow hook up into the Cloud Controller (CC) to capture ORG, SPACE and USER deletion, insertion and update. So far, I considered some approaches, such as forking the CC (the least favorite) and modifying the code with some hooks, tapping into Nginx to capture the requests, and using triggers in the database to capture each event and send the necessary info to a service. What do you think? Any other idea you might have? Thanks! -- Mis mejores deseos, Best wishes, Meilleurs vœux, Juan Pablo ------------------------------------------------------ http://www.jpgenovese.com _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev -- Mis mejores deseos, Best wishes, Meilleurs vœux, Juan Pablo ------------------------------------------------------ http://www.jpgenovese.com -- Mis mejores deseos, Best wishes, Meilleurs vœux, Juan Pablo ------------------------------------------------------ http://www.jpgenovese.com _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev -- Mis mejores deseos, Best wishes, Meilleurs vœux, Juan Pablo ------------------------------------------------------ http://www.jpgenovese.com _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev -- -- Matt _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev -- -- Matt _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev -- Jean-Sebastien Sent from my DynaTAC 8000x attachment.html
More All Messages By This Member

View All 23 Messages In Topic

#948

Join cf-dev@lists.cloudfoundry.org to automatically receive all group messages.

Home Hashtags Subgroups Terms