Re: [abacus] Handling Notifications
I've begun to work on supporting notifications in abacus.Great! What you've described here looks like a good start to me. I've
included a few comments and questions below:
We already have a github issue and pivotal tracker story associated withit as seen here:
https://github.com/cloudfoundry-incubator/cf-abacus/issues/161and one more, created from Github #161 by the CF gitbot:
I'm guessing that's the one that'll track the commits under Github #161.
There are two ways that this will probably need to work. One via awebhook and another via letting a client keep and open connection that
would receive a stream of notifications.
I've asked Subhash (@sinduri, author of Github #161) to post more details
of what he needs here. I'm guessing it's Webhooks but will let him
For the registry, I have considered Eureka <...> I'm not sure if alsousing it as a registry for handling notifications would kind of warp the
purpose of its use.
IMO Eureka works fine as a registry of IP addresses for performance
monitoring of our services, but I'm not so sure about it for persistent
registrations of Webhooks and corresponding triggers/criteria, as AIUI
that'd a big deviation from what it's originally designed for. So +1 to
what you said about warping its purpose :)
Documents would probably be keyed to the given URL just for supportingupdates on new criteria for a certain callback URL and prevent multiple
documents for the same URL.
How about keying by criteria as well to know when to trigger a Webhook
call? and maybe allow multiple registrations per URL? (e.g please call me
back at http://foo.com/bar on new usage for org abc, and also when usage
for app xyz goes over 1000, would probably be two separate registration
docs for a single URL)
If a call is an error, it'd be best to retry for a set amount oftime/tries until that particular URL is marked unreachable (and thus isn't
called in subsequent notifications).
+1 for a sort of quarantine on an unreachable Webhook. How about slow
Webhooks causing back pressure problems? Would we quarantine these too?
... And a few more questions that would be good to discuss here as well:
- should we let the rating service app do this or have a separate
notification service app?
- with partitioning of the orgs across multiple deployments of our apps
(for scalability or regional deployments for example) do I need to first
find the right service to register with (e.g. register with the us-south
notification service or the eu-west notification service)? or can I
register with a central notification service that will then figure out
which deployment instance will call me back?
- how do we secure the registration calls and Webhook callbacks?
- do we replay notifications when we can't deliver them?
- can I register to receive all notifications from a certain point in the
logical stream of notifications matching a criteria (e.g. call me back if
this org consumed too much per hour at any point since last week)
- do we have specific examples of notification trigger criteria? (I've
asked Subhash to post some here to initiate that discussion).
On Wed, Nov 11, 2015 at 1:21 PM, Benjamin Cheng <bscheng(a)us.ibm.com> wrote:
I've begun to work on supporting notifications in abacus. We already have