Questions about purge app usage event API


Nitta, Minoru <minoru.nitta@...>
 

Hi,

I have some questions about 'Purge and reseed App Usage Events' API.
https://apidocs.cloudfoundry.org/212/app_usage_events/purge_and_reseed_app_usage_events.html

I am wondering about app started events. The API populates new events for started app.
This may cause the problems if the API and stopping app occur at the same time (race condition).
(e.g. app stopped after the API execution, which populates new app start event but stop app event
never occur).

How can I workaround this problem? Or CloudFoundry handles such race condition internally and
I do not have to consider the workaround for it?

Regards,
Minoru


Hristo Iliev
 

Hi Minoru,

AFAIK the API does not handle the race condition and this is documented in
the link you provided: "There is the potential race condition if apps are
currently being started, stopped, or scaled"

The purging has to be called only once, before you start/connect your
billing infrastructure. Have a look at this blog for more detailed
explanation: https://www.cloudfoundry.org/how-to-bill-on-cloud-foundry/

Regards,
Hristo Iliev

2015-11-30 3:56 GMT+02:00 Nitta, Minoru <minoru.nitta(a)jp.fujitsu.com>:

Hi,

I have some questions about 'Purge and reseed App Usage Events' API.

https://apidocs.cloudfoundry.org/212/app_usage_events/purge_and_reseed_app_usage_events.html

I am wondering about app started events. The API populates new events for
started app.
This may cause the problems if the API and stopping app occur at the same
time (race condition).
(e.g. app stopped after the API execution, which populates new app start
event but stop app event
never occur).

How can I workaround this problem? Or CloudFoundry handles such race
condition internally and
I do not have to consider the workaround for it?

Regards,
Minoru


Nitta, Minoru <minoru.nitta@...>
 

Hi Hristo,

Thank you for your response. If the API does not handle the race condition, it would not be realistic to
create a billing epoch safely, especially after providing (commercial) service because there are customer’s
application installed.

I would like to issue a purge API periodically, specifically once a month, to calculate application working
time easily. I can calculate application working time during a month even if I do not issue the API, but
application start event will be expired and deleted after one month (default) and will not be able to
calculate application working time eventually.

Regards,
Minoru Nitta

From: Hristo Iliev [mailto:hsiliev(a)gmail.com]
Sent: Monday, November 30, 2015 4:48 PM
To: Discussions about Cloud Foundry projects and the system overall. <cf-dev(a)lists.cloudfoundry.org>
Subject: [cf-dev] Re: Questions about purge app usage event API

Hi Minoru,
AFAIK the API does not handle the race condition and this is documented in the link you provided: "There is the potential race condition if apps are currently being started, stopped, or scaled"
The purging has to be called only once, before you start/connect your billing infrastructure. Have a look at this blog for more detailed explanation: https://www.cloudfoundry.org/how-to-bill-on-cloud-foundry/
Regards,
Hristo Iliev

2015-11-30 3:56 GMT+02:00 Nitta, Minoru <minoru.nitta(a)jp.fujitsu.com<mailto:minoru.nitta(a)jp.fujitsu.com>>:
Hi,

I have some questions about 'Purge and reseed App Usage Events' API.
https://apidocs.cloudfoundry.org/212/app_usage_events/purge_and_reseed_app_usage_events.html

I am wondering about app started events. The API populates new events for started app.
This may cause the problems if the API and stopping app occur at the same time (race condition).
(e.g. app stopped after the API execution, which populates new app start event but stop app event
never occur).

How can I workaround this problem? Or CloudFoundry handles such race condition internally and
I do not have to consider the workaround for it?

Regards,
Minoru


Hristo Iliev
 

Hi,

You can do one or more of these:
1) Start billing epoch just once
2) Compare real application state with the events and provide a
compensation logic. We tried this in the Abacus cf-bridge prototype [1]
3) Help improve Abacus cf-bridge [2]

Probably the easiest thing to do is #1 since your implementation would be
much simpler and you won't need to synchronize polling of events with
purging (these might be responsibility of different teams which makes
things harder).

Even with all of these 3 approaches you might end up with problems. The
correct thing to do imho be to implement the idea of Piotr Przybylski [3].
This goes in the direction of continuously pulling the events and storing a
copy into a separate retention DB that can be used for audit,
re-calculation, legal or other reasons.

Regards,
Hristo Iliev

[1]
https://github.com/cloudfoundry-incubator/cf-abacus/blob/master/lib/cf/bridge/src/index.js#L246
[2]
https://github.com/cloudfoundry-incubator/cf-abacus/tree/master/lib/cf/bridge
[3] https://github.com/cloudfoundry-incubator/cf-abacus/issues/30

2015-12-03 12:10 GMT+02:00 Nitta, Minoru <minoru.nitta(a)jp.fujitsu.com>:

Hi Hristo,



Thank you for your response. If the API does not handle the race
condition, it would not be realistic to

create a billing epoch safely, especially after providing (commercial)
service because there are customer’s

application installed.



I would like to issue a purge API periodically, specifically once a month,
to calculate application working

time easily. I can calculate application working time during a month even
if I do not issue the API, but

application start event will be expired and deleted after one month
(default) and will not be able to

calculate application working time eventually.



Regards,

Minoru Nitta



*From:* Hristo Iliev [mailto:hsiliev(a)gmail.com]
*Sent:* Monday, November 30, 2015 4:48 PM
*To:* Discussions about Cloud Foundry projects and the system overall. <
cf-dev(a)lists.cloudfoundry.org>
*Subject:* [cf-dev] Re: Questions about purge app usage event API



Hi Minoru,

AFAIK the API does not handle the race condition and this is documented in
the link you provided: "There is the potential race condition if apps are
currently being started, stopped, or scaled"

The purging has to be called only once, before you start/connect your
billing infrastructure. Have a look at this blog for more detailed
explanation: https://www.cloudfoundry.org/how-to-bill-on-cloud-foundry/

Regards,

Hristo Iliev



2015-11-30 3:56 GMT+02:00 Nitta, Minoru <minoru.nitta(a)jp.fujitsu.com>:

Hi,

I have some questions about 'Purge and reseed App Usage Events' API.

https://apidocs.cloudfoundry.org/212/app_usage_events/purge_and_reseed_app_usage_events.html

I am wondering about app started events. The API populates new events for
started app.
This may cause the problems if the API and stopping app occur at the same
time (race condition).
(e.g. app stopped after the API execution, which populates new app start
event but stop app event
never occur).

How can I workaround this problem? Or CloudFoundry handles such race
condition internally and
I do not have to consider the workaround for it?

Regards,
Minoru