Hi, I am also looking at runtime submissions for Abacus, worked on it for
Bluemix, couple of points for discussion.

In addition to usage events (start followed by stop), the scaling and
duplicate events need to be handled. The former is a START followed by
START with memory or instance count changed, the latter can be STOP
followed by STOP.
We also encountered a situation where the ordering is correct - START
followed by STOP but the timestamp for the START is later than STOP.

For the running applications - your points #2 and #3 are a working though
it may generate fair amount of traffic, depending on frequency and number
of running applications. Eventually we may want to look at alternatives,
for example enhance metering to allow for time based submissions. Instead
of continually submitting usage, submit the state of application -
(STARTED, memory, instances), the metering could then calculate usage for
that application based on the time passed until application is stopped.

I think handling some of above scenarios requires persistence, even if to
log CF events that were used for submission (or skipped). You may want to
persist state to recovery from application failure or restart, as well as
keep track of running/active applications.

Thanks,

Piotr


Hristo Iliev <hsiliev(a)gmail.com> wrote on 08/07/2015 08:07:53 AM:

We want to integrate the metrics provided by Cloud Foundry with Abacus.

We plan to create a billing/metering integration layer that:
fetches the app usage events from CC
inserts runtime usage data to Abacus
The events should be processed to build usage data based on them.
This is much like the idea outlined in Dr.Nic's blog. The
integration layer can do this periodically.

AFAIK Abacus provides usage reports for the current month only and
not for arbitrary period of time. This implies some restrictions to
what we can report when the application is:
1. started and stopped several times in the month
2. started in the current month but not stopped at all
3. started in a previous month and not stopped
The first problem can be solved by iterating through the events and
finding the respective start and end timestamps that have to be
reported to Abacus.

The second issue might be solved by reporting small amounts of
usage, stating from the last start event and continuing to report on
every poll of the integration layer. For example we can report several

usages:

start: 1438945112; end: 1438946000 (current time for the billing

integration)

start: 1438946000 (previous reporting cycle); end: 1438947000
The third issue might be solved by finding the last start event and
reporting in the same manner as with #2.

Reporting usage in small steps might require persistence so we can
store the end time of the previous reporting. We might use in-memory
cache and reach to Abacus as primary storage. If Abacus can
accumulate usage reporting we can even get rid of the persistence and

cache.

Is such integration in the scope of Abacus project?

Regards,
Hristo Iliev