[cf-env] [abacus] Changing how resources are organized


dmangin <dmangin@...>
 

With the current way we have resource_ids being used for metering,
accumulation, and aggregation, we have to have a resource definition for
every resource_id that people are using. Then when we have custom
buildpacks, we will have to start creating a resource definition for each
one of them, inflating the amount of the resource definitions that we have.
So we will be adding a new field called resource_type_id ontop of the
resource_id that will be the resource definition abacus uses, allowing
custom buildpacks to fall under the resource type. So we were thinking of
having the hierarchy changed so resource_id is underneath resource_type_id.

When we implement this, we will have to change abacus to use the
resource_type_id rather than the resource_id to do the calculations. What
changes will this cause in abacus right now and how will this affect the
previous reports abacus has created?

Reagrds,
Daniel Mangin



--
View this message in context: http://cf-dev.70369.x6.nabble.com/cf-env-abacus-Changing-how-resources-are-organized-tp2971.html
Sent from the CF Dev mailing list archive at Nabble.com.


Jean-Sebastien Delfino
 

Hi Daniel,

Is that related to Github issue #38?

https://github.com/cloudfoundry-incubator/cf-abacus/issues/38

Thanks

- Jean-Sebastien

On Fri, Dec 4, 2015 at 11:22 AM, dmangin <dmangin(a)us.ibm.com> wrote:

With the current way we have resource_ids being used for metering,
accumulation, and aggregation, we have to have a resource definition for
every resource_id that people are using. Then when we have custom
buildpacks, we will have to start creating a resource definition for each
one of them, inflating the amount of the resource definitions that we have.
So we will be adding a new field called resource_type_id ontop of the
resource_id that will be the resource definition abacus uses, allowing
custom buildpacks to fall under the resource type. So we were thinking of
having the hierarchy changed so resource_id is underneath resource_type_id.

When we implement this, we will have to change abacus to use the
resource_type_id rather than the resource_id to do the calculations. What
changes will this cause in abacus right now and how will this affect the
previous reports abacus has created?

Reagrds,
Daniel Mangin



--
View this message in context:
http://cf-dev.70369.x6.nabble.com/cf-env-abacus-Changing-how-resources-are-organized-tp2971.html
Sent from the CF Dev mailing list archive at Nabble.com.


dmangin <dmangin@...>
 

Yes, this is related to github issue 38.

https://github.com/cloudfoundry-incubator/cf-abacus/issues/38

We want to group the buildpacks by a resource_type_id and bind resource
definitions to the resource_type_id rather than to the resource_id. However,
when we make this change, how will this affect how abacus is doing all of
the calculations. The only change that I can think of is for abacus to use
the resource_type_id rather than the resource_id when creating the reports.





--
View this message in context: http://cf-dev.70369.x6.nabble.com/cf-dev-abacus-Changing-how-resources-are-organized-tp2971p2991.html
Sent from the CF Dev mailing list archive at Nabble.com.


Jean-Sebastien Delfino
 

It depends if you still want usage aggregation at both the resource_id and
resource_type_id levels (more changes as that'll add another aggregation
level to the reports) or if you only need aggregation at the
resource_type_id level (and are effectively treating that resource_type_id
as a 'more convenient' resource_id).

What aggregation levels do you need, both, or just aggregation at that
resource_type_id level?

- Jean-Sebastien

On Mon, Dec 7, 2015 at 3:19 PM, dmangin <dmangin(a)us.ibm.com> wrote:

Yes, this is related to github issue 38.

https://github.com/cloudfoundry-incubator/cf-abacus/issues/38

We want to group the buildpacks by a resource_type_id and bind resource
definitions to the resource_type_id rather than to the resource_id.
However,
when we make this change, how will this affect how abacus is doing all of
the calculations. The only change that I can think of is for abacus to use
the resource_type_id rather than the resource_id when creating the reports.





--
View this message in context:
http://cf-dev.70369.x6.nabble.com/cf-dev-abacus-Changing-how-resources-are-organized-tp2971p2991.html
Sent from the CF Dev mailing list archive at Nabble.com.


Piotr Przybylski <piotrp@...>
 

The main aggregation needed is at the resource type, however the aggregation within consumer by the resource Id is also something we would like to access - for example to determine that application used two different version of node. Instead of introducing resource type, the alternative approach could be to augment the consumer id with the resource id. 

Piotr

-----Jean-Sebastien Delfino <jsdelfino@...> wrote: -----
To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@...>
From: Jean-Sebastien Delfino <jsdelfino@...>
Date: 12/09/2015 11:51AM
Subject: [cf-dev] Re: Re: Re: [cf-env] [abacus] Changing how resources are organized

It depends if you still want usage aggregation at both the resource_id and resource_type_id levels (more changes as that'll add another aggregation level to the reports) or if you only need aggregation at the resource_type_id level (and are effectively treating that resource_type_id as a 'more convenient' resource_id).

What aggregation levels do you need, both, or just aggregation at that resource_type_id level?

- Jean-Sebastien


On Mon, Dec 7, 2015 at 3:19 PM, dmangin <dmangin@...> wrote:
Yes, this is related to github issue 38.

https://github.com/cloudfoundry-incubator/cf-abacus/issues/38

We want to group the buildpacks by a resource_type_id and bind resource
definitions to the resource_type_id rather than to the resource_id. However,
when we make this change, how will this affect how abacus is doing all of
the calculations. The only change that I can think of is for abacus to use
the resource_type_id rather than the resource_id when creating the reports.





--
View this message in context: http://cf-dev.70369.x6.nabble.com/cf-dev-abacus-Changing-how-resources-are-organized-tp2971p2991.html
Sent from the CF Dev mailing list archive at Nabble.com.



Jean-Sebastien Delfino
 

Thanks Piotr,

The main aggregation needed is at the resource type, however the
aggregation within consumer by the resource Id is also something we would
like to access - for example to determine that application used two
different version of node.

OK so then that means a new aggregation level, not rocket science, but a
rather mechanical addition of a new aggregation level similar to the
existing ones to the aggregator, reporting, tests, demos, schemas and API
doc. I'm out on vacation tomorrow Friday but tomorrow's IPM could be a good
opportunity to get the team to point that story's work with Max -- and that
way I won't be able to influence the point'ing :).

Instead of introducing resource type, the alternative approach could be
to augment the consumer id with the resource id

Not sure how that would work given that a consumer can use/consume multiple
(service) resources, and this 'resource type' aggregation should work for
all types of resources (not just runtime buildpack resources).

- Jean-Sebastien

On Thu, Dec 10, 2015 at 12:57 PM, Piotr Przybylski <piotrp(a)us.ibm.com>
wrote:

The main aggregation needed is at the resource type, however the
aggregation within consumer by the resource Id is also something we would
like to access - for example to determine that application used two
different version of node. Instead of introducing resource type, the
alternative approach could be to augment the consumer id with the resource
id.

Piotr

-----Jean-Sebastien Delfino <jsdelfino(a)gmail.com> wrote: -----
To: "Discussions about Cloud Foundry projects and the system overall." <
cf-dev(a)lists.cloudfoundry.org>
From: Jean-Sebastien Delfino <jsdelfino(a)gmail.com>
Date: 12/09/2015 11:51AM
Subject: [cf-dev] Re: Re: Re: [cf-env] [abacus] Changing how resources are
organized


It depends if you still want usage aggregation at both the resource_id and
resource_type_id levels (more changes as that'll add another aggregation
level to the reports) or if you only need aggregation at the
resource_type_id level (and are effectively treating that resource_type_id
as a 'more convenient' resource_id).

What aggregation levels do you need, both, or just aggregation at that
resource_type_id level?

- Jean-Sebastien

On Mon, Dec 7, 2015 at 3:19 PM, dmangin <dmangin(a)us.ibm.com> wrote:

Yes, this is related to github issue 38.

https://github.com/cloudfoundry-incubator/cf-abacus/issues/38

We want to group the buildpacks by a resource_type_id and bind resource
definitions to the resource_type_id rather than to the resource_id.
However,
when we make this change, how will this affect how abacus is doing all of
the calculations. The only change that I can think of is for abacus to use
the resource_type_id rather than the resource_id when creating the
reports.





--
View this message in context:
http://cf-dev.70369.x6.nabble.com/cf-dev-abacus-Changing-how-resources-are-organized-tp2971p2991.html
Sent from the CF Dev mailing list archive at Nabble.com.