Assk,
can you confirm that the same scope (abacus.usage.write) is sufficient to
retrieve usage ?

Piotr




----- Original message -----
From: Saravanakumar A Srinivasan/Burlingame/IBM(a)IBMUS
To: <cf-dev(a)lists.cloudfoundry.org>
Cc:
Subject: [cf-dev] Re: Re: Re: [cf-dev][abacus] Usage submission
authorization
Date: Thu, Oct 15, 2015 7:06 PM

We have enabled scope based authorization for REST endpoints at usage
collector and usage reporting service. While we are working on using system
OAuth bearer access token at internal Abacus pipeline, Submitting usage to
a secured Abacus needs a OAuth bearer access token with
'abacus.usage.write' scope.

Thanks,
Saravanakumar Srinivasan (Assk),

Bay Area Lab, 1001, E Hillsdale Blvd, Ste 400, Foster City, CA - 94404.
E-mail: sasrin(a)us.ibm.com
Phone: 650 645 8251 (T/L 367-8251)


-----Jean-Sebastien Delfino <jsdelfino(a)gmail.com> wrote: -----
To: "Discussions about Cloud Foundry projects and the system overall." <
cf-dev(a)lists.cloudfoundry.org>
From: Jean-Sebastien Delfino <jsdelfino(a)gmail.com>
Date: 10/12/2015 07:50PM
Subject: [cf-dev] Re: Re: [cf-dev][abacus] Usage submission authorization

Also, resource id is an arbitrary identifier, making it part of the

scope may create quite complex names e.g.
'abacus.runtimes/node/v12-07.revision-2-buildpack-guid-a3d7ff4d-3cb1-4cc3-a855-fae98e20cf57.write.

Do you have a specific issue in mind with putting the resource uuid in the
scope name? We have uuids all over the place in CF, in most of the APIs,
the usage docs etc so I'm not sure why it'd be a problem to have one here.

Any naming convention may not be generic enough, for example for my UAA

instance requires the scope names to start with component using it,
followed by proper name - 'bss.runtimes.abacus.<resource id>.write'.

Like I said before, if you can't or don't want to use a specific scope per
resource, then you can use abacus.usage.write (with the same
disclaimers/warnings I gave in my previous post.)

I must be missing something though :) ... aren't you happily using
cloud_controller.write for example (or similar other CF scopes) without
renaming it to <your client component>.cloud_controller.write? Why would
you treat abacus.usage.write different?

Also, I must admit to find a bit surprising a naming convention that will
tie the scope name to the client that presents it. Isn't the scope
typically defined by the owner of the resource it protects instead of the
client? In that case the owner of the resource is not the client
component... it is the CF abacus project, hence <abacus>.usage.write.
Wouldn't that make more sense?

Finally, I'm also not quite sure how this will work at all if for example
Abacus needs to authorize resource access from multiple clients. That would
have to be really dynamic then, as each new client would require Abacus to
know about a new client specific naming convention (or client component
name prefix in the example you gave...)

Now, all that being said, looks like I'm not really following how you're
envisioning this to work, so do you think you could maybe submit a pull
request with how you concretely propose to make that dynamic scope naming
work when it includes client component names, or follows client component
specific naming conventions?

Thanks!

- Jean-Sebastien

On Mon, Oct 12, 2015 at 5:22 PM, Piotr Przybylski <piotrp(a)us.ibm.com>
wrote:

Hi Sebastien,
I am not sure why allowing resource provider to explicitly specify scope
with which particular resource usage will be submitted is a problem. Just
allowing to pick a name would not compromise submission security in any
way. It could be done for example by adding scope name to the resource
definition.

Any naming convention may not be generic enough, for example for my UAA
instance requires the scope names to start with component using it,
followed by proper name - 'bss.runtimes.abacus.<resource id>.write'. Also,
resource id is an arbitrary identifier, making it part of the scope may
create quite complex names e.g.
'abacus.runtimes/node/v12-07.revision-2-buildpack-guid-a3d7ff4d-3cb1-4cc3-a855-fae98e20cf57.write.


Piotr



[image: Inactive hide details for Jean-Sebastien Delfino ---10/09/2015
09:38:09 PM---Hey Piotr, >>> In some cases it may not be possibl]Jean-Sebastien
Delfino ---10/09/2015 09:38:09 PM---Hey Piotr, >>> In some cases it may not
be possible or viable to create new scope for

From: Jean-Sebastien Delfino <jsdelfino(a)gmail.com>
To: "Discussions about Cloud Foundry projects and the system overall." <
cf-dev(a)lists.cloudfoundry.org>
Date: 10/09/2015 09:38 PM
Subject: [cf-dev] Re: Re: Re: Re: Re: [abacus] Usage submission
authorization

------------------------------



Hey Piotr,

In some cases it may not be possible or viable to create new scope for

each resource id e.g. short lived resources.

Why wouldn't that be possible? What type of short-lived resources did

you have in mind?

For example experimental service version (beta) replaced by release

version, usage of which may be reported and metered but not necessarily
billed.

OK, that use case makes sense to me. So, your resource is going to be
available for a few hours or days. I'm assuming that to get it on board CF
and meter it with Abacus you're going to run a cf create-service-broker
command or cf update-service-broker, define the resource config specifying
how to meter it, and store that config where your Abacus provisioning
endpoint implementation can retrieve it.

To secure the submission of usage for it, if I understand correctly how
UAA works, you'll then need to do this:
uaac client update <your service provider's client id> --authorities "...
existing permissions... abacus.<your resource id>.write"

That's all...

If that's really too much of a burden (really?) compared to the other
steps, you're basically looking to do *nothing* to secure that resource.
You could just submit usage with the abacus.usage.write scope, but that's
the equivalent of the CF cloud_controller.write scope for Abacus, close to
all powers... I'd probably advise against it as that's a serious risk but
that may be what you're looking for.

The scope names may need to follow adopter specific conventions so

creating scope with predefined name 'abacus.usage....' may not fit that
scheme. Abacus should offer ability to adjust the scope names, otherwise
submission may not be possible.

These are simple names that we expect in the token used to submit usage.
They're just constants like the names of our APIs, parameters, options,
fields in our JSON schemas... basically the contract/interface between the
Abacus user and its implementation. Not sure if there's a specific issue
with that abacus naming convention or if it's just a theoretical question,
but I'll be happy to discuss alternate naming conventions:

Do you have another naming convention in mind that you'd like to use?

Is there a specific issue with abacus.usage.write? Is the 'abacus' part in
the name a problem?

Would you prefer to submit usage with an existing CF scope like
cloud_controller.write or another of these high power scopes?
(again, I'd advise against it though...)

- Jean-Sebastien

- Jean-Sebastien

On Thu, Oct 8, 2015 at 5:24 PM, Piotr Przybylski <*piotrp(a)us.ibm.com*
<piotrp(a)us.ibm.com>> wrote:

- Hi Sebastien,

>> In some cases it may not be possible or viable to create new scope
for each resource id e.g. short lived resources.

>Why wouldn't that be possible? What type of short-lived resources did
you have in mind?

For example experimental service version (beta) replaced by release
version, usage of which may be reported and metered but not necessarily
billed.
The scope names may need to follow adopter specific conventions so
creating scope with predefined name 'abacus.usage....' may not fit
that scheme. Abacus should offer ability to adjust the scope names,
otherwise submission may not be possible.


> Another reason why I'm not sure about short lived resources, is that
although you may decide to stop offering a type a resource at some point,
once you've metered it, and sent a bill for it >to a customer, I don't
think you can really 'forget' about its existence anymore... So in that
sense I'm not sure how it can be 'short lived'.
The short lived resource is only for submission, once it is not
offered, its specific scope is not needed. Thad does not mean erasing
history of usage.


Piotr




[image: Inactive hide details for Jean-Sebastien Delfino ---10/08/2015
11:10:16 AM---Hi Piotr, > In some cases it may not be possible o]Jean-Sebastien
Delfino ---10/08/2015 11:10:16 AM---Hi Piotr, > In some cases it may not be
possible or viable to create new scope for

From: Jean-Sebastien Delfino <*jsdelfino(a)gmail.com*
<jsdelfino(a)gmail.com>>
To: "Discussions about Cloud Foundry projects and the system overall."
<*cf-dev(a)lists.cloudfoundry.org* <cf-dev(a)lists.cloudfoundry.org>>
Date: 10/08/2015 11:10 AM
Subject: [cf-dev] Re: Re: Re: [abacus] Usage submission authorization


------------------------------



Hi Piotr,

> In some cases it may not be possible or viable to create new scope
for each resource id e.g. short lived resources.

Why wouldn't that be possible? What type of short-lived resources did
you have in mind?

The typical use case I've seen is for a Cloud platform to decide to
offer a new type of database or analytics or messaging service, or a new
type of runtime for example. Before that new resource is offered on the
platform, their resource provider needs to get on board, get a user id,
auth credentials defined in UAA etc... You probably also need to define how
you're going to meter that new resource and the pricing for it.

Couldn't a scope be created in UAA at that time along all these other
on boarding steps?

Another reason why I'm not sure about short lived resources, is that
although you may decide to stop offering a type a resource at some point,
once you've metered it, and sent a bill for it to a customer, I don't think
you can really 'forget' about its existence anymore... So in that sense I'm
not sure how it can be 'short lived'.

> Some flexibility would also help to accommodate changes related to
grouping resources by type as discussed in [1].

We discussed two options in [1]:
a) support a resource_type in addition to resource_id for grouping
many resource_ids under a single type
b) a common resource_id for several resources (something like 'node'
for all your versions of Node.js build packs for example)

Since option (a) is not implemented at this point and Issue #38 is
actually assigned to a 'future' milestone, AIUI resource providers need to
use option (b) with a common resource_id for multiple resources. Is
creating a scope for that common id still too much of a burden then?

[1] - *https://github.com/cloudfoundry-incubator/cf-abacus/issues/38*
<https://github.com/cloudfoundry-incubator/cf-abacus/issues/38>

Thoughts?

- Jean-Sebastien

On Wed, Oct 7, 2015 at 5:51 PM, Piotr Przybylski <*piotrp(a)us.ibm.com*
<piotrp(a)us.ibm.com>> wrote:
-
- Hi Sebastien,

> That OAuth token should include:
> - a user id uniquely identifying that resource provider;
> - an OAuth scope named like abacus.usage.<resource_id>.write

What kind of customization of the above do you plan to expose?
In some cases it may not be possible or viable to create new scope for each
resource id e.g. short lived resources. The ability to either configure
scope to use for validation or provide scope 'mapping' would help to adapt
it to existing deployments. Some flexibility would also help to accommodate
changes related to grouping resources by type as discussed in [1].

[1] -
*https://github.com/cloudfoundry-incubator/cf-abacus/issues/38*
<https://github.com/cloudfoundry-incubator/cf-abacus/issues/38>


Piotr



[image: Inactive hide details for Jean-Sebastien Delfino
---10/07/2015 12:30:05 AM---Hi Piotr, > what kind of authorization is
required]Jean-Sebastien Delfino ---10/07/2015 12:30:05 AM---Hi
Piotr, > what kind of authorization is required to submit usage to Abacus ?

From: Jean-Sebastien Delfino <*jsdelfino(a)gmail.com*
<jsdelfino(a)gmail.com>>
To: "Discussions about Cloud Foundry projects and the system
overall." <*cf-dev(a)lists.cloudfoundry.org*
<cf-dev(a)lists.cloudfoundry.org>>
Date: 10/07/2015 12:30 AM
Subject: [cf-dev] Re: [abacus] Usage submission authorization
------------------------------




Hi Piotr,

> what kind of authorization is required to submit usage to
Abacus ?
> Is the oauth token used for submission [1] required to have
particular scope, specific to resource or resource provider ?

A resource provider is expected to present an OAuth token with
the usage it submits for a (service or runtime) resource.

That OAuth token should include:
- a user id uniquely identifying that resource provider;
- an OAuth scope named like abacus.usage.<resource_id>.write.

The precise naming syntax for that scope may still evolve in the
next few days as we progress with the implementation of user story
101703426 [1].

> Is there a different scope required to submit runtimes usage
(like cf bridge) versus other services or its possible to use single scope
for all the submissions

I'd like to handle runtimes and services consistently as they're
basically just different types of resources, i.e. one scope per 'service'
resource, one scope per 'runtime' resource.

We're still working on the detailed design and implementation,
but I'm not sure we'd want to share scopes across (service and runtime)
resource providers as that'd allow a resource provider to submit usage for
resources owned by another...

@assk / @sasrin, anything I missed? Thoughts?

-- Jean-Sebastien


On Tue, Oct 6, 2015 at 6:29 PM, Piotr Przybylski <
*piotrp(a)us.ibm.com* <piotrp(a)us.ibm.com>> wrote:
-
-
-
- Hi,
what kind of authorization is required to submit
usage to Abacus ?
Is the oauth token used for submission [1] required
to have particular scope, specific to resource or resource provider ? Is
there a different scope required to submit runtimes usage (like cf bridge)
versus other services or its possible to use single scope for all the
submissions ?


[1] -
*https://www.pivotaltracker.com/story/show/101703426*
<https://www.pivotaltracker.com/story/show/101703426>

Piotr