----- Original message -----
From: Saravanakumar A Srinivasan/Burlingame/IBM@IBMUS
To: <cf-dev@...>
Cc:
Subject: [cf-dev] Re: Re: Re: [cf-dev][abacus] Usage submission authorization
Date: Thu, Oct 15, 2015 7:06 PM
 

We have enabled scope based authorization for REST endpoints at usage collector and usage reporting service. While we are working on using system OAuth bearer access token at internal Abacus pipeline, Submitting usage to a secured Abacus needs a OAuth bearer access token with 'abacus.usage.write' scope.

Thanks,
Saravanakumar Srinivasan (Assk),

Bay Area Lab, 1001, E Hillsdale Blvd, Ste 400, Foster City, CA - 94404.
E-mail: sasrin@...
Phone: 650 645 8251 (T/L 367-8251)

-----Jean-Sebastien Delfino <jsdelfino@...> wrote: -----

To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@...>
From: Jean-Sebastien Delfino <jsdelfino@...>
Date: 10/12/2015 07:50PM
Subject: [cf-dev] Re: Re: [cf-dev][abacus] Usage submission authorization
 

> Also, resource id is an arbitrary identifier, making it part of the scope may create quite complex names e.g. 'abacus.runtimes/node/v12-07.revision-2-buildpack-guid-a3d7ff4d-3cb1-4cc3-a855-fae98e20cf57.write.

 

Do you have a specific issue in mind with putting the resource uuid in the scope name? We have uuids all over the place in CF, in most of the APIs, the usage docs etc so I'm not sure why it'd be a problem to have one here.

 

> Any naming convention may not be generic enough, for example for my UAA instance requires the scope names to start with component using it, followed by proper name - 'bss.runtimes.abacus.<resource id>.write'.

 

Like I said before, if you can't or don't want to use a specific scope per resource, then you can use abacus.usage.write (with the same disclaimers/warnings I gave in my previous post.)

 

I must be missing something though :) ... aren't you happily using cloud_controller.write for example (or similar other CF scopes) without renaming it to <your client component>.cloud_controller.write? Why would you treat abacus.usage.write different?

 

Also, I must admit to find a bit surprising a naming convention that will tie the scope name to the client that presents it. Isn't the scope typically defined by the owner of the resource it protects instead of the client? In that case the owner of the resource is not the client component... it is the CF abacus project, hence <abacus>.usage.write. Wouldn't that make more sense?

 

Finally, I'm also not quite sure how this will work at all if for example Abacus needs to authorize resource access from multiple clients. That would have to be really dynamic then, as each new client would require Abacus to know about a new client specific naming convention (or client component name prefix in the example you gave...)

 

Now, all that being said, looks like I'm not really following how you're envisioning this to work, so do you think you could maybe submit a pull request with how you concretely propose to make that dynamic scope naming work when it includes client component names, or follows client component specific naming conventions?

 

Thanks!

 

- Jean-Sebastien

 

On Mon, Oct 12, 2015 at 5:22 PM, Piotr Przybylski <piotrp@...> wrote:

Hi Sebastien,
I am not sure why allowing resource provider to explicitly specify scope with which particular resource usage will be submitted is a problem. Just allowing to pick a name would not compromise submission security in any way. It could be done for example by adding scope name to the resource definition.

Any naming convention may not be generic enough, for example for my UAA instance requires the scope names to start with component using it, followed by proper name - 'bss.runtimes.abacus.<resource id>.write'. Also, resource id is an arbitrary identifier, making it part of the scope may create quite complex names e.g.
'abacus.runtimes/node/v12-07.revision-2-buildpack-guid-a3d7ff4d-3cb1-4cc3-a855-fae98e20cf57.write.

Piotr



Jean-Sebastien Delfino ---10/09/2015 09:38:09 PM---Hey Piotr, >>> In some cases it may not be possible or viable to create new scope for

From: Jean-Sebastien Delfino <jsdelfino@...>
To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@...>
Date: 10/09/2015 09:38 PM
Subject: [cf-dev] Re: Re: Re: Re: Re: [abacus] Usage submission authorization

 

---

Hey Piotr,

>>> In some cases it may not be possible or viable to create new scope for each resource id e.g. short lived resources.

>> Why wouldn't that be possible? What type of short-lived resources did you have in mind?

> For example experimental service version (beta) replaced by release version, usage of which may be reported and metered but not necessarily billed. 

OK, that use case makes sense to me. So, your resource is going to be available for a few hours or days. I'm assuming that to get it on board CF and meter it with Abacus you're going to run a cf create-service-broker command or cf update-service-broker, define the resource config specifying how to meter it, and store that config where your Abacus provisioning endpoint implementation can retrieve it.

To secure the submission of usage for it, if I understand correctly how UAA works, you'll then need to do this:
uaac client update <your service provider's client id> --authorities "... existing permissions... abacus.<your resource id>.write"

That's all...

If that's really too much of a burden (really?) compared to the other steps, you're basically looking to do *nothing* to secure that resource. You could just submit usage with the abacus.usage.write scope, but that's the equivalent of the CF cloud_controller.write scope for Abacus, close to all powers... I'd probably advise against it as that's a serious risk but that may be what you're looking for.

> The scope names may need to follow adopter specific conventions so creating scope with predefined name 'abacus.usage....' may not fit that scheme. Abacus should offer ability to adjust the scope names, otherwise submission may not be possible.

These are simple names that we expect in the token used to submit usage. They're just constants like the names of our APIs, parameters, options, fields in our JSON schemas... basically the contract/interface between the Abacus user and its implementation. Not sure if there's a specific issue with that abacus naming convention or if it's just a theoretical question, but I'll be happy to discuss alternate naming conventions:

Do you have another naming convention in mind that you'd like to use?

Is there a specific issue with abacus.usage.write? Is the 'abacus' part in the name a problem?

Would you prefer to submit usage with an existing CF scope like cloud_controller.write or another of these high power scopes?
(again, I'd advise against it though...)

- Jean-Sebastien

- Jean-Sebastien

On Thu, Oct 8, 2015 at 5:24 PM, Piotr Przybylski <piotrp@...> wrote:

• Hi Sebastien,
  
  >> In some cases it may not be possible or viable to create new scope for each resource id e.g. short lived resources.
  
  >Why wouldn't that be possible? What type of short-lived resources did you have in mind?
  
  For example experimental service version (beta) replaced by release version, usage of which may be reported and metered but not necessarily billed.
  The scope names may need to follow adopter specific conventions so creating scope with predefined name 'abacus.usage....' may not fit that scheme. Abacus should offer ability to adjust the scope names, otherwise submission may not be possible.
  
  
  > Another reason why I'm not sure about short lived resources, is that although you may decide to stop offering a type a resource at some point, once you've metered it, and sent a bill for it >to a customer, I don't think you can really 'forget' about its existence anymore... So in that sense I'm not sure how it can be 'short lived'.
  The short lived resource is only for submission, once it is not offered, its specific scope is not needed. Thad does not mean erasing history of usage.
  
  
  Piotr
  
  
  
  
  Jean-Sebastien Delfino ---10/08/2015 11:10:16 AM---Hi Piotr, > In some cases it may not be possible or viable to create new scope for
  
  From: Jean-Sebastien Delfino <jsdelfino@...>
  To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@...>
  Date: 10/08/2015 11:10 AM
  Subject: [cf-dev] Re: Re: Re: [abacus] Usage submission authorization

   

  ---

  
  
  
  Hi Piotr,
  
  > In some cases it may not be possible or viable to create new scope for each resource id e.g. short lived resources.
  
  Why wouldn't that be possible? What type of short-lived resources did you have in mind?
  
  The typical use case I've seen is for a Cloud platform to decide to offer a new type of database or analytics or messaging service, or a new type of runtime for example. Before that new resource is offered on the platform, their resource provider needs to get on board, get a user id, auth credentials defined in UAA etc... You probably also need to define how you're going to meter that new resource and the pricing for it.
  
  Couldn't a scope be created in UAA at that time along all these other on boarding steps?
  
  Another reason why I'm not sure about short lived resources, is that although you may decide to stop offering a type a resource at some point, once you've metered it, and sent a bill for it to a customer, I don't think you can really 'forget' about its existence anymore... So in that sense I'm not sure how it can be 'short lived'.
  
  > Some flexibility would also help to accommodate changes related to grouping resources by type as discussed in [1].
  
  We discussed two options in [1]:
  a) support a resource_type in addition to resource_id for grouping many resource_ids under a single type
  b) a common resource_id for several resources (something like 'node' for all your versions of Node.js build packs for example) 
  
  Since option (a) is not implemented at this point and Issue #38 is actually assigned to a 'future' milestone, AIUI resource providers need to use option (b) with a common resource_id for multiple resources. Is creating a scope for that common id still too much of a burden then?
  
  [1] - https://github.com/cloudfoundry-incubator/cf-abacus/issues/38
  
  Thoughts?
  
  - Jean-Sebastien
  
  On Wed, Oct 7, 2015 at 5:51 PM, Piotr Przybylski <piotrp@...> wrote:
  • • Hi Sebastien,
      
      > That OAuth token should include:
      > - a user id uniquely identifying that resource provider;
      > - an OAuth scope named like abacus.usage.<resource_id>.write
      
      What kind of customization of the above do you plan to expose? In some cases it may not be possible or viable to create new scope for each resource id e.g. short lived resources. The ability to either configure scope to use for validation or provide scope 'mapping' would help to adapt it to existing deployments. Some flexibility would also help to accommodate changes related to grouping resources by type as discussed in [1].
      
      [1] - https://github.com/cloudfoundry-incubator/cf-abacus/issues/38
      
      
      Piotr
      
      
      
      Jean-Sebastien Delfino ---10/07/2015 12:30:05 AM---Hi Piotr, > what kind of authorization is required to submit usage to Abacus ?
      
      From: Jean-Sebastien Delfino <jsdelfino@...>
      To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@...>
      Date: 10/07/2015 12:30 AM
      Subject: [cf-dev] Re: [abacus] Usage submission authorization

      ---

      
      
      
      
      Hi Piotr,
      
      > what kind of authorization is required to submit usage to Abacus ? 
      > Is the oauth token used for submission [1] required to have particular scope, specific to resource or resource provider ?
      
      A resource provider is expected to present an OAuth token with the usage it submits for a (service or runtime) resource.
      
      That OAuth token should include:
      - a user id uniquely identifying that resource provider;
      - an OAuth scope named like abacus.usage.<resource_id>.write.
      
      The precise naming syntax for that scope may still evolve in the next few days as we progress with the implementation of user story 101703426 [1].
      
      > Is there a different scope required to submit runtimes usage (like cf bridge) versus other services or its possible to use single scope for all the submissions
      
      I'd like to handle runtimes and services consistently as they're basically just different types of resources, i.e. one scope per 'service' resource, one scope per 'runtime' resource.
      
      We're still working on the detailed design and implementation, but I'm not sure we'd want to share scopes across (service and runtime) resource providers as that'd allow a resource provider to submit usage for resources owned by another...
      
      @assk / @sasrin, anything I missed? Thoughts?
      
      -- Jean-Sebastien
      
      
      On Tue, Oct 6, 2015 at 6:29 PM, Piotr Przybylski <piotrp@...> wrote:
      • • • • Hi,
              what kind of authorization is required to submit usage to Abacus ?
              Is the oauth token used for submission [1] required to have particular scope, specific to resource or resource provider ? Is there a different scope required to submit runtimes usage (like cf bridge) versus other services or its possible to use single scope for all the submissions ?
              
              
              [1] - https://www.pivotaltracker.com/story/show/101703426
              
              Piotr

   

   

 

 

 

Assk,
can you confirm that the same scope (abacus.usage.write) is sufficient to
retrieve usage ?

Piotr




----- Original message -----
From: Saravanakumar A Srinivasan/Burlingame/IBM(a)IBMUS
To: <cf-dev(a)lists.cloudfoundry.org>
Cc:
Subject: [cf-dev] Re: Re: Re: [cf-dev][abacus] Usage submission
authorization
Date: Thu, Oct 15, 2015 7:06 PM

We have enabled scope based authorization for REST endpoints at usage
collector and usage reporting service. While we are working on using system
OAuth bearer access token at internal Abacus pipeline, Submitting usage to
a secured Abacus needs a OAuth bearer access token with
'abacus.usage.write' scope.

Thanks,
Saravanakumar Srinivasan (Assk),

Bay Area Lab, 1001, E Hillsdale Blvd, Ste 400, Foster City, CA - 94404.
E-mail: sasrin(a)us.ibm.com
Phone: 650 645 8251 (T/L 367-8251)


-----Jean-Sebastien Delfino <jsdelfino(a)gmail.com> wrote: -----
To: "Discussions about Cloud Foundry projects and the system overall." <
cf-dev(a)lists.cloudfoundry.org>
From: Jean-Sebastien Delfino <jsdelfino(a)gmail.com>
Date: 10/12/2015 07:50PM
Subject: [cf-dev] Re: Re: [cf-dev][abacus] Usage submission authorization

Also, resource id is an arbitrary identifier, making it part of the

scope may create quite complex names e.g.
'abacus.runtimes/node/v12-07.revision-2-buildpack-guid-a3d7ff4d-3cb1-4cc3-a855-fae98e20cf57.write.

Do you have a specific issue in mind with putting the resource uuid in the
scope name? We have uuids all over the place in CF, in most of the APIs,
the usage docs etc so I'm not sure why it'd be a problem to have one here.

Any naming convention may not be generic enough, for example for my UAA

instance requires the scope names to start with component using it,
followed by proper name - 'bss.runtimes.abacus.<resource id>.write'.

Like I said before, if you can't or don't want to use a specific scope per
resource, then you can use abacus.usage.write (with the same
disclaimers/warnings I gave in my previous post.)

I must be missing something though :) ... aren't you happily using
cloud_controller.write for example (or similar other CF scopes) without
renaming it to <your client component>.cloud_controller.write? Why would
you treat abacus.usage.write different?

Also, I must admit to find a bit surprising a naming convention that will
tie the scope name to the client that presents it. Isn't the scope
typically defined by the owner of the resource it protects instead of the
client? In that case the owner of the resource is not the client
component... it is the CF abacus project, hence <abacus>.usage.write.
Wouldn't that make more sense?

Finally, I'm also not quite sure how this will work at all if for example
Abacus needs to authorize resource access from multiple clients. That would
have to be really dynamic then, as each new client would require Abacus to
know about a new client specific naming convention (or client component
name prefix in the example you gave...)

Now, all that being said, looks like I'm not really following how you're
envisioning this to work, so do you think you could maybe submit a pull
request with how you concretely propose to make that dynamic scope naming
work when it includes client component names, or follows client component
specific naming conventions?

Thanks!

- Jean-Sebastien

On Mon, Oct 12, 2015 at 5:22 PM, Piotr Przybylski <piotrp(a)us.ibm.com>
wrote:

Hi Sebastien,
I am not sure why allowing resource provider to explicitly specify scope
with which particular resource usage will be submitted is a problem. Just
allowing to pick a name would not compromise submission security in any
way. It could be done for example by adding scope name to the resource
definition.

Any naming convention may not be generic enough, for example for my UAA
instance requires the scope names to start with component using it,
followed by proper name - 'bss.runtimes.abacus.<resource id>.write'. Also,
resource id is an arbitrary identifier, making it part of the scope may
create quite complex names e.g.
'abacus.runtimes/node/v12-07.revision-2-buildpack-guid-a3d7ff4d-3cb1-4cc3-a855-fae98e20cf57.write.


Piotr



[image: Inactive hide details for Jean-Sebastien Delfino ---10/09/2015
09:38:09 PM---Hey Piotr, >>> In some cases it may not be possibl]Jean-Sebastien
Delfino ---10/09/2015 09:38:09 PM---Hey Piotr, >>> In some cases it may not
be possible or viable to create new scope for

From: Jean-Sebastien Delfino <jsdelfino(a)gmail.com>
To: "Discussions about Cloud Foundry projects and the system overall." <
cf-dev(a)lists.cloudfoundry.org>
Date: 10/09/2015 09:38 PM
Subject: [cf-dev] Re: Re: Re: Re: Re: [abacus] Usage submission
authorization

------------------------------



Hey Piotr,

In some cases it may not be possible or viable to create new scope for

each resource id e.g. short lived resources.

Why wouldn't that be possible? What type of short-lived resources did

you have in mind?

For example experimental service version (beta) replaced by release

version, usage of which may be reported and metered but not necessarily
billed.

OK, that use case makes sense to me. So, your resource is going to be
available for a few hours or days. I'm assuming that to get it on board CF
and meter it with Abacus you're going to run a cf create-service-broker
command or cf update-service-broker, define the resource config specifying
how to meter it, and store that config where your Abacus provisioning
endpoint implementation can retrieve it.

To secure the submission of usage for it, if I understand correctly how
UAA works, you'll then need to do this:
uaac client update <your service provider's client id> --authorities "...
existing permissions... abacus.<your resource id>.write"

That's all...

If that's really too much of a burden (really?) compared to the other
steps, you're basically looking to do *nothing* to secure that resource.
You could just submit usage with the abacus.usage.write scope, but that's
the equivalent of the CF cloud_controller.write scope for Abacus, close to
all powers... I'd probably advise against it as that's a serious risk but
that may be what you're looking for.

The scope names may need to follow adopter specific conventions so

creating scope with predefined name 'abacus.usage....' may not fit that
scheme. Abacus should offer ability to adjust the scope names, otherwise
submission may not be possible.

These are simple names that we expect in the token used to submit usage.
They're just constants like the names of our APIs, parameters, options,
fields in our JSON schemas... basically the contract/interface between the
Abacus user and its implementation. Not sure if there's a specific issue
with that abacus naming convention or if it's just a theoretical question,
but I'll be happy to discuss alternate naming conventions:

Do you have another naming convention in mind that you'd like to use?

Is there a specific issue with abacus.usage.write? Is the 'abacus' part in
the name a problem?

Would you prefer to submit usage with an existing CF scope like
cloud_controller.write or another of these high power scopes?
(again, I'd advise against it though...)

- Jean-Sebastien

- Jean-Sebastien

On Thu, Oct 8, 2015 at 5:24 PM, Piotr Przybylski <*piotrp(a)us.ibm.com*
<piotrp(a)us.ibm.com>> wrote:

- Hi Sebastien,

>> In some cases it may not be possible or viable to create new scope
for each resource id e.g. short lived resources.

>Why wouldn't that be possible? What type of short-lived resources did
you have in mind?

For example experimental service version (beta) replaced by release
version, usage of which may be reported and metered but not necessarily
billed.
The scope names may need to follow adopter specific conventions so
creating scope with predefined name 'abacus.usage....' may not fit
that scheme. Abacus should offer ability to adjust the scope names,
otherwise submission may not be possible.


> Another reason why I'm not sure about short lived resources, is that
although you may decide to stop offering a type a resource at some point,
once you've metered it, and sent a bill for it >to a customer, I don't
think you can really 'forget' about its existence anymore... So in that
sense I'm not sure how it can be 'short lived'.
The short lived resource is only for submission, once it is not
offered, its specific scope is not needed. Thad does not mean erasing
history of usage.


Piotr




[image: Inactive hide details for Jean-Sebastien Delfino ---10/08/2015
11:10:16 AM---Hi Piotr, > In some cases it may not be possible o]Jean-Sebastien
Delfino ---10/08/2015 11:10:16 AM---Hi Piotr, > In some cases it may not be
possible or viable to create new scope for

From: Jean-Sebastien Delfino <*jsdelfino(a)gmail.com*
<jsdelfino(a)gmail.com>>
To: "Discussions about Cloud Foundry projects and the system overall."
<*cf-dev(a)lists.cloudfoundry.org* <cf-dev(a)lists.cloudfoundry.org>>
Date: 10/08/2015 11:10 AM
Subject: [cf-dev] Re: Re: Re: [abacus] Usage submission authorization


------------------------------



Hi Piotr,

> In some cases it may not be possible or viable to create new scope
for each resource id e.g. short lived resources.

Why wouldn't that be possible? What type of short-lived resources did
you have in mind?

The typical use case I've seen is for a Cloud platform to decide to
offer a new type of database or analytics or messaging service, or a new
type of runtime for example. Before that new resource is offered on the
platform, their resource provider needs to get on board, get a user id,
auth credentials defined in UAA etc... You probably also need to define how
you're going to meter that new resource and the pricing for it.

Couldn't a scope be created in UAA at that time along all these other
on boarding steps?

Another reason why I'm not sure about short lived resources, is that
although you may decide to stop offering a type a resource at some point,
once you've metered it, and sent a bill for it to a customer, I don't think
you can really 'forget' about its existence anymore... So in that sense I'm
not sure how it can be 'short lived'.

> Some flexibility would also help to accommodate changes related to
grouping resources by type as discussed in [1].

We discussed two options in [1]:
a) support a resource_type in addition to resource_id for grouping
many resource_ids under a single type
b) a common resource_id for several resources (something like 'node'
for all your versions of Node.js build packs for example)

Since option (a) is not implemented at this point and Issue #38 is
actually assigned to a 'future' milestone, AIUI resource providers need to
use option (b) with a common resource_id for multiple resources. Is
creating a scope for that common id still too much of a burden then?

[1] - *https://github.com/cloudfoundry-incubator/cf-abacus/issues/38*
<https://github.com/cloudfoundry-incubator/cf-abacus/issues/38>

Thoughts?

- Jean-Sebastien

On Wed, Oct 7, 2015 at 5:51 PM, Piotr Przybylski <*piotrp(a)us.ibm.com*
<piotrp(a)us.ibm.com>> wrote:
-
- Hi Sebastien,

> That OAuth token should include:
> - a user id uniquely identifying that resource provider;
> - an OAuth scope named like abacus.usage.<resource_id>.write

What kind of customization of the above do you plan to expose?
In some cases it may not be possible or viable to create new scope for each
resource id e.g. short lived resources. The ability to either configure
scope to use for validation or provide scope 'mapping' would help to adapt
it to existing deployments. Some flexibility would also help to accommodate
changes related to grouping resources by type as discussed in [1].

[1] -
*https://github.com/cloudfoundry-incubator/cf-abacus/issues/38*
<https://github.com/cloudfoundry-incubator/cf-abacus/issues/38>


Piotr



[image: Inactive hide details for Jean-Sebastien Delfino
---10/07/2015 12:30:05 AM---Hi Piotr, > what kind of authorization is
required]Jean-Sebastien Delfino ---10/07/2015 12:30:05 AM---Hi
Piotr, > what kind of authorization is required to submit usage to Abacus ?

From: Jean-Sebastien Delfino <*jsdelfino(a)gmail.com*
<jsdelfino(a)gmail.com>>
To: "Discussions about Cloud Foundry projects and the system
overall." <*cf-dev(a)lists.cloudfoundry.org*
<cf-dev(a)lists.cloudfoundry.org>>
Date: 10/07/2015 12:30 AM
Subject: [cf-dev] Re: [abacus] Usage submission authorization
------------------------------




Hi Piotr,

> what kind of authorization is required to submit usage to
Abacus ?
> Is the oauth token used for submission [1] required to have
particular scope, specific to resource or resource provider ?

A resource provider is expected to present an OAuth token with
the usage it submits for a (service or runtime) resource.

That OAuth token should include:
- a user id uniquely identifying that resource provider;
- an OAuth scope named like abacus.usage.<resource_id>.write.

The precise naming syntax for that scope may still evolve in the
next few days as we progress with the implementation of user story
101703426 [1].

> Is there a different scope required to submit runtimes usage
(like cf bridge) versus other services or its possible to use single scope
for all the submissions

I'd like to handle runtimes and services consistently as they're
basically just different types of resources, i.e. one scope per 'service'
resource, one scope per 'runtime' resource.

We're still working on the detailed design and implementation,
but I'm not sure we'd want to share scopes across (service and runtime)
resource providers as that'd allow a resource provider to submit usage for
resources owned by another...

@assk / @sasrin, anything I missed? Thoughts?

-- Jean-Sebastien


On Tue, Oct 6, 2015 at 6:29 PM, Piotr Przybylski <
*piotrp(a)us.ibm.com* <piotrp(a)us.ibm.com>> wrote:
-
-
-
- Hi,
what kind of authorization is required to submit
usage to Abacus ?
Is the oauth token used for submission [1] required
to have particular scope, specific to resource or resource provider ? Is
there a different scope required to submit runtimes usage (like cf bridge)
versus other services or its possible to use single scope for all the
submissions ?


[1] -
*https://www.pivotaltracker.com/story/show/101703426*
<https://www.pivotaltracker.com/story/show/101703426>

Piotr

----- Original message -----
From: Saravanakumar A Srinivasan/Burlingame/IBM@IBMUS
To: <cf-dev@...>
Cc:
Subject: [cf-dev] Re: Re: Re: [cf-dev][abacus] Usage submission authorization
Date: Thu, Oct 15, 2015 7:06 PM
 

We have enabled scope based authorization for REST endpoints at usage collector and usage reporting service. While we are working on using system OAuth bearer access token at internal Abacus pipeline, Submitting usage to a secured Abacus needs a OAuth bearer access token with 'abacus.usage.write' scope.

Thanks,
Saravanakumar Srinivasan (Assk),

Bay Area Lab, 1001, E Hillsdale Blvd, Ste 400, Foster City, CA - 94404.
E-mail: sasrin@...
Phone: 650 645 8251 (T/L 367-8251)

-----Jean-Sebastien Delfino <jsdelfino@...> wrote: -----

To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@...>
From: Jean-Sebastien Delfino <jsdelfino@...>
Date: 10/12/2015 07:50PM
Subject: [cf-dev] Re: Re: [cf-dev][abacus] Usage submission authorization
 

> Also, resource id is an arbitrary identifier, making it part of the scope may create quite complex names e.g. 'abacus.runtimes/node/v12-07.revision-2-buildpack-guid-a3d7ff4d-3cb1-4cc3-a855-fae98e20cf57.write.

 

Do you have a specific issue in mind with putting the resource uuid in the scope name? We have uuids all over the place in CF, in most of the APIs, the usage docs etc so I'm not sure why it'd be a problem to have one here.

 

> Any naming convention may not be generic enough, for example for my UAA instance requires the scope names to start with component using it, followed by proper name - 'bss.runtimes.abacus.<resource id>.write'.

 

Like I said before, if you can't or don't want to use a specific scope per resource, then you can use abacus.usage.write (with the same disclaimers/warnings I gave in my previous post.)

 

I must be missing something though :) ... aren't you happily using cloud_controller.write for example (or similar other CF scopes) without renaming it to <your client component>.cloud_controller.write? Why would you treat abacus.usage.write different?

 

Also, I must admit to find a bit surprising a naming convention that will tie the scope name to the client that presents it. Isn't the scope typically defined by the owner of the resource it protects instead of the client? In that case the owner of the resource is not the client component... it is the CF abacus project, hence <abacus>.usage.write. Wouldn't that make more sense?

 

Finally, I'm also not quite sure how this will work at all if for example Abacus needs to authorize resource access from multiple clients. That would have to be really dynamic then, as each new client would require Abacus to know about a new client specific naming convention (or client component name prefix in the example you gave...)

 

Now, all that being said, looks like I'm not really following how you're envisioning this to work, so do you think you could maybe submit a pull request with how you concretely propose to make that dynamic scope naming work when it includes client component names, or follows client component specific naming conventions?

 

Thanks!

 

- Jean-Sebastien

 

On Mon, Oct 12, 2015 at 5:22 PM, Piotr Przybylski <piotrp@...> wrote:

Hi Sebastien,
I am not sure why allowing resource provider to explicitly specify scope with which particular resource usage will be submitted is a problem. Just allowing to pick a name would not compromise submission security in any way. It could be done for example by adding scope name to the resource definition.

Any naming convention may not be generic enough, for example for my UAA instance requires the scope names to start with component using it, followed by proper name - 'bss.runtimes.abacus.<resource id>.write'. Also, resource id is an arbitrary identifier, making it part of the scope may create quite complex names e.g.
'abacus.runtimes/node/v12-07.revision-2-buildpack-guid-a3d7ff4d-3cb1-4cc3-a855-fae98e20cf57.write.

Piotr



Jean-Sebastien Delfino ---10/09/2015 09:38:09 PM---Hey Piotr, >>> In some cases it may not be possible or viable to create new scope for

From: Jean-Sebastien Delfino <jsdelfino@...>
To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@...>
Date: 10/09/2015 09:38 PM
Subject: [cf-dev] Re: Re: Re: Re: Re: [abacus] Usage submission authorization

 

---

Hey Piotr,

>>> In some cases it may not be possible or viable to create new scope for each resource id e.g. short lived resources.

>> Why wouldn't that be possible? What type of short-lived resources did you have in mind?

> For example experimental service version (beta) replaced by release version, usage of which may be reported and metered but not necessarily billed. 

OK, that use case makes sense to me. So, your resource is going to be available for a few hours or days. I'm assuming that to get it on board CF and meter it with Abacus you're going to run a cf create-service-broker command or cf update-service-broker, define the resource config specifying how to meter it, and store that config where your Abacus provisioning endpoint implementation can retrieve it.

To secure the submission of usage for it, if I understand correctly how UAA works, you'll then need to do this:
uaac client update <your service provider's client id> --authorities "... existing permissions... abacus.<your resource id>.write"

That's all...

If that's really too much of a burden (really?) compared to the other steps, you're basically looking to do *nothing* to secure that resource. You could just submit usage with the abacus.usage.write scope, but that's the equivalent of the CF cloud_controller.write scope for Abacus, close to all powers... I'd probably advise against it as that's a serious risk but that may be what you're looking for.

> The scope names may need to follow adopter specific conventions so creating scope with predefined name 'abacus.usage....' may not fit that scheme. Abacus should offer ability to adjust the scope names, otherwise submission may not be possible.

These are simple names that we expect in the token used to submit usage. They're just constants like the names of our APIs, parameters, options, fields in our JSON schemas... basically the contract/interface between the Abacus user and its implementation. Not sure if there's a specific issue with that abacus naming convention or if it's just a theoretical question, but I'll be happy to discuss alternate naming conventions:

Do you have another naming convention in mind that you'd like to use?

Is there a specific issue with abacus.usage.write? Is the 'abacus' part in the name a problem?

Would you prefer to submit usage with an existing CF scope like cloud_controller.write or another of these high power scopes?
(again, I'd advise against it though...)

- Jean-Sebastien

- Jean-Sebastien

On Thu, Oct 8, 2015 at 5:24 PM, Piotr Przybylski <piotrp@...> wrote:

• Hi Sebastien,
  
  >> In some cases it may not be possible or viable to create new scope for each resource id e.g. short lived resources.
  
  >Why wouldn't that be possible? What type of short-lived resources did you have in mind?
  
  For example experimental service version (beta) replaced by release version, usage of which may be reported and metered but not necessarily billed.
  The scope names may need to follow adopter specific conventions so creating scope with predefined name 'abacus.usage....' may not fit that scheme. Abacus should offer ability to adjust the scope names, otherwise submission may not be possible.
  
  
  > Another reason why I'm not sure about short lived resources, is that although you may decide to stop offering a type a resource at some point, once you've metered it, and sent a bill for it >to a customer, I don't think you can really 'forget' about its existence anymore... So in that sense I'm not sure how it can be 'short lived'.
  The short lived resource is only for submission, once it is not offered, its specific scope is not needed. Thad does not mean erasing history of usage.
  
  
  Piotr
  
  
  
  
  Jean-Sebastien Delfino ---10/08/2015 11:10:16 AM---Hi Piotr, > In some cases it may not be possible or viable to create new scope for
  
  From: Jean-Sebastien Delfino <jsdelfino@...>
  To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@...>
  Date: 10/08/2015 11:10 AM
  Subject: [cf-dev] Re: Re: Re: [abacus] Usage submission authorization

   

  ---

  
  
  
  Hi Piotr,
  
  > In some cases it may not be possible or viable to create new scope for each resource id e.g. short lived resources.
  
  Why wouldn't that be possible? What type of short-lived resources did you have in mind?
  
  The typical use case I've seen is for a Cloud platform to decide to offer a new type of database or analytics or messaging service, or a new type of runtime for example. Before that new resource is offered on the platform, their resource provider needs to get on board, get a user id, auth credentials defined in UAA etc... You probably also need to define how you're going to meter that new resource and the pricing for it.
  
  Couldn't a scope be created in UAA at that time along all these other on boarding steps?
  
  Another reason why I'm not sure about short lived resources, is that although you may decide to stop offering a type a resource at some point, once you've metered it, and sent a bill for it to a customer, I don't think you can really 'forget' about its existence anymore... So in that sense I'm not sure how it can be 'short lived'.
  
  > Some flexibility would also help to accommodate changes related to grouping resources by type as discussed in [1].
  
  We discussed two options in [1]:
  a) support a resource_type in addition to resource_id for grouping many resource_ids under a single type
  b) a common resource_id for several resources (something like 'node' for all your versions of Node.js build packs for example) 
  
  Since option (a) is not implemented at this point and Issue #38 is actually assigned to a 'future' milestone, AIUI resource providers need to use option (b) with a common resource_id for multiple resources. Is creating a scope for that common id still too much of a burden then?
  
  [1] - https://github.com/cloudfoundry-incubator/cf-abacus/issues/38
  
  Thoughts?
  
  - Jean-Sebastien
  
  On Wed, Oct 7, 2015 at 5:51 PM, Piotr Przybylski <piotrp@...> wrote:
  • • Hi Sebastien,
      
      > That OAuth token should include:
      > - a user id uniquely identifying that resource provider;
      > - an OAuth scope named like abacus.usage.<resource_id>.write
      
      What kind of customization of the above do you plan to expose? In some cases it may not be possible or viable to create new scope for each resource id e.g. short lived resources. The ability to either configure scope to use for validation or provide scope 'mapping' would help to adapt it to existing deployments. Some flexibility would also help to accommodate changes related to grouping resources by type as discussed in [1].
      
      [1] - https://github.com/cloudfoundry-incubator/cf-abacus/issues/38
      
      
      Piotr
      
      
      
      Jean-Sebastien Delfino ---10/07/2015 12:30:05 AM---Hi Piotr, > what kind of authorization is required to submit usage to Abacus ?
      
      From: Jean-Sebastien Delfino <jsdelfino@...>
      To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@...>
      Date: 10/07/2015 12:30 AM
      Subject: [cf-dev] Re: [abacus] Usage submission authorization

      ---

      
      
      
      
      Hi Piotr,
      
      > what kind of authorization is required to submit usage to Abacus ? 
      > Is the oauth token used for submission [1] required to have particular scope, specific to resource or resource provider ?
      
      A resource provider is expected to present an OAuth token with the usage it submits for a (service or runtime) resource.
      
      That OAuth token should include:
      - a user id uniquely identifying that resource provider;
      - an OAuth scope named like abacus.usage.<resource_id>.write.
      
      The precise naming syntax for that scope may still evolve in the next few days as we progress with the implementation of user story 101703426 [1].
      
      > Is there a different scope required to submit runtimes usage (like cf bridge) versus other services or its possible to use single scope for all the submissions
      
      I'd like to handle runtimes and services consistently as they're basically just different types of resources, i.e. one scope per 'service' resource, one scope per 'runtime' resource.
      
      We're still working on the detailed design and implementation, but I'm not sure we'd want to share scopes across (service and runtime) resource providers as that'd allow a resource provider to submit usage for resources owned by another...
      
      @assk / @sasrin, anything I missed? Thoughts?
      
      -- Jean-Sebastien
      
      
      On Tue, Oct 6, 2015 at 6:29 PM, Piotr Przybylski <piotrp@...> wrote:
      • • • • Hi,
              what kind of authorization is required to submit usage to Abacus ?
              Is the oauth token used for submission [1] required to have particular scope, specific to resource or resource provider ? Is there a different scope required to submit runtimes usage (like cf bridge) versus other services or its possible to use single scope for all the submissions ?
              
              
              [1] - https://www.pivotaltracker.com/story/show/101703426
              
              Piotr

   

   

 

 

 

----- Original message -----
From: Jean-Sebastien Delfino <jsdelfino@...>
To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@...>
Cc:
Subject: [cf-dev] Re: Re: Re: Re: Re: [cf-dev][abacus] Usage submission authorization
Date: Thu, Oct 15, 2015 9:11 PM
 

Hey Piotr,

 

To read usage I believe you'll need 'abacus.usage.read', as 'abacus.usage.write' is for, well... writing.

 

P.S. That reminds me of a period of my life long time ago when I was a contractor for some big company and they had hired me to write code for them but had not given me the authorization to read the confidential code I was writing :)

 

- Jean-Sebastien

 

On Thu, Oct 15, 2015 at 7:28 PM, Piotr Przybylski <piotrp@...> wrote:

Assk,

can you confirm that the same scope (abacus.usage.write) is sufficient to retrieve usage ? 

Piotr

 

 

----- Original message -----
From: Saravanakumar A Srinivasan/Burlingame/IBM@IBMUS
To: <cf-dev@...>
Cc:
Subject: [cf-dev] Re: Re: Re: [cf-dev][abacus] Usage submission authorization
Date: Thu, Oct 15, 2015 7:06 PM
 

We have enabled scope based authorization for REST endpoints at usage collector and usage reporting service. While we are working on using system OAuth bearer access token at internal Abacus pipeline, Submitting usage to a secured Abacus needs a OAuth bearer access token with 'abacus.usage.write' scope.

Thanks,
Saravanakumar Srinivasan (Assk),

Bay Area Lab, 1001, E Hillsdale Blvd, Ste 400, Foster City, CA - 94404.
E-mail: sasrin@...
Phone: 650 645 8251 (T/L 367-8251)

-----Jean-Sebastien Delfino <jsdelfino@...> wrote: -----

To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@...>
From: Jean-Sebastien Delfino <jsdelfino@...>
Date: 10/12/2015 07:50PM
Subject: [cf-dev] Re: Re: [cf-dev][abacus] Usage submission authorization
 

> Also, resource id is an arbitrary identifier, making it part of the scope may create quite complex names e.g. 'abacus.runtimes/node/v12-07.revision-2-buildpack-guid-a3d7ff4d-3cb1-4cc3-a855-fae98e20cf57.write.

 

Do you have a specific issue in mind with putting the resource uuid in the scope name? We have uuids all over the place in CF, in most of the APIs, the usage docs etc so I'm not sure why it'd be a problem to have one here.

 

> Any naming convention may not be generic enough, for example for my UAA instance requires the scope names to start with component using it, followed by proper name - 'bss.runtimes.abacus.<resource id>.write'.

 

Like I said before, if you can't or don't want to use a specific scope per resource, then you can use abacus.usage.write (with the same disclaimers/warnings I gave in my previous post.)

 

I must be missing something though :) ... aren't you happily using cloud_controller.write for example (or similar other CF scopes) without renaming it to <your client component>.cloud_controller.write? Why would you treat abacus.usage.write different?

 

Also, I must admit to find a bit surprising a naming convention that will tie the scope name to the client that presents it. Isn't the scope typically defined by the owner of the resource it protects instead of the client? In that case the owner of the resource is not the client component... it is the CF abacus project, hence <abacus>.usage.write. Wouldn't that make more sense?

 

Finally, I'm also not quite sure how this will work at all if for example Abacus needs to authorize resource access from multiple clients. That would have to be really dynamic then, as each new client would require Abacus to know about a new client specific naming convention (or client component name prefix in the example you gave...)

 

Now, all that being said, looks like I'm not really following how you're envisioning this to work, so do you think you could maybe submit a pull request with how you concretely propose to make that dynamic scope naming work when it includes client component names, or follows client component specific naming conventions?

 

Thanks!

 

- Jean-Sebastien

 

On Mon, Oct 12, 2015 at 5:22 PM, Piotr Przybylski <piotrp@...> wrote:

Hi Sebastien,
I am not sure why allowing resource provider to explicitly specify scope with which particular resource usage will be submitted is a problem. Just allowing to pick a name would not compromise submission security in any way. It could be done for example by adding scope name to the resource definition.

Any naming convention may not be generic enough, for example for my UAA instance requires the scope names to start with component using it, followed by proper name - 'bss.runtimes.abacus.<resource id>.write'. Also, resource id is an arbitrary identifier, making it part of the scope may create quite complex names e.g.
'abacus.runtimes/node/v12-07.revision-2-buildpack-guid-a3d7ff4d-3cb1-4cc3-a855-fae98e20cf57.write.

Piotr



Jean-Sebastien Delfino ---10/09/2015 09:38:09 PM---Hey Piotr, >>> In some cases it may not be possible or viable to create new scope for

From: Jean-Sebastien Delfino <jsdelfino@...>
To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@...>
Date: 10/09/2015 09:38 PM
Subject: [cf-dev] Re: Re: Re: Re: Re: [abacus] Usage submission authorization

 

---

Hey Piotr,

>>> In some cases it may not be possible or viable to create new scope for each resource id e.g. short lived resources.

>> Why wouldn't that be possible? What type of short-lived resources did you have in mind?

> For example experimental service version (beta) replaced by release version, usage of which may be reported and metered but not necessarily billed. 

OK, that use case makes sense to me. So, your resource is going to be available for a few hours or days. I'm assuming that to get it on board CF and meter it with Abacus you're going to run a cf create-service-broker command or cf update-service-broker, define the resource config specifying how to meter it, and store that config where your Abacus provisioning endpoint implementation can retrieve it.

To secure the submission of usage for it, if I understand correctly how UAA works, you'll then need to do this:
uaac client update <your service provider's client id> --authorities "... existing permissions... abacus.<your resource id>.write"

That's all...

If that's really too much of a burden (really?) compared to the other steps, you're basically looking to do *nothing* to secure that resource. You could just submit usage with the abacus.usage.write scope, but that's the equivalent of the CF cloud_controller.write scope for Abacus, close to all powers... I'd probably advise against it as that's a serious risk but that may be what you're looking for.

> The scope names may need to follow adopter specific conventions so creating scope with predefined name 'abacus.usage....' may not fit that scheme. Abacus should offer ability to adjust the scope names, otherwise submission may not be possible.

These are simple names that we expect in the token used to submit usage. They're just constants like the names of our APIs, parameters, options, fields in our JSON schemas... basically the contract/interface between the Abacus user and its implementation. Not sure if there's a specific issue with that abacus naming convention or if it's just a theoretical question, but I'll be happy to discuss alternate naming conventions:

Do you have another naming convention in mind that you'd like to use?

Is there a specific issue with abacus.usage.write? Is the 'abacus' part in the name a problem?

Would you prefer to submit usage with an existing CF scope like cloud_controller.write or another of these high power scopes?
(again, I'd advise against it though...)

- Jean-Sebastien

- Jean-Sebastien

On Thu, Oct 8, 2015 at 5:24 PM, Piotr Przybylski <piotrp@...> wrote:

• Hi Sebastien,
  
  >> In some cases it may not be possible or viable to create new scope for each resource id e.g. short lived resources.
  
  >Why wouldn't that be possible? What type of short-lived resources did you have in mind?
  
  For example experimental service version (beta) replaced by release version, usage of which may be reported and metered but not necessarily billed.
  The scope names may need to follow adopter specific conventions so creating scope with predefined name 'abacus.usage....' may not fit that scheme. Abacus should offer ability to adjust the scope names, otherwise submission may not be possible.
  
  
  > Another reason why I'm not sure about short lived resources, is that although you may decide to stop offering a type a resource at some point, once you've metered it, and sent a bill for it >to a customer, I don't think you can really 'forget' about its existence anymore... So in that sense I'm not sure how it can be 'short lived'.
  The short lived resource is only for submission, once it is not offered, its specific scope is not needed. Thad does not mean erasing history of usage.
  
  
  Piotr
  
  
  
  
  Jean-Sebastien Delfino ---10/08/2015 11:10:16 AM---Hi Piotr, > In some cases it may not be possible or viable to create new scope for
  
  From: Jean-Sebastien Delfino <jsdelfino@...>
  To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@...>
  Date: 10/08/2015 11:10 AM
  Subject: [cf-dev] Re: Re: Re: [abacus] Usage submission authorization

   

  ---

  
  
  
  Hi Piotr,
  
  > In some cases it may not be possible or viable to create new scope for each resource id e.g. short lived resources.
  
  Why wouldn't that be possible? What type of short-lived resources did you have in mind?
  
  The typical use case I've seen is for a Cloud platform to decide to offer a new type of database or analytics or messaging service, or a new type of runtime for example. Before that new resource is offered on the platform, their resource provider needs to get on board, get a user id, auth credentials defined in UAA etc... You probably also need to define how you're going to meter that new resource and the pricing for it.
  
  Couldn't a scope be created in UAA at that time along all these other on boarding steps?
  
  Another reason why I'm not sure about short lived resources, is that although you may decide to stop offering a type a resource at some point, once you've metered it, and sent a bill for it to a customer, I don't think you can really 'forget' about its existence anymore... So in that sense I'm not sure how it can be 'short lived'.
  
  > Some flexibility would also help to accommodate changes related to grouping resources by type as discussed in [1].
  
  We discussed two options in [1]:
  a) support a resource_type in addition to resource_id for grouping many resource_ids under a single type
  b) a common resource_id for several resources (something like 'node' for all your versions of Node.js build packs for example) 
  
  Since option (a) is not implemented at this point and Issue #38 is actually assigned to a 'future' milestone, AIUI resource providers need to use option (b) with a common resource_id for multiple resources. Is creating a scope for that common id still too much of a burden then?
  
  [1] - https://github.com/cloudfoundry-incubator/cf-abacus/issues/38
  
  Thoughts?
  
  - Jean-Sebastien
  
  On Wed, Oct 7, 2015 at 5:51 PM, Piotr Przybylski <piotrp@...> wrote:
  • • Hi Sebastien,
      
      > That OAuth token should include:
      > - a user id uniquely identifying that resource provider;
      > - an OAuth scope named like abacus.usage.<resource_id>.write
      
      What kind of customization of the above do you plan to expose? In some cases it may not be possible or viable to create new scope for each resource id e.g. short lived resources. The ability to either configure scope to use for validation or provide scope 'mapping' would help to adapt it to existing deployments. Some flexibility would also help to accommodate changes related to grouping resources by type as discussed in [1].
      
      [1] - https://github.com/cloudfoundry-incubator/cf-abacus/issues/38
      
      
      Piotr
      
      
      
      Jean-Sebastien Delfino ---10/07/2015 12:30:05 AM---Hi Piotr, > what kind of authorization is required to submit usage to Abacus ?
      
      From: Jean-Sebastien Delfino <jsdelfino@...>
      To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@...>
      Date: 10/07/2015 12:30 AM
      Subject: [cf-dev] Re: [abacus] Usage submission authorization

      ---

      
      
      
      
      Hi Piotr,
      
      > what kind of authorization is required to submit usage to Abacus ? 
      > Is the oauth token used for submission [1] required to have particular scope, specific to resource or resource provider ?
      
      A resource provider is expected to present an OAuth token with the usage it submits for a (service or runtime) resource.
      
      That OAuth token should include:
      - a user id uniquely identifying that resource provider;
      - an OAuth scope named like abacus.usage.<resource_id>.write.
      
      The precise naming syntax for that scope may still evolve in the next few days as we progress with the implementation of user story 101703426 [1].
      
      > Is there a different scope required to submit runtimes usage (like cf bridge) versus other services or its possible to use single scope for all the submissions
      
      I'd like to handle runtimes and services consistently as they're basically just different types of resources, i.e. one scope per 'service' resource, one scope per 'runtime' resource.
      
      We're still working on the detailed design and implementation, but I'm not sure we'd want to share scopes across (service and runtime) resource providers as that'd allow a resource provider to submit usage for resources owned by another...
      
      @assk / @sasrin, anything I missed? Thoughts?
      
      -- Jean-Sebastien
      
      
      On Tue, Oct 6, 2015 at 6:29 PM, Piotr Przybylski <piotrp@...> wrote:
      • • • • Hi,
              what kind of authorization is required to submit usage to Abacus ?
              Is the oauth token used for submission [1] required to have particular scope, specific to resource or resource provider ? Is there a different scope required to submit runtimes usage (like cf bridge) versus other services or its possible to use single scope for all the submissions ?
              
              
              [1] - https://www.pivotaltracker.com/story/show/101703426
              
              Piotr

   

   

 

 

 

----- Original message -----
From: Jean-Sebastien Delfino <jsdelfino@...>
To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@...>
Cc:
Subject: [cf-dev] Re: Re: Re: Re: Re: [cf-dev][abacus] Usage submission authorization
Date: Thu, Oct 15, 2015 9:11 PM
 

Hey Piotr,

 

To read usage I believe you'll need 'abacus.usage.read', as 'abacus.usage.write' is for, well... writing.

 

P.S. That reminds me of a period of my life long time ago when I was a contractor for some big company and they had hired me to write code for them but had not given me the authorization to read the confidential code I was writing :)

 

- Jean-Sebastien

 

On Thu, Oct 15, 2015 at 7:28 PM, Piotr Przybylski <piotrp@...> wrote:

Assk,

can you confirm that the same scope (abacus.usage.write) is sufficient to retrieve usage ? 

Piotr

 

 

----- Original message -----
From: Saravanakumar A Srinivasan/Burlingame/IBM@IBMUS
To: <cf-dev@...>
Cc:
Subject: [cf-dev] Re: Re: Re: [cf-dev][abacus] Usage submission authorization
Date: Thu, Oct 15, 2015 7:06 PM
 

We have enabled scope based authorization for REST endpoints at usage collector and usage reporting service. While we are working on using system OAuth bearer access token at internal Abacus pipeline, Submitting usage to a secured Abacus needs a OAuth bearer access token with 'abacus.usage.write' scope.

Thanks,
Saravanakumar Srinivasan (Assk),

Bay Area Lab, 1001, E Hillsdale Blvd, Ste 400, Foster City, CA - 94404.
E-mail: sasrin@...
Phone: 650 645 8251 (T/L 367-8251)

-----Jean-Sebastien Delfino <jsdelfino@...> wrote: -----

To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@...>
From: Jean-Sebastien Delfino <jsdelfino@...>
Date: 10/12/2015 07:50PM
Subject: [cf-dev] Re: Re: [cf-dev][abacus] Usage submission authorization
 

> Also, resource id is an arbitrary identifier, making it part of the scope may create quite complex names e.g. 'abacus.runtimes/node/v12-07.revision-2-buildpack-guid-a3d7ff4d-3cb1-4cc3-a855-fae98e20cf57.write.

 

Do you have a specific issue in mind with putting the resource uuid in the scope name? We have uuids all over the place in CF, in most of the APIs, the usage docs etc so I'm not sure why it'd be a problem to have one here.

 

> Any naming convention may not be generic enough, for example for my UAA instance requires the scope names to start with component using it, followed by proper name - 'bss.runtimes.abacus.<resource id>.write'.

 

Like I said before, if you can't or don't want to use a specific scope per resource, then you can use abacus.usage.write (with the same disclaimers/warnings I gave in my previous post.)

 

I must be missing something though :) ... aren't you happily using cloud_controller.write for example (or similar other CF scopes) without renaming it to <your client component>.cloud_controller.write? Why would you treat abacus.usage.write different?

 

Also, I must admit to find a bit surprising a naming convention that will tie the scope name to the client that presents it. Isn't the scope typically defined by the owner of the resource it protects instead of the client? In that case the owner of the resource is not the client component... it is the CF abacus project, hence <abacus>.usage.write. Wouldn't that make more sense?

 

Finally, I'm also not quite sure how this will work at all if for example Abacus needs to authorize resource access from multiple clients. That would have to be really dynamic then, as each new client would require Abacus to know about a new client specific naming convention (or client component name prefix in the example you gave...)

 

Now, all that being said, looks like I'm not really following how you're envisioning this to work, so do you think you could maybe submit a pull request with how you concretely propose to make that dynamic scope naming work when it includes client component names, or follows client component specific naming conventions?

 

Thanks!

 

- Jean-Sebastien

 

On Mon, Oct 12, 2015 at 5:22 PM, Piotr Przybylski <piotrp@...> wrote:

Hi Sebastien,
I am not sure why allowing resource provider to explicitly specify scope with which particular resource usage will be submitted is a problem. Just allowing to pick a name would not compromise submission security in any way. It could be done for example by adding scope name to the resource definition.

Any naming convention may not be generic enough, for example for my UAA instance requires the scope names to start with component using it, followed by proper name - 'bss.runtimes.abacus.<resource id>.write'. Also, resource id is an arbitrary identifier, making it part of the scope may create quite complex names e.g.
'abacus.runtimes/node/v12-07.revision-2-buildpack-guid-a3d7ff4d-3cb1-4cc3-a855-fae98e20cf57.write.

Piotr



Jean-Sebastien Delfino ---10/09/2015 09:38:09 PM---Hey Piotr, >>> In some cases it may not be possible or viable to create new scope for

From: Jean-Sebastien Delfino <jsdelfino@...>
To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@...>
Date: 10/09/2015 09:38 PM
Subject: [cf-dev] Re: Re: Re: Re: Re: [abacus] Usage submission authorization

 

---

Hey Piotr,

>>> In some cases it may not be possible or viable to create new scope for each resource id e.g. short lived resources.

>> Why wouldn't that be possible? What type of short-lived resources did you have in mind?

> For example experimental service version (beta) replaced by release version, usage of which may be reported and metered but not necessarily billed. 

OK, that use case makes sense to me. So, your resource is going to be available for a few hours or days. I'm assuming that to get it on board CF and meter it with Abacus you're going to run a cf create-service-broker command or cf update-service-broker, define the resource config specifying how to meter it, and store that config where your Abacus provisioning endpoint implementation can retrieve it.

To secure the submission of usage for it, if I understand correctly how UAA works, you'll then need to do this:
uaac client update <your service provider's client id> --authorities "... existing permissions... abacus.<your resource id>.write"

That's all...

If that's really too much of a burden (really?) compared to the other steps, you're basically looking to do *nothing* to secure that resource. You could just submit usage with the abacus.usage.write scope, but that's the equivalent of the CF cloud_controller.write scope for Abacus, close to all powers... I'd probably advise against it as that's a serious risk but that may be what you're looking for.

> The scope names may need to follow adopter specific conventions so creating scope with predefined name 'abacus.usage....' may not fit that scheme. Abacus should offer ability to adjust the scope names, otherwise submission may not be possible.

These are simple names that we expect in the token used to submit usage. They're just constants like the names of our APIs, parameters, options, fields in our JSON schemas... basically the contract/interface between the Abacus user and its implementation. Not sure if there's a specific issue with that abacus naming convention or if it's just a theoretical question, but I'll be happy to discuss alternate naming conventions:

Do you have another naming convention in mind that you'd like to use?

Is there a specific issue with abacus.usage.write? Is the 'abacus' part in the name a problem?

Would you prefer to submit usage with an existing CF scope like cloud_controller.write or another of these high power scopes?
(again, I'd advise against it though...)

- Jean-Sebastien

- Jean-Sebastien

On Thu, Oct 8, 2015 at 5:24 PM, Piotr Przybylski <piotrp@...> wrote:

• Hi Sebastien,
  
  >> In some cases it may not be possible or viable to create new scope for each resource id e.g. short lived resources.
  
  >Why wouldn't that be possible? What type of short-lived resources did you have in mind?
  
  For example experimental service version (beta) replaced by release version, usage of which may be reported and metered but not necessarily billed.
  The scope names may need to follow adopter specific conventions so creating scope with predefined name 'abacus.usage....' may not fit that scheme. Abacus should offer ability to adjust the scope names, otherwise submission may not be possible.
  
  
  > Another reason why I'm not sure about short lived resources, is that although you may decide to stop offering a type a resource at some point, once you've metered it, and sent a bill for it >to a customer, I don't think you can really 'forget' about its existence anymore... So in that sense I'm not sure how it can be 'short lived'.
  The short lived resource is only for submission, once it is not offered, its specific scope is not needed. Thad does not mean erasing history of usage.
  
  
  Piotr
  
  
  
  
  Jean-Sebastien Delfino ---10/08/2015 11:10:16 AM---Hi Piotr, > In some cases it may not be possible or viable to create new scope for
  
  From: Jean-Sebastien Delfino <jsdelfino@...>
  To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@...>
  Date: 10/08/2015 11:10 AM
  Subject: [cf-dev] Re: Re: Re: [abacus] Usage submission authorization

   

  ---

  
  
  
  Hi Piotr,
  
  > In some cases it may not be possible or viable to create new scope for each resource id e.g. short lived resources.
  
  Why wouldn't that be possible? What type of short-lived resources did you have in mind?
  
  The typical use case I've seen is for a Cloud platform to decide to offer a new type of database or analytics or messaging service, or a new type of runtime for example. Before that new resource is offered on the platform, their resource provider needs to get on board, get a user id, auth credentials defined in UAA etc... You probably also need to define how you're going to meter that new resource and the pricing for it.
  
  Couldn't a scope be created in UAA at that time along all these other on boarding steps?
  
  Another reason why I'm not sure about short lived resources, is that although you may decide to stop offering a type a resource at some point, once you've metered it, and sent a bill for it to a customer, I don't think you can really 'forget' about its existence anymore... So in that sense I'm not sure how it can be 'short lived'.
  
  > Some flexibility would also help to accommodate changes related to grouping resources by type as discussed in [1].
  
  We discussed two options in [1]:
  a) support a resource_type in addition to resource_id for grouping many resource_ids under a single type
  b) a common resource_id for several resources (something like 'node' for all your versions of Node.js build packs for example) 
  
  Since option (a) is not implemented at this point and Issue #38 is actually assigned to a 'future' milestone, AIUI resource providers need to use option (b) with a common resource_id for multiple resources. Is creating a scope for that common id still too much of a burden then?
  
  [1] - https://github.com/cloudfoundry-incubator/cf-abacus/issues/38
  
  Thoughts?
  
  - Jean-Sebastien
  
  On Wed, Oct 7, 2015 at 5:51 PM, Piotr Przybylski <piotrp@...> wrote:
  • • Hi Sebastien,
      
      > That OAuth token should include:
      > - a user id uniquely identifying that resource provider;
      > - an OAuth scope named like abacus.usage.<resource_id>.write
      
      What kind of customization of the above do you plan to expose? In some cases it may not be possible or viable to create new scope for each resource id e.g. short lived resources. The ability to either configure scope to use for validation or provide scope 'mapping' would help to adapt it to existing deployments. Some flexibility would also help to accommodate changes related to grouping resources by type as discussed in [1].
      
      [1] - https://github.com/cloudfoundry-incubator/cf-abacus/issues/38
      
      
      Piotr
      
      
      
      Jean-Sebastien Delfino ---10/07/2015 12:30:05 AM---Hi Piotr, > what kind of authorization is required to submit usage to Abacus ?
      
      From: Jean-Sebastien Delfino <jsdelfino@...>
      To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@...>
      Date: 10/07/2015 12:30 AM
      Subject: [cf-dev] Re: [abacus] Usage submission authorization

      ---

      
      
      
      
      Hi Piotr,
      
      > what kind of authorization is required to submit usage to Abacus ? 
      > Is the oauth token used for submission [1] required to have particular scope, specific to resource or resource provider ?
      
      A resource provider is expected to present an OAuth token with the usage it submits for a (service or runtime) resource.
      
      That OAuth token should include:
      - a user id uniquely identifying that resource provider;
      - an OAuth scope named like abacus.usage.<resource_id>.write.
      
      The precise naming syntax for that scope may still evolve in the next few days as we progress with the implementation of user story 101703426 [1].
      
      > Is there a different scope required to submit runtimes usage (like cf bridge) versus other services or its possible to use single scope for all the submissions
      
      I'd like to handle runtimes and services consistently as they're basically just different types of resources, i.e. one scope per 'service' resource, one scope per 'runtime' resource.
      
      We're still working on the detailed design and implementation, but I'm not sure we'd want to share scopes across (service and runtime) resource providers as that'd allow a resource provider to submit usage for resources owned by another...
      
      @assk / @sasrin, anything I missed? Thoughts?
      
      -- Jean-Sebastien
      
      
      On Tue, Oct 6, 2015 at 6:29 PM, Piotr Przybylski <piotrp@...> wrote:
      • • • • Hi,
              what kind of authorization is required to submit usage to Abacus ?
              Is the oauth token used for submission [1] required to have particular scope, specific to resource or resource provider ? Is there a different scope required to submit runtimes usage (like cf bridge) versus other services or its possible to use single scope for all the submissions ?
              
              
              [1] - https://www.pivotaltracker.com/story/show/101703426
              
              Piotr