bits-service fog usage question


Simon D Moser
 

Hi everybody,

we are beginning work on the Bits Service proposed here [1] for handling
all kinds of blobstore related operations - a functionality that is in the
cloud controller today. Cloud controller currently uses the ruby gem 'fog'
[2] for interactions with the back-end blobstore.
We are thinking about removing the dependency on the ruby fog gem and
would like to better understand what blobstores and configurations you, in
the community, are using today. Unfortunately, the fact that this gem is
used leaks out to the cc configuration [3] by taking an opaque fog hash,
and therefore it could potentially be used in many ways.

We would like to get feedback on whether it would be fine if the new bits
service only supported s3 compatible blobstores, externally provided nfs,
and webdav (and no longer everything that fog potentially supports).
Would you be impacted by such a decision, and how?
What connection parameters/features of fog do you currently make use of?


[1]
https://docs.google.com/document/d/1kIjBuJJ0ZiJRPzMJW8dtce26jhAHbK7KotY9416YMEI/edit?usp=sharing
[2] https://github.com/fog/fog
[3]
https://github.com/cloudfoundry/cf-release/blob/master/jobs/cloud_controller_ng/spec#L250-L251


Mit freundlichen Grüßen / Kind regards

Simon Moser

Senior Technical Staff Member / IBM Master Inventor
Bluemix Application Platform Lead Architect
Dept. C727, IBM Research & Development Boeblingen

-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland
Schoenaicher Str. 220
71032 Boeblingen
Phone: +49-7031-16-4304
Fax: +49-7031-16-4890
E-Mail: smoser(a)de.ibm.com
-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland Research & Development GmbH / Vorsitzender des
Aufsichtsrats: Martina Koederitz
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht
Stuttgart, HRB 243294

**
Great minds discuss ideas; average minds discuss events; small minds
discuss people.
Eleanor Roosevelt


Simon D Moser
 

ps: forgot to mention that openStack Swift is of course in scope, too.

Mit freundlichen Grüßen / Kind regards

Simon Moser

Senior Technical Staff Member / IBM Master Inventor
Bluemix Application Platform Lead Architect
Dept. C727, IBM Research & Development Boeblingen

-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland
Schoenaicher Str. 220
71032 Boeblingen
Phone: +49-7031-16-4304
Fax: +49-7031-16-4890
E-Mail: smoser(a)de.ibm.com
-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland Research & Development GmbH / Vorsitzender des
Aufsichtsrats: Martina Koederitz
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht
Stuttgart, HRB 243294

**
Great minds discuss ideas; average minds discuss events; small minds
discuss people.
Eleanor Roosevelt



From: Simon D Moser/Germany/IBM(a)IBMDE
To: "CF Developers Mailing List" <cf-dev(a)lists.cloudfoundry.org>
Date: 03/02/2016 11:51
Subject: [cf-dev] bits-service fog usage question



Hi everybody,

we are beginning work on the Bits Service proposed here [1] for handling
all kinds of blobstore related operations - a functionality that is in the
cloud controller today. Cloud controller currently uses the ruby gem 'fog'
[2] for interactions with the back-end blobstore.
We are thinking about removing the dependency on the ruby fog gem and
would like to better understand what blobstores and configurations you, in
the community, are using today. Unfortunately, the fact that this gem is
used leaks out to the cc configuration [3] by taking an opaque fog hash,
and therefore it could potentially be used in many ways.

We would like to get feedback on whether it would be fine if the new bits
service only supported s3 compatible blobstores, externally provided nfs,
and webdav (and no longer everything that fog potentially supports).
Would you be impacted by such a decision, and how?
What connection parameters/features of fog do you currently make use of?


[1]
https://docs.google.com/document/d/1kIjBuJJ0ZiJRPzMJW8dtce26jhAHbK7KotY9416YMEI/edit?usp=sharing

[2] https://github.com/fog/fog
[3]
https://github.com/cloudfoundry/cf-release/blob/master/jobs/cloud_controller_ng/spec#L250-L251



Mit freundlichen Grüßen / Kind regards

Simon Moser

Senior Technical Staff Member / IBM Master Inventor
Bluemix Application Platform Lead Architect
Dept. C727, IBM Research & Development Boeblingen

-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland
Schoenaicher Str. 220
71032 Boeblingen
Phone: +49-7031-16-4304
Fax: +49-7031-16-4890
E-Mail: smoser(a)de.ibm.com
-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland Research & Development GmbH / Vorsitzender des
Aufsichtsrats: Martina Koederitz
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht
Stuttgart, HRB 243294

**
Great minds discuss ideas; average minds discuss events; small minds
discuss people.
Eleanor Roosevelt


Tammer Saleh
 

I can't comment on the common use cases, and who this change would break.
However, I'm very much in favor of hiding this implementation detail from
our API. Letting a library's internal API leak into our product surface
area is a very bad thing for us long term
<http://tammersaleh.com/posts/mind-the-map/>.

Cheers,
Tammer Saleh
Senior Director of Engineering, Pivotal CF, London
http://pivotal.io | http://tammersaleh.com | +44 7463 939332

On Wed, Feb 3, 2016 at 1:07 PM, Simon D Moser <SMOSER(a)de.ibm.com> wrote:

ps: forgot to mention that openStack Swift is of course in scope, too.

Mit freundlichen Grüßen / Kind regards

Simon Moser

Senior Technical Staff Member / IBM Master Inventor
Bluemix Application Platform Lead Architect
Dept. C727, IBM Research & Development Boeblingen


-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland
Schoenaicher Str. 220
71032 Boeblingen
Phone: +49-7031-16-4304
Fax: +49-7031-16-4890
E-Mail: smoser(a)de.ibm.com

-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland Research & Development GmbH / Vorsitzender des
Aufsichtsrats: Martina Koederitz
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht
Stuttgart, HRB 243294

**
Great minds discuss ideas; average minds discuss events; small minds
discuss people.
Eleanor Roosevelt



From: Simon D Moser/Germany/IBM(a)IBMDE
To: "CF Developers Mailing List" <cf-dev(a)lists.cloudfoundry.org>
Date: 03/02/2016 11:51
Subject: [cf-dev] bits-service fog usage question
------------------------------



Hi everybody,

we are beginning work on the Bits Service proposed here [1] for handling
all kinds of blobstore related operations - a functionality that is in the
cloud controller today. Cloud controller currently uses the ruby gem 'fog'
[2] for interactions with the back-end blobstore.
We are thinking about removing the dependency on the ruby fog gem and
would like to better understand what blobstores and configurations you, in
the community, are using today. Unfortunately, the fact that this gem is
used leaks out to the cc configuration [3] by taking an opaque fog hash,
and therefore it could potentially be used in many ways.

We would like to get feedback on whether it would be fine if the new bits
service only supported s3 compatible blobstores, externally provided nfs,
and webdav (and no longer everything that fog potentially supports).
Would you be impacted by such a decision, and how?
What connection parameters/features of fog do you currently make use of?


[1]
*https://docs.google.com/document/d/1kIjBuJJ0ZiJRPzMJW8dtce26jhAHbK7KotY9416YMEI/edit?usp=sharing*
<https://docs.google.com/document/d/1kIjBuJJ0ZiJRPzMJW8dtce26jhAHbK7KotY9416YMEI/edit?usp=sharing>
[2] *https://github.com/fog/fog* <https://github.com/fog/fog>
[3]
*https://github.com/cloudfoundry/cf-release/blob/master/jobs/cloud_controller_ng/spec#L250-L251*
<https://github.com/cloudfoundry/cf-release/blob/master/jobs/cloud_controller_ng/spec#L250-L251>


Mit freundlichen Grüßen / Kind regards

Simon Moser

Senior Technical Staff Member / IBM Master Inventor
Bluemix Application Platform Lead Architect
Dept. C727, IBM Research & Development Boeblingen


-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland
Schoenaicher Str. 220
71032 Boeblingen
Phone: +49-7031-16-4304
Fax: +49-7031-16-4890
E-Mail: smoser(a)de.ibm.com

-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland Research & Development GmbH / Vorsitzender des
Aufsichtsrats: Martina Koederitz
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht
Stuttgart, HRB 243294

**
Great minds discuss ideas; average minds discuss events; small minds
discuss people.
Eleanor Roosevelt



Tim Lawrence <tim.lawrence1984@...>
 

We would potentially be affected as we use Fog's Atmos provider for now. In the future we plan to implement fully s3 compatible
Object storage so no problem then but may require some backwards compatibility for a while..

Tim

Sent from my iPhone

On 3 Feb 2016, at 14:21, Tammer Saleh <tsaleh(a)pivotal.io> wrote:

I can't comment on the common use cases, and who this change would break. However, I'm very much in favor of hiding this implementation detail from our API. Letting a library's internal API leak into our product surface area is a very bad thing for us long term.

Cheers,
Tammer Saleh
Senior Director of Engineering, Pivotal CF, London
http://pivotal.io | http://tammersaleh.com | +44 7463 939332

On Wed, Feb 3, 2016 at 1:07 PM, Simon D Moser <SMOSER(a)de.ibm.com> wrote:
ps: forgot to mention that openStack Swift is of course in scope, too.

Mit freundlichen Grüßen / Kind regards

Simon Moser

Senior Technical Staff Member / IBM Master Inventor
Bluemix Application Platform Lead Architect
Dept. C727, IBM Research & Development Boeblingen

-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland
Schoenaicher Str. 220
71032 Boeblingen
Phone: +49-7031-16-4304
Fax: +49-7031-16-4890
E-Mail: smoser(a)de.ibm.com
-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland Research & Development GmbH / Vorsitzender des
Aufsichtsrats: Martina Koederitz
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht
Stuttgart, HRB 243294

**
Great minds discuss ideas; average minds discuss events; small minds discuss people.
Eleanor Roosevelt



From: Simon D Moser/Germany/IBM(a)IBMDE
To: "CF Developers Mailing List" <cf-dev(a)lists.cloudfoundry.org>
Date: 03/02/2016 11:51
Subject: [cf-dev] bits-service fog usage question



Hi everybody,

we are beginning work on the Bits Service proposed here [1] for handling all kinds of blobstore related operations - a functionality that is in the cloud controller today. Cloud controller currently uses the ruby gem 'fog' [2] for interactions with the back-end blobstore.
We are thinking about removing the dependency on the ruby fog gem and would like to better understand what blobstores and configurations you, in the community, are using today. Unfortunately, the fact that this gem is used leaks out to the cc configuration [3] by taking an opaque fog hash, and therefore it could potentially be used in many ways.

We would like to get feedback on whether it would be fine if the new bits service only supported s3 compatible blobstores, externally provided nfs, and webdav (and no longer everything that fog potentially supports).
Would you be impacted by such a decision, and how?
What connection parameters/features of fog do you currently make use of?


[1] https://docs.google.com/document/d/1kIjBuJJ0ZiJRPzMJW8dtce26jhAHbK7KotY9416YMEI/edit?usp=sharing
[2] https://github.com/fog/fog
[3] https://github.com/cloudfoundry/cf-release/blob/master/jobs/cloud_controller_ng/spec#L250-L251


Mit freundlichen Grüßen / Kind regards

Simon Moser

Senior Technical Staff Member / IBM Master Inventor
Bluemix Application Platform Lead Architect
Dept. C727, IBM Research & Development Boeblingen

-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland
Schoenaicher Str. 220
71032 Boeblingen
Phone: +49-7031-16-4304
Fax: +49-7031-16-4890
E-Mail: smoser(a)de.ibm.com
-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland Research & Development GmbH / Vorsitzender des
Aufsichtsrats: Martina Koederitz
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht
Stuttgart, HRB 243294

**
Great minds discuss ideas; average minds discuss events; small minds discuss people.
Eleanor Roosevelt