Hi, all,

This email serves two purposes: 1) introduce CF-Swagger, and 2) shares the results of the CF service broker compliance survey I sent out a couple of weeks ago.

------
My IBM Research colleague, Mohamed (on cc:), and I have been working on creating Swagger descriptions for some CF APIs.

Our main goal was to explore what useful tools or utilities we could build with these Swagger descriptions once created.

The initial results of this exploratory research is CF-Swagger which is included in the following:

See presentation here: https://goo.gl/Y16plT
Video demo here: http://goo.gl/C8Nz5p
Temp repo here: https://github.com/maximilien/cf-swagger

The gist of of our work and results are:

1. We created a full Swagger description of the CF service broker
2. Using this description you can use the Swagger editor to create a neat API docs that is browsable and even callable
3. Using the description you can create client and server stubs for service brokers in a variety of languages, e.g., JS, Java, Ruby, etc.
4. We've extended go-swagger to generate workable client and server stubs for service brokers in Golang. We plan to submit all changes to go-swagger back to that project
5. We've extended go-swagger to generate prototypes of working Ginkgo tests to service brokers
6. We've extended go-swagger to generate a CF service broker Ginkgo Test Compliance Kit (TCK) that anyone could use to validate their broker's compliance with any Swagger-described version of spec
7. We've created a custom Ginkgo reporter that when ran with TCK will give you a summary of your compliance, e.g., 100% compliant with v2.5 but 90% compliant with v2.6 due to failing test X, Y, Z... (in Ginkgo fashion)
8. The survey results (all included in the presentation) indicate that over 50% of respondants believe TCK tests for service broker would be valuable to them. Many (over 50%) are using custom proprietary tests, and this project maybe a way to get everyone to converge to a common set of tests we could all use and improve...

------
We plan to propose this work to become a CF incubator at the next CAB and PMC calls, especially the TCK part for service brokers. The overall approach and project could be useful for other parts of the CF APIs but we will start with CF Service Brokers.

The actual Swagger descriptions should ideally come from the teams who own the APIs. So for service brokers, the CAPI team. We are engaging them as they have also been looking at improving APIs docs and descriptions. Maybe there are potential for synergies and at a minimum making sure what we generate ends up becoming useful to their pipelines.

Finally, while the repo is temporary and will change, I welcome you to take a look at presentation and video and code and let us know your thoughts and feedback.

Thanks for your time and interest.

Mohamed and Max
IBM

Since I know various folks are looking at better API docs. I went ahead
and did some quick investigation on what other kind of docs formats could
be generated from Swagger.

Found a bunch, but experimented with Swagger2Markup and was able to
generate the following from the Service Broker Swagger definition here:
https://github.com/maximilien/cf-swagger/blob/master/descriptions/cloudfoundry/service_broker/service_broker.json

1. ASSCIIDoc:
https://github.com/maximilien/cf-swagger/tree/master/markup/cloudfoundry/service_broker/assciidoc
2. GitHub Markdown:
https://github.com/maximilien/cf-swagger/tree/master/markup/cloudfoundry/service_broker/markdown

These are generated from the JSON above without any customization or
changes.

Best,

------
dr.max
ibm cloud labs
silicon valley, ca
maximilien.org



Michael Maximilien/Almaden/IBM
09/18/2015 04:51 PM

To
cf-dev(a)lists.cloudfoundry.org
cc
Mohamed Mohamed/Almaden/IBM(a)ibmus, Christopher B Ferris/Waltham/IBM(a)ibmus,
Alex Tarpinian/Austin/IBM(a)ibmus, Heiko Ludwig/Watson/IBM(a)ibmus
Subject
Introducing CF-Swagger






Hi, all,

This email serves two purposes: 1) introduce CF-Swagger, and 2) shares the
results of the CF service broker compliance survey I sent out a couple of
weeks ago.

------
My IBM Research colleague, Mohamed (on cc:), and I have been working on
creating Swagger descriptions for some CF APIs.

Our main goal was to explore what useful tools or utilities we could build
with these Swagger descriptions once created.

The initial results of this exploratory research is CF-Swagger which is
included in the following:

See presentation here: https://goo.gl/Y16plT
Video demo here: http://goo.gl/C8Nz5p
Temp repo here: https://github.com/maximilien/cf-swagger

The gist of of our work and results are:

1. We created a full Swagger description of the CF service broker
2. Using this description you can use the Swagger editor to create a neat
API docs that is browsable and even callable
3. Using the description you can create client and server stubs for
service brokers in a variety of languages, e.g., JS, Java, Ruby, etc.
4. We've extended go-swagger to generate workable client and server stubs
for service brokers in Golang. We plan to submit all changes to go-swagger
back to that project
5. We've extended go-swagger to generate prototypes of working Ginkgo
tests to service brokers
6. We've extended go-swagger to generate a CF service broker Ginkgo Test
Compliance Kit (TCK) that anyone could use to validate their broker's
compliance with any Swagger-described version of spec
7. We've created a custom Ginkgo reporter that when ran with TCK will give
you a summary of your compliance, e.g., 100% compliant with v2.5 but 90%
compliant with v2.6 due to failing test X, Y, Z... (in Ginkgo fashion)
8. The survey results (all included in the presentation) indicate that
over 50% of respondants believe TCK tests for service broker would be
valuable to them. Many (over 50%) are using custom proprietary tests, and
this project maybe a way to get everyone to converge to a common set of
tests we could all use and improve...

------
We plan to propose this work to become a CF incubator at the next CAB and
PMC calls, especially the TCK part for service brokers. The overall
approach and project could be useful for other parts of the CF APIs but we
will start with CF Service Brokers.

The actual Swagger descriptions should ideally come from the teams who own
the APIs. So for service brokers, the CAPI team. We are engaging them as
they have also been looking at improving APIs docs and descriptions. Maybe
there are potential for synergies and at a minimum making sure what we
generate ends up becoming useful to their pipelines.

Finally, while the repo is temporary and will change, I welcome you to
take a look at presentation and video and code and let us know your
thoughts and feedback.

Thanks for your time and interest.

Mohamed and Max
IBM

Thanks Mohamed and Max for sharing this great work. Besides the supporting
an official TCK, the cf-swagger repo seems great to ease the delivery of
acceptance tests as part of a a service broker release (e.g. scheduled
through bosh errands).

+1 for formal description of CF APIs allowing partly? automated client
generation, and lowering the maintenance burden w.r.t; existing CC API v2
manually maintained clients (e.g. cf-java-client, go-cfclient, nodejs, php
clients...). I had also suggested swagger for consideration in the CC API
v3 [1].

It seems the CAPI team was initially considering Swagger as a documentation
media for CC API v3 into [2] . Dieu, would it be possible to share the "Doc
of comparisons of pros and cons of different options" at [3] which does not
yet seem public ?

Thanks,

Guillaume.

[1] https://github.com/cloudfoundry/cc-api-v3-style-guide/issues/46
[2] https://www.pivotaltracker.com/n/projects/966314/stories/99237980
[3]
https://docs.google.com/a/pivotal.io/document/d/1aVOZfd0n7BOLuJvK0_Sgie9Y3D7GT6NUF4V-bVG-BCs/edit?usp=sharing

On Tue, Sep 22, 2015 at 9:12 PM, Michael Maximilien <maxim(a)us.ibm.com>
wrote:

Since I know various folks are looking at better API docs. I went ahead
and did some quick investigation on what other kind of docs formats could
be generated from Swagger.

Found a bunch, but experimented with Swagger2Markup
<https://github.com/Swagger2Markup/swagger2markup> and was able to
generate the following from the Service Broker Swagger definition here:
https://github.com/maximilien/cf-swagger/blob/master/descriptions/cloudfoundry/service_broker/service_broker.json

1. ASSCIIDoc:
https://github.com/maximilien/cf-swagger/tree/master/markup/cloudfoundry/service_broker/assciidoc
2. GitHub Markdown:
https://github.com/maximilien/cf-swagger/tree/master/markup/cloudfoundry/service_broker/markdown

These are generated from the JSON above without any customization or
changes.

Best,

------
dr.max
ibm cloud labs
silicon valley, ca
maximilien.org


*Michael Maximilien/Almaden/IBM*

09/18/2015 04:51 PM
To
cf-dev(a)lists.cloudfoundry.org
cc
Mohamed Mohamed/Almaden/IBM(a)ibmus, Christopher B Ferris/Waltham/IBM(a)ibmus,
Alex Tarpinian/Austin/IBM(a)ibmus, Heiko Ludwig/Watson/IBM(a)ibmus
Subject
Introducing CF-Swagger




Hi, all,

This email serves two purposes: 1) introduce CF-Swagger, and 2) shares the
results of the CF service broker compliance survey I sent out a couple of
weeks ago.

------
My IBM Research colleague, Mohamed (on cc:), and I have been working on
creating Swagger descriptions for some CF APIs.

Our main goal was to explore what useful tools or utilities we could build
with these Swagger descriptions once created.

The initial results of this exploratory research is CF-Swagger which is
included in the following:

See presentation here: *https://goo.gl/Y16plT* <https://goo.gl/Y16plT>
Video demo here: *http://goo.gl/C8Nz5p* <http://goo.gl/C8Nz5p>
Temp repo here: *https://github.com/maximilien/cf-swagger*
<https://github.com/maximilien/cf-swagger>

The gist of of our work and results are:

1. We created a full Swagger description of the CF service broker
2. Using this description you can use the Swagger editor to create a neat
API docs that is browsable and even callable
3. Using the description you can create client and server stubs for
service brokers in a variety of languages, e.g., JS, Java, Ruby, etc.
4. We've extended go-swagger to generate workable client and server stubs
for service brokers in Golang. We plan to submit all changes to go-swagger
back to that project
5. We've extended go-swagger to generate prototypes of working Ginkgo
tests to service brokers
6. We've extended go-swagger to generate a CF service broker Ginkgo Test
Compliance Kit (TCK) that anyone could use to validate their broker's
compliance with any Swagger-described version of spec
7. We've created a custom Ginkgo reporter that when ran with TCK will give
you a summary of your compliance, e.g., 100% compliant with v2.5 but 90%
compliant with v2.6 due to failing test X, Y, Z... (in Ginkgo fashion)
8. The survey results (all included in the presentation) indicate that
over 50% of respondants believe TCK tests for service broker would be
valuable to them. Many (over 50%) are using custom proprietary tests, and
this project maybe a way to get everyone to converge to a common set of
tests we could all use and improve...

------
We plan to propose this work to become a CF incubator at the next CAB and
PMC calls, especially the TCK part for service brokers. The overall
approach and project could be useful for other parts of the CF APIs but we
will start with CF Service Brokers.

The actual Swagger descriptions should ideally come from the teams who own
the APIs. So for service brokers, the CAPI team. We are engaging them as
they have also been looking at improving APIs docs and descriptions. Maybe
there are potential for synergies and at a minimum making sure what we
generate ends up becoming useful to their pipelines.

Finally, while the repo is temporary and will change, I welcome you to
take a look at presentation and video and code and let us know your
thoughts and feedback.

Thanks for your time and interest.

Mohamed and Max
IBM

Hi Mohamed and Dr. Max, I fully support this effort. By having Swagger based “Application Interface” capability as part of the overall CF PaaS platform would be very useful for the CF community as a whole. As a matter of fact, I also initiated a similar thread few months ago on cf-dev alias (see email text below). Your work exactly matches up with what our current thinking is.

By having “Swagger” based “Application Interface” is a very good start along those lines. This opens up lots of other possibilities such as building out “Deployment Governance” capabilities not merely for Cloud Foundry API or Services assets but for the whole Application landscape built & deployed within CF PaaS environment and subsequently exposed as APIs to end consumers.

As described below in my email I sent out earlier that “Deployment Governance” as part of overall API Management is what we are striving towards in order to expose comprehensive telecom API Management capabilities within the public cloud environment.

Dr. Max, as I mentioned to you during our brief discussion few days ago that “Heroku” folks also have a similar initiative ongoing. They have gone lightweight “JSON” schema route versus Swagger/WADL/RAML etc.

In any case, I am fully in support of your proposal. Thanks.

Regards,
Deepak Vij

=============================
Hi folks, I would like to start a thread on the need for machine-readable “Application Interface” supported at the platform level. Essentially, this interface describes details such as available methods/operations, inputs/outputs data types (schema), application dependencies etc. Any standard specifications language can be used for this purpose, as long as it clearly describes the schema of the requests and responses – one can use Web Application Description Language (WADL), Swagger, RESTful API Modeling Language (RAML), JSON Schema (something like JSON Schema for Heroku Platform APIs) or any other language that provides similar functionality. These specifications are to be automatically derived from the code and are typically part of the application development process (e.g. generated by the build system).

Such functionality can have lots of usage scenarios:

1. First and foremost, Deployment Governance for API Management (our main vested interest) – API Versioning & Backward Compatibility, Dependency Management and many more as part of the comprehensive telecom API Management capabilities which we are currently in the process of building out.

2. Auto-creating client libraries for your favorite programming language.

3. Automatic generation of up-to-date documentation.

4. Writing automatic acceptance and integration tests etc.

From historical perspective, in the early 2000s when SOA started out, the mindset was to author the application contract-first (application interface using WSDL at that time) and subsequently generate and author code from the application interface. With the advent of RESTful services, REST community initially took a stand against such metadata for applications. Although, a number of metadata standards have none-the-less emerged over the last couple of years, mainly fueled by the use case scenarios described earlier.

Based on my knowledge, none of this currently exists within Cloud Foundry at the platform level. It would be highly desirable to have a standard common “application interface” definition at the platform level, agnostic of the underlying application development frameworks.

I hope this all makes sense. I think this is something could be very relevant to the “Utilities” PMC. I will also copy&paste this text under “Utilities” PMC-notes on the github.

I would love to hear from the community on this. Thanks.

Regards,
Deepak Vij

From: Michael Maximilien [mailto:maxim(a)us.ibm.com]
Sent: Friday, September 18, 2015 4:52 PM
To: cf-dev(a)lists.cloudfoundry.org
Cc: Heiko Ludwig; Mohamed Mohamed; Alex Tarpinian; Christopher B Ferris
Subject: [cf-dev] Introducing CF-Swagger

Hi, all,


This email serves two purposes: 1) introduce CF-Swagger, and 2) shares the results of the CF service broker compliance survey I sent out a couple of weeks ago.


------
My IBM Research colleague, Mohamed (on cc:), and I have been working on creating Swagger descriptions for some CF APIs.


Our main goal was to explore what useful tools or utilities we could build with these Swagger descriptions once created.


The initial results of this exploratory research is CF-Swagger which is included in the following:


See presentation here: https://goo.gl/Y16plT
Video demo here: http://goo.gl/C8Nz5p
Temp repo here: https://github.com/maximilien/cf-swagger


The gist of of our work and results are:


1. We created a full Swagger description of the CF service broker
2. Using this description you can use the Swagger editor to create a neat API docs that is browsable and even callable
3. Using the description you can create client and server stubs for service brokers in a variety of languages, e.g., JS, Java, Ruby, etc.
4. We've extended go-swagger to generate workable client and server stubs for service brokers in Golang. We plan to submit all changes to go-swagger back to that project
5. We've extended go-swagger to generate prototypes of working Ginkgo tests to service brokers
6. We've extended go-swagger to generate a CF service broker Ginkgo Test Compliance Kit (TCK) that anyone could use to validate their broker's compliance with any Swagger-described version of spec
7. We've created a custom Ginkgo reporter that when ran with TCK will give you a summary of your compliance, e.g., 100% compliant with v2.5 but 90% compliant with v2.6 due to failing test X, Y, Z... (in Ginkgo fashion)
8. The survey results (all included in the presentation) indicate that over 50% of respondants believe TCK tests for service broker would be valuable to them. Many (over 50%) are using custom proprietary tests, and this project maybe a way to get everyone to converge to a common set of tests we could all use and improve...


------
We plan to propose this work to become a CF incubator at the next CAB and PMC calls, especially the TCK part for service brokers. The overall approach and project could be useful for other parts of the CF APIs but we will start with CF Service Brokers.


The actual Swagger descriptions should ideally come from the teams who own the APIs. So for service brokers, the CAPI team. We are engaging them as they have also been looking at improving APIs docs and descriptions. Maybe there are potential for synergies and at a minimum making sure what we generate ends up becoming useful to their pipelines.


Finally, while the repo is temporary and will change, I welcome you to take a look at presentation and video and code and let us know your thoughts and feedback.


Thanks for your time and interest.


Mohamed and Max
IBM

Separate from this proposal, the CAPI team has stories for spiking on a few
different api documentation options for the cloud controller api [1].
Swagger is one of the options we are looking into, but it is not the only
one.

[1] https://www.pivotaltracker.com/epic/show/2093796



On Wed, Sep 23, 2015 at 11:00 AM, Deepak Vij (A) <deepak.vij(a)huawei.com>
wrote:

Hi Mohamed and Dr. Max, I fully support this effort. By having Swagger
based “Application Interface” capability as part of the overall CF PaaS
platform would be very useful for the CF community as a whole. As a
matter of fact, I also initiated a similar thread few months ago on cf-dev
alias (see email text below). Your work exactly matches up with what our
current thinking is.



By having “Swagger” based “Application Interface” is a very good start
along those lines. This opens up lots of other possibilities such as
building out “Deployment Governance” capabilities not merely for Cloud
Foundry API or Services assets but for the whole Application landscape
built & deployed within CF PaaS environment and subsequently exposed as
APIs to end consumers.



As described below in my email I sent out earlier that “Deployment
Governance” as part of overall API Management is what we are striving
towards in order to expose comprehensive telecom API Management
capabilities within the public cloud environment.



Dr. Max, as I mentioned to you during our brief discussion few days ago
that “Heroku” folks also have a similar initiative ongoing. They have gone
lightweight “JSON” schema route versus Swagger/WADL/RAML etc.



In any case, I am fully in support of your proposal. Thanks.



Regards,

Deepak Vij



=============================

Hi folks, I would like to start a thread on the need for machine-readable “*Application
Interface*” supported at the platform level. Essentially, this interface
describes details such as available methods/operations, inputs/outputs data
types (schema), application dependencies etc. Any standard specifications
language can be used for this purpose, as long as it clearly describes the
schema of the requests and responses – one can use Web Application
Description Language (WADL), Swagger, RESTful API Modeling Language (RAML),
JSON Schema (something like *JSON Schema for Heroku Platform APIs*) or
any other language that provides similar functionality. These
specifications are to be automatically derived from the code and are
typically part of the application development process (e.g. generated by
the build system).



Such functionality can have lots of usage scenarios:

1. First and foremost, Deployment Governance for API Management (our
main vested interest) – API Versioning & Backward Compatibility,
Dependency Management and many more as part of the comprehensive telecom
API Management capabilities which we are currently in the process of
building out.

2. Auto-creating client libraries for your favorite programming
language.

3. Automatic generation of up-to-date documentation.

4. Writing automatic acceptance and integration tests etc.



From historical perspective, in the early 2000s when SOA started out, the
mindset was to author the application contract-first (application interface
using WSDL at that time) and subsequently generate and author code from the
application interface. With the advent of RESTful services, REST community
initially took a stand against such metadata for applications. Although, a
number of metadata standards have none-the-less emerged over the last
couple of years, mainly fueled by the use case scenarios described earlier.



Based on my knowledge, none of this currently exists within Cloud Foundry
at the platform level. It would be highly desirable to have a standard
common “*application interface*” definition at the platform level,
agnostic of the underlying application development frameworks.



I hope this all makes sense. I think this is something could be very
relevant to the “Utilities” PMC. I will also copy&paste this text under
“Utilities” PMC-notes on the github.



I would love to hear from the community on this. Thanks.



Regards,

Deepak Vij



*From:* Michael Maximilien [mailto:maxim(a)us.ibm.com]
*Sent:* Friday, September 18, 2015 4:52 PM
*To:* cf-dev(a)lists.cloudfoundry.org
*Cc:* Heiko Ludwig; Mohamed Mohamed; Alex Tarpinian; Christopher B Ferris
*Subject:* [cf-dev] Introducing CF-Swagger



Hi, all,



This email serves two purposes: 1) introduce CF-Swagger, and 2) shares the
results of the CF service broker compliance survey I sent out a couple of
weeks ago.



------

My IBM Research colleague, Mohamed (on cc:), and I have been working on
creating Swagger descriptions for some CF APIs.



Our main goal was to explore what useful tools or utilities we could build
with these Swagger descriptions once created.



The initial results of this exploratory research is CF-Swagger which is
included in the following:



See presentation here: https://goo.gl/Y16plT

Video demo here: http://goo.gl/C8Nz5p

Temp repo here: https://github.com/maximilien/cf-swagger



The gist of of our work and results are:



1. We created a full Swagger description of the CF service broker

2. Using this description you can use the Swagger editor to create a neat
API docs that is browsable and even callable

3. Using the description you can create client and server stubs for
service brokers in a variety of languages, e.g., JS, Java, Ruby, etc.

4. We've extended go-swagger to generate workable client and server stubs
for service brokers in Golang. We plan to submit all changes to go-swagger
back to that project

5. We've extended go-swagger to generate prototypes of working Ginkgo
tests to service brokers

6. We've extended go-swagger to generate a CF service broker Ginkgo Test
Compliance Kit (TCK) that anyone could use to validate their broker's
compliance with any Swagger-described version of spec

7. We've created a custom Ginkgo reporter that when ran with TCK will give
you a summary of your compliance, e.g., 100% compliant with v2.5 but 90%
compliant with v2.6 due to failing test X, Y, Z... (in Ginkgo fashion)

8. The survey results (all included in the presentation) indicate that
over 50% of respondants believe TCK tests for service broker would be
valuable to them. Many (over 50%) are using custom proprietary tests, and
this project maybe a way to get everyone to converge to a common set of
tests we could all use and improve...



------

We plan to propose this work to become a CF incubator at the next CAB and
PMC calls, especially the TCK part for service brokers. The overall
approach and project could be useful for other parts of the CF APIs but we
will start with CF Service Brokers.



The actual Swagger descriptions should ideally come from the teams who own
the APIs. So for service brokers, the CAPI team. We are engaging them as
they have also been looking at improving APIs docs and descriptions. Maybe
there are potential for synergies and at a minimum making sure what we
generate ends up becoming useful to their pipelines.



Finally, while the repo is temporary and will change, I welcome you to
take a look at presentation and video and code and let us know your
thoughts and feedback.



Thanks for your time and interest.



Mohamed and Max

IBM