The "cached" packages of the buildpacks contain lots of runtime binaries,
like OpenJDK, Node.js, PHP etc, built by Cloud Foundry. Has anybody gone
through a review of their export control classification? For example, ASF
has this nice page [1] listing the classification of their products. Such
information would make things easier for vendors who want to redistribute
Cloud Foundry.

Jack

[1] http://www.apache.org/licenses/exports/

I just wanted to follow up on this.

Github 110 is now closed as we've finished removing explicit references to
regions in the Abacus API in favor of a more generic namespace/scoping
mechanism. With these changes, organization ids, resource instance ids, etc
can include a namespace/scope allowing you to ensure they're really
universally unique across your multiple CF deployments (if you're
paranoid...)

For example the following ids will be happily accepted:
a3d7fe4d-3cb1-4cc3-a831-ffe98e20cf27
us-south:a3d7fe4d-3cb1-4cc3-a831-ffe98e20cf27
my-deployment:foo:bar:xyz23:a3d7fe4d-3cb1-4cc3-a831-ffe98e20cf27
...

We're keeping the existing region field as optional for now in the usage
schemas to avoid breaking the folks that use it, but it's not needed
anymore, and I'd like to remove it as well at some point.

HTH

- Jean-Sebastien

Any update at all?

Thanks

- Jean-Sebastien

On Tue, Nov 10, 2015 at 11:28 AM, Jean-Sebastien Delfino <
jsdelfino(a)gmail.com> wrote:

Hi all,

I'd like to start looking into how to leverage these notifications in the
Abacus project.

Has there been any progress on this?

Thanks

- Jean-Sebastien

On Fri, Sep 4, 2015 at 7:58 AM, Dieu Cao <dcao(a)pivotal.io> wrote:

To follow up on this, I've been working with Simon Moser on an initial
proposal for this and he is now taking lead on it. Simon just completed a
PM dojo at the end of August.

Dieu


On Tuesday, August 18, 2015, Dieu Cao <dcao(a)pivotal.io> wrote:

I planned to put together a proposal for this a couple of weeks ago as a
strawman to describe use cases, but just have not had the time.
I still hope to tackle this in the next week or so and will post to this
list.

For reference, see this thread [1] where this was previously discussed.

-Dieu
CF CAPI PM

[1]
http://cf-dev.70369.x6.nabble.com/cf-dev-Notifications-on-ORG-SPACE-and-USER-modifications-tt827.html#none

On Tue, Aug 18, 2015 at 5:47 PM, Vineet Banga <vineetbanga1(a)gmail.com>
wrote:

Thanks Juan, I will try to setup a poller for this to achieve similar
functionality. Do you know if there is already proposal for the better
notifications - if yes, could you point me to it? I Would like to see if
it would meet our needs at some point in the future.

On Fri, Aug 14, 2015 at 4:26 PM, Juan Pablo Genovese <
juanpgenovese(a)gmail.com> wrote:

Vineet,

there is some proposals to add better notifications to CF in general
and the CC in particular, but for now you can poll the CC API to get those
events. See http://apidocs.cloudfoundry.org/214/

Thanks!

2015-08-14 18:31 GMT-03:00 Vineet Banga <vineetbanga1(a)gmail.com>:

Is there any notification pub/sub mechanism in cloud foundry when
services are created/updated/deleted. We are exposing few services in CF
using service brokers and we would like some common actions to occur when
our services are created/delete/updated.

--
Mis mejores deseos,
Best wishes,
Meilleurs vœux,

Juan Pablo
------------------------------------------------------
http://www.jpgenovese.com

On Thu, Nov 12, 2015 at 1:41 PM, Benjamin Cheng <bscheng(a)us.ibm.com> wrote:

How about keying by criteria as well to know when to trigger a Webhook

call? and maybe allow multiple registrations per URL? (e.g please call me
back at http://foo.com/bar on new usage for org abc, and also when usage
for app xyz goes over 1000, would probably be two separate registration
docs for a single URL)
I think that would work well too. I guess the main thing between the two
approaches is number of docs versus size of a doc. Building a query for
either wouldn't be too hard, but putting multiple registrations inside a
doc would probably add additional complexity to the notification logic.

Yes, that's why I was suggesting one doc per registration keyed by trigger
criteria.

+1 for a sort of quarantine on an unreachable Webhook. How about slow

Webhooks causing back pressure problems? Would we quarantine these too?
This is kind of hard for me to figure out. If they continue to cause
problems, leaving the slow webhooks would probably compound the problems as
time goes on, but at the same time, they aren't quite in the same category
as an unreachable one since they're actually reachable. Somehow the client
needs to know their webhook is causing issues or unreachable.

I thought a bit more about this. We could set a timeout (using our circuit
breaker module for example) causing slow Webhooks to error and then get
handled like other errors.

- should we let the rating service app do this or have a separate

notification service app?
I don't have a concrete opinion on this, but both sides have their merits.
With a separate notification app, this keeps notification logic outside of
something like aggregator such that aggregator stays doing aggregator logic
and doesn't expand to something that it might not want to do. One thing to
consider is dealing with the load of everything coming to a single
notification app. I'm also not sure if this approach would design the logic
in such a way that it won't be pluggable in any other app such that someone
would not have to the notifications application running to use it.
I don't have as much to say on putting the logic inside of rating. The
only thing I can bring up is my previous statement of having rating doing
notification logic instead of strictly rating logic, and it does give a
bigger separation of logic in terms of determining if a criteria is met or
not.

Right, so I'd vote for a separate service initially for a cleaner
separation of concerns, and we can always merge it back in later if we want.

- with partitioning of the orgs across multiple deployments of our apps

(for scalability or regional deployments for example) do I need to first
find the right service to register with (e.g. register with the us-south
notification service or the eu-west notification service)? or can I
register with a central notification service that will then figure out
which deployment instance will call me back?
It might work better on a central notification service to prevent things
like a client registering to an incorrect region. If we keep it separate
and configured criteria continue to evolve, would a central notification
service help along the lines of something like "notify if all my
organizations across all the regions exceed X amount?"

Good point, I had not thought about notifications on total usage across
multiple regional organizations.

- how do we secure the registration calls and Webhook callbacks?

The security on registration would probably just validate if the user's
token has access to that organization/space/etc.

OK sounds good, that'll also be consistent with how our usage reporting
works.

For the webhook, I'm not sure if this falls within abacus.usage.read or
abacus.usage.write, or if we would need a new scope to handle this case.

It'd be good to get some use case input from Subhash, Piotr etc on this.

- do we replay notifications when we can't deliver them?

Ideally, it would make sense. Whether it's a simple series of retries or
have replay logic that would later play through the set of unsent
notifications. I think this is related to the quarantine logic. Some
notifications when replayed will probably go through, some will fail and
may continue to fail. Those that continue to fail may be due to the webhook
their attached to, and thus, it may make sense to quarantine that webhook
based upon these replay failures.

+1

- can I register to receive all notifications from a certain point in the

logical stream of notifications matching a criteria (e.g. call me back if
this org consumed too much per hour at any point since last week)
I'm not sure if I completely understand this question. Is this pretty much
setting a adhoc time range upon which notifications should be sent?

I was thinking about a kind of 'cursor' mechanism or something along the
lines of what CF app events provide, where you can request notifications
from a sequence number, a timestamp, or a page number for example... That
cursor mechanism will be handy too when you'll want to replay missed
notifications after a failure. Makes sense?

- Jean-Sebastien

Hi,

We have three Cloud foundry environment (Dev, STG and PROD) deployed using internal/private S3 blobstore (Cloudian) on internal VSphere cloud - all of a sudden we start getting 'SignatureDoesNotMatch' (or sometime 403 Forbidden) error while uploading release using bosh upload or trying to recreate any VM.

We are suspecting that we might need to pass/configure use signature-v2 option somewhere in the bosh and cloud foundry manifest files - I have few question on this

1/ What is the default request type (S2 or V4 signature) whey we do bosh deploy (to deploy bosh director using Micro Bosh)?
2/ How/Where can I change the signature request in Micro bosh manifest file?
3/ How/Where can I change the signature request in Bosh manifest file?
4/ In case of Cloud foundry - I think we can pass 'aws_signature_version' in fog connections config - please correct me if I am wrong
5/ can we use fog connections in Micro bosh and/or Bosh manifest

- Anuj

Perfect that worked. Thanks.

Eric,

The python buildpack does bundle the libffi dependency. It only get's
installed
<https://github.com/cloudfoundry/python-buildpack/blob/master/bin/steps/cryptography#L23>
if you added cffi to your requirements.txt. With `pip` you have to explicit
set all your dependencies.

Give that a try.

Kind Regards,

JT

I haven't tried this, but I think it should work.

1.) Make a directory. In that directory put your JAR file, your start
script and anything else the app needs to run.
2.) From that directory, run `cf push <app-name> -b java_buildpack -c
'$PWD/start-script.sh'`. This will upload your script, the JAR file and
everything else in the current directory. It will also tell CF that you
specifically want to use the Java build pack (which will install Java) and
that you want to use your script to start your app.

What could be tricky about this is your start script. It's going to need
to reference JAVA_HOME as `/home/vcap/app/.java-buildpack/open_jdk_jre`,
and `java` as `$JAVA_HOME/bin/java` since `java` is not going to be on the
$PATH.

You're also going to need to handle some of the things that the JBP would
normally do like set -Xmx and other JVM memory settings to keep the JVM
from exceeding the containers MEMORY_LIMIT. Note, *all* memory needs to
fit under the limit, not just the JVM's heap. In other words, setting -Xmx
== MEMORY_LIMIT is 100% wrong.

Beyond that, you'd need to make sure the app is listening on $PORT or if
it's not taking web requests, disable that health check (`cf push
--no-route` & `cf set-health-check none`).

Dan

Hi ankit,

First of all, do you think who is responsible to verify the signature? Your
application? Or (one of) the components of Cloud Foundry? I assume the
former is your answer. I think there is no functionality in Cloud Foundry to
verify client signature.

Then, if you use the Cloud Foundry java-buildpack to deploy your
application, I think there is the only one way to send key files with your
app on deployment. It is to add your key files to your app's war / jar / zip
file, primitively like:

```
jar uvf your-war-jar-zip-file path-to-your-key-files-or-directories
```

But you may add a maven / gradle task to do such a thing.

This is because the Cloud Foundry java-buildpack accepts only one zip-format
file on a deployment.

# Please correct this post if I am wrong. Thank you.



ankit wrote

Suppose my application is deployed on the cloud foundry and my client
sends a POST request that contains some message but that message is
digitally signed by client’s private key. So, I need client’s public
key(digital id of client) to verify my client for inbound calls in the
cloud foundry where application is running. So, can you tell me where can
I put these public keys(digital IDs of clients) in java build pack or any
other place.
Similarly, for outbound calls I want my message to be digitally signed and
for that I need private key to be used. So, where can I put that also?

-----
I'm not a ...
noburou taniguchi
--
View this message in context: http://cf-dev.70369.x6.nabble.com/regarding-using-public-key-to-verify-client-tp2711p2719.html
Sent from the CF Dev mailing list archive at Nabble.com.

Hi Dammina,

I think your case doesn't seem to fit the Cloud Foundry standard Java
buildpack.

One thing I can suggest is to use heroku-buildpack-runnable-jar [1]. With
this buildpack, you can start your app with a shell script.

But you probably have to modify it to fit your purpose. Also you should
calculate and specify appropriate memory for your app in the start script by
yourself.

Or if you can use Diego, it may be a solution to create and use a Docker
image for your app. But I don't know much about Diego, so this may be wrong.

[1] https://github.com/energizedwork/heroku-buildpack-runnable-jar


dammina sahabandu wrote

Hi All,
I have a java application which is a composition of several jars in the
running environment. The main jar is invoked by a shell script. Is it
possible to push such application into cloud foundry? And if it is
possible can you please help me on how to achieve that. A simple guide
will be really helpful.

Thank you in advance,
Dammina

-----
I'm not a ...
noburou taniguchi
--
View this message in context: http://cf-dev.70369.x6.nabble.com/cf-dev-Deploying-a-shell-script-driven-java-application-to-cf-tp2697p2717.html
Sent from the CF Dev mailing list archive at Nabble.com.

Hi zooba,

Though I haven't tried them by myself, I conceived the following 2 methods:

(a) Push your app with an auth token and a refresh token, and refresh the
auth token using the refresh token

or,

(b) Create another user only for your app and push the app with the
credentials of the user


For (a), I'm not sure how an auth token is refreshed, but it seems able to
be done via /oauth/token endpoint in UAA [1].

For (b), it seems that you need just revokable credentials, so I think you
can use a user dedicated to the app as revokable credentials.


[1]
https://github.com/cloudfoundry/uaa/blob/master/docs/UAA-APIs.rst#oauth2-token-endpoint-post-oauth-token


I hope this helps.


zooba Sir wrote

Actually my app needs to get info of other apps running in same cf api
domain. To get this info, I'm calling APIs of other apps using
HTTP.GETrequest with authentication token in request header. And to get
this authentication token, I'm using HTTP:POST request to
'AuthorizationURL/oauth/token' with username and password in the request
header.
1.CF environment variables doesn't have all the info needed and so I'm
calling app's API directly and getting needed info from the returned JSON.
2. There is a config.json file in my local system in "C:\Users\MyUser\.cf"
path and has cf target api and auth token, but it gets populated only when
I'm logged into cf in command line. My app once deployed and running in
cf, should get the info of other apps even if I'm not logged in. So
getting auth token from local system's config.json is not a good idea.

Please suggest me another way to get auth token.

-----
I'm not a ...
noburou taniguchi
--
View this message in context: http://cf-dev.70369.x6.nabble.com/cf-dev-how-to-get-cf-authentication-token-programmatically-tp2668p2716.html
Sent from the CF Dev mailing list archive at Nabble.com.

Hey,

Can you elaborate on what qps is? Query per second?

Zhou Yu
Software Engineer
Pivotal.io

Dear all！
Do you have the solution to limit the qps of the app？

Hi,

I am using cf version 211 and CC API version 2.28.0. I am curious to know why the "inline-relations-depth" is going to be deprecated. It seems to be a useful feature.

For instance, I have a use case where for an APP, I need to display the service bindings details. The details to be displayed are: "Instance name, Plan name, Service name, Dashboard url, Credentials etc".

The calls that had to be fired to achieve this are:
1. GET /v2/apps/0f27ab6c-c132-413d-8d6a-64551dcb73fc/service_bindings
2. GET /v2/service_instances/fbd24d3e-3fe5-4d89-9ef1-5f43b8bc3767
3. GET /v2/service_plans/32bd0e93-e856-4c89-9f97-ba5c09c84ac6
4. GET /v2/services/ffc81a4b-98e0-4aff-9901-399ef98638e0

Without this feature, performance delay is introduced for multiple calls, if the data is quite large. Not only this use case but we have other use cases where the "inline-relational-data" has to be retrieved and displayed.

Is there any other api which replaces this particular feature going to be introduced?. Any other solution also would help.

Thanks.
---------
Ponraj

We also had the exact same issue and tried using https://api.bosh-lite.com also but it did not work. It gave an error unexpected EOF.

Can someone help us?

Thanks in advance.

Suppose my application is deployed on the cloud foundry and my client sends
a POST request that contains some message but that message is digitally
signed by client’s private key. So, I need client’s public key(digital id of
client) to verify my client for inbound calls in the cloud foundry where
application is running. So, can you tell me where can I put these public
keys(digital IDs of clients) in java build pack or any other place.
Similarly, for outbound calls I want my message to be digitally signed and
for that I need private key to be used. So, where can I put that also?




--
View this message in context: http://cf-dev.70369.x6.nabble.com/regarding-using-public-key-to-verify-client-tp2711.html
Sent from the CF Dev mailing list archive at Nabble.com.

Is it possible to add commands to get run during staging? I am using the python buildpack but one of my dependencies is on PyNaCL which requires libffi-dev to build properly. Since that package is not part of cflinuxfs2 stack my push fails. So is there any way to add staging steps? I have googled around a bit but not really come up with anything.

Hi Idit, good to hear from you. In that case, we have covered all bases and good to go on this. We will touch base as we discussed earlier. Thanks.


- Deepak

From: Gmail [mailto:idit.levine(a)gmail.com]
Sent: Friday, November 13, 2015 1:43 PM
To: Discussions about Cloud Foundry projects and the system overall.
Subject: [cf-dev] Re: Re: Re: Re: Re: Pluggable Resource Scheduling

We can add to mesosphere a native support for Garden containers. That is very easy to do ....

Sent from my iPhone

On Nov 13, 2015, at 2:51 PM, Deepak Vij (A) <deepak.vij(a)huawei.com<mailto:deepak.vij(a)huawei.com>> wrote:
If Mesosphere can do it for Kubernetes, CF/Diego is doable too. I am not worried about that, it is a solved problem.

The only concern I have is regarding deployment of Garden Container environment in Mesos/Slave/Executor. Although, this is not CF/Diego issues.

Because Mesos is not Garden Container environment aware, its underlying DRF scheduling algorithm may not have visibility to the resources being consumed within the Garden Container. Unless, we wrap Garden Container within the Docker container as Mesos supports Docker container environment. Although, this may not be right approach as it opens up another can of worms – Gardner Container nested within Docker Container. For Kubernetes environment this is not an issue as it uses Docker container to begin with.


- Deepak

From: resouer(a)163.com<mailto:resouer(a)163.com> [mailto:resouer(a)163.com] On Behalf Of Zhang Lei
Sent: Thursday, November 12, 2015 8:24 PM
To: Discussions about Cloud Foundry projects and the system overall.
Subject: [cf-dev] Re: Re: Re: Pluggable Resource Scheduling

You can add different scheduling strategy into Diego by implementing a scheduler plugin.


But not Mesos, that would be a huge task and another story.

The reason Kubernetes can integrate Mesos as scheduler (can work, not perfect) is due to Mesosphere is doing that part, I'm afraid ...

在 2015-11-13 03:57:52，"Deepak Vij (A)" <deepak.vij(a)huawei.com<mailto:deepak.vij(a)huawei.com>> 写道：


I did not mean to replace the whole “Diego” environment itself. What I was thinking was more in terms of plug-ability within Diego itself. This is so that “Auctioneer” component can be turned into a “Mesos Framework” as one of the scheduling options. By doing that, “Auctioneer” can start accepting “Mesos Offers” instead of native “Auctioning based Diego Resource Scheduling”. Rest of the runtime environment including Garden, Rep etc., they all stay the same. Nothing else changes. I hope this makes sense.


- Deepak

From: Gwenn Etourneau [mailto:getourneau(a)pivotal.io<mailto:getourneau(a)pivotal.io>]
Sent: Wednesday, November 11, 2015 5:10 PM
To: Discussions about Cloud Foundry projects and the system overall.
Subject: [cf-dev] Re: Pluggable Resource Scheduling

Hi,

Interesting proposition, wondering if it make sense to hook into Diego or CF.
Diego is connected to CF by the CC-Bridge (big picture) why not create a CC-Bridge for other scheduling system ?



Thanks

On Thu, Nov 12, 2015 at 5:13 AM, Deepak Vij (A) <deepak.vij(a)huawei.com<mailto:deepak.vij(a)huawei.com>> wrote:
Hi folks, I would like to start a discussion thread and get community thoughts regarding availability of Pluggable Resource Scheduling within CF/Diego. Just like Kubernetes does, wouldn’t it be nice to have an option of choosing Diego native scheduling or other uber/global resource management environments, specifically Mesos.

Look forward to comments and feedback from the community. Thanks.

Regards,
Deepak Vij
(Huawei Software Lab., Santa Clara)

We can add to mesosphere a native support for Garden containers. That is very easy to do ....

Sent from my iPhone

It seems like the specs for the packages don't include the LICENSE and NOTICE files for each package.

Example: https://github.com/cloudfoundry/cf-release/blob/master/packages/cloud_controller_ng/spec

The end result is that the bosh release tarballs don't include licensing information for each of these packages.
I think it would be best if they were included, but I'm also asking to see if there's some reason that they're not.

If no reason exists I'd like to open a PR to add them to the packaging specs.