Date   

Re: Pointer to the CF code that generates org and service instance guids?

Eric Malm <emalm@...>
 

Hi, Jean-Sebastien,

V4 UUIDs contain 122 bits of information (
https://tools.ietf.org/html/rfc4122#page-14), so there are approximately
5.3 × 10^36 distinct ones. Since the Ruby implementation that Joseph
referred to generates them randomly, we can assume them to be distributed
uniformly across the sample space. From the approximation formulas in
https://en.wikipedia.org/wiki/Birthday_problem#Cast_as_a_collision_problem,
for there to be even a one-in-a-billion probability of a collision, you
would have to be using more than 10^14 UUIDs simultaneously as organization
guids. If you have, say, only a million org guids in play, the probability
of a collision decreases to about 10^-25.

Thanks,
Eric

On Tue, Oct 20, 2015 at 10:28 PM, Jean-Sebastien Delfino <
jsdelfino(a)gmail.com> wrote:

Thanks that helps. I have a few follow up questions:

Can you quantify 'mathematically incredibly likely'?

How do you guarantee uniqueness for a single deployment (since AIUI
they're generated randomly)? Are you using the CC database for that?

What would it take to guarantee uniqueness across deployments? (I'm asking
as cloud platforms out there usually run in several datacenters or regions,
i.e. multiple deployments, so to me it would make sense to provide a way to
implement that guarantee, for example by allowing the uuid to be generated
from a namespace/name like described in RFC 4122 section 4.3 [1], but
there's probably many other ways to do this...)

[1] https://www.ietf.org/rfc/rfc4122.txt

Thanks!

- Jean-Sebastien

On Fri, Oct 9, 2015 at 11:08 PM, CF Runtime <cfruntime(a)gmail.com> wrote:

Org and Service Instance guids are generated by the Cloud Controller. It
simply uses the SecureRandom.uuid ruby method:
http://docs.ruby-lang.org/en/2.2.0/SecureRandom.html#method-c-uuid

SecureRandom.uuid implements RFC 4122 and generates guids randomly. The
guids for orgs are guaranteed to be unique for a single deployment, and
mathematically incredibly likely to be unique across all deployments.

Joseph
CF Release Integration Team

On Fri, Oct 9, 2015 at 2:51 PM, Jean-Sebastien Delfino <
jsdelfino(a)gmail.com> wrote:

Hi all,

Could somebody from the CF team point me to the CF code that generates
org and service instance guids?

Are the generated guids globally unique across multiple
deployments/instances of CF? or only unique within a single instance of CF?

Thanks!

- Jean-Sebastien


Re: region qualifier for organizations

Jean-Sebastien Delfino
 

Hi all,

If there's no objection and nobody comes up with a better idea, I'll start
to work on adding a GET /v1/regions/us/orgs/ path as discussed here
sometime today.

- Jean-Sebastien

On Tue, Oct 20, 2015 at 10:11 PM, Jean-Sebastien Delfino <
jsdelfino(a)gmail.com> wrote:

Hi Bharath,

Sorry for the delay, I didn't realize this was a question for the Abacus
project as it didn't have the [abacus] subject tag we've been using for
Abacus discussions recently. I guess from now on I'll just check all
threads just in case :)

This is a good question. With independent deployments of CF in multiple
datacenters or regions you may need to distinguish between organization
86d0482c-7208-4f2f-8606-935c080cad41 in region 'us' and the same
organization id in region 'eu' for example.

We could add another path to the API for the cases where you care about
the region with GET /v1/regions/us/orgs/86d0482c-7208-4f2f-8606-935c080cad41/...
if that helps.

I could also sympathize with another approach, where we'd say that the
organization id being a guid should truly be *globally unique*. It looks
like the the current CF guid generation algorithm doesn't *guarantee*
uniqueness across deployments [1] but combining the region with it would
make it unique. IIUC I think that's what you're suggesting.

What do others think?

[1]
http://cf-dev.70369.x6.nabble.com/cf-dev-Pointer-to-the-CF-code-that-generates-org-and-service-instance-guids-tp2192.html

- Jean-Sebastien

On Tue, Oct 20, 2015 at 11:42 AM, Bharath Sekar <bsekar14(a)gmail.com>
wrote:

Hi,
account service implementations could need additional qualifiers to
uniquely identify an organization. For example, the implementation I'm
working on needs a region along with the guid of the org.
The API to get an account given org information looks like this

GET /v1/orgs/:org_id/account

How do we want to support the additional qualifier in abacus? One
solution that I can think of is including the region in the guid. org_id
could be 'guid_region'. ex:
GET /v1/orgs/86d0482c-7208-4f2f-8606-935c080cad41_us/account
Thoughts?


HM9000 gets stuck in bad state

kyle havlovitz <kylehav@...>
 

I'm having an issue where after a day or two or running, the health manager
is getting stuck in a bad state and doesn't display the state of apps
correctly. The logs show messages like this: "Store is not fresh -
Error:Actual and desired state are not fresh" and "Daemon returned an
error. Continuining... - Error:Actual and desired state are not fresh".

Restarting the process fixes the issue, but I'm wondering how to avoid this
problem altogether. Is this a known issue?


[abacus] Configuring Abacus applications

Piotr Przybylski <piotrp@...>
 

Hi, 
couple of questions about configuring Abacus, specifically the recommended settings and how to configure them

- is there a recommended minimal number of instances of Abacus applications
- how would above depend on expected number of submissions or documents to be processed
- is there a dependency between number of instances of applications i.e. do they have to match 
- what is the default and recommended number of DB partitions and how can they be configured (time based as well as key based)
- how would above depend on expected number of documents

Thank you,

Piotr



Re: REST API endpoint for accessing application logs

Johannes Tuchscherer
 

No, that is 100 as in 100 log lines. Given that a log line is transported
as a UDP package and the UDP max package size is 64k, with a buffer of 100
log lines you require 6.3MB of memory per app on your Doppler server.

On Wednesday, October 21, 2015, Ponraj E <ponraj.e(a)gmail.com> wrote:

Hi,

Short update reg the question #2 above: I came to know from here
http://docs.cloudfoundry.org/loggregator/ops.html that the number/size of
log messages drained to the doppler can be controlled by a bosh deployment
manifest configuration : doppler.message_drain_buffer_size

It is specified that the doppler.message_drain_buffer_size default value
is 100.

Is it 100MB?


How to detect this case: CF-AppMemoryQuo taExceeded

Juan Antonio Breña Moral <bren at juanantonio.info...>
 

Hi,

doing some tests, I detected in my testing environment the following scenario:

Error: the string "{\n \"code\": 100005,\n \"description\": \"You have ex
ceeded your organization's memory limit.\",\n \"error_code\": \"CF-AppMemoryQuo
taExceeded\"\n}\n" was thrown, throw an Error :)

Does exist some REST Call to know if the org/space has reached the limit?

Many thanks in advance

Juan Antonio


Re: Different behaviour in Login with Pivotal and Bluemix

Juan Antonio Breña Moral <bren at juanantonio.info...>
 

Hi Matthew,

many thanks for the technical explanation.

In my tests, I discovered that at the moment it is compatible to get access using this field:
"authorization_endpoint":"https://login.run.pivotal.io"
"authorization_endpoint":"https://login.eu-gb.bluemix.net/UAALoginServerWAR"

Cheers

Juan Antonio


Re: Need Help: Seeking tutorial on CF and WordPress Multisite

Daniel Mikusa
 

On Tue, Oct 20, 2015 at 10:19 PM, v Bailey <v2bailey(a)gmail.com> wrote:

I am totally green about asking the proper question so here's what I'm
researching...

I'm a decently adept web designer who specializes in WordPress. I would
like to pursue some larger clients and offer to move their sites to the
cloud (as well as perform their makeovers, of course) but in a couple of
cases, the organizations have multiple branches, each with their own
website, and it seems to make sense to use Multisite instead of individual
websites. That said, I've never used Multisite so my concern is how does
it work to create multiple websites+database for a single client...on the
cloud?
If you have a little experience with the command prompt or a terminal it's
fairly easy to create multiple sites and even services using Cloud
Foundry. This document gives you a quick overview of the cf cli.

http://docs.cloudfoundry.org/devguide/installcf/whats-new-v6.html

The main task to run your application on CF is called "cf push". You can
see more about that here.

http://docs.cloudfoundry.org/devguide/installcf/whats-new-v6.html#push

And there's a walkthrough of deploying an app here.

http://docs.cloudfoundry.org/devguide/deploy-apps/deploy-app.html

In regards to databases, you can also create those with the cf client.
This document has some instructions on that.

https://docs.cloudfoundry.org/devguide/services/managing-services.html

Since you mentioned Wordpress, you might find this blog post useful. It
walks through running Wordpress on CF.


http://blog.pivotal.io/pivotal-cloud-foundry/products/getting-started-with-wordpress-on-cloud-foundry


I suspect I should be asking more but since I don't entirely understand
what the setup should be I'll start with a simple initial question. I'd
LOVE to hear some insight into how to setup these kind of clients.

Please be gentle with me, I bruise easy. Simple explanations get more
mileage with me. Thank you kindly in advance!!
The best thing would be to sign up for an account with a public CF provider
(PWS, Bluemix, AnyNines, etc...). It's pretty easy and most offer free
trials. That would allow you to play around with deploying an app or two
to CF and get used to the cf client.

Hope that helps!

Dan


Re: Different behaviour in Login with Pivotal and Bluemix

Matthew Sykes <matthew.sykes@...>
 

Now that I've had a cup of coffee, I should clarify that the links may not
be present in future deployments so if you're looking to redirect to a
graphical page, you may not be able to do that directly. You may also run
into deployments that have not changed the default link values so they
point to the wrong target...

Perhaps what you really want to pursue is the use of the OAuth based flows
and register your application so clients will automatically be redirected
to authenticate and approve your application to act on their behalf. If you
go with that mechanism, you'll be in line with everything else.

On Wed, Oct 21, 2015 at 6:42 AM, Matthew Sykes <matthew.sykes(a)gmail.com>
wrote:

Because Bluemix, like many other installations, uses a custom login server
to provide additional integration points and branding for its users.

Not too long ago, login and UAA were separate jobs and vendors like IBM
would deploy their own login server per a documented contract with the UAA.
Then the UAA team decided to merge UAA and Login to make some things easier
for the team to manage in the code base.

When that merge happened, vendors could still deploy their own login
server but there was no way to completely disable the one inside of the
UAA. Instead vendors could simply advertise links to the custom login
server.

In addition to the fact that the login endpoint in the UAA cannot be
disabled, it includes a fair amount of branding that can't be completely
removed or replaced without forking the UAA so there are definitely some
side effects. This topic has been raised with the Identity team a few times
- most recently during the PMC yesterday.

If you want those advertised links, you can hit the /info (or /login)
endpoint on the UAA with an 'Accept: application/json' header:

```
$ curl -s -H 'Accept: application/json' https://uaa.eu-gb.bluemix.net/info
| jq .links.login
"https://login.eu-gb.bluemix.net"
```

That pattern should work across all providers. Hope that helps a bit.

On Wed, Oct 21, 2015 at 4:06 AM, Juan Antonio Breña Moral <
bren(a)juanantonio.info> wrote:

I am doing some tests to get the token required in any operation using CF
API and yesterday, I noticed a difference between the login process in
Pivotal and the same process in Bluemix.

CF API in Pivotal:
https://api.run.pivotal.io/v2/info

has defined a URL to get token:
"token_endpoint":"https://uaa.run.pivotal.io"

and Bluemix has the same behaviour:
https://api.eu-gb.bluemix.net/v2/info
"token_endpoint":"https://uaa.eu-gb.bluemix.net"

In Pivotal and using local instance from CF, it is possible to
authenticate with the URL:
https://uaa.run.pivotal.io/login

but in bluemix:
http://uaa.eu-gb.bluemix.net/login

I can't do the same thing.

Using the field: authorization_endpoint from:
https://api.eu-gb.bluemix.net/v2/info
"authorization_endpoint":"
https://login.eu-gb.bluemix.net/UAALoginServerWAR" it is possible to get
token but why in bluemix the auth process using the UAA is disabled or it
is not running in the same way that other CF instances?
http://uaa.eu-gb.bluemix.net/login

Juan Antonio


--
Matthew Sykes
matthew.sykes(a)gmail.com


--
Matthew Sykes
matthew.sykes(a)gmail.com


Re: cf": error=2, No such file or directory and error=2

Matthew Sykes <matthew.sykes@...>
 

Sounds like the path is probably not setup correctly in the java launcher.
You should dump your path before attempting to run `cf` and make sure the
directory that contains it is actually on the PATH.

System.out.println(System.getenv("PATH"));

On Wed, Oct 21, 2015 at 1:50 AM, Varsha Nagraj <n.varsha(a)gmail.com> wrote:

This question is regarding "cf" - cloudfoundry. I have installed eclipse
Mars on Mac. Also have installed cf and run a "brew intsall protobuf". I am
executing the following in my program from eclipse :

p = Runtime.getRuntime().exec(cmd);

This is giving me: Cannot run program "cf": error=2, No such file or
directory and error=2, No such file or directory errors. The "cf" command
works fine from the command line but I see this error while running any
"cf" commands from eclipse.
--
Matthew Sykes
matthew.sykes(a)gmail.com


Re: Different behaviour in Login with Pivotal and Bluemix

Matthew Sykes <matthew.sykes@...>
 

Because Bluemix, like many other installations, uses a custom login server
to provide additional integration points and branding for its users.

Not too long ago, login and UAA were separate jobs and vendors like IBM
would deploy their own login server per a documented contract with the UAA.
Then the UAA team decided to merge UAA and Login to make some things easier
for the team to manage in the code base.

When that merge happened, vendors could still deploy their own login server
but there was no way to completely disable the one inside of the UAA.
Instead vendors could simply advertise links to the custom login server.

In addition to the fact that the login endpoint in the UAA cannot be
disabled, it includes a fair amount of branding that can't be completely
removed or replaced without forking the UAA so there are definitely some
side effects. This topic has been raised with the Identity team a few times
- most recently during the PMC yesterday.

If you want those advertised links, you can hit the /info (or /login)
endpoint on the UAA with an 'Accept: application/json' header:

```
$ curl -s -H 'Accept: application/json' https://uaa.eu-gb.bluemix.net/info
| jq .links.login
"https://login.eu-gb.bluemix.net"
```

That pattern should work across all providers. Hope that helps a bit.

On Wed, Oct 21, 2015 at 4:06 AM, Juan Antonio Breña Moral <
bren(a)juanantonio.info> wrote:

I am doing some tests to get the token required in any operation using CF
API and yesterday, I noticed a difference between the login process in
Pivotal and the same process in Bluemix.

CF API in Pivotal:
https://api.run.pivotal.io/v2/info

has defined a URL to get token:
"token_endpoint":"https://uaa.run.pivotal.io"

and Bluemix has the same behaviour:
https://api.eu-gb.bluemix.net/v2/info
"token_endpoint":"https://uaa.eu-gb.bluemix.net"

In Pivotal and using local instance from CF, it is possible to
authenticate with the URL:
https://uaa.run.pivotal.io/login

but in bluemix:
http://uaa.eu-gb.bluemix.net/login

I can't do the same thing.

Using the field: authorization_endpoint from:
https://api.eu-gb.bluemix.net/v2/info
"authorization_endpoint":"
https://login.eu-gb.bluemix.net/UAALoginServerWAR" it is possible to get
token but why in bluemix the auth process using the UAA is disabled or it
is not running in the same way that other CF instances?
http://uaa.eu-gb.bluemix.net/login

Juan Antonio


--
Matthew Sykes
matthew.sykes(a)gmail.com


Re: Loggregator/Doppler Syslog Drain Missing Logs

Mehran Saliminia
 

Hi Michael,

We have the same issue now. How did you resolve this problem?

Regards,
Mehran


Different behaviour in Login with Pivotal and Bluemix

Juan Antonio Breña Moral <bren at juanantonio.info...>
 

I am doing some tests to get the token required in any operation using CF API and yesterday, I noticed a difference between the login process in Pivotal and the same process in Bluemix.

CF API in Pivotal:
https://api.run.pivotal.io/v2/info

has defined a URL to get token:
"token_endpoint":"https://uaa.run.pivotal.io"

and Bluemix has the same behaviour:
https://api.eu-gb.bluemix.net/v2/info
"token_endpoint":"https://uaa.eu-gb.bluemix.net"

In Pivotal and using local instance from CF, it is possible to authenticate with the URL:
https://uaa.run.pivotal.io/login

but in bluemix:
http://uaa.eu-gb.bluemix.net/login

I can't do the same thing.

Using the field: authorization_endpoint from: https://api.eu-gb.bluemix.net/v2/info
"authorization_endpoint":"https://login.eu-gb.bluemix.net/UAALoginServerWAR" it is possible to get token but why in bluemix the auth process using the UAA is disabled or it is not running in the same way that other CF instances?
http://uaa.eu-gb.bluemix.net/login

Juan Antonio


Re: REST API endpoint for accessing application logs

Ponraj E
 

Hi,

Short update reg the question #2 above: I came to know from here http://docs.cloudfoundry.org/loggregator/ops.html that the number/size of log messages drained to the doppler can be controlled by a bosh deployment
manifest configuration : doppler.message_drain_buffer_size

It is specified that the doppler.message_drain_buffer_size default value is 100.

Is it 100MB?


Re: Diego and Maven support

Krzysztof Wilk
 

Hello Dan,

In my recent log I have found the following issue:
[INFO] 2015-10-18T14:16:02.329+0200 [HEALTH] OUT healthcheck failed
[INFO] 2015-10-18T14:16:02.330+0200 [HEALTH] OUT Exit status 1

Prior attempting to push application with Maven I have disabled health check:
cf set-health-check KrzysztofWilkCurrencyConverter none


Relevant snippet of my Maven configuration (pom.xml) looks as follows:
<plugin>
<groupId>org.cloudfoundry</groupId>
<artifactId>cf-maven-plugin</artifactId>
<version>${cloudfoundry.maven.version}</version>
<configuration>
<server>mycloudfoundry-instance</server>
<target>http://api.run.pivotal.io</target>
<org>krzysztofwilk.pl</org>
<space>development</space>
<appname>KrzysztofWilkCurrencyConverter</appname>
<url>krzysztofwilkcurrencyconverter.cfapps.io</url>
<env>
<JDBC_DB_URL>${jdbc.db.url}</JDBC_DB_URL>
<JDBC_USERNAME>${jdbc.username}</JDBC_USERNAME>
<JDBC_PASSWORD>${jdbc.password}</JDBC_PASSWORD>
</env>
<services>
<service>
<name>CurrencyConverterDB</name>
<label>MySQL</label>
<plan>spark</plan>
</service>
</services>
</configuration>
</plugin>

All variables are resolved correctly.

Should I also disable health check via Maven? How should I do it?

Best regards,
Krzysztof


Re: Experience Deploying CF on OpenStack

Michael Maximilien
 

Hi, all,

Final reminder. Please consider taking survey if you have not done so already:

https://www.surveymonkey.com/r/VQQZ5ZP

We plan to close the survey on Thursday. We will share the results with everyone as soon as we aggregate them.

Best,

dr.max
ibm cloud labs
sillicon valley, ca

Sent from my iPhone

On Oct 14, 2015, at 7:08 PM, Michael Maximilien <maxim(a)us.ibm.com> wrote:

Hi, BOSH, CF, and OpenStack users,

In our continued commitment to make CF the ONE true cross-cloud PaaS, we have been wondering what your experience was when deploying CF onto OpenStack using BOSH.

We created this short survey: https://www.surveymonkey.com/r/VQQZ5ZPto aggregate your experiences.

This is a short survey that will take 5 minutes (or less) of your time to fill it out.

After a week, we will share the results with the community. Our goal is to learn the most common pain point(s) and hopefully help fix these in both the CF and OpenStack communities.

Best regards,

CF BOSH team and contributors

PS: feel free to share this survey to others who might have good input to these questions


cf": error=2, No such file or directory and error=2

Varsha Nagraj
 

This question is regarding "cf" - cloudfoundry. I have installed eclipse Mars on Mac. Also have installed cf and run a "brew intsall protobuf". I am executing the following in my program from eclipse :

p = Runtime.getRuntime().exec(cmd);

This is giving me: Cannot run program "cf": error=2, No such file or directory and error=2, No such file or directory errors. The "cf" command works fine from the command line but I see this error while running any "cf" commands from eclipse.


Re: Pointer to the CF code that generates org and service instance guids?

Jean-Sebastien Delfino
 

Thanks that helps. I have a few follow up questions:

Can you quantify 'mathematically incredibly likely'?

How do you guarantee uniqueness for a single deployment (since AIUI they're
generated randomly)? Are you using the CC database for that?

What would it take to guarantee uniqueness across deployments? (I'm asking
as cloud platforms out there usually run in several datacenters or regions,
i.e. multiple deployments, so to me it would make sense to provide a way to
implement that guarantee, for example by allowing the uuid to be generated
from a namespace/name like described in RFC 4122 section 4.3 [1], but
there's probably many other ways to do this...)

[1] https://www.ietf.org/rfc/rfc4122.txt

Thanks!

- Jean-Sebastien

On Fri, Oct 9, 2015 at 11:08 PM, CF Runtime <cfruntime(a)gmail.com> wrote:

Org and Service Instance guids are generated by the Cloud Controller. It
simply uses the SecureRandom.uuid ruby method:
http://docs.ruby-lang.org/en/2.2.0/SecureRandom.html#method-c-uuid

SecureRandom.uuid implements RFC 4122 and generates guids randomly. The
guids for orgs are guaranteed to be unique for a single deployment, and
mathematically incredibly likely to be unique across all deployments.

Joseph
CF Release Integration Team

On Fri, Oct 9, 2015 at 2:51 PM, Jean-Sebastien Delfino <
jsdelfino(a)gmail.com> wrote:

Hi all,

Could somebody from the CF team point me to the CF code that generates
org and service instance guids?

Are the generated guids globally unique across multiple
deployments/instances of CF? or only unique within a single instance of CF?

Thanks!

- Jean-Sebastien


Re: REST API endpoint for accessing application logs

Ponraj E
 

Hi Warren,

Thanks. Have some questions regarding the loggregator.

1. I see the loggregator has all the logs drained by cf components + applications as well. Is the logs timebound? If yes, where do we configure that?

2. Are the logs space bound?- like loggregator can store this much MB of data. I guess its yes, because that's why we have third party log management services. So, If yes where do we specify or configure that?

3. The 'getRecentLogs' output dumped from cf-java-client truncates the data compared to the output of cf logs app_name --recent. Only some of the data is displayed. What could be the reason?


---
Ponraj


Re: region qualifier for organizations

Jean-Sebastien Delfino
 

Hi Bharath,

Sorry for the delay, I didn't realize this was a question for the Abacus
project as it didn't have the [abacus] subject tag we've been using for
Abacus discussions recently. I guess from now on I'll just check all
threads just in case :)

This is a good question. With independent deployments of CF in multiple
datacenters or regions you may need to distinguish between organization
86d0482c-7208-4f2f-8606-935c080cad41 in region 'us' and the same
organization id in region 'eu' for example.

We could add another path to the API for the cases where you care about the
region with GET /v1/regions/us/orgs/86d0482c-7208-4f2f-8606-935c080cad41/...
if that helps.

I could also sympathize with another approach, where we'd say that the
organization id being a guid should truly be *globally unique*. It looks
like the the current CF guid generation algorithm doesn't *guarantee*
uniqueness across deployments [1] but combining the region with it would
make it unique. IIUC I think that's what you're suggesting.

What do others think?

[1]
http://cf-dev.70369.x6.nabble.com/cf-dev-Pointer-to-the-CF-code-that-generates-org-and-service-instance-guids-tp2192.html

- Jean-Sebastien

On Tue, Oct 20, 2015 at 11:42 AM, Bharath Sekar <bsekar14(a)gmail.com> wrote:

Hi,
account service implementations could need additional qualifiers to
uniquely identify an organization. For example, the implementation I'm
working on needs a region along with the guid of the org.
The API to get an account given org information looks like this

GET /v1/orgs/:org_id/account

How do we want to support the additional qualifier in abacus? One solution
that I can think of is including the region in the guid. org_id could be
'guid_region'. ex:
GET /v1/orgs/86d0482c-7208-4f2f-8606-935c080cad41_us/account
Thoughts?

7041 - 7060 of 9422