Date   

Re: Loggregator/Doppler Syslog Drain Missing Logs

Mehran Saliminia
 

Hi Michael,

We have the same issue now. How did you resolve this problem?

Regards,
Mehran


Different behaviour in Login with Pivotal and Bluemix

Juan Antonio Breña Moral <bren at juanantonio.info...>
 

I am doing some tests to get the token required in any operation using CF API and yesterday, I noticed a difference between the login process in Pivotal and the same process in Bluemix.

CF API in Pivotal:
https://api.run.pivotal.io/v2/info

has defined a URL to get token:
"token_endpoint":"https://uaa.run.pivotal.io"

and Bluemix has the same behaviour:
https://api.eu-gb.bluemix.net/v2/info
"token_endpoint":"https://uaa.eu-gb.bluemix.net"

In Pivotal and using local instance from CF, it is possible to authenticate with the URL:
https://uaa.run.pivotal.io/login

but in bluemix:
http://uaa.eu-gb.bluemix.net/login

I can't do the same thing.

Using the field: authorization_endpoint from: https://api.eu-gb.bluemix.net/v2/info
"authorization_endpoint":"https://login.eu-gb.bluemix.net/UAALoginServerWAR" it is possible to get token but why in bluemix the auth process using the UAA is disabled or it is not running in the same way that other CF instances?
http://uaa.eu-gb.bluemix.net/login

Juan Antonio


Re: REST API endpoint for accessing application logs

Ponraj E
 

Hi,

Short update reg the question #2 above: I came to know from here http://docs.cloudfoundry.org/loggregator/ops.html that the number/size of log messages drained to the doppler can be controlled by a bosh deployment
manifest configuration : doppler.message_drain_buffer_size

It is specified that the doppler.message_drain_buffer_size default value is 100.

Is it 100MB?


Re: Diego and Maven support

Krzysztof Wilk
 

Hello Dan,

In my recent log I have found the following issue:
[INFO] 2015-10-18T14:16:02.329+0200 [HEALTH] OUT healthcheck failed
[INFO] 2015-10-18T14:16:02.330+0200 [HEALTH] OUT Exit status 1

Prior attempting to push application with Maven I have disabled health check:
cf set-health-check KrzysztofWilkCurrencyConverter none


Relevant snippet of my Maven configuration (pom.xml) looks as follows:
<plugin>
<groupId>org.cloudfoundry</groupId>
<artifactId>cf-maven-plugin</artifactId>
<version>${cloudfoundry.maven.version}</version>
<configuration>
<server>mycloudfoundry-instance</server>
<target>http://api.run.pivotal.io</target>
<org>krzysztofwilk.pl</org>
<space>development</space>
<appname>KrzysztofWilkCurrencyConverter</appname>
<url>krzysztofwilkcurrencyconverter.cfapps.io</url>
<env>
<JDBC_DB_URL>${jdbc.db.url}</JDBC_DB_URL>
<JDBC_USERNAME>${jdbc.username}</JDBC_USERNAME>
<JDBC_PASSWORD>${jdbc.password}</JDBC_PASSWORD>
</env>
<services>
<service>
<name>CurrencyConverterDB</name>
<label>MySQL</label>
<plan>spark</plan>
</service>
</services>
</configuration>
</plugin>

All variables are resolved correctly.

Should I also disable health check via Maven? How should I do it?

Best regards,
Krzysztof


Re: Experience Deploying CF on OpenStack

Michael Maximilien
 

Hi, all,

Final reminder. Please consider taking survey if you have not done so already:

https://www.surveymonkey.com/r/VQQZ5ZP

We plan to close the survey on Thursday. We will share the results with everyone as soon as we aggregate them.

Best,

dr.max
ibm cloud labs
sillicon valley, ca

Sent from my iPhone

On Oct 14, 2015, at 7:08 PM, Michael Maximilien <maxim(a)us.ibm.com> wrote:

Hi, BOSH, CF, and OpenStack users,

In our continued commitment to make CF the ONE true cross-cloud PaaS, we have been wondering what your experience was when deploying CF onto OpenStack using BOSH.

We created this short survey: https://www.surveymonkey.com/r/VQQZ5ZPto aggregate your experiences.

This is a short survey that will take 5 minutes (or less) of your time to fill it out.

After a week, we will share the results with the community. Our goal is to learn the most common pain point(s) and hopefully help fix these in both the CF and OpenStack communities.

Best regards,

CF BOSH team and contributors

PS: feel free to share this survey to others who might have good input to these questions


cf": error=2, No such file or directory and error=2

Varsha Nagraj
 

This question is regarding "cf" - cloudfoundry. I have installed eclipse Mars on Mac. Also have installed cf and run a "brew intsall protobuf". I am executing the following in my program from eclipse :

p = Runtime.getRuntime().exec(cmd);

This is giving me: Cannot run program "cf": error=2, No such file or directory and error=2, No such file or directory errors. The "cf" command works fine from the command line but I see this error while running any "cf" commands from eclipse.


Re: Pointer to the CF code that generates org and service instance guids?

Jean-Sebastien Delfino
 

Thanks that helps. I have a few follow up questions:

Can you quantify 'mathematically incredibly likely'?

How do you guarantee uniqueness for a single deployment (since AIUI they're
generated randomly)? Are you using the CC database for that?

What would it take to guarantee uniqueness across deployments? (I'm asking
as cloud platforms out there usually run in several datacenters or regions,
i.e. multiple deployments, so to me it would make sense to provide a way to
implement that guarantee, for example by allowing the uuid to be generated
from a namespace/name like described in RFC 4122 section 4.3 [1], but
there's probably many other ways to do this...)

[1] https://www.ietf.org/rfc/rfc4122.txt

Thanks!

- Jean-Sebastien

On Fri, Oct 9, 2015 at 11:08 PM, CF Runtime <cfruntime(a)gmail.com> wrote:

Org and Service Instance guids are generated by the Cloud Controller. It
simply uses the SecureRandom.uuid ruby method:
http://docs.ruby-lang.org/en/2.2.0/SecureRandom.html#method-c-uuid

SecureRandom.uuid implements RFC 4122 and generates guids randomly. The
guids for orgs are guaranteed to be unique for a single deployment, and
mathematically incredibly likely to be unique across all deployments.

Joseph
CF Release Integration Team

On Fri, Oct 9, 2015 at 2:51 PM, Jean-Sebastien Delfino <
jsdelfino(a)gmail.com> wrote:

Hi all,

Could somebody from the CF team point me to the CF code that generates
org and service instance guids?

Are the generated guids globally unique across multiple
deployments/instances of CF? or only unique within a single instance of CF?

Thanks!

- Jean-Sebastien


Re: REST API endpoint for accessing application logs

Ponraj E
 

Hi Warren,

Thanks. Have some questions regarding the loggregator.

1. I see the loggregator has all the logs drained by cf components + applications as well. Is the logs timebound? If yes, where do we configure that?

2. Are the logs space bound?- like loggregator can store this much MB of data. I guess its yes, because that's why we have third party log management services. So, If yes where do we specify or configure that?

3. The 'getRecentLogs' output dumped from cf-java-client truncates the data compared to the output of cf logs app_name --recent. Only some of the data is displayed. What could be the reason?


---
Ponraj


Re: region qualifier for organizations

Jean-Sebastien Delfino
 

Hi Bharath,

Sorry for the delay, I didn't realize this was a question for the Abacus
project as it didn't have the [abacus] subject tag we've been using for
Abacus discussions recently. I guess from now on I'll just check all
threads just in case :)

This is a good question. With independent deployments of CF in multiple
datacenters or regions you may need to distinguish between organization
86d0482c-7208-4f2f-8606-935c080cad41 in region 'us' and the same
organization id in region 'eu' for example.

We could add another path to the API for the cases where you care about the
region with GET /v1/regions/us/orgs/86d0482c-7208-4f2f-8606-935c080cad41/...
if that helps.

I could also sympathize with another approach, where we'd say that the
organization id being a guid should truly be *globally unique*. It looks
like the the current CF guid generation algorithm doesn't *guarantee*
uniqueness across deployments [1] but combining the region with it would
make it unique. IIUC I think that's what you're suggesting.

What do others think?

[1]
http://cf-dev.70369.x6.nabble.com/cf-dev-Pointer-to-the-CF-code-that-generates-org-and-service-instance-guids-tp2192.html

- Jean-Sebastien

On Tue, Oct 20, 2015 at 11:42 AM, Bharath Sekar <bsekar14(a)gmail.com> wrote:

Hi,
account service implementations could need additional qualifiers to
uniquely identify an organization. For example, the implementation I'm
working on needs a region along with the guid of the org.
The API to get an account given org information looks like this

GET /v1/orgs/:org_id/account

How do we want to support the additional qualifier in abacus? One solution
that I can think of is including the region in the guid. org_id could be
'guid_region'. ex:
GET /v1/orgs/86d0482c-7208-4f2f-8606-935c080cad41_us/account
Thoughts?


Need Help: Seeking tutorial on CF and WordPress Multisite

v Bailey
 

I am totally green about asking the proper question so here's what I'm researching...

I'm a decently adept web designer who specializes in WordPress. I would like to pursue some larger clients and offer to move their sites to the cloud (as well as perform their makeovers, of course) but in a couple of cases, the organizations have multiple branches, each with their own website, and it seems to make sense to use Multisite instead of individual websites. That said, I've never used Multisite so my concern is how does it work to create multiple websites+database for a single client...on the cloud?

I suspect I should be asking more but since I don't entirely understand what the setup should be I'll start with a simple initial question. I'd LOVE to hear some insight into how to setup these kind of clients.

Please be gentle with me, I bruise easy. Simple explanations get more mileage with me. Thank you kindly in advance!!


Re: Open sourcing our S3 service broker

Gwenn Etourneau
 

Maybe as the same we have for bosh release (bosh.io) we can create
brokers.cloudfoundry.org

On Wed, Oct 21, 2015 at 6:29 AM, David Ehringer <david.ehringer(a)gmail.com>
wrote:

I think discovery mechanism would be very valuable, especially in cases
like this where there was already an S3 service broker open sourced last
year: https://github.com/cloudfoundry-community/s3-cf-service-broker .
Not sure if this meets the same needs but it would be great for people to
be able to find existing broker projects they can contribute enhancements
to if it makes sense rather than duplicate effort, let alone those just
interested in running brokers.

I would be interested in working on something although I'm not sure how
much bandwidth I have to contribute at the moment. I'm assuming whatever
is built could be hosted in a public CF instance just like
plugins.cloudfoundry.org? If dynamic features like reviews were desired
and to be supported, would any specific database services be available for
use?


Re: Defining a singular plan for all services

Shannon Coen
 

Hello Kayode,

So glad to hear you're building a broker for your API Gateway!

If your broker offers multiple services, and/or multiple plans of a
service, it is up to you as the broker author to expose all of these plans
individually in the catalog endpoint [1]. The Service Broker API doesn't
support the notion of a "default plan" shared by multiple services because,
while plans may have metadata in common, every plan must have a unique
identifier.

As Cloud Foundry is not prescriptive about implementation or deployment of
your broker, mention of settings.yml is a bit confusing. While you may use
a configuration file called settings.yml, another service broker author may
use a very different mechanism for configuration.

[1] http://docs.cloudfoundry.org/services/api.html#catalog-mgmt

Warm regards,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Mon, Oct 19, 2015 at 6:17 AM, Kayode Odeyemi <dreyemi(a)gmail.com> wrote:

A custom built service.

We're building a custom API Gateway for our apps.


On Fri, Oct 16, 2015 at 11:29 PM, CF Runtime <cfruntime(a)gmail.com> wrote:

What services do you want to define this default plan for?

Natalie & Mikhail
Runtime & OSS Integration

On Wed, Oct 14, 2015 at 2:22 AM, Kayode Odeyemi <dreyemi(a)gmail.com>
wrote:

Hi,

How can I define a default plan to be used by more than one services
without repeating the same plan definition in settings.yml file?

Appreciate your help.


Re: Open sourcing our S3 service broker

David Ehringer
 

I think discovery mechanism would be very valuable, especially in cases like this where there was already an S3 service broker open sourced last year: https://github.com/cloudfoundry-community/s3-cf-service-broker . Not sure if this meets the same needs but it would be great for people to be able to find existing broker projects they can contribute enhancements to if it makes sense rather than duplicate effort, let alone those just interested in running brokers.

I would be interested in working on something although I'm not sure how much bandwidth I have to contribute at the moment. I'm assuming whatever is built could be hosted in a public CF instance just like plugins.cloudfoundry.org? If dynamic features like reviews were desired and to be supported, would any specific database services be available for use?


Re: Open sourcing our S3 service broker

Shannon Coen
 

Awesome! Thank you for the contribution.

Would love to see brokers be more easily discoverable. Would someone from
the community have the bandwidth to develop a site to make brokered
services for CF discoverable? I'm thinking reviews, commercial vs OSS,
logos, deployment strategy, links to install instructions, etc.


Best,
Shannon

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Tue, Oct 20, 2015 at 11:58 AM, Eric Poelke <epoelke(a)gmail.com> wrote:

Throwing our S3 service broker out there for others that may find it
useful --

https://github.com/epoelke/s3-service-broker

Currently has default plans for most regions as well as location
constraint support for buckets created in those regions. IAM users get
created for each bucket, as well as a access policy that restricts the user
to only the bucket created for the user. API keys and endpoints are
returned as VCAP credentials when binding the service to an app.


Open sourcing our S3 service broker

Eric Poelke
 

Throwing our S3 service broker out there for others that may find it useful --

https://github.com/epoelke/s3-service-broker

Currently has default plans for most regions as well as location constraint support for buckets created in those regions. IAM users get created for each bucket, as well as a access policy that restricts the user to only the bucket created for the user. API keys and endpoints are returned as VCAP credentials when binding the service to an app.


region qualifier for organizations

Bharath Sekar
 

Hi,
account service implementations could need additional qualifiers to uniquely identify an organization. For example, the implementation I'm working on needs a region along with the guid of the org.
The API to get an account given org information looks like this

GET /v1/orgs/:org_id/account

How do we want to support the additional qualifier in abacus? One solution that I can think of is including the region in the guid. org_id could be 'guid_region'. ex:
GET /v1/orgs/86d0482c-7208-4f2f-8606-935c080cad41_us/account
Thoughts?


Action Required : UAA Backwards Compatibility for OpenID Connect ID Token Response Type

Sree Tummidi
 

Hi All,

Starting with *CF Release 220*, we have added OpenID Connect ID Token
support in UAA.

Currently this feature is *disabled* by default to remain fully backwards
compatible.


uaa.id_token.disable

description: When set to true, requests to /oauth/authorize will ignore the
response_type=id_token parameter

*default: true*


Our plan is to provide a *30 Day window *(roughly cf-release slated for 2nd
week of November) for Service Authors and other integrators using UAA as
the token server. We will be removing the feature flag and support ID token
response by default after 2nd week of November

*Action Required:*

1. If you have a custom implementation of OAuth client, please update
the parsing logic to handle the response of the /oauth/authorize endpoint.
The main change is that the Location header will have a Fragment (#) and
not a Query String (?). *See [1] below*
2. If you are using the cf-uaa-lib
<https://github.com/cloudfoundry/cf-uaa-lib>, Please update to version
*3.2.4*
3. If you are using a third party library for OAuth/OpenID Connect, most
likely it already supports parsing the response properly and there is no
action required.



Please test your Services or Application using UAA with the
*uaa.id_token.disable* set to *false*



*[1] Details*

During invokation of the /oauth/authorize URL, the normal process is to
specify response_type=code

Some libraries have been specifying response_type=code+id_token

This is a OpenID Connect extension. Previously the UAA ignored the id_token
response_type, but now we have added support. This changes the response of
the /oauth/authorize. The main change is that the Location header will have
a Fragment (#) and not a Query String (?)

$ uaac target http://login. <http://login.identity.cf-app.com/>
{system-domain}

$ uaac token owner get cf marissa -s "" -p koala

$ uaac curl "
<http://login.identity.cf-app.com/oauth/authorize?grant_type=authorization_code&>
http://login. <http://login.identity.cf-app.com/>{system-domain}
/oauth/authorize?grant_type=authorization_code&response_type=code
&client_id=ssh-proxy&redirect_uri=http://localhost"

$ uaac curl "http://login. <http://login.identity.cf-app.com/>
{system-domain}/oauth/authorize?grant_type=authorization_code&
<http://login.identity.cf-app.com/oauth/authorize?grant_type=authorization_code&>
response_type=code+id_token&client_id=ssh-proxy&redirect_uri=
http://localhost"

Location: http://localhost?code=5G9Rm1


*response_type=code+id_token*

Location: http://localhost#
token_type=bearer&id_token=eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiIwMjJmMGUzOC0xNTZlLTRjY2EtYTFmMS1kMGMwNTNkMDY1ZGMiLCJ1c2VyX25hbWUiOiJtYXJpc3NhIiwib3JpZ2luIjoibGRhcCIsImlzcyI6Imh0dHBzOi8vdWFhLmlkZW50aXR5LmNmLWFwcC5jb20vb2F1dGgvdG9rZW4iLCJjbGllbnRfaWQiOiJzc2gtcHJveHkiLCJhdWQiOlsic3NoLXByb3h5Il0sInppZCI6InVhYSIsImdyYW50X3R5cGUiOiJhdXRob3JpemF0aW9uX2NvZGUiLCJ1c2VyX2lkIjoiMDIyZjBlMzgtMTU2ZS00Y2NhLWExZjEtZDBjMDUzZDA2NWRjIiwiYXpwIjoic3NoLXByb3h5Iiwic2NvcGUiOlsib3BlbmlkIl0sImV4cCI6MTQ0MjQ2NDM5MSwiaWF0IjoxNDQyNDIxMTkxLCJqdGkiOiI2ZTBmOThlZS02YzNjLTQzN2UtOGYzZi0yNjE2ZDZhM2U2NzkiLCJlbWFpbCI6Im1hcmlzc2FAdGVzdC5jb20iLCJyZXZfc2lnIjoiZGJlYjYxN2UiLCJjaWQiOiJzc2gtcHJveHkifQ.AQtGa5N9QW47jAehCEIz9K46yNTIJpK5SKrp5IqAZ2AqZhNLzwryLw0wJmZkZ3v3wXqpwm_IryJnJmzui0hsM8ZYsyeDjtGtmd35QjxBe785nVGe2GZeyVQ0CqLNW1C7w34VCnvSfIxo8ecstaS4O0MkBxjQR0CQRQ3XHS4RIJg&
code=aGp5Ig
&expires_in=43199&scope=cloud_controller.read%20cloud_controller.write%20openid&jti=6e0f98ee-6c3c-437e-8f3f-2616d6a3e679


Thanks,
Sree Tummidi
Sr. Product Manager
Identity - Pivotal Cloud Foundry


Re: How to specify "no default shared domain" in the cf-deployment-manifest.yml?

Amit Kumar Gupta
 

Hi Jim,

Try setting the `app_domains` property in your manifest to an empty array.

Cheers,
Amit

On Mon, Oct 19, 2015 at 7:05 PM, Jim Lin <jimlintw922(a)gmail.com> wrote:

Hi all

I don't need the default shared domain (i.e., ${CF_IP}.xip.io). I can use
command line to remove it after CF deployment. However, every time I
re-deploy CF, I need manually remove it again. My question is how to
specify "no default shared domain" in the cf-deployment-manifest.yml?

Thanks all.

Sincerely,
Jim


Re: Some results running CloudController under JRuby

Steffen Uhlig
 

Hi,
 
Dieu wrote:
 
> Would be interested to see how this performs on aws or on soft layer.
 
Good point  - is there an AWS envrionment we could use? We'll look into Softlayer.
 
> Also, were you able to run CATS? 
 
We did not attempt to run CATS, because we focused on one endpoint. There others would probably not have functioned.
 
> Or is more work needed to deal with NATS etc?
 
More work is certainly needed for that. We haven't bothered so far to update the depencencies we have 'patched' for the spike. 
 
> Other pros/cons that you've found?
 
Pros not mentioned yet:
 
- There are a lot more tools to run and monitor JVM based applications.
- JRuby in interpreted mode did not seem to be slower, in terms of development workflow.
 
Cons:
 
- JRruby is an additional layer (of complexity) on top of Ruby
- JRuby is still more in a niche
 
Amit wrote:
 
> what are differences to pre-packaging dependencies?
> any differences in pre-packaging time?
> what differences for packaging/compilation dependencies?
 
Our approach was minimal; we captured code changes to CC and dependencies that need updates on a branch. Those changes would need to be translated into packaging.
From our spike, we think that the (pre-) packaging would be similar to how UAA handles the OpenJDK. JRuby itself goes on top of that, and is simply an additional unzip command.
Some gems will need updates to work with JRuby, (see Gemfile in the branch). Mostly around yaml support, thin, and NATS. 
 
> any difference in packaging/compilation time?
 
We did not measure times. From a Ruby perspective, JRuby did not feel different. The Ruby-to-Java compilation can happen in two modes; interpreted and compiled. We only worked in the interpreted mode. In compiled, the Ruby code is compiled to Java upfront, which should lead to further improvements.
 
> there are going to be some new job and packages blobs, and maybe some old ones go away? what are the size differences?
 
We can re-use OpenJDK from UAA. JRuby is a 40 MiB download, where we can eliminate samples etc., so probably 30 MiB in addition.
 
> any changes to start-up or update times during bosh deploy/update?
 
Sorry, we did not measure this, either ;-)
 
 
Regards
 
Marc & Steffen
 
 

----- Original message -----
From: Amit Gupta <agupta@...>
To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@...>
Cc:
Subject: [cf-dev] Re: Re: Some results running CloudController under JRuby
Date: Mon, Oct 19, 2015 11:40 PM
 
Awesome!
 
Got some questions:
- what are differences to pre-packaging dependencies?
- any differences in pre-packaging time?
- what differences for packaging/compilation dependencies?
- any difference in packaging/compilation time?
- there are going to be some new job and packages blobs, and maybe some old ones go away? what are the size differences?
- any changes to start-up or update times during bosh deploy/update?
 
Amit
 
On Mon, Oct 19, 2015 at 2:06 PM, Dieu Cao <dcao@...> wrote:
Hi Steffen,
 
Cool stuff.  It's good to see there's some improvement under jruby.
Would be interested to see how this performs on aws or on soft layer.
Also, were you able to run CATS? 
Or is more work needed to deal with NATS etc?
Other pros/cons that you've found?
 
-Dieu
CF CAPI PM
 
 
 
On Mon, Oct 19, 2015 at 1:25 PM, Steffen Uhlig <Steffen.Uhlig@...> wrote:
Hi,
 
the Flintstone team recently spent some time researching the potential benefits of running the CloudController under JRuby. We were hoping to find evidence that JRuby (and the underlying JVM) would allow us to make better use of multiple cores, and maybe also lead to significant response time improvements when answering many parallel requests.
 
This exercise wasn't set up as a scientific benchmark; it is more of a spike that would allow us to judge whether it is worth investigating the next level of detail.
 
We would like to share some early results in the hope to get feedback from the community.
 
In our measurements, we saw 20..30% improvement in both average response time and throughput when 10 or more concurrent requests were made (using ApacheBench against the `/v2/orgs/*/spaces` endpoint).
 
Graphs:
    * Throughput: https://goo.gl/NuWkvf
    * Response Time: https://goo.gl/ItPBHN
 
We patched a CC VM to use JRuby 9000 under OpenJDK 8. WEBrick was used as we weren't able to quickly find a drop-in replacement for Thin (as used under MRI). All measurements were taken on a 2014 MacBook Pro running Cloudfoundry in a BOSH Lite environment. Simulating network latency by adding a 100 ms sleep to each request did not change the overall picture.
 
For more details see the spreadsheet* and our repository** with the test scripts.
 
Steffen
(on behalf of the Flintstone Team)
 
 
 


Re: How to explicitly specify the password for the account admin?

Jim Lin <jimlintw922@...>
 

It works!. Thanks!