Date   

SSO Kerberos with Spring

Leumas Yajiv
 

Hi.

I am trying to integrate with application(Java/Spring) authentication with an SSO through Kerberos. Has anyone done this before, I am using tomcat 8, openjdk 7 with java-buildpack version 3.0. And I am using r170 release of CF.

I have gone through this documentation, https://spring.io/blog/2009/09/28/spring-security-kerberos-spnego-extension, but I fail to understand how that will work on CF.

ooo Leuma


Re: Garden Port Assignment Story

Will Pragnell <wpragnell@...>
 

Hi Mike,

What I think you're saying is that once the new
`prune_on_config_unavailable` property is available in the router, and if
it's set to `false`, there's a case when NATs is not reachable from the
router in which potentially stale routes will continue to exist until the
router can reach NATs again. Is that correct?

(Sorry to repeat you back at yourself, just want to make sure I've
understood you correctly.)

Will

On 23 November 2015 at 19:02, Mike Youngstrom <youngm(a)gmail.com> wrote:

Hi Will,

Though I see the main reason for the issue assuming a healthy running
environment I've also experienced a deploy related issue that more unique
port assignment could help defend against. During one of our deploys the
routers finished deployed before the DEAs. When the DEAs started rolling,
for some reason some of our routers stopped getting route updates from
NATs. This caused their route tables to go stale and as apps started
rolling new apps started getting assigned ports previously held by other
apps. Which caused a number of our hosts to be mis-routed.

Though the root cause was probably some bug in the Nats client in GoRouter
the runtime team had apparently experienced a similar issue in the past [0]
which caused them to implement code that would delete stale routes even
then a router couldn't connect to NATs. The Router team is now planning to
optionally remove this failsafe [1]. I'm hoping that with the removal of
this failsafe (which I'm planning to take advantage of) this tracker story
will help protect us from the problem we experienced before from happening
again.

If the ports simply reset on a stemcell upgrade this issue provides no
defense for the problem we had before.

Does that make sense Will?

Mike

[0]
https://groups.google.com/a/cloudfoundry.org/d/msg/vcap-dev/yuVYCZkMLG8/7t8FHnFzWEsJ
[1] https://www.pivotaltracker.com/story/show/108659764

On Mon, Nov 23, 2015 at 11:11 AM, Will Pragnell <wpragnell(a)pivotal.io>
wrote:

Hi Mike,

What's the motivation for wanting rolling port assignment to persist
across e.g. stemcell upgrade? The motivation for this story is to prevent
stale routes from sending traffic to the wrong containers. Our assumption
is that stale routes won't ever exist for anything close to the amount of
time it takes BOSH to destroy and recreate a VM. Have we missed something
in making that assumption?

On your second point, I see your concern. We've talked about the
possibility of implementing FIFO semantics on free ports (when a port that
was in use becomes free, it goes to the end of the queue of available
ports) to decrease the chances of traffic reaching the wrong container as
far as possible. It's possible that the rolling ports approach is "good
enough" though. We're still trying to understand whether that's actually
the case.

The consistent hashing idea is interesting, but a few folks have
suggested that with a relatively small range of available ports (5000 by
default) that the chances of collision are actually higher than we'd want.
I'll see if someone wants to lay down some maths to give that idea some
credence.

Cheers,
Will

On 23 November 2015 at 08:47, Mike Youngstrom <youngm(a)gmail.com> wrote:

Since I cannot comment in tracker I'm starting this thread to discuss
story:
https://www.pivotaltracker.com/n/projects/1158420/stories/92085170

Some comments I have:

* Although I can see how a rolling port assignment could be maintained
across garden/diego restarts I'd also like the story to ensure that the
rolling port assignments get maintained across a Stemcell upgrade without
the need for persistent disks on each cell. Perhaps etcd?

* Another thing to keep in mind. Although a rolling port value may not
duplicate ports 100% of the time in a short lived container in a long lived
container it seems to me that a rolling port assignment becomes no more
successful than a random port assignment if the container lives long enough
for the port assignment loop to loop a few times.

* Has there been any consideration to using an incremental consistent
hash of the app_guid to assign ports? A consistent hash would have the
benefit of being stateless. It also would have the benefit of increasing
the likely hood that if a request is sent to a stale route it may be to the
correct app anyway.

Thoughts?

Mike


Re: CF-RELEASE v202 UPLOAD ERROR

Parthiban Annadurai <senjiparthi@...>
 

Okay.. Let me try with it.. Thanks..

On 24 November 2015 at 14:02, ronak banka <ronakbanka.cse(a)gmail.com> wrote:

Subnet ranges on which your other components are provisioned.

allow_from_entries:
- 192.168.33.0/24




On Tue, Nov 24, 2015 at 5:16 PM, Parthiban Annadurai <
senjiparthi(a)gmail.com> wrote:

Hello Ronak,
Actually, previously i have given the values for
ALLOW_FROM_ENTRIES, after seeing some mail groups only i changed it to
NULL. Could you tell me which IP i need to give their or something else??
Thanks..

On 24 November 2015 at 13:23, ronak banka <ronakbanka.cse(a)gmail.com>
wrote:

Hi Parthiban,

In your manifest , there is a global property block

nfs_server:
address: 192.168.33.53
allow_from_entries:
- null
- null
share: null

allow from entries are provided for cc individual property and not for actual debian nfs server, that is possible reason cc is not able to write to nfs


https://github.com/cloudfoundry/cf-release/blob/master/jobs/debian_nfs_server/spec#L20

Thanks
Ronak



On Tue, Nov 24, 2015 at 3:42 PM, Parthiban Annadurai <
senjiparthi(a)gmail.com> wrote:

Thanks Amit for your faster reply. FYI, I have shared my deployment
manifest too. I got struck in this issue for past couple of weeks. Thanks..

On 24 November 2015 at 12:00, Amit Gupta <agupta(a)pivotal.io> wrote:

Hi Parthiban,

Sorry to hear your deployment is still getting stuck. As Warren
points out, based on your log output, it looks like an issue with NFS
configuration. I will ask the CAPI team, who are experts on cloud
controller and NFS server, to take a look at your question.

Best,
Amit

On Thu, Nov 19, 2015 at 8:11 PM, Parthiban Annadurai <
senjiparthi(a)gmail.com> wrote:

Thanks for your suggestions Warren. I am attaching the Manifest file
which used for the deployment. Am also suspecting that the problem with the
NFS Server Configuration only.

On 19 November 2015 at 22:32, Warren Fernandes <wfernandes(a)pivotal.io
wrote:
Hey Parthiban,

It seems that there may be a misconfiguration in your manifest.
Did you configure the nfs_server properties?


https://github.com/cloudfoundry/cf-release/blob/master/templates/cf-jobs.yml#L19-L22

The api_z1 pulls the above properties in here.
https://github.com/cloudfoundry/cf-release/blob/master/templates/cf-jobs.yml#L368
.

Is it possible to share your manifest with us via a gist or
attachment? Please remove any sensitive information like passwords, certs
and keys.

Thanks.


Re: CF-RELEASE v202 UPLOAD ERROR

Ronak Banka
 

Subnet ranges on which your other components are provisioned.

allow_from_entries:
- 192.168.33.0/24




On Tue, Nov 24, 2015 at 5:16 PM, Parthiban Annadurai <senjiparthi(a)gmail.com>
wrote:

Hello Ronak,
Actually, previously i have given the values for
ALLOW_FROM_ENTRIES, after seeing some mail groups only i changed it to
NULL. Could you tell me which IP i need to give their or something else??
Thanks..

On 24 November 2015 at 13:23, ronak banka <ronakbanka.cse(a)gmail.com>
wrote:

Hi Parthiban,

In your manifest , there is a global property block

nfs_server:
address: 192.168.33.53
allow_from_entries:
- null
- null
share: null

allow from entries are provided for cc individual property and not for actual debian nfs server, that is possible reason cc is not able to write to nfs


https://github.com/cloudfoundry/cf-release/blob/master/jobs/debian_nfs_server/spec#L20

Thanks
Ronak



On Tue, Nov 24, 2015 at 3:42 PM, Parthiban Annadurai <
senjiparthi(a)gmail.com> wrote:

Thanks Amit for your faster reply. FYI, I have shared my deployment
manifest too. I got struck in this issue for past couple of weeks. Thanks..

On 24 November 2015 at 12:00, Amit Gupta <agupta(a)pivotal.io> wrote:

Hi Parthiban,

Sorry to hear your deployment is still getting stuck. As Warren points
out, based on your log output, it looks like an issue with NFS
configuration. I will ask the CAPI team, who are experts on cloud
controller and NFS server, to take a look at your question.

Best,
Amit

On Thu, Nov 19, 2015 at 8:11 PM, Parthiban Annadurai <
senjiparthi(a)gmail.com> wrote:

Thanks for your suggestions Warren. I am attaching the Manifest file
which used for the deployment. Am also suspecting that the problem with the
NFS Server Configuration only.

On 19 November 2015 at 22:32, Warren Fernandes <wfernandes(a)pivotal.io>
wrote:

Hey Parthiban,

It seems that there may be a misconfiguration in your manifest.
Did you configure the nfs_server properties?


https://github.com/cloudfoundry/cf-release/blob/master/templates/cf-jobs.yml#L19-L22

The api_z1 pulls the above properties in here.
https://github.com/cloudfoundry/cf-release/blob/master/templates/cf-jobs.yml#L368
.

Is it possible to share your manifest with us via a gist or
attachment? Please remove any sensitive information like passwords, certs
and keys.

Thanks.


Re: CF-RELEASE v202 UPLOAD ERROR

Parthiban Annadurai <senjiparthi@...>
 

Hello Ronak,
Actually, previously i have given the values for
ALLOW_FROM_ENTRIES, after seeing some mail groups only i changed it to
NULL. Could you tell me which IP i need to give their or something else??
Thanks..

On 24 November 2015 at 13:23, ronak banka <ronakbanka.cse(a)gmail.com> wrote:

Hi Parthiban,

In your manifest , there is a global property block

nfs_server:
address: 192.168.33.53
allow_from_entries:
- null
- null
share: null

allow from entries are provided for cc individual property and not for actual debian nfs server, that is possible reason cc is not able to write to nfs


https://github.com/cloudfoundry/cf-release/blob/master/jobs/debian_nfs_server/spec#L20

Thanks
Ronak



On Tue, Nov 24, 2015 at 3:42 PM, Parthiban Annadurai <
senjiparthi(a)gmail.com> wrote:

Thanks Amit for your faster reply. FYI, I have shared my deployment
manifest too. I got struck in this issue for past couple of weeks. Thanks..

On 24 November 2015 at 12:00, Amit Gupta <agupta(a)pivotal.io> wrote:

Hi Parthiban,

Sorry to hear your deployment is still getting stuck. As Warren points
out, based on your log output, it looks like an issue with NFS
configuration. I will ask the CAPI team, who are experts on cloud
controller and NFS server, to take a look at your question.

Best,
Amit

On Thu, Nov 19, 2015 at 8:11 PM, Parthiban Annadurai <
senjiparthi(a)gmail.com> wrote:

Thanks for your suggestions Warren. I am attaching the Manifest file
which used for the deployment. Am also suspecting that the problem with the
NFS Server Configuration only.

On 19 November 2015 at 22:32, Warren Fernandes <wfernandes(a)pivotal.io>
wrote:

Hey Parthiban,

It seems that there may be a misconfiguration in your manifest.
Did you configure the nfs_server properties?


https://github.com/cloudfoundry/cf-release/blob/master/templates/cf-jobs.yml#L19-L22

The api_z1 pulls the above properties in here.
https://github.com/cloudfoundry/cf-release/blob/master/templates/cf-jobs.yml#L368
.

Is it possible to share your manifest with us via a gist or
attachment? Please remove any sensitive information like passwords, certs
and keys.

Thanks.


Re: CF-RELEASE v202 UPLOAD ERROR

Ronak Banka
 

Hi Parthiban,

In your manifest , there is a global property block

nfs_server:
address: 192.168.33.53
allow_from_entries:
- null
- null
share: null

allow from entries are provided for cc individual property and not for
actual debian nfs server, that is possible reason cc is not able to
write to nfs

https://github.com/cloudfoundry/cf-release/blob/master/jobs/debian_nfs_server/spec#L20

Thanks
Ronak



On Tue, Nov 24, 2015 at 3:42 PM, Parthiban Annadurai <senjiparthi(a)gmail.com>
wrote:

Thanks Amit for your faster reply. FYI, I have shared my deployment
manifest too. I got struck in this issue for past couple of weeks. Thanks..

On 24 November 2015 at 12:00, Amit Gupta <agupta(a)pivotal.io> wrote:

Hi Parthiban,

Sorry to hear your deployment is still getting stuck. As Warren points
out, based on your log output, it looks like an issue with NFS
configuration. I will ask the CAPI team, who are experts on cloud
controller and NFS server, to take a look at your question.

Best,
Amit

On Thu, Nov 19, 2015 at 8:11 PM, Parthiban Annadurai <
senjiparthi(a)gmail.com> wrote:

Thanks for your suggestions Warren. I am attaching the Manifest file
which used for the deployment. Am also suspecting that the problem with the
NFS Server Configuration only.

On 19 November 2015 at 22:32, Warren Fernandes <wfernandes(a)pivotal.io>
wrote:

Hey Parthiban,

It seems that there may be a misconfiguration in your manifest.
Did you configure the nfs_server properties?


https://github.com/cloudfoundry/cf-release/blob/master/templates/cf-jobs.yml#L19-L22

The api_z1 pulls the above properties in here.
https://github.com/cloudfoundry/cf-release/blob/master/templates/cf-jobs.yml#L368
.

Is it possible to share your manifest with us via a gist or attachment?
Please remove any sensitive information like passwords, certs and keys.

Thanks.


Re: CF-RELEASE v202 UPLOAD ERROR

Parthiban Annadurai <senjiparthi@...>
 

Thanks Amit for your faster reply. FYI, I have shared my deployment
manifest too. I got struck in this issue for past couple of weeks. Thanks..

On 24 November 2015 at 12:00, Amit Gupta <agupta(a)pivotal.io> wrote:

Hi Parthiban,

Sorry to hear your deployment is still getting stuck. As Warren points
out, based on your log output, it looks like an issue with NFS
configuration. I will ask the CAPI team, who are experts on cloud
controller and NFS server, to take a look at your question.

Best,
Amit

On Thu, Nov 19, 2015 at 8:11 PM, Parthiban Annadurai <
senjiparthi(a)gmail.com> wrote:

Thanks for your suggestions Warren. I am attaching the Manifest file
which used for the deployment. Am also suspecting that the problem with the
NFS Server Configuration only.

On 19 November 2015 at 22:32, Warren Fernandes <wfernandes(a)pivotal.io>
wrote:

Hey Parthiban,

It seems that there may be a misconfiguration in your manifest.
Did you configure the nfs_server properties?


https://github.com/cloudfoundry/cf-release/blob/master/templates/cf-jobs.yml#L19-L22

The api_z1 pulls the above properties in here.
https://github.com/cloudfoundry/cf-release/blob/master/templates/cf-jobs.yml#L368
.

Is it possible to share your manifest with us via a gist or attachment?
Please remove any sensitive information like passwords, certs and keys.

Thanks.


Re: CF-RELEASE v202 UPLOAD ERROR

Amit Kumar Gupta
 

Hi Parthiban,

Sorry to hear your deployment is still getting stuck. As Warren points
out, based on your log output, it looks like an issue with NFS
configuration. I will ask the CAPI team, who are experts on cloud
controller and NFS server, to take a look at your question.

Best,
Amit

On Thu, Nov 19, 2015 at 8:11 PM, Parthiban Annadurai <senjiparthi(a)gmail.com>
wrote:

Thanks for your suggestions Warren. I am attaching the Manifest file which
used for the deployment. Am also suspecting that the problem with the NFS
Server Configuration only.

On 19 November 2015 at 22:32, Warren Fernandes <wfernandes(a)pivotal.io>
wrote:

Hey Parthiban,

It seems that there may be a misconfiguration in your manifest.
Did you configure the nfs_server properties?


https://github.com/cloudfoundry/cf-release/blob/master/templates/cf-jobs.yml#L19-L22

The api_z1 pulls the above properties in here.
https://github.com/cloudfoundry/cf-release/blob/master/templates/cf-jobs.yml#L368
.

Is it possible to share your manifest with us via a gist or attachment?
Please remove any sensitive information like passwords, certs and keys.

Thanks.


Re: CF-RELEASE v202 UPLOAD ERROR

Parthiban Annadurai <senjiparthi@...>
 

Hello All,
Is there anything i need to change on my CF Deployment
Manifest for the deployments or not? Could you please help on this to
solve? Thanks.

On 20 November 2015 at 09:41, Parthiban Annadurai <senjiparthi(a)gmail.com>
wrote:

Thanks for your suggestions Warren. I am attaching the Manifest file which
used for the deployment. Am also suspecting that the problem with the NFS
Server Configuration only.

On 19 November 2015 at 22:32, Warren Fernandes <wfernandes(a)pivotal.io>
wrote:

Hey Parthiban,

It seems that there may be a misconfiguration in your manifest.
Did you configure the nfs_server properties?


https://github.com/cloudfoundry/cf-release/blob/master/templates/cf-jobs.yml#L19-L22

The api_z1 pulls the above properties in here.
https://github.com/cloudfoundry/cf-release/blob/master/templates/cf-jobs.yml#L368
.

Is it possible to share your manifest with us via a gist or attachment?
Please remove any sensitive information like passwords, certs and keys.

Thanks.


Re: Bosh release for Docker

Vedran Lerenc <vlerenc@...>
 

Hi Rajesh,

Is Elastic Runtime aware of any of these containers or the services or the swarm cluster?
Not if you don't make it aware.

How does the route to the containers or services get registered or traffic get routed?
Only the broker gets registered at the router, then it gets registered at the CC. When containers are created (on request of the CC/broker, by the Docker engine, potentially through Swarm), their exposed ports were automatically mapped in the past by the Docker engine into the ephemeral port range and are today (by default, that is) mapped by the broker (to keep stable port mappings as the automatic port mappings are unstable/may change when the Docker engine/VM crashes/is restarted). The broker returns the coordinates (host, port, credentials, etc.) through the CC to the app which then (given access is permitted, e.g. security groups) can access the container directly.

Can you take logs out of the containers (similar to Diego containers)?
Yes, by means of [Docker logging drivers](https://docs.docker.com/engine/reference/logging/overview/), see job spec https://github.com/cloudfoundry-community/docker-boshrelease/blob/master/jobs/docker/spec#L71-L77

Can you bring your own container/container stem cell?
Nope, not if you don't create an image that plays by the rules of the broker (obeys its contract) and add it to the services section of the broker configuration/expose it as new service.

Can you connect to an internal Docker Repo?
Yes, by means of a registry mirror, though I never tried it, see job spec https://github.com/cloudfoundry-community/docker-boshrelease/blob/master/jobs/docker/spec#L80-L81

Cheers, Vedran


Re: Pluggable Resource Scheduling

taichi nakashima
 

+1 having this pluggable scheduling feature !

--
tcnksm

2015年11月14日(土) 6:55 Deepak Vij (A) <deepak.vij(a)huawei.com>:

Hi Idit, good to hear from you. In that case, we have covered all bases
and good to go on this. We will touch base as we discussed earlier. Thanks.



- Deepak



*From:* Gmail [mailto:idit.levine(a)gmail.com]
*Sent:* Friday, November 13, 2015 1:43 PM


*To:* Discussions about Cloud Foundry projects and the system overall.

*Subject:* [cf-dev] Re: Re: Re: Re: Re: Pluggable Resource Scheduling



We can add to mesosphere a native support for Garden containers. That is
very easy to do ....

Sent from my iPhone


On Nov 13, 2015, at 2:51 PM, Deepak Vij (A) <deepak.vij(a)huawei.com> wrote:

If Mesosphere can do it for Kubernetes, CF/Diego is doable too. I am not
worried about that, it is a solved problem.



The only concern I have is regarding deployment of Garden Container
environment in Mesos/Slave/Executor. Although, this is not CF/Diego issues.



Because Mesos is not Garden Container environment aware, its underlying
DRF scheduling algorithm may not have visibility to the resources being
consumed within the Garden Container. Unless, we wrap Garden Container
within the Docker container as Mesos supports Docker container environment.
Although, this may not be right approach as it opens up another can of
worms – Gardner Container nested within Docker Container. For Kubernetes
environment this is not an issue as it uses Docker container to begin with.



- Deepak



*From:* resouer(a)163.com [mailto:resouer(a)163.com <resouer(a)163.com>] *On
Behalf Of *Zhang Lei
*Sent:* Thursday, November 12, 2015 8:24 PM
*To:* Discussions about Cloud Foundry projects and the system overall.
*Subject:* [cf-dev] Re: Re: Re: Pluggable Resource Scheduling



You can add different scheduling strategy into Diego by implementing a
scheduler plugin.


But not Mesos, that would be a huge task and another story.

The reason Kubernetes can integrate Mesos as scheduler (can work, not
perfect) is due to Mesosphere is doing that part, I'm afraid ...

在 2015-11-13 03:57:52,"Deepak Vij (A)" <deepak.vij(a)huawei.com> 写道:


I did not mean to replace the whole “Diego” environment itself. What I was
thinking was more in terms of plug-ability within Diego itself. This is so
that “Auctioneer” component can be turned into a “Mesos Framework” as one
of the scheduling options. By doing that, “Auctioneer” can start accepting
“Mesos Offers” instead of native “Auctioning based Diego Resource
Scheduling”. Rest of the runtime environment including Garden, Rep etc.,
they all stay the same. Nothing else changes. I hope this makes sense.



- Deepak



*From:* Gwenn Etourneau [mailto:getourneau(a)pivotal.io]
*Sent:* Wednesday, November 11, 2015 5:10 PM
*To:* Discussions about Cloud Foundry projects and the system overall.
*Subject:* [cf-dev] Re: Pluggable Resource Scheduling



Hi,



Interesting proposition, wondering if it make sense to hook into Diego or
CF.

Diego is connected to CF by the CC-Bridge (big picture) why not create a
CC-Bridge for other scheduling system ?







Thanks



On Thu, Nov 12, 2015 at 5:13 AM, Deepak Vij (A) <deepak.vij(a)huawei.com>
wrote:

Hi folks, I would like to start a discussion thread and get community
thoughts regarding availability of Pluggable Resource Scheduling within
CF/Diego. Just like Kubernetes does, wouldn’t it be nice to have an option
of choosing Diego native scheduling or other uber/global resource
management environments, specifically Mesos.



Look forward to comments and feedback from the community. Thanks.



Regards,

Deepak Vij

(Huawei Software Lab., Santa Clara)




bosh-lite diego "found no compatible cell"

Christian Stocker <chregu@...>
 

Hi

I installed bosh-lite on my mac according to the docs on
https://github.com/cloudfoundry-incubator/diego-release

That worked all fine, I can deploy apps without diego enabled and they
run as expected. But when I enable diego for an app and then restart it,
I get

0 of 1 instances running, 1 starting (found no compatible cell)

Any idea where to look at?

Greetings

christian


Bosh release for Docker

Rajesh Jain <rjain15@...>
 

I have a few questions on the Bosh release for Docker

https://github.com/cloudfoundry-community/docker-boshrelease

The documentation says:
There are 3 deployment scenarios for this BOSH release:

Deploy statically defined Docker containers: see CONTAINERS.md for deployment instructions.
Deploy a Docker Service Broker for your Cloud Foundry deployment: see SERVICE_BROKER.md for deployment instructions.
Deploy a Docker Swarm cluster: see SWARM.md for deployment instructions.

Is Elastic Runtime aware of any of these containers or the services or the swarm cluster?
How does the route to the containers or services get registered or traffic get routed?
Can you take logs out of the containers (similar to Diego containers) ?
Can you bring your own container/container stem cell?
Can you connect to an internal Docker Repo?
And how actively is this Bosh released being maintained and worked on.

Thanks, Rajesh


Re: Setting up multinode PCF on macbook pro

Daniel Mikusa
 

Install Virtualbox and follow the instructions to setup bosh-lite.

https://github.com/cloudfoundry/bosh-lite#prepare-the-environment

Then install CF into Bosh lite.

http://docs.cloudfoundry.org/deploying/boshlite/index.html

After that, you can install some services into bosh-lite as well.

https://github.com/drnic/cassandra-boshrelease
https://github.com/emc-cloudfoundry/cassandra-cf-service-boshrelease
https://github.com/Altoros/mongo-bosh

Or you could just point to an existing Cassandra / Mongo installation that
runs elsewhere with a user provided service.

https://docs.cloudfoundry.org/devguide/services/user-provided.html

This is what I typically prefer as my laptop is resource challenged.

Dan


On Mon, Nov 23, 2015 at 3:18 PM, Basanth Gowda <basanth.gowda(a)gmail.com>
wrote:

Hello,
I am new to CloudFoundry. My client wants to use PCF with Cassandra/Mongo.
I wanted to try it out locally on my macbook pro. Any pointers would help

thank you
Basu


Re: diego: disk filling up over time

Eric Malm <emalm@...>
 

Hi, Tom,

Thanks for asking about this. Could you provide some more details about
your deployment?

- What are the exact errors you're seeing when CF users are trying to make
containers? The errors from CF CLI logs or rep/garden logs would be great
to see.
- What's the total amount of disk space available on the volume attached to
/var/vcap/data? You should be able to see this from `df` command output.
- How much space is the rep configured to allocate for its executor cache?
Is it the default 10GB provided by the rep's job spec in
https://github.com/cloudfoundry-incubator/diego-release/blob/v0.1398.0/jobs/rep/spec#L70-L72?
How much disk is actually used in /var/vcap/data/executor_cache (based on
reporting from `du`, say)?
- How much space have you directed garden-linux to allocate for its btrfs
store? This is provided via the diego.garden-linux.btrfs_store_size_mb BOSH
property, and with Diego 0.1398.0 I believe it has to be specified
explicitly. Also, how much space is actually used in the btrfs filesystem?
You should be able to inspect this with the btrfs tools available on the
cell VM in '/var/vcap/packages/btrfs-tools/bin'. I think running
`/var/vcap/packages/btrfs-tools/bin/btrfs filesystem usage
/var/vcap/data/garden-linux/btrfs_graph` should be a good starting point.

You may also find some useful information in the cf-dev thread from August
about overcommitting disk on Diego cells:
https://lists.cloudfoundry.org/archives/list/cf-dev(a)lists.cloudfoundry.org/thread/VBDM2TMHQSOFILSHRCV4G2CCPRBP5WKA/#VBDM2TMHQSOFILSHRCV4G2CCPRBP5WKA

Thanks,
Eric

On Wed, Nov 18, 2015 at 6:52 AM, Tom Sherrod <tom.sherrod(a)gmail.com> wrote:

diego release 0.1398.0

After a couple of weeks of dev, the cells end up filling their disks. Did
I miss a clean up job somewhere?
Currently, once pushes start failing, I get bosh to recreate the machine.

Other options?

Thanks,
Tom


Re: Deploy Cloud foundry using bosh or terraform

Marco Nicosia
 

I tried to install CF via BOSH-lite on my 8GB Macbook last week. The
results were ugly. Several compile tasks failed leaving a mangled,
half-finished install. I did not pursue further; it's possible allocating
more resources to the VM (Vagrant, in this case) would have helped. During
the process, however, the MacBook itself was pretty saturated.

If you can find more RAM, definitely recommend.

On Monday, November 23, 2015, James Bayer <jbayer(a)pivotal.io> wrote:

8GB of RAM may not be enough memory for full cloud foundry. from the
terraform openstack installer repo, you can see the recommended resource
sizes:


https://github.com/cloudfoundry-community/terraform-openstack-cf-install/blob/master/variables.tf#L177-L184

if you're going to use cloud foundry in production, i believe you should
use BOSH so that you can evolve and update the installation.

On Mon, Nov 23, 2015 at 8:47 AM, Rubab Syed <rubab.syed21(a)gmail.com
<javascript:_e(%7B%7D,'cvml','rubab.syed21(a)gmail.com');>> wrote:

Hello!
I want to deploy cloud foundry on my openstack instance. I have 8gb RAM
and 500gb hard-disk. Can someone please tell me should I use terraform or
bosh for that purpose. I am a newbie in this world.
Thanks!


--
Thank you,

James Bayer
--
--
Marco Nicosia
Product Manager
Pivotal Software, Inc.
mnicosia(a)pivotal.io
c: 650-796-2948


Setting up multinode PCF on macbook pro

Basanth Gowda
 

Hello,
I am new to CloudFoundry. My client wants to use PCF with Cassandra/Mongo. I wanted to try it out locally on my macbook pro. Any pointers would help

thank you
Basu


Re: Deploy Cloud foundry using bosh or terraform

James Bayer
 

8GB of RAM may not be enough memory for full cloud foundry. from the
terraform openstack installer repo, you can see the recommended resource
sizes:

https://github.com/cloudfoundry-community/terraform-openstack-cf-install/blob/master/variables.tf#L177-L184

if you're going to use cloud foundry in production, i believe you should
use BOSH so that you can evolve and update the installation.

On Mon, Nov 23, 2015 at 8:47 AM, Rubab Syed <rubab.syed21(a)gmail.com> wrote:

Hello!
I want to deploy cloud foundry on my openstack instance. I have 8gb RAM
and 500gb hard-disk. Can someone please tell me should I use terraform or
bosh for that purpose. I am a newbie in this world.
Thanks!
--
Thank you,

James Bayer


Re: CF CLI v6.14.0 Released Today

James Bayer
 

this is a great release. people have wanted the ability to self-manage org
and space users without an admin in the cli for a long time.

thanks for the detailed notes dies!

On Mon, Nov 23, 2015 at 9:56 AM, Dieu Cao <dcao(a)pivotal.io> wrote:

To clarify the point above, a space manager can manage space level roles
for any user of the organization. They will not be able to add users to
the space if that user is not yet a member of the space's organization.

I'll see about getting the docs updated to describe these new feature
flags.
These flags enable whether setting org/space roles can be managed by
passing a username instead of a user's guid to cloud controller, which in
effect makes it so that Org Managers and Space Managers can manage roles.
With this change, the CLI no longer needs to do the lookup to UAA (on an
admin only end point) to get user guids.

-Dieu
CF CAPI PM

On Mon, Nov 23, 2015 at 4:24 AM, Voelz, Marco <marco.voelz(a)sap.com> wrote:

Thanks for the clarifications, Dies! :)

On 23/11/15 11:56, "Koper, Dies" <diesk(a)fast.au.fujitsu.com> wrote:

Hi Marco,



With this release of the CLI, Org Managers can assign org and space
roles to users. Also, Space Managers can assign space roles to users in
their org, using the existing `cf set-org-role` and `cf set-space-role`
(and equivalent ‘unset’) commands. This feature was previously only
available to admin users.

The above text reads like Space Managers can assign roles in an Org,
i.e. outside "their" Space, is that true? My assumption was that Space
Managers can assign roles in "their" Space only, while Org Managers can
assign roles in all Spaces in that Org and also create new Spaces in that
Org.



Space Managers can assign **space** roles for in **their** space only.

But, Space Managers can do more than just changing the space roles of the
users already in their space:

They can also assign a space role to a user not yet in their space: This
user will be added to their space and then assigned the space role, with a
single set-space-role command invocation.

This is the realization of the following in the manual:

https://docs.cloudfoundry.org/concepts/roles.html#space-roles

A Space Manager can do the following:

- Add and manage users in the space



Can we please add documentation about the new feature flags in
https://docs.cloudfoundry.org/adminguide/listing-feature-flags.html to
clarify what exactly is enabled/disabled by them? And it seems like the new
flags are enabled by default



I believe they are, but the flags are CC flags, not CLI flags, so please
check with Dieu.



Related to the above, as an Org Manager creating an org using `cf
create-org`, you will now be assigned Org Manager role in it
automatically so you can start managing it straight away.

Does that mean Org Managers can create Organizations now? Or should this
rather be `cf create-space` inside the Org you are managing as Org Manager?
Is this related to the feature flag `user_org_creation`? Ot should this
rather be about Org Managers creating a Space in "their" Org?



Already, when you create a space, you are automatically assigned
SpaceManager and SpaceDeveloper roles in it automatically.

In this CLI release we extend the equivalent functionality to creating an
org.

We have not changed anything about the ability to create organizations or
not. I believe non-admins can create orgs only if feature flag
`user_org_creation` is enabled, which is disabled by default.



I’m not too familiar with the `user_org_creation` feature itself, but now
I think “as a non-user creating an org” may have been a more accurate
description to use in my release notes.



- The version reported by `cf -v` is now SemVer <http://semver.org/>
compliant and easier to understand by leaving out the build time component: cf
version 6.14.0+2654a47-2015-11-18

I don't see how including the build time component is making it non
SemVer compliant, see Point #10, which explicit names examples such as 1.0.0-alpha+001,
1.0.0+20130313144700, 1.0.0-beta+exp.sha.5114f85. I'm not saying we
should keep it if we don't need it – just that removing it would not have
been necessary to comply with SemVer.



We did two things with the version, 1) make it SemVer compliant, 2) make
it easier to understand.

Previous version : 6.12.4-b4b6af1-2015-09-18T10:55:12+00:00

This version : 6.14.0+2654a47-2015-11-18



The SemVer compliance is in the `+` and the lack of `:` and `+` after the
initial `+`.

The easier to understand part is the shortening of it, and the omission
of digits nobody should care about:

The date is useful for users to get a quick idea of how recent their cli
is by viewing the version and not having to then visit our release page to
confirm what version number we are at; They see the date and go, “hey, it’s
three months old, maybe I should check if there is a new version before I
report the bug I think I just found”. The hours, minutes, seconds, TZ are
just noise.



Thanks for asking!



Cheers,

Dies Koper
Cloud Foundry CLI PM



*From:* Voelz, Marco [mailto:marco.voelz(a)sap.com <marco.voelz(a)sap.com>]
*Sent:* Monday, November 23, 2015 8:22 PM
*To:* Discussions about Cloud Foundry projects and the system overall.
*Subject:* [cf-dev] Re: CF CLI v6.14.0 Released Today



Dear Dies,



thanks for the new release, I'm really happy about the RBAC part. Could
you maybe have a look at my question about details in the release notes
below:



On 19/11/15 02:19, "Koper, Dies" <diesk(a)fast.au.fujitsu.com> wrote:

*RBAC for Org and Space Managers*



With this release of the CLI, Org Managers can assign org and space roles
to users. Also, Space Managers can assign space roles to users in their
org, using the existing `cf set-org-role` and `cf set-space-role` (and
equivalent ‘unset’) commands. This feature was previously only available to
admin users.



The above text reads like Space Managers can assign roles in an Org, i.e.
outside "their" Space, is that true? My assumption was that Space Managers
can assign roles in "their" Space only, while Org Managers can assign roles
in all Spaces in that Org and also create new Spaces in that Org.



Note that this feature requires the target CF release to be v219 (CC API
v2.37.0) or higher, and the feature flags ‘set_roles_by_username’ and ’
set_roles_by_username’ to be enabled (use `cf feature-flags` to check).



Can we please add documentation about the new feature flags in
https://docs.cloudfoundry.org/adminguide/listing-feature-flags.html to
clarify what exactly is enabled/disabled by them? And it seems like the new
flags are enabled by default:
https://github.com/cloudfoundry/cloud_controller_ng/blob/965dbc4bdf65df89f382329aef39f86a916b3f05/app/models/runtime/feature_flag.rb#L16-L17
?



Related to the above, as an Org Manager creating an org using `cf
create-org`, you will now be assigned Org Manager role in it
automatically so you can start managing it straight away.



Does that mean Org Managers can create Organizations now? Or should this
rather be `cf create-space` inside the Org you are managing as Org Manager?
Is this related to the feature flag `user_org_creation`? Ot should this
rather be about Org Managers creating a Space in "their" Org?



*Other Features:*

- The version reported by `cf -v` is now SemVer <http://semver.org/>
compliant and easier to understand by leaving out the build time component: cf
version 6.14.0+2654a47-2015-11-18



I don't see how including the build time component is making it non
SemVer compliant, see Point #10, which explicit names examples such as 1.0.0-alpha+001,
1.0.0+20130313144700, 1.0.0-beta+exp.sha.5114f85. I'm not saying we
should keep it if we don't need it – just that removing it would not have
been necessary to comply with SemVer.



*New Plugins:*

- *Manifest Generator Plugin *
*https://github.com/ArthurHlt/plugin-cf-manifest-generator*
<https://github.com/ArthurHlt/plugin-cf-manifest-generator>



Nice!



Thanks and warm regards

Marco



--
Thank you,

James Bayer


Re: Garden Port Assignment Story

Mike Youngstrom
 

Hi Will,

Though I see the main reason for the issue assuming a healthy running
environment I've also experienced a deploy related issue that more unique
port assignment could help defend against. During one of our deploys the
routers finished deployed before the DEAs. When the DEAs started rolling,
for some reason some of our routers stopped getting route updates from
NATs. This caused their route tables to go stale and as apps started
rolling new apps started getting assigned ports previously held by other
apps. Which caused a number of our hosts to be mis-routed.

Though the root cause was probably some bug in the Nats client in GoRouter
the runtime team had apparently experienced a similar issue in the past [0]
which caused them to implement code that would delete stale routes even
then a router couldn't connect to NATs. The Router team is now planning to
optionally remove this failsafe [1]. I'm hoping that with the removal of
this failsafe (which I'm planning to take advantage of) this tracker story
will help protect us from the problem we experienced before from happening
again.

If the ports simply reset on a stemcell upgrade this issue provides no
defense for the problem we had before.

Does that make sense Will?

Mike

[0]
https://groups.google.com/a/cloudfoundry.org/d/msg/vcap-dev/yuVYCZkMLG8/7t8FHnFzWEsJ
[1] https://www.pivotaltracker.com/story/show/108659764

On Mon, Nov 23, 2015 at 11:11 AM, Will Pragnell <wpragnell(a)pivotal.io>
wrote:

Hi Mike,

What's the motivation for wanting rolling port assignment to persist
across e.g. stemcell upgrade? The motivation for this story is to prevent
stale routes from sending traffic to the wrong containers. Our assumption
is that stale routes won't ever exist for anything close to the amount of
time it takes BOSH to destroy and recreate a VM. Have we missed something
in making that assumption?

On your second point, I see your concern. We've talked about the
possibility of implementing FIFO semantics on free ports (when a port that
was in use becomes free, it goes to the end of the queue of available
ports) to decrease the chances of traffic reaching the wrong container as
far as possible. It's possible that the rolling ports approach is "good
enough" though. We're still trying to understand whether that's actually
the case.

The consistent hashing idea is interesting, but a few folks have suggested
that with a relatively small range of available ports (5000 by default)
that the chances of collision are actually higher than we'd want. I'll see
if someone wants to lay down some maths to give that idea some credence.

Cheers,
Will

On 23 November 2015 at 08:47, Mike Youngstrom <youngm(a)gmail.com> wrote:

Since I cannot comment in tracker I'm starting this thread to discuss
story: https://www.pivotaltracker.com/n/projects/1158420/stories/92085170

Some comments I have:

* Although I can see how a rolling port assignment could be maintained
across garden/diego restarts I'd also like the story to ensure that the
rolling port assignments get maintained across a Stemcell upgrade without
the need for persistent disks on each cell. Perhaps etcd?

* Another thing to keep in mind. Although a rolling port value may not
duplicate ports 100% of the time in a short lived container in a long lived
container it seems to me that a rolling port assignment becomes no more
successful than a random port assignment if the container lives long enough
for the port assignment loop to loop a few times.

* Has there been any consideration to using an incremental consistent
hash of the app_guid to assign ports? A consistent hash would have the
benefit of being stateless. It also would have the benefit of increasing
the likely hood that if a request is sent to a stale route it may be to the
correct app anyway.

Thoughts?

Mike

6561 - 6580 of 9422