Date   

Re: can't login with cf CLI but the UAAC tool works

kyle havlovitz <kylehav@...>
 

Following up on this, I fixed the issue by using a verification key between
the CC and the UAA instead of a token secret

On Fri, Sep 4, 2015 at 6:51 PM, Filip Hanik <fhanik(a)pivotal.io> wrote:

those urls do not look right. but they are dependent on what url you
deployed the uaa under. ( ie / or /uaa )

what's returned by uaac info is dependent on your uaa config.

I'd have to get back to you in CC config. not my area of expertise. but
yes I believe the CC will have an oauth client registered with the UAA

Filip

On Friday, September 4, 2015, kyle havlovitz <kylehav(a)gmail.com> wrote:

I realize it's a pain, but I'm setting these up without bosh. I'm just
unfamiliar with how the config between the CC and the UAA needs to be set.

The http://localhost:8080/login and http://localhost:8080/uaa seem to be
the correct URLs (they're whats returned by 'uaac info')
Likewise, the CLI seems to be pointed at the right places, it's just
getting this invalid token error, as if the CC can't correctly talk to the
uaa or something.

what should the uaa.resource_id and uaa.symmetric_secret fields in the CC
config be set to if I'm using the default config/clients? Are there any
other values in the CC config that might be the issue here?

On Fri, Sep 4, 2015 at 6:26 PM, Filip Hanik <fhanik(a)pivotal.io> wrote:

ok, is that the correct URL?

you're attempting to configure a very large eco system by hand. That can
be a bit difficult. If you want a local cloud foundry, I would suggest
bosh-lite

basically, clone cloudfoundry/cf-release and cloudfoundry/bosh-lite

cd bosh-lite
vagrant up (this launches a VM with bosh director on it)
bin/add-route (sets up network routing)
bin/provision-cf (builds and publishing cloud foundry to the VM

cf api api.10.244.0.34.xip.io
cf login







On Fri, Sep 4, 2015 at 4:18 PM, kyle havlovitz <kylehav(a)gmail.com>
wrote:

The cloud controller logs have "Invalid bearer token:
#<CF::UAA::InvalidSignature: Signature verification failed>" and the 401
invalid auth message.

On Fri, Sep 4, 2015 at 6:14 PM, kyle havlovitz <kylehav(a)gmail.com>
wrote:

Ok, I set those 2 properties to http://localhost:8080 and it looks
identical; same error, same endpoints requested.
Could something be wrong with the cloud controller config?

On Fri, Sep 4, 2015 at 5:58 PM, Filip Hanik <fhanik(a)pivotal.io> wrote:

yes, this is the URL misconfiguration I was talking about.

In your uaa.yml you should have two properties

login.url: http://localhost:8080
uaa.url: http://localhost:8080

set those two, and let's see that trace again



On Fri, Sep 4, 2015 at 2:58 PM, kyle havlovitz <kylehav(a)gmail.com>
wrote:

The CLI seems to be able to get a token now though, it's failing for
a different reason:

cf login
API endpoint: http://localhost:8181
REQUEST: [2015-09-04T20:46:51Z]
GET /v2/info HTTP/1.1
Host: localhost:8181
Accept: application/json
Content-Type: application/json
User-Agent: go-cli 6.12.3-c0c9a03 / linux


RESPONSE: [2015-09-04T20:46:51Z]
HTTP/1.1 200 OK
Content-Length: 406
Connection: keep-alive
Content-Type: application/json;charset=utf-8
Server: thin
X-Content-Type-Options: nosniff
X-Vcap-Request-Id: d44503ef-3b2c-4340-9958-ad91daf3706c
{"name":"vcap","build":"2222","support":"
http://support.local.example.com","version":2,"description":"CF v2
test environment","authorization_endpoint":"http://localhost:8080
","token_endpoint":"http://localhost:8080/uaa
","min_cli_version":null,"min_recommended_cli_version":null,"api_version":"2.34.0","app_ssh_endpoint":null,"app_ssh_host_key_fingerprint":null,"logging_endpoint":"ws://
127.0.0.1:9090"}
Warning: Insecure http API endpoint detected: secure https API
endpoints are recommended

REQUEST: [2015-09-04T20:46:51Z]
GET /login HTTP/1.1
Host: localhost:8080
Accept: application/json
Content-Type: application/json
User-Agent: go-cli 6.12.3-c0c9a03 / linux


RESPONSE: [2015-09-04T20:46:51Z]
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Access-Control-Allow-Origin: *
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Cache-Control: no-cache, no-store, max-age=0
Content-Language: en-US
Content-Type: application/json;charset=UTF-8
Date: Fri, 04 Sep 2015 20:46:51 GMT
Expires: 0
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Pragma: no-cache
Pragma: no-cache
Server: Apache-Coyote/1.1
Strict-Transport-Security: max-age=31536000
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Xss-Protection: 1; mode=block
1fd

{"timestamp":"2015-08-05T00:00:41+0000","app":{"version":"2.5.1"},"idpDefinitions":[],"fieldUsernameShow":true,"zone_name":"uaa","commit_id":"eae6724","prompts":{"username":["text","Email"],"password":["password","Password"]},"forgotPasswordLink":"/forgot_password","createAccountLink":"/create_account","links":{"register":"/create_account","passwd":"/forgot_password","login":"
http://localhost:8080/login","uaa":"http://localhost:8080/uaa
"},"entityID":"cloudfoundry-saml-login","linkCreateAccountShow":true}
0


Email> admin
Password>
Authenticating...
REQUEST: [2015-09-04T20:46:58Z]
POST /oauth/token HTTP/1.1
Host: localhost:8080
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Content-Type: application/x-www-form-urlencoded
User-Agent: go-cli 6.12.3-c0c9a03 / linux
grant_type=password&password=[PRIVATE DATA
HIDDEN]&scope=&username=admin
RESPONSE: [2015-09-04T20:46:58Z]
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Access-Control-Allow-Origin: *
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Cache-Control: no-store
Content-Type: application/json;charset=UTF-8
Date: Fri, 04 Sep 2015 20:46:58 GMT
Expires: 0
Pragma: no-cache
Pragma: no-cache
Server: Apache-Coyote/1.1
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Xss-Protection: 1; mode=block
738
{"access_token":"[PRIVATE DATA
HIDDEN]","token_type":"bearer","refresh_token":"[PRIVATE DATA
HIDDEN]","expires_in":43199,"scope":"scim.read scim.userids
cloud_controller.admin scim.write cloud_controller.write password.write
openid cloud_controller.read","jti":"cbda4e10-cf04-4696-a560-2e1f4d2c610c"}
0

OK

REQUEST: [2015-09-04T20:46:58Z]
GET /v2/organizations HTTP/1.1
Host: localhost:8181
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Content-Type: application/json
User-Agent: go-cli 6.12.3-c0c9a03 / linux


RESPONSE: [2015-09-04T20:46:58Z]
HTTP/1.1 401 Unauthorized
Content-Length: 97
Connection: keep-alive
Content-Type: application/json;charset=utf-8
Server: thin
X-Content-Type-Options: nosniff
X-Vcap-Request-Id: b7658709-8145-4e31-a7ed-12a7831e99da
{
"code": 1000,
"description": "Invalid Auth Token",
"error_code": "CF-InvalidAuthToken"
}

REQUEST: [2015-09-04T20:46:58Z]
POST /oauth/token HTTP/1.1
Host: localhost:8080
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Content-Type: application/x-www-form-urlencoded
User-Agent: go-cli 6.12.3-c0c9a03 / linux

grant_type=refresh_token&refresh_token=eyJhbGciOiJIUzI1NiJ9.eyJqdGkiOiJlNzFmOTNmZS0yMmEyLTQ3ZjgtODgwNC0xN2ZmNmU1YzM1NmMiLCJzdWIiOiIzMDBkZTc1YS1mN2RhLTRjMWQtYjA0Yi02YWZhZjI1ZmE5MjgiLCJzY29wZSI6WyJzY2ltLnJlYWQiLCJzY2ltLnVzZXJpZHMiLCJjbG91ZF9jb250cm9sbGVyLmFkbWluIiwic2NpbS53cml0ZSIsImNsb3VkX2NvbnRyb2xsZXIud3JpdGUiLCJwYXNzd29yZC53cml0ZSIsIm9wZW5pZCIsImNsb3VkX2NvbnRyb2xsZXIucmVhZCJdLCJpYXQiOjE0NDEzOTk2MTgsImV4cCI6MTQ0Mzk5MTYxOCwiY2lkIjoiY2YiLCJjbGllbnRfaWQiOiJjZiIsImlzcyI6Imh0dHA6Ly9sb2NhbGhvc3Q6ODA4MC91YWEvb2F1dGgvdG9rZW4iLCJ6aWQiOiJ1YWEiLCJncmFudF90eXBlIjoicGFzc3dvcmQiLCJ1c2VyX25hbWUiOiJhZG1pbiIsInVzZXJfaWQiOiIzMDBkZTc1YS1mN2RhLTRjMWQtYjA0Yi02YWZhZjI1ZmE5MjgiLCJyZXZfc2lnIjoiOTAyODliNjgiLCJhdWQiOlsiY2YiLCJzY2ltIiwiY2xvdWRfY29udHJvbGxlciIsInBhc3N3b3JkIiwib3BlbmlkIl19.-eGB2RWZfYVZkTSvT7c4lUzsY5QZMWgXFHMGGx4HEN8&scope=
RESPONSE: [2015-09-04T20:46:58Z]
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Access-Control-Allow-Origin: *
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Cache-Control: no-store
Content-Type: application/json;charset=UTF-8
Date: Fri, 04 Sep 2015 20:46:58 GMT
Expires: 0
Pragma: no-cache
Pragma: no-cache
Server: Apache-Coyote/1.1
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Xss-Protection: 1; mode=block
738
{"access_token":"[PRIVATE DATA
HIDDEN]","token_type":"bearer","refresh_token":"[PRIVATE DATA
HIDDEN]","expires_in":43199,"scope":"scim.userids scim.read
cloud_controller.admin password.write scim.write openid
cloud_controller.write
cloud_controller.read","jti":"e62d3265-9ab7-441e-b2b2-69ca92d81d7c"}
0


REQUEST: [2015-09-04T20:46:58Z]
GET /v2/organizations HTTP/1.1
Host: localhost:8181
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Content-Type: application/json
User-Agent: go-cli 6.12.3-c0c9a03 / linux


RESPONSE: [2015-09-04T20:46:58Z]
HTTP/1.1 401 Unauthorized
Content-Length: 97
Connection: keep-alive
Content-Type: application/json;charset=utf-8
Server: thin
X-Content-Type-Options: nosniff
X-Vcap-Request-Id: 7b07e39c-60f0-4db4-9274-6a59e23b8f29
{
"code": 1000,
"description": "Invalid Auth Token",
"error_code": "CF-InvalidAuthToken"
}
FAILED
Error finding available orgs
Invalid auth token: Invalid Auth Token
FAILED
Error finding available orgs
Invalid auth token: Invalid Auth Token

API endpoint: http://localhost:8181 (API version: 2.34.0)
User: admin
No org or space targeted, use 'cf target -o ORG -s SPACE'


On Fri, Sep 4, 2015 at 4:49 PM, kyle havlovitz <kylehav(a)gmail.com>
wrote:

Ok, thanks for the helpful links.
I replaced my config with the uaa.yml and login.yml from there and
now the uaac commands from above work and I can run 'uaac token
owner get'. I still can't login to cf with the cli though.

On Fri, Sep 4, 2015 at 4:15 PM, Filip Hanik <fhanik(a)pivotal.io>
wrote:

Minimalist defaults are in the UAA repo (uaa.yml and login.yml)

https://github.com/cloudfoundry/uaa/tree/master/uaa/src/main/resources

Yaml is very sensitive to indentation. So hand crafting it may
become a bit difficult.

If you want the UAA to provide all default values (including
admin/adminsecret client and cf/<blank password> client, then don't add any
uaa.yml config file at all. Just start up UAA with it's defaults.
It will suck in client defaults from

https://github.com/cloudfoundry/uaa/blob/feature/invitations_flow_by_email_domain/uaa/src/main/webapp/WEB-INF/spring/oauth-clients.xml#L47-L172

Filip


On Fri, Sep 4, 2015 at 2:05 PM, kyle havlovitz <kylehav(a)gmail.com>
wrote:

is there an example somewhere of a minimalist working config for
them? I'm going through at the moment and trying to make mine resemble the
config here:
https://github.com/cloudfoundry/cf-release/blob/master/jobs/uaa/templates/uaa.yml.erb

I'm also defining a test admin user in the scim users section

On Fri, Sep 4, 2015 at 4:00 PM, Filip Hanik <fhanik(a)pivotal.io>
wrote:

ok, that tells me that your configuration of the UAA clients is
incorrect



On Fri, Sep 4, 2015 at 1:44 PM, kyle havlovitz <
kylehav(a)gmail.com> wrote:

ok so the 'uaac token client get' fails, and the error is 'Bad
credentials'

On Fri, Sep 4, 2015 at 3:33 PM, Filip Hanik <fhanik(a)pivotal.io>
wrote:

ok, so we can validate that

uaac target http://localhost:8080
uaac token client get admin -s <your admin client secret>
uaac clients

Should show your 'cf' client in the list

then we can do

uaac token owner get cf <username> -s "" -p <user password>

and if that works, we can take it to the next step



On Fri, Sep 4, 2015 at 1:26 PM, kyle havlovitz <
kylehav(a)gmail.com> wrote:

I started the uaa by building from the tagged version in
cf-release v215 and running it via tomcat with a custom config file, but I
didn't specify a database. I have both a cf and admin section in the uaa
clients config:

cf:

id: cf
override: true
authorized-grant-types: password,implicit,refresh_token
authorities: uaa.none
scope:
cloud_controller.read,cloud_controller.write,openid,password.write,cloud_controller.admin,scim.read,scim.write
secret: 'xxxxxxxxxx'

admin:

id: admin
authorized-grant-types: client_credentials
authorities:
clients.read,clients.write,clients.secret,password.write,scim.read,uaa.admin
scope: read,write,password
resource-ids: clients
secret: 'xxxxxxxxxx'


On Fri, Sep 4, 2015 at 3:09 PM, Filip Hanik <
fhanik(a)pivotal.io> wrote:

ok, so the URL you have is /oauth/token, that's fine. your
trace returns

"authorization_endpoint":"http://localhost:8080
","token_endpoint":"http://localhost:8080/uaa"

indicating that there is a misconfiguration somewhere, but
we can fix that later.

How did you start the UAA? Are you sure that your UAA has a
client named 'cf' in its database?



On Fri, Sep 4, 2015 at 1:05 PM, kyle havlovitz <
kylehav(a)gmail.com> wrote:

Running that command against /uaa/oauth/token gives just a
redirect to /login. Doing it with /oauth/token gives a 401 unauthorized,
same as the cf cli.

What do you mean by deploy it as root "/"? As in, a
override the url it hosts the endpoints at?


Re: Security Question --- Securely wipe data on warden container removal / destruction???

Will Pragnell <wpragnell@...>
 

In Diego/Garden, container files are stored on btrfs subvolumes. When a
container is destroyed, the subvolume is removed with btrfs subvolume delete.
I don’t think this does anything particularly fancy, and I don’t think it
classifies as “secure deletion”.

On 17 September 2015 at 11:53, Chris K <christopherkugler2(a)yahoo.de> wrote:

Hello again,

I'm sorry for having to revive this topic, but I'm still unaware about the
deletion process.

[... ] i believe with standard removal tools.
Could you please specify the term "standard removal tool". What's the
standard? Is standard just the deletion of the pointer pointing on files /
segments, or is secure deletion state of the art?
I'd be thankful for any reference on documentation regarding this topic.

Thanks in advance.

Cheers Chris


Re: Security Question --- Securely wipe data on warden container removal / destruction???

Chris K
 

Hello again,

I'm sorry for having to revive this topic, but I'm still unaware about the deletion process.

[... ] i believe with standard removal tools.
Could you please specify the term "standard removal tool". What's the standard? Is standard just the deletion of the pointer pointing on files / segments, or is secure deletion state of the art?
I'd be thankful for any reference on documentation regarding this topic.

Thanks in advance.

Cheers Chris


Re: Relationship between HM9000 and router jobs

Sylvain FAUVE
 

Thank you for your reply.
True i think it was a coincidence ...
I've try on a playground config to shutdown hm9000 and etd, and I did not lose routes at all, apps were still running.


Re: User cannot do CF login when UAA is being updated

Yunata, Ricky <rickyy@...>
 

Thank you Joseph, I will pass it to Dies

Ricky Yunata
Software & Solution Specialist

Fujitsu Australia Software Technology Pty Ltd
14 Rodborough Road, Frenchs Forest NSW 2086, Australia
T +61 2 9452 9128 M +61 433 977 739 F +61 2 9975 2899
rickyy(a)fast.au.fujitsu.com<mailto:rickyy(a)fast.au.fujitsu.com>
fastware.com.au<http://fastware.com.au>

[cid:image001.jpg(a)01D0F138.DC30E980]
[cid:image002.jpg(a)01D0F138.DC30E980]
From: CF Runtime [mailto:cfruntime(a)gmail.com]
Sent: Wednesday, 16 September 2015 7:08 PM
To: Discussions about Cloud Foundry projects and the system overall.
Subject: [cf-dev] Re: Re: Re: Re: Re: Re: User cannot do CF login when UAA is being updated

If you can't get the list to accept the attachment, you can give it to Dies and he should be able to get it to us.

Joseph
OSS Release Integration Team

On Tue, Sep 15, 2015 at 7:19 PM, Yunata, Ricky <rickyy(a)fast.au.fujitsu.com<mailto:rickyy(a)fast.au.fujitsu.com>> wrote:
Hi Joseph,

Yes that is the case. I have sent my test result but it seems that my e-mail does not get through. How can I sent attachment in this mailing list?

Regards,
Ricky


From: CF Runtime [mailto:cfruntime(a)gmail.com<mailto:cfruntime(a)gmail.com>]
Sent: Tuesday, 15 September 2015 8:10 PM
To: Discussions about Cloud Foundry projects and the system overall.
Subject: [cf-dev] Re: Re: Re: Re: User cannot do CF login when UAA is being updated

Couple of updates here for clarity. No databases are stored on NFS in any default installation. NFS is only used to store blobstore data. If you are using the postgres job from cf-release, since it is single node there will be downtime during a stemcell deploy.

I talked with Dies from Fujitsu earlier and confirmed they are NOT using the postgres job but an external non-cf deployed postgres instance. So during a deploy, the UAA db should be up and available the entire time.

The issue they are seeing is that even though the database is up, and I'm guessing there is at least a single node of UAA up during the deploy, there are still login failures.

Joseph
OSS Release Integration Team

On Mon, Sep 14, 2015 at 6:39 PM, Filip Hanik <fhanik(a)pivotal.io<mailto:fhanik(a)pivotal.io>> wrote:
Amit, see previous comment.

Postgresql database is stored on NFS that is restarted during nfs job update.
UAA, while being up, is non functional while the NFS job is updated because it can't get to the DB.



On Mon, Sep 14, 2015 at 5:09 PM, Amit Gupta <agupta(a)pivotal.io<mailto:agupta(a)pivotal.io>> wrote:
Hi Ricky,

My understanding is that you still need help, and the issues Jiang and Alexander raised are different. To avoid confusion, let's keep this thread focused on your issue.

Can you confirm that you have two UAA VMs in separate bosh jobs, separate AZs, etc. Can you confirm that when you roll the UAAs, only one goes down at a time? The simplest way to affect a roll is to change some trivial property in the manifest for your UAA jobs. If you're using v215, any of the properties referenced here will do:

https://github.com/cloudfoundry/cf-release/blob/v215/jobs/uaa/spec#L321-L335

You should confirm that only one UAA is down at a time, and comes back up before bosh moves on to updating the other UAA.

While this roll is happening, can you just do `CF_TRACE=true cf auth USERNAME PASSWORD` in a loop, and if you see one that fails, post the output, along with noting the state of the bosh deploy when the error happens.

Thanks,
Amit

On Mon, Sep 14, 2015 at 10:51 AM, Amit Gupta <agupta(a)pivotal.io<mailto:agupta(a)pivotal.io>> wrote:
Ricky, Jiang, Alexander, are the three of you working together? It's hard to tell since you've got Fujitsu, Gmail, and Altoros email addresses. Are you folks talking about the same issue with the same deployment, or three separate issues.

Ricky, if you still need assistance with your issue, please let us know.

On Mon, Sep 14, 2015 at 10:16 AM, Lomov Alexander <alexander.lomov(a)altoros.com<mailto:alexander.lomov(a)altoros.com>> wrote:
Yes, the problem is that postgresql database is stored on NFS that is restarted during nfs job update. I’m sure that you’ll be able to run updates without outage with several customizations.

It is hard to tell without knowing your environment, but in common case steps will be following:


1. Add additional instances to nfs job and customize it to make replications (for instance use this docs for release customization [1])
2. Make your NFS job to update sequently without our jobs updates in parallel (like it is done for postgresql [2])
3. Check your options in update section [3].

[1] https://help.ubuntu.com/community/HighlyAvailableNFS
[2] https://github.com/cloudfoundry/cf-release/blob/master/example_manifests/minimal-aws.yml#L115-L116
[3] https://github.com/cloudfoundry/cf-release/blob/master/example_manifests/minimal-aws.yml#L57-L62

On Sep 14, 2015, at 9:47 AM, Yitao Jiang <jiangyt.cn(a)gmail.com<mailto:jiangyt.cn(a)gmail.com>> wrote:

On upgrading the deployment, the uaa not working due the uaadb filesystem hangup.Under my environment , the nfs-wal-server's ip changed which causing uaadb,ccdb hang up. Hard reboot the uaadb, restart uaa service solve the issue.

Hopes can help you.

On Mon, Sep 14, 2015 at 2:13 PM, Yunata, Ricky <rickyy(a)fast.au.fujitsu.com<mailto:rickyy(a)fast.au.fujitsu.com>> wrote:
Hello,

I have a question regarding UAA in Cloud Foundry. I’m currently running Cloud Foundry on Openstack.
I have 2 availability zones and redundancy of the important VMs including UAA.
Whenever I do an upgrade of either stemcell or CF release, user will not be able to do CF login when when CF is updating UAA VM.
My question is, is this a normal behaviour? If I have redundant UAA VM, shouldn’t user still be able to still login to the apps even though it’s being updated?
I’ve done this test a few times, with different CF version and stemcells and all of them are giving me the same result. The latest test that I’ve done was to upgrade CF version from 212 to 215.
Has anyone experienced the same issue?

Regards,
Ricky
Disclaimer

The information in this e-mail is confidential and may contain content that is subject to copyright and/or is commercial-in-confidence and is intended only for the use of the above named addressee. If you are not the intended recipient, you are hereby notified that dissemination, copying or use of the information is strictly prohibited. If you have received this e-mail in error, please telephone Fujitsu Australia Software Technology Pty Ltd on + 61 2 9452 9000<tel:%2B%2061%202%209452%209000> or by reply e-mail to the sender and delete the document and all copies thereof.


Whereas Fujitsu Australia Software Technology Pty Ltd would not knowingly transmit a virus within an email communication, it is the receiver’s responsibility to scan all communication and any files attached for computer viruses and other defects. Fujitsu Australia Software Technology Pty Ltd does not accept liability for any loss or damage (whether direct, indirect, consequential or economic) however caused, and whether by negligence or otherwise, which may result directly or indirectly from this communication or any files attached.


If you do not wish to receive commercial and/or marketing email messages from Fujitsu Australia Software Technology Pty Ltd, please email unsubscribe(a)fast.au.fujitsu.com<mailto:unsubscribe(a)fast.au.fujitsu.com>




--

Regards,

Yitao
jiangyt.github.io<http://jiangyt.github.io/>





Disclaimer

The information in this e-mail is confidential and may contain content that is subject to copyright and/or is commercial-in-confidence and is intended only for the use of the above named addressee. If you are not the intended recipient, you are hereby notified that dissemination, copying or use of the information is strictly prohibited. If you have received this e-mail in error, please telephone Fujitsu Australia Software Technology Pty Ltd on + 61 2 9452 9000<tel:%2B%2061%202%209452%209000> or by reply e-mail to the sender and delete the document and all copies thereof.


Whereas Fujitsu Australia Software Technology Pty Ltd would not knowingly transmit a virus within an email communication, it is the receiver’s responsibility to scan all communication and any files attached for computer viruses and other defects. Fujitsu Australia Software Technology Pty Ltd does not accept liability for any loss or damage (whether direct, indirect, consequential or economic) however caused, and whether by negligence or otherwise, which may result directly or indirectly from this communication or any files attached.


If you do not wish to receive commercial and/or marketing email messages from Fujitsu Australia Software Technology Pty Ltd, please email unsubscribe(a)fast.au.fujitsu.com<mailto:unsubscribe(a)fast.au.fujitsu.com>

Disclaimer

The information in this e-mail is confidential and may contain content that is subject to copyright and/or is commercial-in-confidence and is intended only for the use of the above named addressee. If you are not the intended recipient, you are hereby notified that dissemination, copying or use of the information is strictly prohibited. If you have received this e-mail in error, please telephone Fujitsu Australia Software Technology Pty Ltd on + 61 2 9452 9000 or by reply e-mail to the sender and delete the document and all copies thereof.


Whereas Fujitsu Australia Software Technology Pty Ltd would not knowingly transmit a virus within an email communication, it is the receiver’s responsibility to scan all communication and any files attached for computer viruses and other defects. Fujitsu Australia Software Technology Pty Ltd does not accept liability for any loss or damage (whether direct, indirect, consequential or economic) however caused, and whether by negligence or otherwise, which may result directly or indirectly from this communication or any files attached.


If you do not wish to receive commercial and/or marketing email messages from Fujitsu Australia Software Technology Pty Ltd, please email unsubscribe(a)fast.au.fujitsu.com


Re: Relation between Network property in Resource pool block and Network property in Actual Job block

CF Runtime
 

A lot of this is set to change in the near future to make things more
clear. But here is what is currently going on.

BOSH will pre-create machines it is going to need, and it does that without
any input from the jobs that will eventually run on them. A machine needs a
network to be created, so that is the reason it is needed in the resource
pool section.

Later when BOSH configures the machine for running a job on it, it will
update the network to match the details provided in the job.

You'll probably get a faster response to questions like this in the cf-bosh
mailing list in the future.

Joseph
OSS Release Integration Team

On Tue, Sep 8, 2015 at 6:09 PM, ronak banka <ronakbanka.cse(a)gmail.com>
wrote:

Hello All,

I have a resource pool , let say small_z1

- name: small_z1
network: cf1
stemcell: stemcell-xyz
cloud_properties:
instance_type: m1.small
availability_zone: zone1

and a Job , router having two networks assigned to it

- name: router
instances: 1
networks:
- name: router_internal
default: [dns, gateway]
static_ips:
- xy.xy.xy.xy
- name: router_external
static_ips:
- yz.yz.yz.yz
gateway: yy.yy.yy.yy
networks:
apps: router_internal
management: router_internal
resource_pool: small_z1

With these properties there are no issues anywhere.

what is the network property in resource pool responsible for, if the
created job networks and not linked to the one in pool??

Regards,
Ronak



--
View this message in context:
http://cf-dev.70369.x6.nabble.com/Relation-between-Network-property-in-Resource-pool-block-and-Network-property-in-Actual-Job-block-tp1562.html
Sent from the CF Dev mailing list archive at Nabble.com.


Re: tcp-routing in Lattice

Atul Kshirsagar
 

That's true. This is one of the limitations of pure tcp (layer 4) routing. We will be hit by scalability limits in terms of public IPs. However, per public IP we can theoretically provide 64K ports and so can potentially accommodate many apps with tcp routing requirement (provided these apps and their clients can work with non-standard ports).

One alternative to this is to use SNI based solution where host header is provided in initial TLS handshake by client, which will help router to decide how to route the connection request. However, this will be limited to be used only over TLS, which may not be a very big limitation :)


Re: Proposal: Decomposing cf-release and Extracting Deployment Strategies

Mike Youngstrom
 

Another situation we have that you may want to keep in mind while
developing cf-deployment:

* We are using vsphere and currently we have a cf installation with 2 AZ
using 2 separate vsphere "Datacenters" (more details:
https://github.com/cloudfoundry/bosh-notes/issues/7). This means we have a
CF installation that is actually made up of 2 deployments. So, we need to
generate a manifest for az1 and another for az2. The job names in each
deployment must be unique across the installation (e.g.
cloud_controller_az1 and cloud_controller_az2) would be the cc job names in
each deployment.

Mike

On Wed, Sep 16, 2015 at 3:38 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

Here are some of the examples:

* Sensitive property management as part of manifest generation (encrypted
or acquired from an outside source)

* We have our own internal bosh releases and config that we'll need to
merge in with the things cf-deployment is doing. For example, if
cf-deployment tags v250 as including Diego 3333 and etcd 34 with given
templates perhaps we'd like to augment this with our own release jobs and
config that we know to work with cf-deployment 250's and perhaps tag it as
v250.lds and that becomes what we use to generate our manifests and upload
releases.

* Occasionally we may wish to use some config from a stock release not
currently exposed in a cf-deployment template. I'd like to be sure there
is a way we can add that config, in a not hacky way, without waiting for a
PR to be accepted and subsequent release.

* If for some reason we are forced to fork a stock release we'd like to be
able to use that forked release we are building instead of the publicly
available one for manifest generation and release uploads, etc.

Does that help?

Mike



On Tue, Sep 15, 2015 at 9:50 PM, Amit Gupta <agupta(a)pivotal.io> wrote:

Thanks for the feedback Mike!

Can you tell us more specifically what sort of extensions you need? It
would be great if cf-deployment provided an interface that could serve the
needs of essentially all operators of CF.

Thanks,
Amit

On Tue, Sep 15, 2015 at 4:02 PM, Mike Youngstrom <youngm(a)gmail.com>
wrote:

This is great stuff! My organization currently maintains our own custom
ways to generate manifests, include secure properties, and manage release
versions.

We would love to base the next generation of our solution on
cf-deployment. Have you put any thought into how others might customize or
extend cf-deployment? Our needs are very similar to yours just sometimes a
little different.

Perhaps a private fork periodically merged with a known good release
combination (tag) might be appropriate? Or some way to include the same
tools into a wholly private repo?

Mike


On Tue, Sep 8, 2015 at 1:22 PM, Amit Gupta <agupta(a)pivotal.io> wrote:

Hi all,

The CF OSS Release Integration team (casually referred to as the "MEGA
team") is trying to solve a lot of tightly interrelated problems, and make
many of said problems less interrelated. It is difficult to address just
one issue without touching the others, so the following proposal addresses
several issues, but the most important ones are:

* decompose cf-release into many independently manageable,
independently testable, independently usable releases
* separate manifest generation strategies from the release source,
paving the way for Diego to be part of the standard deployment

This proposal will outline a picture of how manifest generation will
work in a unified manner in development, test, and integration
environments. It will also outline a picture of what each release’s test
pipelines will look like, how they will feed into a common integration
environment, and how feedback from the integration environment will feed
back into the test environments. Finally, it will propose a picture for
what the integration environment will look like, and how we get from the
current integration environment to where we want to be.

For further details, please feel free to view and comment here:


https://docs.google.com/document/d/1Viga_TzUB2nLxN_ILqksmUiILM1hGhq7MBXxgLaUOkY

Thanks,
Amit, CF OSS Release Integration team


Re: [vcap-dev] Re: Proposal for Context Path Routing

Shannon Coen
 

Stevo,
Just a confirmation that I also saw your suggestion for defining the http
method for a route.
Thank you,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Wed, Sep 16, 2015 at 2:55 PM, Shannon Coen <scoen(a)pivotal.io> wrote:

Hello Stevo,

Thank you for your suggestion of support for wildcards in route paths. I
have noted the request and will keep an ear out for the same request from
others in order to help me prioritize.

Best,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Mon, Sep 14, 2015 at 8:34 AM, Stevo Slavić <sslavic(a)gmail.com> wrote:

Thanks for heads up and fast reply!

Will read spec routing services/API specs in more detail. For the moment,
for use cases I mentioned, one can use/deploy something like Netflix Zuul
(Spring Boot) CF app, as a routing service. Covers a lot of use cases, but
I expected support in CF itself.

Thanks once more!

Kind regards,
Stevo Slavic.

On Mon, Sep 14, 2015 at 4:47 PM, Dieu Cao <dcao(a)pivotal.io> wrote:

Shannon, the Product Manager for the Routing team, has this story [1] in
the CF Routing tracker project for doing a PR to the CLI to add support for
context path routes.

Yes, that's correct that the context path route support does not
currently support the particular example you've specified with wildcards or
routing based on HTTP methods.
If I recall correctly, we wanted to add some basic support for context
path routes without adding too much overhead in what gorouter needed to
reason about, as any logic added could impact every request to the gorouter.

There's a currently in progress tracks of work to support Route Services
[2] [3].
That would allow you to insert you're own proxy to support all sorts of
use cases and could be used to support what you've described.

Shannon is currently on vacation, but should be returning later this
week. I'll forward this request to him for consideration. You might also
consider opening up an issue against the gorouter with these enhancement
requests.

[1] https://www.pivotaltracker.com/story/show/93368928
[2] https://www.pivotaltracker.com/epic/show/2031344
[3] https://www.pivotaltracker.com/epic/show/1884060

On Mon, Sep 14, 2015 at 6:44 AM, Stevo Slavić <sslavic(a)gmail.com> wrote:

In this proposal and current Routes API docs
<http://apidocs.cloudfoundry.org/217/routes/creating_a_route.html> I
don't see support for wildcards when defining context path based route. Am
I missing something?
E.g. for resources like

/foos/bar/1/doSomeAction
/foos/baz/2/doSomeAction

and

/foos/bar/1/doSomeOtherAction
/foos/baz/2/doSomeOtherAction

I'd like all /foos/*/*/doSomeAction to be routed to one CF app, and all
/foos/*/*/doSomeOtherAction to be routed to a different CF app.

Did you consider supporting HTTP methods to be used for routing as
well? E.g. similar to previous example, I'd like some HTTP methods on
particular resources to be handled by separate app then the rest of the
requests on same resources.

DELETE /foos -> route to one cf app
* /foos -> route to another cf app, where * means any HTTP method, most
specific rule wins strategy

Ideally one would be able to combine both wildcards in HTTP methods and
in path segments to define a routing rule.


Also I do not see context path routing support in CF CLI (v6.12.3) or a
ticket in CF CLI issues to add the support. Did I miss something there? If
not should I create new ticket?

Kind regards,
Stevo Slavic.

On Tue, May 19, 2015 at 4:41 PM, sabith ks <sabithksme(a)gmail.com>
wrote:

Thanks

On Tue, May 19, 2015 at 7:16 AM James Bayer <jbayer(a)pivotal.io> wrote:

context path based routing is currently experimental and mostly
complete:
http://apidocs.cloudfoundry.org/208/routes/creating_a_route.html

see this epic: https://www.pivotaltracker.com/epic/show/1808212

On Tue, May 19, 2015 at 5:55 AM, sabith ks <sabithksme(a)gmail.com>
wrote:

Hi all,

Was there any follow ups on this proposal

-- Thanks
Sabith

On Fri, Apr 24, 2015 at 08:35 Mike Youngstrom <youngm(a)gmail.com>
wrote:

+1 couldn't have designed the feature better myself. :)

Mike

On Fri, Apr 24, 2015 at 3:52 AM, Dieu Cao <dcao(a)pivotal.io> wrote:

Ooops!

The link to the proposal is here:

[1]
https://docs.google.com/document/d/1H_adSiY7wGR85av9YfxxPRylSO8Q8U0ANJJTg6wpYRQ/edit?usp=sharing

On Friday, April 24, 2015 at 1:11:15 AM UTC-7, Johannes Hiemer
wrote:

Hi Dieu,
link missing? :-/


On Friday, April 24, 2015 at 10:04:39 AM UTC+2, Dieu Cao wrote:

Hi All,

We've put together a proposal for Context Path Routing [1].

The goal of this feature is to enhance the gorouter, the cloud
controller, the routing api, and the cf cli to support the ability for a
space developer to create and map a route with a path to app1 and a route
with a different path to app2.

We'd like to be able to iterate on this document quickly, so
please keep your input and feedback in the comments of the document.

Thanks!
-Dieu
CF Runtime PM

--
You received this message because you are subscribed to the Google
Groups "Cloud Foundry Developers" group.
To view this discussion on the web visit
https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/6f24eea9-7d1c-4547-8ed8-4a4643f1329b%40cloudfoundry.org
<https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/6f24eea9-7d1c-4547-8ed8-4a4643f1329b%40cloudfoundry.org?utm_medium=email&utm_source=footer>
.

To unsubscribe from this group and stop receiving emails from it,
send an email to vcap-dev+unsubscribe(a)cloudfoundry.org.
--
You received this message because you are subscribed to the Google
Groups "Cloud Foundry Developers" group.
To view this discussion on the web visit
https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAEoPEDpqCbFDS3Ogn%2B%3D_Q7FzX6Nr9OnO5_T0o1eYkBOmpEpUxQ%40mail.gmail.com
<https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAEoPEDpqCbFDS3Ogn%2B%3D_Q7FzX6Nr9OnO5_T0o1eYkBOmpEpUxQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
.

To unsubscribe from this group and stop receiving emails from it,
send an email to vcap-dev+unsubscribe(a)cloudfoundry.org.
--
This mailing list is for closed, and is available for archival
purposes only. For active discussion, please visit
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev or email
cf-dev(a)lists.cloudfoundry.org
---
You received this message because you are subscribed to the Google
Groups "Cloud Foundry Developers" group.
To view this discussion on the web visit
https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAHCF%3DJd%2BmiWb5apYUKjS3rVF7sr8vpedKFLQJS5XT5N25HCAcA%40mail.gmail.com
<https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAHCF%3DJd%2BmiWb5apYUKjS3rVF7sr8vpedKFLQJS5XT5N25HCAcA%40mail.gmail.com?utm_medium=email&utm_source=footer>
.


--
Thank you,

James Bayer

--
This mailing list is for closed, and is available for archival
purposes only. For active discussion, please visit
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev or email
cf-dev(a)lists.cloudfoundry.org
---
You received this message because you are subscribed to the Google
Groups "Cloud Foundry Developers" group.
To view this discussion on the web visit
https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAB%3Dt-sWEN8jaxrfRa3zRBbt-AyjaULBnTPHb0%2BpoboqhYp06jQ%40mail.gmail.com
<https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAB%3Dt-sWEN8jaxrfRa3zRBbt-AyjaULBnTPHb0%2BpoboqhYp06jQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
.
--
This mailing list is for closed, and is available for archival
purposes only. For active discussion, please visit
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev or email
cf-dev(a)lists.cloudfoundry.org
---
You received this message because you are subscribed to the Google
Groups "Cloud Foundry Developers" group.
To view this discussion on the web visit
https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAHCF%3DJdyN5BtFJbH0e8sMqgWJ4Z89tx%3D8Ou1kBju%2BWQ3AGAfVw%40mail.gmail.com
<https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAHCF%3DJdyN5BtFJbH0e8sMqgWJ4Z89tx%3D8Ou1kBju%2BWQ3AGAfVw%40mail.gmail.com?utm_medium=email&utm_source=footer>
.


Re: [vcap-dev] Re: Proposal for Context Path Routing

Shannon Coen
 

Hello Stevo,

Thank you for your suggestion of support for wildcards in route paths. I
have noted the request and will keep an ear out for the same request from
others in order to help me prioritize.

Best,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Mon, Sep 14, 2015 at 8:34 AM, Stevo Slavić <sslavic(a)gmail.com> wrote:

Thanks for heads up and fast reply!

Will read spec routing services/API specs in more detail. For the moment,
for use cases I mentioned, one can use/deploy something like Netflix Zuul
(Spring Boot) CF app, as a routing service. Covers a lot of use cases, but
I expected support in CF itself.

Thanks once more!

Kind regards,
Stevo Slavic.

On Mon, Sep 14, 2015 at 4:47 PM, Dieu Cao <dcao(a)pivotal.io> wrote:

Shannon, the Product Manager for the Routing team, has this story [1] in
the CF Routing tracker project for doing a PR to the CLI to add support for
context path routes.

Yes, that's correct that the context path route support does not
currently support the particular example you've specified with wildcards or
routing based on HTTP methods.
If I recall correctly, we wanted to add some basic support for context
path routes without adding too much overhead in what gorouter needed to
reason about, as any logic added could impact every request to the gorouter.

There's a currently in progress tracks of work to support Route Services
[2] [3].
That would allow you to insert you're own proxy to support all sorts of
use cases and could be used to support what you've described.

Shannon is currently on vacation, but should be returning later this
week. I'll forward this request to him for consideration. You might also
consider opening up an issue against the gorouter with these enhancement
requests.

[1] https://www.pivotaltracker.com/story/show/93368928
[2] https://www.pivotaltracker.com/epic/show/2031344
[3] https://www.pivotaltracker.com/epic/show/1884060

On Mon, Sep 14, 2015 at 6:44 AM, Stevo Slavić <sslavic(a)gmail.com> wrote:

In this proposal and current Routes API docs
<http://apidocs.cloudfoundry.org/217/routes/creating_a_route.html> I
don't see support for wildcards when defining context path based route. Am
I missing something?
E.g. for resources like

/foos/bar/1/doSomeAction
/foos/baz/2/doSomeAction

and

/foos/bar/1/doSomeOtherAction
/foos/baz/2/doSomeOtherAction

I'd like all /foos/*/*/doSomeAction to be routed to one CF app, and all
/foos/*/*/doSomeOtherAction to be routed to a different CF app.

Did you consider supporting HTTP methods to be used for routing as well?
E.g. similar to previous example, I'd like some HTTP methods on particular
resources to be handled by separate app then the rest of the requests on
same resources.

DELETE /foos -> route to one cf app
* /foos -> route to another cf app, where * means any HTTP method, most
specific rule wins strategy

Ideally one would be able to combine both wildcards in HTTP methods and
in path segments to define a routing rule.


Also I do not see context path routing support in CF CLI (v6.12.3) or a
ticket in CF CLI issues to add the support. Did I miss something there? If
not should I create new ticket?

Kind regards,
Stevo Slavic.

On Tue, May 19, 2015 at 4:41 PM, sabith ks <sabithksme(a)gmail.com> wrote:

Thanks

On Tue, May 19, 2015 at 7:16 AM James Bayer <jbayer(a)pivotal.io> wrote:

context path based routing is currently experimental and mostly
complete:
http://apidocs.cloudfoundry.org/208/routes/creating_a_route.html

see this epic: https://www.pivotaltracker.com/epic/show/1808212

On Tue, May 19, 2015 at 5:55 AM, sabith ks <sabithksme(a)gmail.com>
wrote:

Hi all,

Was there any follow ups on this proposal

-- Thanks
Sabith

On Fri, Apr 24, 2015 at 08:35 Mike Youngstrom <youngm(a)gmail.com>
wrote:

+1 couldn't have designed the feature better myself. :)

Mike

On Fri, Apr 24, 2015 at 3:52 AM, Dieu Cao <dcao(a)pivotal.io> wrote:

Ooops!

The link to the proposal is here:

[1]
https://docs.google.com/document/d/1H_adSiY7wGR85av9YfxxPRylSO8Q8U0ANJJTg6wpYRQ/edit?usp=sharing

On Friday, April 24, 2015 at 1:11:15 AM UTC-7, Johannes Hiemer
wrote:

Hi Dieu,
link missing? :-/


On Friday, April 24, 2015 at 10:04:39 AM UTC+2, Dieu Cao wrote:

Hi All,

We've put together a proposal for Context Path Routing [1].

The goal of this feature is to enhance the gorouter, the cloud
controller, the routing api, and the cf cli to support the ability for a
space developer to create and map a route with a path to app1 and a route
with a different path to app2.

We'd like to be able to iterate on this document quickly, so
please keep your input and feedback in the comments of the document.

Thanks!
-Dieu
CF Runtime PM

--
You received this message because you are subscribed to the Google
Groups "Cloud Foundry Developers" group.
To view this discussion on the web visit
https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/6f24eea9-7d1c-4547-8ed8-4a4643f1329b%40cloudfoundry.org
<https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/6f24eea9-7d1c-4547-8ed8-4a4643f1329b%40cloudfoundry.org?utm_medium=email&utm_source=footer>
.

To unsubscribe from this group and stop receiving emails from it,
send an email to vcap-dev+unsubscribe(a)cloudfoundry.org.
--
You received this message because you are subscribed to the Google
Groups "Cloud Foundry Developers" group.
To view this discussion on the web visit
https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAEoPEDpqCbFDS3Ogn%2B%3D_Q7FzX6Nr9OnO5_T0o1eYkBOmpEpUxQ%40mail.gmail.com
<https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAEoPEDpqCbFDS3Ogn%2B%3D_Q7FzX6Nr9OnO5_T0o1eYkBOmpEpUxQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
.

To unsubscribe from this group and stop receiving emails from it,
send an email to vcap-dev+unsubscribe(a)cloudfoundry.org.
--
This mailing list is for closed, and is available for archival
purposes only. For active discussion, please visit
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev or email
cf-dev(a)lists.cloudfoundry.org
---
You received this message because you are subscribed to the Google
Groups "Cloud Foundry Developers" group.
To view this discussion on the web visit
https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAHCF%3DJd%2BmiWb5apYUKjS3rVF7sr8vpedKFLQJS5XT5N25HCAcA%40mail.gmail.com
<https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAHCF%3DJd%2BmiWb5apYUKjS3rVF7sr8vpedKFLQJS5XT5N25HCAcA%40mail.gmail.com?utm_medium=email&utm_source=footer>
.


--
Thank you,

James Bayer

--
This mailing list is for closed, and is available for archival
purposes only. For active discussion, please visit
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev or email
cf-dev(a)lists.cloudfoundry.org
---
You received this message because you are subscribed to the Google
Groups "Cloud Foundry Developers" group.
To view this discussion on the web visit
https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAB%3Dt-sWEN8jaxrfRa3zRBbt-AyjaULBnTPHb0%2BpoboqhYp06jQ%40mail.gmail.com
<https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAB%3Dt-sWEN8jaxrfRa3zRBbt-AyjaULBnTPHb0%2BpoboqhYp06jQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
.
--
This mailing list is for closed, and is available for archival purposes
only. For active discussion, please visit
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev or email
cf-dev(a)lists.cloudfoundry.org
---
You received this message because you are subscribed to the Google
Groups "Cloud Foundry Developers" group.
To view this discussion on the web visit
https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAHCF%3DJdyN5BtFJbH0e8sMqgWJ4Z89tx%3D8Ou1kBju%2BWQ3AGAfVw%40mail.gmail.com
<https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAHCF%3DJdyN5BtFJbH0e8sMqgWJ4Z89tx%3D8Ou1kBju%2BWQ3AGAfVw%40mail.gmail.com?utm_medium=email&utm_source=footer>
.


Re: DEA/Warden staging error

CF Runtime
 

Also worth noting: if you're cloning from the ssh endpoint at github you
must have an ssh key in your ssh agent that authenticates against a real
github user, even if the repo is public. It seems as though you're using
the https endpoint so theoretically this should not affect you.

Zak + Natalie,
CF OSS Integration

On Wed, Sep 16, 2015 at 8:55 AM, kyle havlovitz <kylehav(a)gmail.com> wrote:

It's just the staticfile buildpack from
https://github.com/cloudfoundry/staticfile-buildpack.git, I can try git
cloning it from inside the warden container though

On Wed, Sep 16, 2015 at 9:30 AM, Mike Dalessio <mdalessio(a)pivotal.io>
wrote:

Worth noting that the git repo also needs to allow anonymous access. If
it's a private repo, then the 'git clone' is going to fail.

Can you verify that you can download the buildpack from your repo without
authenticating?

On Tue, Sep 15, 2015 at 7:43 PM, CF Runtime <cfruntime(a)gmail.com> wrote:

It's not something we've ever seen before.

In theory, the warden container needs the git binary, which I think it
gets from the cflinuxfs2 stack; and internet access to wherever the git
repo lives.

If the warden container has both of those things, I can't think of any
reason why it wouldn't work.

Joseph
OSS Release Integration Team

On Tue, Sep 15, 2015 at 2:06 PM, kyle havlovitz <kylehav(a)gmail.com>
wrote:

I tried deploying via uploading a buildpack to the CC (had to set up
nginx first, I didnt have it running/configured before) and that worked! So
that's awesome, but I'm not sure what the problem with using a remote
buildpack is. Even with nginx, I still get the exact same error as before
when pushing using a remote buildpack from git.

On Tue, Sep 15, 2015 at 6:57 AM, CF Runtime <cfruntime(a)gmail.com>
wrote:

Looking at the logs, we can see it finishing downloading the app
package. The next step should be to download and run the buildpack. Since
you mention there is no output after this, I'm guessing it doesn't get that
far.

It might be having trouble downloading the buildpack from the remote
git url. Could you try uploading the buildpack to Cloud Controller and then
having it use that buildpack to see if that makes a difference?


http://apidocs.cloudfoundry.org/217/buildpacks/creates_an_admin_buildpack.html

http://apidocs.cloudfoundry.org/217/buildpacks/upload_the_bits_for_an_admin_buildpack.html

Joseph
OSS Release Integration Team

On Mon, Sep 14, 2015 at 5:37 PM, kyle havlovitz <kylehav(a)gmail.com>
wrote:

Here's the full dea_ng and warden debug logs:
https://gist.github.com/MrEnzyme/6dcc74174482ac62c1cf

Are there any other places I should look for logs?

On Mon, Sep 14, 2015 at 8:14 PM, CF Runtime <cfruntime(a)gmail.com>
wrote:

That's not an error we normally get. It's not clear if the
staging_info.yml error is the source of the problem or an artifact of it.
Having more logs would allow us to speculate more.

Joseph & Dan
OSS Release Integration Team

On Mon, Sep 14, 2015 at 2:24 PM, kyle havlovitz <kylehav(a)gmail.com>
wrote:

I have the cloudfoundry components built, configured and running on
one VM (not in BOSH), and when I push an app I'm getting a generic 'FAILED
StagingError' message after '-----> Downloaded app package (460K)'.

There's nothing in the logs for the dea/warden that seems suspect
other than these 2 things:


{
"timestamp": 1441985105.8883495,

"message": "Exited with status 1 (35.120s):
[[\"/opt/cloudfoundry/warden/warden/src/closefds/closefds\",
\"/opt/cloudfoundry/warden/warden/src/closefds/closefds\"],
\"/var/warden/containers/18vf956il5v/bin/iomux-link\", \"-w\",
\"/var/warden/containers/18vf956il5v/jobs/8/cursors\",
\"/var/warden/containers/18vf956il5v/jobs/8\"]",
"log_level": "warn",

"source": "Warden::Container::Linux",

"data": {

"handle": "18vf956il5v",

"stdout": "",

"stderr": ""

},

"thread_id": 69890836968240,

"fiber_id": 69890849112480,

"process_id": 17063,

"file":
"/opt/cloudfoundry/warden/warden/lib/warden/container/spawn.rb",
"lineno": 135,

"method": "set_deferred_success"

}



{
"timestamp": 1441985105.94083,

"message": "Exited with status 23 (0.023s):
[[\"/opt/cloudfoundry/warden/warden/src/closefds/closefds\",
\"/opt/cloudfoundry/warden/warden/src/closefds/closefds\"], \"rsync\",
\"-e\", \"/var/warden/containers/18vf956il5v/bin/wsh --socket
/var/warden/containers/18vf956il5v/run/wshd.sock --rsh\", \"-r\", \"-p\",
\"--links\", \"vcap(a)container:/tmp/staged/staging_info.yml\",
\"/tmp/dea_ng/staging/d20150911-17093-1amg6y8\"]",
"log_level": "warn",

"source": "Warden::Container::Linux",

"data": {

"handle": "18vf956il5v",

"stdout": "",

"stderr": "rsync: link_stat
\"/tmp/staged/staging_info.yml\" failed: No such file or directory
(2)\nrsync error: some files/attrs were not transferred (see previous
errors) (code 23) at main.c(1655) [Receiver=3.1.0]\nrsync: [Receiver] write
error: Broken pipe (32)\n"
},

"thread_id": 69890836968240,

"fiber_id": 69890849112480,

"process_id": 17063,

"file":
"/opt/cloudfoundry/warden/warden/lib/warden/container/spawn.rb",
"lineno": 135,

"method": "set_deferred_success"

}


And I think the second error is just during cleanup, only failing
because the staging process didn't get far enough in to create the
'staging_info.yml'. The one about iomux-link exiting with status 1 is
pretty mysterious though and I have no idea what caused it. Does anyone
know why this might be happening?


Re: Proposal: Decomposing cf-release and Extracting Deployment Strategies

Mike Youngstrom
 

Here are some of the examples:

* Sensitive property management as part of manifest generation (encrypted
or acquired from an outside source)

* We have our own internal bosh releases and config that we'll need to
merge in with the things cf-deployment is doing. For example, if
cf-deployment tags v250 as including Diego 3333 and etcd 34 with given
templates perhaps we'd like to augment this with our own release jobs and
config that we know to work with cf-deployment 250's and perhaps tag it as
v250.lds and that becomes what we use to generate our manifests and upload
releases.

* Occasionally we may wish to use some config from a stock release not
currently exposed in a cf-deployment template. I'd like to be sure there
is a way we can add that config, in a not hacky way, without waiting for a
PR to be accepted and subsequent release.

* If for some reason we are forced to fork a stock release we'd like to be
able to use that forked release we are building instead of the publicly
available one for manifest generation and release uploads, etc.

Does that help?

Mike

On Tue, Sep 15, 2015 at 9:50 PM, Amit Gupta <agupta(a)pivotal.io> wrote:

Thanks for the feedback Mike!

Can you tell us more specifically what sort of extensions you need? It
would be great if cf-deployment provided an interface that could serve the
needs of essentially all operators of CF.

Thanks,
Amit

On Tue, Sep 15, 2015 at 4:02 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

This is great stuff! My organization currently maintains our own custom
ways to generate manifests, include secure properties, and manage release
versions.

We would love to base the next generation of our solution on
cf-deployment. Have you put any thought into how others might customize or
extend cf-deployment? Our needs are very similar to yours just sometimes a
little different.

Perhaps a private fork periodically merged with a known good release
combination (tag) might be appropriate? Or some way to include the same
tools into a wholly private repo?

Mike


On Tue, Sep 8, 2015 at 1:22 PM, Amit Gupta <agupta(a)pivotal.io> wrote:

Hi all,

The CF OSS Release Integration team (casually referred to as the "MEGA
team") is trying to solve a lot of tightly interrelated problems, and make
many of said problems less interrelated. It is difficult to address just
one issue without touching the others, so the following proposal addresses
several issues, but the most important ones are:

* decompose cf-release into many independently manageable, independently
testable, independently usable releases
* separate manifest generation strategies from the release source,
paving the way for Diego to be part of the standard deployment

This proposal will outline a picture of how manifest generation will
work in a unified manner in development, test, and integration
environments. It will also outline a picture of what each release’s test
pipelines will look like, how they will feed into a common integration
environment, and how feedback from the integration environment will feed
back into the test environments. Finally, it will propose a picture for
what the integration environment will look like, and how we get from the
current integration environment to where we want to be.

For further details, please feel free to view and comment here:


https://docs.google.com/document/d/1Viga_TzUB2nLxN_ILqksmUiILM1hGhq7MBXxgLaUOkY

Thanks,
Amit, CF OSS Release Integration team


Re: Environment variable names (was RE: Environment variable changes in DIEGO)

john mcteague <john.mcteague@...>
 

Add another +1. We are implementing something akin to this within out
environment, having it standardised would be a big win.

On Wed, Sep 16, 2015 at 8:49 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

+1 We'd love a metadata service as well when the dust settles.

Mike

On Wed, Sep 16, 2015 at 1:17 PM, Christopher B Ferris <chrisfer(a)us.ibm.com
wrote:
Onsi,

I can tell you that we (IBM) would welcome a secure metadata service for
retrieving credentials, etc. I'm sure we'd help make that happen.

Cheers,

Christopher Ferris
IBM Distinguished Engineer, CTO Open Technology
IBM Cloud, Open Technologies
email: chrisfer(a)us.ibm.com
twitter: @christo4ferris
blog: http://thoughtsoncloud.com/index.php/author/cferris/
phone: +1 508 667 0402



----- Original message -----
From: Onsi Fakhouri <ofakhouri(a)pivotal.io>
To: "Discussions about Cloud Foundry projects and the system overall." <
cf-dev(a)lists.cloudfoundry.org>
Cc:
Subject: [cf-dev] Re: Environment variable names (was RE: Environment
variable changes in DIEGO)
Date: Wed, Sep 16, 2015 11:40 AM


Would if we could! Environment variables are part of the surface area
that constitutes the contract between the platform and the application.
While we could have duplicates with CF_* and eventually deprecate VCAP_*
the new runtime needs to support droplets staged by the legacy runtime
seamlessly.

Once the dust settles I'd sooner see us step back and reconsider this
particular abstraction. For example, instead of env vars an authenticated
metadata service a la AWS's http://169.254.169.254/ would give us more
flexibility, allow us to dynamically control metadata made available to
apps, and even version metadata sanely.

Onsi

Sent from my iPad

On Sep 16, 2015, at 8:30 AM, john mcteague <john.mcteague(a)gmail.com>
wrote:


On a related, but slightly off, topic, while renaming the VCAP_* vars
would have a big impact, is it not time we thought about renaming these to
CF_* ?

John.
On 16 Sep 2015 16:09, "Matthew Sykes" <matthew.sykes(a)gmail.com> wrote:

The changes, in general, were intentional. The `application_uris` data
was always broken as it didn't reflect route changes. I can't speak
directly to the time stamp data.

The host is present still so I don't know why you don't see it.

We also have a migration guide [1]. If you think additional information
is needed there, pull requests would be welcome.

[1]:
https://github.com/cloudfoundry-incubator/diego-design-notes/blob/master/migrating-to-diego.md

On Wed, Sep 16, 2015 at 10:19 AM, Jack Cai <greensight(a)gmail.com> wrote:

I notice the below changes in the environment variables of DIEGO:
1. VCAP_APP_HOST & VCAP_APP_PORT are removed.
2. These fields are removed from VCAP_APPLICATION value:
application_uris, started_at, start, started_at_timestamp, host,
state_timestamp

I suppose #1 is intentional. How about #2?

Jack




--
Matthew Sykes
matthew.sykes(a)gmail.com




Re: Environment variable names (was RE: Environment variable changes in DIEGO)

Mike Youngstrom
 

+1 We'd love a metadata service as well when the dust settles.

Mike

On Wed, Sep 16, 2015 at 1:17 PM, Christopher B Ferris <chrisfer(a)us.ibm.com>
wrote:

Onsi,

I can tell you that we (IBM) would welcome a secure metadata service for
retrieving credentials, etc. I'm sure we'd help make that happen.

Cheers,

Christopher Ferris
IBM Distinguished Engineer, CTO Open Technology
IBM Cloud, Open Technologies
email: chrisfer(a)us.ibm.com
twitter: @christo4ferris
blog: http://thoughtsoncloud.com/index.php/author/cferris/
phone: +1 508 667 0402



----- Original message -----
From: Onsi Fakhouri <ofakhouri(a)pivotal.io>
To: "Discussions about Cloud Foundry projects and the system overall." <
cf-dev(a)lists.cloudfoundry.org>
Cc:
Subject: [cf-dev] Re: Environment variable names (was RE: Environment
variable changes in DIEGO)
Date: Wed, Sep 16, 2015 11:40 AM


Would if we could! Environment variables are part of the surface area
that constitutes the contract between the platform and the application.
While we could have duplicates with CF_* and eventually deprecate VCAP_*
the new runtime needs to support droplets staged by the legacy runtime
seamlessly.

Once the dust settles I'd sooner see us step back and reconsider this
particular abstraction. For example, instead of env vars an authenticated
metadata service a la AWS's http://169.254.169.254/ would give us more
flexibility, allow us to dynamically control metadata made available to
apps, and even version metadata sanely.

Onsi

Sent from my iPad

On Sep 16, 2015, at 8:30 AM, john mcteague <john.mcteague(a)gmail.com>
wrote:


On a related, but slightly off, topic, while renaming the VCAP_* vars
would have a big impact, is it not time we thought about renaming these to
CF_* ?

John.
On 16 Sep 2015 16:09, "Matthew Sykes" <matthew.sykes(a)gmail.com> wrote:

The changes, in general, were intentional. The `application_uris` data was
always broken as it didn't reflect route changes. I can't speak directly to
the time stamp data.

The host is present still so I don't know why you don't see it.

We also have a migration guide [1]. If you think additional information is
needed there, pull requests would be welcome.

[1]:
https://github.com/cloudfoundry-incubator/diego-design-notes/blob/master/migrating-to-diego.md

On Wed, Sep 16, 2015 at 10:19 AM, Jack Cai <greensight(a)gmail.com> wrote:

I notice the below changes in the environment variables of DIEGO:
1. VCAP_APP_HOST & VCAP_APP_PORT are removed.
2. These fields are removed from VCAP_APPLICATION value: application_uris,
started_at, start, started_at_timestamp, host, state_timestamp

I suppose #1 is intentional. How about #2?

Jack




--
Matthew Sykes
matthew.sykes(a)gmail.com




Re: Packaging CF app as bosh-release

Amit Kumar Gupta
 

My very limited understanding is that NFS writes to the actual filesystem,
and achieves persistence by having centralized NFS servers where it writes
to a real mounted device, whereas the clients write to an ephemeral
nfs-mount.

My very limited understanding of HDFS is that it's all userland FS, does
not write to the actual filesystem, and relies on replication to other
nodes in the HDFS cluster. Being a userland FS, you don't have to worry
about the data being wiped when a container is shut down, if you were to
run it as an app.

I think one main issue is going to be ensuring that you never lose too many
instances (whether they are containers or VMs), since you might then lose
all replicas of a given data shard. Whether you go with apps or BOSH VMs
doesn't make a big difference here.

Deploying as an app may be a better way to go, it's simpler right now to
configure and deploy and app, than to configure and deploy a full BOSH
release. It's also likely to be a more efficient use of resources, since a
BOSH VM can only run one of these spark-job-processors, but a CF
container-runner can run lots of other things. That actually brings up a
different question: is your compute environment a multi-tenant one that
will be running multiple different workloads? E.g. could someone also use
the CF to push their own apps? Or is the whole thing just for your spark
jobs, in which case you might only be running one container per VM anyways?

Assuming you can make use of the VMs for other workloads, I think this
would be an ideal use case for Diego. You probably don't need all the
extra logic around apps, like staging and routing, you just need Diego to
efficiently schedule containers for you.

On Wed, Sep 16, 2015 at 1:13 PM, Kayode Odeyemi <dreyemi(a)gmail.com> wrote:

Thanks Dmitriy,

Just for clarity, are you saying multiple instances of a VM cannot share a
single shared filesystem?

On Wed, Sep 16, 2015 at 6:59 PM, Dmitriy Kalinin <dkalinin(a)pivotal.io>
wrote:

BOSH allocates a persistent disk per instance. It never shares persistent
disks between multiple instances at the same time.

If you need a shared file system, you will have to use some kind of a
release for it. It's not any different from what people do with nfs
server/client.

On Wed, Sep 16, 2015 at 7:09 AM, Amit Gupta <agupta(a)pivotal.io> wrote:

The shared file system aspect is an interesting wrinkle to the problem.
Unless you use some network layer to how you write to the shared file
system, e.g. SSHFS, I think apps will not work because they get isolated to
run in a container, they're given a chroot "jail" for their file system,
and it gets blown away whenever the app is stopped or restarted (which will
commonly happen, e.g. during a rolling deploy of the container-runner VMs).

Do you have something that currently works? How do your VMs currently
access this shared FS? I'm not sure BOSH has the abstractions for choosing
a shared, already-existing "persistent disk" to be attached to multiple
VMs. I also don't know what happens when you scale your VMs down, because
BOSH would generally destroy the associated persistent disk, but you don't
want to destroy the shared data.

Dmitriy, any idea how BOSH can work with a shared filesystem (e.g. HDFS)?

Amit

On Wed, Sep 16, 2015 at 6:54 AM, Kayode Odeyemi <dreyemi(a)gmail.com>
wrote:


On Wed, Sep 16, 2015 at 3:44 PM, Amit Gupta <agupta(a)pivotal.io> wrote:

Are the spark jobs tasks that you expect to end, or apps that you
expect to run forever?
They are tasks that run forever. The jobs are subscribers to RabbitMQ
queues that process
messages in batches.


Do your jobs need to write to the file system, or do they access a
shared/distributed file system somehow?
The jobs write to shared filesystem.


Do you need things like a static IP allocated to your jobs?
No.


Are your spark jobs serving any web traffic?
No.




Re: Environment variable names (was RE: Environment variable changes in DIEGO)

Christopher B Ferris <chrisfer@...>
 

Onsi,
 
I can tell you that we (IBM) would welcome a secure metadata service for retrieving credentials, etc. I'm sure we'd help make that happen.
 
Cheers,

Christopher Ferris
IBM Distinguished Engineer, CTO Open Technology
IBM Cloud, Open Technologies
email: chrisfer@...
twitter: @christo4ferris
blog: http://thoughtsoncloud.com/index.php/author/cferris/
phone: +1 508 667 0402
 
 

----- Original message -----
From: Onsi Fakhouri <ofakhouri@...>
To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@...>
Cc:
Subject: [cf-dev] Re: Environment variable names (was RE: Environment variable changes in DIEGO)
Date: Wed, Sep 16, 2015 11:40 AM
 
 
Would if we could!  Environment variables are part of the surface area that constitutes the contract between the platform and the application.  While we could have duplicates with CF_* and eventually deprecate VCAP_* the new runtime needs to support droplets staged by the legacy runtime seamlessly.
 
Once the dust settles I'd sooner see us step back and reconsider this particular abstraction.  For example, instead of env vars an authenticated metadata service a la AWS's http://169.254.169.254/ would give us more flexibility, allow us to dynamically control metadata made available to apps, and even version metadata sanely.
 
Onsi

Sent from my iPad

On Sep 16, 2015, at 8:30 AM, john mcteague <john.mcteague@...> wrote:
 

On a related, but slightly off, topic, while renaming the VCAP_* vars would have a big impact, is it not time we thought about renaming these to CF_*  ?

John.

On 16 Sep 2015 16:09, "Matthew Sykes" <matthew.sykes@...> wrote:
The changes, in general, were intentional. The `application_uris` data was always broken as it didn't reflect route changes. I can't speak directly to the time stamp data.
 
The host is present still so I don't know why you don't see it.
 
We also have a migration guide [1]. If you think additional information is needed there, pull requests would be welcome.
 
 
On Wed, Sep 16, 2015 at 10:19 AM, Jack Cai <greensight@...> wrote:
I notice the below changes in the environment variables of DIEGO:
1. VCAP_APP_HOST & VCAP_APP_PORT are removed.
2. These fields are removed from VCAP_APPLICATION value: application_uris, started_at, start, started_at_timestamp, host, state_timestamp
 
I suppose #1 is intentional. How about #2?
 
Jack
 
 
 
--
Matthew Sykes
matthew.sykes@...
 


Re: consolidated routing api

Shannon Coen
 

Checking once more that no one is using the Routing API yet, so we can
proceed with backward incompatible API changes. I believe we still have
this opportunity as we have not yet announced the component is ready for
use.

Our current plan is that the first consumers of the Routing API will be:
- CC (to register it's own route: api.system-domain)
- TCP Route Emitter (to register routes for Diego LRPs with TCP Routes)
- TCP Router (to fetch routing table of TCP routes)

Thank you,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Wed, Sep 9, 2015 at 6:05 PM, Shannon Coen <scoen(a)pivotal.io> wrote:

Some of the proposed changes to the Routing API are backward incompatible.
We don't believe anyone is using it yet, as adoption has generally be
blocked on securing connections to Consul, but we'd like to confirm.

Please raise your hand if you're using the routing API.

Thank you!

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Wed, Sep 9, 2015 at 12:10 PM, Shannon Coen <scoen(a)pivotal.io> wrote:

We currently have two routing APIs in CF.
1. HTTP Routing API in cf-release:
https://github.com/cloudfoundry-incubator/routing-api
2. TCP Routing API in cf-routing-release:
https://github.com/cloudfoundry-incubator/cf-routing-release

The TCP Routing API is quite basic and we want to extend it for high
availability, authentication, etc. However, instead of enhancing the
existing TCP Routing API, we plan to add support for TCP route registration
to the Routing API in cf-release, as it already supports many of the
desired features. We'll get rid of the current API in cf-routing-release
and submodule in the Routing API from cf-release. Eventually we'll move the
Routing API (along with GoRouter and HAProxy) from cf-release into
cf-routing-release and submodule them into cf-release.

This consolidation, along with our not having any API consumer besides
GoRouter yet, gives us the opportunity to consider a common behavior for
routing API endpoints. We welcome your comments in our design doc:


https://docs.google.com/document/d/1v941oy3Y7RRI80gmLfhPZlaahElK_Q0C3pCQewK8Z3g/edit?usp=sharing

Thank you,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.


Re: Relationship between HM9000 and router jobs

CF Runtime
 

The DEAs are responsible for broadcasting the routes for the apps they are
running. I can't think of why an hm9000 problem would cause routes to get
lost, unless there was some problem with NATS itself.

Joseph
OSS Release Integration Team

On Wed, Sep 16, 2015 at 1:37 AM, Sylvain FAUVE <sylvain.fauve(a)sap.com>
wrote:

Hello,

My team was working on solving inconsistencies issue on etcd jobs, and
realized that two hm9000 jobs were running at same time.
When fixing this, we experienced route loss to our apps (then restage apps
was needed).

As far as I could read/understand there is no direct communication between
router and hm9000...
Router is getting info from NATS, and NATS gets it from ...? hm9000 ?
I wonder which component is sending routes update to the router to keep
them alive ?


Thank you for your help
Regards,
Sylvain.


Re: cc metric: total_users

CF Runtime
 

Yeah, just as it is possible to create users in UAA, it is also possible to
delete them there, thus orphaning the user accounts in the Cloud Controller.

The orphaned users won't cause any problems in the Cloud Controller, but
the metrics may not be what you expect at that point.

The cf CLI automatically deletes users out of both when it is used.

Joseph
OSS Release Integration Team

On Wed, Sep 16, 2015 at 4:29 AM, Klevenz, Stephan <stephan.klevenz(a)sap.com>
wrote:

Hi,

I did look deeper into implementation. Actually there are two databases:
ccdb and uaadb. Each of them has its own users table. The user count value
of the uaadb/users is reported to admin ui and the user count of the
ccdb/users is used for total_user metric. A ccdb/users table entry contains
just a refid and for getting user details from uaa.

So, we have two totals which can be different if users created for uaa
which a not cc users. That's fine and this is what I did get from the first
answer.

But there is a remaining open point. The total of our ccdb/users is bigger
than total of uaa/users. This is an inconsistency in ccdb/users which
contains references to uaa users that do not exist. If this diffs grows
over time then this is maybe a problem.

Regards,
Stephan







Von: CF Runtime
Antworten an: "Discussions about Cloud Foundry projects and the system
overall."
Datum: Mittwoch, 16. September 2015 10:56
An: "Discussions about Cloud Foundry projects and the system overall."
Betreff: [cf-dev] Re: Re: Re: Re: cc metric: total_users

The users reported by CloudController.total_users are the users the Cloud
Controller has in its database. This is normally the same set of users that
exist in UAA.

However, there is nothing that prevents you from creating users via the
UAAC cli tool, or creating new UAA clients that can create users themselves.

Joseph
OSS Release Integration Team

On Wed, Sep 16, 2015 at 12:03 AM, Voelz, Marco <marco.voelz(a)sap.com>
wrote:

Hi John,

thanks for your answer, however, I don’t understand that completely. For
the open source version of CF, what does “register in CF console” mean? And
what might be an example of “other applications” you are referring to?

Thanks and warm regards
Marco

On 16/09/15 08:29, "Klevenz, Stephan" <stephan.klevenz(a)sap.com> wrote:

Thanks for clarification

-- Stephan

Von: John Liptak
Antworten an: "Discussions about Cloud Foundry projects and the system
overall."
Datum: Dienstag, 15. September 2015 18:17
An: "Discussions about Cloud Foundry projects and the system overall."
Betreff: [cf-dev] Re: cc metric: total_users

Cloud Controller reports the number of users register in CF console. UAAC
reports additional users who may have access to other applications. So
they are both correct, depending on what you need.

For example, if you call the REST API for a UAAC user that isn't in the
CF console, but still call the cloud controller REST API, you will get a
404.

On Tue, Sep 15, 2015 at 10:10 AM, Klevenz, Stephan <
stephan.klevenz(a)sap.com> wrote:

Hi all,

I have a question, hopefully a small one :-)

The CloudController.total_users metric
(/CF\.CloudController\.0\..*\.total_users/) differs from number of users
reported by uaac (uaac users command / admin ui). Can someone explain why
this differs or which number is the correct one?

Regards,
Stephan





Re: Packaging CF app as bosh-release

Paul Bakare
 

Thanks Dmitriy,

Just for clarity, are you saying multiple instances of a VM cannot share a
single shared filesystem?

On Wed, Sep 16, 2015 at 6:59 PM, Dmitriy Kalinin <dkalinin(a)pivotal.io>
wrote:

BOSH allocates a persistent disk per instance. It never shares persistent
disks between multiple instances at the same time.

If you need a shared file system, you will have to use some kind of a
release for it. It's not any different from what people do with nfs
server/client.

On Wed, Sep 16, 2015 at 7:09 AM, Amit Gupta <agupta(a)pivotal.io> wrote:

The shared file system aspect is an interesting wrinkle to the problem.
Unless you use some network layer to how you write to the shared file
system, e.g. SSHFS, I think apps will not work because they get isolated to
run in a container, they're given a chroot "jail" for their file system,
and it gets blown away whenever the app is stopped or restarted (which will
commonly happen, e.g. during a rolling deploy of the container-runner VMs).

Do you have something that currently works? How do your VMs currently
access this shared FS? I'm not sure BOSH has the abstractions for choosing
a shared, already-existing "persistent disk" to be attached to multiple
VMs. I also don't know what happens when you scale your VMs down, because
BOSH would generally destroy the associated persistent disk, but you don't
want to destroy the shared data.

Dmitriy, any idea how BOSH can work with a shared filesystem (e.g. HDFS)?

Amit

On Wed, Sep 16, 2015 at 6:54 AM, Kayode Odeyemi <dreyemi(a)gmail.com>
wrote:


On Wed, Sep 16, 2015 at 3:44 PM, Amit Gupta <agupta(a)pivotal.io> wrote:

Are the spark jobs tasks that you expect to end, or apps that you
expect to run forever?
They are tasks that run forever. The jobs are subscribers to RabbitMQ
queues that process
messages in batches.


Do your jobs need to write to the file system, or do they access a
shared/distributed file system somehow?
The jobs write to shared filesystem.


Do you need things like a static IP allocated to your jobs?
No.


Are your spark jobs serving any web traffic?
No.



7581 - 7600 of 9398