Date   

Re: Failed to deploy diego 0.1452.0 on openstack: database_z2/0 is not running after update

Adrian Zankich
 

Hi Ricky,

We deconstructed the certs you provided in your manifest and think that you may have missed a step when you generated your peer ssl cert. Your peer cert is missing the DNS wildcard entry '*.etcd.service.cf.internal`, it will show up like this if you deconstruct your cert

X509v3 Subject Alternative Name:
DNS:*.etcd.service.cf.internal, DNS:etcd.service.cf.internal

If you regenerate your peer ssl cert with:

$ certstrap --depot-path peer request-cert --common-name "etcd.service.cf.internal" --domain "*.etcd.service.cf.internal,etcd.service.cf.internal"

It is detailed in https://github.com/cloudfoundry-incubator/diego-release#generating-tls-certificates step #8.

That should fix the ssl error you're experiencing.

- Adrian


Re: Using swift as a blobstore in cloud foundry with keystone v3

Marco Voelz
 

Hi Dies, Nick,

I can confirm that a swift blobstore works with domains different than 'default'. Here is our configuration on an OpenStack Liberty

fog_connection: &fog_connection
provider: OpenStack
openstack_auth_url: https://<keystone-url>:5000/v3/auth/tokens<https://cluster-4.eu-de-1.cloud.sap:5000/v3/auth/tokens>
openstack_username: <username>
openstack_api_key: <password>
openstack_project_name: <openstack project>
openstack_domain_name: MY_CUSTOM_DOMAIN
# register tem url key with swift:
# swift post -m "Temp-URL-Key:tempurlkey"
openstack_temp_url_key: tempurlkey

Warm regards
Marco

On 01/04/16 14:55, "Koper, Dies" <diesk(a)fast.au.fujitsu.com<mailto:diesk(a)fast.au.fujitsu.com>> wrote:

Hi Nick,

Can you try with a domain other than “default”?
We initially found that with the BOSH CPI, Keystone V3 worked for the “default” domain, but not for any other domain.
Note that the point of using Keystone V3 is to have different domains.

Also, we can connect to swift using the temp urls with the openstack cli fine.

Cheers,
Dies Koper


From: Nicholas Calugar [mailto:ncalugar(a)pivotal.io]
Sent: Thursday, March 31, 2016 11:18 AM
To: Discussions about Cloud Foundry projects and the system overall.
Subject: [cf-dev] Re: Re: Re: Re: Using swift as a blobstore in cloud foundry with keystone v3

Before confirming the new version of fog fixed this, I tried to reproduce the error using CF v233. Instead of reproducing the error, it actually worked.

V2 CONFIG:
provider: 'OpenStack'
openstack_tenant: 'admin'
openstack_username: 'admin'
openstack_api_key: '*************'
openstack_auth_url: 'https://my-openstack:5000/v2.0/tokens'
openstack_temp_url_key: 'b3968d0207b54ece87cccc06515a89d4'
connection_options:
ssl_verify_peer: false

V3 CONFIG:
provider: 'OpenStack'
openstack_tenant: 'admin'
openstack_project_name: 'admin'
openstack_domain_name: 'default'
openstack_username: 'admin'
openstack_api_key: '**************'
openstack_auth_url: 'https://my-openstack:5000/v3/auth/tokens'
openstack_temp_url_key: 'b3968d0207b54ece87cccc06515a89d4'
connection_options:
ssl_verify_peer: false

I would recommend using the openstack / swift cli to ensure you can correctly generate temp urls. The DEAs / CELLs use the tempurl feature to download objects from the blobstore and I think that is the error you are running into. Once you can get it to work via CLI, translate the correct variables into the fog_configuration for Cloud Foundry.


Thanks,

Nick

On Mon, Mar 21, 2016 at 10:21 AM Nicholas Calugar <ncalugar(a)pivotal.io<mailto:ncalugar(a)pivotal.io>> wrote:
We have a story in flight: https://www.pivotaltracker.com/story/show/115793253

If upgrading fog doesn't work, we'll happily take a PR that resolves this.

On Thu, Mar 17, 2016 at 5:33 PM Gwenn Etourneau <getourneau(a)pivotal.io<mailto:getourneau(a)pivotal.io>> wrote:
This can be nice ! As Bosh support V3 it can be nice that the CC do the same.
To be able to fully use v3.



On Thu, Mar 17, 2016 at 6:30 PM, Voelz, Marco <marco.voelz(a)sap.com<mailto:marco.voelz(a)sap.com>> wrote:
Dear Nicholas,

if desired, we can also do a PR for allowing the CC to connect to Swift using Keystone v3. A couple of months ago we did the same thing in the OpenStack CPI. We also have some test envs available where we can validate the change. What do you think?

Warm regards
Marco

On 16/03/16 18:13, "Nicholas Calugar" <ncalugar(a)pivotal.io<mailto:ncalugar(a)pivotal.io>> wrote:

Hi Muhammad,

Unfortunately, we don't have an environment using keystone v3. We are passing the configuration to fog as-is. There are several fixes that have been made in later releases of fog, for example:

https://github.com/fog/fog/pull/3806

I'll get a story prioritized to upgrade fog to v1.37.0

Thanks,

Nick

On Sun, Mar 13, 2016 at 11:58 PM Altaf, Muhammad <Muhammada(a)fast.au.fujitsu.com<mailto:Muhammada(a)fast.au.fujitsu.com>> wrote:
Hi All,
I am trying to configure cloud foundry to use swift on OpenStack. I have followed the instructions at https://docs.cloudfoundry.org/deploying/openstack/using_swift_blobstore.html
When used keystone v2, I am able to start my apps on DEA which is good. However when using keystone V3, I am not able to start my apps. The error I am getting is:

“FAILED
Server error, status code: 400, error code: 170001, message: Staging error: failed to stage application:
Error downloading: HTTP status: 401”

Tried to debug by adding some ‘puts’ statements in openstack/core.rb file and it looks like tokens are being generated successfully so there is no problem with the authentication. The generated response to auth request shows that the user has “ResellerAdmin” role as well.

When I look into runner_z1/0 /var/vcap/data/dea_next/tmp/ app-package-download.tgz2016*, I find error saying: “401 Unauthorized: Temp URL invalid xxxxx”

/var/vcap/sys/log/dea_next/dea_next.log shows some download URLs, and if I curl those URLs, I get exact same error message. Below are the fog_connection settings in cloud foundry manifest:

fog_connection: &fog_connection
provider: 'OpenStack'
openstack_username: 'cf-admin2'
openstack_tenant: 'cf2'
openstack_project_name: 'cf2'
openstack_api_key: 'passw0rd'
openstack_auth_url: 'http://<OPENSTACK_IP>:5000/v3/auth/tokens'
openstack_domain_name: 'cf_domain'
openstack_user_domain_name: 'cf_domain'
openstack_temp_url_key: 'b3968d0207b54ece87cccc06515a89d4'


Account has a valid temp_url_key configured. Please see below:
curl -v -X GET http://SWIFT_IP:SWIFT_PORT/v2/Auth_b34a51e551ec4796a461168c886c734f -H "X-Auth-Token: TOKEN"
* Hostname was NOT found in DNS cache
* Trying SWIFT_IP...
* Connected to SWIFT_IP (SWIFT_IP) port SWIFT_PORT (#0)
GET /v2/Auth_b34a51e551ec4796a461168c886c734f HTTP/1.1
User-Agent: curl/7.35.0
Host: SWIFT_IP:SWIFT_PORT
Accept: */*
X-Auth-Token: TOKEN
< HTTP/1.1 204 No Content
< Content-Length: 0
< X-Account-Object-Count: 0
< X-Timestamp: 1457918518.21777
< X-Account-Meta-Temp-Url-Key: b3968d0207b54ece87cccc06515a89d4
< X-Account-Bytes-Used: 0
< X-Account-Container-Count: 0
< Content-Type: text/plain; charset=utf-8
< Accept-Ranges: bytes
< X-Trans-Id: txfc362c27bdda4355a942a-0056e65d93
< Date: Mon, 14 Mar 2016 06:43:31 GMT
<
* Connection #0 to host SWIFT_IP left intact

Also, I can see that the containers are created on swift, so obviously it is able to authenticate.
$ openstack container list
+---------------+
| Name |
+---------------+
| cc-buildpacks |
| cc-droplets |
| cc-packages |
| cc-resources |
+---------------+

I would appreciate if someone can help me fixing this issue.

Regards,
Muhammad Altaf
Software Development Engineer

Fujitsu Australia Software Technology Pty Ltd
14 Rodborough Road, Frenchs Forest NSW 2086, Australia
T +61 2 9452 9067<tel:%2B61%202%209452%209067>F +61 2 9975 2899<tel:%2B61%202%209975%202899>
Muhammada(a)fast.au.fujitsu.com<mailto:Muhammada(a)fast.au.fujitsu.com>
fastware.com.au<http://fastware.com.au>

Disclaimer

The information in this e-mail is confidential and may contain content that is subject to copyright and/or is commercial-in-confidence and is intended only for the use of the above named addressee. If you are not the intended recipient, you are hereby notified that dissemination, copying or use of the information is strictly prohibited. If you have received this e-mail in error, please telephone Fujitsu Australia Software Technology Pty Ltd on + 61 2 9452 9000<tel:%2B%2061%202%209452%209000> or by reply e-mail to the sender and delete the document and all copies thereof.


Whereas Fujitsu Australia Software Technology Pty Ltd would not knowingly transmit a virus within an email communication, it is the receiver’s responsibility to scan all communication and any files attached for computer viruses and other defects. Fujitsu Australia Software Technology Pty Ltd does not accept liability for any loss or damage (whether direct, indirect, consequential or economic) however caused, and whether by negligence or otherwise, which may result directly or indirectly from this communication or any files attached.


If you do not wish to receive commercial and/or marketing email messages from Fujitsu Australia Software Technology Pty Ltd, please email unsubscribe(a)fast.au.fujitsu.com<mailto:unsubscribe(a)fast.au.fujitsu.com>



--
Nicholas Calugar
CAPI Product Manager
Pivotal Software, Inc.


App running even after delete. Pointers on finding it and debugging?

Tom Sherrod <tom.sherrod@...>
 

cf 230, diego 0.1450.0, etcd 27, garden-linux 0.330.0
Default to diego true.

Developer deployed a java application. Deleted the application: cf delete <app> No errors.
The app still responds. The only thing left is the route.
I've not encountered this before. Delete has been delete and even if route remains, 404 Not Found: Requested route ('<hostname.domain>') does not exist. is returned.

Pointers on tracking this down appreciated.

Tom


Re: cf v231: Issue with new webdav blobstore job

Rich Wohlstadter
 

Excellent. Thanks for your investigation on this.

-Rich


Re: Using swift as a blobstore in cloud foundry with keystone v3

Koper, Dies <diesk@...>
 

Hi Nick,

Can you try with a domain other than “default”?
We initially found that with the BOSH CPI, Keystone V3 worked for the “default” domain, but not for any other domain.
Note that the point of using Keystone V3 is to have different domains.

Also, we can connect to swift using the temp urls with the openstack cli fine.

Cheers,
Dies Koper


From: Nicholas Calugar [mailto:ncalugar(a)pivotal.io]
Sent: Thursday, March 31, 2016 11:18 AM
To: Discussions about Cloud Foundry projects and the system overall.
Subject: [cf-dev] Re: Re: Re: Re: Using swift as a blobstore in cloud foundry with keystone v3

Before confirming the new version of fog fixed this, I tried to reproduce the error using CF v233. Instead of reproducing the error, it actually worked.

V2 CONFIG:
provider: 'OpenStack'
openstack_tenant: 'admin'
openstack_username: 'admin'
openstack_api_key: '*************'
openstack_auth_url: 'https://my-openstack:5000/v2.0/tokens'
openstack_temp_url_key: 'b3968d0207b54ece87cccc06515a89d4'
connection_options:
ssl_verify_peer: false

V3 CONFIG:
provider: 'OpenStack'
openstack_tenant: 'admin'
openstack_project_name: 'admin'
openstack_domain_name: 'default'
openstack_username: 'admin'
openstack_api_key: '**************'
openstack_auth_url: 'https://my-openstack:5000/v3/auth/tokens'
openstack_temp_url_key: 'b3968d0207b54ece87cccc06515a89d4'
connection_options:
ssl_verify_peer: false

I would recommend using the openstack / swift cli to ensure you can correctly generate temp urls. The DEAs / CELLs use the tempurl feature to download objects from the blobstore and I think that is the error you are running into. Once you can get it to work via CLI, translate the correct variables into the fog_configuration for Cloud Foundry.


Thanks,

Nick

On Mon, Mar 21, 2016 at 10:21 AM Nicholas Calugar <ncalugar(a)pivotal.io<mailto:ncalugar(a)pivotal.io>> wrote:
We have a story in flight: https://www.pivotaltracker.com/story/show/115793253

If upgrading fog doesn't work, we'll happily take a PR that resolves this.

On Thu, Mar 17, 2016 at 5:33 PM Gwenn Etourneau <getourneau(a)pivotal.io<mailto:getourneau(a)pivotal.io>> wrote:
This can be nice ! As Bosh support V3 it can be nice that the CC do the same.
To be able to fully use v3.



On Thu, Mar 17, 2016 at 6:30 PM, Voelz, Marco <marco.voelz(a)sap.com<mailto:marco.voelz(a)sap.com>> wrote:
Dear Nicholas,

if desired, we can also do a PR for allowing the CC to connect to Swift using Keystone v3. A couple of months ago we did the same thing in the OpenStack CPI. We also have some test envs available where we can validate the change. What do you think?

Warm regards
Marco

On 16/03/16 18:13, "Nicholas Calugar" <ncalugar(a)pivotal.io<mailto:ncalugar(a)pivotal.io>> wrote:

Hi Muhammad,

Unfortunately, we don't have an environment using keystone v3. We are passing the configuration to fog as-is. There are several fixes that have been made in later releases of fog, for example:

https://github.com/fog/fog/pull/3806

I'll get a story prioritized to upgrade fog to v1.37.0

Thanks,

Nick

On Sun, Mar 13, 2016 at 11:58 PM Altaf, Muhammad <Muhammada(a)fast.au.fujitsu.com<mailto:Muhammada(a)fast.au.fujitsu.com>> wrote:
Hi All,
I am trying to configure cloud foundry to use swift on OpenStack. I have followed the instructions at https://docs.cloudfoundry.org/deploying/openstack/using_swift_blobstore.html
When used keystone v2, I am able to start my apps on DEA which is good. However when using keystone V3, I am not able to start my apps. The error I am getting is:

“FAILED
Server error, status code: 400, error code: 170001, message: Staging error: failed to stage application:
Error downloading: HTTP status: 401”

Tried to debug by adding some ‘puts’ statements in openstack/core.rb file and it looks like tokens are being generated successfully so there is no problem with the authentication. The generated response to auth request shows that the user has “ResellerAdmin” role as well.

When I look into runner_z1/0 /var/vcap/data/dea_next/tmp/ app-package-download.tgz2016*, I find error saying: “401 Unauthorized: Temp URL invalid xxxxx”

/var/vcap/sys/log/dea_next/dea_next.log shows some download URLs, and if I curl those URLs, I get exact same error message. Below are the fog_connection settings in cloud foundry manifest:

fog_connection: &fog_connection
provider: 'OpenStack'
openstack_username: 'cf-admin2'
openstack_tenant: 'cf2'
openstack_project_name: 'cf2'
openstack_api_key: 'passw0rd'
openstack_auth_url: 'http://<OPENSTACK_IP>:5000/v3/auth/tokens'
openstack_domain_name: 'cf_domain'
openstack_user_domain_name: 'cf_domain'
openstack_temp_url_key: 'b3968d0207b54ece87cccc06515a89d4'


Account has a valid temp_url_key configured. Please see below:
curl -v -X GET http://SWIFT_IP:SWIFT_PORT/v2/Auth_b34a51e551ec4796a461168c886c734f -H "X-Auth-Token: TOKEN"
* Hostname was NOT found in DNS cache
* Trying SWIFT_IP...
* Connected to SWIFT_IP (SWIFT_IP) port SWIFT_PORT (#0)
GET /v2/Auth_b34a51e551ec4796a461168c886c734f HTTP/1.1
User-Agent: curl/7.35.0
Host: SWIFT_IP:SWIFT_PORT
Accept: */*
X-Auth-Token: TOKEN
< HTTP/1.1 204 No Content
< Content-Length: 0
< X-Account-Object-Count: 0
< X-Timestamp: 1457918518.21777
< X-Account-Meta-Temp-Url-Key: b3968d0207b54ece87cccc06515a89d4
< X-Account-Bytes-Used: 0
< X-Account-Container-Count: 0
< Content-Type: text/plain; charset=utf-8
< Accept-Ranges: bytes
< X-Trans-Id: txfc362c27bdda4355a942a-0056e65d93
< Date: Mon, 14 Mar 2016 06:43:31 GMT
<
* Connection #0 to host SWIFT_IP left intact

Also, I can see that the containers are created on swift, so obviously it is able to authenticate.
$ openstack container list
+---------------+
| Name |
+---------------+
| cc-buildpacks |
| cc-droplets |
| cc-packages |
| cc-resources |
+---------------+

I would appreciate if someone can help me fixing this issue.

Regards,
Muhammad Altaf
Software Development Engineer

Fujitsu Australia Software Technology Pty Ltd
14 Rodborough Road, Frenchs Forest NSW 2086, Australia
T +61 2 9452 9067<tel:%2B61%202%209452%209067> F +61 2 9975 2899<tel:%2B61%202%209975%202899>
Muhammada(a)fast.au.fujitsu.com<mailto:Muhammada(a)fast.au.fujitsu.com>
fastware.com.au<http://fastware.com.au>

Disclaimer

The information in this e-mail is confidential and may contain content that is subject to copyright and/or is commercial-in-confidence and is intended only for the use of the above named addressee. If you are not the intended recipient, you are hereby notified that dissemination, copying or use of the information is strictly prohibited. If you have received this e-mail in error, please telephone Fujitsu Australia Software Technology Pty Ltd on + 61 2 9452 9000<tel:%2B%2061%202%209452%209000> or by reply e-mail to the sender and delete the document and all copies thereof.


Whereas Fujitsu Australia Software Technology Pty Ltd would not knowingly transmit a virus within an email communication, it is the receiver’s responsibility to scan all communication and any files attached for computer viruses and other defects. Fujitsu Australia Software Technology Pty Ltd does not accept liability for any loss or damage (whether direct, indirect, consequential or economic) however caused, and whether by negligence or otherwise, which may result directly or indirectly from this communication or any files attached.


If you do not wish to receive commercial and/or marketing email messages from Fujitsu Australia Software Technology Pty Ltd, please email unsubscribe(a)fast.au.fujitsu.com<mailto:unsubscribe(a)fast.au.fujitsu.com>



--
Nicholas Calugar
CAPI Product Manager
Pivotal Software, Inc.


Re: Intended UAA-specific user identity fields in JWT access token ?

Guillaume Berche
 

Great, thanks Filip!


Guillaume.

On Thu, Mar 31, 2016 at 9:50 PM, Filip Hanik <fhanik(a)pivotal.io> wrote:

yes, they are always returned.

introducing an option sounds like a good idea for the systems that wish to
turn it off, thanks for the idea.

https://www.pivotaltracker.com/story/show/116726159

Filip


On Thu, Mar 31, 2016 at 1:39 PM, Guillaume Berche <bercheg(a)gmail.com>
wrote:

Thanks Filip for your answer. Wouldn't it make sense to progressively
change this behavior, possibly controlled by a configuration option to give
clients time to handle this incompatible change?

Scanning quickly through the code I suspect the username and email fields
are systematically returned in the access token, regardless of the presence
of the openid scope (I still have to double check by actually testing it),
therefore disclosing some user identity without his/her consent.

Guillaume.

On Thu, Mar 31, 2016 at 3:03 PM, Filip Hanik <fhanik(a)pivotal.io> wrote:

The access token used to double down as an identity token before OpenID
Connect was standardized, now that we have implemented id_token, we don't
really need it. but removing it would cause an backwards incompatible
change.


Filip


On Thu, Mar 31, 2016 at 6:50 AM, Guillaume Berche <bercheg(a)gmail.com>
wrote:

Hi,

I wonder the rationale for apparenty uaa-specific [0] user-related
fields in the access token (username, email [1]) while they are now
returned in a standard maneer in the openidconnect id token.

Is it something that would change in the future ([2] seemed similar
decoupling) ? Or is it a standard practice that avoids clients to request
the idtoken to get access to basic user identity ?

Thanks in advance,

Guillaume.

ps: please let me know if such questions are better suited for a GH
issue on the UAA repo.

[0]
https://github.com/cloudfoundry/uaa/blob/master/docs/UAA-Tokens.md#getting-started

Some of these fields are described in the JSON web tokens
specification. However, the vendor may add additional fields, or
attributes, to the token itself.

[1]
https://github.com/cloudfoundry/uaa/blob/9b5c13d793ebfe358e26559cedc6b528a557b43f/server/src/main/java/org/cloudfoundry/identity/uaa/oauth/UaaTokenServices.java#L493-L497
[2] https://www.pivotaltracker.com/story/show/102090344

Guillaume.


Re: Issues syncing blobs and creating releases for several releases

Claire Laurence
 

Hi all,

The blobs in the S3 bucket appear to be back up and running. If you
believe something is still missing please let us know and we will file a
support ticket with AWS on your behalf.

Best,
Claire

On Thu, Mar 31, 2016 at 2:18 PM, Amit Gupta <agupta(a)pivotal.io> wrote:

Hi all,

Several releases, including cf-release and diego-release, use an AWS
account belonging to the Cloud Foundry Foundation to store final release
blobs. This account was accidentally closed, causing any attempt to "bosh
sync blobs" or "bosh create release" for some of these releases to fail
with blobstore errors. The admin team has been in contact with AWS support
and the account is being reinstated. All resources (including the S3
buckets for final release blobs) should be recovered within half an hour.

Please feel free to reach out if you have any questions.

Cheers,
Amit
--
Claire Laurence
Pivotal Cloud Foundry San Francisco
Administrative Assistant


Issues syncing blobs and creating releases for several releases

Amit Kumar Gupta
 

Hi all,

Several releases, including cf-release and diego-release, use an AWS
account belonging to the Cloud Foundry Foundation to store final release
blobs. This account was accidentally closed, causing any attempt to "bosh
sync blobs" or "bosh create release" for some of these releases to fail
with blobstore errors. The admin team has been in contact with AWS support
and the account is being reinstated. All resources (including the S3
buckets for final release blobs) should be recovered within half an hour.

Please feel free to reach out if you have any questions.

Cheers,
Amit


Re: Reg the serial property

CF Runtime
 

If the question can be paraphrased as "when colocating multiple job (templates) on a single VM, will they start in parallel or instead in some sort of order?" then the jobs will generally come online in parallel unless one job has a monit-level dependency on another, in which case monit will start them in order of the dependencies.

Given you are colocating jobs which are usually on multiple VMs and hence the ordering would be handled by bosh jobs (bosh-level dependencies), these job templates will not have monit-level dependencies configured and so you will likely see most jobs starting together. This may or may not work; we haven't tried starting all jobs in parallel.

The serial: true configuration in the bosh manifest applies at the bosh-level to multiple VMs, but as you only have a single VM it has no effect.

Thanks,
Rob Dimsdale
CF Release Integration
Pivotal


Re: Intended UAA-specific user identity fields in JWT access token ?

Filip Hanik
 

yes, they are always returned.

introducing an option sounds like a good idea for the systems that wish to
turn it off, thanks for the idea.

https://www.pivotaltracker.com/story/show/116726159

Filip

On Thu, Mar 31, 2016 at 1:39 PM, Guillaume Berche <bercheg(a)gmail.com> wrote:

Thanks Filip for your answer. Wouldn't it make sense to progressively
change this behavior, possibly controlled by a configuration option to give
clients time to handle this incompatible change?

Scanning quickly through the code I suspect the username and email fields
are systematically returned in the access token, regardless of the presence
of the openid scope (I still have to double check by actually testing it),
therefore disclosing some user identity without his/her consent.

Guillaume.

On Thu, Mar 31, 2016 at 3:03 PM, Filip Hanik <fhanik(a)pivotal.io> wrote:

The access token used to double down as an identity token before OpenID
Connect was standardized, now that we have implemented id_token, we don't
really need it. but removing it would cause an backwards incompatible
change.


Filip


On Thu, Mar 31, 2016 at 6:50 AM, Guillaume Berche <bercheg(a)gmail.com>
wrote:

Hi,

I wonder the rationale for apparenty uaa-specific [0] user-related
fields in the access token (username, email [1]) while they are now
returned in a standard maneer in the openidconnect id token.

Is it something that would change in the future ([2] seemed similar
decoupling) ? Or is it a standard practice that avoids clients to request
the idtoken to get access to basic user identity ?

Thanks in advance,

Guillaume.

ps: please let me know if such questions are better suited for a GH
issue on the UAA repo.

[0]
https://github.com/cloudfoundry/uaa/blob/master/docs/UAA-Tokens.md#getting-started

Some of these fields are described in the JSON web tokens
specification. However, the vendor may add additional fields, or
attributes, to the token itself.

[1]
https://github.com/cloudfoundry/uaa/blob/9b5c13d793ebfe358e26559cedc6b528a557b43f/server/src/main/java/org/cloudfoundry/identity/uaa/oauth/UaaTokenServices.java#L493-L497
[2] https://www.pivotaltracker.com/story/show/102090344

Guillaume.


Re: Intended UAA-specific user identity fields in JWT access token ?

Guillaume Berche
 

Thanks Filip for your answer. Wouldn't it make sense to progressively
change this behavior, possibly controlled by a configuration option to give
clients time to handle this incompatible change?

Scanning quickly through the code I suspect the username and email fields
are systematically returned in the access token, regardless of the presence
of the openid scope (I still have to double check by actually testing it),
therefore disclosing some user identity without his/her consent.

Guillaume.

On Thu, Mar 31, 2016 at 3:03 PM, Filip Hanik <fhanik(a)pivotal.io> wrote:

The access token used to double down as an identity token before OpenID
Connect was standardized, now that we have implemented id_token, we don't
really need it. but removing it would cause an backwards incompatible
change.


Filip


On Thu, Mar 31, 2016 at 6:50 AM, Guillaume Berche <bercheg(a)gmail.com>
wrote:

Hi,

I wonder the rationale for apparenty uaa-specific [0] user-related fields
in the access token (username, email [1]) while they are now returned in a
standard maneer in the openidconnect id token.

Is it something that would change in the future ([2] seemed similar
decoupling) ? Or is it a standard practice that avoids clients to request
the idtoken to get access to basic user identity ?

Thanks in advance,

Guillaume.

ps: please let me know if such questions are better suited for a GH issue
on the UAA repo.

[0]
https://github.com/cloudfoundry/uaa/blob/master/docs/UAA-Tokens.md#getting-started

Some of these fields are described in the JSON web tokens
specification. However, the vendor may add additional fields, or
attributes, to the token itself.

[1]
https://github.com/cloudfoundry/uaa/blob/9b5c13d793ebfe358e26559cedc6b528a557b43f/server/src/main/java/org/cloudfoundry/identity/uaa/oauth/UaaTokenServices.java#L493-L497
[2] https://www.pivotaltracker.com/story/show/102090344

Guillaume.


Re: Cloud Controller System Domain vs App Domains

john mcteague <john.mcteague@...>
 

For option 2 would it not be simpler to have a single property such as
cc.blacklisted_system_domain_routes that contains the desired list and have
the CC deny route requests for those routes?

I don't see what storing them in the DB or creating real routes actually
buys us here. Everything would be available via config.

I'm in favour of some form of option 2 for those of us who have existing
deployments but would rather not make a change in either app or system
domains but are exposed to this issue.

John.

On 31 Mar 2016 5:47 a.m., "Nicholas Calugar" <ncalugar(a)pivotal.io> wrote:

Hi Cloud Foundry,

We've had a recurring issue brought to our attention regarding a Cloud
Foundry deployment using a system_domain that is in the list of
app_domains. When the system domain is in the list of app domains, a Shared
Domain is created for the system domain. This is problematic because it
allows users to create routes on the system domain, see this [1] recent
issue as an example.

[1] https://github.com/cloudfoundry/cloud_controller_ng/issues/568

I'd like to propose two solutions and get some feedback regarding which
the community would prefer. Please respond with your preferred solution and
a brief reason why.

*Solution 1 - Require a Unique System Domain*

Instead of recommending a unique system domain, we would enforce this in
the Cloud Controller. The proposed change is as follows:

1. REQUIRE system_domain to NOT be in the list of app_domains
2. REQUIRE a system_domain_organization

This will create a Private Domain for the system domain. Failure to
configure correctly would not allow the Cloud Controller to start.

If we decide to implement this, an operator should ensure their deployment
uses a unique system domain and a system_domain_organization and correct
DNS entries before upgrading.

Example for BOSH-lite

- app_domains: [bosh-lite.com]
- system_domain: system.bosh-lite.com
- system_domain_organization: system


- api endpoint: api.system.bosh-lite.com
- sample app endpoint: dora.bosh-lite.com

Example for a PaaS:

- app_domains: [yuge-paas-apps.io]
- system_domain: yuge-pass.com
- system_domain_organization: yuge-system-org


- api endpoint: api.yuge-paas.com
- sample app endpoint: dora.yuge-paas-apps.io

*Pro: *Cloud Controller now enforces what was previously the recommended
configuration for separate system and apps domains.
Con: Second SSL cert for the system domain and possibly a second DNS
record if system domain is not covered by the current wildcard record.

*Solution 2 - Cloud Controller Seeds System Routes*

To prevent a non-system app from requesting a hostname on a shared system
domain, the Cloud Controller will take a list of hostnames and seed the
database with routes. As routes are associated with a space, we will
require a system_organization and system_space. An operator could choose to
omit hostnames as desired.

cc.system_hostnames:
description: List of hostnames for which routes will be created on the
system domain.
default: [api,uaa,login,doppler,loggregator,hm9000]
cc.system_space:
description: Space where system routes will be created.
default: system

*Pro:* Significantly less change for operators running Cloud Foundry with
matching system and apps domains.
*Con: *Cloud Controller has knowledge of unrelated system components and
the list of defaults needs to be maintained as we add and remove components.


Thanks,

-Nick
--
Nicholas Calugar
CAPI Product Manager
Pivotal Software, Inc.


Re: Intended UAA-specific user identity fields in JWT access token ?

Filip Hanik
 

The access token used to double down as an identity token before OpenID
Connect was standardized, now that we have implemented id_token, we don't
really need it. but removing it would cause an backwards incompatible
change.


Filip

On Thu, Mar 31, 2016 at 6:50 AM, Guillaume Berche <bercheg(a)gmail.com> wrote:

Hi,

I wonder the rationale for apparenty uaa-specific [0] user-related fields
in the access token (username, email [1]) while they are now returned in a
standard maneer in the openidconnect id token.

Is it something that would change in the future ([2] seemed similar
decoupling) ? Or is it a standard practice that avoids clients to request
the idtoken to get access to basic user identity ?

Thanks in advance,

Guillaume.

ps: please let me know if such questions are better suited for a GH issue
on the UAA repo.

[0]
https://github.com/cloudfoundry/uaa/blob/master/docs/UAA-Tokens.md#getting-started

Some of these fields are described in the JSON web tokens specification.
However, the vendor may add additional fields, or attributes, to the token
itself.

[1]
https://github.com/cloudfoundry/uaa/blob/9b5c13d793ebfe358e26559cedc6b528a557b43f/server/src/main/java/org/cloudfoundry/identity/uaa/oauth/UaaTokenServices.java#L493-L497
[2] https://www.pivotaltracker.com/story/show/102090344

Guillaume.


Intended UAA-specific user identity fields in JWT access token ?

Guillaume Berche
 

Hi,

I wonder the rationale for apparenty uaa-specific [0] user-related fields
in the access token (username, email [1]) while they are now returned in a
standard maneer in the openidconnect id token.

Is it something that would change in the future ([2] seemed similar
decoupling) ? Or is it a standard practice that avoids clients to request
the idtoken to get access to basic user identity ?

Thanks in advance,

Guillaume.

ps: please let me know if such questions are better suited for a GH issue
on the UAA repo.

[0]
https://github.com/cloudfoundry/uaa/blob/master/docs/UAA-Tokens.md#getting-started

Some of these fields are described in the JSON web tokens specification.
However, the vendor may add additional fields, or attributes, to the token
itself.

[1]
https://github.com/cloudfoundry/uaa/blob/9b5c13d793ebfe358e26559cedc6b528a557b43f/server/src/main/java/org/cloudfoundry/identity/uaa/oauth/UaaTokenServices.java#L493-L497
[2] https://www.pivotaltracker.com/story/show/102090344

Guillaume.


Cloud Controller System Domain vs App Domains

Nicholas Calugar
 

Hi Cloud Foundry,

We've had a recurring issue brought to our attention regarding a Cloud
Foundry deployment using a system_domain that is in the list of
app_domains. When the system domain is in the list of app domains, a Shared
Domain is created for the system domain. This is problematic because it
allows users to create routes on the system domain, see this [1] recent
issue as an example.

[1] https://github.com/cloudfoundry/cloud_controller_ng/issues/568

I'd like to propose two solutions and get some feedback regarding which the
community would prefer. Please respond with your preferred solution and a
brief reason why.

*Solution 1 - Require a Unique System Domain*

Instead of recommending a unique system domain, we would enforce this in
the Cloud Controller. The proposed change is as follows:

1. REQUIRE system_domain to NOT be in the list of app_domains
2. REQUIRE a system_domain_organization

This will create a Private Domain for the system domain. Failure to
configure correctly would not allow the Cloud Controller to start.

If we decide to implement this, an operator should ensure their deployment
uses a unique system domain and a system_domain_organization and correct
DNS entries before upgrading.

Example for BOSH-lite

- app_domains: [bosh-lite.com]
- system_domain: system.bosh-lite.com
- system_domain_organization: system


- api endpoint: api.system.bosh-lite.com
- sample app endpoint: dora.bosh-lite.com

Example for a PaaS:

- app_domains: [yuge-paas-apps.io]
- system_domain: yuge-pass.com
- system_domain_organization: yuge-system-org


- api endpoint: api.yuge-paas.com
- sample app endpoint: dora.yuge-paas-apps.io

*Pro: *Cloud Controller now enforces what was previously the recommended
configuration for separate system and apps domains.
Con: Second SSL cert for the system domain and possibly a second DNS record
if system domain is not covered by the current wildcard record.

*Solution 2 - Cloud Controller Seeds System Routes*

To prevent a non-system app from requesting a hostname on a shared system
domain, the Cloud Controller will take a list of hostnames and seed the
database with routes. As routes are associated with a space, we will
require a system_organization and system_space. An operator could choose to
omit hostnames as desired.

cc.system_hostnames:
description: List of hostnames for which routes will be created on the
system domain.
default: [api,uaa,login,doppler,loggregator,hm9000]
cc.system_space:
description: Space where system routes will be created.
default: system

*Pro:* Significantly less change for operators running Cloud Foundry with
matching system and apps domains.
*Con: *Cloud Controller has knowledge of unrelated system components and
the list of defaults needs to be maintained as we add and remove components.


Thanks,

-Nick
--
Nicholas Calugar
CAPI Product Manager
Pivotal Software, Inc.


Re: Question regarding "cf services" and "cf service"

Nicholas Calugar
 

Hi Nils,

I cannot seem to reproduce this. I am logged in as an org-manager:

$ cf services
Getting services in org ncalugar-a1 / space ncalugar-a1 as
ncalugar+a1-org-manager(a)pivotal.io...
OK

name service plan bound apps last operation
nick-mysql p-mysql 100mb create succeeded
nick-papertrail user-provided

$ cf service nick-mysql

Service instance: nick-mysql
Service: p-mysql
Tags:
Plan: 100mb
Description: MySQL databases on demand
Documentation url:
https://github.com/cloudfoundry/cf-mysql-release/blob/master/README.md
Dashboard:

Last Operation
Status: create succeeded
Message:
Started: 2016-03-31T00:54:03Z
Updated:

$ cf service nick-papertrail

Service instance: nick-papertrail
Service: user-provided


Thanks,

Nick

On Wed, Mar 30, 2016 at 4:41 AM Nils Eckert <Nils.Eckert(a)de.ibm.com> wrote:

Hi,

There seems to be a difference between how "cf services" and "cf service"
are working in regard of permissions and I would like to understand why.

As organization manager but without a role in a space, the "cf services"
command shows me all service instances that are available within the space.
However, when using "cf service" to show the service instance info for
such a service, I receive an error message telling me that the service
instance was not found.

EXAMPLE

cf services
Getting services in org Multi Region Team / space dev as
nils.eckert(a)de.ibm.com...
OK

name service plan bound apps last operation
logstash-drain user-provided

cf service logstash-drain
FAILED
Service instance logstash-drain not found


Mit freundlichen Grüßen / Kind regards

*Nils Eckert*
[image: 09842089.gif]
Software Engineer, Bluemix Development
IBM Cloud Platform Services

*IBM Deutschland Research & Development GmbH*
Schoenaicher Strasse 220
D-71032 Boeblingen

Phone:+49-7031-164297
eMail: *nils.eckert(a)de.ibm.com* <nils.eckert(a)de.ibm.com>

[image: 09558563.gif]
IBM Deutschland Research & Development GmbH / Vorsitzende des
Aufsichtsrats: Martina Koederitz
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart,
HRB 243294


Re: Failed to deploy diego 0.1452.0 on openstack: database_z2/0 is not running after update

Yunata, Ricky <rickyy@...>
 

Hi Ryan,

Thanks for your e-mail. I'm following the instructions from https://github.com/cloudfoundry-incubator/diego-release
This is how I generate my certificates

CA certificate
$ certstrap init --common-name "diegoCA"

Etcd server certificate
$ certstrap request-cert --common-name "etcd.service.cf.internal" --domain "*.etcd.service.cf.internal,etcd.service.cf.internal"
$ certstrap sign etcd.service.cf.internal --CA diegoCA

Etcd client certificate
$ certstrap request-cert --common-name "client"
$ certstrap sign client --CA diegoCA

Ricky Yunata


Please consider the environment before printing this email

-----Original Message-----
From: Ryan Moran [mailto:rmoran(a)pivotal.io]
Sent: Thursday, 31 March 2016 9:01 AM
To: cf-dev(a)lists.cloudfoundry.org
Subject: [cf-dev] Re: Re: Re: Re: Re: Re: Failed to deploy diego 0.1452.0 on openstack: database_z2/0 is not running after update

Hi Ricky,

We think there might be an issue with the TLS certs that you provide in the etcd properties of the manifest. Can you tell us how you generated these certs? We also discovered an issue with the current behavior of the etcd_ctl start script. We will fix the bug, but we want to also help you get a working cluster.

Thanks,
Ryan
Disclaimer

The information in this e-mail is confidential and may contain content that is subject to copyright and/or is commercial-in-confidence and is intended only for the use of the above named addressee. If you are not the intended recipient, you are hereby notified that dissemination, copying or use of the information is strictly prohibited. If you have received this e-mail in error, please telephone Fujitsu Australia Software Technology Pty Ltd on + 61 2 9452 9000 or by reply e-mail to the sender and delete the document and all copies thereof.


Whereas Fujitsu Australia Software Technology Pty Ltd would not knowingly transmit a virus within an email communication, it is the receiver’s responsibility to scan all communication and any files attached for computer viruses and other defects. Fujitsu Australia Software Technology Pty Ltd does not accept liability for any loss or damage (whether direct, indirect, consequential or economic) however caused, and whether by negligence or otherwise, which may result directly or indirectly from this communication or any files attached.


If you do not wish to receive commercial and/or marketing email messages from Fujitsu Australia Software Technology Pty Ltd, please email unsubscribe(a)fast.au.fujitsu.com


Re: Using swift as a blobstore in cloud foundry with keystone v3

Nicholas Calugar
 

Before confirming the new version of fog fixed this, I tried to reproduce
the error using CF v233. Instead of reproducing the error, it actually
worked.

V2 CONFIG:
provider: 'OpenStack'
openstack_tenant: 'admin'
openstack_username: 'admin'
openstack_api_key: '*************'
openstack_auth_url: 'https://my-openstack:5000/v2.0/tokens'
openstack_temp_url_key: 'b3968d0207b54ece87cccc06515a89d4'
connection_options:
ssl_verify_peer: false

V3 CONFIG:
provider: 'OpenStack'
openstack_tenant: 'admin'
openstack_project_name: 'admin'
openstack_domain_name: 'default'
openstack_username: 'admin'
openstack_api_key: '**************'
openstack_auth_url: 'https://my-openstack:5000/v3/auth/tokens'
openstack_temp_url_key: 'b3968d0207b54ece87cccc06515a89d4'
connection_options:
ssl_verify_peer: false

I would recommend using the openstack / swift cli to ensure you can
correctly generate temp urls. The DEAs / CELLs use the tempurl feature to
download objects from the blobstore and I think that is the error you are
running into. Once you can get it to work via CLI, translate the correct
variables into the fog_configuration for Cloud Foundry.


Thanks,

Nick

On Mon, Mar 21, 2016 at 10:21 AM Nicholas Calugar <ncalugar(a)pivotal.io>
wrote:

We have a story in flight:
https://www.pivotaltracker.com/story/show/115793253

If upgrading fog doesn't work, we'll happily take a PR that resolves this.

On Thu, Mar 17, 2016 at 5:33 PM Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

This can be nice ! As Bosh support V3 it can be nice that the CC do the
same.
To be able to fully use v3.



On Thu, Mar 17, 2016 at 6:30 PM, Voelz, Marco <marco.voelz(a)sap.com>
wrote:

Dear Nicholas,

if desired, we can also do a PR for allowing the CC to connect to Swift
using Keystone v3. A couple of months ago we did the same thing in the
OpenStack CPI. We also have some test envs available where we can validate
the change. What do you think?

Warm regards
Marco

On 16/03/16 18:13, "Nicholas Calugar" <ncalugar(a)pivotal.io> wrote:

Hi Muhammad,

Unfortunately, we don't have an environment using keystone v3. We are
passing the configuration to fog as-is. There are several fixes that have
been made in later releases of fog, for example:

https://github.com/fog/fog/pull/3806

I'll get a story prioritized to upgrade fog to v1.37.0

Thanks,

Nick

On Sun, Mar 13, 2016 at 11:58 PM Altaf, Muhammad <
Muhammada(a)fast.au.fujitsu.com> wrote:

Hi All,

I am trying to configure cloud foundry to use swift on OpenStack. I
have followed the instructions at
https://docs.cloudfoundry.org/deploying/openstack/using_swift_blobstore.html

When used keystone v2, I am able to start my apps on DEA which is good.
However when using keystone V3, I am not able to start my apps. The error I
am getting is:



“FAILED

Server error, status code: 400, error code: 170001, message: Staging
error: failed to stage application:

Error downloading: HTTP status: 401”



Tried to debug by adding some ‘puts’ statements in openstack/core.rb
file and it looks like tokens are being generated successfully so there is
no problem with the authentication. The generated response to auth request
shows that the user has “ResellerAdmin” role as well.



When I look into runner_z1/0 /var/vcap/data/dea_next/tmp/
app-package-download.tgz2016*, I find error saying: “401 Unauthorized:
Temp URL invalid xxxxx”



/var/vcap/sys/log/dea_next/dea_next.log shows some download URLs, and
if I curl those URLs, I get exact same error message. Below are the
fog_connection settings in cloud foundry manifest:



fog_connection: &fog_connection

provider: 'OpenStack'

openstack_username: 'cf-admin2'

openstack_tenant: 'cf2'

openstack_project_name: 'cf2'

openstack_api_key: 'passw0rd'

openstack_auth_url: 'http://<OPENSTACK_IP>:5000/v3/auth/tokens'

openstack_domain_name: 'cf_domain'

openstack_user_domain_name: 'cf_domain'

openstack_temp_url_key: 'b3968d0207b54ece87cccc06515a89d4'





Account has a valid temp_url_key configured. Please see below:

curl -v -X GET
http://SWIFT_IP:SWIFT_PORT/v2/Auth_b34a51e551ec4796a461168c886c734f -H
"X-Auth-Token: TOKEN"

* Hostname was NOT found in DNS cache

* Trying SWIFT_IP...

* Connected to SWIFT_IP (SWIFT_IP) port SWIFT_PORT (#0)

GET /v2/Auth_b34a51e551ec4796a461168c886c734f HTTP/1.1
User-Agent: curl/7.35.0
Host: SWIFT_IP:SWIFT_PORT
Accept: */*
X-Auth-Token: TOKEN
< HTTP/1.1 204 No Content

< Content-Length: 0

< X-Account-Object-Count: 0

< X-Timestamp: 1457918518.21777

< X-Account-Meta-Temp-Url-Key: *b3968d0207b54ece87cccc06515a89d4*

< X-Account-Bytes-Used: 0

< X-Account-Container-Count: 0

< Content-Type: text/plain; charset=utf-8

< Accept-Ranges: bytes

< X-Trans-Id: txfc362c27bdda4355a942a-0056e65d93

< Date: Mon, 14 Mar 2016 06:43:31 GMT

<

* Connection #0 to host SWIFT_IP left intact



Also, I can see that the containers are created on swift, so obviously
it is able to authenticate.

$ openstack container list

+---------------+

| Name |

+---------------+

| cc-buildpacks |

| cc-droplets |

| cc-packages |

| cc-resources |

+---------------+



I would appreciate if someone can help me fixing this issue.



Regards,




*Muhammad Altaf Software Development Engineer Fujitsu Australia
Software Technology Pty Ltd*
14 Rodborough Road, Frenchs Forest NSW 2086, Australia
*T* +61 2 9452 9067 *F* +61 2 9975 2899
Muhammada(a)fast.au.fujitsu.com
fastware.com.au

[image: image001.jpg]
[image: image002.jpg]

Disclaimer

The information in this e-mail is confidential and may contain content
that is subject to copyright and/or is commercial-in-confidence and is
intended only for the use of the above named addressee. If you are not the
intended recipient, you are hereby notified that dissemination, copying or
use of the information is strictly prohibited. If you have received this
e-mail in error, please telephone Fujitsu Australia Software Technology Pty
Ltd on + 61 2 9452 9000 or by reply e-mail to the sender and delete
the document and all copies thereof.

Whereas Fujitsu Australia Software Technology Pty Ltd would not
knowingly transmit a virus within an email communication, it is the
receiver’s responsibility to scan all communication and any files attached
for computer viruses and other defects. Fujitsu Australia Software
Technology Pty Ltd does not accept liability for any loss or damage
(whether direct, indirect, consequential or economic) however caused, and
whether by negligence or otherwise, which may result directly or indirectly
from this communication or any files attached.

If you do not wish to receive commercial and/or marketing email
messages from Fujitsu Australia Software Technology Pty Ltd, please email
unsubscribe(a)fast.au.fujitsu.com

--
Nicholas Calugar
CAPI Product Manager
Pivotal Software, Inc.


Re: Failed to deploy diego 0.1452.0 on openstack: database_z2/0 is not running after update

Ryan Moran
 

Hi Ricky,

We think there might be an issue with the TLS certs that you provide in the etcd properties of the manifest. Can you tell us how you generated these certs? We also discovered an issue with the current behavior of the etcd_ctl start script. We will fix the bug, but we want to also help you get a working cluster.

Thanks,
Ryan


Re: cf v231: Issue with new webdav blobstore job

Nicholas Calugar
 

Hi Rich,

We investigated and determined the recursive chown was not necessary. The
commit is still working it's way through CI, but will hopefully make it in
the next CF-Release:

https://www.pivotaltracker.com/story/show/116176983

Thanks,

Nick

On Wed, Mar 23, 2016 at 6:02 AM Rich Wohlstadter <lethwin(a)gmail.com> wrote:

That would probably work. I'm wondering if the recursive is even
neccessary. The nfs service didnt do that. Why would the webdav
replacement? If it does indeed need to be done, then I'm thinking it
should be done outside the monit startup or in a way that does not delay
webdav from starting otherwise its going to be a continuing issue depending
on customer blobstore sizes.

-Rich
--
Nicholas Calugar
CAPI Product Manager
Pivotal Software, Inc.

5001 - 5020 of 9426