Re: Using swift as a blobstore in cloud foundry with keystone v3


Koper, Dies <diesk@...>
 

Hi Nick,

Can you try with a domain other than “default”?
We initially found that with the BOSH CPI, Keystone V3 worked for the “default” domain, but not for any other domain.
Note that the point of using Keystone V3 is to have different domains.

Also, we can connect to swift using the temp urls with the openstack cli fine.

Cheers,
Dies Koper


From: Nicholas Calugar [mailto:ncalugar(a)pivotal.io]
Sent: Thursday, March 31, 2016 11:18 AM
To: Discussions about Cloud Foundry projects and the system overall.
Subject: [cf-dev] Re: Re: Re: Re: Using swift as a blobstore in cloud foundry with keystone v3

Before confirming the new version of fog fixed this, I tried to reproduce the error using CF v233. Instead of reproducing the error, it actually worked.

V2 CONFIG:
provider: 'OpenStack'
openstack_tenant: 'admin'
openstack_username: 'admin'
openstack_api_key: '*************'
openstack_auth_url: 'https://my-openstack:5000/v2.0/tokens'
openstack_temp_url_key: 'b3968d0207b54ece87cccc06515a89d4'
connection_options:
ssl_verify_peer: false

V3 CONFIG:
provider: 'OpenStack'
openstack_tenant: 'admin'
openstack_project_name: 'admin'
openstack_domain_name: 'default'
openstack_username: 'admin'
openstack_api_key: '**************'
openstack_auth_url: 'https://my-openstack:5000/v3/auth/tokens'
openstack_temp_url_key: 'b3968d0207b54ece87cccc06515a89d4'
connection_options:
ssl_verify_peer: false

I would recommend using the openstack / swift cli to ensure you can correctly generate temp urls. The DEAs / CELLs use the tempurl feature to download objects from the blobstore and I think that is the error you are running into. Once you can get it to work via CLI, translate the correct variables into the fog_configuration for Cloud Foundry.


Thanks,

Nick

On Mon, Mar 21, 2016 at 10:21 AM Nicholas Calugar <ncalugar(a)pivotal.io<mailto:ncalugar(a)pivotal.io>> wrote:
We have a story in flight: https://www.pivotaltracker.com/story/show/115793253

If upgrading fog doesn't work, we'll happily take a PR that resolves this.

On Thu, Mar 17, 2016 at 5:33 PM Gwenn Etourneau <getourneau(a)pivotal.io<mailto:getourneau(a)pivotal.io>> wrote:
This can be nice ! As Bosh support V3 it can be nice that the CC do the same.
To be able to fully use v3.



On Thu, Mar 17, 2016 at 6:30 PM, Voelz, Marco <marco.voelz(a)sap.com<mailto:marco.voelz(a)sap.com>> wrote:
Dear Nicholas,

if desired, we can also do a PR for allowing the CC to connect to Swift using Keystone v3. A couple of months ago we did the same thing in the OpenStack CPI. We also have some test envs available where we can validate the change. What do you think?

Warm regards
Marco

On 16/03/16 18:13, "Nicholas Calugar" <ncalugar(a)pivotal.io<mailto:ncalugar(a)pivotal.io>> wrote:

Hi Muhammad,

Unfortunately, we don't have an environment using keystone v3. We are passing the configuration to fog as-is. There are several fixes that have been made in later releases of fog, for example:

https://github.com/fog/fog/pull/3806

I'll get a story prioritized to upgrade fog to v1.37.0

Thanks,

Nick

On Sun, Mar 13, 2016 at 11:58 PM Altaf, Muhammad <Muhammada(a)fast.au.fujitsu.com<mailto:Muhammada(a)fast.au.fujitsu.com>> wrote:
Hi All,
I am trying to configure cloud foundry to use swift on OpenStack. I have followed the instructions at https://docs.cloudfoundry.org/deploying/openstack/using_swift_blobstore.html
When used keystone v2, I am able to start my apps on DEA which is good. However when using keystone V3, I am not able to start my apps. The error I am getting is:

“FAILED
Server error, status code: 400, error code: 170001, message: Staging error: failed to stage application:
Error downloading: HTTP status: 401”

Tried to debug by adding some ‘puts’ statements in openstack/core.rb file and it looks like tokens are being generated successfully so there is no problem with the authentication. The generated response to auth request shows that the user has “ResellerAdmin” role as well.

When I look into runner_z1/0 /var/vcap/data/dea_next/tmp/ app-package-download.tgz2016*, I find error saying: “401 Unauthorized: Temp URL invalid xxxxx”

/var/vcap/sys/log/dea_next/dea_next.log shows some download URLs, and if I curl those URLs, I get exact same error message. Below are the fog_connection settings in cloud foundry manifest:

fog_connection: &fog_connection
provider: 'OpenStack'
openstack_username: 'cf-admin2'
openstack_tenant: 'cf2'
openstack_project_name: 'cf2'
openstack_api_key: 'passw0rd'
openstack_auth_url: 'http://<OPENSTACK_IP>:5000/v3/auth/tokens'
openstack_domain_name: 'cf_domain'
openstack_user_domain_name: 'cf_domain'
openstack_temp_url_key: 'b3968d0207b54ece87cccc06515a89d4'


Account has a valid temp_url_key configured. Please see below:
curl -v -X GET http://SWIFT_IP:SWIFT_PORT/v2/Auth_b34a51e551ec4796a461168c886c734f -H "X-Auth-Token: TOKEN"
* Hostname was NOT found in DNS cache
* Trying SWIFT_IP...
* Connected to SWIFT_IP (SWIFT_IP) port SWIFT_PORT (#0)
GET /v2/Auth_b34a51e551ec4796a461168c886c734f HTTP/1.1
User-Agent: curl/7.35.0
Host: SWIFT_IP:SWIFT_PORT
Accept: */*
X-Auth-Token: TOKEN
< HTTP/1.1 204 No Content
< Content-Length: 0
< X-Account-Object-Count: 0
< X-Timestamp: 1457918518.21777
< X-Account-Meta-Temp-Url-Key: b3968d0207b54ece87cccc06515a89d4
< X-Account-Bytes-Used: 0
< X-Account-Container-Count: 0
< Content-Type: text/plain; charset=utf-8
< Accept-Ranges: bytes
< X-Trans-Id: txfc362c27bdda4355a942a-0056e65d93
< Date: Mon, 14 Mar 2016 06:43:31 GMT
<
* Connection #0 to host SWIFT_IP left intact

Also, I can see that the containers are created on swift, so obviously it is able to authenticate.
$ openstack container list
+---------------+
| Name |
+---------------+
| cc-buildpacks |
| cc-droplets |
| cc-packages |
| cc-resources |
+---------------+

I would appreciate if someone can help me fixing this issue.

Regards,
Muhammad Altaf
Software Development Engineer

Fujitsu Australia Software Technology Pty Ltd
14 Rodborough Road, Frenchs Forest NSW 2086, Australia
T +61 2 9452 9067<tel:%2B61%202%209452%209067> F +61 2 9975 2899<tel:%2B61%202%209975%202899>
Muhammada(a)fast.au.fujitsu.com<mailto:Muhammada(a)fast.au.fujitsu.com>
fastware.com.au<http://fastware.com.au>

Disclaimer

The information in this e-mail is confidential and may contain content that is subject to copyright and/or is commercial-in-confidence and is intended only for the use of the above named addressee. If you are not the intended recipient, you are hereby notified that dissemination, copying or use of the information is strictly prohibited. If you have received this e-mail in error, please telephone Fujitsu Australia Software Technology Pty Ltd on + 61 2 9452 9000<tel:%2B%2061%202%209452%209000> or by reply e-mail to the sender and delete the document and all copies thereof.


Whereas Fujitsu Australia Software Technology Pty Ltd would not knowingly transmit a virus within an email communication, it is the receiver’s responsibility to scan all communication and any files attached for computer viruses and other defects. Fujitsu Australia Software Technology Pty Ltd does not accept liability for any loss or damage (whether direct, indirect, consequential or economic) however caused, and whether by negligence or otherwise, which may result directly or indirectly from this communication or any files attached.


If you do not wish to receive commercial and/or marketing email messages from Fujitsu Australia Software Technology Pty Ltd, please email unsubscribe(a)fast.au.fujitsu.com<mailto:unsubscribe(a)fast.au.fujitsu.com>



--
Nicholas Calugar
CAPI Product Manager
Pivotal Software, Inc.

Join cf-dev@lists.cloudfoundry.org to automatically receive all group messages.