Hi Dmitriy,

Actually, after I generated the bosh release and bosh-openstack-cpi release
locally, I deployed them with bosh-init and failed with keystone v3.

I found this is caused by fog-1.27.0 code which is installed during
packaging. (attached detailed log below). Do you know how to fix it?
(Note: new bosh-openstack-cpi supporting keystone v3 has dependency on fog
1.31.0)

File:
1. bosh-openstack-cpi-release\packages\bosh_openstack_cpi\packaging

cat >> Gemfile <<EOF
gem 'bosh_openstack_cpi'
EOF

Here, it depends on fog-1.27.0 (
https://rubygems.org/gems/bosh_openstack_cpi/versions/1.2972.0) which
does not support keystone v3.

when it is executed during packaging, it will install fog-1.27.0. (I guess)

2.
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/gem_home/ruby/1.9.1/gems/fog-1.27.0/lib/fog/openstack/core.rb

if @openstack_auth_uri.path =~ /\/v2.0/
credentials = Fog::OpenStack.authenticate_v2(options,
@connection_options)
else
credentials = Fog::OpenStack.authenticate_v1(options,
@connection_options)
end



Command 'deploy' failed:
creating stemcell (bosh-openstack-kvm-ubuntu-trusty-go_agent 2950):
Unmarshalling external CPI command output: STDOUT: '', STDERR: 'D,
[2015-07-14T01:06:53.287951 #17191] DEBUG -- : excon.request
{:chunk_size=>1048576, :ciphers=>"HIGH:!SSLv2:!aNULL:!eNULL:!3DES",
:connect_timeout=>60, :debug_request=>false, :debug_response=>true,
:headers=>{"User-Agent"=>"fog/1.27.0 fog-core/1.32.0",
"X-Auth-Key"=>"passw0rd", "X-Auth-User"=>"demo", "Host"=>"
137.172.74.130:5000"}, :idempotent=>false, :instrumentor_name=>"excon",
:middlewares=>[Excon::Middleware::ResponseParser,
Excon::Middleware::Expects, Excon::Middleware::Idempotent,
Excon::Middleware::Instrumentor, Excon::Middleware::Mock], :mock=>false,
:nonblock=>true, :omit_default_port=>false, :persistent=>false,
:read_timeout=>60, :retry_limit=>4, :ssl_verify_peer=>true,
:tcp_nodelay=>false, :thread_safe_sockets=>true, :uri_parser=>URI,
:versions=>"excon/0.45.3 (x86_64-linux) ruby/1.9.3", :write_timeout=>60,
:host=>"137.172.74.130", :hostname=>"137.172.74.130", :path=>"/v3/tokens",
:port=>5000, :query=>nil, :scheme=>"http",
:instrumentor=>Bosh::OpenStackCloud::ExconLoggingInstrumentor,
:expects=>[200, 204], :method=>"GET", :retries_remaining=>4,
:connection=>#<Excon::Connection:30fe238 @data={:chunk_size=>1048576,
:ciphers=>"HIGH:!SSLv2:!aNULL:!eNULL:!3DES", :connect_timeout=>60,
:debug_request=>false, :debug_response=>true,
:headers=>{"User-Agent"=>"fog/1.27.0 fog-core/1.32.0"}, :idempotent=>false,
:instrumentor_name=>"excon",
:middlewares=>[Excon::Middleware::ResponseParser,
Excon::Middleware::Expects, Excon::Middleware::Idempotent,
Excon::Middleware::Instrumentor, Excon::Middleware::Mock], :mock=>false,
:nonblock=>true, :omit_default_port=>false, :persistent=>false,
:read_timeout=>60, :retry_limit=>4, :ssl_verify_peer=>true,
:tcp_nodelay=>false, :thread_safe_sockets=>true, :uri_parser=>URI,
:versions=>"excon/0.45.3 (x86_64-linux) ruby/1.9.3", :write_timeout=>60,
:host=>"137.172.74.130", :hostname=>"137.172.74.130", :path=>"/v3/tokens",
:port=>5000, :query=>nil, :scheme=>"http",
:instrumentor=>Bosh::OpenStackCloud::ExconLoggingInstrumentor} @socket_key="
http://137.172.74.130:5000">,
:stack=>#<Excon::Middleware::ResponseParser:0x000000030fbb00
@stack=#<Excon::Middleware::Expects:0x000000030fbb28
@stack=#<Excon::Middleware::Idempotent:0x000000030fbbc8
@stack=#<Excon::Middleware::Instrumentor:0x000000030fbbf0
@stack=#<Excon::Middleware::Mock:0x000000030fbe48
@stack=#<Excon::Connection:30fe238 @data={:chunk_size=>1048576,
:ciphers=>"HIGH:!SSLv2:!aNULL:!eNULL:!3DES", :connect_timeout=>60,
:debug_request=>false, :debug_response=>true,
:headers=>{"User-Agent"=>"fog/1.27.0 fog-core/1.32.0"}, :idempotent=>false,
:instrumentor_name=>"excon",
:middlewares=>[Excon::Middleware::ResponseParser,
Excon::Middleware::Expects, Excon::Middleware::Idempotent,
Excon::Middleware::Instrumentor, Excon::Middleware::Mock], :mock=>false,
:nonblock=>true, :omit_default_port=>false, :persistent=>false,
:read_timeout=>60, :retry_limit=>4, :ssl_verify_peer=>true,
:tcp_nodelay=>false, :thread_safe_sockets=>true, :uri_parser=>URI,
:versions=>"excon/0.45.3 (x86_64-linux) ruby/1.9.3", :write_timeout=>60,
:host=>"137.172.74.130", :hostname=>"137.172.74.130", :path=>"/v3/tokens",
:port=>5000, :query=>nil, :scheme=>"http",
:instrumentor=>Bosh::OpenStackCloud::ExconLoggingInstrumentor} @socket_key="
http://137.172.74.130:5000">>>>>>}
D, [2015-07-14T01:06:53.292721 #17191] DEBUG -- : excon.error
{:error=>#<Excon::Errors::NotFound: Expected([200, 204]) <=> Actual(404 Not
Found)
excon.error.response
:body => "{\"error\": {\"message\": \"The resource could not be
found.\", \"code\": 404, \"title\": \"Not Found\"}}"
:headers => {
"Connection" => "close"
"Content-Length" => "93"
"Content-Type" => "application/json"
"Date" => "Tue, 14 Jul 2015 01:06:53 GMT"
"Server" => "Apache/2.4.6 (CentOS)"
"Vary" => "X-Auth-Token"
}
:local_address => "172.16.0.20"
:local_port => 45399
:reason_phrase => "Not Found"
:remote_ip => "137.172.74.130"
:status => 404
:status_line => "HTTP/1.1 404 Not Found\r\n"

}

E, [2015-07-14T01:06:53.292820 #17191] ERROR -- : Expected([200, 204]) <=>
Actual(404 Not Found)
excon.error.response
:body => "{\"error\": {\"message\": \"The resource could not be
found.\", \"code\": 404, \"title\": \"Not Found\"}}"
:headers => {
"Connection" => "close"
"Content-Length" => "93"
"Content-Type" => "application/json"
"Date" => "Tue, 14 Jul 2015 01:06:53 GMT"
"Server" => "Apache/2.4.6 (CentOS)"
"Vary" => "X-Auth-Token"
}
:local_address => "172.16.0.20"
:local_port => 45399
:reason_phrase => "Not Found"
:remote_ip => "137.172.74.130"
:status => 404
:status_line => "HTTP/1.1 404 Not Found\r\n"
(Excon::Errors::NotFound)
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/gem_home/ruby/1.9.1/gems/excon-0.45.3/lib/excon/middlewares/expects.rb:10:in
`response_call'
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/gem_home/ruby/1.9.1/gems/excon-0.45.3/lib/excon/middlewares/response_parser.rb:8:in
`response_call'
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/gem_home/ruby/1.9.1/gems/excon-0.45.3/lib/excon/connection.rb:372:in
`response'
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/gem_home/ruby/1.9.1/gems/excon-0.45.3/lib/excon/connection.rb:236:in
`request'
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/gem_home/ruby/1.9.1/gems/fog-core-1.32.0/lib/fog/core/connection.rb:81:in
`request'
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/gem_home/ruby/1.9.1/gems/fog-1.27.0/lib/fog/openstack/core.rb:70:in
`authenticate_v1'
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/gem_home/ruby/1.9.1/gems/fog-1.27.0/lib/fog/openstack/compute.rb:397:in
`authenticate'
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/gem_home/ruby/1.9.1/gems/fog-1.27.0/lib/fog/openstack/compute.rb:320:in
`initialize'
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/gem_home/ruby/1.9.1/gems/fog-core-1.32.0/lib/fog/core/service.rb:115:in
`new'
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/gem_home/ruby/1.9.1/gems/fog-core-1.32.0/lib/fog/core/service.rb:115:in
`new'
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/gem_home/ruby/1.9.1/gems/fog-core-1.32.0/lib/fog/core/services_mixin.rb:16:in
`new'
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/gem_home/ruby/1.9.1/gems/fog-core-1.32.0/lib/fog/compute.rb:62:in
`new'
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/gem_home/ruby/1.9.1/gems/bosh_openstack_cpi-1.3012.0/lib/cloud/openstack/cloud.rb:67:in
`initialize'
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/gem_home/ruby/1.9.1/gems/bosh_openstack_cpi-1.3012.0/bin/openstack_cpi:17:in
`new'
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/gem_home/ruby/1.9.1/gems/bosh_openstack_cpi-1.3012.0/bin/openstack_cpi:17:in
`<top (required)>'
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/bin/openstack_cpi:16:in
`load'
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/bin/openstack_cpi:16:in
`<main>'
E, [2015-07-14T01:06:53.292878 #17191] ERROR -- : Unable to connect to the
OpenStack Compute API. Check task debug log for details.
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/gem_home/ruby/1.9.1/gems/bosh_openstack_cpi-1.3012.0/lib/cloud/openstack/helpers.rb:20:in
`cloud_error': Unable to connect to the OpenStack Compute API. Check task
debug log for details. (Bosh::Clouds::CloudError)
from
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/gem_home/ruby/1.9.1/gems/bosh_openstack_cpi-1.3012.0/lib/cloud/openstack/cloud.rb:70:in
`rescue in initialize'
from
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/gem_home/ruby/1.9.1/gems/bosh_openstack_cpi-1.3012.0/lib/cloud/openstack/cloud.rb:66:in
`initialize'
from
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/gem_home/ruby/1.9.1/gems/bosh_openstack_cpi-1.3012.0/bin/openstack_cpi:17:in
`new'
from
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/gem_home/ruby/1.9.1/gems/bosh_openstack_cpi-1.3012.0/bin/openstack_cpi:17:in
`<top (required)>'
from
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/bin/openstack_cpi:16:in
`load'
from
/home/ubuntu/.bosh_init/installations/4ae17957-e371-4116-74d2-8b50ad18c1b8/packages/bosh_openstack_cpi/bin/openstack_cpi:16:in
`<main>'

On Thu, Jul 16, 2015 at 5:34 PM, Guangcai Wang <guangcai.wang(a)gmail.com>
wrote:

Hi Dmitriy,

Thanks for your guiding.

I can run bundle exec rake spec:lifecycle under bosh_openstack_cpi in bosh
repo.
but cannot know what's the command "be rspec spec/unit/"? It seems you
missed some words. Could you help correct?




On Thu, Jul 16, 2015 at 3:42 PM, Dmitriy Kalinin <dkalinin(a)pivotal.io>
wrote:

First two snippets need to run from bosh repo (
github.com/cloudfoundry/bosh). Last one is from
bosh-openstack-cpi-release (
https://github.com/cloudfoundry-incubator/bosh-openstack-cpi-release).

On Wed, Jul 15, 2015 at 10:00 PM, Guangcai Wang <guangcai.wang(a)gmail.com>
wrote:

I tried to run these commands. but got failure. Anything specific to
set? How to get the Gemfile?

ubuntu(a)boshclivm:~/git/bosh-openstack-cpi-release$ bundle install
Could not locate Gemfile or .bundle/ directory


My environment (using rbenv):
ubuntu(a)boshclivm:~/git/bosh-openstack-cpi-release$ ruby -v
ruby 2.1.5p273 (2014-11-13 revision 48405) [x86_64-linux]

ubuntu(a)boshclivm:~/git/bosh-openstack-cpi-release$ bundle -v
Bundler version 1.10.5

ubuntu(a)boshclivm:~/git/bosh-openstack-cpi-release$ which ruby
/home/ubuntu/.rbenv/shims/ruby

ubuntu(a)boshclivm:~/git/bosh-openstack-cpi-release$ which bundle
/home/ubuntu/.rbenv/shims/bundle






On Thu, Jul 16, 2015 at 11:05 AM, Dmitriy Kalinin <dkalinin(a)pivotal.io>
wrote:

Hey,

ci/ directory is meant to be used with concourse CI (
http://concourse.ci/). Alternatively...

To run unit tests:

cd bosh_openstack_cpi
bundle install
be rspec spec/unit/

To run lifecycle tests locally:

cd bosh_openstack_cpi
bundle install
# set env variables specified by
https://github.com/cloudfoundry/bosh/blob/master/bosh_openstack_cpi/spec/integration/lifecycle_spec.rb
bundle exec rake spec:lifecycle

Finally to build a CPI release with an updated gem you will have to:

cd bosh-openstack-cpi-release
# reference openstack gem with gem 'bosh_openstack_cpi', path:
'/path/to/bosh/repo/bosh_openstack_cpi'
./scripts/vendor_gems
bosh create release --with-tarball
# use that tarball with bosh-init (
https://bosh.io/docs/init-openstack.html) to deploy the director+cpi

On Mon, Jul 13, 2015 at 11:57 PM, Guangcai Wang <
guangcai.wang(a)gmail.com> wrote:

Hi,

we are modifying bosh-openstack-cpi to support keystone v3. Now, I am
trying to test it locally. Who can share me how to run the test locally?


I tried to build a docker image and run the shell script under
ci/tasks/ inside docker. But it failed. Anything wrong?

root(a)cb30b6f9604c:/git/bosh-openstack-cpi-release#
./ci/tasks/run-lifecycle.sh
Don't run Bundler as root. Bundler can ask for sudo if it is needed,
and installing your bundle as root will break this application for all
non-root users on
this machine.
Could not locate Gemfile or .bundle/ directory



_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh