BTW did someone has a proper doc (like a swagger doc) for the api and for
the nats rpc? or knows how to generate it from the code

2017-04-27 21:00 GMT+02:00 Leandro David Cacciagioni <
leandro.21.2008(a)gmail.com>:

Guys, I'm not saying that the director is bad or wrong, actually what I
want is maybe to improve it a little bit without touching the logic or the
api, my final goal is maybe to create a drop in replacement but keeping the
agent and the logic in place. I know it can be hard work but OTP solves a
lot of "edge cases" of the classic languages out of the box.

Geoff by downtime I mean that, no matter what, in languages like
ruby/python/go or any "classical" language you need to stop and start again
the server to read the new code while in erlang / elixir there is no need
for this,since it has a feature that it is called "hot code reloading" (You
can read about it here
<http://learnyousomeerlang.com/designing-a-concurrent-application#hot-code-loving>,
here <http://erlang.org/doc/man/code.html> and here
<http://www.unstablebuild.com/2016/03/18/hot-code-reload-in-elixir.html>)
it is one of the moto's of erlang 99.9999999% (nine nines of availability)
and you can read more here <https://pragprog.com/articles/erlang>.

Marco good catch and thanks for the suggestion for the license, maybe I'll
evaluate some others like Apache or LGPL.

Thanks,
Leandro.-

2017-04-27 20:26 GMT+02:00 Voelz, Marco <marco.voelz(a)sap.com>:

Dear Leandro,

I'd love to see your experiment grow – keep in mind that the Director is around for quite a while and has some pretty complicated corner cases. Just like any rewrite: It is pretty simple to get 80% right, but then you'll spend much time on getting the remaining 20%.

A word on the license: If your target audience is really companies like IBM, don't go with GPL. I now that for example GPL is a no-go for us at SAP. I would assume a similar policy is in place in pretty much every big enterprise.

Warm regards
Marco

From: Leandro David Cacciagioni <leandro.21.2008(a)gmail.com>
Reply-To: "Discussions about the Cloud Foundry BOSH project." <cf-bosh(a)lists.cloudfoundry.org>
Date: Thursday, 27. April 2017 at 19:57
To: "Discussions about the Cloud Foundry BOSH project." <cf-bosh(a)lists.cloudfoundry.org>
Subject: [cf-bosh] Re: Re: Elixir for bosh director?

OK what you quote is certainly amazing, anyway that only tackle the Scalability in part (I know for sure that elixir/erlang can hold the same a lot better) but it didn't solve the Fault Tolerance part or the true no downtime deployments (I know that people like IBM will love to update BOSH with true / zero downtime). Plus all the simplification in the Director logic that can come from using the proper tool for the right job. Anyway I think I'll start a POC as under GPL license to make a compatible "BOSH director" using elixir. Anyone who will like to help more than welcomed.

2017-04-27 17:59 GMT+02:00 Geoff Franks <geoff(a)starkandwayne.com<mailto:geoff(a)starkandwayne.com>>:
FWIW, we've managed BOSHes with many deployments, some of which consist of ~1000 VMs, and not seen any direct performance issues of the BOSH director. Just lengthy deploys due to having so many VMs to iterate through.

I've also seen a significant uptick in responsiveness from the bosh cli when using the v2 cli, since ruby isn't parsing for tons of gemfiles every time I start the CLI up.

What kind of downtime are they seeing when upgrading the BOSH director? IIRC there were things like binary-bosh to keep bosh alive + responsive all the time, but in practice, the main cause of downtime for BOSH is the delay for compilation + VM creation/deletion during `create-env`, which I don't think rewriting in another language can solve (stemcell upgrades). Downtime from restarting the director is pretty minimal, and BOSH itself isn't a critical path for the availability of the VMs it deploys.

We've mitigated the create-env issues by using create-env to create a BOSH that will manage upper-level infrastructure that isn't as important, and have that BOSH deploy other BOSHes to deploy prod (lower risk of not restarting a failed VM due to a BOSH upgrade).

OK what you quote is certainly amazing, anyway that only tackle the
Scalability in part (I know for sure that elixir/erlang can hold the same a
lot better) but it didn't solve the Fault Tolerance part or the true no
downtime deployments (I know that people like IBM will love to update BOSH
with true / zero downtime). Plus all the simplification in the Director
logic that can come from using the proper tool for the right job. Anyway I
think I'll start a POC as under GPL license to make a compatible "BOSH
director" using elixir. Anyone who will like to help more than welcomed.

2017-04-27 17:59 GMT+02:00 Geoff Franks <geoff(a)starkandwayne.com>:

I think you first have to ask what real (as opposed to theoretical) problems does current bosh director have, where the effort required to fix them is such that it would be more efficient (in terms of time, resources and cost) to re-write the whole thing, possibly in another language.

I have experienced a couple of times where it was decided to re-write a well established major piece of software form the ground up. Without exception, it always took far longer than the original estimate and didn’t’ have the features of the original.

Jo

From: Leandro David Cacciagioni <leandro.21.2008(a)gmail.com<mailto:leandro.21.2008(a)gmail.com>>
Reply-To: "cf-bosh(a)lists.cloudfoundry.org<mailto:cf-bosh(a)lists.cloudfoundry.org>" <cf-bosh(a)lists.cloudfoundry.org<mailto:cf-bosh(a)lists.cloudfoundry.org>>
Date: Thursday, April 27, 2017 at 6:01 AM
To: "cf-bosh(a)lists.cloudfoundry.org<mailto:cf-bosh(a)lists.cloudfoundry.org>" <cf-bosh(a)lists.cloudfoundry.org<mailto:cf-bosh(a)lists.cloudfoundry.org>>
Subject: [EXTERNAL] - [cf-bosh] Elixir for bosh director?

After more than 6 months working with elixir in prod, it crossed my mind that maybe it deserves some time of experiment and think on the possibility of a TOTAL REWRITE OF BOSH DIRECTOR USING ELIXIR.
Some of the pros that I can list out of the box (without digging to much in the technical side) are:

* Ruby like syntax (I know I know... This means a lot for people that don't like erlang syntax) (I'm used to both so far)
* Easiness of development thanks to OTP & FP.
* Scalability (ex: http://www.phoenixframework.org/blog/the-road-to-2-million-websocket-connections<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.phoenixframework.org_blog_the-2Droad-2Dto-2D2-2Dmillion-2Dwebsocket-2Dconnections&d=DwMFaQ&c=ZgVRmm3mf2P1-XDAyDsu4A&r=62hyuFHUBRZl3K63suuxEK4tHN8enyiCVaHEAagYkK4&m=ITkJUBdozfcbUccDPmz2sSgeNGohUNiSGP_SRUbDV1U&s=lfCUFpdQRuvF6BimtsLsV3CX4oXrU7gmdhjpJXwx7zs&e=>)
* Fault-tolerance
* True no downtime updates.
* Simplification:
* nats can be deprecated.
* All the other jobs (Director, Registry, Blobstore, HM & CPI) can to be OTP/Apps (Mix powered) under the same umbrella project.
* Clustering out of the box
* Perfomance wins, giving the nature of elixir/erlang/OTP is easy to guess that a single bosh instance will gonna be able to manage more deployments and bigger deployments than it does now.

This is a suggestion and I would like to know if you agree or don't and why.

Thanks,
Leandro.-

FWIW, we've managed BOSHes with many deployments, some of which consist of ~1000 VMs, and not seen any direct performance issues of the BOSH director. Just lengthy deploys due to having so many VMs to iterate through.

I've also seen a significant uptick in responsiveness from the bosh cli when using the v2 cli, since ruby isn't parsing for tons of gemfiles every time I start the CLI up.

After more than 6 months working with elixir in prod, it crossed my mind
that maybe it deserves some time of experiment and think on the possibility
of a *TOTAL REWRITE OF BOSH DIRECTOR USING ELIXIR*.
Some of the pros that I can list out of the box (without digging to much in
the technical side) are:

- Ruby like syntax (I know I know... This means a lot for people that
don't like erlang syntax) (I'm used to both so far)
- Easiness of development thanks to OTP & FP.
- Scalability (ex:
http://www.phoenixframework.org/blog/the-road-to-2-million-websocket-connections
)
- Fault-tolerance
- True no downtime updates.
- Simplification:
- nats can be deprecated.
- All the other jobs (Director, Registry, Blobstore, HM & CPI) can to
be OTP/Apps (Mix powered) under the same umbrella project.
- Clustering out of the box
- Perfomance wins, giving the nature of elixir/erlang/OTP is easy to
guess that a single bosh instance will gonna be able to manage more
deployments and bigger deployments than it does now.

This is a suggestion and I would like to know if you agree or don't and why.

Thanks,
Leandro.-

Awesome work and huge congratulations!

________________________________
From: Colin Jackson <cojackson(a)pivotal.io>
Sent: Wednesday, April 26, 2017 12:02:06 AM
To: cf-bosh(a)lists.cloudfoundry.org
Subject: [cf-bosh] BOSH Windows is now GA


Hi cf-bosh,


BOSH Windows, the new deployment mechanism for Windows cells, is now generally available.


About BOSH Windows: At this point BOSH Windows has been successfully running 100% of the CI/CD workload for the BOSH Windows and Greenhouse teams for more than a month. The Diego team has also just finished an effort to integrate workers deployed via BOSH Windows in their CI/CD pipeline. In all cases it has held up remarkably well. Given this I'm happy to recommend people start using BOSH Windows as soon as possible.


BOSH Windows uses the exact same agent code as BOSH Linux for running jobs, so when you run jobs you know they run similarly on Windows as elsewhere.


Thanks all very much, and again a huge thanks to the BOSH Windows team!


Sincerely,

William Martin, BOSH Windows PM

Colin Jackson, BOSH Windows PM

Hi cf-bosh,


BOSH Windows, the new deployment mechanism for Windows cells, is now
generally available.

About BOSH Windows: At this point BOSH Windows has been successfully
running 100% of the CI/CD workload for the BOSH Windows and Greenhouse
teams for more than a month. The Diego team has also just finished an
effort to integrate workers deployed via BOSH Windows in their CI/CD
pipeline. In all cases it has held up remarkably well. Given this I'm happy
to recommend people start using BOSH Windows as soon as possible.


BOSH Windows uses the exact same agent code as BOSH Linux for running jobs,
so when you run jobs you know they run similarly on Windows as elsewhere.

Thanks all very much, and again a huge thanks to the BOSH Windows team!


Sincerely,

William Martin, BOSH Windows PM

Colin Jackson, BOSH Windows PM

Thanks Stanislav,
this is a bad news. :(
I'm trying to use vSphere to do the installation.

Thanks again.

Thanks Sze Siong Teo,

I will test this.

Unless you really have problem using vagrant to setup bosh-lite, then you can setup the bosh-lite VM is a slightly more manual way.

You can download this and create a VM manually then use it as your disk image:
https://atlas.hashicorp.com/cloudfoundry/boxes/bosh-lite/versions/9000.137.0/providers/virtualbox.box

In any case if you want to import into AWS or some other cloud, just rename virtualbox.box to virtualbox.tar.gz, unpack it and you will get VMDK file inside.

Regards,
Teo

majorErrorCode="500" message="[ d992ba15-cec3-4953-9af3-4132d6a111fd-e3c9

] Unable to perform this action. Contact your cloud administrator."
minorErrorCode="INTERNAL_SERVER_ERROR"
you might have to contact vcloud support to understand root cause of this
error.

On Fri, Apr 21, 2017 at 2:14 AM, Phong Tran <phong.tran(a)emc.com> wrote:

We're currently trying to deploying Single-VM Bosh Director on vCloud
Director with bosh-init for one of our test environments and we ran into
the error on attaching disk to the VM.

vCloud Director Version: 8.0.2
bosh
bosh-v216-4.tgz
bosh-vcloud-cpi
bosh-vcloud-cpi-v24.tgz
stemcell:
bosh-vcloud-esxi-ubuntu-trusty-go_agent-v3363-15.tgz

boshadmin(a)bosh-init-ubuntu:~/bosh-init$ bosh-init deploy bosh.yml
Deployment manifest: '/home/boshadmin/bosh-init/bosh.yml'
Deployment state: '/home/boshadmin/bosh-init/bosh-state.json'

.... intentionally removed package compiling messages to keep it short ...

Started deploying
Creating VM for instance 'bosh/0' from stemcell 'urn:vcloud:catalogitem:
75d662d5-4de9-4b20-8fc2-7754f261314e'... Finished (00:02:59)
Waiting for the agent on VM 'urn:vcloud:vm:a2e227c7-3484-4bdc-afb3-b022dc44198d'
to be ready... Finished (00:00:38)
Creating disk... Finished (00:00:22)
Attaching disk 'urn:vcloud:disk:a5da07db-a772-46cd-b65f-6e38cbb7dfcb'
to VM 'urn:vcloud:vm:a2e227c7-3484-4bdc-afb3-b022dc44198d'... Failed
(00:01:42)
Failed deploying (00:05:42)

Stopping registry... Finished (00:00:00)
Cleaning up rendered CPI jobs... Finished (00:00:00)

Command 'deploy' failed:
Deploying:
Creating instance 'bosh/0':
Updating instance disks:
Updating disks:
Deploying disk:
Attaching disk in the cloud:
CPI 'attach_disk' method responded with error:
CmdError{"type":"Unknown","message":"re raising exception Task
urn:vcloud:task:f28ba0a0-35a8-48f6-9e7a-e14018af9324 Attached Disk to
Virtual Machine 1bcc2f50-c70c-40db-7ffa-eab1d721c876(a2e227c7-3484-4bdc-afb3-b022dc44198d)
completed unsuccessfully, Details: [ fba153f1-3b4e-42f7-83e9-82990457c75b-4971
] The requested operation cannot be performed because th... in
attach_disk","ok_to_retry":false}

From the logs in vCloud Director portal, we can see that the disk was
successfully attached to the VM on the first try, but it retried to attach
the disk 9 more times resulting in the error above as the disk is already
attached.

It turned out that the attach disk operation consists of multiple steps.
The first step is to sending a request to vCD to attach a disk to the VM
and next steps are to update and delete catalog media (ISO) on the VM. All
of these steps are wrapped in a retry logic so if one of them fails, all
steps are retried for a fixed number of times. With
BOSH_INIT_LOG_LEVEL=DEBUG, we can see the real problem is in one of steps
used for deleting the catalog item/media failed with 500 server error:

D, [2017-04-20T21:03:48.986129 #6143] DEBUG -- : REST REQ DELETE
https://10.145.114.95/api/catalogItem/cf9fca53-df90-4eff-9773-f8b14c5a86d1
{:Accept=>"application/*+xml;version=5.5", :content_type=>"*/*",
"X-VMWARE-VCLOUD-CLIENT-REQUEST-ID"=>"d992ba15-cec3-4953-9af3-4132d6a111fd",
:x_vcloud_authorization=>"39a0728f4c5143f1b5c94fef578c2bd2"}
{"vcloud-token"=>"39a0728f4c5143f1b5c94fef578c2bd2", "Path"=>"%2F"}
D, [2017-04-20T21:03:49.015256 #6143] DEBUG -- : REST RES 500
{:date=>"Thu, 20 Apr 2017 20:03:49 GMT", :x_vmware_vcloud_request_id=>"
d992ba15-cec3-4953-9af3-4132d6a111fd-e3c9", :x_vcloud_authorization=>"
39a0728f4c5143f1b5c94fef578c2bd2", :set_cookie=>["vcloud-token=
39a0728f4c5143f1b5c94fef578c2bd2; Secure; Path=/"],
:content_type=>"application/vnd.vmware.vcloud.error+xml;version=5.5",
:x_vmware_vcloud_request_execution_time=>"8", :content_length=>"435"}
<?xml version="1.0" encoding="UTF-8"?>
<Error xmlns="http://www.vmware.com/vcloud/v1.5" majorErrorCode="500"
message="[ d992ba15-cec3-4953-9af3-4132d6a111fd-e3c9 ] Unable to perform
this action. Contact your cloud administrator." minorErrorCode="INTERNAL_SERVER_ERROR"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
http://www.vmware.com/vcloud/v1.5 http://10.145.114.95/api/v1.5/
schema/master.xsd"></Error>
W, [2017-04-20T21:03:51.469053 #6143] WARN -- : Caught an exception
during attach_disk Exception 500 Internal Server Error, Type RestClient::InternalServerError
Message 500 Internal Server Error
W, [2017-04-20T21:03:51.469143 #6143] WARN -- : Exception trace
/home/boshadmin/.bosh_init/installations/445016e5-c428-
4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/
vendor/bundle/ruby/2.3.0/gems/rest-client-1.6.9/lib/
restclient/abstract_response.rb:48:in `return!'\n/home/boshadmin/.
bosh_init/installations/445016e5-c428-4cf2-57c4-
11f983dfc4aa/packages/bosh_vcloud_cpi/lib/cloud/vcloud/vcd_client.rb:290:in
`block (2 levels) in send_request'\n/home/boshadmin/.bosh_init/
installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/
packages/bosh_vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/
rest-client-1.6.9/lib/restclient/request.rb:267:in
`process_result'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-
4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/
vendor/bundle/ruby/2.3.0/gems/rest-client-1.6.9/lib/restclient/request.rb:212:in
`block in transmit'\n/home/boshadmin/.bosh_init/installations/
445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/ruby_
vcloud_cpi/lib/ruby/2.3.0/net/http.rb:853:in
`start'\
n/home/boshadmin/.bosh_init/installations/445016e5-c428-
4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/
vendor/bundle/ruby/2.3.0/gems/rest-client-1.6.9/lib/restclient/request.rb:206:in
`transmit'\n/home/boshadmin/.bosh_init/installations/
445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_
vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/rest-client-1.6.9/
lib/restclient/request.rb:68:in `execute'\n/home/boshadmin/.
bosh_init/installations/445016e5-c428-4cf2-57c4-
11f983dfc4aa/packages/bosh_vcloud_cpi/vendor/bundle/ruby/
2.3.0/gems/rest-client-1.6.9/lib/restclient/request.rb:35:in
`execute'\n/home/boshadmin/.bosh_init/installations/
445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_
vcloud_cpi/lib/cloud/vcloud/vcd_client.rb:287:in `block in
send_request'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-
4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/
vendor/bundle/ruby/2.3.0/gems/bosh_common-1.3262.1/lib/common/retryable.rb:28:in
`block in retryer'\n/home/boshadmin/.bosh_init/install
ations/4
45016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_
vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/bosh_common-1.3262.
1/lib/common/retryable.rb:26:in `loop'\n/home/boshadmin/.bosh_
init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/
packages/bosh_vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/
bosh_common-1.3262.1/lib/common/retryable.rb:26:in
`retryer'\n/home/boshadmin/.bosh_init/installations/
445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_
vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/bosh_common-1.3262.1/lib/common/common.rb:119:in
`retryable'\n/home/boshadmin/.bosh_init/installations/
445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_
vcloud_cpi/lib/cloud/vcloud/vcd_client.rb:283:in `send_request'\n/home/
boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/
packages/bosh_vcloud_cpi/lib/cloud/vcloud/vcd_client.rb:126:in
`invoke'\n/home/boshadmin/.bosh_init/installations/
445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_
vcloud_cpi/lib/cloud/vcloud/steps/delete_catalog_
media.rb
:14:in `block in perform'\n/home/boshadmin/.bosh_init/installations/
445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_
vcloud_cpi/lib/cloud/vcloud/vcd_client.rb:198:in
`timed_loop'\n/home/boshadmin/.bosh_init/installations/
445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_
vcloud_cpi/lib/cloud/vcloud/steps/delete_catalog_media.rb:10:in
`perform'\n/home/boshadmin/.bosh_init/installations/
445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_
vcloud_cpi/lib/cloud/vcloud/steps.rb:37:in `next'\n/home/boshadmin/.bosh_
init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/
packages/bosh_vcloud_cpi/lib/cloud/vcloud/cloud.rb:406:in
`save_agent_env'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-
4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/lib/cloud/vcloud/cloud.rb:290:in
`block (2 levels) in attach_disk'\n/home/boshadmin/
.bosh_init/installations/445016e5-c428-4cf2-57c4-
11f983dfc4aa/packages/bosh_vcloud_cpi/vendor/bundle/ruby/
2.3.0/gems/bosh_common-1.3262.1/lib/common/retryab
le.rb:28
:in `block in retryer'\n/home/boshadmin/.bosh_init/installations/
445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_
vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/bosh_common-1.3262.
1/lib/common/retryable.rb:26:in `loop'\n/home/boshadmin/.bosh_
init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/
packages/bosh_vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/
bosh_common-1.3262.1/lib/common/retryable.rb:26:in
`retryer'\n/home/boshadmin/.bosh_init/installations/
445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_
vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/bosh_common-1.3262.1/lib/common/common.rb:119:in
`retryable'\n/home/boshadmin/.bosh_init/installations/
445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_
vcloud_cpi/lib/cloud/vcloud/cloud.rb:279:in `block in
attach_disk'\n/home/boshadmin/.bosh_init/installations/
445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_
vcloud_cpi/lib/cloud/vcloud/steps.rb:44:in `block in
perform'\n/home/boshadmin/.bosh_init/installations/
445016e5-c428-4cf2-57c4-11f
983dfc4a
a/packages/bosh_vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/
bosh_common-1.3262.1/lib/common/thread_formatter.rb:49:in
`with_thread_name'\n/home/boshadmin/.bosh_init/
installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/
packages/bosh_vcloud_cpi/lib/cloud/vcloud/steps.rb:41:in
`perform'\n/home/boshadmin/.bosh_init/installations/
445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_
vcloud_cpi/lib/cloud/vcloud/steps.rb:58:in `perform'\n/home/boshadmin/.
bosh_init/installations/445016e5-c428-4cf2-57c4-
11f983dfc4aa/packages/bosh_vcloud_cpi/lib/cloud/vcloud/cloud.rb:371:in
`steps'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-
11f983dfc4aa/packages/bosh_vcloud_cpi/lib/cloud/vcloud/cloud.rb:270:in
`attach_disk'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-
4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/
vendor/bundle/ruby/2.3.0/gems/bosh_cpi-1.3262.1/lib/bosh/cpi/cli.rb:71:in
`public_send'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-
4cf2-57c4-11f983dfc
4aa/pack
ages/bosh_vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/bosh_
cpi-1.3262.1/lib/bosh/cpi/cli.rb:71:in `run'\n/home/boshadmin/.bosh_
init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/
packages/bosh_vcloud_cpi/bin/vcloud_cpi:31:in `<top
(required)>'\n/home/boshadmin/.bosh_init/installations/
445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/ruby_
vcloud_cpi/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/cli/exec.rb:63:in
`load'\n/home/boshadmin/.bosh_init/installations/445016e5-
c428-4cf2-57c4-11f983dfc4aa/packages/ruby_vcloud_cpi/lib/
ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/cli/exec.rb:63:in
`kernel_load'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-
4cf2-57c4-11f983dfc4aa/packages/ruby_vcloud_cpi/lib/
ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/cli/exec.rb:24:in
`run'\n/home/boshadmin/.bosh_init/installations/445016e5-
c428-4cf2-57c4-11f983dfc4aa/packages/ruby_vcloud_cpi/lib/
ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/cli.rb:304:in
`exec'\n/home/boshadmin/.
bosh_ini
t/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/
packages/ruby_vcloud_cpi/lib/ruby/gems/2.3.0/gems/bundler-
1.12.4/lib/bundler/vendor/thor/lib/thor/command.rb:27:in
`run'\n/home/boshadmin/.bosh_init/installations/445016e5-
c428-4cf2-57c4-11f983dfc4aa/packages/ruby_vcloud_cpi/lib/
ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/vendor/
thor/lib/thor/invocation.rb:126:in `invoke_command'\n/home/
boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/
packages/ruby_vcloud_cpi/lib/ruby/gems/2.3.0/gems/bundler-
1.12.4/lib/bundler/vendor/thor/lib/thor.rb:359:in
`dispatch'\n/home/boshadmin/.bosh_init/installations/
445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/ruby_
vcloud_cpi/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/lib/
bundler/vendor/thor/lib/thor/base.rb:440:in `start'\n/home/boshadmin/.
bosh_init/installations/445016e5-c428-4cf2-57c4-
11f983dfc4aa/packages/ruby_vcloud_cpi/lib/ruby/gems/2.3.
0/gems/bundler-1.12.4/lib/bundler/cli.rb:11:in `start'\n/home/boshadmin/.
bosh_in
it/insta
llations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/
ruby_vcloud_cpi/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/exe/bundle:27:in
`block in <top (required)>'\n/home/boshadmin/.bosh_init/installations/
445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/ruby_
vcloud_cpi/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/lib/
bundler/friendly_errors.rb:98:in `with_friendly_errors'\n/home/
boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/
packages/ruby_vcloud_cpi/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/exe/bundle:19:in
`<top (required)>'\n/home/boshadmin/.bosh_init/installations/
445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/ruby_vcloud_cpi/bin/bundle:22:in
`load'\n/home/boshadmin/.bosh_init/installations/445016e5-
c428-4cf2-57c4-11f983dfc4aa/packages/ruby_vcloud_cpi/bin/bundle:22:in
`<main>'

This 500 server error on deletion of catalog item triggered the retry loop
and thus indirectly caused the attach disk step to fail on subsequent
attempts.

Any help on how to resolve this issue would be greatly appreciated.

Does anyone deploy Bosh/CF on vCloud Director and how is your experience
with Bosh CPI for vCloud?

Thanks,
Phong

Dear Adrian,

Thanks for your feedback.
The screenshot didnot showup in the web. Could you please send again?

about you mentioned: Select the 'Enable VT-x/AMD-V' option, because i'm in the VM Ubuntu, and now i'm stopped at BOTS-LITE installation, it installed a new VM automaticlly, i do not know where can i select the option..... :(

Please advise. Thanks..

VirtualBox does not support nested virtualization https://www.virtualbox.org/ticket/4032.

I'd not install it in the vm. Just use vagrant to start the bosh-lite on the same host. So you have two vms: 1. You Ubuntu vm, 2. A bosh-lite

I used this setup myself to start developing bosh releases.



You should be able to enable virtualization inside your vm using the advanced setting of the processer configuration. I've added a screenshot of the config.

Select the 'Enable VT-x/AMD-V' option.

[cid:image002.png(a)01D2BA95.826E5D10]



I'm pretty sure you won't be happy with the nested virtualization as it is really slow due to the huge amount of overhead.

So please just start the bosh-lite in the same virualbox instance as your Ubuntu vm is running in.



Have a nice weekend as well!

Kind regards

Adrian

Is there any ETA for the BOSH cli v1 deprecation?

Thanks,
Leandro.-

We're currently trying to deploying Single-VM Bosh Director on vCloud Director with bosh-init for one of our test environments and we ran into the error on attaching disk to the VM.

vCloud Director Version: 8.0.2
bosh
bosh-v216-4.tgz
bosh-vcloud-cpi
bosh-vcloud-cpi-v24.tgz
stemcell:
bosh-vcloud-esxi-ubuntu-trusty-go_agent-v3363-15.tgz

boshadmin(a)bosh-init-ubuntu:~/bosh-init$ bosh-init deploy bosh.yml
Deployment manifest: '/home/boshadmin/bosh-init/bosh.yml'
Deployment state: '/home/boshadmin/bosh-init/bosh-state.json'

.... intentionally removed package compiling messages to keep it short ...

Started deploying
Creating VM for instance 'bosh/0' from stemcell 'urn:vcloud:catalogitem:75d662d5-4de9-4b20-8fc2-7754f261314e'... Finished (00:02:59)
Waiting for the agent on VM 'urn:vcloud:vm:a2e227c7-3484-4bdc-afb3-b022dc44198d' to be ready... Finished (00:00:38)
Creating disk... Finished (00:00:22)
Attaching disk 'urn:vcloud:disk:a5da07db-a772-46cd-b65f-6e38cbb7dfcb' to VM 'urn:vcloud:vm:a2e227c7-3484-4bdc-afb3-b022dc44198d'... Failed (00:01:42)
Failed deploying (00:05:42)

Stopping registry... Finished (00:00:00)
Cleaning up rendered CPI jobs... Finished (00:00:00)

Command 'deploy' failed:
Deploying:
Creating instance 'bosh/0':
Updating instance disks:
Updating disks:
Deploying disk:
Attaching disk in the cloud:
CPI 'attach_disk' method responded with error: CmdError{"type":"Unknown","message":"re raising exception Task urn:vcloud:task:f28ba0a0-35a8-48f6-9e7a-e14018af9324 Attached Disk to Virtual Machine 1bcc2f50-c70c-40db-7ffa-eab1d721c876(a2e227c7-3484-4bdc-afb3-b022dc44198d) completed unsuccessfully, Details: [ fba153f1-3b4e-42f7-83e9-82990457c75b-4971 ] The requested operation cannot be performed because th... in attach_disk","ok_to_retry":false}

From the logs in vCloud Director portal, we can see that the disk was successfully attached to the VM on the first try, but it retried to attach the disk 9 more times resulting in the error above as the disk is already attached.

It turned out that the attach disk operation consists of multiple steps. The first step is to sending a request to vCD to attach a disk to the VM and next steps are to update and delete catalog media (ISO) on the VM. All of these steps are wrapped in a retry logic so if one of them fails, all steps are retried for a fixed number of times. With BOSH_INIT_LOG_LEVEL=DEBUG, we can see the real problem is in one of steps used for deleting the catalog item/media failed with 500 server error:

D, [2017-04-20T21:03:48.986129 #6143] DEBUG -- : REST REQ DELETE https://10.145.114.95/api/catalogItem/cf9fca53-df90-4eff-9773-f8b14c5a86d1 {:Accept=>"application/*+xml;version=5.5", :content_type=>"*/*", "X-VMWARE-VCLOUD-CLIENT-REQUEST-ID"=>"d992ba15-cec3-4953-9af3-4132d6a111fd", :x_vcloud_authorization=>"39a0728f4c5143f1b5c94fef578c2bd2"} {"vcloud-token"=>"39a0728f4c5143f1b5c94fef578c2bd2", "Path"=>"%2F"}
D, [2017-04-20T21:03:49.015256 #6143] DEBUG -- : REST RES 500 {:date=>"Thu, 20 Apr 2017 20:03:49 GMT", :x_vmware_vcloud_request_id=>"d992ba15-cec3-4953-9af3-4132d6a111fd-e3c9", :x_vcloud_authorization=>"39a0728f4c5143f1b5c94fef578c2bd2", :set_cookie=>["vcloud-token=39a0728f4c5143f1b5c94fef578c2bd2; Secure; Path=/"], :content_type=>"application/vnd.vmware.vcloud.error+xml;version=5.5", :x_vmware_vcloud_request_execution_time=>"8", :content_length=>"435"} <?xml version="1.0" encoding="UTF-8"?>
<Error xmlns="http://www.vmware.com/vcloud/v1.5" majorErrorCode="500" message="[ d992ba15-cec3-4953-9af3-4132d6a111fd-e3c9 ] Unable to perform this action. Contact your cloud administrator." minorErrorCode="INTERNAL_SERVER_ERROR" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.vmware.com/vcloud/v1.5 http://10.145.114.95/api/v1.5/schema/master.xsd"></Error>
W, [2017-04-20T21:03:51.469053 #6143] WARN -- : Caught an exception during attach_disk Exception 500 Internal Server Error, Type RestClient::InternalServerError Message 500 Internal Server Error
W, [2017-04-20T21:03:51.469143 #6143] WARN -- : Exception trace /home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/rest-client-1.6.9/lib/restclient/abstract_response.rb:48:in `return!'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/lib/cloud/vcloud/vcd_client.rb:290:in `block (2 levels) in send_request'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/rest-client-1.6.9/lib/restclient/request.rb:267:in `process_result'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/rest-client-1.6.9/lib/restclient/request.rb:212:in `block in transmit'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/ruby_vcloud_cpi/lib/ruby/2.3.0/net/http.rb:853:in `start'\
n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/rest-client-1.6.9/lib/restclient/request.rb:206:in `transmit'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/rest-client-1.6.9/lib/restclient/request.rb:68:in `execute'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/rest-client-1.6.9/lib/restclient/request.rb:35:in `execute'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/lib/cloud/vcloud/vcd_client.rb:287:in `block in send_request'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/bosh_common-1.3262.1/lib/common/retryable.rb:28:in `block in retryer'\n/home/boshadmin/.bosh_init/installations/4
45016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/bosh_common-1.3262.1/lib/common/retryable.rb:26:in `loop'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/bosh_common-1.3262.1/lib/common/retryable.rb:26:in `retryer'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/bosh_common-1.3262.1/lib/common/common.rb:119:in `retryable'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/lib/cloud/vcloud/vcd_client.rb:283:in `send_request'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/lib/cloud/vcloud/vcd_client.rb:126:in `invoke'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/lib/cloud/vcloud/steps/delete_catalog_media.rb
:14:in `block in perform'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/lib/cloud/vcloud/vcd_client.rb:198:in `timed_loop'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/lib/cloud/vcloud/steps/delete_catalog_media.rb:10:in `perform'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/lib/cloud/vcloud/steps.rb:37:in `next'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/lib/cloud/vcloud/cloud.rb:406:in `save_agent_env'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/lib/cloud/vcloud/cloud.rb:290:in `block (2 levels) in attach_disk'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/bosh_common-1.3262.1/lib/common/retryable.rb:28
:in `block in retryer'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/bosh_common-1.3262.1/lib/common/retryable.rb:26:in `loop'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/bosh_common-1.3262.1/lib/common/retryable.rb:26:in `retryer'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/bosh_common-1.3262.1/lib/common/common.rb:119:in `retryable'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/lib/cloud/vcloud/cloud.rb:279:in `block in attach_disk'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/lib/cloud/vcloud/steps.rb:44:in `block in perform'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4a
a/packages/bosh_vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/bosh_common-1.3262.1/lib/common/thread_formatter.rb:49:in `with_thread_name'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/lib/cloud/vcloud/steps.rb:41:in `perform'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/lib/cloud/vcloud/steps.rb:58:in `perform'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/lib/cloud/vcloud/cloud.rb:371:in `steps'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/lib/cloud/vcloud/cloud.rb:270:in `attach_disk'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/bosh_cpi-1.3262.1/lib/bosh/cpi/cli.rb:71:in `public_send'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/pack
ages/bosh_vcloud_cpi/vendor/bundle/ruby/2.3.0/gems/bosh_cpi-1.3262.1/lib/bosh/cpi/cli.rb:71:in `run'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/bosh_vcloud_cpi/bin/vcloud_cpi:31:in `<top (required)>'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/ruby_vcloud_cpi/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/cli/exec.rb:63:in `load'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/ruby_vcloud_cpi/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/cli/exec.rb:63:in `kernel_load'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/ruby_vcloud_cpi/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/cli/exec.rb:24:in `run'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/ruby_vcloud_cpi/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/cli.rb:304:in `exec'\n/home/boshadmin/.bosh_ini
t/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/ruby_vcloud_cpi/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/ruby_vcloud_cpi/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/ruby_vcloud_cpi/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/vendor/thor/lib/thor.rb:359:in `dispatch'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/ruby_vcloud_cpi/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/vendor/thor/lib/thor/base.rb:440:in `start'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/ruby_vcloud_cpi/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/cli.rb:11:in `start'\n/home/boshadmin/.bosh_init/insta
llations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/ruby_vcloud_cpi/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/exe/bundle:27:in `block in <top (required)>'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/ruby_vcloud_cpi/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/lib/bundler/friendly_errors.rb:98:in `with_friendly_errors'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/ruby_vcloud_cpi/lib/ruby/gems/2.3.0/gems/bundler-1.12.4/exe/bundle:19:in `<top (required)>'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/ruby_vcloud_cpi/bin/bundle:22:in `load'\n/home/boshadmin/.bosh_init/installations/445016e5-c428-4cf2-57c4-11f983dfc4aa/packages/ruby_vcloud_cpi/bin/bundle:22:in `<main>'

This 500 server error on deletion of catalog item triggered the retry loop and thus indirectly caused the attach disk step to fail on subsequent attempts.

Any help on how to resolve this issue would be greatly appreciated.

Does anyone deploy Bosh/CF on vCloud Director and how is your experience with Bosh CPI for vCloud?

Thanks,
Phong

Dear Adrian,

Thanks a lot for your feedback.
because i do not have enough HW resource, i can not install Ubuntu to a phical server.

Could you please share how can i "enable the virtualization inside the advanced cpu config of virtualbox"? does that mean i should do something in my Ubuntu machine (virtual machine), and how?

Thanks in advance.
Wish you have a nice weekend.