Hi,

I'm trying to create my own 'bosh-lite' VM from scratch instead of using the one that's available via vagrant, but I have a hard time figuring out what the deployment manifest actually needs to contain.

Current version of my deployment manifest: https://gist.github.com/zralt/6550e1f2bab3fc3da95cb0a7db1ed220

I tried to use the template from the bosh-lite repo after filling out the release url's, but it
failed with this:

Command 'deploy' failed:
Parsing installation manifest '/root/litebosh.yml':
Validating installation manifest:
cloud_provider.template.name must be provided
cloud_provider.template.release must be provided
cloud_provider.template.release '' must refer to a release in releases

I added a cloud_provider block to end of my yml, but I'm not sure that it's correct, please tell.
After adding, I get this:
...
Validating deployment manifest... Failed (00:00:00)
Failed validating (00:00:03)

Command 'deploy' failed:
Validating deployment manifest:
jobs[0].resource_pool must be provided

How is this resource pool need to be defined, what/where needs to be specified in the yml?

Thanks in advance!

Okay, continued to progress with this, manifest now looks like this:
https://gist.github.com/zralt/eb7f8baa6b2346bdc38a1a4027fe4581

Currently I'm failing with this:
bosh-init deploy litebosh.yml
Deployment manifest: '/root/litebosh.yml'
Deployment state: '/root/litebosh-state.json'

Started validating
Downloading release 'bosh'... Skipped [Found in local cache] (00:00:00)
Validating release 'bosh'... Finished (00:00:01)
Downloading release 'bosh-warden-cpi'... Skipped [Found in local cache] (00:00:00)
Validating release 'bosh-warden-cpi'... Finished (00:00:00)
Downloading release 'garden-runc'... Skipped [Found in local cache] (00:00:00)
Validating release 'garden-runc'... Finished (00:00:01)
Validating cpi release... Finished (00:00:00)
Validating deployment manifest... Finished (00:00:00)
Downloading stemcell... Skipped [Found in local cache] (00:00:00)
Validating stemcell... Finished (00:00:06)
Finished validating (00:00:10)

Started installing CPI
Compiling package 'golang_1.3/fc3bc1b4431e8913d91362c1183c9852809d35f6'... Finished (00:00:04)
Compiling package 'warden_cpi/9665dea8a7152fd9333072824bf61c09193b7a00'... Finished (00:00:00)
Installing packages... Finished (00:00:11)
Rendering job templates... Finished (00:00:01)
Installing job 'warden_cpi'... Finished (00:00:00)
Finished installing CPI (00:00:16)

Starting registry... Finished (00:00:00)
Uploading stemcell 'bosh-warden-boshlite-ubuntu-trusty-go_agent/3312.12'... Skipped [Stemcell already uploaded] (00:00:00)

Started deploying
Creating VM for instance 'bosh/0' from stemcell '8938de5b-74d1-4170-4546-0bae8f5a4340'... Failed (00:00:00)
Failed deploying (00:00:00)

Stopping registry... Finished (00:00:00)
Cleaning up rendered CPI jobs... Finished (00:00:00)

Command 'deploy' failed:
Deploying:
Creating instance 'bosh/0':
Creating VM:
Creating vm with stemcell cid '8938de5b-74d1-4170-4546-0bae8f5a4340':
CPI 'create_vm' method responded with error: CmdError{"type":"Bosh::Clouds::CloudError","message":"Creating VM with agent ID '4b41101d-9100-458b-54e4-609b5f2a1258': Creating container: Post http://api/containers: dial tcp 127.0.0.1:7777: connection refused","ok_to_retry":false}

I would assume that the warden would be installed locally, then it would be used to start the vm's (containers), but as I understand this wants to connect to an already existing warden process. Any suggestions?

I compiled garden-linux (as it had a makefile, garden-runc suggests that I use BOSH, which I'm trying to install, and no steps how to create it 'manually'), so that the create_vm can succeed. It brought up a VM for the warden, but now it hangs with ' Waiting for the agent on VM'. Some process got stuck in it after connecting to the container, and now it just waits.
Any ideas how to get this done from scratch? Am I even walking on the right path messing with this, or just I'm missing something really obvious?

Hi,

You might try setting up your hand-rolled BOSH-Lite with the
bosh-deployment repo (https://github.com/cloudfoundry/bosh-deployment) and
the v2 BOSH CLI (https://github.com/cloudfoundry/bosh-cli). I was able to
create one a few weeks ago as follows:

bosh interpolate \
$BOSH_DEPLOYMENT_DIR/bosh.yml \
--ops-file $BOSH_DEPLOYMENT_DIR/virtualbox/cpi.yml \
--ops-file $BOSH_DEPLOYMENT_DIR/bosh-lite.yml \
--vars-store $DEPLOYMENT_DIR/creds.yml \
-v director_name="Bosh Lite Director" \
-v internal_ip=192.168.50.6 \
-v internal_gw=192.168.50.1 \
-v internal_cidr=192.168.50.0/24 \
-v network_name=vboxnet0 \
--var-errs \
--var-errs-unused \
> $DEPLOYMENT_DIR/bosh-lite.yml

bosh create-env \
$DEPLOYMENT_DIR/bosh-lite.yml \
--state $DEPLOYMENT_DIR/state.json \
--vars-store $DEPLOYMENT_DIR/creds.yml

The env vars above are BOSH_DEPLOYMENT_DIR, the path to your local copy of
bosh-deployment repo, and DEPLOYMENT_DIR, the path to a directory for the
BOSH-Lite deployment manifest and state.

Note that this does not set up the Virtualbox VM with NAT. I had to make a
few extra changes with a separate ops-file to put the VM on a Virtualbox
NAT network, which time permitting I'll PR into the bosh-deployment repo
soon.

Best,
Eric

On Sun, Jan 8, 2017 at 11:03 AM, zr alt <zralt1(a)gmail.com> wrote:

I compiled garden-linux (as it had a makefile, garden-runc suggests that I
use BOSH, which I'm trying to install, and no steps how to create it
'manually'), so that the create_vm can succeed. It brought up a VM for the
warden, but now it hangs with ' Waiting for the agent on VM'. Some process
got stuck in it after connecting to the container, and now it just waits.
Any ideas how to get this done from scratch? Am I even walking on the
right path messing with this, or just I'm missing something really obvious?

Thanks Eric, I guess this should work if I were on a linux host, but unfortunately I'm running the test VM on virtualbox, and virtualbox is running on windows. There is a warden folder in the bosh-deployment repo, but it's missing the cpi.yml.
Would it be possible to use warden as a CPI ?

https://github.com/cloudfoundry/bosh-deployment/blob/master/docs/bosh-lite-on-vbox.md describes how to create a bosh lite vm via bosh create-env instead of vagrant.

Sent from my iPhone

Hi Dmitriy,

Thanks, following those instructions on a clean ubuntu host gives this result:

{create_stemcell [/root/.bosh/installations/cc1dfc37-eb0a-4813-651f-6115cecc08f0/tmp/stemcell-manager600120437/image map[infrastructure:vsphere os_distro:ubuntu architecture:x86_64 container_format:bare hypervisor:esxi name:bosh-vsphere-esxi-ubuntu-trusty-go_agent os_type:linux root_device_name:/dev/sda1 version:3312 disk:%!s(float64=3072) disk_format:ovf]]}
********************
[File System] 2017/01/15 12:31:54 DEBUG - Creating temp dir with prefix bosh-virtualbox-cpi-stemcell-upload
[Cmd Runner] 2017/01/15 12:31:54 DEBUG - Running command 'tar --no-same-owner -xzvf /root/.bosh/installations/cc1dfc37-eb0a-4813-651f-6115cecc08f0/tmp/stemcell-manager600120437/image -C /tmp/bosh-virtualbox-cpi-stemcell-upload227941448'
[Cmd Runner] 2017/01/15 12:31:57 DEBUG - Stdout: image.ovf
image.mf
image-disk1.vmdk
[Cmd Runner] 2017/01/15 12:31:57 DEBUG - Stderr:
[Cmd Runner] 2017/01/15 12:31:57 DEBUG - Successful: true (0)
[driver.LocalRunner] 2017/01/15 12:31:57 DEBUG - Execute 'sh -c' 'echo ~'
[Cmd Runner] 2017/01/15 12:31:57 DEBUG - Running command 'sh -c echo ~'
[Cmd Runner] 2017/01/15 12:31:57 DEBUG - Stdout: ~
[Cmd Runner] 2017/01/15 12:31:57 DEBUG - Stderr:
[Cmd Runner] 2017/01/15 12:31:57 DEBUG - Successful: true (0)
[File System] 2017/01/15 12:31:57 DEBUG - Remove all /tmp/bosh-virtualbox-cpi-stemcell-upload227941448
[json] 2017/01/15 12:31:57 DEBUG - CloudError response bytes
********************
{"result":null,"error":{"type":"Bosh::Clouds::CloudError","message":"Importing stemcell from '/root/.bosh/installations/cc1dfc37-eb0a-4813-651f-6115cecc08f0/tmp/stemcell-manager600120437/image': Creating stemcell parent: Failed to expand path '~\n\n'","ok_to_retry":false},"log":""}
********************
'

Full debug log:
https://gist.github.com/zralt/c4b4e4e3ccfc14a24c8f7cd1a5b7e2ac

Just for the record, I finally managed to get it deployed.
The bosh-cli that creates the final config uses the default vars of these virtualbox cpi properties:
bin_path:
description: Path to VBoxManage binary on the SSH-ed machine
default: "/usr/local/bin/VBoxManage"
store_dir:
description: Directory path to use to store stemcells, disks, and VMs managed by VirtualBox ('~' will be expanded)
default: "~/.bosh_virtualbox_cpi"

The "~/.bosh_virtualbox_cpi" gets wrongly evaluated somehow, and gets used as "~\n\n". Also VBoxManage is not under "/usr/local/bin/VBoxManage". After setting these properties, the deployment finished successfully.