Date   

Re: [cf-dev] BOSH Links: Why and How

Dr Nic Williams
 

Amit, this doc has been awesomely helpful to me over the last week or so. I've been steadily upgrading bosh releases. I like them & the deployment manifests a lot more.


________________________________
From: Amit Gupta <agupta(a)pivotal.io>
Sent: Thursday, March 30, 2017 8:35:53 AM
To: Discussions about the Cloud Foundry BOSH project.; Discussions about Cloud Foundry projects and the system overall.
Subject: [cf-dev] BOSH Links: Why and How

Hi all,

"BOSH Links" is a feature which simplifies how data is shared between BOSH-deployed jobs that need to collaborate with one another (e.g. a web server and its backing database). Like many BOSH things, the whole "BOSH Links" thing can seem counter-intuitive at first, and it may not be clear why things are the way they are. This note linked below hopes to show some of the powerful benefits of BOSH Links, and provide examples and explanations that make things more intuitive.

Hope folks find it useful!

https://gist.github.com/Amit-PivotalLabs/c39528248b8cdc4ba8e347f8aa68abb6

Best,
Amit


BOSH Links: Why and How

Amit Kumar Gupta
 

Hi all,

"BOSH Links" is a feature which simplifies how data is shared between
BOSH-deployed jobs that need to collaborate with one another (e.g. a web
server and its backing database). Like many BOSH things, the whole "BOSH
Links" thing can seem counter-intuitive at first, and it may not be clear
why things are the way they are. This note linked below hopes to show some
of the powerful benefits of BOSH Links, and provide examples and
explanations that make things more intuitive.

Hope folks find it useful!

https://gist.github.com/Amit-PivotalLabs/c39528248b8cdc4ba8e347f8aa68abb6

Best,
Amit


Re: VMs are not added to AWS ELB target groups

Dmitriy Kalinin
 

We didnt yet publish cpi release with AWS v2 integration (includes ALBs).
You are welcome to build a local copy (bosh create release --with-tarball)
from github.com/cloudfoundry-incubator/bosh-aws-cpi-release. We are
planning to promote release with AWS v2 shortly pending stability testing
in our pipelines. Feel free to hop into #bosh slack channel for more
details.

On Wed, Mar 29, 2017 at 9:27 AM, Danny Berger <dberger(a)pivotal.io> wrote:

Instead of `lbtargetgroups`, try `lb_target_groups` - the markup
formatting on the bosh.io docs page is rendering poorly.

Danny

On Wed, Mar 29, 2017 at 9:10 AM, Tsvetan Valchev <bgtsetso(a)gmail.com>
wrote:

Hi,

In my cloud config descriptor in the vm_types section I have specified
that I want the created via bosh deployment VM instances to be added to a
specific load balancer target group using the lbtargetgroups attribute,
e.g.:
vm_types:
- name: vm_myserver
cloud_properties: { instance_type: m4.xlarge, lbtargetgroups:
[my_target_group] }

Although the balancer and target groups are already created, the newly
created VMs are not added to the target group and I have to do this
manually via the aws console.
Do you have any ideas?

Thanks,
Tsvetan


--
Danny Berger


Re: VMs are not added to AWS ELB target groups

Danny Berger
 

Instead of `lbtargetgroups`, try `lb_target_groups` - the markup formatting
on the bosh.io docs page is rendering poorly.

Danny

On Wed, Mar 29, 2017 at 9:10 AM, Tsvetan Valchev <bgtsetso(a)gmail.com> wrote:

Hi,

In my cloud config descriptor in the vm_types section I have specified
that I want the created via bosh deployment VM instances to be added to a
specific load balancer target group using the lbtargetgroups attribute,
e.g.:
vm_types:
- name: vm_myserver
cloud_properties: { instance_type: m4.xlarge, lbtargetgroups:
[my_target_group] }

Although the balancer and target groups are already created, the newly
created VMs are not added to the target group and I have to do this
manually via the aws console.
Do you have any ideas?

Thanks,
Tsvetan
--
Danny Berger


VMs are not added to AWS ELB target groups

Tsvetan Valchev
 

Hi,

In my cloud config descriptor in the vm_types section I have specified that I want the created via bosh deployment VM instances to be added to a specific load balancer target group using the lbtargetgroups attribute, e.g.:
vm_types:
- name: vm_myserver
cloud_properties: { instance_type: m4.xlarge, lbtargetgroups: [my_target_group] }

Although the balancer and target groups are already created, the newly created VMs are not added to the target group and I have to do this manually via the aws console.
Do you have any ideas?

Thanks,
Tsvetan


Creating & Deploying BOSH Releases on Windows

Jonathan Stockley
 

We have been using bosh for some time with Ubuntu stemcells on vSphere.
We have recently setup Diego cells on Windows using Windows stemcell.
Is this the only thing that can be deployed on Windows with bosh or
can we build a bosh release of any application (for example MySQL or RabbitMQ) using the bosh CLI and then deploy it on Windows?
I appreciate that any scripts would probably have to be powershelgl rather than bash (although…)

Any pointer to docs or examples would be much appreciated.

Cheers,
Jo


Re: bosh-init install question

Marco Voelz
 

Correction: I didn't mean manifest templates, I meant job templates of the installed releases, which are .erb templates, even for releases written in different languages. See e.g. something like https://github.com/cppforlife/bosh-virtualbox-cpi-release/blob/master/jobs/virtualbox_cpi/templates/cpi.json.erb in the virtualbox cpi, which is written in golang.

Warm regards
Marco

From: "Voelz, Marco" <marco.voelz(a)sap.com>
Date: Friday, 17 March 2017 at 11:04
To: "Discussions about the Cloud Foundry BOSH project." <cf-bosh(a)lists.cloudfoundry.org>
Subject: Re: [cf-bosh] Re: bosh-init install question

Hi Lynn,

The ruby dependencies are necessary for bosh-init, because it renders manifest templates with ruby. See https://github.com/cloudfoundry/bosh-init/blob/master/templatescompiler/erbrenderer/template_evaluation_context_rb.go for all its beauty.

Warm regards
Marco


From: Dr Nic Williams <drnicwilliams(a)gmail.com>
Reply-To: "Discussions about the Cloud Foundry BOSH project." <cf-bosh(a)lists.cloudfoundry.org>
Date: Friday, 17 March 2017 at 06:18
To: "Discussions about the Cloud Foundry BOSH project." <cf-bosh(a)lists.cloudfoundry.org>
Subject: [cf-bosh] Re: bosh-init install question

bosh-init (or the new bosh2 cli's create-env command) will download some yet-to-be-compiled modules called CPIs. Some CPIs are written in Ruby - the ruby itself is also downloaded automatically and installed in a special private place; but Ruby has some system dependencies to work. Instead of also downloading those as well, the CPI module will just require that you have them pre installed.

On Thu, Mar 16, 2017 at 8:34 PM -0700, "Lynn Lin" <lynn.lin(a)emc.com<mailto:lynn.lin(a)emc.com>> wrote:

All,

I am on the page http://bosh.io/docs/install-bosh-init.html and am wondering what's purpose for this section ? Is this for ruby2?



Depending on your platform install following packages:



Ubuntu Trusty



$ sudo apt-get install -y build-essential zlibc zlib1g-dev ruby ruby-dev openssl libxslt-dev libxml2-dev libssl-dev libreadline6 libreadline6-dev libyaml-dev libsqlite3-dev sqlite3


Re: bosh-init install question

Marco Voelz
 

Hi Lynn,

The ruby dependencies are necessary for bosh-init, because it renders manifest templates with ruby. See https://github.com/cloudfoundry/bosh-init/blob/master/templatescompiler/erbrenderer/template_evaluation_context_rb.go for all its beauty.

Warm regards
Marco


From: Dr Nic Williams <drnicwilliams(a)gmail.com>
Reply-To: "Discussions about the Cloud Foundry BOSH project." <cf-bosh(a)lists.cloudfoundry.org>
Date: Friday, 17 March 2017 at 06:18
To: "Discussions about the Cloud Foundry BOSH project." <cf-bosh(a)lists.cloudfoundry.org>
Subject: [cf-bosh] Re: bosh-init install question

bosh-init (or the new bosh2 cli's create-env command) will download some yet-to-be-compiled modules called CPIs. Some CPIs are written in Ruby - the ruby itself is also downloaded automatically and installed in a special private place; but Ruby has some system dependencies to work. Instead of also downloading those as well, the CPI module will just require that you have them pre installed.

On Thu, Mar 16, 2017 at 8:34 PM -0700, "Lynn Lin" <lynn.lin(a)emc.com<mailto:lynn.lin(a)emc.com>> wrote:

All,

I am on the page http://bosh.io/docs/install-bosh-init.html and am wondering what's purpose for this section ? Is this for ruby2?



Depending on your platform install following packages:



Ubuntu Trusty



$ sudo apt-get install -y build-essential zlibc zlib1g-dev ruby ruby-dev openssl libxslt-dev libxml2-dev libssl-dev libreadline6 libreadline6-dev libyaml-dev libsqlite3-dev sqlite3


Re: bosh-init install question

Dr Nic Williams
 

bosh-init (or the new bosh2 cli's create-env command) will download some yet-to-be-compiled modules called CPIs. Some CPIs are written in Ruby - the ruby itself is also downloaded automatically and installed in a special private place; but Ruby has some system dependencies to work. Instead of also downloading those as well, the CPI module will just require that you have them pre installed.

On Thu, Mar 16, 2017 at 8:34 PM -0700, "Lynn Lin" <lynn.lin(a)emc.com> wrote:










All,
I am on the page http://bosh.io/docs/install-bosh-init.html and am wondering what's purpose for this section ? Is this for ruby2?

Depending on your platform install following packages:

Ubuntu Trusty

$ sudo apt-get install -y build-essential zlibc zlib1g-dev ruby ruby-dev openssl libxslt-dev libxml2-dev libssl-dev libreadline6 libreadline6-dev libyaml-dev libsqlite3-dev sqlite3


bosh-init install question

Lynn Lin
 

All,
I am on the page http://bosh.io/docs/install-bosh-init.html and am wondering what's purpose for this section ? Is this for ruby2?

Depending on your platform install following packages:

Ubuntu Trusty

$ sudo apt-get install -y build-essential zlibc zlib1g-dev ruby ruby-dev openssl libxslt-dev libxml2-dev libssl-dev libreadline6 libreadline6-dev libyaml-dev libsqlite3-dev sqlite3


Re: 2017 CF Summit Silicon Valley Contributor Code

Gwenn Etourneau
 

Thanks !
Good to know.


On Wed, Mar 8, 2017 at 11:36 PM, Chip Childers <cchilders(a)cloudfoundry.org>
wrote:

We can consider cloudfoundry-community as a contributor, sure! ;-)
Technically the Foundation doesn't actually own that GH org, but we love
what's happening there!

On Wed, Mar 8, 2017 at 9:35 AM Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

Chip,

When you means "this project", are you including cloudfoundry-community ?
of just cloudfoundry repository ?


Thanks
Gwenn

On Thu, Jan 19, 2017 at 1:17 AM, Chip Childers <
cchilders(a)cloudfoundry.org> wrote:

Hi all!

On today's CAB call, Dr. Max asked if this thread could be bumped so
everyone is reminded to see it.

Contributors: Don't forget to register!

-chip

On Wed, Nov 30, 2016 at 12:22 PM Chip Childers <
cchilders(a)cloudfoundry.org> wrote:

Hi all!

Some of you may have noticed that the registration is now open for the
upcoming CF Summit in Silicon Valley, and we are offering free passes
for contributors to the project again.

This code can be used by anyone that is a contributor to a Cloud Foundry
or BOSH project. We consider contributions to be project leads, dedicated
committers or even if you have sent in a pull request to one of the
projects.

Use of the code is on the honor system... ;-)

https://www.cloudfoundry.org/summit2017/?utm_source=flash&
utm_campaign=summit_2017_sv&utm_medium=eml&utm_term=cloud%
20foundry%20summit

Code: CFSV17CONT
Feel free to reach out to me or to events(a)cloudfoundry.org if you have
any questions.

See you there!
--
Chip Childers
CTO, Cloud Foundry Foundation
1.267.250.0815 <(267)%20250-0815>

--
Chip Childers
CTO, Cloud Foundry Foundation
1.267.250.0815 <(267)%20250-0815>


--
Chip Childers
CTO, Cloud Foundry Foundation
1.267.250.0815 <(267)%20250-0815>


Re: 2017 CF Summit Silicon Valley Contributor Code

Chip Childers
 

We can consider cloudfoundry-community as a contributor, sure! ;-)
Technically the Foundation doesn't actually own that GH org, but we love
what's happening there!

On Wed, Mar 8, 2017 at 9:35 AM Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

Chip,

When you means "this project", are you including cloudfoundry-community ?
of just cloudfoundry repository ?


Thanks
Gwenn

On Thu, Jan 19, 2017 at 1:17 AM, Chip Childers <cchilders(a)cloudfoundry.org
wrote:
Hi all!

On today's CAB call, Dr. Max asked if this thread could be bumped so
everyone is reminded to see it.

Contributors: Don't forget to register!

-chip

On Wed, Nov 30, 2016 at 12:22 PM Chip Childers <cchilders(a)cloudfoundry.org>
wrote:

Hi all!

Some of you may have noticed that the registration is now open for the
upcoming CF Summit in Silicon Valley, and we are offering free passes for
contributors to the project again.

This code can be used by anyone that is a contributor to a Cloud Foundry
or BOSH project. We consider contributions to be project leads, dedicated
committers or even if you have sent in a pull request to one of the
projects.

Use of the code is on the honor system... ;-)


https://www.cloudfoundry.org/summit2017/?utm_source=flash&utm_campaign=summit_2017_sv&utm_medium=eml&utm_term=cloud%20foundry%20summit


Code: CFSV17CONT
Feel free to reach out to me or to events(a)cloudfoundry.org if you have
any questions.

See you there!
--
Chip Childers
CTO, Cloud Foundry Foundation
1.267.250.0815 <(267)%20250-0815>

--
Chip Childers
CTO, Cloud Foundry Foundation
1.267.250.0815 <(267)%20250-0815>


--
Chip Childers
CTO, Cloud Foundry Foundation
1.267.250.0815


Re: 2017 CF Summit Silicon Valley Contributor Code

Gwenn Etourneau
 

Chip,

When you means "this project", are you including cloudfoundry-community ?
of just cloudfoundry repository ?


Thanks
Gwenn

On Thu, Jan 19, 2017 at 1:17 AM, Chip Childers <cchilders(a)cloudfoundry.org>
wrote:

Hi all!

On today's CAB call, Dr. Max asked if this thread could be bumped so
everyone is reminded to see it.

Contributors: Don't forget to register!

-chip

On Wed, Nov 30, 2016 at 12:22 PM Chip Childers <cchilders(a)cloudfoundry.org>
wrote:

Hi all!

Some of you may have noticed that the registration is now open for the
upcoming CF Summit in Silicon Valley, and we are offering free passes
for contributors to the project again.

This code can be used by anyone that is a contributor to a Cloud Foundry
or BOSH project. We consider contributions to be project leads, dedicated
committers or even if you have sent in a pull request to one of the
projects.

Use of the code is on the honor system... ;-)

https://www.cloudfoundry.org/summit2017/?utm_source=flash&
utm_campaign=summit_2017_sv&utm_medium=eml&utm_term=cloud%
20foundry%20summit

Code: CFSV17CONT
Feel free to reach out to me or to events(a)cloudfoundry.org if you have
any questions.

See you there!
--
Chip Childers
CTO, Cloud Foundry Foundation
1.267.250.0815 <(267)%20250-0815>
--
Chip Childers
CTO, Cloud Foundry Foundation
1.267.250.0815 <(267)%20250-0815>


Re: BOSH Stemcells and vulnerability scanning

Daniel Jones
 

Just to chip in, we've helped folks consume updates for their CF
deployments that averaged out at a new *thing* (stemcell, release) every 4
days. Having your entire platform pipelined with something like Concourse
makes a massive difference - if you're not used to this pace of change and
are trying to do things manually, you'll never keep up. It shouldn't take
more than hours to get a new stemcell tested and into production.

Regards,
Daniel Jones - CTO
+44 (0)79 8000 9153
@DanielJonesEB <https://twitter.com/DanielJonesEB>
*EngineerBetter* Ltd <http://www.engineerbetter.com> - UK Cloud Foundry
Specialists

On 2 March 2017 at 06:02, James Bayer <jbayer(a)pivotal.io> wrote:

pivotal deploys updated stemcells regularly to PWS. high and critical CVEs
have a 48hr goal. we catch up on lows and mediums generally approximately
once per month.

On Wed, Mar 1, 2017 at 5:48 PM, Jonathan Stockley <jstockle(a)opentext.com>
wrote:

Hi, before deploying/upgrading a stemcell in production our security
group runs vulnerability scans on our staging deployments.
The problem is that by the time we get the stemcell into staging (about a
4-6 weeks), they have updated the vulnerability database and then there
scan find new issues.

How often are people upgrading stemcells in production?
How do you handle vulnerability scanning of BOSH deployed apps?
How about run.pivotal.io? How do they address this?

Thanks,
Jo


--
Thank you,

James Bayer


Re: BOSH Stemcells and vulnerability scanning

James Bayer
 

pivotal deploys updated stemcells regularly to PWS. high and critical CVEs
have a 48hr goal. we catch up on lows and mediums generally approximately
once per month.

On Wed, Mar 1, 2017 at 5:48 PM, Jonathan Stockley <jstockle(a)opentext.com>
wrote:

Hi, before deploying/upgrading a stemcell in production our security group
runs vulnerability scans on our staging deployments.
The problem is that by the time we get the stemcell into staging (about a
4-6 weeks), they have updated the vulnerability database and then there
scan find new issues.

How often are people upgrading stemcells in production?
How do you handle vulnerability scanning of BOSH deployed apps?
How about run.pivotal.io? How do they address this?

Thanks,
Jo


--
Thank you,

James Bayer


BOSH Stemcells and vulnerability scanning

Jonathan Stockley
 

Hi, before deploying/upgrading a stemcell in production our security group runs vulnerability scans on our staging deployments.
The problem is that by the time we get the stemcell into staging (about a 4-6 weeks), they have updated the vulnerability database and then there scan find new issues.

How often are people upgrading stemcells in production?
How do you handle vulnerability scanning of BOSH deployed apps?
How about run.pivotal.io? How do they address this?

Thanks,
Jo


Re: Migrate some deployments from one bosh to another

Suren R
 

If your deployment have zone1 and zone 2 jobs, the best, safest and
easiest way to achieve this is to remove Z2 job in manifest and perform
deploy in old bosh. Deploy Z2 job in new bosh. Repeat this for Zone 1 jobs.


On Mon, Feb 27, 2017 at 2:55 PM, Grifalconi, Michael <
michael.grifalconi(a)sap.com> wrote:

Hello all,



I would just bring to your attention the discussion I opened on GitHub
https://github.com/cloudfoundry/bosh/issues/1601



I am looking for the best way to achieve that or the most valid reason why
I should not even try it :)



Thanks and regards,

Michael


Re: Documentation Update AWS

Leandro David Cacciagioni
 

Thanks a lot for the answer, and yes the pain points are those ones...
Those are normally not pain points for someone who has already deployed CF
for a long time... But for new comers they are quite huge problems, that
many times lead to a more difficult adoption of CF among new comers... It
makes CF looks a little bit unprofessional. It is also a pain point to
explain to your coworkers why you do things that are not in the docs.
Once again thanks a lot for the new tools, I'll take a look to them this
week.

Cheers,
Leandro.-

2017-02-25 2:43 GMT+01:00 David Sabeti <dsabeti(a)pivotal.io>:

Hey Leandro,

Thanks for this feedback. One of the things the CF team has been working
on is better tooling around deploying Cloud Foundry, including IaaS setup.
A lot of this work is still in flight, which is why the CF docs don't have
any information about it yet; when the work is complete, there will
definitely be an overhaul of the docs to explain how to use the new tools.

In your outline, I think I'm noticing the following concerns:
1. How do I set up an AWS account so that I can deploy a CF? What are the
requirements (for example, quotas) for my account?
2. Can we default to using the most up-to-date instances?
3. Once my account is set up, how do I deploy CF (with Diego)?
4. How do I understand what I'm deploying and how I can modify it on my
own?

Let me know if I've understood your questions correctly: I'll make sure
that we take this feedback into account when we build the docs for the new
tools.

In the meantime, I'm happy to point people to the new tools, provided that
I emphasize that* these tools are not ready for production and still
undergoing active development. You should use these at your own risk.*
- To set up IaaS for CF: Take a look a "bbl" (short for bosh-bootloader
<https://github.com/cloudfoundry/bosh-bootloader>). This is tool that
does all the work to get you to a bosh director -- it create IaaS resources
like VPCs, NAT boxes, etc., and the creates a BOSH director that you can
target.
- To deploy CF: Take a look at cf-deployment
<https://github.com/cloudfoundry/cf-deployment>. It contains a BOSH
manifest for deploying Cloud Foundry (with Diego!). It also uses new BOSH
features to make manifest generation a good deal simpler. This will be the
future for how to deploy CF when we deprecated cf-release.

Again, these aren't ready for prime time yet, but it should be good for
development, testing, and experimentation -- not to mention that we'd love
feedback on this from the community. Feel free to follow up if you have any
questions.

David Sabeti
Product Manager, CF Release Integration Team


On Thu, Feb 23, 2017 at 12:04 AM Leandro David Cacciagioni <
leandro.21.2008(a)gmail.com> wrote:

Guys,

I have had a few issues over the last weeks deploying Cloud Foundry to
AWS and I think that maybe it will be nice to:

- Update the docs to be able to deploy without using the bosh aws gem
(It's broken at least with ruby 2.3.3/rbenv & Fedora Linux 25).
- Also is not nice that this GEM deletes everything when you try
to tear down all that was created, it not only delete what was create by
it, but everything else in the account if possible.
- Update the docs specifying the minimum AWS quotas required for
minimal HA deployment.
- Update the stub sample to use m4 instances, since this has been in
AWS for quite some time and they are already in most of the AWS regions.
- Beyond AWS it would be nice a detailed step by step guide to deploy
CF with Diego enabled, since for those who have never touched CF is a
little difficult to understand all the moving parts of CF.
- In my case I was giving more than a week of training, at least
one hour per day until they catch up the concepts and understand the basics
of how CF works (Forget about make them modify a bosh deployment to match a
Diego deployment if that's not in the docs)

Hope this help us all.

Cheers, Leandro.-


Re: Migrate some deployments from one bosh to another

Dr Nic Williams
 

I guess technically a deployment is just a bunch of rows in the bosh Postgres database; if you move them them to another bosh then that bosh will think it owns those VMs.
At the same time, initially the VMs will still be listening to the original bosh nats & blobstore.
But perhaps on the new bosh you do a recreate of all vms in the deployment and it will replace all the settings on each vms so that they call home to the new bosh.
Somewhere above you'd remove the same rows from the old bosh director Postgres DB 
New bosh would need the blobs - the releases and compiled packages - but I guess they could be recreated if you did a "bosh deploy" rather than a "bosh recreate".
Conceptually it should be possible - it's just rows in a database and some blobs; and some VMs who need an attitude adjustment about who's the boss.
Dr Nic

On Mon, Feb 27, 2017 at 7:25 PM +1000, "Grifalconi, Michael" <michael.grifalconi(a)sap.com> wrote:






















Hello all,


 


I would just bring to your attention the discussion I opened on GitHub
https://github.com/cloudfoundry/bosh/issues/1601



 


I am looking for the best way to achieve that or the most valid reason why I should not even try it  :)


 


Thanks and regards,


Michael


Migrate some deployments from one bosh to another

Grifalconi, Michael
 

Hello all,

I would just bring to your attention the discussion I opened on GitHub https://github.com/cloudfoundry/bosh/issues/1601

I am looking for the best way to achieve that or the most valid reason why I should not even try it :)

Thanks and regards,
Michael