Date   
Re: bosh-lite CI link

Dmitriy Kalinin
 

Bosh-lite pipeline got moved to the main bosh CI: https://main.bosh-ci.cf-app.com/pipelines/bosh-lite

I'm making changes to bosh-lite to use official garden-linux release, and I'll update the README shortly.

Sent from my iPhone

On Sep 10, 2015, at 6:31 AM, Matthew Sykes <matthew.sykes(a)gmail.com> wrote:

The bosh-lite README points to http://lite.bosh-ci.cf-app.com:8080 as the concourse instance for CI. Can someone please verify that link as I'm unable to access it.

Thanks.

--
Matthew Sykes
matthew.sykes(a)gmail.com

bosh-lite CI link

Matthew Sykes <matthew.sykes@...>
 

The bosh-lite README points to http://lite.bosh-ci.cf-app.com:8080 as the
concourse instance for CI. Can someone please verify that link as I'm
unable to access it.

Thanks.

--
Matthew Sykes
matthew.sykes(a)gmail.com

Virus alert: Returned mail: see transcript for details

postmaster@...
 

A email from cf-bosh(a)lists.cloudfoundry.org may contains virus. Discarded by system

Re: BOSH/CF package compilation time

Dmitriy Kalinin
 

Few bugs were fixed on 3068.

On Wed, Sep 9, 2015 at 9:53 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

Hi Dmitriy,

Is "export release" considered stable in 3042?

Thanks,
Mike

On Wed, Sep 9, 2015 at 9:12 PM, Dmitriy Kalinin <dkalinin(a)pivotal.io>
wrote:

We just recently introduced compiled releases feature. I have not had a
chance to document it yet, but in short it allows you to:

bosh export release cf/215 ubuntu-trusty/2068 on one Director

and later

bosh upload release release-cf-215-on...tgz on a second Director

No compilation will occur for a stemcell version 2068 on the second
Director when deploying cf.

On Wed, Sep 9, 2015 at 7:38 PM, Satya Thokachichu <tsnraju(a)yahoo.com>
wrote:

during bosh deploy..

Re: BOSH/CF package compilation time

Mike Youngstrom
 

Hi Dmitriy,

Is "export release" considered stable in 3042?

Thanks,
Mike

On Wed, Sep 9, 2015 at 9:12 PM, Dmitriy Kalinin <dkalinin(a)pivotal.io> wrote:

We just recently introduced compiled releases feature. I have not had a
chance to document it yet, but in short it allows you to:

bosh export release cf/215 ubuntu-trusty/2068 on one Director

and later

bosh upload release release-cf-215-on...tgz on a second Director

No compilation will occur for a stemcell version 2068 on the second
Director when deploying cf.

On Wed, Sep 9, 2015 at 7:38 PM, Satya Thokachichu <tsnraju(a)yahoo.com>
wrote:

during bosh deploy..

Re: BOSH/CF package compilation time

Satya Thokachichu
 

Thanks much..I will try and let you know how it goes

Re: Deploy CF Diego Release on AWS

王小锋 <zzuwxf at gmail.com...>
 

Sorry, It did mention AWS.

Not sure how /path/to/iaas-settings.yml should look like, any reference?
thanks.

iaas-settings.yml

spiff merge \
manifest-generation/misc-templates/aws-iaas-settings.yml \
/path/to/iaas-settings.yml \
> /tmp/iaas-settings.yml



2015-09-10 9:50 GMT+08:00 jinsenglin <jinsenglin(a)iii.org.tw>:

Actually It does.

-----Original Message-----
From: 王小锋 [mailto:zzuwxf(a)gmail.com]
Sent: Thursday, September 10, 2015 9:32 AM
To: Discussions about the Cloud Foundry BOSH project.
Subject: [cf-bosh] Re: Re: Deploy CF Diego Release on AWS

Thanks, but this document does not mention how to deploy diego to AWS
using bosh.


2015-09-10 9:08 GMT+08:00 jinsenglin <jinsenglin(a)iii.org.tw>:


https://github.com/cloudfoundry-incubator/diego-release

FYI


-----Original Message-----
From: 王小锋 [mailto:zzuwxf(a)gmail.com]
Sent: Wednesday, September 09, 2015 8:22 PM
To: Discussions about the Cloud Foundry BOSH project.
Subject: [cf-bosh] Deploy CF Diego Release on AWS

Hi, there


I've deployed CloudFoundry 212 on AWS using BOSH, now I want to
deploy Diego release using BOSH, but I have no idea how to generate the
diego manifest file, anyone has such experience? Any guide to share?

Best Regards!




Re: BOSH/CF package compilation time

Dmitriy Kalinin
 

We just recently introduced compiled releases feature. I have not had a
chance to document it yet, but in short it allows you to:

bosh export release cf/215 ubuntu-trusty/2068 on one Director

and later

bosh upload release release-cf-215-on...tgz on a second Director

No compilation will occur for a stemcell version 2068 on the second
Director when deploying cf.

On Wed, Sep 9, 2015 at 7:38 PM, Satya Thokachichu <tsnraju(a)yahoo.com> wrote:

during bosh deploy..

Re: Adding security groups in resource_pools instead of networks

Dmitriy Kalinin
 

It's a bit unclear to me if you are proposing to security groups feature in
the Director or in the OpenStack CPI specifically.

If in the OpenStack CPI, then I think it makes sense to pull in security
groups config into resource pool's cloud_properties section. For example:

resource_pools:
- name: my-fancy-web
stemcell: { ... }
cloud_properties:
instance_type: m3.xlarge
security_groups: [web]

- name: my-fancy-worker
stemcell: { ... }
cloud_properties:
instance_type: m3.xlarge
security_groups: [worker]

will assign web to VMs of type my-fancy-web and worker to VMs of type
my-fancy-worker. When resource pool's cloud_properties are changed, VMs
will be recreated with new config. Like you point out there is a case of
configure_networks, imho we can just raise an error in create_vm if both
networks' cloud_properties specifies security groups and resource pool cp's
specifies security_groups.

If you are proposing changes in the Director, then I think it gets a bit
more complicated. I'm not sure we have enough good usage patterns to figure
out a good abstraction *yet* (e.g. Azure has two different types of
security groups: network and vm, which imho makes a lot of sense). Also if
it's becoming a first class feature, may be BOSH at this point should be
creating security groups automatically with certain rules and may be even
with links generating these rules dynamically, etc.

Btw we are currently implementing first class support for AZs + links in
the Director, so all of the bosh-director core classes are going through
significant changes on global-net branch. Related to that I am actually
thinking we split up resource pool config into two pieces: vm type and
stemcell. With recent cloud config changes it would make it more sense to
keep stemcell os/version in the deployment manifest (just like releases)
and vm type with iaas specifics in the cloud config file.

On Wed, Sep 9, 2015 at 1:38 PM, john mcteague <john.mcteague(a)gmail.com>
wrote:

Would look forward to seeing this go through, it relates to a previous
thread from early this year -
https://groups.google.com/a/cloudfoundry.org/forum/#!topic/bosh-users/LJ2Kym6QCak.
It fell off my radar and have not had time to revisit.

On Wed, Sep 9, 2015 at 9:46 AM, Voelz, Marco <marco.voelz(a)sap.com> wrote:


Dear Boshers,

we currently encounter a problem which has been discussed briefly before
on the list [1]: Adding security groups in resource pools should be
possible. On a side note: we are dealing with openstack, so references
below might be openstack specific.

Here is a use-case: having machines in the same network, but with
different incoming/outgoing rules. Example: only the runners/DEAs of a CF
deployment should be able to access some service VMs. Currently, this means
that we have to have the same network configuration twice in our manifests,
the only difference being the set of security groups. I’d like to propose a
change to allow specifying them on resource_pool level and discuss some
implementation-specifics and impacts before writing code, so we are all on
the same page.

If we are introducing anything new, my assumption is that current
behavior of specifying security groups in networks should not break or
change. If you have a manifest specifying security groups, you probably
expect it to work also when a new feature is added.


Analysis of current state
* global default security groups can be specified when setting up your
director
* network security groups override those when specified for a deployment
* security groups are not a first-class concept. They are transported
through the entities known to bosh within the cloud_properties of a
network. Therefore, only methods dealing with the network entities obtained
from the manifest or director db actually know about them implicitly.

Concept proposal
* introduce ability to specify security groups for resource pools
* keep current behavior:
** if there are global default security groups only, use them.
** if there are network security groups, use them instead of anything
else. Don’t care about global groups or the new resource pool groups
** if there are resource pool groups AND no network security groups, use
them. Don’t care about global groups.
** probably remove security groups on networks at some point in time with
a heads-up to everyone currently using it. I have no idea if this is
feasible.

Implementation proposal
* create_vm and configure_networks of CPI seem to be the relevant calls
setting up the security groups: the former or creating a vm, the latter for
updating an existing one. Any changes done here would be CPI specific!
* adapting create_vm could be done straight forward: it is already using
the network_configurator to merge the security groups [2], and has access
to the resource_pool for the vm as well. We could simply add logic here to
take security groups within a resource pool into account.
* adapting configure_networks is more tricky: It gets a network spec and
compares the security groups in there with the ones currently present on a
vm [3]. It has no idea about the resource pool of that vm. The CPI is
called by the director’s network_updater [4] which gets initialized for a
specific instance and is called by the instance_updater [5].
* The instance entity combines all there is to know in terms of
configuration for a specific vm (e.g. network settings [6]), so this could
be the point to include the new feature

So, what could be changed now?
* introduce a new method security_groups on
bosh-director/lib/bosh/director/deployment_plan/instance.rb called
security_groups, providing information about security groups. If there are
secrurity groups on networks, return them, otherwise return security groups
defined on resource pools if there are any. Just like the desired behavior
we assumed above.
* adapt create_vm and configure_networks to accept security_groups as an
additional argument. Instead of having the CPIs extract security groups
from the network’s cloud_properties, take them from the argument and keep
the current logic of the methods.

What are your thoughts on this?

I would love to have the change isolated from the actual CPI coding, so
we don’t need to adapt all of them at the same time. However, this seems
like an API change might be in order, so I’m not sure on how to do it.

Given any form of agreement on how to proceed, we could provide a PR as a
further means for discussion. However, this change will impact the API, so
I wanted to get your feedback on this before actually implementing anything.

Warm regards
Marco

[1]
https://groups.google.com/a/cloudfoundry.org/forum/#!topic/bosh-users/LJ2Kym6QCak
[2]
https://github.com/cloudfoundry-incubator/bosh-openstack-cpi-release/blob/master/src/bosh_openstack_cpi/lib/cloud/openstack/cloud.rb#L226
[3]
https://github.com/cloudfoundry-incubator/bosh-openstack-cpi-release/blob/master/src/bosh_openstack_cpi/lib/cloud/openstack/cloud.rb#L397
[4]
https://github.com/cloudfoundry/bosh/blob/master/bosh-director/lib/bosh/director/instance_updater/network_updater.rb#L28
[5]
https://github.com/cloudfoundry/bosh/blob/master/bosh-director/lib/bosh/director/instance_updater.rb#L283-L286
[6]
https://github.com/cloudfoundry/bosh/blob/master/bosh-director/lib/bosh/director/deployment_plan/instance.rb#L186-L222

Re: BOSH/CF package compilation time

Satya Thokachichu
 

during bosh deploy..

Re: Deploy CF Diego Release on AWS

Jim
 

Actually It does.

-----Original Message-----
From: 王小锋 [mailto:zzuwxf(a)gmail.com]
Sent: Thursday, September 10, 2015 9:32 AM
To: Discussions about the Cloud Foundry BOSH project.
Subject: [cf-bosh] Re: Re: Deploy CF Diego Release on AWS

Thanks, but this document does not mention how to deploy diego to AWS using bosh.


2015-09-10 9:08 GMT+08:00 jinsenglin <jinsenglin(a)iii.org.tw>:


https://github.com/cloudfoundry-incubator/diego-release

FYI


-----Original Message-----
From: 王小锋 [mailto:zzuwxf(a)gmail.com]
Sent: Wednesday, September 09, 2015 8:22 PM
To: Discussions about the Cloud Foundry BOSH project.
Subject: [cf-bosh] Deploy CF Diego Release on AWS

Hi, there


I've deployed CloudFoundry 212 on AWS using BOSH, now I want to deploy Diego release using BOSH, but I have no idea how to generate the diego manifest file, anyone has such experience? Any guide to share?

Best Regards!

Re: Deploy CF Diego Release on AWS

王小锋 <zzuwxf at gmail.com...>
 

Thanks, but this document does not mention how to deploy diego to AWS using
bosh.

2015-09-10 9:08 GMT+08:00 jinsenglin <jinsenglin(a)iii.org.tw>:

https://github.com/cloudfoundry-incubator/diego-release

FYI

-----Original Message-----
From: 王小锋 [mailto:zzuwxf(a)gmail.com]
Sent: Wednesday, September 09, 2015 8:22 PM
To: Discussions about the Cloud Foundry BOSH project.
Subject: [cf-bosh] Deploy CF Diego Release on AWS

Hi, there


I've deployed CloudFoundry 212 on AWS using BOSH, now I want to deploy
Diego release using BOSH, but I have no idea how to generate the diego
manifest file, anyone has such experience? Any guide to share?

Best Regards!

Re: Deploy CF Diego Release on AWS

Jim
 

-----Original Message-----
From: 王小锋 [mailto:zzuwxf(a)gmail.com]
Sent: Wednesday, September 09, 2015 8:22 PM
To: Discussions about the Cloud Foundry BOSH project.
Subject: [cf-bosh] Deploy CF Diego Release on AWS

Hi, there


I've deployed CloudFoundry 212 on AWS using BOSH, now I want to deploy Diego release using BOSH, but I have no idea how to generate the diego manifest file, anyone has such experience? Any guide to share?

Best Regards!

Re: BOSH/CF package compilation time

Dmitriy Kalinin
 

Do you when using bosh-init or during bosh deploy?

On Wed, Sep 9, 2015 at 3:13 PM, Satya Thokachichu <tsnraju(a)yahoo.com> wrote:

Is there anyway to reduce CF/BOSH package compilation time during
deployments? It is taking roughly 15 minutes for each ..Just wanted to
check if we have any option to pre compile once and use it forever sorta
solution? Thanks for your help

BOSH/CF package compilation time

Satya Thokachichu
 

Is there anyway to reduce CF/BOSH package compilation time during deployments? It is taking roughly 15 minutes for each ..Just wanted to check if we have any option to pre compile once and use it forever sorta solution? Thanks for your help

Re: Adding security groups in resource_pools instead of networks

john mcteague
 

Would look forward to seeing this go through, it relates to a previous
thread from early this year -
https://groups.google.com/a/cloudfoundry.org/forum/#!topic/bosh-users/LJ2Kym6QCak.
It fell off my radar and have not had time to revisit.

On Wed, Sep 9, 2015 at 9:46 AM, Voelz, Marco <marco.voelz(a)sap.com> wrote:


Dear Boshers,

we currently encounter a problem which has been discussed briefly before
on the list [1]: Adding security groups in resource pools should be
possible. On a side note: we are dealing with openstack, so references
below might be openstack specific.

Here is a use-case: having machines in the same network, but with
different incoming/outgoing rules. Example: only the runners/DEAs of a CF
deployment should be able to access some service VMs. Currently, this means
that we have to have the same network configuration twice in our manifests,
the only difference being the set of security groups. I’d like to propose a
change to allow specifying them on resource_pool level and discuss some
implementation-specifics and impacts before writing code, so we are all on
the same page.

If we are introducing anything new, my assumption is that current behavior
of specifying security groups in networks should not break or change. If
you have a manifest specifying security groups, you probably expect it to
work also when a new feature is added.


Analysis of current state
* global default security groups can be specified when setting up your
director
* network security groups override those when specified for a deployment
* security groups are not a first-class concept. They are transported
through the entities known to bosh within the cloud_properties of a
network. Therefore, only methods dealing with the network entities obtained
from the manifest or director db actually know about them implicitly.

Concept proposal
* introduce ability to specify security groups for resource pools
* keep current behavior:
** if there are global default security groups only, use them.
** if there are network security groups, use them instead of anything
else. Don’t care about global groups or the new resource pool groups
** if there are resource pool groups AND no network security groups, use
them. Don’t care about global groups.
** probably remove security groups on networks at some point in time with
a heads-up to everyone currently using it. I have no idea if this is
feasible.

Implementation proposal
* create_vm and configure_networks of CPI seem to be the relevant calls
setting up the security groups: the former or creating a vm, the latter for
updating an existing one. Any changes done here would be CPI specific!
* adapting create_vm could be done straight forward: it is already using
the network_configurator to merge the security groups [2], and has access
to the resource_pool for the vm as well. We could simply add logic here to
take security groups within a resource pool into account.
* adapting configure_networks is more tricky: It gets a network spec and
compares the security groups in there with the ones currently present on a
vm [3]. It has no idea about the resource pool of that vm. The CPI is
called by the director’s network_updater [4] which gets initialized for a
specific instance and is called by the instance_updater [5].
* The instance entity combines all there is to know in terms of
configuration for a specific vm (e.g. network settings [6]), so this could
be the point to include the new feature

So, what could be changed now?
* introduce a new method security_groups on
bosh-director/lib/bosh/director/deployment_plan/instance.rb called
security_groups, providing information about security groups. If there are
secrurity groups on networks, return them, otherwise return security groups
defined on resource pools if there are any. Just like the desired behavior
we assumed above.
* adapt create_vm and configure_networks to accept security_groups as an
additional argument. Instead of having the CPIs extract security groups
from the network’s cloud_properties, take them from the argument and keep
the current logic of the methods.

What are your thoughts on this?

I would love to have the change isolated from the actual CPI coding, so we
don’t need to adapt all of them at the same time. However, this seems like
an API change might be in order, so I’m not sure on how to do it.

Given any form of agreement on how to proceed, we could provide a PR as a
further means for discussion. However, this change will impact the API, so
I wanted to get your feedback on this before actually implementing anything.

Warm regards
Marco

[1]
https://groups.google.com/a/cloudfoundry.org/forum/#!topic/bosh-users/LJ2Kym6QCak
[2]
https://github.com/cloudfoundry-incubator/bosh-openstack-cpi-release/blob/master/src/bosh_openstack_cpi/lib/cloud/openstack/cloud.rb#L226
[3]
https://github.com/cloudfoundry-incubator/bosh-openstack-cpi-release/blob/master/src/bosh_openstack_cpi/lib/cloud/openstack/cloud.rb#L397
[4]
https://github.com/cloudfoundry/bosh/blob/master/bosh-director/lib/bosh/director/instance_updater/network_updater.rb#L28
[5]
https://github.com/cloudfoundry/bosh/blob/master/bosh-director/lib/bosh/director/instance_updater.rb#L283-L286
[6]
https://github.com/cloudfoundry/bosh/blob/master/bosh-director/lib/bosh/director/deployment_plan/instance.rb#L186-L222

Deploy CF Diego Release on AWS

王小锋 <zzuwxf at gmail.com...>
 

Hi, there

I've deployed CloudFoundry 212 on AWS using BOSH, now I want to deploy
Diego release using BOSH, but I have no idea how to generate the diego
manifest file, anyone has such experience? Any guide to share?

Best Regards!

Adding security groups in resource_pools instead of networks

Marco Voelz
 

Dear Boshers,

we currently encounter a problem which has been discussed briefly before on the list [1]: Adding security groups in resource pools should be possible. On a side note: we are dealing with openstack, so references below might be openstack specific.

Here is a use-case: having machines in the same network, but with different incoming/outgoing rules. Example: only the runners/DEAs of a CF deployment should be able to access some service VMs. Currently, this means that we have to have the same network configuration twice in our manifests, the only difference being the set of security groups. I’d like to propose a change to allow specifying them on resource_pool level and discuss some implementation-specifics and impacts before writing code, so we are all on the same page.

If we are introducing anything new, my assumption is that current behavior of specifying security groups in networks should not break or change. If you have a manifest specifying security groups, you probably expect it to work also when a new feature is added.


Analysis of current state
* global default security groups can be specified when setting up your director
* network security groups override those when specified for a deployment
* security groups are not a first-class concept. They are transported through the entities known to bosh within the cloud_properties of a network. Therefore, only methods dealing with the network entities obtained from the manifest or director db actually know about them implicitly.

Concept proposal
* introduce ability to specify security groups for resource pools
* keep current behavior:
** if there are global default security groups only, use them.
** if there are network security groups, use them instead of anything else. Don’t care about global groups or the new resource pool groups
** if there are resource pool groups AND no network security groups, use them. Don’t care about global groups.
** probably remove security groups on networks at some point in time with a heads-up to everyone currently using it. I have no idea if this is feasible.

Implementation proposal
* create_vm and configure_networks of CPI seem to be the relevant calls setting up the security groups: the former or creating a vm, the latter for updating an existing one. Any changes done here would be CPI specific!
* adapting create_vm could be done straight forward: it is already using the network_configurator to merge the security groups [2], and has access to the resource_pool for the vm as well. We could simply add logic here to take security groups within a resource pool into account.
* adapting configure_networks is more tricky: It gets a network spec and compares the security groups in there with the ones currently present on a vm [3]. It has no idea about the resource pool of that vm. The CPI is called by the director’s network_updater [4] which gets initialized for a specific instance and is called by the instance_updater [5].
* The instance entity combines all there is to know in terms of configuration for a specific vm (e.g. network settings [6]), so this could be the point to include the new feature

So, what could be changed now?
* introduce a new method security_groups on bosh-director/lib/bosh/director/deployment_plan/instance.rb called security_groups, providing information about security groups. If there are secrurity groups on networks, return them, otherwise return security groups defined on resource pools if there are any. Just like the desired behavior we assumed above.
* adapt create_vm and configure_networks to accept security_groups as an additional argument. Instead of having the CPIs extract security groups from the network’s cloud_properties, take them from the argument and keep the current logic of the methods.

What are your thoughts on this?

I would love to have the change isolated from the actual CPI coding, so we don’t need to adapt all of them at the same time. However, this seems like an API change might be in order, so I’m not sure on how to do it.

Given any form of agreement on how to proceed, we could provide a PR as a further means for discussion. However, this change will impact the API, so I wanted to get your feedback on this before actually implementing anything.

Warm regards
Marco

[1] https://groups.google.com/a/cloudfoundry.org/forum/#!topic/bosh-users/LJ2Kym6QCak
[2] https://github.com/cloudfoundry-incubator/bosh-openstack-cpi-release/blob/master/src/bosh_openstack_cpi/lib/cloud/openstack/cloud.rb#L226
[3] https://github.com/cloudfoundry-incubator/bosh-openstack-cpi-release/blob/master/src/bosh_openstack_cpi/lib/cloud/openstack/cloud.rb#L397
[4] https://github.com/cloudfoundry/bosh/blob/master/bosh-director/lib/bosh/director/instance_updater/network_updater.rb#L28
[5] https://github.com/cloudfoundry/bosh/blob/master/bosh-director/lib/bosh/director/instance_updater.rb#L283-L286
[6] https://github.com/cloudfoundry/bosh/blob/master/bosh-director/lib/bosh/director/deployment_plan/instance.rb#L186-L222

Re: Started updating job api_z1 > api_z1/0. Failed: `api_z1/0' is not running after update

Parthiban Annadurai <senjiparthi@...>
 

Great Ramesh..

On 8 September 2015 at 10:48, Ramesh Sambandan <rsamban(a)gmail.com> wrote:


I finally figured it out. It is my bad, I missed the
properties.uaa.jwt.signing_key/verification_key in my manifest. It is
something I thought I took care of it, but must have overwritten in
subsequent edits.

The clue came from looking at the log files in router_api in api_z1 vm
(/var/vcap/sys/log/router* folder)

And I successfully deployed cloud foundry in vsphere. :):)

Relation between Network property in Resource pool property and Network property in Actual Job block

Ronak Banka
 

Hello All,

I have a resource pool , let say small_z1

- name: small_z1
network: cf1
stemcell: stemcell-xyz
cloud_properties:
instance_type: m1.small
availability_zone: zone1

and a Job , router having two networks assigned to it

- name: router
instances: 1
networks:
- name: router_internal
default: [dns, gateway]
static_ips:
- xy.xy.xy.xy
- name: router_external
static_ips:
- yz.yz.yz.yz
gateway: yy.yy.yy.yy
networks:
apps: router_internal
management: router_internal
resource_pool: small_z1

With these properties there are no issues anywhere.

what is the network property in resource pool responsible for, if the
created job networks and not linked to the one in pool??

Regards,
Ronak




--
View this message in context: http://cf-bosh.70367.x6.nabble.com/Relation-between-Network-property-in-Resource-pool-property-and-Network-property-in-Actual-Job-block-tp649.html
Sent from the CF BOSH mailing list archive at Nabble.com.