Date   

Re: Update Parallelization in Cloud Foundry

Marco Voelz
 

Does NATS also need to come up before any of the other components?

On 07/03/16 21:16, "Amit Gupta" <agupta(a)pivotal.io<mailto:agupta(a)pivotal.io>> wrote:

Hey Omar,

You can set the "serial" property at the global level of a deployment (you can think of it as setting a default for all jobs), and then override it at the individual job levels. You will want the consul server jobs to be deployed first, with serial: true, and max_in_flight: 1. The important thing here is, if you have more than one server in your consul cluster, they need to come up one at a time to ensure the cluster orchestration goes smoothly. The same is true if your etcd cluster has more than one server in it. If you're using the postgres job for CCDB and/or UAADB (instead of some external database), then you will want the postgres job to come up before CC and/or UAA. Similarly, if you're using the provided blobstore job instead of an external blobstore, you'll want it up before CC comes up.

You might be able to get away with parallelizing some of the things above. E.g. if you bring the CC and blobstore up at the same time, CC might fail to start for a while until Blobstore comes up, and then CC might successfully start up. Monit also generally keeps retrying even after BOSH gives up. So your deploy might fail but later on, you might see everything up and running.

Cheers,
Amit

On Mon, Mar 7, 2016 at 5:54 AM, Omar Elazhary <omazhary(a)gmail.com<mailto:omazhary(a)gmail.com>> wrote:
Hello everyone,

I know it is possible to update and redeploy components in parallel in cloud foundry by setting the "serial" property in the deployment manifest to "false". However, is such a thing recommended? Are there particular job dependencies that I need to pay attention to?

Regards,
Omar


how to debug "BuildpackCompileFailed" issue?

Ning Fu
 

Hi,

Does anyone know how to debug "BuildpackCompileFailed" issue?
When I push a ruby app:
========================
...
Done uploadingOK
Starting app happy in org funorg / space development as funcloud...
FAILEDBuildpackCompileFailed
TIP: use 'cf logs happy --recent' for more
informationPivotals-iMac:happy-root-route-app pivotal$ cf logs happy
--recent
Connected, dumping recent logs for app happy in org funorg / space
development as funcloud...
========================
But I got nothing from "cf logs happy --recent".
It is "ruby '2.2.2'" in Gemfile, and my ruby build back is cached-1.6.11.
I've also tried "bundle package --all" before I push.

Any suggestions? Doesn't ruby build pack provide any logs?

Thanks,
Ning


Re: CF CAB call for March is next Wednesday March 9th, 2016 @ 8a PDT

Michael Maximilien
 

Final reminder to all. Please note that we will be implementing new approach to CAN call based on survey results which we will review briefly...

Also, if you are a PM, please still update your status section of the agenda doc. This is for all to read on their own time and for posterity.

Thanks and talk soon. Best,

dr.max
ibm cloud labs
sillicon valley, ca

Sent from my iPhone

On Mar 3, 2016, at 4:06 PM, Michael Maximilien <maxim(a)us.ibm.com> wrote:

Quick reminder of the CAB call next Wednesday, March 9th @ 8a PDT. All info in link:

https://docs.google.com/document/d/1SCOlAquyUmNM-AQnekCOXiwhLs6gveTxAcduvDcW_xI/edit#heading=h.o44xhgvum2we

Please NOTE that for this CAB we will also do a quick review of the feedback from the "Future of CAB Survey" (see: https://goo.gl/5wBHMu). Two main changes from survey results: a) less status during call; and b) more future discussions and QAs.

Talk to you all next week. We'll send one more reminder on this list.

Best,

Chip, James, and Max

dr.max
ibm cloud labs
sillicon valley, ca

Sent from my iPhone


Re: `api_z1/0' is not running after update to CF v231

Filip Hanik
 

follow the steps in here

https://github.com/cloudfoundry/bosh-lite/blob/master/bin/provision_cf

for a working bosh-lite. key is to start with a virtualbox image. the
fusion/workstation are not up to date.

Filip

On Monday, March 7, 2016, Wayne Ha <wayne.h.ha(a)gmail.com> wrote:

Filip,

I am running with the latest CF v231. Initially, I ran it with older
stemcell and got:

`api_z1/0' is not running after update

After running with the latest stemcell, I got a successful deployment but
failed to login with error:

Error performing request: Get https://login.bosh-lite.com/login: stopped
after 1 redirect

Could there be some configurations that I missed? Note that I am using
default bosh-lite-v231.yml.

Thanks,


Re: `api_z1/0' is not running after update to CF v231

Amit Kumar Gupta
 

Hey Wayne,

What command did you run to generate your manifest?

Amit

On Mon, Mar 7, 2016 at 8:03 PM, Wayne Ha <wayne.h.ha(a)gmail.com> wrote:

Filip,

I am running with the latest CF v231. Initially, I ran it with older
stemcell and got:

`api_z1/0' is not running after update

After running with the latest stemcell, I got a successful deployment but
failed to login with error:

Error performing request: Get https://login.bosh-lite.com/login: stopped
after 1 redirect

Could there be some configurations that I missed? Note that I am using
default bosh-lite-v231.yml.

Thanks,


Re: `api_z1/0' is not running after update to CF v231

Wayne Ha <wayne.h.ha@...>
 

Filip,

I am running with the latest CF v231. Initially, I ran it with older stemcell and got:

`api_z1/0' is not running after update

After running with the latest stemcell, I got a successful deployment but failed to login with error:

Error performing request: Get https://login.bosh-lite.com/login: stopped after 1 redirect

Could there be some configurations that I missed? Note that I am using default bosh-lite-v231.yml.

Thanks,


Re: `api_z1/0' is not running after update to CF v231

Filip Hanik
 

Error performing request: Get https://login.bosh-lite.com/login: stopped
after 1 redirect


that's the error right there. it's a redirect loop

what version of CF is this? upgrade to the latest.

On Monday, March 7, 2016, sridhar vennela <sridhar.vennela(a)gmail.com> wrote:

Hi Wayne,

I am not seeing any errors in above. To capture UAA errors, It is better
to open 2 terminals in one terminal you can do tail -f uaa.log and in
another terminal try to do cf login -a api.bosh-lite.com -u admin -p
admin --skip-ssl-validation.

Thank you,
Sridhar


Re: `api_z1/0' is not running after update to CF v231

sridhar vennela
 

Hi Wayne,

I am not seeing any errors in above. To capture UAA errors, It is better to open 2 terminals in one terminal you can do tail -f uaa.log and in another terminal try to do cf login -a api.bosh-lite.com -u admin -p admin --skip-ssl-validation.

Thank you,
Sridhar


Re: `api_z1/0' is not running after update to CF v231

Wayne Ha <wayne.h.ha@...>
 

Zach,

After using the latest stemcell, I got a successful deployment. But after that, cf login fails:

vagrant(a)agent-id-bosh-0:~$ cf login -a api.bosh-lite.com -u admin -p admin
API endpoint: api.bosh-lite.com
FAILED
Invalid SSL Cert for api.bosh-lite.com
TIP: Use 'cf login --skip-ssl-validation' to continue with an insecure API endpoint

vagrant(a)agent-id-bosh-0:~$ cf login -a api.bosh-lite.com -u admin -p admin --skip-ssl-validation
API endpoint: api.bosh-lite.com
FAILED
Error performing request: Get https://login.bosh-lite.com/login: stopped after 1 redirect
API endpoint: https://api.bosh-lite.com (API version: 2.51.0)
Not logged in. Use 'cf login' to log in.

I saw the following in uaa.log:

root(a)d142fabc-f823-40df-b9ea-97d306bf7209:/var/vcap/sys/log/uaa# grep -A9 -i error uaa.log | cut -c 65-650
DEBUG --- AntPathRequestMatcher: Checking match of request : '/login'; against '/error'
DEBUG --- AntPathRequestMatcher: Checking match of request : '/login'; against '/email_sent'
DEBUG --- AntPathRequestMatcher: Checking match of request : '/login'; against '/create_account*'
DEBUG --- AntPathRequestMatcher: Checking match of request : '/login'; against '/accounts/email_sent'
DEBUG --- AntPathRequestMatcher: Checking match of request : '/login'; against '/invalid_request'
DEBUG --- AntPathRequestMatcher: Checking match of request : '/login'; against '/saml_error'
DEBUG --- UaaRequestMatcher: [loginAuthenticateRequestMatcher] Checking match of request : '/login'; '/authenticate' with parameters={} and headers {Authorization=[bearer ], accept=[application/json]}
DEBUG --- AntPathRequestMatcher: Checking match of request : '/login'; against '/authenticate/**'
DEBUG --- UaaRequestMatcher: [loginAuthorizeRequestMatcher] Checking match of request : '/login'; '/oauth/authorize' with parameters={source=login} and headers {accept=[application/json]}
DEBUG --- UaaRequestMatcher: [loginTokenRequestMatcher] Checking match of request : '/login'; '/oauth/token' with parameters={source=login, grant_type=password, add_new=} and headers {Authorization=[bearer ], accept=[application/json]}
DEBUG --- UaaRequestMatcher: [loginAuthorizeRequestMatcherOld] Checking match of request : '/login'; '/oauth/authorize' with parameters={login={} and headers {accept=[application/json]}
DEBUG --- AntPathRequestMatcher: Checking match of request : '/login'; against '/password_*'
DEBUG --- AntPathRequestMatcher: Checking match of request : '/login'; against '/email_*'
DEBUG --- AntPathRequestMatcher: Checking match of request : '/login'; against '/oauth/token/revoke/**'
DEBUG --- UaaRequestMatcher: [passcodeTokenMatcher] Checking match of request : '/login'; '/oauth/token' with parameters={grant_type=password, passcode=} and headers {accept=[application/json, application/x-www-form-urlencoded]}

But I don't know what the above mean.

Thanks,


Re: New CF Service Broker "chaos-galago" - a chaos-monkey for your Cloud Foundry

Cornelia Davis <cdavis@...>
 

wicked cool!

On Mon, Mar 7, 2016 at 4:57 AM, Sam Bryant <srbry(a)hotmail.com> wrote:

For anyone interested we have also now added a smoke tests project for
chaos-galago that can be used to monitor the service-broker. This can be
found: https://github.com/FidelityInternational/chaos-galago-smoke-tests

Details are also on the README for chaos-galago.

Regards,
Sam
--
Cornelia Davis
(805) 452 8941


OAuth Client Details changed for CF CLI

Madhura Bhave
 

Hi All,

The oauth client with client ID *cf* is meant to be used by the CF CLI for
communicating with the UAA using password grant type. There are default
values for this client here:
https://github.com/cloudfoundry/cf-release/blob/master/templates/cf-properties.yml#L337-L344

We have noticed that this client has been used by applications other than
the CF CLI using the implicit grant type. We have decided that this client
should not have implicit grant type and therefore this will not work going
forward. Along with this change, we will also remove `autoapprove: true`
for this client. For anyone using this client with implicit grant type,
please consider making a separate client for that purpose.

Thanks,
Jeremy & Madhura


Re: ERR_CONNECTION_TIMED_OUT

Amit Kumar Gupta
 

Upcoming support for TCP routing (not just HTTP routing) will allow you to
run a container that serves HTTPS traffic.

Currently the wso2 product ships with an untrusted self-signed cert. How
do you plan on using wso2? Will it be exposed to external consumers, or is
it just something internal that other applications need to talk to? If
exposed to external consumers, will you want to configure it with a real
trusted cert? What about DNS, would it be exposed under a proper DNS
hostname (instead of clients requesting a raw IP)? Do you plan on being
able to horizontally scale your wso2 application, i.e. run more than one
instance of the server to share traffic load and provide fault tolerance?

Cheers,
Amit

On Thu, Mar 3, 2016 at 8:39 PM, Nanduni Nimalsiri <nandunibw(a)gmail.com>
wrote:

Hi Eric,

Thank you very much for looking into this matter. Actually I was not well
and I couldn't reply you. I am really really grateful for you and Amit for
looking into this matter very generously.

Actually, your description explains me well. The problem here is that WSO2
products use HTTPS, and there's no way to change that behavior because that
will be affecting the overall company process.

WSO2 products are based on Carbon platform and they are run with HTTPS
mode. [1] is a basic WSO2 product repo which runs the WSO2 ESB.
[1] https://github.com/wso2/product-esb

So, can you propose me any other way to deploy WSO2 products in Cloud
Foundry, if I'm unable to handle with Docker images. As a company, we are
now in the strategy of trying out our products in different PaaSes. So your
support is highly appreciated.

Best regards,
Nanduni.


Re: Reg the minimal-openstack yml files

Amit Kumar Gupta
 

You need some static IPs, because some components don't use service
discovery, so clients need to be configured up front to know those
components' IPs. For example, the route-registrar jobs talk to the NATS
cluster to register routes for CC, HM9k, UAA, and Doppler. NATS servers
must be assigned static IPs in the manifest and components like
route-registrar that talk to NATS need to have those IPs provided in their
config in the manifest.

This current requirement on static IPs exists for some other components as
well. Some other components however would be fine with dynamic networking.
A private vendor may be able to work with you to manage a Cloud Foundry
installation that suits your networking requirements, but said guidance is
beyond the scope of this mailing list. The deployments that the Foundation
tests continuously use static IPs for simplicity.

Cheers,
Amit

On Monday, March 7, 2016, Jayarajan Ramapurath Kozhummal (jayark) <
jayark(a)cisco.com> wrote:

Thanks Amit!

A minor correction in our case.
We are not using static IP. We are only using dynamic IP.
None of the Cloud Foundry components are using Floating IP.

What is Pivotal recommendation for the networking? Is it static or dynamic?
Given that we are using dynamic IPs, do you foresee any issues with our
current approach to generate the manifest for CF-229?

Regards
Jayaraj

From: Amit Gupta <agupta(a)pivotal.io>
Date: Monday, March 7, 2016 at 12:24 PM
To: "Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco)" <
ngnanase(a)cisco.com>
Cc: Rohit Kumar <rokumar(a)pivotal.io>, Jayarajan Ramapurath Kozhummal <
jayark(a)cisco.com>, "Discussions about Cloud Foundry projects and the
system overall." <cf-dev(a)lists.cloudfoundry.org>
Subject: Re: Reg the minimal-openstack yml files

We're not likely to be able to create and maintain a new minimal manifest
any time soon.

As for generating the manifest, you can look at these docs for what needs
to go into the OpenStack stub as of the latest release:

http://docs.cloudfoundry.org/deploying/openstack/cf-stub.html

Combining this with updating your current template guided by the errors is
a good way to generate the manifest. This is not guaranteed to work for
100% of use cases, e.g. if you want to use all floating IPs instead of
static IPs. But it is a good starting point.

Cheers,
Amit

On Mon, Mar 7, 2016 at 4:09 AM, Nithiyasri Gnanasekaran -X (ngnanase -
TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com> wrote:

Hi



We are trying to upgrade our deployment with the latest cloud-foundry,
from 205 to 230 release, as per your advice.



We could see minimal-aws.yml available in the GIT repo. *Can we have a
similar one available for openstack environment, with which we can deploy
the basic cloud foundry* and do our custom changes on top of it



Parallely we are updating our stub to match the template yml files guided
by the errors given by the generate_deployment_manifest script. Kindly let
us know if this is the correct way to generate the manifest.





Regards

Nithiyasri


Re: Reg the minimal-openstack yml files

Jayarajan Ramapurath Kozhummal (jayark) <jayark@...>
 

Thanks Amit!

A minor correction in our case.
We are not using static IP. We are only using dynamic IP.
None of the Cloud Foundry components are using Floating IP.

What is Pivotal recommendation for the networking? Is it static or dynamic?
Given that we are using dynamic IPs, do you foresee any issues with our current approach to generate the manifest for CF-229?

Regards
Jayaraj

From: Amit Gupta <agupta(a)pivotal.io<mailto:agupta(a)pivotal.io>>
Date: Monday, March 7, 2016 at 12:24 PM
To: "Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco)" <ngnanase(a)cisco.com<mailto:ngnanase(a)cisco.com>>
Cc: Rohit Kumar <rokumar(a)pivotal.io<mailto:rokumar(a)pivotal.io>>, Jayarajan Ramapurath Kozhummal <jayark(a)cisco.com<mailto:jayark(a)cisco.com>>, "Discussions about Cloud Foundry projects and the system overall." <cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>>
Subject: Re: Reg the minimal-openstack yml files

We're not likely to be able to create and maintain a new minimal manifest any time soon.

As for generating the manifest, you can look at these docs for what needs to go into the OpenStack stub as of the latest release:

http://docs.cloudfoundry.org/deploying/openstack/cf-stub.html

Combining this with updating your current template guided by the errors is a good way to generate the manifest. This is not guaranteed to work for 100% of use cases, e.g. if you want to use all floating IPs instead of static IPs. But it is a good starting point.

Cheers,
Amit

On Mon, Mar 7, 2016 at 4:09 AM, Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com<mailto:ngnanase(a)cisco.com>> wrote:
Hi

We are trying to upgrade our deployment with the latest cloud-foundry, from 205 to 230 release, as per your advice.

We could see minimal-aws.yml available in the GIT repo. Can we have a similar one available for openstack environment, with which we can deploy the basic cloud foundry and do our custom changes on top of it

Parallely we are updating our stub to match the template yml files guided by the errors given by the generate_deployment_manifest script. Kindly let us know if this is the correct way to generate the manifest.


Regards
Nithiyasri


Re: Reg the minimal-openstack yml files

Amit Kumar Gupta
 

We're not likely to be able to create and maintain a new minimal manifest
any time soon.

As for generating the manifest, you can look at these docs for what needs
to go into the OpenStack stub as of the latest release:

http://docs.cloudfoundry.org/deploying/openstack/cf-stub.html

Combining this with updating your current template guided by the errors is
a good way to generate the manifest. This is not guaranteed to work for
100% of use cases, e.g. if you want to use all floating IPs instead of
static IPs. But it is a good starting point.

Cheers,
Amit

On Mon, Mar 7, 2016 at 4:09 AM, Nithiyasri Gnanasekaran -X (ngnanase - TECH
MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com> wrote:

Hi



We are trying to upgrade our deployment with the latest cloud-foundry,
from 205 to 230 release, as per your advice.



We could see minimal-aws.yml available in the GIT repo. *Can we have a
similar one available for openstack environment, with which we can deploy
the basic cloud foundry* and do our custom changes on top of it



Parallely we are updating our stub to match the template yml files guided
by the errors given by the generate_deployment_manifest script. Kindly let
us know if this is the correct way to generate the manifest.





Regards

Nithiyasri


Re: Update Parallelization in Cloud Foundry

Amit Kumar Gupta
 

Hey Omar,

You can set the "serial" property at the global level of a deployment (you
can think of it as setting a default for all jobs), and then override it at
the individual job levels. You will want the consul server jobs to be
deployed first, with serial: true, and max_in_flight: 1. The important
thing here is, if you have more than one server in your consul cluster,
they need to come up one at a time to ensure the cluster orchestration goes
smoothly. The same is true if your etcd cluster has more than one server
in it. If you're using the postgres job for CCDB and/or UAADB (instead of
some external database), then you will want the postgres job to come up
before CC and/or UAA. Similarly, if you're using the provided blobstore
job instead of an external blobstore, you'll want it up before CC comes up.

You might be able to get away with parallelizing some of the things above.
E.g. if you bring the CC and blobstore up at the same time, CC might fail
to start for a while until Blobstore comes up, and then CC might
successfully start up. Monit also generally keeps retrying even after BOSH
gives up. So your deploy might fail but later on, you might see everything
up and running.

Cheers,
Amit

On Mon, Mar 7, 2016 at 5:54 AM, Omar Elazhary <omazhary(a)gmail.com> wrote:

Hello everyone,

I know it is possible to update and redeploy components in parallel in
cloud foundry by setting the "serial" property in the deployment manifest
to "false". However, is such a thing recommended? Are there particular job
dependencies that I need to pay attention to?

Regards,
Omar


Re: `api_z1/0' is not running after update to CF v231

Wayne Ha <wayne.h.ha@...>
 

Zach,

Thanks for the hints. You are right, I am not using latest stemcell:

vagrant(a)agent-id-bosh-0:~$ bosh stemcells
+---------------------------------------------+---------+--------------------------------------+
| Name | Version | CID |
+---------------------------------------------+---------+--------------------------------------+
| bosh-warden-boshlite-ubuntu-trusty-go_agent | 389* | cb6ee28c-a703-4a7e-581b-b63be2302e3d |

I will try the stemcell you recommended to see if it helps.

Thanks,


Re: Increase CAB meeting to 1.5 or 2 hours?

Steven Benario
 

One of the things we've started going in the Runtime-specific meetings is
asking that individual team status updates are provided ahead of time, and
then we use the time in the meeting for Q&A only, instead of reading those
notes aloud.

I'm a big fan and recommend this model.

Cheers

On Sun, Feb 14, 2016 at 11:19 AM, Dr Nic Williams <drnicwilliams(a)gmail.com>
wrote:

Since we started the CAB calls in late 2013 the size of Cloud Foundry
community, the number of engineering teams, and the number of code bases
(both core & community) has grown.

There was a time when community projects might get to be introduced once
during a CAB meeting; and shared discussion of what was going on amongst
the community.

Recently, due to the hard limit of 1 hour - there is only just time for
core team projects to discuss their status.

With increased number of CF office locations, the tread will continue to
create more teams, which in turn will create more repositories & tracker
roadmaps, and will create more demand on the limited CAB call time slot.

I like the name "Community Advisory Board meeting". It seems like a good
goal for the meeting.

Could we increase the CAB meeting to 1.5 or 2 hours?

Or could the summaries by PMs be delivered before the meeting (I have the
feeling many of them are organized and have notes that they are reading).
This might give us more time to discuss the projects, and discuss
non-core/community projects and ambitions.

It might also allow time for the community to chat with itself and bond as
a community.

with valentine's love
Dr Nic


--
Dr Nic Williams
Stark & Wayne LLC - consultancy for Cloud Foundry users
http://drnicwilliams.com
http://starkandwayne.com
cell +1 (415) 860-2185
twitter @drnic


Re: `api_z1/0' is not running after update to CF v231

Zach Robinson
 

Wayne,

Can you verify that you are using the latest bosh-lite stemcell 3147? Older stemcells are known to have issues with consul which is what many of the CF components use for service discovery.

Latest bosh-lite stemcells can be found at http://bosh.io Just search for lite.

See this similar issue: https://github.com/cloudfoundry/cf-release/issues/919

-Zach


Re: `api_z1/0' is not running after update to CF v231

Amit Kumar Gupta
 

As of cf v231, CC has switched from using NFS to WebDav as the default
blobstore. There are more details in the release notes:
https://github.com/cloudfoundry/cf-release/releases/tag/v231. I don't know
off-hand how to debug the issue you're seeing, but I will reach out to some
folks with more knowledge of Cloud Controller.

Best,
Amit

On Mon, Mar 7, 2016 at 8:48 AM, Wayne Ha <wayne.h.ha(a)gmail.com> wrote:

Kayode,

I am using the default bosh-lite-v231.yml file and the instances for nfs
server is set to 0:

vagrant(a)agent-id-bosh-0:~$ egrep -i "name:.*nfs|instances"
bosh-lite-v231.yml.1603041454
etc...
- instances: 0
- instances: 0
- instances: 0
name: nfs_z1
- name: debian_nfs_server
- instances: 1
- instances: 1
- instances: 1
etc...

So it is not running.

Thanks,

5341 - 5360 of 9425