|
Re: Moving Diego Repositories
I've found when doing this the easiest thing to do is to use sed to remove
existing imports followed by goimports -w on the same files.
KH
toggle quoted message
Show quoted text
On Sunday, June 26, 2016, Amit Gupta <agupta(a)pivotal.io> wrote: Congrats on making it official!
I assume you folks are going to build some scripts or some "go fmt -r" to
find/change all import paths that need updating. If you find anything
useful that other teams might be able to leverage, sharing would be much
appreciated.
Cheers,
Amit
On Fri, Jun 24, 2016 at 4:20 PM, Onsi Fakhouri <ofakhouri(a)pivotal.io
<javascript:_e(%7B%7D,'cvml','ofakhouri(a)pivotal.io');>> wrote:
That stuff has always technically been foundation code and should have
moved out of pivotal-golang into cloudfoundry-incubator long ago. Back
when we made the org the rules about how orgs are structured weren't quite
so clear.
Onsi
On Fri, Jun 24, 2016 at 4:18 PM, Alex Suraci <asuraci(a)pivotal.io
<javascript:_e(%7B%7D,'cvml','asuraci(a)pivotal.io');>> wrote:
why are we moving the pivotal-golang repos? I thought the point of that
was to be able to create those cheaply for super generic packages that we
just need to have somewhere in cases where the repo incubation foundation
lifecycle crap gets in the way. are we now able to create repos willy-nilly
under the cloudfoundry org?
On Sat, Jun 25, 2016, 6:13 AM James Myers <jmyers(a)pivotal.io
<javascript:_e(%7B%7D,'cvml','jmyers(a)pivotal.io');>> wrote:
Hi all,
We are currently in the process of moving all of the Diego repositories
from `cloudfoundry-incubator` to the `cloudfoundry` organization.
Specifically we are moving the following repositories to the
`cloudfoundry` organization:
- auction
- auctioneer
- bbs
- benchmark-bbs
- buildpack_app_lifecycle
- cacheddownloader
- cf-debug-server
- cf-lager
- cf_http
- consuladapter
- converger
- diego-ssh
- diego-upgrade-stability-tests
- docker_app_lifecycle
- executor
- file-server
- healthcheck
- inigo
- locket
- rep
- route-emitter
- runtime-schema
- vizzini
- diego-cf-compatibility
- diego-ci-pools
- diego-ci
- diego-design-notes
- diego-dev-notes
- diego-dockerfiles
- diego-perf-release
- diego-release
- diego-stress-tests
We are also moving the following from `pivotal-golang` to
`cloudfoundry` as well:
- archiver
- bytefmt
- clock
- eventhub
- lager
- localip
- operationq
We are also renaming the following, and will be updating their package
names accordingly:
- benchmark-bbs -> benchmarkbbs
- buildpack_app_lifecycle -> buildpackapplifecycle
- cf-debug-server -> debugserver
- cf-lager -> cflager
- cf_http -> cfhttp
- docker_app_lifecycle -> dockerapplifecycle
- file-server -> fileserver
- runtime-schema -> runtimeschema
You might be asking yourself, what does this mean for me?
Generally it means the following:
- If you are importing any of the above repos in your golang code, you
should change it from `github.com/cloudfoundry-incubator/REPO_NAME`
<http://github.com/cloudfoundry-incubator/REPO_NAME> to `
code.cloudfoundry.org/REPO_NAME`
<http://code.cloudfoundry.org/REPO_NAME>.
- Update your golang code when you update your dependencies to
reference the new package names marked above.
- If you are consuming the Diego bosh release from bosh.io, you will
need to update the location to
http://bosh.io/releases/github.com/cloudfoundry/diego-release.
Other than that, github redirects should handle most of the issues for
you.
As a side note we are also moving the following deprecated repositories
to the `cloudfoundry-attic`:
- diego-acceptance-tests
- diego-smoke-tests
- receptor
- runtime-metrics-server
Let us know if you have any questions.
Best,
Jim + Andrew, CF Diego Team
|
|
|
Re: Moving Diego Repositories
Congrats on making it official!
I assume you folks are going to build some scripts or some "go fmt -r" to
find/change all import paths that need updating. If you find anything
useful that other teams might be able to leverage, sharing would be much
appreciated.
Cheers,
Amit
toggle quoted message
Show quoted text
On Fri, Jun 24, 2016 at 4:20 PM, Onsi Fakhouri <ofakhouri(a)pivotal.io> wrote: That stuff has always technically been foundation code and should have
moved out of pivotal-golang into cloudfoundry-incubator long ago. Back
when we made the org the rules about how orgs are structured weren't quite
so clear.
Onsi
On Fri, Jun 24, 2016 at 4:18 PM, Alex Suraci <asuraci(a)pivotal.io> wrote:
why are we moving the pivotal-golang repos? I thought the point of that
was to be able to create those cheaply for super generic packages that we
just need to have somewhere in cases where the repo incubation foundation
lifecycle crap gets in the way. are we now able to create repos willy-nilly
under the cloudfoundry org?
On Sat, Jun 25, 2016, 6:13 AM James Myers <jmyers(a)pivotal.io> wrote:
Hi all,
We are currently in the process of moving all of the Diego repositories
from `cloudfoundry-incubator` to the `cloudfoundry` organization.
Specifically we are moving the following repositories to the
`cloudfoundry` organization:
- auction
- auctioneer
- bbs
- benchmark-bbs
- buildpack_app_lifecycle
- cacheddownloader
- cf-debug-server
- cf-lager
- cf_http
- consuladapter
- converger
- diego-ssh
- diego-upgrade-stability-tests
- docker_app_lifecycle
- executor
- file-server
- healthcheck
- inigo
- locket
- rep
- route-emitter
- runtime-schema
- vizzini
- diego-cf-compatibility
- diego-ci-pools
- diego-ci
- diego-design-notes
- diego-dev-notes
- diego-dockerfiles
- diego-perf-release
- diego-release
- diego-stress-tests
We are also moving the following from `pivotal-golang` to `cloudfoundry`
as well:
- archiver
- bytefmt
- clock
- eventhub
- lager
- localip
- operationq
We are also renaming the following, and will be updating their package
names accordingly:
- benchmark-bbs -> benchmarkbbs
- buildpack_app_lifecycle -> buildpackapplifecycle
- cf-debug-server -> debugserver
- cf-lager -> cflager
- cf_http -> cfhttp
- docker_app_lifecycle -> dockerapplifecycle
- file-server -> fileserver
- runtime-schema -> runtimeschema
You might be asking yourself, what does this mean for me?
Generally it means the following:
- If you are importing any of the above repos in your golang code, you
should change it from `github.com/cloudfoundry-incubator/REPO_NAME`
<http://github.com/cloudfoundry-incubator/REPO_NAME> to `
code.cloudfoundry.org/REPO_NAME`
<http://code.cloudfoundry.org/REPO_NAME>.
- Update your golang code when you update your dependencies to reference
the new package names marked above.
- If you are consuming the Diego bosh release from bosh.io, you will
need to update the location to
http://bosh.io/releases/github.com/cloudfoundry/diego-release.
Other than that, github redirects should handle most of the issues for
you.
As a side note we are also moving the following deprecated repositories
to the `cloudfoundry-attic`:
- diego-acceptance-tests
- diego-smoke-tests
- receptor
- runtime-metrics-server
Let us know if you have any questions.
Best,
Jim + Andrew, CF Diego Team
|
|
|
Re: Creating new user UAA
|
|
|
Re: Retrieve __VCAP__ID from instance_ID
Installed the sample app and tried it on Bluemix and I don't see CF_INSTANCE_GUID or INSTANCE_GUID in Environment section. How do I know if its Diego or DEA ?.
Thanks,
Vinod
|
|
|
Re: How shoulld I debug a blobstore error?
You can use xip.io for temporary domain, try generating manifest again and
deploy.
toggle quoted message
Show quoted text
On Saturday, 25 June 2016, Eyal Shalev <eshalev(a)cisco.com> wrote: As I was not planning on sigining exposing this deployment to the outside,
I was planning on using HAPROXY through it's floating IP.
I did not set a domain, because I have not yet interfaced with my DNS
server. So I just left the domain as "DOMAIN"
When looking at manifest generated from the stuff, i see the following 2
lines repeating sveral times:
private_endpoint: https://blobstore.service.cf.internal
public_endpoint: http://blobstore.DOMAIN
Could it be that the api-node is trying to access blobstore through the
haproxy node?
Is there anyway to avoid setting up a domain for CF? (as this is just a
lab experiment)
|
|
|
Re: Creating new user UAA
That was the problem. I removed the MySQL profile and it worked. Thanks!
|
|
|
Re: Moving Diego Repositories
Onsi Fakhouri <ofakhouri@...>
That stuff has always technically been foundation code and should have
moved out of pivotal-golang into cloudfoundry-incubator long ago. Back
when we made the org the rules about how orgs are structured weren't quite
so clear.
Onsi
toggle quoted message
Show quoted text
On Fri, Jun 24, 2016 at 4:18 PM, Alex Suraci <asuraci(a)pivotal.io> wrote: why are we moving the pivotal-golang repos? I thought the point of that
was to be able to create those cheaply for super generic packages that we
just need to have somewhere in cases where the repo incubation foundation
lifecycle crap gets in the way. are we now able to create repos willy-nilly
under the cloudfoundry org?
On Sat, Jun 25, 2016, 6:13 AM James Myers <jmyers(a)pivotal.io> wrote:
Hi all,
We are currently in the process of moving all of the Diego repositories
from `cloudfoundry-incubator` to the `cloudfoundry` organization.
Specifically we are moving the following repositories to the
`cloudfoundry` organization:
- auction
- auctioneer
- bbs
- benchmark-bbs
- buildpack_app_lifecycle
- cacheddownloader
- cf-debug-server
- cf-lager
- cf_http
- consuladapter
- converger
- diego-ssh
- diego-upgrade-stability-tests
- docker_app_lifecycle
- executor
- file-server
- healthcheck
- inigo
- locket
- rep
- route-emitter
- runtime-schema
- vizzini
- diego-cf-compatibility
- diego-ci-pools
- diego-ci
- diego-design-notes
- diego-dev-notes
- diego-dockerfiles
- diego-perf-release
- diego-release
- diego-stress-tests
We are also moving the following from `pivotal-golang` to `cloudfoundry`
as well:
- archiver
- bytefmt
- clock
- eventhub
- lager
- localip
- operationq
We are also renaming the following, and will be updating their package
names accordingly:
- benchmark-bbs -> benchmarkbbs
- buildpack_app_lifecycle -> buildpackapplifecycle
- cf-debug-server -> debugserver
- cf-lager -> cflager
- cf_http -> cfhttp
- docker_app_lifecycle -> dockerapplifecycle
- file-server -> fileserver
- runtime-schema -> runtimeschema
You might be asking yourself, what does this mean for me?
Generally it means the following:
- If you are importing any of the above repos in your golang code, you
should change it from `github.com/cloudfoundry-incubator/REPO_NAME`
<http://github.com/cloudfoundry-incubator/REPO_NAME> to `
code.cloudfoundry.org/REPO_NAME` <http://code.cloudfoundry.org/REPO_NAME>
.
- Update your golang code when you update your dependencies to reference
the new package names marked above.
- If you are consuming the Diego bosh release from bosh.io, you will
need to update the location to
http://bosh.io/releases/github.com/cloudfoundry/diego-release.
Other than that, github redirects should handle most of the issues for
you.
As a side note we are also moving the following deprecated repositories
to the `cloudfoundry-attic`:
- diego-acceptance-tests
- diego-smoke-tests
- receptor
- runtime-metrics-server
Let us know if you have any questions.
Best,
Jim + Andrew, CF Diego Team
|
|
|
Re: How shoulld I debug a blobstore error?
As I was not planning on sigining exposing this deployment to the outside, I was planning on using HAPROXY through it's floating IP. I did not set a domain, because I have not yet interfaced with my DNS server. So I just left the domain as "DOMAIN" When looking at manifest generated from the stuff, i see the following 2 lines repeating sveral times: private_endpoint: https://blobstore.service.cf.internalpublic_endpoint: http://blobstore.DOMAINCould it be that the api-node is trying to access blobstore through the haproxy node? Is there anyway to avoid setting up a domain for CF? (as this is just a lab experiment)
|
|
|
Re: How shoulld I debug a blobstore error?
Hello Eyal,
Can you paste the domain properties from your final manifest ( system
domain & app domain part).
Thanks
Ronak
toggle quoted message
Show quoted text
On Saturday, 25 June 2016, Eyal Shalev <eshalev(a)cisco.com> wrote: Hello Amit,
I have removed the lines that you have marked.
Now I am getting a different error...
Process 'consul_agent' running
Process 'cloud_controller_ng' Connection failed
Process 'cloud_controller_worker_local_1' not monitored
Process 'cloud_controller_worker_local_2' not monitored
Process 'nginx_cc' initializing
Process 'cloud_controller_migration' running
Process 'metron_agent' running
Process 'statsd-injector' running
Process 'route_registrar' running
System 'system_localhost' running
The blob store is available, but still the process fails:
[2016-06-24 20:13:17+0000] ------------ STARTING
cloud_controller_worker_ctl at Fri Jun 24 20:13:17 UTC 2016 --------------
[2016-06-24 20:13:17+0000] Removing stale pidfile
[2016-06-24 20:13:17+0000] Checking for blobstore availability
[2016-06-24 20:13:17+0000] Blobstore is available
[2016-06-24 20:13:18+0000] Buildpacks installation failed
and also:
[2016-06-24 20:33:16+0000] ------------ STARTING cloud_controller_ng_ctl
at Fri Jun 24 20:33:16 UTC 2016 --------------
[2016-06-24 20:33:16+0000] Checking for blobstore availability
[2016-06-24 20:33:16+0000] Blobstore is available
[2016-06-24 20:33:38+0000] Killing
/var/vcap/sys/run/cloud_controller_ng/cloud_controller_ng.pid: 28368
[2016-06-24 20:33:38+0000] Stopped
[2016-06-24 20:33:39+0000] ------------ STARTING cloud_controller_ng_ctl
at Fri Jun 24 20:33:39 UTC 2016 --------------
[2016-06-24 20:33:39+0000] Checking for blobstore availability
[2016-06-24 20:33:39+0000] Blobstore is available
[2016-06-24 20:34:02+0000] Killing
/var/vcap/sys/run/cloud_controller_ng/cloud_controller_ng.pid: 28818
[2016-06-24 20:34:03+0000] Stopped
Which brings me to another question:
Do you have a stable old release of CF for openstack? I don't mind
downgrading, if the new releases are unstable. If that is not possible, can
you post a valid cf-stub.yml without the need for any manual removal of
invalid lines? (that way I have a reference to what tried and tested stub
should look like)
Thanks alot for your help,
Eyal
|
|
|
Re: Retrieve __VCAP__ID from instance_ID
|
|
|
Moving Diego Repositories
Hi all, We are currently in the process of moving all of the Diego repositories from `cloudfoundry-incubator` to the `cloudfoundry` organization. Specifically we are moving the following repositories to the `cloudfoundry` organization: - auction - auctioneer - bbs - benchmark-bbs - buildpack_app_lifecycle - cacheddownloader - cf-debug-server - cf-lager - cf_http - consuladapter - converger - diego-ssh - diego-upgrade-stability-tests - docker_app_lifecycle - executor - file-server - healthcheck - inigo - locket - rep - route-emitter - runtime-schema - vizzini - diego-cf-compatibility - diego-ci-pools - diego-ci - diego-design-notes - diego-dev-notes - diego-dockerfiles - diego-perf-release - diego-release - diego-stress-tests We are also moving the following from `pivotal-golang` to `cloudfoundry` as well: - archiver - bytefmt - clock - eventhub - lager - localip - operationq We are also renaming the following, and will be updating their package names accordingly: - benchmark-bbs -> benchmarkbbs - buildpack_app_lifecycle -> buildpackapplifecycle - cf-debug-server -> debugserver - cf-lager -> cflager - cf_http -> cfhttp - docker_app_lifecycle -> dockerapplifecycle - file-server -> fileserver - runtime-schema -> runtimeschema You might be asking yourself, what does this mean for me? Generally it means the following: - If you are importing any of the above repos in your golang code, you should change it from `github.com/cloudfoundry-incubator/REPO_NAME` to ` code.cloudfoundry.org/REPO_NAME`. - Update your golang code when you update your dependencies to reference the new package names marked above. - If you are consuming the Diego bosh release from bosh.io, you will need to update the location to http://bosh.io/releases/github.com/cloudfoundry/diego-release. Other than that, github redirects should handle most of the issues for you. As a side note we are also moving the following deprecated repositories to the `cloudfoundry-attic`: - diego-acceptance-tests - diego-smoke-tests - receptor - runtime-metrics-server Let us know if you have any questions. Best, Jim + Andrew, CF Diego Team
|
|
|
Re: How shoulld I debug a blobstore error?
Hello Amit,
I have removed the lines that you have marked.
Now I am getting a different error...
Process 'consul_agent' running
Process 'cloud_controller_ng' Connection failed
Process 'cloud_controller_worker_local_1' not monitored
Process 'cloud_controller_worker_local_2' not monitored
Process 'nginx_cc' initializing
Process 'cloud_controller_migration' running
Process 'metron_agent' running
Process 'statsd-injector' running
Process 'route_registrar' running
System 'system_localhost' running
The blob store is available, but still the process fails:
[2016-06-24 20:13:17+0000] ------------ STARTING cloud_controller_worker_ctl at Fri Jun 24 20:13:17 UTC 2016 --------------
[2016-06-24 20:13:17+0000] Removing stale pidfile
[2016-06-24 20:13:17+0000] Checking for blobstore availability
[2016-06-24 20:13:17+0000] Blobstore is available
[2016-06-24 20:13:18+0000] Buildpacks installation failed
and also:
[2016-06-24 20:33:16+0000] ------------ STARTING cloud_controller_ng_ctl at Fri Jun 24 20:33:16 UTC 2016 --------------
[2016-06-24 20:33:16+0000] Checking for blobstore availability
[2016-06-24 20:33:16+0000] Blobstore is available
[2016-06-24 20:33:38+0000] Killing /var/vcap/sys/run/cloud_controller_ng/cloud_controller_ng.pid: 28368
[2016-06-24 20:33:38+0000] Stopped
[2016-06-24 20:33:39+0000] ------------ STARTING cloud_controller_ng_ctl at Fri Jun 24 20:33:39 UTC 2016 --------------
[2016-06-24 20:33:39+0000] Checking for blobstore availability
[2016-06-24 20:33:39+0000] Blobstore is available
[2016-06-24 20:34:02+0000] Killing /var/vcap/sys/run/cloud_controller_ng/cloud_controller_ng.pid: 28818
[2016-06-24 20:34:03+0000] Stopped
Which brings me to another question:
Do you have a stable old release of CF for openstack? I don't mind downgrading, if the new releases are unstable. If that is not possible, can you post a valid cf-stub.yml without the need for any manual removal of invalid lines? (that way I have a reference to what tried and tested stub should look like)
Thanks alot for your help,
Eyal
|
|
|
Re: Retrieve __VCAP__ID from instance_ID
ok thanks, will wait for someone to reply. Do you know if the behavior should be same or has it been changed from DEA to Diego ?.
Thanks,
Vinod
|
|
|
Re: How shoulld I debug a blobstore error?
Yes sir, please try removing them and regenerating your manifest before
redeploying.
Cheers,
Amit
toggle quoted message
Show quoted text
On Friday, June 24, 2016, Eyal Shalev <eshalev(a)cisco.com> wrote: Hello Amit,
I used the latest stub from GIT. As I was working last week, I could swear
I saw the stub change as I was working...
I guess that that explains it.
Should I remove the lines in the highlighted block?
Thanks,
Eyal
|
|
|
Re: How shoulld I debug a blobstore error?
Hello Amit,
I used the latest stub from GIT. As I was working last week, I could swear I saw the stub change as I was working...
I guess that that explains it.
Should I remove the lines in the highlighted block?
Thanks,
Eyal
|
|
|
Re: Creating new user UAA
Hi Bryan,
UAA needs a database to store user accounts. By default it uses in-memory db, In your case, it is trying to connect MySQL DB. I am not sure. How are you running you UAA server? Are you using below command? If yes, you need to have MySQL running on your local machine
./gradlew -Dspring.profiles.active=default,mysql run
[2016-06-24 02:30:56.279] cloudfoundry-identity-server/uaa - ???? [localhost-startStop-1] .... WARN --- XmlWebApplicationContext: Caused by: java.net.ConnectException: Connection refused
[2016-06-24 02:30:56.279] cloudfoundry-identity-server/uaa - ???? [localhost-startStop-1] .... WARN --- XmlWebApplicationContext: at java.net.PlainSocketImpl.socketConnect(Native Method)
[2016-06-24 02:30:56.279] cloudfoundry-identity-server/uaa - ???? [localhost-startStop-1] .... WARN --- XmlWebApplicationContext: at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
[2016-06-24 02:30:56.279] cloudfoundry-identity-server/uaa - ???? [localhost-startStop-1] .... WARN --- XmlWebApplicationContext: at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
[2016-06-24 02:30:56.279] cloudfoundry-identity-server/uaa - ???? [localhost-startStop-1] .... WARN --- XmlWebApplicationContext: at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
[2016-06-24 02:30:56.279] cloudfoundry-identity-server/uaa - ???? [localhost-startStop-1] .... WARN --- XmlWebApplicationContext: at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
[2016-06-24 02:30:56.279] cloudfoundry-identity-server/uaa - ???? [localhost-startStop-1] .... WARN --- XmlWebApplicationContext: at java.net.Socket.connect(Socket.java:589)
[2016-06-24 02:30:56.279] cloudfoundry-identity-server/uaa - ???? [localhost-startStop-1] .... WARN --- XmlWebApplicationContext: at java.net.Socket.connect(Socket.java:538)
[2016-06-24 02:30:56.279] cloudfoundry-identity-server/uaa - ???? [localhost-startStop-1] .... WARN --- XmlWebApplicationContext: at org.mariadb.jdbc.internal.mysql.MySQLProtocol.connect(MySQLProtocol.java:387)
[2016-06-24 02:30:56.279] cloudfoundry-identity-server/uaa - ???? [localhost-startStop-1] .... WARN --- XmlWebApplicationContext: at org.mariadb.jdbc.internal.mysql.MySQLProtocol.connect(MySQLProtocol.java:709)
[2016-06-24 02:30:56.279] cloudfoundry-identity-server/uaa - ???? [localhost-startStop-1] .... WARN --- XmlWebApplicationContext: ... 89 more
|
|
|
Re: How shoulld I debug a blobstore error?
Those stubs are out of date. Our OpenStack stubs may remain out of date for a bit while we find more resource to support automated testing of the stubs against and OpenStack environment. In the mean time, you should be fine to just delete those offending lines. Please try that and let me know if it works. Best, Amit On Fri, Jun 24, 2016 at 10:40 AM, Pramod Mandagere <nagapramod(a)gmail.com> wrote: Hi Amit,
We have the exact same issue and our cf-stub looking exactly like the one
u pointed out. Stuck with this issue,
Regards,
Pramod
On Fri, Jun 24, 2016 at 10:26 AM Amit Gupta <agupta(a)pivotal.io> wrote:
Hi Eyal,
No need to tear things down. By any chance, did you generate your
manifest using a stub? Do you have something that looks like this in your
manifest?
<https://github.com/cloudfoundry/cf-release/blob/3290ed8/spec/fixtures/openstack/cf-stub.yml#L155-L187>
https://github.com/cloudfoundry/cf-release/blob/3290ed8/spec/fixtures/openstack/cf-stub.yml#L155-L187
Best,
Amit
On Fri, Jun 24, 2016 at 6:54 AM, Eyal Shalev <eshalev(a)cisco.com> wrote:
Hello, I am trying to deploy cloud foundry on openstack.
running "bosh deploy" gives me the following error:
Started updating job consul_z1 > consul_z1/0
(d248a6da-476a-4fb1-bc08-01ce23c94eb4) (canary). Done (00:01:03)
Started updating job ha_proxy_z1 > ha_proxy_z1/0
(f416b7c5-24d3-4d58-bbf9-066740971065) (canary). Done (00:00:43)
Started updating job nats_z1 > nats_z1/0
(074d00e8-96b3-4b7c-af1a-347549ff51dd) (canary). Done (00:00:44)
Started updating job etcd_z1 > etcd_z1/0
(ef27b75d-e0d0-4a95-8ec1-4b0c9371fa93) (canary). Done (00:01:23)
Started updating job stats_z1 > stats_z1/0
(20c1f5e9-3b7a-4c84-a940-3628cadf942c) (canary). Done (00:00:47)
Started updating job blobstore_z1 > blobstore_z1/0
(28f3cf80-502b-4e83-8dff-ba3477894a13) (canary). Done (00:01:20)
Started updating job postgres_z1 > postgres_z1/0
(601262d5-97d2-436c-b67d-1e63680f9dc6) (canary). Done (00:01:16)
Started updating job uaa_z1 > uaa_z1/0
(9792327b-05c6-4bd8-8434-8a48fa364a91) (canary). Done (00:00:52)
Started updating job api_z1 > api_z1/0
(03f63cee-dfc1-4a93-9ddc-1bf129eb7396) (canary). Failed: 'api_z1/0
(03f63cee-dfc1-4a93-9ddc-1bf129eb7396)' is not running after update. Review
logs for failed jobs: cloud_controller_ng, cloud_controller_worker_local_1,
cloud_controller_worker_local_2, nginx_cc, cloud_controller_worker_1
(00:11:11)
Error 400007: 'api_z1/0 (03f63cee-dfc1-4a93-9ddc-1bf129eb7396)' is not
running after update. Review logs for failed jobs: cloud_controller_ng,
cloud_controller_worker_local_1, cloud_controller_worker_local_2, nginx_cc,
cloud_controller_worker_1
"bosh vms" command seem to show blobstore be up and running:
+---------------------------------------------------------------------------+---------+-----+-----------+---------------+
| VM
| State | AZ | VM Type | IPs |
+---------------------------------------------------------------------------+---------+-----+-----------+---------------+
| api_z1/0 (03f63cee-dfc1-4a93-9ddc-1bf129eb7396)
| failing | n/a | large_z1 | 192.168.10.54 |
| blobstore_z1/0 (28f3cf80-502b-4e83-8dff-ba3477894a13)
| running | n/a | medium_z1 | 192.168.10.52 |
| consul_z1/0 (d248a6da-476a-4fb1-bc08-01ce23c94eb4)
| running | n/a | small_z1 | 192.168.10.76 |
| doppler_z1/0 (ec0232f4-e736-4bbf-9fe4-b35b3d0c673d)
| running | n/a | medium_z1 | 192.168.10.57 |
| etcd_z1/0 (ef27b75d-e0d0-4a95-8ec1-4b0c9371fa93)
| running | n/a | medium_z1 | 192.168.10.72 |
| ha_proxy_z1/0 (f416b7c5-24d3-4d58-bbf9-066740971065)
| running | n/a | router_z1 | 192.168.10.64 |
|
| | | | XX.XX.XX.XX |
| hm9000_z1/0 (c50fbd8e-1c8f-4b96-a1c8-a427fdabba79)
| running | n/a | medium_z1 | 192.168.10.55 |
| loggregator_trafficcontroller_z1/0
(e187f9c1-dc5d-4d9b-850d-ca6118513228) | running | n/a | small_z1 |
192.168.10.58 |
| nats_z1/0 (074d00e8-96b3-4b7c-af1a-347549ff51dd)
| running | n/a | medium_z1 | 192.168.10.66 |
| postgres_z1/0 (601262d5-97d2-436c-b67d-1e63680f9dc6)
| running | n/a | medium_z1 | 192.168.10.68 |
| router_z1/0 (4da988b9-5cbb-4a6f-b082-5c4ed25e6a7c)
| running | n/a | router_z1 | 192.168.10.69 |
| runner_z1/0 (4bce5874-2edc-4a0f-a074-0a136933de65)
| running | n/a | runner_z1 | 192.168.10.56 |
| stats_z1/0 (20c1f5e9-3b7a-4c84-a940-3628cadf942c)
| running | n/a | small_z1 | 192.168.10.51 |
| uaa_z1/0 (9792327b-05c6-4bd8-8434-8a48fa364a91)
| running | n/a | medium_z1 | 192.168.10.53 |
+---------------------------------------------------------------------------+---------+-----+-----------+---------------+
The vcap job logs also seem to show it thinks that the blobstore node is
healthey,
However, when I ssh to the api node, I see problems:
"monit summary" yields:
Process 'cloud_controller_ng' not monitored
Process 'cloud_controller_worker_local_1' not monitored
Process 'cloud_controller_worker_local_2' not monitored
Process 'nginx_cc' not monitored
Process 'cloud_controller_migration' running
Process 'cloud_controller_clock' running
Process 'cloud_controller_worker_1' running
Process 'metron_agent' running
Process 'statsd-injector' running
Process 'route_registrar' running
System 'system_localhost' running
, /var/vcap/sys/log/cloud_controller_ng/cloud_controller_ng_ctl.log
looks like this:
[2016-06-24 12:32:51+0000] Checking for blobstore availability
[2016-06-24 12:33:38+0000] ------------ STARTING cloud_controller_ng_ctl
at Fri Jun 24 12:33:38 UTC 2016 --------------
[2016-06-24 12:33:38+0000] Checking for blobstore availability
[2016-06-24 12:34:24+0000] ------------ STARTING cloud_controller_ng_ctl
at Fri Jun 24 12:34:24 UTC 2016 --------------
[2016-06-24 12:34:24+0000] Checking for blobstore availability
[2016-06-24 12:34:41+0000] Blobstore is not available
Other jobs on the same node are failing as well, all with similar
messages both in stderr and stdout logs.
However, I do not see any additional information, such as an IP address
or URL that I can test.
Additionally on the blobstore node, both "monit" and the vcap job logs
seem to think that it is healthy.
How should I go about debugging this error? (I am looking for more
verbose information on the error "blobstore is not available")
I do not mind tearing down this deployment and starting over with more
verbose flags to expose the problem.
It seems to occur consitantly.
PS: my stemcell is
bosh-stemcell-3232.6-openstack-kvm-ubuntu-trusty-go_agent.tgz , and I have
not done anything special to configure blobstore on cf-stub besides adding
TLS Ca's and certs. Also, I could find no instructions on how to add these
certs to cf-stub, so I just added some self-generated keys & certs.
|
|
|
Re: How shoulld I debug a blobstore error?
Hi Amit,
We have the exact same issue and our cf-stub looking exactly like the one u
pointed out. Stuck with this issue,
Regards,
Pramod
toggle quoted message
Show quoted text
On Fri, Jun 24, 2016 at 10:26 AM Amit Gupta <agupta(a)pivotal.io> wrote: Hi Eyal,
No need to tear things down. By any chance, did you generate your
manifest using a stub? Do you have something that looks like this in your
manifest?
<https://github.com/cloudfoundry/cf-release/blob/3290ed8/spec/fixtures/openstack/cf-stub.yml#L155-L187>
https://github.com/cloudfoundry/cf-release/blob/3290ed8/spec/fixtures/openstack/cf-stub.yml#L155-L187
Best,
Amit
On Fri, Jun 24, 2016 at 6:54 AM, Eyal Shalev <eshalev(a)cisco.com> wrote:
Hello, I am trying to deploy cloud foundry on openstack.
running "bosh deploy" gives me the following error:
Started updating job consul_z1 > consul_z1/0
(d248a6da-476a-4fb1-bc08-01ce23c94eb4) (canary). Done (00:01:03)
Started updating job ha_proxy_z1 > ha_proxy_z1/0
(f416b7c5-24d3-4d58-bbf9-066740971065) (canary). Done (00:00:43)
Started updating job nats_z1 > nats_z1/0
(074d00e8-96b3-4b7c-af1a-347549ff51dd) (canary). Done (00:00:44)
Started updating job etcd_z1 > etcd_z1/0
(ef27b75d-e0d0-4a95-8ec1-4b0c9371fa93) (canary). Done (00:01:23)
Started updating job stats_z1 > stats_z1/0
(20c1f5e9-3b7a-4c84-a940-3628cadf942c) (canary). Done (00:00:47)
Started updating job blobstore_z1 > blobstore_z1/0
(28f3cf80-502b-4e83-8dff-ba3477894a13) (canary). Done (00:01:20)
Started updating job postgres_z1 > postgres_z1/0
(601262d5-97d2-436c-b67d-1e63680f9dc6) (canary). Done (00:01:16)
Started updating job uaa_z1 > uaa_z1/0
(9792327b-05c6-4bd8-8434-8a48fa364a91) (canary). Done (00:00:52)
Started updating job api_z1 > api_z1/0
(03f63cee-dfc1-4a93-9ddc-1bf129eb7396) (canary). Failed: 'api_z1/0
(03f63cee-dfc1-4a93-9ddc-1bf129eb7396)' is not running after update. Review
logs for failed jobs: cloud_controller_ng, cloud_controller_worker_local_1,
cloud_controller_worker_local_2, nginx_cc, cloud_controller_worker_1
(00:11:11)
Error 400007: 'api_z1/0 (03f63cee-dfc1-4a93-9ddc-1bf129eb7396)' is not
running after update. Review logs for failed jobs: cloud_controller_ng,
cloud_controller_worker_local_1, cloud_controller_worker_local_2, nginx_cc,
cloud_controller_worker_1
"bosh vms" command seem to show blobstore be up and running:
+---------------------------------------------------------------------------+---------+-----+-----------+---------------+
| VM
| State | AZ | VM Type | IPs |
+---------------------------------------------------------------------------+---------+-----+-----------+---------------+
| api_z1/0 (03f63cee-dfc1-4a93-9ddc-1bf129eb7396)
| failing | n/a | large_z1 | 192.168.10.54 |
| blobstore_z1/0 (28f3cf80-502b-4e83-8dff-ba3477894a13)
| running | n/a | medium_z1 | 192.168.10.52 |
| consul_z1/0 (d248a6da-476a-4fb1-bc08-01ce23c94eb4)
| running | n/a | small_z1 | 192.168.10.76 |
| doppler_z1/0 (ec0232f4-e736-4bbf-9fe4-b35b3d0c673d)
| running | n/a | medium_z1 | 192.168.10.57 |
| etcd_z1/0 (ef27b75d-e0d0-4a95-8ec1-4b0c9371fa93)
| running | n/a | medium_z1 | 192.168.10.72 |
| ha_proxy_z1/0 (f416b7c5-24d3-4d58-bbf9-066740971065)
| running | n/a | router_z1 | 192.168.10.64 |
|
| | | | XX.XX.XX.XX |
| hm9000_z1/0 (c50fbd8e-1c8f-4b96-a1c8-a427fdabba79)
| running | n/a | medium_z1 | 192.168.10.55 |
| loggregator_trafficcontroller_z1/0
(e187f9c1-dc5d-4d9b-850d-ca6118513228) | running | n/a | small_z1 |
192.168.10.58 |
| nats_z1/0 (074d00e8-96b3-4b7c-af1a-347549ff51dd)
| running | n/a | medium_z1 | 192.168.10.66 |
| postgres_z1/0 (601262d5-97d2-436c-b67d-1e63680f9dc6)
| running | n/a | medium_z1 | 192.168.10.68 |
| router_z1/0 (4da988b9-5cbb-4a6f-b082-5c4ed25e6a7c)
| running | n/a | router_z1 | 192.168.10.69 |
| runner_z1/0 (4bce5874-2edc-4a0f-a074-0a136933de65)
| running | n/a | runner_z1 | 192.168.10.56 |
| stats_z1/0 (20c1f5e9-3b7a-4c84-a940-3628cadf942c)
| running | n/a | small_z1 | 192.168.10.51 |
| uaa_z1/0 (9792327b-05c6-4bd8-8434-8a48fa364a91)
| running | n/a | medium_z1 | 192.168.10.53 |
+---------------------------------------------------------------------------+---------+-----+-----------+---------------+
The vcap job logs also seem to show it thinks that the blobstore node is
healthey,
However, when I ssh to the api node, I see problems:
"monit summary" yields:
Process 'cloud_controller_ng' not monitored
Process 'cloud_controller_worker_local_1' not monitored
Process 'cloud_controller_worker_local_2' not monitored
Process 'nginx_cc' not monitored
Process 'cloud_controller_migration' running
Process 'cloud_controller_clock' running
Process 'cloud_controller_worker_1' running
Process 'metron_agent' running
Process 'statsd-injector' running
Process 'route_registrar' running
System 'system_localhost' running
, /var/vcap/sys/log/cloud_controller_ng/cloud_controller_ng_ctl.log looks
like this:
[2016-06-24 12:32:51+0000] Checking for blobstore availability
[2016-06-24 12:33:38+0000] ------------ STARTING cloud_controller_ng_ctl
at Fri Jun 24 12:33:38 UTC 2016 --------------
[2016-06-24 12:33:38+0000] Checking for blobstore availability
[2016-06-24 12:34:24+0000] ------------ STARTING cloud_controller_ng_ctl
at Fri Jun 24 12:34:24 UTC 2016 --------------
[2016-06-24 12:34:24+0000] Checking for blobstore availability
[2016-06-24 12:34:41+0000] Blobstore is not available
Other jobs on the same node are failing as well, all with similar
messages both in stderr and stdout logs.
However, I do not see any additional information, such as an IP address
or URL that I can test.
Additionally on the blobstore node, both "monit" and the vcap job logs
seem to think that it is healthy.
How should I go about debugging this error? (I am looking for more
verbose information on the error "blobstore is not available")
I do not mind tearing down this deployment and starting over with more
verbose flags to expose the problem.
It seems to occur consitantly.
PS: my stemcell is
bosh-stemcell-3232.6-openstack-kvm-ubuntu-trusty-go_agent.tgz , and I have
not done anything special to configure blobstore on cf-stub besides adding
TLS Ca's and certs. Also, I could find no instructions on how to add these
certs to cf-stub, so I just added some self-generated keys & certs.
|
|
|
Re: How shoulld I debug a blobstore error?
toggle quoted message
Show quoted text
On Fri, Jun 24, 2016 at 6:54 AM, Eyal Shalev <eshalev(a)cisco.com> wrote: Hello, I am trying to deploy cloud foundry on openstack.
running "bosh deploy" gives me the following error:
Started updating job consul_z1 > consul_z1/0
(d248a6da-476a-4fb1-bc08-01ce23c94eb4) (canary). Done (00:01:03)
Started updating job ha_proxy_z1 > ha_proxy_z1/0
(f416b7c5-24d3-4d58-bbf9-066740971065) (canary). Done (00:00:43)
Started updating job nats_z1 > nats_z1/0
(074d00e8-96b3-4b7c-af1a-347549ff51dd) (canary). Done (00:00:44)
Started updating job etcd_z1 > etcd_z1/0
(ef27b75d-e0d0-4a95-8ec1-4b0c9371fa93) (canary). Done (00:01:23)
Started updating job stats_z1 > stats_z1/0
(20c1f5e9-3b7a-4c84-a940-3628cadf942c) (canary). Done (00:00:47)
Started updating job blobstore_z1 > blobstore_z1/0
(28f3cf80-502b-4e83-8dff-ba3477894a13) (canary). Done (00:01:20)
Started updating job postgres_z1 > postgres_z1/0
(601262d5-97d2-436c-b67d-1e63680f9dc6) (canary). Done (00:01:16)
Started updating job uaa_z1 > uaa_z1/0
(9792327b-05c6-4bd8-8434-8a48fa364a91) (canary). Done (00:00:52)
Started updating job api_z1 > api_z1/0
(03f63cee-dfc1-4a93-9ddc-1bf129eb7396) (canary). Failed: 'api_z1/0
(03f63cee-dfc1-4a93-9ddc-1bf129eb7396)' is not running after update. Review
logs for failed jobs: cloud_controller_ng, cloud_controller_worker_local_1,
cloud_controller_worker_local_2, nginx_cc, cloud_controller_worker_1
(00:11:11)
Error 400007: 'api_z1/0 (03f63cee-dfc1-4a93-9ddc-1bf129eb7396)' is not
running after update. Review logs for failed jobs: cloud_controller_ng,
cloud_controller_worker_local_1, cloud_controller_worker_local_2, nginx_cc,
cloud_controller_worker_1
"bosh vms" command seem to show blobstore be up and running:
+---------------------------------------------------------------------------+---------+-----+-----------+---------------+
| VM
| State | AZ | VM Type | IPs |
+---------------------------------------------------------------------------+---------+-----+-----------+---------------+
| api_z1/0 (03f63cee-dfc1-4a93-9ddc-1bf129eb7396)
| failing | n/a | large_z1 | 192.168.10.54 |
| blobstore_z1/0 (28f3cf80-502b-4e83-8dff-ba3477894a13)
| running | n/a | medium_z1 | 192.168.10.52 |
| consul_z1/0 (d248a6da-476a-4fb1-bc08-01ce23c94eb4)
| running | n/a | small_z1 | 192.168.10.76 |
| doppler_z1/0 (ec0232f4-e736-4bbf-9fe4-b35b3d0c673d)
| running | n/a | medium_z1 | 192.168.10.57 |
| etcd_z1/0 (ef27b75d-e0d0-4a95-8ec1-4b0c9371fa93)
| running | n/a | medium_z1 | 192.168.10.72 |
| ha_proxy_z1/0 (f416b7c5-24d3-4d58-bbf9-066740971065)
| running | n/a | router_z1 | 192.168.10.64 |
|
| | | | XX.XX.XX.XX |
| hm9000_z1/0 (c50fbd8e-1c8f-4b96-a1c8-a427fdabba79)
| running | n/a | medium_z1 | 192.168.10.55 |
| loggregator_trafficcontroller_z1/0
(e187f9c1-dc5d-4d9b-850d-ca6118513228) | running | n/a | small_z1 |
192.168.10.58 |
| nats_z1/0 (074d00e8-96b3-4b7c-af1a-347549ff51dd)
| running | n/a | medium_z1 | 192.168.10.66 |
| postgres_z1/0 (601262d5-97d2-436c-b67d-1e63680f9dc6)
| running | n/a | medium_z1 | 192.168.10.68 |
| router_z1/0 (4da988b9-5cbb-4a6f-b082-5c4ed25e6a7c)
| running | n/a | router_z1 | 192.168.10.69 |
| runner_z1/0 (4bce5874-2edc-4a0f-a074-0a136933de65)
| running | n/a | runner_z1 | 192.168.10.56 |
| stats_z1/0 (20c1f5e9-3b7a-4c84-a940-3628cadf942c)
| running | n/a | small_z1 | 192.168.10.51 |
| uaa_z1/0 (9792327b-05c6-4bd8-8434-8a48fa364a91)
| running | n/a | medium_z1 | 192.168.10.53 |
+---------------------------------------------------------------------------+---------+-----+-----------+---------------+
The vcap job logs also seem to show it thinks that the blobstore node is
healthey,
However, when I ssh to the api node, I see problems:
"monit summary" yields:
Process 'cloud_controller_ng' not monitored
Process 'cloud_controller_worker_local_1' not monitored
Process 'cloud_controller_worker_local_2' not monitored
Process 'nginx_cc' not monitored
Process 'cloud_controller_migration' running
Process 'cloud_controller_clock' running
Process 'cloud_controller_worker_1' running
Process 'metron_agent' running
Process 'statsd-injector' running
Process 'route_registrar' running
System 'system_localhost' running
, /var/vcap/sys/log/cloud_controller_ng/cloud_controller_ng_ctl.log looks
like this:
[2016-06-24 12:32:51+0000] Checking for blobstore availability
[2016-06-24 12:33:38+0000] ------------ STARTING cloud_controller_ng_ctl
at Fri Jun 24 12:33:38 UTC 2016 --------------
[2016-06-24 12:33:38+0000] Checking for blobstore availability
[2016-06-24 12:34:24+0000] ------------ STARTING cloud_controller_ng_ctl
at Fri Jun 24 12:34:24 UTC 2016 --------------
[2016-06-24 12:34:24+0000] Checking for blobstore availability
[2016-06-24 12:34:41+0000] Blobstore is not available
Other jobs on the same node are failing as well, all with similar messages
both in stderr and stdout logs.
However, I do not see any additional information, such as an IP address or
URL that I can test.
Additionally on the blobstore node, both "monit" and the vcap job logs
seem to think that it is healthy.
How should I go about debugging this error? (I am looking for more verbose
information on the error "blobstore is not available")
I do not mind tearing down this deployment and starting over with more
verbose flags to expose the problem.
It seems to occur consitantly.
PS: my stemcell is
bosh-stemcell-3232.6-openstack-kvm-ubuntu-trusty-go_agent.tgz , and I have
not done anything special to configure blobstore on cf-stub besides adding
TLS Ca's and certs. Also, I could find no instructions on how to add these
certs to cf-stub, so I just added some self-generated keys & certs.
|
|
|
Re: Consul Encryption in CF v234+
Hi there!
We found that if you monit stop all the consul nodes before upgrading and adding ssl. The deployment should upgrade without any issues.
Thanks,
Long
toggle quoted message
Show quoted text
On June 23, 2016 at 11:56:04 AM, Hiort, Carsten, Springer DE (carsten.hiort(a)springer.com) wrote:
Hi,
CF v234 enforces the use of SSL for Consul. We are currently wondering if there is a supposed upgrade path.
When you switch to SSL and the Consul cluster get’s upgraded all machines that are not yet upgraded will be blind with respect to service discovery/ DNS through Consul. This particularly affects the DEAs as they are not able to figure out where to get the droplets from when staging causing a 500 when cf pushing. I did already try deploying the Certs on 231 with require_ssl=false but then setting require_ssl true or upgrading to v234+ will still rest in this situation.
Any thoughts highly appreciated!
Thanks,
Carsten
---
Carsten Hiort
Platform Engineer
Platform Engineering
SpringerNature
Abraham-Lincoln-Str. 46, 65189 Wiesbaden, Germany
T +49 611 7878665
M +49 175 2965802
carsten.hiort(a)springernature.com
www.springernature.com
—
Springer Nature is one of the world’s leading global research, educational and professional publishers, created in May 2015 through the combination of Nature Publishing Group,
Palgrave Macmillan, Macmillan Education and Springer Science+Business Media
—
Springer Fachmedien Wiesbaden GmbH
Registered Office: Wiesbaden | Amtsgericht Wiesbaden, HRB 9754
Directors: Armin Gross, Joachim Krieger, Dr. Niels Peter Thomas
|
|
|
