i am hitting an error when i execute bosh deploy in minimal-aws.yml based
cf deployment.

Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not
running after update (00:12:11)

Error 400007: `api_z1/0' is not running after update

Any suggestions to resolve this? I tried in #bosh freenode but no one
responded.

I see this error related to api_z1 in the ./cloud_controller_worker_ctl.log

./cloud_controller_worker_ctl.log:{"timestamp":1434996874.6546366,"message":"(0.003843s)
UPDATE \"delayed_jobs\" SET \"guid\" =
'a3af8feb-30b7-4b4d-91fd-112045501a0d', \"created_at\" = '2015-06-22
18:14:32.651086+0000', \"updated_at\" = CURRENT_TIMESTAMP, \"priority\" =
0, \"attempts\" = 0, \"handler\" = '---
!ruby/struct:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler:
!ruby/struct:VCAP::CloudController::Jobs::RequestJob\n job:
!ruby/struct:VCAP::CloudController::Jobs::TimeoutJob\n job:
!ruby/struct:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n
name: java_buildpack\n file:
\"/var/vcap/packages/buildpack_java/java-buildpack-v2.5.zip\"\n opts:
{}\n config: \n request_id: \n', \"last_error\" = NULL, \"run_at\" =
'2015-06-22 18:14:32.660520+0000', \"locked_at\" = '2015-06-22
18:14:34.649651+0000', \"failed_at\" = NULL, \"locked_by\" = 'cc_api_worker.
*api_z1*.0.2', \"queue\" = 'cc-*api_z1*-0', \"cf_api_error\" = NULL WHERE
(\"id\" =
1)","log_level":"debug2","source":"cc.background","data":{},"thread_id":69964119192340,"fiber_id":69964148702360,"process_id":2727,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

I searched online but it seems like there are some others hitting the same
issue. I could not find a proper solution.


Sincerely,

Dulanjalie

Don t think is an error, please check all files.

On Tue, Jun 23, 2015 at 1:15 PM, Dulanjalie Dhanapala <dulanjalie(a)gmail.com>
wrote:

i am hitting an error when i execute bosh deploy in minimal-aws.yml based
cf deployment.

Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not
running after update (00:12:11)

Error 400007: `api_z1/0' is not running after update

Any suggestions to resolve this? I tried in #bosh freenode but no one
responded.

I see this error related to api_z1 in the ./cloud_controller_worker_
ctl.log

./cloud_controller_worker_ctl.log:{"timestamp":1434996874.6546366,"message":"(0.003843s)
UPDATE \"delayed_jobs\" SET \"guid\" =
'a3af8feb-30b7-4b4d-91fd-112045501a0d', \"created_at\" = '2015-06-22
18:14:32.651086+0000', \"updated_at\" = CURRENT_TIMESTAMP, \"priority\" =
0, \"attempts\" = 0, \"handler\" = '---
!ruby/struct:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler:
!ruby/struct:VCAP::CloudController::Jobs::RequestJob\n job:
!ruby/struct:VCAP::CloudController::Jobs::TimeoutJob\n job:
!ruby/struct:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n
name: java_buildpack\n file:
\"/var/vcap/packages/buildpack_java/java-buildpack-v2.5.zip\"\n opts:
{}\n config: \n request_id: \n', \"last_error\" = NULL, \"run_at\" =
'2015-06-22 18:14:32.660520+0000', \"locked_at\" = '2015-06-22
18:14:34.649651+0000', \"failed_at\" = NULL, \"locked_by\" = 'cc_api_worker.
*api_z1*.0.2', \"queue\" = 'cc-*api_z1*-0', \"cf_api_error\" = NULL WHERE
(\"id\" =
1)","log_level":"debug2","source":"cc.background","data":{},"thread_id":69964119192340,"fiber_id":69964148702360,"process_id":2727,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

I searched online but it seems like there are some others hitting the same
issue. I could not find a proper solution.


Sincerely,

Dulanjalie

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

I am seeing this error. I am checking other logs
=======================cloud_controller_worker_ctl.err.log==============================

rake aborted!

SignalException: SIGTERM

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient/session.rb:20:in
`require'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient/session.rb:20:in
`<top (required)>'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient.rb:17:in
`require'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient.rb:17:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/app/controllers/runtime/files_controller.rb:1:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/app/controllers/runtime/files_controller.rb:1:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:4:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:4:in
`block in <top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:3:in
`each'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:3:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller.rb:46:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller.rb:46:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/Rakefile:10:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/Rakefile:10:in
`<top (required)>'

(See full trace by running task with --trace)

On Mon, Jun 22, 2015 at 9:39 PM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

Don t think is an error, please check all files.

On Tue, Jun 23, 2015 at 1:15 PM, Dulanjalie Dhanapala <
dulanjalie(a)gmail.com> wrote:

i am hitting an error when i execute bosh deploy in minimal-aws.yml
based cf deployment.

Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not
running after update (00:12:11)

Error 400007: `api_z1/0' is not running after update

Any suggestions to resolve this? I tried in #bosh freenode but no one
responded.

I see this error related to api_z1 in the ./cloud_controller_worker_
ctl.log

./cloud_controller_worker_ctl.log:{"timestamp":1434996874.6546366,"message":"(0.003843s)
UPDATE \"delayed_jobs\" SET \"guid\" =
'a3af8feb-30b7-4b4d-91fd-112045501a0d', \"created_at\" = '2015-06-22
18:14:32.651086+0000', \"updated_at\" = CURRENT_TIMESTAMP, \"priority\" =
0, \"attempts\" = 0, \"handler\" = '---
!ruby/struct:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler:
!ruby/struct:VCAP::CloudController::Jobs::RequestJob\n job:
!ruby/struct:VCAP::CloudController::Jobs::TimeoutJob\n job:
!ruby/struct:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n
name: java_buildpack\n file:
\"/var/vcap/packages/buildpack_java/java-buildpack-v2.5.zip\"\n opts:
{}\n config: \n request_id: \n', \"last_error\" = NULL, \"run_at\" =
'2015-06-22 18:14:32.660520+0000', \"locked_at\" = '2015-06-22
18:14:34.649651+0000', \"failed_at\" = NULL, \"locked_by\" = 'cc_api_worker.
*api_z1*.0.2', \"queue\" = 'cc-*api_z1*-0', \"cf_api_error\" = NULL
WHERE (\"id\" =
1)","log_level":"debug2","source":"cc.background","data":{},"thread_id":69964119192340,"fiber_id":69964148702360,"process_id":2727,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

I searched online but it seems like there are some others hitting the
same issue. I could not find a proper solution.


Sincerely,

Dulanjalie

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

--
Dulanjalie

Some more error messages

------------ STARTING cloud_controller_clock_ctl at Tue Jun 23 04:58:50 UTC
2015 --------------

c9/cloud_controller_ng/lib/cloud_controller/background_job_environment.rb:10:in
`setup_environment'

lib/tasks/clock.rake:6:in `block (2 levels) in <top (required)>'

PG::UndefinedTable: ERROR: relation "apps" does not exist

LINE 1: SELECT * FROM "apps" LIMIT 1


------------ STARTING cloud_controller_ng_ctl at Tue Jun 23 04:58:52 UTC
2015 --------------

Preparing local package directory

Preparing local resource_pool directory

Preparing local droplet directory

{"timestamp":1435035533.863138,"message":"PG::UndefinedTable: ERROR:
relation \"schema_migrations\" does not exist\nLINE 1: SELECT NULL AS
\"nil\" FROM \"schema_migrations\" LIMIT 1\n
^: SELECT NULL AS \"nil\" FROM \"schema_migrations\" LIMIT
1","log_level":"error","source":"cc.db.migrations","data":{},"thread_id":69988649218860,"fiber_id":69988678772900,"process_id":2704,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}


=====================
cloud_controller_clock_ctl.log===========================

{"timestamp":1435035531.961396,"message":"PG::UndefinedTable: ERROR:
relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM
\"billing_events\" LIMIT 1\n ^: SELECT * FROM
\"billing_events\" LIMIT
1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

{"timestamp":1435035531.9634626,"message":"PG::UndefinedTable: ERROR:
relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM
\"billing_events\" LIMIT 1\n ^: SELECT * FROM
\"billing_events\" LIMIT
1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

{"timestamp":1435035531.9643211,"message":"PG::UndefinedTable: ERROR:
relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM
\"billing_events\" LIMIT 1\n ^: SELECT * FROM
\"billing_events\" LIMIT
1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

--
Dulanjalie

this looks like DB migrations may be messed up.

--
Thank you,

James Bayer

Best way delete deployment and redeploy to start from clean env.

Thanks a lot for the respond.
Pablo, Lev and Nic also gave me some pointers. I am redeploying with cf-212.

thanks all.
Sincerely,
Dulanjalie

On Wed, Jun 24, 2015 at 2:03 AM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

Best way delete deployment and redeploy to start from clean env.



On Wed, Jun 24, 2015 at 5:07 PM, James Bayer <jbayer(a)pivotal.io> wrote:

this looks like DB migrations may be messed up.

On Mon, Jun 22, 2015 at 10:32 PM, Dulanjalie Dhanapala <
dulanjalie(a)gmail.com> wrote:

Some more error messages

------------ STARTING cloud_controller_clock_ctl at Tue Jun 23 04:58:50
UTC 2015 --------------

c9/cloud_controller_ng/lib/cloud_controller/background_job_environment.rb:10:in
`setup_environment'

lib/tasks/clock.rake:6:in `block (2 levels) in <top (required)>'

PG::UndefinedTable: ERROR: relation "apps" does not exist

LINE 1: SELECT * FROM "apps" LIMIT 1


------------ STARTING cloud_controller_ng_ctl at Tue Jun 23 04:58:52 UTC
2015 --------------

Preparing local package directory

Preparing local resource_pool directory

Preparing local droplet directory

{"timestamp":1435035533.863138,"message":"PG::UndefinedTable: ERROR:
relation \"schema_migrations\" does not exist\nLINE 1: SELECT NULL AS
\"nil\" FROM \"schema_migrations\" LIMIT 1\n
^: SELECT NULL AS \"nil\" FROM \"schema_migrations\" LIMIT
1","log_level":"error","source":"cc.db.migrations","data":{},"thread_id":69988649218860,"fiber_id":69988678772900,"process_id":2704,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}


=====================
cloud_controller_clock_ctl.log===========================

{"timestamp":1435035531.961396,"message":"PG::UndefinedTable: ERROR:
relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM
\"billing_events\" LIMIT 1\n ^: SELECT * FROM
\"billing_events\" LIMIT
1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

{"timestamp":1435035531.9634626,"message":"PG::UndefinedTable: ERROR:
relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM
\"billing_events\" LIMIT 1\n ^: SELECT * FROM
\"billing_events\" LIMIT
1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

{"timestamp":1435035531.9643211,"message":"PG::UndefinedTable: ERROR:
relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM
\"billing_events\" LIMIT 1\n ^: SELECT * FROM
\"billing_events\" LIMIT
1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

On Mon, Jun 22, 2015 at 10:17 PM, Dulanjalie Dhanapala <
dulanjalie(a)gmail.com> wrote:

I am seeing this error. I am checking other logs

=======================cloud_controller_worker_ctl.err.log==============================

rake aborted!

SignalException: SIGTERM

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient/session.rb:20:in
`require'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient/session.rb:20:in
`<top (required)>'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient.rb:17:in
`require'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient.rb:17:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/app/controllers/runtime/files_controller.rb:1:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/app/controllers/runtime/files_controller.rb:1:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:4:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:4:in
`block in <top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:3:in
`each'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:3:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller.rb:46:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller.rb:46:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/Rakefile:10:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/Rakefile:10:in
`<top (required)>'

(See full trace by running task with --trace)

On Mon, Jun 22, 2015 at 9:39 PM, Gwenn Etourneau <getourneau(a)pivotal.io

wrote:

Don t think is an error, please check all files.

On Tue, Jun 23, 2015 at 1:15 PM, Dulanjalie Dhanapala <
dulanjalie(a)gmail.com> wrote:

i am hitting an error when i execute bosh deploy in minimal-aws.yml
based cf deployment.

Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not
running after update (00:12:11)

Error 400007: `api_z1/0' is not running after update

Any suggestions to resolve this? I tried in #bosh freenode but no one
responded.

I see this error related to api_z1 in the ./cloud_controller_worker_
ctl.log

./cloud_controller_worker_ctl.log:{"timestamp":1434996874.6546366,"message":"(0.003843s)
UPDATE \"delayed_jobs\" SET \"guid\" =
'a3af8feb-30b7-4b4d-91fd-112045501a0d', \"created_at\" = '2015-06-22
18:14:32.651086+0000', \"updated_at\" = CURRENT_TIMESTAMP, \"priority\" =
0, \"attempts\" = 0, \"handler\" = '---
!ruby/struct:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler:
!ruby/struct:VCAP::CloudController::Jobs::RequestJob\n job:
!ruby/struct:VCAP::CloudController::Jobs::TimeoutJob\n job:
!ruby/struct:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n
name: java_buildpack\n file:
\"/var/vcap/packages/buildpack_java/java-buildpack-v2.5.zip\"\n opts:
{}\n config: \n request_id: \n', \"last_error\" = NULL, \"run_at\" =
'2015-06-22 18:14:32.660520+0000', \"locked_at\" = '2015-06-22
18:14:34.649651+0000', \"failed_at\" = NULL, \"locked_by\" = 'cc_api_worker.
*api_z1*.0.2', \"queue\" = 'cc-*api_z1*-0', \"cf_api_error\" = NULL
WHERE (\"id\" =
1)","log_level":"debug2","source":"cc.background","data":{},"thread_id":69964119192340,"fiber_id":69964148702360,"process_id":2727,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

I searched online but it seems like there are some others hitting the
same issue. I could not find a proper solution.


Sincerely,

Dulanjalie

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

--
Dulanjalie

--
Dulanjalie

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

--
Thank you,

James Bayer

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

--
Dulanjalie

Hi all,
I deleted my "cf" deployment
git cloned the latest https://github.com/cloudfoundry/cf-release.git

I am using the latest stemcells 2999

+-----------------------------------------+---------+--------------------+

| Name | Version | CID |

+-----------------------------------------+---------+--------------------+

| bosh-aws-xen-hvm-ubuntu-trusty-go_agent | 2989 | ami-8bc83be0 light |

| bosh-aws-xen-hvm-ubuntu-trusty-go_agent | 2999* | ami-4dcc3526 light |

+-----------------------------------------+---------+--------------------+

(*) Currently in-use

Finally, i am using 212 release

bosh releases

+------+----------+-------------+

| Name | Versions | Commit Hash |

+------+----------+-------------+

| cf | 193 | 54613ebb+ |

| | 212* | ae2ec7a5+ |

+------+----------+-------------+

(*) Currently deployed

(+) Uncommitted changes

Still fails with

Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not
running after update (00:12:18)

Error 400007: `api_z1/0' is not running after update


But error logs are different:
http://fpaste.org/236236/65314143/
http://fpaste.org/236241/16579314/

Still "Buildpacks installation failed"

Sincerely,
Dulanjalie

On Wed, Jun 24, 2015 at 7:04 AM, Dulanjalie Dhanapala <dulanjalie(a)gmail.com>
wrote:

Thanks a lot for the respond.
Pablo, Lev and Nic also gave me some pointers. I am redeploying with
cf-212.

thanks all.
Sincerely,
Dulanjalie

On Wed, Jun 24, 2015 at 2:03 AM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

Best way delete deployment and redeploy to start from clean env.



On Wed, Jun 24, 2015 at 5:07 PM, James Bayer <jbayer(a)pivotal.io> wrote:

this looks like DB migrations may be messed up.

On Mon, Jun 22, 2015 at 10:32 PM, Dulanjalie Dhanapala <
dulanjalie(a)gmail.com> wrote:

Some more error messages

------------ STARTING cloud_controller_clock_ctl at Tue Jun 23 04:58:50
UTC 2015 --------------

c9/cloud_controller_ng/lib/cloud_controller/background_job_environment.rb:10:in
`setup_environment'

lib/tasks/clock.rake:6:in `block (2 levels) in <top (required)>'

PG::UndefinedTable: ERROR: relation "apps" does not exist

LINE 1: SELECT * FROM "apps" LIMIT 1


------------ STARTING cloud_controller_ng_ctl at Tue Jun 23 04:58:52
UTC 2015 --------------

Preparing local package directory

Preparing local resource_pool directory

Preparing local droplet directory

{"timestamp":1435035533.863138,"message":"PG::UndefinedTable: ERROR:
relation \"schema_migrations\" does not exist\nLINE 1: SELECT NULL AS
\"nil\" FROM \"schema_migrations\" LIMIT 1\n
^: SELECT NULL AS \"nil\" FROM \"schema_migrations\" LIMIT
1","log_level":"error","source":"cc.db.migrations","data":{},"thread_id":69988649218860,"fiber_id":69988678772900,"process_id":2704,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}


=====================
cloud_controller_clock_ctl.log===========================

{"timestamp":1435035531.961396,"message":"PG::UndefinedTable: ERROR:
relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM
\"billing_events\" LIMIT 1\n ^: SELECT * FROM
\"billing_events\" LIMIT
1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

{"timestamp":1435035531.9634626,"message":"PG::UndefinedTable: ERROR:
relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM
\"billing_events\" LIMIT 1\n ^: SELECT * FROM
\"billing_events\" LIMIT
1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

{"timestamp":1435035531.9643211,"message":"PG::UndefinedTable: ERROR:
relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM
\"billing_events\" LIMIT 1\n ^: SELECT * FROM
\"billing_events\" LIMIT
1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

On Mon, Jun 22, 2015 at 10:17 PM, Dulanjalie Dhanapala <
dulanjalie(a)gmail.com> wrote:

I am seeing this error. I am checking other logs

=======================cloud_controller_worker_ctl.err.log==============================

rake aborted!

SignalException: SIGTERM

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient/session.rb:20:in
`require'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient/session.rb:20:in
`<top (required)>'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient.rb:17:in
`require'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient.rb:17:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/app/controllers/runtime/files_controller.rb:1:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/app/controllers/runtime/files_controller.rb:1:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:4:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:4:in
`block in <top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:3:in
`each'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:3:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller.rb:46:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller.rb:46:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/Rakefile:10:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/Rakefile:10:in
`<top (required)>'

(See full trace by running task with --trace)

On Mon, Jun 22, 2015 at 9:39 PM, Gwenn Etourneau <
getourneau(a)pivotal.io> wrote:

Don t think is an error, please check all files.

On Tue, Jun 23, 2015 at 1:15 PM, Dulanjalie Dhanapala <
dulanjalie(a)gmail.com> wrote:

i am hitting an error when i execute bosh deploy in minimal-aws.yml
based cf deployment.

Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not
running after update (00:12:11)

Error 400007: `api_z1/0' is not running after update

Any suggestions to resolve this? I tried in #bosh freenode but no
one responded.

I see this error related to api_z1 in the ./cloud_controller_worker_
ctl.log

./cloud_controller_worker_ctl.log:{"timestamp":1434996874.6546366,"message":"(0.003843s)
UPDATE \"delayed_jobs\" SET \"guid\" =
'a3af8feb-30b7-4b4d-91fd-112045501a0d', \"created_at\" = '2015-06-22
18:14:32.651086+0000', \"updated_at\" = CURRENT_TIMESTAMP, \"priority\" =
0, \"attempts\" = 0, \"handler\" = '---
!ruby/struct:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler:
!ruby/struct:VCAP::CloudController::Jobs::RequestJob\n job:
!ruby/struct:VCAP::CloudController::Jobs::TimeoutJob\n job:
!ruby/struct:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n
name: java_buildpack\n file:
\"/var/vcap/packages/buildpack_java/java-buildpack-v2.5.zip\"\n opts:
{}\n config: \n request_id: \n', \"last_error\" = NULL, \"run_at\" =
'2015-06-22 18:14:32.660520+0000', \"locked_at\" = '2015-06-22
18:14:34.649651+0000', \"failed_at\" = NULL, \"locked_by\" = 'cc_api_worker.
*api_z1*.0.2', \"queue\" = 'cc-*api_z1*-0', \"cf_api_error\" = NULL
WHERE (\"id\" =
1)","log_level":"debug2","source":"cc.background","data":{},"thread_id":69964119192340,"fiber_id":69964148702360,"process_id":2727,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

I searched online but it seems like there are some others hitting
the same issue. I could not find a proper solution.


Sincerely,

Dulanjalie

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

--
Dulanjalie

--
Dulanjalie

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

--
Thank you,

James Bayer

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

--
Dulanjalie

--
Dulanjalie

Dulanjalie,
did you ever get this resolved?
I am running into the same issue, with a blank-slate 212 install:

Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not
running after update (00:12:18)

Error 400007: `api_z1/0' is not running after update

thanks,
Matthias

From: cf-bosh-bounces(a)lists.cloudfoundry.org [mailto:cf-bosh-bounces(a)lists.cloudfoundry.org] On Behalf Of Dulanjalie Dhanapala
Sent: Wednesday, June 24, 2015 1:14 PM
To: Discussions about the Cloud Foundry BOSH project.
Subject: Re: [cf-bosh] Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not running after update

Hi all,
I deleted my "cf" deployment
git cloned the latest https://github.com/cloudfoundry/cf-release.git

I am using the latest stemcells 2999

+-----------------------------------------+---------+--------------------+

| Name | Version | CID |

+-----------------------------------------+---------+--------------------+

| bosh-aws-xen-hvm-ubuntu-trusty-go_agent | 2989 | ami-8bc83be0 light |

| bosh-aws-xen-hvm-ubuntu-trusty-go_agent | 2999* | ami-4dcc3526 light |

+-----------------------------------------+---------+--------------------+

(*) Currently in-use

Finally, i am using 212 release

bosh releases

+------+----------+-------------+

| Name | Versions | Commit Hash |

+------+----------+-------------+

| cf | 193 | 54613ebb+ |

| | 212* | ae2ec7a5+ |

+------+----------+-------------+

(*) Currently deployed

(+) Uncommitted changes

Still fails with

Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not running after update (00:12:18)

Error 400007: `api_z1/0' is not running after update



But error logs are different:
http://fpaste.org/236236/65314143/
http://fpaste.org/236241/16579314/

Still "Buildpacks installation failed"

Sincerely,
Dulanjalie

On Wed, Jun 24, 2015 at 7:04 AM, Dulanjalie Dhanapala <dulanjalie(a)gmail.com<mailto:dulanjalie(a)gmail.com>> wrote:
Thanks a lot for the respond.
Pablo, Lev and Nic also gave me some pointers. I am redeploying with cf-212.

thanks all.
Sincerely,
Dulanjalie

On Wed, Jun 24, 2015 at 2:03 AM, Gwenn Etourneau <getourneau(a)pivotal.io<mailto:getourneau(a)pivotal.io>> wrote:
Best way delete deployment and redeploy to start from clean env.



On Wed, Jun 24, 2015 at 5:07 PM, James Bayer <jbayer(a)pivotal.io<mailto:jbayer(a)pivotal.io>> wrote:
this looks like DB migrations may be messed up.

On Mon, Jun 22, 2015 at 10:32 PM, Dulanjalie Dhanapala <dulanjalie(a)gmail.com<mailto:dulanjalie(a)gmail.com>> wrote:

Some more error messages

------------ STARTING cloud_controller_clock_ctl at Tue Jun 23 04:58:50 UTC 2015 --------------

c9/cloud_controller_ng/lib/cloud_controller/background_job_environment.rb:10:in `setup_environment'

lib/tasks/clock.rake:6:in `block (2 levels) in <top (required)>'

PG::UndefinedTable: ERROR: relation "apps" does not exist

LINE 1: SELECT * FROM "apps" LIMIT 1



------------ STARTING cloud_controller_ng_ctl at Tue Jun 23 04:58:52 UTC 2015 --------------

Preparing local package directory

Preparing local resource_pool directory

Preparing local droplet directory

{"timestamp":1435035533.863138,"message":"PG::UndefinedTable: ERROR: relation \"schema_migrations\" does not exist\nLINE 1: SELECT NULL AS \"nil\" FROM \"schema_migrations\" LIMIT 1\n ^: SELECT NULL AS \"nil\" FROM \"schema_migrations\" LIMIT 1","log_level":"error","source":"cc.db.migrations","data":{},"thread_id":69988649218860,"fiber_id":69988678772900,"process_id":2704,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}



=====================cloud_controller_clock_ctl.log===========================

{"timestamp":1435035531.961396,"message":"PG::UndefinedTable: ERROR: relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM \"billing_events\" LIMIT 1\n ^: SELECT * FROM \"billing_events\" LIMIT 1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}

{"timestamp":1435035531.9634626,"message":"PG::UndefinedTable: ERROR: relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM \"billing_events\" LIMIT 1\n ^: SELECT * FROM \"billing_events\" LIMIT 1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}

{"timestamp":1435035531.9643211,"message":"PG::UndefinedTable: ERROR: relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM \"billing_events\" LIMIT 1\n ^: SELECT * FROM \"billing_events\" LIMIT 1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}

On Mon, Jun 22, 2015 at 10:17 PM, Dulanjalie Dhanapala <dulanjalie(a)gmail.com<mailto:dulanjalie(a)gmail.com>> wrote:
I am seeing this error. I am checking other logs
=======================cloud_controller_worker_ctl.err.log==============================

rake aborted!

SignalException: SIGTERM

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient/session.rb:20:in `require'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient/session.rb:20:in `<top (required)>'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient.rb:17:in `require'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient.rb:17:in `<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/app/controllers/runtime/files_controller.rb:1:in `require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/app/controllers/runtime/files_controller.rb:1:in `<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:4:in `require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:4:in `block in <top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:3:in `each'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:3:in `<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller.rb:46:in `require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller.rb:46:in `<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/Rakefile:10:in `require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/Rakefile:10:in `<top (required)>'

(See full trace by running task with --trace)

On Mon, Jun 22, 2015 at 9:39 PM, Gwenn Etourneau <getourneau(a)pivotal.io<mailto:getourneau(a)pivotal.io>> wrote:
Don t think is an error, please check all files.

On Tue, Jun 23, 2015 at 1:15 PM, Dulanjalie Dhanapala <dulanjalie(a)gmail.com<mailto:dulanjalie(a)gmail.com>> wrote:

i am hitting an error when i execute bosh deploy in minimal-aws.yml based cf deployment.


Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not running after update (00:12:11)

Error 400007: `api_z1/0' is not running after update

Any suggestions to resolve this? I tried in #bosh freenode but no one responded.

I see this error related to api_z1 in the ./cloud_controller_worker_ctl.log

./cloud_controller_worker_ctl.log:{"timestamp":1434996874.6546366,"message":"(0.003843s) UPDATE \"delayed_jobs\" SET \"guid\" = 'a3af8feb-30b7-4b4d-91fd-112045501a0d', \"created_at\" = '2015-06-22 18:14:32.651086+0000', \"updated_at\" = CURRENT_TIMESTAMP, \"priority\" = 0, \"attempts\" = 0, \"handler\" = '--- !ruby/struct:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler: !ruby/struct:VCAP::CloudController::Jobs::RequestJob\n job: !ruby/struct:VCAP::CloudController::Jobs::TimeoutJob\n job: !ruby/struct:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n name: java_buildpack\n file: \"/var/vcap/packages/buildpack_java/java-buildpack-v2.5.zip\"\n opts: {}\n config: \n request_id: \n', \"last_error\" = NULL, \"run_at\" = '2015-06-22 18:14:32.660520+0000', \"locked_at\" = '2015-06-22 18:14:34.649651+0000', \"failed_at\" = NULL, \"locked_by\" = 'cc_api_worker.api_z1.0.2', \"queue\" = 'cc-api_z1-0', \"cf_api_error\" = NULL WHERE (\"id\" = 1)","log_level":"debug2","source":"cc.background","data":{},"thread_id":69964119192340,"fiber_id":69964148702360,"process_id":2727,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}

I searched online but it seems like there are some others hitting the same issue. I could not find a proper solution.



Sincerely,

Dulanjalie

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org<mailto:cf-bosh(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh


_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org<mailto:cf-bosh(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh



--
Dulanjalie



--
Dulanjalie

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org<mailto:cf-bosh(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh



--
Thank you,

James Bayer

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org<mailto:cf-bosh(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh


_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org<mailto:cf-bosh(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh



--
Dulanjalie



--
Dulanjalie

Hi Matthias,
did you get into api_z1 0 via ssh and look at the specific error? The
problem here is, that the failure might arise from different sources.

On Fri, Jul 24, 2015 at 6:57 PM, Matthias Ender <Matthias.Ender(a)sas.com>
wrote:

Dulanjalie,

did you ever get this resolved?

I am running into the same issue, with a blank-slate 212 install:



Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not

running after update (00:12:18)



Error 400007: `api_z1/0' is not running after update



thanks,

Matthias



*From:* cf-bosh-bounces(a)lists.cloudfoundry.org [mailto:
cf-bosh-bounces(a)lists.cloudfoundry.org] *On Behalf Of *Dulanjalie
Dhanapala
*Sent:* Wednesday, June 24, 2015 1:14 PM
*To:* Discussions about the Cloud Foundry BOSH project.
*Subject:* Re: [cf-bosh] Failed updating job api_z1 > api_z1/0 (canary):
`api_z1/0' is not running after update



Hi all,

I deleted my "cf" deployment

git cloned the latest https://github.com/cloudfoundry/cf-release.git

I am using the latest stemcells 2999

+-----------------------------------------+---------+--------------------+

| Name | Version | CID |

+-----------------------------------------+---------+--------------------+

| bosh-aws-xen-hvm-ubuntu-trusty-go_agent | 2989 | ami-8bc83be0 light |

| bosh-aws-xen-hvm-ubuntu-trusty-go_agent | 2999* | ami-4dcc3526 light |

+-----------------------------------------+---------+--------------------+

(*) Currently in-use

Finally, i am using 212 release

bosh releases

+------+----------+-------------+

| Name | Versions | Commit Hash |

+------+----------+-------------+

| cf | 193 | 54613ebb+ |

| | 212* | ae2ec7a5+ |

+------+----------+-------------+

(*) Currently deployed

(+) Uncommitted changes

Still fails with

Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not
running after update (00:12:18)

Error 400007: `api_z1/0' is not running after update



But error logs are different:

http://fpaste.org/236236/65314143/

http://fpaste.org/236241/16579314/



Still "Buildpacks installation failed"



Sincerely,

Dulanjalie



On Wed, Jun 24, 2015 at 7:04 AM, Dulanjalie Dhanapala <
dulanjalie(a)gmail.com> wrote:

Thanks a lot for the respond.

Pablo, Lev and Nic also gave me some pointers. I am redeploying with
cf-212.



thanks all.

Sincerely,

Dulanjalie



On Wed, Jun 24, 2015 at 2:03 AM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

Best way delete deployment and redeploy to start from clean env.







On Wed, Jun 24, 2015 at 5:07 PM, James Bayer <jbayer(a)pivotal.io> wrote:

this looks like DB migrations may be messed up.



On Mon, Jun 22, 2015 at 10:32 PM, Dulanjalie Dhanapala <
dulanjalie(a)gmail.com> wrote:

Some more error messages

------------ STARTING cloud_controller_clock_ctl at Tue Jun 23 04:58:50
UTC 2015 --------------

c9/cloud_controller_ng/lib/cloud_controller/background_job_environment.rb:10:in
`setup_environment'

lib/tasks/clock.rake:6:in `block (2 levels) in <top (required)>'

PG::UndefinedTable: ERROR: relation "apps" does not exist

LINE 1: SELECT * FROM "apps" LIMIT 1



------------ STARTING cloud_controller_ng_ctl at Tue Jun 23 04:58:52 UTC
2015 --------------

Preparing local package directory

Preparing local resource_pool directory

Preparing local droplet directory

{"timestamp":1435035533.863138,"message":"PG::UndefinedTable: ERROR:
relation \"schema_migrations\" does not exist\nLINE 1: SELECT NULL AS
\"nil\" FROM \"schema_migrations\" LIMIT 1\n
^: SELECT NULL AS \"nil\" FROM \"schema_migrations\" LIMIT
1","log_level":"error","source":"cc.db.migrations","data":{},"thread_id":69988649218860,"fiber_id":69988678772900,"process_id":2704,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}




=====================cloud_controller_clock_ctl.log===========================

{"timestamp":1435035531.961396,"message":"PG::UndefinedTable: ERROR:
relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM
\"billing_events\" LIMIT 1\n ^: SELECT * FROM
\"billing_events\" LIMIT
1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

{"timestamp":1435035531.9634626,"message":"PG::UndefinedTable: ERROR:
relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM
\"billing_events\" LIMIT 1\n ^: SELECT * FROM
\"billing_events\" LIMIT
1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

{"timestamp":1435035531.9643211,"message":"PG::UndefinedTable: ERROR:
relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM
\"billing_events\" LIMIT 1\n ^: SELECT * FROM
\"billing_events\" LIMIT
1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}



On Mon, Jun 22, 2015 at 10:17 PM, Dulanjalie Dhanapala <
dulanjalie(a)gmail.com> wrote:

I am seeing this error. I am checking other logs


=======================cloud_controller_worker_ctl.err.log==============================

rake aborted!

SignalException: SIGTERM

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient/session.rb:20:in
`require'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient/session.rb:20:in
`<top (required)>'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient.rb:17:in
`require'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient.rb:17:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/app/controllers/runtime/files_controller.rb:1:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/app/controllers/runtime/files_controller.rb:1:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:4:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:4:in
`block in <top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:3:in
`each'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:3:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller.rb:46:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller.rb:46:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/Rakefile:10:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/Rakefile:10:in
`<top (required)>'

(See full trace by running task with --trace)



On Mon, Jun 22, 2015 at 9:39 PM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

Don t think is an error, please check all files.



On Tue, Jun 23, 2015 at 1:15 PM, Dulanjalie Dhanapala <
dulanjalie(a)gmail.com> wrote:


i am hitting an error when i execute bosh deploy in minimal-aws.yml
based cf deployment.



Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not
running after update (00:12:11)

Error 400007: `api_z1/0' is not running after update

Any suggestions to resolve this? I tried in #bosh freenode but no one
responded.



I see this error related to api_z1 in the ./cloud_controller_worker_ctl.log

./cloud_controller_worker_ctl.log:{"timestamp":1434996874.6546366,"message":"(0.003843s)
UPDATE \"delayed_jobs\" SET \"guid\" =
'a3af8feb-30b7-4b4d-91fd-112045501a0d', \"created_at\" = '2015-06-22
18:14:32.651086+0000', \"updated_at\" = CURRENT_TIMESTAMP, \"priority\" =
0, \"attempts\" = 0, \"handler\" = '---
!ruby/struct:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler:
!ruby/struct:VCAP::CloudController::Jobs::RequestJob\n job:
!ruby/struct:VCAP::CloudController::Jobs::TimeoutJob\n job:
!ruby/struct:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n
name: java_buildpack\n file:
\"/var/vcap/packages/buildpack_java/java-buildpack-v2.5.zip\"\n opts:
{}\n config: \n request_id: \n', \"last_error\" = NULL, \"run_at\" =
'2015-06-22 18:14:32.660520+0000', \"locked_at\" = '2015-06-22
18:14:34.649651+0000', \"failed_at\" = NULL, \"locked_by\" = 'cc_api_worker.
*api_z1*.0.2', \"queue\" = 'cc-*api_z1*-0', \"cf_api_error\" = NULL WHERE
(\"id\" =
1)","log_level":"debug2","source":"cc.background","data":{},"thread_id":69964119192340,"fiber_id":69964148702360,"process_id":2727,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

I searched online but it seems like there are some others hitting the same
issue. I could not find a proper solution.



Sincerely,

Dulanjalie



_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh




_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh





--

Dulanjalie





--

Dulanjalie


_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh





--

Thank you,



James Bayer


_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh




_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh





--

Dulanjalie





--

Dulanjalie

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

--
Mit freundlichen Grüßen

Johannes Hiemer

I noticed this in /var/vcap/sys/log/routing-api/routing-api.log

{"timestamp":"1437740486.292583704","source":"routing-api","message":"routing-api.database","log_level":1,"data":{"etcd-addresses":["http://10.10.3.8:4001"]}}
{"timestamp":"1437740486.293358803","source":"routing-api","message":"routing-api.failed to connect to etcd","log_level":2,"data":{"error":"sync cluster failed"}}

From: cf-bosh-bounces(a)lists.cloudfoundry.org [mailto:cf-bosh-bounces(a)lists.cloudfoundry.org] On Behalf Of Johannes Hiemer
Sent: Friday, July 24, 2015 1:12 PM
To: Discussions about the Cloud Foundry BOSH project.
Subject: Re: [cf-bosh] Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not running after update

Hi Matthias,
did you get into api_z1 0 via ssh and look at the specific error? The problem here is, that the failure might arise from different sources.

On Fri, Jul 24, 2015 at 6:57 PM, Matthias Ender <Matthias.Ender(a)sas.com<mailto:Matthias.Ender(a)sas.com>> wrote:
Dulanjalie,
did you ever get this resolved?
I am running into the same issue, with a blank-slate 212 install:

Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not
running after update (00:12:18)

Error 400007: `api_z1/0' is not running after update

thanks,
Matthias

From: cf-bosh-bounces(a)lists.cloudfoundry.org<mailto:cf-bosh-bounces(a)lists.cloudfoundry.org> [mailto:cf-bosh-bounces(a)lists.cloudfoundry.org<mailto:cf-bosh-bounces(a)lists.cloudfoundry.org>] On Behalf Of Dulanjalie Dhanapala
Sent: Wednesday, June 24, 2015 1:14 PM
To: Discussions about the Cloud Foundry BOSH project.
Subject: Re: [cf-bosh] Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not running after update

Hi all,
I deleted my "cf" deployment
git cloned the latest https://github.com/cloudfoundry/cf-release.git

I am using the latest stemcells 2999

+-----------------------------------------+---------+--------------------+

| Name | Version | CID |

+-----------------------------------------+---------+--------------------+

| bosh-aws-xen-hvm-ubuntu-trusty-go_agent | 2989 | ami-8bc83be0 light |

| bosh-aws-xen-hvm-ubuntu-trusty-go_agent | 2999* | ami-4dcc3526 light |

+-----------------------------------------+---------+--------------------+

(*) Currently in-use

Finally, i am using 212 release

bosh releases

+------+----------+-------------+

| Name | Versions | Commit Hash |

+------+----------+-------------+

| cf | 193 | 54613ebb+ |

| | 212* | ae2ec7a5+ |

+------+----------+-------------+

(*) Currently deployed

(+) Uncommitted changes

Still fails with

Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not running after update (00:12:18)

Error 400007: `api_z1/0' is not running after update



But error logs are different:
http://fpaste.org/236236/65314143/
http://fpaste.org/236241/16579314/

Still "Buildpacks installation failed"

Sincerely,
Dulanjalie

On Wed, Jun 24, 2015 at 7:04 AM, Dulanjalie Dhanapala <dulanjalie(a)gmail.com<mailto:dulanjalie(a)gmail.com>> wrote:
Thanks a lot for the respond.
Pablo, Lev and Nic also gave me some pointers. I am redeploying with cf-212.

thanks all.
Sincerely,
Dulanjalie

On Wed, Jun 24, 2015 at 2:03 AM, Gwenn Etourneau <getourneau(a)pivotal.io<mailto:getourneau(a)pivotal.io>> wrote:
Best way delete deployment and redeploy to start from clean env.



On Wed, Jun 24, 2015 at 5:07 PM, James Bayer <jbayer(a)pivotal.io<mailto:jbayer(a)pivotal.io>> wrote:
this looks like DB migrations may be messed up.

On Mon, Jun 22, 2015 at 10:32 PM, Dulanjalie Dhanapala <dulanjalie(a)gmail.com<mailto:dulanjalie(a)gmail.com>> wrote:

Some more error messages

------------ STARTING cloud_controller_clock_ctl at Tue Jun 23 04:58:50 UTC 2015 --------------

c9/cloud_controller_ng/lib/cloud_controller/background_job_environment.rb:10:in `setup_environment'

lib/tasks/clock.rake:6:in `block (2 levels) in <top (required)>'

PG::UndefinedTable: ERROR: relation "apps" does not exist

LINE 1: SELECT * FROM "apps" LIMIT 1



------------ STARTING cloud_controller_ng_ctl at Tue Jun 23 04:58:52 UTC 2015 --------------

Preparing local package directory

Preparing local resource_pool directory

Preparing local droplet directory

{"timestamp":1435035533.863138,"message":"PG::UndefinedTable: ERROR: relation \"schema_migrations\" does not exist\nLINE 1: SELECT NULL AS \"nil\" FROM \"schema_migrations\" LIMIT 1\n ^: SELECT NULL AS \"nil\" FROM \"schema_migrations\" LIMIT 1","log_level":"error","source":"cc.db.migrations","data":{},"thread_id":69988649218860,"fiber_id":69988678772900,"process_id":2704,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}



=====================cloud_controller_clock_ctl.log===========================

{"timestamp":1435035531.961396,"message":"PG::UndefinedTable: ERROR: relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM \"billing_events\" LIMIT 1\n ^: SELECT * FROM \"billing_events\" LIMIT 1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}

{"timestamp":1435035531.9634626,"message":"PG::UndefinedTable: ERROR: relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM \"billing_events\" LIMIT 1\n ^: SELECT * FROM \"billing_events\" LIMIT 1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}

{"timestamp":1435035531.9643211,"message":"PG::UndefinedTable: ERROR: relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM \"billing_events\" LIMIT 1\n ^: SELECT * FROM \"billing_events\" LIMIT 1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}

On Mon, Jun 22, 2015 at 10:17 PM, Dulanjalie Dhanapala <dulanjalie(a)gmail.com<mailto:dulanjalie(a)gmail.com>> wrote:
I am seeing this error. I am checking other logs
=======================cloud_controller_worker_ctl.err.log==============================

rake aborted!

SignalException: SIGTERM

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient/session.rb:20:in `require'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient/session.rb:20:in `<top (required)>'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient.rb:17:in `require'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient.rb:17:in `<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/app/controllers/runtime/files_controller.rb:1:in `require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/app/controllers/runtime/files_controller.rb:1:in `<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:4:in `require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:4:in `block in <top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:3:in `each'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:3:in `<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller.rb:46:in `require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller.rb:46:in `<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/Rakefile:10:in `require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/Rakefile:10:in `<top (required)>'

(See full trace by running task with --trace)

On Mon, Jun 22, 2015 at 9:39 PM, Gwenn Etourneau <getourneau(a)pivotal.io<mailto:getourneau(a)pivotal.io>> wrote:
Don t think is an error, please check all files.

On Tue, Jun 23, 2015 at 1:15 PM, Dulanjalie Dhanapala <dulanjalie(a)gmail.com<mailto:dulanjalie(a)gmail.com>> wrote:

i am hitting an error when i execute bosh deploy in minimal-aws.yml based cf deployment.


Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not running after update (00:12:11)

Error 400007: `api_z1/0' is not running after update

Any suggestions to resolve this? I tried in #bosh freenode but no one responded.

I see this error related to api_z1 in the ./cloud_controller_worker_ctl.log

./cloud_controller_worker_ctl.log:{"timestamp":1434996874.6546366,"message":"(0.003843s) UPDATE \"delayed_jobs\" SET \"guid\" = 'a3af8feb-30b7-4b4d-91fd-112045501a0d', \"created_at\" = '2015-06-22 18:14:32.651086+0000', \"updated_at\" = CURRENT_TIMESTAMP, \"priority\" = 0, \"attempts\" = 0, \"handler\" = '--- !ruby/struct:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler: !ruby/struct:VCAP::CloudController::Jobs::RequestJob\n job: !ruby/struct:VCAP::CloudController::Jobs::TimeoutJob\n job: !ruby/struct:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n name: java_buildpack\n file: \"/var/vcap/packages/buildpack_java/java-buildpack-v2.5.zip\"\n opts: {}\n config: \n request_id: \n', \"last_error\" = NULL, \"run_at\" = '2015-06-22 18:14:32.660520+0000', \"locked_at\" = '2015-06-22 18:14:34.649651+0000', \"failed_at\" = NULL, \"locked_by\" = 'cc_api_worker.api_z1.0.2', \"queue\" = 'cc-api_z1-0', \"cf_api_error\" = NULL WHERE (\"id\" = 1)","log_level":"debug2","source":"cc.background","data":{},"thread_id":69964119192340,"fiber_id":69964148702360,"process_id":2727,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}

I searched online but it seems like there are some others hitting the same issue. I could not find a proper solution.



Sincerely,

Dulanjalie

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org<mailto:cf-bosh(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh


_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org<mailto:cf-bosh(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh



--
Dulanjalie



--
Dulanjalie

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org<mailto:cf-bosh(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh



--
Thank you,

James Bayer

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org<mailto:cf-bosh(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh


_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org<mailto:cf-bosh(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh



--
Dulanjalie



--
Dulanjalie

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org<mailto:cf-bosh(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh



--
Mit freundlichen Grüßen

Johannes Hiemer

couple more:
/var/vcap/sys/log/cloud_controller_worker_ctl.err.log

[2015-07-24 12:22:57+0000] ------------ STARTING cloud_controller_worker_ctl at Fri Jul 24 12:22:57 UTC 2015 --------------
[2015-07-24 12:22:59+0000] ------------ STARTING cloud_controller_worker_ctl at Fri Jul 24 12:22:58 UTC 2015 --------------
[2015-07-24 12:23:00+0000] rake aborted!
[2015-07-24 12:23:00+0000] Sequel::Migrator::NotCurrentError: migrator is not current
[2015-07-24 12:23:00+0000] Tasks: TOP => buildpacks:install
[2015-07-24 12:23:00+0000] (See full trace by running task with --trace)
[2015-07-24 12:23:02+0000] rake aborted!
[2015-07-24 12:23:02+0000] Sequel::Migrator::NotCurrentError: migrator is not current
[2015-07-24 12:23:02+0000] Tasks: TOP => jobs:local
[2015-07-24 12:23:02+0000] (See full trace by running task with --trace)
[2015-07-24 12:24:08+0000] ------------ STARTING cloud_controller_worker_ctl at Fri Jul 24 12:24:08 UTC 2015 --------------
[2015-07-24 12:24:09+0000] ------------ STARTING cloud_controller_worker_ctl at Fri Jul 24 12:24:09 UTC 2015 --------------
[2015-07-24 17:06:46+0000] ------------ STARTING cloud_controller_worker_ctl at Fri Jul 24 17:06:46 UTC 2015 --------------
[2015-07-24 17:06:47+0000] ------------ STARTING cloud_controller_worker_ctl at Fri Jul 24 17:06:47 UTC 2015 --------------

/var/vcap/sys/log/cloud_controller_ng_ctl.err.log:

[2015-07-24 12:24:05+0000] chown: changing ownership of â/var/vcap/nfs/sharedâ: Operation not permitted
[2015-07-24 12:24:05+0000] chown: changing ownership of â/var/vcap/nfs/sharedâ: Operation not permitted
[2015-07-24 12:24:05+0000] chown: changing ownership of â/var/vcap/nfs/sharedâ: Operation not permitted

From: Matthias Ender
Sent: Friday, July 24, 2015 1:16 PM
To: Discussions about the Cloud Foundry BOSH project.
Subject: RE: [cf-bosh] Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not running after update

I noticed this in /var/vcap/sys/log/routing-api/routing-api.log

{"timestamp":"1437740486.292583704","source":"routing-api","message":"routing-api.database","log_level":1,"data":{"etcd-addresses":["http://10.10.3.8:4001"]}}
{"timestamp":"1437740486.293358803","source":"routing-api","message":"routing-api.failed to connect to etcd","log_level":2,"data":{"error":"sync cluster failed"}}

From: cf-bosh-bounces(a)lists.cloudfoundry.org<mailto:cf-bosh-bounces(a)lists.cloudfoundry.org> [mailto:cf-bosh-bounces(a)lists.cloudfoundry.org] On Behalf Of Johannes Hiemer
Sent: Friday, July 24, 2015 1:12 PM
To: Discussions about the Cloud Foundry BOSH project.
Subject: Re: [cf-bosh] Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not running after update

Hi Matthias,
did you get into api_z1 0 via ssh and look at the specific error? The problem here is, that the failure might arise from different sources.

On Fri, Jul 24, 2015 at 6:57 PM, Matthias Ender <Matthias.Ender(a)sas.com<mailto:Matthias.Ender(a)sas.com>> wrote:
Dulanjalie,
did you ever get this resolved?
I am running into the same issue, with a blank-slate 212 install:

Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not
running after update (00:12:18)

Error 400007: `api_z1/0' is not running after update

thanks,
Matthias

From: cf-bosh-bounces(a)lists.cloudfoundry.org<mailto:cf-bosh-bounces(a)lists.cloudfoundry.org> [mailto:cf-bosh-bounces(a)lists.cloudfoundry.org<mailto:cf-bosh-bounces(a)lists.cloudfoundry.org>] On Behalf Of Dulanjalie Dhanapala
Sent: Wednesday, June 24, 2015 1:14 PM
To: Discussions about the Cloud Foundry BOSH project.
Subject: Re: [cf-bosh] Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not running after update

Hi all,
I deleted my "cf" deployment
git cloned the latest https://github.com/cloudfoundry/cf-release.git

I am using the latest stemcells 2999

+-----------------------------------------+---------+--------------------+

| Name | Version | CID |

+-----------------------------------------+---------+--------------------+

| bosh-aws-xen-hvm-ubuntu-trusty-go_agent | 2989 | ami-8bc83be0 light |

| bosh-aws-xen-hvm-ubuntu-trusty-go_agent | 2999* | ami-4dcc3526 light |

+-----------------------------------------+---------+--------------------+

(*) Currently in-use

Finally, i am using 212 release

bosh releases

+------+----------+-------------+

| Name | Versions | Commit Hash |

+------+----------+-------------+

| cf | 193 | 54613ebb+ |

| | 212* | ae2ec7a5+ |

+------+----------+-------------+

(*) Currently deployed

(+) Uncommitted changes

Still fails with

Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not running after update (00:12:18)

Error 400007: `api_z1/0' is not running after update



But error logs are different:
http://fpaste.org/236236/65314143/
http://fpaste.org/236241/16579314/

Still "Buildpacks installation failed"

Sincerely,
Dulanjalie

On Wed, Jun 24, 2015 at 7:04 AM, Dulanjalie Dhanapala <dulanjalie(a)gmail.com<mailto:dulanjalie(a)gmail.com>> wrote:
Thanks a lot for the respond.
Pablo, Lev and Nic also gave me some pointers. I am redeploying with cf-212.

thanks all.
Sincerely,
Dulanjalie

On Wed, Jun 24, 2015 at 2:03 AM, Gwenn Etourneau <getourneau(a)pivotal.io<mailto:getourneau(a)pivotal.io>> wrote:
Best way delete deployment and redeploy to start from clean env.



On Wed, Jun 24, 2015 at 5:07 PM, James Bayer <jbayer(a)pivotal.io<mailto:jbayer(a)pivotal.io>> wrote:
this looks like DB migrations may be messed up.

On Mon, Jun 22, 2015 at 10:32 PM, Dulanjalie Dhanapala <dulanjalie(a)gmail.com<mailto:dulanjalie(a)gmail.com>> wrote:

Some more error messages

------------ STARTING cloud_controller_clock_ctl at Tue Jun 23 04:58:50 UTC 2015 --------------

c9/cloud_controller_ng/lib/cloud_controller/background_job_environment.rb:10:in `setup_environment'

lib/tasks/clock.rake:6:in `block (2 levels) in <top (required)>'

PG::UndefinedTable: ERROR: relation "apps" does not exist

LINE 1: SELECT * FROM "apps" LIMIT 1



------------ STARTING cloud_controller_ng_ctl at Tue Jun 23 04:58:52 UTC 2015 --------------

Preparing local package directory

Preparing local resource_pool directory

Preparing local droplet directory

{"timestamp":1435035533.863138,"message":"PG::UndefinedTable: ERROR: relation \"schema_migrations\" does not exist\nLINE 1: SELECT NULL AS \"nil\" FROM \"schema_migrations\" LIMIT 1\n ^: SELECT NULL AS \"nil\" FROM \"schema_migrations\" LIMIT 1","log_level":"error","source":"cc.db.migrations","data":{},"thread_id":69988649218860,"fiber_id":69988678772900,"process_id":2704,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}



=====================cloud_controller_clock_ctl.log===========================

{"timestamp":1435035531.961396,"message":"PG::UndefinedTable: ERROR: relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM \"billing_events\" LIMIT 1\n ^: SELECT * FROM \"billing_events\" LIMIT 1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}

{"timestamp":1435035531.9634626,"message":"PG::UndefinedTable: ERROR: relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM \"billing_events\" LIMIT 1\n ^: SELECT * FROM \"billing_events\" LIMIT 1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}

{"timestamp":1435035531.9643211,"message":"PG::UndefinedTable: ERROR: relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM \"billing_events\" LIMIT 1\n ^: SELECT * FROM \"billing_events\" LIMIT 1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}

On Mon, Jun 22, 2015 at 10:17 PM, Dulanjalie Dhanapala <dulanjalie(a)gmail.com<mailto:dulanjalie(a)gmail.com>> wrote:
I am seeing this error. I am checking other logs
=======================cloud_controller_worker_ctl.err.log==============================

rake aborted!

SignalException: SIGTERM

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient/session.rb:20:in `require'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient/session.rb:20:in `<top (required)>'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient.rb:17:in `require'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient.rb:17:in `<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/app/controllers/runtime/files_controller.rb:1:in `require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/app/controllers/runtime/files_controller.rb:1:in `<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:4:in `require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:4:in `block in <top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:3:in `each'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:3:in `<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller.rb:46:in `require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller.rb:46:in `<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/Rakefile:10:in `require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/Rakefile:10:in `<top (required)>'

(See full trace by running task with --trace)

On Mon, Jun 22, 2015 at 9:39 PM, Gwenn Etourneau <getourneau(a)pivotal.io<mailto:getourneau(a)pivotal.io>> wrote:
Don t think is an error, please check all files.

On Tue, Jun 23, 2015 at 1:15 PM, Dulanjalie Dhanapala <dulanjalie(a)gmail.com<mailto:dulanjalie(a)gmail.com>> wrote:

i am hitting an error when i execute bosh deploy in minimal-aws.yml based cf deployment.


Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not running after update (00:12:11)

Error 400007: `api_z1/0' is not running after update

Any suggestions to resolve this? I tried in #bosh freenode but no one responded.

I see this error related to api_z1 in the ./cloud_controller_worker_ctl.log

./cloud_controller_worker_ctl.log:{"timestamp":1434996874.6546366,"message":"(0.003843s) UPDATE \"delayed_jobs\" SET \"guid\" = 'a3af8feb-30b7-4b4d-91fd-112045501a0d', \"created_at\" = '2015-06-22 18:14:32.651086+0000', \"updated_at\" = CURRENT_TIMESTAMP, \"priority\" = 0, \"attempts\" = 0, \"handler\" = '--- !ruby/struct:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler: !ruby/struct:VCAP::CloudController::Jobs::RequestJob\n job: !ruby/struct:VCAP::CloudController::Jobs::TimeoutJob\n job: !ruby/struct:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n name: java_buildpack\n file: \"/var/vcap/packages/buildpack_java/java-buildpack-v2.5.zip\"\n opts: {}\n config: \n request_id: \n', \"last_error\" = NULL, \"run_at\" = '2015-06-22 18:14:32.660520+0000', \"locked_at\" = '2015-06-22 18:14:34.649651+0000', \"failed_at\" = NULL, \"locked_by\" = 'cc_api_worker.api_z1.0.2', \"queue\" = 'cc-api_z1-0', \"cf_api_error\" = NULL WHERE (\"id\" = 1)","log_level":"debug2","source":"cc.background","data":{},"thread_id":69964119192340,"fiber_id":69964148702360,"process_id":2727,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}

I searched online but it seems like there are some others hitting the same issue. I could not find a proper solution.



Sincerely,

Dulanjalie

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org<mailto:cf-bosh(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh


_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org<mailto:cf-bosh(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh



--
Dulanjalie



--
Dulanjalie

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org<mailto:cf-bosh(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh



--
Thank you,

James Bayer

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org<mailto:cf-bosh(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh


_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org<mailto:cf-bosh(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh



--
Dulanjalie



--
Dulanjalie

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org<mailto:cf-bosh(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh



--
Mit freundlichen Grüßen

Johannes Hiemer

Hello Matthias,
The issue i hit was a bug. It cause cos i was using NFS. I removed NFS
server and used S3 bucket (Credit goes to Pablo Alonso Rodriguez). then i
was able to deploy successfully. See the below link. I have added the work
around that Pablo gave me. Or use the build that Pivotal has suggested to
avoid the issue.

https://github.com/cloudfoundry/cf-release/issues/712
https://www.pivotaltracker.com/n/projects/966314/stories/98158086

Sincerely,
DUlanjalie

On Fri, Jul 24, 2015 at 10:11 AM, Johannes Hiemer <jvhiemer(a)gmail.com>
wrote:

Hi Matthias,
did you get into api_z1 0 via ssh and look at the specific error? The
problem here is, that the failure might arise from different sources.

On Fri, Jul 24, 2015 at 6:57 PM, Matthias Ender <Matthias.Ender(a)sas.com>
wrote:

Dulanjalie,

did you ever get this resolved?

I am running into the same issue, with a blank-slate 212 install:



Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not

running after update (00:12:18)



Error 400007: `api_z1/0' is not running after update



thanks,

Matthias



*From:* cf-bosh-bounces(a)lists.cloudfoundry.org [mailto:
cf-bosh-bounces(a)lists.cloudfoundry.org] *On Behalf Of *Dulanjalie
Dhanapala
*Sent:* Wednesday, June 24, 2015 1:14 PM
*To:* Discussions about the Cloud Foundry BOSH project.
*Subject:* Re: [cf-bosh] Failed updating job api_z1 > api_z1/0 (canary):
`api_z1/0' is not running after update



Hi all,

I deleted my "cf" deployment

git cloned the latest https://github.com/cloudfoundry/cf-release.git

I am using the latest stemcells 2999

+-----------------------------------------+---------+--------------------+

| Name | Version | CID |

+-----------------------------------------+---------+--------------------+

| bosh-aws-xen-hvm-ubuntu-trusty-go_agent | 2989 | ami-8bc83be0 light |

| bosh-aws-xen-hvm-ubuntu-trusty-go_agent | 2999* | ami-4dcc3526 light |

+-----------------------------------------+---------+--------------------+

(*) Currently in-use

Finally, i am using 212 release

bosh releases

+------+----------+-------------+

| Name | Versions | Commit Hash |

+------+----------+-------------+

| cf | 193 | 54613ebb+ |

| | 212* | ae2ec7a5+ |

+------+----------+-------------+

(*) Currently deployed

(+) Uncommitted changes

Still fails with

Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not
running after update (00:12:18)

Error 400007: `api_z1/0' is not running after update



But error logs are different:

http://fpaste.org/236236/65314143/

http://fpaste.org/236241/16579314/



Still "Buildpacks installation failed"



Sincerely,

Dulanjalie



On Wed, Jun 24, 2015 at 7:04 AM, Dulanjalie Dhanapala <
dulanjalie(a)gmail.com> wrote:

Thanks a lot for the respond.

Pablo, Lev and Nic also gave me some pointers. I am redeploying with
cf-212.



thanks all.

Sincerely,

Dulanjalie



On Wed, Jun 24, 2015 at 2:03 AM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

Best way delete deployment and redeploy to start from clean env.







On Wed, Jun 24, 2015 at 5:07 PM, James Bayer <jbayer(a)pivotal.io> wrote:

this looks like DB migrations may be messed up.



On Mon, Jun 22, 2015 at 10:32 PM, Dulanjalie Dhanapala <
dulanjalie(a)gmail.com> wrote:

Some more error messages

------------ STARTING cloud_controller_clock_ctl at Tue Jun 23 04:58:50
UTC 2015 --------------

c9/cloud_controller_ng/lib/cloud_controller/background_job_environment.rb:10:in
`setup_environment'

lib/tasks/clock.rake:6:in `block (2 levels) in <top (required)>'

PG::UndefinedTable: ERROR: relation "apps" does not exist

LINE 1: SELECT * FROM "apps" LIMIT 1



------------ STARTING cloud_controller_ng_ctl at Tue Jun 23 04:58:52 UTC
2015 --------------

Preparing local package directory

Preparing local resource_pool directory

Preparing local droplet directory

{"timestamp":1435035533.863138,"message":"PG::UndefinedTable: ERROR:
relation \"schema_migrations\" does not exist\nLINE 1: SELECT NULL AS
\"nil\" FROM \"schema_migrations\" LIMIT 1\n
^: SELECT NULL AS \"nil\" FROM \"schema_migrations\" LIMIT
1","log_level":"error","source":"cc.db.migrations","data":{},"thread_id":69988649218860,"fiber_id":69988678772900,"process_id":2704,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}




=====================cloud_controller_clock_ctl.log===========================

{"timestamp":1435035531.961396,"message":"PG::UndefinedTable: ERROR:
relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM
\"billing_events\" LIMIT 1\n ^: SELECT * FROM
\"billing_events\" LIMIT
1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

{"timestamp":1435035531.9634626,"message":"PG::UndefinedTable: ERROR:
relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM
\"billing_events\" LIMIT 1\n ^: SELECT * FROM
\"billing_events\" LIMIT
1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

{"timestamp":1435035531.9643211,"message":"PG::UndefinedTable: ERROR:
relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM
\"billing_events\" LIMIT 1\n ^: SELECT * FROM
\"billing_events\" LIMIT
1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}



On Mon, Jun 22, 2015 at 10:17 PM, Dulanjalie Dhanapala <
dulanjalie(a)gmail.com> wrote:

I am seeing this error. I am checking other logs


=======================cloud_controller_worker_ctl.err.log==============================

rake aborted!

SignalException: SIGTERM

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient/session.rb:20:in
`require'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient/session.rb:20:in
`<top (required)>'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient.rb:17:in
`require'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient.rb:17:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/app/controllers/runtime/files_controller.rb:1:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/app/controllers/runtime/files_controller.rb:1:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:4:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:4:in
`block in <top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:3:in
`each'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:3:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller.rb:46:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller.rb:46:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/Rakefile:10:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/Rakefile:10:in
`<top (required)>'

(See full trace by running task with --trace)



On Mon, Jun 22, 2015 at 9:39 PM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

Don t think is an error, please check all files.



On Tue, Jun 23, 2015 at 1:15 PM, Dulanjalie Dhanapala <
dulanjalie(a)gmail.com> wrote:


i am hitting an error when i execute bosh deploy in minimal-aws.yml
based cf deployment.



Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not
running after update (00:12:11)

Error 400007: `api_z1/0' is not running after update

Any suggestions to resolve this? I tried in #bosh freenode but no one
responded.



I see this error related to api_z1 in
the ./cloud_controller_worker_ctl.log

./cloud_controller_worker_ctl.log:{"timestamp":1434996874.6546366,"message":"(0.003843s)
UPDATE \"delayed_jobs\" SET \"guid\" =
'a3af8feb-30b7-4b4d-91fd-112045501a0d', \"created_at\" = '2015-06-22
18:14:32.651086+0000', \"updated_at\" = CURRENT_TIMESTAMP, \"priority\" =
0, \"attempts\" = 0, \"handler\" = '---
!ruby/struct:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler:
!ruby/struct:VCAP::CloudController::Jobs::RequestJob\n job:
!ruby/struct:VCAP::CloudController::Jobs::TimeoutJob\n job:
!ruby/struct:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n
name: java_buildpack\n file:
\"/var/vcap/packages/buildpack_java/java-buildpack-v2.5.zip\"\n opts:
{}\n config: \n request_id: \n', \"last_error\" = NULL, \"run_at\" =
'2015-06-22 18:14:32.660520+0000', \"locked_at\" = '2015-06-22
18:14:34.649651+0000', \"failed_at\" = NULL, \"locked_by\" = 'cc_api_worker.
*api_z1*.0.2', \"queue\" = 'cc-*api_z1*-0', \"cf_api_error\" = NULL
WHERE (\"id\" =
1)","log_level":"debug2","source":"cc.background","data":{},"thread_id":69964119192340,"fiber_id":69964148702360,"process_id":2727,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

I searched online but it seems like there are some others hitting the
same issue. I could not find a proper solution.



Sincerely,

Dulanjalie



_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh




_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh





--

Dulanjalie





--

Dulanjalie


_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh





--

Thank you,



James Bayer


_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh




_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh





--

Dulanjalie





--

Dulanjalie

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

--
Mit freundlichen Grüßen

Johannes Hiemer

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

--
Dulanjalie

Hi Mathias,
I faced the same issue on vSphere with NFS. At the following to the
property section of your properties in cf-stub.yml:

debian_nfs_server:
no_root_squash: true

And run bosh deploy again.

On Fri, Jul 24, 2015 at 7:15 PM, Matthias Ender <Matthias.Ender(a)sas.com>
wrote:

I noticed this in /var/vcap/sys/log/routing-api/routing-api.log




{"timestamp":"1437740486.292583704","source":"routing-api","message":"routing-api.database","log_level":1,"data":{"etcd-addresses":["
http://10.10.3.8:4001"]}}

{"timestamp":"1437740486.293358803","source":"routing-api","message":"routing-api.failed
to connect to etcd","log_level":2,"data":{"error":"sync cluster failed"}}



*From:* cf-bosh-bounces(a)lists.cloudfoundry.org [mailto:
cf-bosh-bounces(a)lists.cloudfoundry.org] *On Behalf Of *Johannes Hiemer
*Sent:* Friday, July 24, 2015 1:12 PM

*To:* Discussions about the Cloud Foundry BOSH project.
*Subject:* Re: [cf-bosh] Failed updating job api_z1 > api_z1/0 (canary):
`api_z1/0' is not running after update



Hi Matthias,

did you get into api_z1 0 via ssh and look at the specific error? The
problem here is, that the failure might arise from different sources.



On Fri, Jul 24, 2015 at 6:57 PM, Matthias Ender <Matthias.Ender(a)sas.com>
wrote:

Dulanjalie,

did you ever get this resolved?

I am running into the same issue, with a blank-slate 212 install:



Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not

running after update (00:12:18)



Error 400007: `api_z1/0' is not running after update



thanks,

Matthias



*From:* cf-bosh-bounces(a)lists.cloudfoundry.org [mailto:
cf-bosh-bounces(a)lists.cloudfoundry.org] *On Behalf Of *Dulanjalie
Dhanapala
*Sent:* Wednesday, June 24, 2015 1:14 PM
*To:* Discussions about the Cloud Foundry BOSH project.
*Subject:* Re: [cf-bosh] Failed updating job api_z1 > api_z1/0 (canary):
`api_z1/0' is not running after update



Hi all,

I deleted my "cf" deployment

git cloned the latest https://github.com/cloudfoundry/cf-release.git

I am using the latest stemcells 2999

+-----------------------------------------+---------+--------------------+

| Name | Version | CID |

+-----------------------------------------+---------+--------------------+

| bosh-aws-xen-hvm-ubuntu-trusty-go_agent | 2989 | ami-8bc83be0 light |

| bosh-aws-xen-hvm-ubuntu-trusty-go_agent | 2999* | ami-4dcc3526 light |

+-----------------------------------------+---------+--------------------+

(*) Currently in-use

Finally, i am using 212 release

bosh releases

+------+----------+-------------+

| Name | Versions | Commit Hash |

+------+----------+-------------+

| cf | 193 | 54613ebb+ |

| | 212* | ae2ec7a5+ |

+------+----------+-------------+

(*) Currently deployed

(+) Uncommitted changes

Still fails with

Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not
running after update (00:12:18)

Error 400007: `api_z1/0' is not running after update



But error logs are different:

http://fpaste.org/236236/65314143/

http://fpaste.org/236241/16579314/



Still "Buildpacks installation failed"



Sincerely,

Dulanjalie



On Wed, Jun 24, 2015 at 7:04 AM, Dulanjalie Dhanapala <
dulanjalie(a)gmail.com> wrote:

Thanks a lot for the respond.

Pablo, Lev and Nic also gave me some pointers. I am redeploying with
cf-212.



thanks all.

Sincerely,

Dulanjalie



On Wed, Jun 24, 2015 at 2:03 AM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

Best way delete deployment and redeploy to start from clean env.







On Wed, Jun 24, 2015 at 5:07 PM, James Bayer <jbayer(a)pivotal.io> wrote:

this looks like DB migrations may be messed up.



On Mon, Jun 22, 2015 at 10:32 PM, Dulanjalie Dhanapala <
dulanjalie(a)gmail.com> wrote:

Some more error messages

------------ STARTING cloud_controller_clock_ctl at Tue Jun 23 04:58:50
UTC 2015 --------------

c9/cloud_controller_ng/lib/cloud_controller/background_job_environment.rb:10:in
`setup_environment'

lib/tasks/clock.rake:6:in `block (2 levels) in <top (required)>'

PG::UndefinedTable: ERROR: relation "apps" does not exist

LINE 1: SELECT * FROM "apps" LIMIT 1



------------ STARTING cloud_controller_ng_ctl at Tue Jun 23 04:58:52 UTC
2015 --------------

Preparing local package directory

Preparing local resource_pool directory

Preparing local droplet directory

{"timestamp":1435035533.863138,"message":"PG::UndefinedTable: ERROR:
relation \"schema_migrations\" does not exist\nLINE 1: SELECT NULL AS
\"nil\" FROM \"schema_migrations\" LIMIT 1\n
^: SELECT NULL AS \"nil\" FROM \"schema_migrations\" LIMIT
1","log_level":"error","source":"cc.db.migrations","data":{},"thread_id":69988649218860,"fiber_id":69988678772900,"process_id":2704,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}




=====================cloud_controller_clock_ctl.log===========================

{"timestamp":1435035531.961396,"message":"PG::UndefinedTable: ERROR:
relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM
\"billing_events\" LIMIT 1\n ^: SELECT * FROM
\"billing_events\" LIMIT
1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

{"timestamp":1435035531.9634626,"message":"PG::UndefinedTable: ERROR:
relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM
\"billing_events\" LIMIT 1\n ^: SELECT * FROM
\"billing_events\" LIMIT
1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

{"timestamp":1435035531.9643211,"message":"PG::UndefinedTable: ERROR:
relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM
\"billing_events\" LIMIT 1\n ^: SELECT * FROM
\"billing_events\" LIMIT
1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}



On Mon, Jun 22, 2015 at 10:17 PM, Dulanjalie Dhanapala <
dulanjalie(a)gmail.com> wrote:

I am seeing this error. I am checking other logs


=======================cloud_controller_worker_ctl.err.log==============================

rake aborted!

SignalException: SIGTERM

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient/session.rb:20:in
`require'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient/session.rb:20:in
`<top (required)>'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient.rb:17:in
`require'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient.rb:17:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/app/controllers/runtime/files_controller.rb:1:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/app/controllers/runtime/files_controller.rb:1:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:4:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:4:in
`block in <top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:3:in
`each'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:3:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller.rb:46:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller.rb:46:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/Rakefile:10:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/Rakefile:10:in
`<top (required)>'

(See full trace by running task with --trace)



On Mon, Jun 22, 2015 at 9:39 PM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

Don t think is an error, please check all files.



On Tue, Jun 23, 2015 at 1:15 PM, Dulanjalie Dhanapala <
dulanjalie(a)gmail.com> wrote:


i am hitting an error when i execute bosh deploy in minimal-aws.yml
based cf deployment.



Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not
running after update (00:12:11)

Error 400007: `api_z1/0' is not running after update

Any suggestions to resolve this? I tried in #bosh freenode but no one
responded.



I see this error related to api_z1 in the ./cloud_controller_worker_ctl.log

./cloud_controller_worker_ctl.log:{"timestamp":1434996874.6546366,"message":"(0.003843s)
UPDATE \"delayed_jobs\" SET \"guid\" =
'a3af8feb-30b7-4b4d-91fd-112045501a0d', \"created_at\" = '2015-06-22
18:14:32.651086+0000', \"updated_at\" = CURRENT_TIMESTAMP, \"priority\" =
0, \"attempts\" = 0, \"handler\" = '---
!ruby/struct:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler:
!ruby/struct:VCAP::CloudController::Jobs::RequestJob\n job:
!ruby/struct:VCAP::CloudController::Jobs::TimeoutJob\n job:
!ruby/struct:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n
name: java_buildpack\n file:
\"/var/vcap/packages/buildpack_java/java-buildpack-v2.5.zip\"\n opts:
{}\n config: \n request_id: \n', \"last_error\" = NULL, \"run_at\" =
'2015-06-22 18:14:32.660520+0000', \"locked_at\" = '2015-06-22
18:14:34.649651+0000', \"failed_at\" = NULL, \"locked_by\" = 'cc_api_worker.
*api_z1*.0.2', \"queue\" = 'cc-*api_z1*-0', \"cf_api_error\" = NULL WHERE
(\"id\" =
1)","log_level":"debug2","source":"cc.background","data":{},"thread_id":69964119192340,"fiber_id":69964148702360,"process_id":2727,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

I searched online but it seems like there are some others hitting the same
issue. I could not find a proper solution.



Sincerely,

Dulanjalie



_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh




_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh





--

Dulanjalie





--

Dulanjalie


_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh





--

Thank you,



James Bayer


_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh




_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh





--

Dulanjalie





--

Dulanjalie


_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh





--

Mit freundlichen Grüßen

Johannes Hiemer

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

--
Mit freundlichen Grüßen

Johannes Hiemer

I am attempting the v213 install on Openstack. Initially with
cf-boshworkspace, now as a manual install, api_z1/0 is not running after
update.
-- I did include
debian_nfs_server:
no_root_squash: true

SSH to the box and the only error out of all *.err.* logs:

[2015-07-29 12:19:38+0000] ------------ STARTING cloud_controller_ng_ctl at
Wed

Jul 29 12:19:38 UTC 2015 --------------

[2015-07-29 12:20:18+0000] ------------ STARTING cloud_controller_ng_ctl at
Wed

Jul 29 12:20:18 UTC 2015 --------------

[2015-07-29 12:20:18+0000] chown: changing ownership of
‘/var/vcap/nfs/shared’:

Operation not permitted

[2015-07-29 12:20:18+0000] chown: changing ownership of
‘/var/vcap/nfs/shared’:

Operation not permitted

[2015-07-29 12:20:18+0000] chown: changing ownership of
‘/var/vcap/nfs/shared’:

Operation not permitted

From another thread, nfs_mounter is the last in the template lists also.

Pointers??

213 have been deprecated and should not be use .


https://github.com/cloudfoundry/cf-release/releases

"*IMPORTANT*: This release has a backwards incompatible Cloud Controller
database migration that causes downtime for Cloud Controller during the
deploy. We are working on getting this sorted out. We don't recommend
upgrading to this release."

To clarify, 213 is fine to use as a fresh deploy.
It's not recommended to use it if you desire not to have down time of cloud
controller during the update of the cloud controller job due to a backwards
incompatible database migration. Once all the cc vms have been updated, cc
should behave normally.

Tom, the issue you are seeing looks different from the race condition where
nfs may not be mounted in time (addressed in cf-release 213 [1].

[1] https://www.pivotaltracker.com/story/show/98158086

Could you open a new thread or github issue for the issue you are seeing
Tom?

-Dieu
CF CAPI PM


On Wed, Jul 29, 2015 at 6:34 PM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

213 have been deprecated and should not be use .


https://github.com/cloudfoundry/cf-release/releases

"*IMPORTANT*: This release has a backwards incompatible Cloud Controller
database migration that causes downtime for Cloud Controller during the
deploy. We are working on getting this sorted out. We don't recommend
upgrading to this release."

On Wed, Jul 29, 2015 at 9:46 PM, Tom Sherrod <tom.sherrod(a)gmail.com>
wrote:

I am attempting the v213 install on Openstack. Initially with
cf-boshworkspace, now as a manual install, api_z1/0 is not running after
update.
-- I did include
debian_nfs_server:
no_root_squash: true

SSH to the box and the only error out of all *.err.* logs:

[2015-07-29 12:19:38+0000] ------------ STARTING cloud_controller_ng_ctl
at Wed

Jul 29 12:19:38 UTC 2015 --------------

[2015-07-29 12:20:18+0000] ------------ STARTING cloud_controller_ng_ctl
at Wed

Jul 29 12:20:18 UTC 2015 --------------

[2015-07-29 12:20:18+0000] chown: changing ownership of
‘/var/vcap/nfs/shared’:

Operation not permitted

[2015-07-29 12:20:18+0000] chown: changing ownership of
‘/var/vcap/nfs/shared’:

Operation not permitted

[2015-07-29 12:20:18+0000] chown: changing ownership of
‘/var/vcap/nfs/shared’:

Operation not permitted

From another thread, nfs_mounter is the last in the template lists also.

Pointers??

On Fri, Jul 24, 2015 at 1:45 PM, Johannes Hiemer <jvhiemer(a)gmail.com>
wrote:

Hi Mathias,
I faced the same issue on vSphere with NFS. At the following to the
property section of your properties in cf-stub.yml:

debian_nfs_server:
no_root_squash: true

And run bosh deploy again.

On Fri, Jul 24, 2015 at 7:15 PM, Matthias Ender <Matthias.Ender(a)sas.com>
wrote:

I noticed this in /var/vcap/sys/log/routing-api/routing-api.log




{"timestamp":"1437740486.292583704","source":"routing-api","message":"routing-api.database","log_level":1,"data":{"etcd-addresses":["
http://10.10.3.8:4001"]}}

{"timestamp":"1437740486.293358803","source":"routing-api","message":"routing-api.failed
to connect to etcd","log_level":2,"data":{"error":"sync cluster failed"}}



*From:* cf-bosh-bounces(a)lists.cloudfoundry.org [mailto:
cf-bosh-bounces(a)lists.cloudfoundry.org] *On Behalf Of *Johannes Hiemer
*Sent:* Friday, July 24, 2015 1:12 PM

*To:* Discussions about the Cloud Foundry BOSH project.
*Subject:* Re: [cf-bosh] Failed updating job api_z1 > api_z1/0
(canary): `api_z1/0' is not running after update



Hi Matthias,

did you get into api_z1 0 via ssh and look at the specific error? The
problem here is, that the failure might arise from different sources.



On Fri, Jul 24, 2015 at 6:57 PM, Matthias Ender <Matthias.Ender(a)sas.com>
wrote:

Dulanjalie,

did you ever get this resolved?

I am running into the same issue, with a blank-slate 212 install:



Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not

running after update (00:12:18)



Error 400007: `api_z1/0' is not running after update



thanks,

Matthias



*From:* cf-bosh-bounces(a)lists.cloudfoundry.org [mailto:
cf-bosh-bounces(a)lists.cloudfoundry.org] *On Behalf Of *Dulanjalie
Dhanapala
*Sent:* Wednesday, June 24, 2015 1:14 PM
*To:* Discussions about the Cloud Foundry BOSH project.
*Subject:* Re: [cf-bosh] Failed updating job api_z1 > api_z1/0
(canary): `api_z1/0' is not running after update



Hi all,

I deleted my "cf" deployment

git cloned the latest https://github.com/cloudfoundry/cf-release.git

I am using the latest stemcells 2999


+-----------------------------------------+---------+--------------------+

| Name | Version | CID
|


+-----------------------------------------+---------+--------------------+

| bosh-aws-xen-hvm-ubuntu-trusty-go_agent | 2989 | ami-8bc83be0
light |

| bosh-aws-xen-hvm-ubuntu-trusty-go_agent | 2999* | ami-4dcc3526
light |


+-----------------------------------------+---------+--------------------+

(*) Currently in-use

Finally, i am using 212 release

bosh releases

+------+----------+-------------+

| Name | Versions | Commit Hash |

+------+----------+-------------+

| cf | 193 | 54613ebb+ |

| | 212* | ae2ec7a5+ |

+------+----------+-------------+

(*) Currently deployed

(+) Uncommitted changes

Still fails with

Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not
running after update (00:12:18)

Error 400007: `api_z1/0' is not running after update



But error logs are different:

http://fpaste.org/236236/65314143/

http://fpaste.org/236241/16579314/



Still "Buildpacks installation failed"



Sincerely,

Dulanjalie



On Wed, Jun 24, 2015 at 7:04 AM, Dulanjalie Dhanapala <
dulanjalie(a)gmail.com> wrote:

Thanks a lot for the respond.

Pablo, Lev and Nic also gave me some pointers. I am redeploying with
cf-212.



thanks all.

Sincerely,

Dulanjalie



On Wed, Jun 24, 2015 at 2:03 AM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

Best way delete deployment and redeploy to start from clean env.







On Wed, Jun 24, 2015 at 5:07 PM, James Bayer <jbayer(a)pivotal.io> wrote:

this looks like DB migrations may be messed up.



On Mon, Jun 22, 2015 at 10:32 PM, Dulanjalie Dhanapala <
dulanjalie(a)gmail.com> wrote:

Some more error messages

------------ STARTING cloud_controller_clock_ctl at Tue Jun 23 04:58:50
UTC 2015 --------------

c9/cloud_controller_ng/lib/cloud_controller/background_job_environment.rb:10:in
`setup_environment'

lib/tasks/clock.rake:6:in `block (2 levels) in <top (required)>'

PG::UndefinedTable: ERROR: relation "apps" does not exist

LINE 1: SELECT * FROM "apps" LIMIT 1



------------ STARTING cloud_controller_ng_ctl at Tue Jun 23 04:58:52
UTC 2015 --------------

Preparing local package directory

Preparing local resource_pool directory

Preparing local droplet directory

{"timestamp":1435035533.863138,"message":"PG::UndefinedTable: ERROR:
relation \"schema_migrations\" does not exist\nLINE 1: SELECT NULL AS
\"nil\" FROM \"schema_migrations\" LIMIT 1\n
^: SELECT NULL AS \"nil\" FROM \"schema_migrations\" LIMIT
1","log_level":"error","source":"cc.db.migrations","data":{},"thread_id":69988649218860,"fiber_id":69988678772900,"process_id":2704,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}




=====================cloud_controller_clock_ctl.log===========================

{"timestamp":1435035531.961396,"message":"PG::UndefinedTable: ERROR:
relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM
\"billing_events\" LIMIT 1\n ^: SELECT * FROM
\"billing_events\" LIMIT
1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

{"timestamp":1435035531.9634626,"message":"PG::UndefinedTable: ERROR:
relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM
\"billing_events\" LIMIT 1\n ^: SELECT * FROM
\"billing_events\" LIMIT
1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

{"timestamp":1435035531.9643211,"message":"PG::UndefinedTable: ERROR:
relation \"billing_events\" does not exist\nLINE 1: SELECT * FROM
\"billing_events\" LIMIT 1\n ^: SELECT * FROM
\"billing_events\" LIMIT
1","log_level":"error","source":"cc.background","data":{},"thread_id":69935043310380,"fiber_id":69935071682700,"process_id":2579,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}



On Mon, Jun 22, 2015 at 10:17 PM, Dulanjalie Dhanapala <
dulanjalie(a)gmail.com> wrote:

I am seeing this error. I am checking other logs


=======================cloud_controller_worker_ctl.err.log==============================

rake aborted!

SignalException: SIGTERM

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient/session.rb:20:in
`require'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient/session.rb:20:in
`<top (required)>'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient.rb:17:in
`require'

/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/httpclient-2.5.1/lib/httpclient.rb:17:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/app/controllers/runtime/files_controller.rb:1:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/app/controllers/runtime/files_controller.rb:1:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:4:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:4:in
`block in <top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:3:in
`each'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller/controllers.rb:3:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller.rb:46:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/lib/cloud_controller.rb:46:in
`<top (required)>'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/Rakefile:10:in
`require'

/var/vcap/data/packages/cloud_controller_ng/27a4f1b135a8415d623996c6f3a7c668801d1ea3.1-2c0961678c940361c8069ddcac42449d59c222c9/cloud_controller_ng/Rakefile:10:in
`<top (required)>'

(See full trace by running task with --trace)



On Mon, Jun 22, 2015 at 9:39 PM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

Don t think is an error, please check all files.



On Tue, Jun 23, 2015 at 1:15 PM, Dulanjalie Dhanapala <
dulanjalie(a)gmail.com> wrote:


i am hitting an error when i execute bosh deploy in minimal-aws.yml
based cf deployment.



Failed updating job api_z1 > api_z1/0 (canary): `api_z1/0' is not
running after update (00:12:11)

Error 400007: `api_z1/0' is not running after update

Any suggestions to resolve this? I tried in #bosh freenode but no one
responded.



I see this error related to api_z1 in
the ./cloud_controller_worker_ctl.log

./cloud_controller_worker_ctl.log:{"timestamp":1434996874.6546366,"message":"(0.003843s)
UPDATE \"delayed_jobs\" SET \"guid\" =
'a3af8feb-30b7-4b4d-91fd-112045501a0d', \"created_at\" = '2015-06-22
18:14:32.651086+0000', \"updated_at\" = CURRENT_TIMESTAMP, \"priority\" =
0, \"attempts\" = 0, \"handler\" = '---
!ruby/struct:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler:
!ruby/struct:VCAP::CloudController::Jobs::RequestJob\n job:
!ruby/struct:VCAP::CloudController::Jobs::TimeoutJob\n job:
!ruby/struct:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n
name: java_buildpack\n file:
\"/var/vcap/packages/buildpack_java/java-buildpack-v2.5.zip\"\n opts:
{}\n config: \n request_id: \n', \"last_error\" = NULL, \"run_at\" =
'2015-06-22 18:14:32.660520+0000', \"locked_at\" = '2015-06-22
18:14:34.649651+0000', \"failed_at\" = NULL, \"locked_by\" = 'cc_api_worker.
*api_z1*.0.2', \"queue\" = 'cc-*api_z1*-0', \"cf_api_error\" = NULL
WHERE (\"id\" =
1)","log_level":"debug2","source":"cc.background","data":{},"thread_id":69964119192340,"fiber_id":69964148702360,"process_id":2727,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.15.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}

I searched online but it seems like there are some others hitting the
same issue. I could not find a proper solution.



Sincerely,

Dulanjalie



_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh




_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh





--

Dulanjalie





--

Dulanjalie


_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh





--

Thank you,



James Bayer


_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh




_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh





--

Dulanjalie





--

Dulanjalie


_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh





--

Mit freundlichen Grüßen

Johannes Hiemer

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

--
Mit freundlichen Grüßen

Johannes Hiemer

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

Hi all,

This thread has become quite confusing, it looks like 3 different people
have raised different issues here. Is this summary correct:

Dulanjalie: problem resolved
Mathias: different problem, not resolved
Tom: different problem, not resolved

If you are continuing to have issues that have not been resolved in this
thread, please open an issue on GitHub:
https://github.com/cloudfoundry/cf-release/issues, or open a new thread on
the mailing list. Any relevant log details you may have copy-pasted in this
thread would probably be useful to include in any new issues you open.

Thanks,
Amit




--
View this message in context: http://cf-bosh.70367.x6.nabble.com/cf-bosh-Failed-updating-job-api-z1-api-z1-0-canary-api-z1-0-is-not-running-after-update-tp308p555.html
Sent from the CF BOSH mailing list archive at Nabble.com.

Amit,
Can be nice to Open issue on the mailing list to avoiding pollution of
Github issue tracker.