Hi Ranga,

The only reason we were using bosh-lite.com for our app_domains was
because we were testing to reproduce on our local bosh-lite.
Using 'cfapp' I managed to reproduce the issue locally. My guess is that
this is because 'cfapp' is not a valid domain (it doesn't end with a valid
top-level domain), and I'm guessing that fixing that will at least fix part
of the problem if not all of it.

Thanks,

Kara

On Thu, Apr 7, 2016 at 8:27 PM, Ranga Rajagopalan <
ranga.rajagopalan(a)gmail.com> wrote:

Here's my deployment manifest. app_domains is set to cfapp. I can't find
bost-lite anywhere in the file at all.

On Thu, Apr 7, 2016 at 4:27 PM, Peter Goetz <peter.gtz(a)gmail.com> wrote:

Hi Ranga,

Looking at your logs we found an error that could possibly cause this
and it is related to the properties.apps_domain in the deployment manifest.
By setting it to 'b%()osh-lite.com' (using special characters), we
could reproduce the following error which we also found in your log file:

Encountered error: name
format\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/model/base.rb:1543:in
`save'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/app/models/runtime/shared_domain.rb:35:in
`block in
find_or_create'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/transactions.rb:134:in
`_transaction'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/transactions.rb:108:in
`block in
transaction'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/connecting.rb:249:in
`block in
synchronize'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/connection_pool/threaded.rb:103:in
`hold'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/connecting.rb:249:in
`synchronize'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/transactions.rb:97:in
`transaction'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/app/models/runtime/shared_domain.rb:27:in
`find_or_create'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/seeds.rb:57:in
`block in
create_seed_domains'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/seeds.rb:56:in
`each'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/seeds.rb:56:in
`create_seed_domains'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/seeds.rb:9:in
`write_seed_data'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/runner.rb:93:in
`block in
run!'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/eventmachine-1.0.9.1/lib/eventmachine.rb:193:in
`call'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/eventmachine-1.0.9.1/lib/eventmachine.rb:193:in
`run_machine'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/eventmachine-1.0.9.1/lib/eventmachine.rb:193:in
`run'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/runner.rb:87:in
`run!'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/bin/cloud_controller:8:in
`<main>'

Can you check the apps_domain property and see if there is anything
suspicious with it?

Thanks,
Peter & Kara

On Thu, Apr 7, 2016 at 2:32 PM Ranga Rajagopalan <
ranga.rajagopalan(a)gmail.com> wrote:

Hi Peter,

Attaching /var/vcap/sys/log/cloud_controller_worker/cloud_controller_worker.log.gz
and /var/vcap/sys/log/cloud_controller_worker_ctl.log.gz. There isn't a
/var/vcap/sys/log/cloud_controller_worker/ directory on this node.

vcap(a)572afc33-0735-4727-8ff3-9dc6d7fa8af0:~$ ls /var/vcap/sys/log/
agent_ctl.err.log cloud_controller_worker_ctl.log
nginx_cc/
agent_ctl.log consul_agent/
nginx_ctl.err.log
cloud_controller_migration/ metron_agent/
nginx_ctl.log
cloud_controller_migration_ctl.err.log metron_agent_ctl.err.log
route_registrar/
cloud_controller_migration_ctl.log metron_agent_ctl.log
route_registrar_ctl.err.log
cloud_controller_ng/ monit/
route_registrar_ctl.log
cloud_controller_ng_ctl.err.log nfs_mounter/
statsd-injector/
cloud_controller_ng_ctl.log nfs_mounter_ctl.err.log
statsd-injector-ctl.err.log
cloud_controller_worker_ctl.err.log nfs_mounter_ctl.log
statsd-injector-ctl.log



On Thu, Apr 7, 2016 at 12:21 PM, Peter Goetz <peter.gtz(a)gmail.com>
wrote:

Hi Ranga,

To trouble-shoot this issue could you also provide the contents of
/var/vcap/sys/log/cloud_controller_ng/cloud_controller_ng.log
and /var/vcap/sys/log/cloud_controller_worker/cloud_controller_worker.log?
This should give us more details about what's going on. The ctl script logs
do not provide enough details.

Thanks,
Peter

On Wed, Apr 6, 2016 at 6:12 PM Ranga Rajagopalan <
ranga.rajagopalan(a)gmail.com> wrote:

I tried v231. Unfortunately, same issue.

--
Thanks,

Ranga

--
Thanks,

Ranga