Date   

Re: CF Auto-scaling with an external application

Giovanni Napoli
 

@Daniel Mikusa

Thank you for your support. I have few more questions and hope that you can help me.
If i'll use the library you linked, i'd like to have some suggestions about the task i have to solve:

- is there a way to collect in some way the data i need in a struct that i could use for send "the scale command"? I mean, would be great to have a struct in wich i could have fields like "AppName", "CpuUse", "MemoryUse", etc. so i cloud just check the App.CpuUse field, for istance, and send a "cf scale" command to CF to solve resource problems.

- i found this client library for CF, do you think could help my work? https://github.com/cloudfoundry/cf-java-client/ I'm asking you cause i'll prefer to use Java, by the way. Is you know, is there a way to have ".jar" library of this repo so i could use as simple as i can? Also i found this http://www.ibm.com/developerworks/cloud/library/cl-bluemix-cloudappswithjava/index.html and a "cloudfoundry-client-lib.jar" online but i don't kwon if could be good for my problem.

I would really thank you for your support, and i'm sorry if i made some stupid or specifical question but i really need help. The app i have to develop must be very simple because it is just a mimal study case for the PaaS Cloud Foundry so the easier it is the best would be.
Still thank you,
Giovanni.


Re: Unsubscribe

Serge Kornfeld <serge.kornfeld@...>
 

Unsubscribe

On Fri, Apr 8, 2016 at 11:51 AM, A Gonzalez <apolonioguy(a)gmail.com> wrote:


Re: Cloud Controller System Domain vs App Domains

Nicholas Calugar
 

Hi Tom,

App developers can currently push an app using api.domain, but that is the
problem we are trying to solve. As this is currently allowed, the Gorouter
will balance traffic between the Cloud Controller and the user app. This is
not desired, all traffic for api.system_domain needs to go to the Cloud
Controller.

At this point we are going to move forward with Solution 2. We think this
will offer the easiest transition for operators.


Thanks,

Nick

On Wed, Apr 6, 2016 at 5:27 PM Tom Sherrod <tom.sherrod(a)gmail.com> wrote:

Would solution 2 allow/work with a system domain?
Solution #2 means api.domain is now taken so there could be no other app
deployed to api.domain. Application developers may not be happy with the
limitation, requiring another domain. Defeats the purpose of solution #2?

Tom

On Tue, Apr 5, 2016 at 2:15 PM, Nicholas Calugar <ncalugar(a)pivotal.io>
wrote:

Hi John,

The point of seeding is so that there aren't two decisions (code paths)
for requests to create a route. Seeding the database requires no changes to
route creation logic.

Would anyone else like to comment on either of the two proposals before
we make a decision and start implementing a fix?


Thanks,

Nick

On Thu, Mar 31, 2016 at 10:52 AM john mcteague <john.mcteague(a)gmail.com>
wrote:

For option 2 would it not be simpler to have a single property such as
cc.blacklisted_system_domain_routes that contains the desired list and have
the CC deny route requests for those routes?

I don't see what storing them in the DB or creating real routes actually
buys us here. Everything would be available via config.

I'm in favour of some form of option 2 for those of us who have existing
deployments but would rather not make a change in either app or system
domains but are exposed to this issue.

John.
On 31 Mar 2016 5:47 a.m., "Nicholas Calugar" <ncalugar(a)pivotal.io>
wrote:

Hi Cloud Foundry,

We've had a recurring issue brought to our attention regarding a Cloud
Foundry deployment using a system_domain that is in the list of
app_domains. When the system domain is in the list of app domains, a Shared
Domain is created for the system domain. This is problematic because it
allows users to create routes on the system domain, see this [1] recent
issue as an example.

[1] https://github.com/cloudfoundry/cloud_controller_ng/issues/568

I'd like to propose two solutions and get some feedback regarding which
the community would prefer. Please respond with your preferred solution and
a brief reason why.

*Solution 1 - Require a Unique System Domain*

Instead of recommending a unique system domain, we would enforce this
in the Cloud Controller. The proposed change is as follows:

1. REQUIRE system_domain to NOT be in the list of app_domains
2. REQUIRE a system_domain_organization

This will create a Private Domain for the system domain. Failure to
configure correctly would not allow the Cloud Controller to start.

If we decide to implement this, an operator should ensure their
deployment uses a unique system domain and a system_domain_organization and
correct DNS entries before upgrading.

Example for BOSH-lite

- app_domains: [bosh-lite.com]
- system_domain: system.bosh-lite.com
- system_domain_organization: system


- api endpoint: api.system.bosh-lite.com
- sample app endpoint: dora.bosh-lite.com

Example for a PaaS:

- app_domains: [yuge-paas-apps.io]
- system_domain: yuge-pass.com
- system_domain_organization: yuge-system-org


- api endpoint: api.yuge-paas.com
- sample app endpoint: dora.yuge-paas-apps.io

*Pro: *Cloud Controller now enforces what was previously the
recommended configuration for separate system and apps domains.
Con: Second SSL cert for the system domain and possibly a second DNS
record if system domain is not covered by the current wildcard record.

*Solution 2 - Cloud Controller Seeds System Routes*

To prevent a non-system app from requesting a hostname on a shared
system domain, the Cloud Controller will take a list of hostnames and seed
the database with routes. As routes are associated with a space, we will
require a system_organization and system_space. An operator could choose to
omit hostnames as desired.

cc.system_hostnames:
description: List of hostnames for which routes will be created on
the system domain.
default: [api,uaa,login,doppler,loggregator,hm9000]
cc.system_space:
description: Space where system routes will be created.
default: system

*Pro:* Significantly less change for operators running Cloud Foundry
with matching system and apps domains.
*Con: *Cloud Controller has knowledge of unrelated system components
and the list of defaults needs to be maintained as we add and remove
components.


Thanks,

-Nick
--
Nicholas Calugar
CAPI Product Manager
Pivotal Software, Inc.
--
Nicholas Calugar
CAPI Product Manager
Pivotal Software, Inc.
--
Nicholas Calugar
CAPI Product Manager
Pivotal Software, Inc.


Re: App running even after delete. Pointers on finding it and debugging?

Eric Malm <emalm@...>
 

Thanks, Tom, glad you were able to use veritas to find and remove the stray
apps. I'd like to know how they remained present in the first place. Do you
have logs from the nsync-bulker jobs on the cc_bridge VMs in your
deployment? That BOSH job has the responsibility of updating the Diego
DesiredLRPs to match the current set of CF apps, so if there are
synchronization errors they should be present in those logs.

Thanks,
Eric, CF Runtime Diego PM

On Fri, Apr 8, 2016 at 8:09 AM, Kris Hicks <khicks(a)pivotal.io> wrote:

It would be nice to figure out the root cause here.

Does having two crashed and two apps have some significance as to why the
delete failed, though appeared successful?

On Friday, April 8, 2016, Tom Sherrod <tom.sherrod(a)gmail.com> wrote:

Thank you.

Veritas is quite informative. I found 2 apps running and 2 crashed.
I deleted them and all appears well.


On Mon, Apr 4, 2016 at 7:03 PM, Amit Gupta <agupta(a)pivotal.io> wrote:

Ok, I would use veritas to look at the Diego BBS, and confirm that it
still thinks the app is there. You can also go onto the router and query
its HTTP endpoint to confirm that the route you're seeing is also still
there: https://github.com/cloudfoundry/gorouter#instrumentation.
Lastly I would connect to the CCDB and confirm that the app and route are
*not* there. This will reduce the problem to figuring out why Diego isn't
being updated to know that the non-existing app is no longer desired.

On Mon, Apr 4, 2016 at 3:47 PM, Tom Sherrod <tom.sherrod(a)gmail.com>
wrote:

The route still exists. I was reluctant to delete it and have the "app"
still running. I wanted some way to track it down, not that it has helped,
other than let me know it is still running.

Pushed the app, with a different name/host, with no problems and it
runs as it should.

On Mon, Apr 4, 2016 at 6:17 PM, Amit Gupta <agupta(a)pivotal.io> wrote:

Tom,

So you're saying that none of the org/spaces shows the app or the
route, but the app continues to run and be routeable?

I could imagine this happen if some CC Bridge components are not able
to talk to either CC or Diego BBS, leaving the data in the Diego BBS
stale. In the case of stale info, Diego may not know that the LRP is no
longer desired, and it will do the safe thing of keeping it around, and
emitting its route to the gorouter, which just does what it's told (it
doesn't check whether CC knows about the route or not).

Are you able to push new apps or delete other apps with the Diego
backend?

Amit

On Fri, Apr 1, 2016 at 1:00 PM, Tom Sherrod <tom.sherrod(a)gmail.com>
wrote:

JT,

Thanks for responding.

This is a test runtime and small. I checked all orgs and spaces. No
routes matching the app.

Found the route information and the result:

{

"total_results": 0,

"total_pages": 1,

"prev_url": null,

"next_url": null,

"resources": []

}

To learn what the output may look like, I check existing routes with
apps and without. The output appears to be the same as if the app has been
deleted.

Even now, the app url still returns a page from the app, even though
it is deleted.

Thanks,

Tom

On Fri, Apr 1, 2016 at 1:52 PM, JT Archie <jarchie(a)pivotal.io> wrote:

Tom,

Are you sure the route isn't bound to another application in another
org/space?

When you do `cf routes` it only show routes for the current space.
You can hit specific API endpoints though to get all the apps for a route.

For example, `cf
curl /v2/routes/89fc2a5e-3a9b-4a88-a360-e405cdbd6f87/apps` will show all
the apps for a particular route. Obviously replacing the route ID with the
correct ID. To find that, I recommend going through `CF_TRACE=true cf
routes` and grabbing the ID.

Let see if you can hunt it down that way.

Kind Regards,

JT

On Fri, Apr 1, 2016 at 8:51 AM, Tom Sherrod <tom.sherrod(a)gmail.com>
wrote:

cf 230, diego 0.1450.0, etcd 27, garden-linux 0.330.0
Default to diego true.

Developer deployed a java application. Deleted the application: cf
delete <app> No errors.
The app still responds. The only thing left is the route.
I've not encountered this before. Delete has been delete and even
if route remains, 404 Not Found: Requested route ('<hostname.domain>') does
not exist. is returned.

Pointers on tracking this down appreciated.

Tom


Re: App running even after delete. Pointers on finding it and debugging?

Kris Hicks <khicks@...>
 

It would be nice to figure out the root cause here.

Does having two crashed and two apps have some significance as to why the
delete failed, though appeared successful?

On Friday, April 8, 2016, Tom Sherrod <tom.sherrod(a)gmail.com> wrote:

Thank you.

Veritas is quite informative. I found 2 apps running and 2 crashed.
I deleted them and all appears well.


On Mon, Apr 4, 2016 at 7:03 PM, Amit Gupta <agupta(a)pivotal.io
<javascript:_e(%7B%7D,'cvml','agupta(a)pivotal.io');>> wrote:

Ok, I would use veritas to look at the Diego BBS, and confirm that it
still thinks the app is there. You can also go onto the router and query
its HTTP endpoint to confirm that the route you're seeing is also still
there: https://github.com/cloudfoundry/gorouter#instrumentation. Lastly
I would connect to the CCDB and confirm that the app and route are *not*
there. This will reduce the problem to figuring out why Diego isn't being
updated to know that the non-existing app is no longer desired.

On Mon, Apr 4, 2016 at 3:47 PM, Tom Sherrod <tom.sherrod(a)gmail.com
<javascript:_e(%7B%7D,'cvml','tom.sherrod(a)gmail.com');>> wrote:

The route still exists. I was reluctant to delete it and have the "app"
still running. I wanted some way to track it down, not that it has helped,
other than let me know it is still running.

Pushed the app, with a different name/host, with no problems and it runs
as it should.

On Mon, Apr 4, 2016 at 6:17 PM, Amit Gupta <agupta(a)pivotal.io
<javascript:_e(%7B%7D,'cvml','agupta(a)pivotal.io');>> wrote:

Tom,

So you're saying that none of the org/spaces shows the app or the
route, but the app continues to run and be routeable?

I could imagine this happen if some CC Bridge components are not able
to talk to either CC or Diego BBS, leaving the data in the Diego BBS
stale. In the case of stale info, Diego may not know that the LRP is no
longer desired, and it will do the safe thing of keeping it around, and
emitting its route to the gorouter, which just does what it's told (it
doesn't check whether CC knows about the route or not).

Are you able to push new apps or delete other apps with the Diego
backend?

Amit

On Fri, Apr 1, 2016 at 1:00 PM, Tom Sherrod <tom.sherrod(a)gmail.com
<javascript:_e(%7B%7D,'cvml','tom.sherrod(a)gmail.com');>> wrote:

JT,

Thanks for responding.

This is a test runtime and small. I checked all orgs and spaces. No
routes matching the app.

Found the route information and the result:

{

"total_results": 0,

"total_pages": 1,

"prev_url": null,

"next_url": null,

"resources": []

}

To learn what the output may look like, I check existing routes with
apps and without. The output appears to be the same as if the app has been
deleted.

Even now, the app url still returns a page from the app, even though
it is deleted.

Thanks,

Tom

On Fri, Apr 1, 2016 at 1:52 PM, JT Archie <jarchie(a)pivotal.io
<javascript:_e(%7B%7D,'cvml','jarchie(a)pivotal.io');>> wrote:

Tom,

Are you sure the route isn't bound to another application in another
org/space?

When you do `cf routes` it only show routes for the current space.
You can hit specific API endpoints though to get all the apps for a route.

For example, `cf
curl /v2/routes/89fc2a5e-3a9b-4a88-a360-e405cdbd6f87/apps` will show all
the apps for a particular route. Obviously replacing the route ID with the
correct ID. To find that, I recommend going through `CF_TRACE=true cf
routes` and grabbing the ID.

Let see if you can hunt it down that way.

Kind Regards,

JT

On Fri, Apr 1, 2016 at 8:51 AM, Tom Sherrod <tom.sherrod(a)gmail.com
<javascript:_e(%7B%7D,'cvml','tom.sherrod(a)gmail.com');>> wrote:

cf 230, diego 0.1450.0, etcd 27, garden-linux 0.330.0
Default to diego true.

Developer deployed a java application. Deleted the application: cf
delete <app> No errors.
The app still responds. The only thing left is the route.
I've not encountered this before. Delete has been delete and even if
route remains, 404 Not Found: Requested route ('<hostname.domain>') does
not exist. is returned.

Pointers on tracking this down appreciated.

Tom


Re: App running even after delete. Pointers on finding it and debugging?

Tom Sherrod <tom.sherrod@...>
 

Thank you.

Veritas is quite informative. I found 2 apps running and 2 crashed.
I deleted them and all appears well.

On Mon, Apr 4, 2016 at 7:03 PM, Amit Gupta <agupta(a)pivotal.io> wrote:

Ok, I would use veritas to look at the Diego BBS, and confirm that it
still thinks the app is there. You can also go onto the router and query
its HTTP endpoint to confirm that the route you're seeing is also still
there: https://github.com/cloudfoundry/gorouter#instrumentation. Lastly
I would connect to the CCDB and confirm that the app and route are *not*
there. This will reduce the problem to figuring out why Diego isn't being
updated to know that the non-existing app is no longer desired.

On Mon, Apr 4, 2016 at 3:47 PM, Tom Sherrod <tom.sherrod(a)gmail.com> wrote:

The route still exists. I was reluctant to delete it and have the "app"
still running. I wanted some way to track it down, not that it has helped,
other than let me know it is still running.

Pushed the app, with a different name/host, with no problems and it runs
as it should.

On Mon, Apr 4, 2016 at 6:17 PM, Amit Gupta <agupta(a)pivotal.io> wrote:

Tom,

So you're saying that none of the org/spaces shows the app or the route,
but the app continues to run and be routeable?

I could imagine this happen if some CC Bridge components are not able to
talk to either CC or Diego BBS, leaving the data in the Diego BBS stale.
In the case of stale info, Diego may not know that the LRP is no longer
desired, and it will do the safe thing of keeping it around, and emitting
its route to the gorouter, which just does what it's told (it doesn't check
whether CC knows about the route or not).

Are you able to push new apps or delete other apps with the Diego
backend?

Amit

On Fri, Apr 1, 2016 at 1:00 PM, Tom Sherrod <tom.sherrod(a)gmail.com>
wrote:

JT,

Thanks for responding.

This is a test runtime and small. I checked all orgs and spaces. No
routes matching the app.

Found the route information and the result:

{

"total_results": 0,

"total_pages": 1,

"prev_url": null,

"next_url": null,

"resources": []

}

To learn what the output may look like, I check existing routes with
apps and without. The output appears to be the same as if the app has been
deleted.

Even now, the app url still returns a page from the app, even though it
is deleted.

Thanks,

Tom

On Fri, Apr 1, 2016 at 1:52 PM, JT Archie <jarchie(a)pivotal.io> wrote:

Tom,

Are you sure the route isn't bound to another application in another
org/space?

When you do `cf routes` it only show routes for the current space. You
can hit specific API endpoints though to get all the apps for a route.

For example, `cf
curl /v2/routes/89fc2a5e-3a9b-4a88-a360-e405cdbd6f87/apps` will show all
the apps for a particular route. Obviously replacing the route ID with the
correct ID. To find that, I recommend going through `CF_TRACE=true cf
routes` and grabbing the ID.

Let see if you can hunt it down that way.

Kind Regards,

JT

On Fri, Apr 1, 2016 at 8:51 AM, Tom Sherrod <tom.sherrod(a)gmail.com>
wrote:

cf 230, diego 0.1450.0, etcd 27, garden-linux 0.330.0
Default to diego true.

Developer deployed a java application. Deleted the application: cf
delete <app> No errors.
The app still responds. The only thing left is the route.
I've not encountered this before. Delete has been delete and even if
route remains, 404 Not Found: Requested route ('<hostname.domain>') does
not exist. is returned.

Pointers on tracking this down appreciated.

Tom


Re: Static IP setup for routers on AWS

Engelke, Johannes <info@...>
 

Hi Amit,
thanks for your answer. I deployed cloud foundry without using static IP’s. It is working well.

As far as I understood the uaa config the entire 10.x.x.x network is allowed to access the UAA Servers anyway, so there is no reason to place the dedicated static IP's of the routers into the config.

Do you see any security improvements, if only routers are allowed to access the UAA?

Best,
Johannes

On 08 Apr 2016, at 02:19, Amit Gupta <agupta(a)pivotal.io> wrote:

The UAA needs to know the router IPs to know which IPs to accept inbound requests from. If you don't care about this, you can try configuring UAA to allow requests from many IPs, and remove the static IPs from gorouter. I would be interested to find out the result of this experiment should you try it out.

Best,
Amit

On Thu, Apr 7, 2016 at 6:28 AM, Engelke, Johannes <info(a)johannes-engelke.de <mailto:info(a)johannes-engelke.de>> wrote:
Hi,
does anybody know, why the routers got static ips in the cf-infrastructure-aws.yml file? https://github.com/cloudfoundry/cf-release/blob/master/templates/cf-infrastructure-aws.yml#L173 <https://github.com/cloudfoundry/cf-release/blob/master/templates/cf-infrastructure-aws.yml#L173>

Bosh is assigning the instances to ELB’s during deploy time, so there should be no need to have static addresses here.

If nobody know’s a good reason should we remove them ;-)

Cheers
Johannes


Re: [Metron Agent] failed to generate job templates with metron agent on top of OpenStack Dynamic network

Vik R <vagcom.ben@...>
 

I have the same issue. It has to do with every release since bosh 248.
However, dynamic networks with older bosh releases + cf-231/cf-231 work.

This must be a bug.

Ben R

On Thu, Apr 7, 2016 at 8:55 PM, Yitao Jiang <jiangyt.cn(a)gmail.com> wrote:

Hi,guys

When deploy CF on top of OpenStack with dynamic network, the jobs failed
with metron-agent
Error filling in template 'syslog_forwarder.conf.erb' (line 44: undefined
method `strip' for nil:NilClass)

here's related logs

​D​
etecting deployment changes
----------------------------
Releases
cf
version type changed: String -> Fixnum
- 233
+ 233

Compilation
No changes

Update
± canaries:
- 1
+ 0

Resource pools
No changes

Disk pools
No changes

Networks
dynamic-net
+ name: dynamic-net
subnets
10.0.0.0/24
cloud_properties
+ net_id: 0700ae03-4b38-464e-b40d-0a9c8dd18ff0
+ security_groups: ["Test OS SG_20160128T070152Z"]
+ dns: ["114.114.114.114", "8.8.8.8"]

+ range: 10.0.0.0/24
+ name: Test OS Sub Internal Network_20160128T070152Z

+ type: dynamic


Jobs
stats_z1

± networks:

- {"name"=>"cf1"}

+ {"name"=>"dynamic-net"}


Properties
No changes


Meta
No changes


Please review all changes carefully

Deploying
---------

Are you sure you want to deploy? (type 'yes' to continue): yes


Director task 57
Started preparing deployment > Preparing deployment. Done (00:00:03)


Error 100: Unable to render instance groups for deployment. Errors are:
- Unable to render jobs for instance group 'stats_z1'. Errors are:

- Unable to render templates for job 'metron_agent'. Errors are:

- Error filling in template 'syslog_forwarder.conf.erb' (line 44:
undefined method `strip' for nil:NilClass)

Task 57 error

For a more detailed error report, run: bosh task 57 --debug
as the ip manged by OpenStack, bosh cannot get the actual ip address of
each vm until vm alive, this lead to the generated job spec doesn't contain
ip address infos
so, must i have to configure network type to manual?​

snippets of deployment yml

1001 - name: dynamic-net
1002 subnets:
1003 - cloud_properties:
1004 net_id: 0700ae03-4b38-464e-b40d-0a9c8dd18ff0
1005 security_groups:
1006 - Test OS SG_20160128T070152Z
1007 dns:
1010 - 114.114.114.114
1011 - 8.8.8.8
1012 range: 10.0.0.0/24
1013 name: Test OS Sub Internal Network_20160128T070152Z
1014 type: dynamic

​Rendered job spec

{"deployment"=>"staging-01", "job"=

{"name"=>"stats_z1", "templates"=>[{"name"=>"collector",
"version"=>"6c210292f18d129e9a037fe7053836db2d494344",
"sha1"=>"38927f47b15c2daf6c8a2e7c760e73e5ff90
dfd4", "blobstore_id"=>"23531029-0ee1-4267-8863-b5f931afaecb"},
{"name"=>"metron_agent",
"version"=>"2b80a211127fc642fc8bb0d14d7eb30c37730db3", "sha1"=>"150f2
7445c2ef960951c1f26606525d41ec629b2",
"blobstore_id"=>"e87174dc-f3f7-4768-94cd-74f299813528"}],
"template"=>"collector", "version"=>"6c210292f18d129e9a037fe70
53836db2d494344", "sha1"=>"38927f47b15c2daf6c8a2e7c760e73e5ff90dfd4",
"blobstore_id"=>"23531029-0ee1-4267-8863-b5f931afaecb"}, "index"=>0,
"bootstrap"=>true,
"name"=>"stats_z1", "id"=>"99f349d0-fb5d-4de7-9912-3de5559d2f19",
"az"=>nil,

*"networks"=>{"dynamic-net"=>{"type"=>"dynamic",
"cloud_properties"=>{"net_id"=>"0700ae03-4b38-464e-b40d-0a9c8dd18ff0",
"security_groups"=>["Test OS SG_20160128T070152Z"]},
"dns"=>["114.114.114.114", "8.8.8.8", "10.0.0.13"], "default"=>["dns",
"gateway"],
"dns_record_name"=>"0.stats-z1.dynamic-net.staging-01.microbosh"}}*,
"properties"=>{"collector"=>{"aws"=>{
"access_key_id"=>nil, "secret_access_key"=>nil},
"datadog"=>{"api_key"=>nil, "application_key"=>nil},
"deployment_name"=>nil, "logging_level"=>"info", "interv
als"=>{"discover"=>60, "healthz"=>30, "local_metrics"=>30,
"nats_ping"=>30, "prune"=>300, "varz"=>30}, "use_aws_cloudwatch"=>false,
"use_datadog"=>false, "use
_tsdb"=>false, "opentsdb"=>{"address"=>nil, "port"=>nil},
"use_graphite"=>false, "graphite"=>{"address"=>nil, "port"=>nil},
"memory_threshold"=>800}, "nats"=>
{"machines"=>["10.0.0.127"], "password"=>"NATS_PASSWORD", "port"=>4222,
"user"=>"NATS_USER"}, "syslog_daemon_config"=>{"address"=>nil, "port"=>nil,
"transport
"=>"tcp", "fallback_addresses"=>[], "custom_rule"=>"",
"max_message_size"=>"4k"},
"metron_agent"=>{"dropsonde_incoming_port"=>3457, "preferred_protocol"=>"udp
", "tls"=>{"client_cert"=>"", "client_key"=>""}, "debug"=>false,
"zone"=>"z1", "deployment"=>"ya-staging-01",
"tcp"=>{"batching_buffer_bytes"=>10240, "batchin
g_buffer_flush_interval_milliseconds"=>100}, "logrotate"=>{"freq_min"=>5,
"rotate"=>7, "size"=>"50M"}, "buffer_size"=>10000, "enable_buffer"=>false},
"metron_
endpoint"=>{"shared_secret"=>"LOGGREGATOR_ENDPOINT_SHARED_SECRET"},
"loggregator"=>{"tls"=>{"ca_cert"=>""}, "dropsonde_incoming_port"=>3457,
"etcd"=>{"machine
s"=>["10.0.0.133"], "maxconcurrentrequests"=>10}}},
"dns_domain_name"=>"microbosh", "links"=>{},
"address"=>"99f349d0-fb5d-4de7-9912-3de5559d2f19.stats-z1.dyn
amic-net.ya-staging-01.microbosh", "persistent_disk"=>0,
"resource_pool"=>"small_z1"}​

--

Regards,

Yitao


Re: cf v233 api_z1/api_z2 failing

Ranga Rajagopalan
 

HI Kara,

Thanks. Let me try with a valid app_domain.

On Thu, Apr 7, 2016 at 9:01 PM, Kara Alexandra <ardnaxelarak(a)gmail.com>
wrote:

Hi Ranga,

The only reason we were using bosh-lite.com for our app_domains was
because we were testing to reproduce on our local bosh-lite.
Using 'cfapp' I managed to reproduce the issue locally. My guess is that
this is because 'cfapp' is not a valid domain (it doesn't end with a valid
top-level domain), and I'm guessing that fixing that will at least fix part
of the problem if not all of it.

Thanks,

Kara

On Thu, Apr 7, 2016 at 8:27 PM, Ranga Rajagopalan <
ranga.rajagopalan(a)gmail.com> wrote:

Here's my deployment manifest. app_domains is set to cfapp. I can't find
bost-lite anywhere in the file at all.

On Thu, Apr 7, 2016 at 4:27 PM, Peter Goetz <peter.gtz(a)gmail.com> wrote:

Hi Ranga,

Looking at your logs we found an error that could possibly cause this
and it is related to the properties.apps_domain in the deployment manifest.
By setting it to 'b%()osh-lite.com' (using special characters), we
could reproduce the following error which we also found in your log file:

Encountered error: name
format\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/model/base.rb:1543:in
`save'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/app/models/runtime/shared_domain.rb:35:in
`block in
find_or_create'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/transactions.rb:134:in
`_transaction'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/transactions.rb:108:in
`block in
transaction'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/connecting.rb:249:in
`block in
synchronize'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/connection_pool/threaded.rb:103:in
`hold'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/connecting.rb:249:in
`synchronize'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/transactions.rb:97:in
`transaction'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/app/models/runtime/shared_domain.rb:27:in
`find_or_create'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/seeds.rb:57:in
`block in
create_seed_domains'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/seeds.rb:56:in
`each'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/seeds.rb:56:in
`create_seed_domains'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/seeds.rb:9:in
`write_seed_data'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/runner.rb:93:in
`block in
run!'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/eventmachine-1.0.9.1/lib/eventmachine.rb:193:in
`call'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/eventmachine-1.0.9.1/lib/eventmachine.rb:193:in
`run_machine'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/eventmachine-1.0.9.1/lib/eventmachine.rb:193:in
`run'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/runner.rb:87:in
`run!'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/bin/cloud_controller:8:in
`<main>'

Can you check the apps_domain property and see if there is anything
suspicious with it?

Thanks,
Peter & Kara

On Thu, Apr 7, 2016 at 2:32 PM Ranga Rajagopalan <
ranga.rajagopalan(a)gmail.com> wrote:

Hi Peter,

Attaching /var/vcap/sys/log/cloud_controller_worker/cloud_controller_worker.log.gz
and /var/vcap/sys/log/cloud_controller_worker_ctl.log.gz. There isn't a
/var/vcap/sys/log/cloud_controller_worker/ directory on this node.

vcap(a)572afc33-0735-4727-8ff3-9dc6d7fa8af0:~$ ls /var/vcap/sys/log/
agent_ctl.err.log cloud_controller_worker_ctl.log
nginx_cc/
agent_ctl.log consul_agent/
nginx_ctl.err.log
cloud_controller_migration/ metron_agent/
nginx_ctl.log
cloud_controller_migration_ctl.err.log metron_agent_ctl.err.log
route_registrar/
cloud_controller_migration_ctl.log metron_agent_ctl.log
route_registrar_ctl.err.log
cloud_controller_ng/ monit/
route_registrar_ctl.log
cloud_controller_ng_ctl.err.log nfs_mounter/
statsd-injector/
cloud_controller_ng_ctl.log nfs_mounter_ctl.err.log
statsd-injector-ctl.err.log
cloud_controller_worker_ctl.err.log nfs_mounter_ctl.log
statsd-injector-ctl.log



On Thu, Apr 7, 2016 at 12:21 PM, Peter Goetz <peter.gtz(a)gmail.com>
wrote:

Hi Ranga,

To trouble-shoot this issue could you also provide the contents of
/var/vcap/sys/log/cloud_controller_ng/cloud_controller_ng.log
and /var/vcap/sys/log/cloud_controller_worker/cloud_controller_worker.log?
This should give us more details about what's going on. The ctl script logs
do not provide enough details.

Thanks,
Peter

On Wed, Apr 6, 2016 at 6:12 PM Ranga Rajagopalan <
ranga.rajagopalan(a)gmail.com> wrote:

I tried v231. Unfortunately, same issue.

--
Thanks,

Ranga

--
Thanks,

Ranga

--
Thanks,

Ranga


Re: cf v233 api_z1/api_z2 failing

Kara Alexandra <ardnaxelarak@...>
 

Hi Ranga,

The only reason we were using bosh-lite.com for our app_domains was because
we were testing to reproduce on our local bosh-lite.
Using 'cfapp' I managed to reproduce the issue locally. My guess is that
this is because 'cfapp' is not a valid domain (it doesn't end with a valid
top-level domain), and I'm guessing that fixing that will at least fix part
of the problem if not all of it.

Thanks,

Kara

On Thu, Apr 7, 2016 at 8:27 PM, Ranga Rajagopalan <
ranga.rajagopalan(a)gmail.com> wrote:

Here's my deployment manifest. app_domains is set to cfapp. I can't find
bost-lite anywhere in the file at all.

On Thu, Apr 7, 2016 at 4:27 PM, Peter Goetz <peter.gtz(a)gmail.com> wrote:

Hi Ranga,

Looking at your logs we found an error that could possibly cause this and
it is related to the properties.apps_domain in the deployment manifest. By
setting it to 'b%()osh-lite.com' (using special characters), we could
reproduce the following error which we also found in your log file:

Encountered error: name
format\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/model/base.rb:1543:in
`save'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/app/models/runtime/shared_domain.rb:35:in
`block in
find_or_create'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/transactions.rb:134:in
`_transaction'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/transactions.rb:108:in
`block in
transaction'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/connecting.rb:249:in
`block in
synchronize'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/connection_pool/threaded.rb:103:in
`hold'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/connecting.rb:249:in
`synchronize'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/transactions.rb:97:in
`transaction'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/app/models/runtime/shared_domain.rb:27:in
`find_or_create'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/seeds.rb:57:in
`block in
create_seed_domains'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/seeds.rb:56:in
`each'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/seeds.rb:56:in
`create_seed_domains'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/seeds.rb:9:in
`write_seed_data'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/runner.rb:93:in
`block in
run!'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/eventmachine-1.0.9.1/lib/eventmachine.rb:193:in
`call'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/eventmachine-1.0.9.1/lib/eventmachine.rb:193:in
`run_machine'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/eventmachine-1.0.9.1/lib/eventmachine.rb:193:in
`run'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/runner.rb:87:in
`run!'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/bin/cloud_controller:8:in
`<main>'

Can you check the apps_domain property and see if there is anything
suspicious with it?

Thanks,
Peter & Kara

On Thu, Apr 7, 2016 at 2:32 PM Ranga Rajagopalan <
ranga.rajagopalan(a)gmail.com> wrote:

Hi Peter,

Attaching /var/vcap/sys/log/cloud_controller_worker/cloud_controller_worker.log.gz
and /var/vcap/sys/log/cloud_controller_worker_ctl.log.gz. There isn't a
/var/vcap/sys/log/cloud_controller_worker/ directory on this node.

vcap(a)572afc33-0735-4727-8ff3-9dc6d7fa8af0:~$ ls /var/vcap/sys/log/
agent_ctl.err.log cloud_controller_worker_ctl.log
nginx_cc/
agent_ctl.log consul_agent/
nginx_ctl.err.log
cloud_controller_migration/ metron_agent/
nginx_ctl.log
cloud_controller_migration_ctl.err.log metron_agent_ctl.err.log
route_registrar/
cloud_controller_migration_ctl.log metron_agent_ctl.log
route_registrar_ctl.err.log
cloud_controller_ng/ monit/
route_registrar_ctl.log
cloud_controller_ng_ctl.err.log nfs_mounter/
statsd-injector/
cloud_controller_ng_ctl.log nfs_mounter_ctl.err.log
statsd-injector-ctl.err.log
cloud_controller_worker_ctl.err.log nfs_mounter_ctl.log
statsd-injector-ctl.log



On Thu, Apr 7, 2016 at 12:21 PM, Peter Goetz <peter.gtz(a)gmail.com>
wrote:

Hi Ranga,

To trouble-shoot this issue could you also provide the contents of
/var/vcap/sys/log/cloud_controller_ng/cloud_controller_ng.log
and /var/vcap/sys/log/cloud_controller_worker/cloud_controller_worker.log?
This should give us more details about what's going on. The ctl script logs
do not provide enough details.

Thanks,
Peter

On Wed, Apr 6, 2016 at 6:12 PM Ranga Rajagopalan <
ranga.rajagopalan(a)gmail.com> wrote:

I tried v231. Unfortunately, same issue.

--
Thanks,

Ranga

--
Thanks,

Ranga


[Metron Agent] failed to generate job templates with metron agent on top of OpenStack Dynamic network

Yitao Jiang
 

Hi,guys

When deploy CF on top of OpenStack with dynamic network, the jobs failed
with metron-agent
Error filling in template 'syslog_forwarder.conf.erb' (line 44: undefined
method `strip' for nil:NilClass)

here's related logs

​D​
etecting deployment changes
----------------------------
Releases
cf
version type changed: String -> Fixnum
- 233
+ 233

Compilation
No changes

Update
± canaries:
- 1
+ 0

Resource pools
No changes

Disk pools
No changes

Networks
dynamic-net
+ name: dynamic-net
subnets
10.0.0.0/24
cloud_properties
+ net_id: 0700ae03-4b38-464e-b40d-0a9c8dd18ff0
+ security_groups: ["Test OS SG_20160128T070152Z"]
+ dns: ["114.114.114.114", "8.8.8.8"]

+ range: 10.0.0.0/24
+ name: Test OS Sub Internal Network_20160128T070152Z

+ type: dynamic


Jobs
stats_z1

± networks:

- {"name"=>"cf1"}

+ {"name"=>"dynamic-net"}


Properties
No changes


Meta
No changes


Please review all changes carefully

Deploying
---------

Are you sure you want to deploy? (type 'yes' to continue): yes


Director task 57
Started preparing deployment > Preparing deployment. Done (00:00:03)


Error 100: Unable to render instance groups for deployment. Errors are:
- Unable to render jobs for instance group 'stats_z1'. Errors are:

- Unable to render templates for job 'metron_agent'. Errors are:

- Error filling in template 'syslog_forwarder.conf.erb' (line 44:
undefined method `strip' for nil:NilClass)

Task 57 error

For a more detailed error report, run: bosh task 57 --debug
as the ip manged by OpenStack, bosh cannot get the actual ip address of
each vm until vm alive, this lead to the generated job spec doesn't contain
ip address infos
so, must i have to configure network type to manual?​

snippets of deployment yml

1001 - name: dynamic-net
1002 subnets:
1003 - cloud_properties:
1004 net_id: 0700ae03-4b38-464e-b40d-0a9c8dd18ff0
1005 security_groups:
1006 - Test OS SG_20160128T070152Z
1007 dns:
1010 - 114.114.114.114
1011 - 8.8.8.8
1012 range: 10.0.0.0/24
1013 name: Test OS Sub Internal Network_20160128T070152Z
1014 type: dynamic

​Rendered job spec

{"deployment"=>"staging-01", "job"=

{"name"=>"stats_z1", "templates"=>[{"name"=>"collector",
"version"=>"6c210292f18d129e9a037fe7053836db2d494344",
"sha1"=>"38927f47b15c2daf6c8a2e7c760e73e5ff90
dfd4", "blobstore_id"=>"23531029-0ee1-4267-8863-b5f931afaecb"},
{"name"=>"metron_agent",
"version"=>"2b80a211127fc642fc8bb0d14d7eb30c37730db3", "sha1"=>"150f2
7445c2ef960951c1f26606525d41ec629b2",
"blobstore_id"=>"e87174dc-f3f7-4768-94cd-74f299813528"}],
"template"=>"collector", "version"=>"6c210292f18d129e9a037fe70
53836db2d494344", "sha1"=>"38927f47b15c2daf6c8a2e7c760e73e5ff90dfd4",
"blobstore_id"=>"23531029-0ee1-4267-8863-b5f931afaecb"}, "index"=>0,
"bootstrap"=>true,
"name"=>"stats_z1", "id"=>"99f349d0-fb5d-4de7-9912-3de5559d2f19",
"az"=>nil,

*"networks"=>{"dynamic-net"=>{"type"=>"dynamic",
"cloud_properties"=>{"net_id"=>"0700ae03-4b38-464e-b40d-0a9c8dd18ff0",
"security_groups"=>["Test OS SG_20160128T070152Z"]},
"dns"=>["114.114.114.114", "8.8.8.8", "10.0.0.13"], "default"=>["dns",
"gateway"],
"dns_record_name"=>"0.stats-z1.dynamic-net.staging-01.microbosh"}}*,
"properties"=>{"collector"=>{"aws"=>{
"access_key_id"=>nil, "secret_access_key"=>nil},
"datadog"=>{"api_key"=>nil, "application_key"=>nil},
"deployment_name"=>nil, "logging_level"=>"info", "interv
als"=>{"discover"=>60, "healthz"=>30, "local_metrics"=>30,
"nats_ping"=>30, "prune"=>300, "varz"=>30}, "use_aws_cloudwatch"=>false,
"use_datadog"=>false, "use
_tsdb"=>false, "opentsdb"=>{"address"=>nil, "port"=>nil},
"use_graphite"=>false, "graphite"=>{"address"=>nil, "port"=>nil},
"memory_threshold"=>800}, "nats"=>
{"machines"=>["10.0.0.127"], "password"=>"NATS_PASSWORD", "port"=>4222,
"user"=>"NATS_USER"}, "syslog_daemon_config"=>{"address"=>nil, "port"=>nil,
"transport
"=>"tcp", "fallback_addresses"=>[], "custom_rule"=>"",
"max_message_size"=>"4k"},
"metron_agent"=>{"dropsonde_incoming_port"=>3457, "preferred_protocol"=>"udp
", "tls"=>{"client_cert"=>"", "client_key"=>""}, "debug"=>false,
"zone"=>"z1", "deployment"=>"ya-staging-01",
"tcp"=>{"batching_buffer_bytes"=>10240, "batchin
g_buffer_flush_interval_milliseconds"=>100}, "logrotate"=>{"freq_min"=>5,
"rotate"=>7, "size"=>"50M"}, "buffer_size"=>10000, "enable_buffer"=>false},
"metron_
endpoint"=>{"shared_secret"=>"LOGGREGATOR_ENDPOINT_SHARED_SECRET"},
"loggregator"=>{"tls"=>{"ca_cert"=>""}, "dropsonde_incoming_port"=>3457,
"etcd"=>{"machine
s"=>["10.0.0.133"], "maxconcurrentrequests"=>10}}},
"dns_domain_name"=>"microbosh", "links"=>{},
"address"=>"99f349d0-fb5d-4de7-9912-3de5559d2f19.stats-z1.dyn
amic-net.ya-staging-01.microbosh", "persistent_disk"=>0,
"resource_pool"=>"small_z1"}​

--

Regards,

Yitao


Re: cf v233 api_z1/api_z2 failing

Ranga Rajagopalan
 

Here's my deployment manifest. app_domains is set to cfapp. I can't find
bost-lite anywhere in the file at all.

On Thu, Apr 7, 2016 at 4:27 PM, Peter Goetz <peter.gtz(a)gmail.com> wrote:

Hi Ranga,

Looking at your logs we found an error that could possibly cause this and
it is related to the properties.apps_domain in the deployment manifest. By
setting it to 'b%()osh-lite.com' (using special characters), we could
reproduce the following error which we also found in your log file:

Encountered error: name
format\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/model/base.rb:1543:in
`save'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/app/models/runtime/shared_domain.rb:35:in
`block in
find_or_create'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/transactions.rb:134:in
`_transaction'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/transactions.rb:108:in
`block in
transaction'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/connecting.rb:249:in
`block in
synchronize'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/connection_pool/threaded.rb:103:in
`hold'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/connecting.rb:249:in
`synchronize'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/transactions.rb:97:in
`transaction'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/app/models/runtime/shared_domain.rb:27:in
`find_or_create'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/seeds.rb:57:in
`block in
create_seed_domains'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/seeds.rb:56:in
`each'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/seeds.rb:56:in
`create_seed_domains'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/seeds.rb:9:in
`write_seed_data'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/runner.rb:93:in
`block in
run!'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/eventmachine-1.0.9.1/lib/eventmachine.rb:193:in
`call'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/eventmachine-1.0.9.1/lib/eventmachine.rb:193:in
`run_machine'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/eventmachine-1.0.9.1/lib/eventmachine.rb:193:in
`run'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/runner.rb:87:in
`run!'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/bin/cloud_controller:8:in
`<main>'

Can you check the apps_domain property and see if there is anything
suspicious with it?

Thanks,
Peter & Kara

On Thu, Apr 7, 2016 at 2:32 PM Ranga Rajagopalan <
ranga.rajagopalan(a)gmail.com> wrote:

Hi Peter,

Attaching /var/vcap/sys/log/cloud_controller_worker/cloud_controller_worker.log.gz
and /var/vcap/sys/log/cloud_controller_worker_ctl.log.gz. There isn't a
/var/vcap/sys/log/cloud_controller_worker/ directory on this node.

vcap(a)572afc33-0735-4727-8ff3-9dc6d7fa8af0:~$ ls /var/vcap/sys/log/
agent_ctl.err.log cloud_controller_worker_ctl.log
nginx_cc/
agent_ctl.log consul_agent/
nginx_ctl.err.log
cloud_controller_migration/ metron_agent/
nginx_ctl.log
cloud_controller_migration_ctl.err.log metron_agent_ctl.err.log
route_registrar/
cloud_controller_migration_ctl.log metron_agent_ctl.log
route_registrar_ctl.err.log
cloud_controller_ng/ monit/
route_registrar_ctl.log
cloud_controller_ng_ctl.err.log nfs_mounter/
statsd-injector/
cloud_controller_ng_ctl.log nfs_mounter_ctl.err.log
statsd-injector-ctl.err.log
cloud_controller_worker_ctl.err.log nfs_mounter_ctl.log
statsd-injector-ctl.log



On Thu, Apr 7, 2016 at 12:21 PM, Peter Goetz <peter.gtz(a)gmail.com> wrote:

Hi Ranga,

To trouble-shoot this issue could you also provide the contents of
/var/vcap/sys/log/cloud_controller_ng/cloud_controller_ng.log
and /var/vcap/sys/log/cloud_controller_worker/cloud_controller_worker.log?
This should give us more details about what's going on. The ctl script logs
do not provide enough details.

Thanks,
Peter

On Wed, Apr 6, 2016 at 6:12 PM Ranga Rajagopalan <
ranga.rajagopalan(a)gmail.com> wrote:

I tried v231. Unfortunately, same issue.

--
Thanks,

Ranga
--
Thanks,

Ranga


Re: Automated Deployment of window cell for diego

Steven Benario
 

BOSH support for Windows is in the pipeline. In the meantime you hopefully
won't have to set up cells very often, but could automate the process with
some powershell scripting.

Cheers,
Steven Benario
Windows PM

On Thu, Apr 7, 2016 at 8:44 PM, Long Nguyen <long.nguyen11288(a)gmail.com>
wrote:

Ah was wondering if anyone came up with anything in meantime.

On April 7, 2016 at 8:16:15 PM, Gwenn Etourneau (getourneau(a)pivotal.io)
wrote:

Bosh support for windows is coming, instead for now you have the msi
installer ..

On Thu, Apr 7, 2016 at 11:27 PM, Long Nguyen <long.nguyen11288(a)gmail.com>
wrote:

Anyone know about automated deployment of window cell for diego? I've
done it manually a few months back and it was very manual and error prone.
Does anyone have anything that automate process?


Re: Automated Deployment of window cell for diego

Long Nguyen
 

Ah was wondering if anyone came up with anything in meantime.

On April 7, 2016 at 8:16:15 PM, Gwenn Etourneau (getourneau(a)pivotal.io) wrote:

Bosh support for windows is coming, instead for now you have the msi installer ..

On Thu, Apr 7, 2016 at 11:27 PM, Long Nguyen <long.nguyen11288(a)gmail.com> wrote:
Anyone know about automated deployment of window cell for diego? I've done it manually a few months back and it was very manual and error prone. Does anyone have anything that automate process?


Re: Static IP setup for routers on AWS

Amit Kumar Gupta
 

The UAA needs to know the router IPs to know which IPs to accept inbound
requests from. If you don't care about this, you can try configuring UAA
to allow requests from many IPs, and remove the static IPs from gorouter.
I would be interested to find out the result of this experiment should you
try it out.

Best,
Amit

On Thu, Apr 7, 2016 at 6:28 AM, Engelke, Johannes <info(a)johannes-engelke.de>
wrote:

Hi,
does anybody know, why the routers got static ips in the
cf-infrastructure-aws.yml file?
https://github.com/cloudfoundry/cf-release/blob/master/templates/cf-infrastructure-aws.yml#L173

Bosh is assigning the instances to ELB’s during deploy time, so there
should be no need to have static addresses here.

If nobody know’s a good reason should we remove them ;-)

Cheers
Johannes


Re: Automated Deployment of window cell for diego

Gwenn Etourneau
 

Bosh support for windows is coming, instead for now you have the msi
installer ..

On Thu, Apr 7, 2016 at 11:27 PM, Long Nguyen <long.nguyen11288(a)gmail.com>
wrote:

Anyone know about automated deployment of window cell for diego? I've done
it manually a few months back and it was very manual and error prone. Does
anyone have anything that automate process?


Re: cf stop not sending SIGTERM

Eric Malm <emalm@...>
 

Hi, William,

What version of the Java buildpack are you using? The introduction of the
memory calculator in v3.1 resulted in a more complicated start-command that
meant the app's java process wouldn't receive a TERM signal when the Diego
cell rep stopped the instance, but this was fixed in v3.4 of the buildpack
with a judiciously placed `exec`. Diego doesn't call the Stop() method on a
garden container, instead opting either to call Signal() on individual
processes that it has invoked or to call Destroy() the container.

Thanks,
Eric, CF Runtime Diego PM

On Thu, Apr 7, 2016 at 12:53 PM, William Tran <wtran(a)pivotal.io> wrote:

I'm not seeing graceful shutdown behaviour when I cf stop a Java app. By
the looks of Garden SIGTERM is sent and if the process is still alive after
10 seconds SIGKILL is sent:
https://github.com/cloudfoundry-incubator/garden-linux/blob/8ab2cbf0696fd0b6bd1a0f15f2d3a463e4acbdac/linux_backend/skeleton/stop.sh#L54-L72

I don't see this in my dead simple app Spring Boot app however:
https://github.com/william-tran/cf-shutdown-hook. When I SIGTERM locally
it only takes a second to gracefully shutdown.

I'm seeing this on PCF 1.6.8-build.1 (cf 225.3, garden 0.330.0)


Re: cf v233 api_z1/api_z2 failing

Peter Goetz <peter.gtz@...>
 

Hi Ranga,

Looking at your logs we found an error that could possibly cause this and
it is related to the properties.apps_domain in the deployment manifest. By
setting it to 'b%()osh-lite.com' (using special characters), we could
reproduce the following error which we also found in your log file:

Encountered error: name
format\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/model/base.rb:1543:in
`save'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/app/models/runtime/shared_domain.rb:35:in
`block in
find_or_create'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/transactions.rb:134:in
`_transaction'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/transactions.rb:108:in
`block in
transaction'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/connecting.rb:249:in
`block in
synchronize'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/connection_pool/threaded.rb:103:in
`hold'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/connecting.rb:249:in
`synchronize'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sequel-4.29.0/lib/sequel/database/transactions.rb:97:in
`transaction'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/app/models/runtime/shared_domain.rb:27:in
`find_or_create'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/seeds.rb:57:in
`block in
create_seed_domains'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/seeds.rb:56:in
`each'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/seeds.rb:56:in
`create_seed_domains'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/seeds.rb:9:in
`write_seed_data'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/runner.rb:93:in
`block in
run!'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/eventmachine-1.0.9.1/lib/eventmachine.rb:193:in
`call'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/eventmachine-1.0.9.1/lib/eventmachine.rb:193:in
`run_machine'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/eventmachine-1.0.9.1/lib/eventmachine.rb:193:in
`run'\n/var/vcap/data/packages/cloud_controller_ng/da452b34d79be56a0784c7e88d6b9c0e1811a9d8.1-f934c136e6019cf54ab7aa04a3c153657226e729/cloud_controller_ng/lib/cloud_controller/runner.rb:87:in
`run!'\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/bin/cloud_controller:8:in
`<main>'

Can you check the apps_domain property and see if there is anything
suspicious with it?

Thanks,
Peter & Kara

On Thu, Apr 7, 2016 at 2:32 PM Ranga Rajagopalan <
ranga.rajagopalan(a)gmail.com> wrote:

Hi Peter,

Attaching /var/vcap/sys/log/cloud_controller_worker/cloud_controller_worker.log.gz
and /var/vcap/sys/log/cloud_controller_worker_ctl.log.gz. There isn't a
/var/vcap/sys/log/cloud_controller_worker/ directory on this node.

vcap(a)572afc33-0735-4727-8ff3-9dc6d7fa8af0:~$ ls /var/vcap/sys/log/
agent_ctl.err.log cloud_controller_worker_ctl.log
nginx_cc/
agent_ctl.log consul_agent/
nginx_ctl.err.log
cloud_controller_migration/ metron_agent/
nginx_ctl.log
cloud_controller_migration_ctl.err.log metron_agent_ctl.err.log
route_registrar/
cloud_controller_migration_ctl.log metron_agent_ctl.log
route_registrar_ctl.err.log
cloud_controller_ng/ monit/
route_registrar_ctl.log
cloud_controller_ng_ctl.err.log nfs_mounter/
statsd-injector/
cloud_controller_ng_ctl.log nfs_mounter_ctl.err.log
statsd-injector-ctl.err.log
cloud_controller_worker_ctl.err.log nfs_mounter_ctl.log
statsd-injector-ctl.log



On Thu, Apr 7, 2016 at 12:21 PM, Peter Goetz <peter.gtz(a)gmail.com> wrote:

Hi Ranga,

To trouble-shoot this issue could you also provide the contents of
/var/vcap/sys/log/cloud_controller_ng/cloud_controller_ng.log
and /var/vcap/sys/log/cloud_controller_worker/cloud_controller_worker.log?
This should give us more details about what's going on. The ctl script logs
do not provide enough details.

Thanks,
Peter

On Wed, Apr 6, 2016 at 6:12 PM Ranga Rajagopalan <
ranga.rajagopalan(a)gmail.com> wrote:

I tried v231. Unfortunately, same issue.

--
Thanks,

Ranga


Re: CF 212 - service in "delete failed" -> how to remove it properly?

Natalie Bennett
 

`cf curl /v2/service_instances/#{service_guid}?purge=true -X DELETE` should
delete stuck service records.

`cf service --guid #{service_name}` will get you the service guid to use in
the above command.

Though note that this doesn't fix whatever caused the service to get into
the state that kept it from being deleted, and it probably still exists on
the system providing the service, so that needs to be dealt with
separately. (Or you might continue being charged for that service, for
instance.)

- Natalie

On Thu, Apr 7, 2016 at 6:27 AM Rafal Radecki <radecki.rafal(a)gmail.com>
wrote:

Hi :)

I have several organizations in which I have services in "delete failed"
state. They prevent me from deleting the organization.

What is the best way to remove metadata about such service?

BR,
Rafal.


Re: cf v233 api_z1/api_z2 failing

Ranga Rajagopalan
 

Hi Peter,

Attaching /var/vcap/sys/log/cloud_controller_worker/cloud_controller_worker.log.gz
and /var/vcap/sys/log/cloud_controller_worker_ctl.log.gz. There isn't a
/var/vcap/sys/log/cloud_controller_worker/ directory on this node.

vcap(a)572afc33-0735-4727-8ff3-9dc6d7fa8af0:~$ ls /var/vcap/sys/log/
agent_ctl.err.log cloud_controller_worker_ctl.log
nginx_cc/
agent_ctl.log consul_agent/
nginx_ctl.err.log
cloud_controller_migration/ metron_agent/
nginx_ctl.log
cloud_controller_migration_ctl.err.log metron_agent_ctl.err.log
route_registrar/
cloud_controller_migration_ctl.log metron_agent_ctl.log
route_registrar_ctl.err.log
cloud_controller_ng/ monit/
route_registrar_ctl.log
cloud_controller_ng_ctl.err.log nfs_mounter/
statsd-injector/
cloud_controller_ng_ctl.log nfs_mounter_ctl.err.log
statsd-injector-ctl.err.log
cloud_controller_worker_ctl.err.log nfs_mounter_ctl.log
statsd-injector-ctl.log

On Thu, Apr 7, 2016 at 12:21 PM, Peter Goetz <peter.gtz(a)gmail.com> wrote:

Hi Ranga,

To trouble-shoot this issue could you also provide the contents of
/var/vcap/sys/log/cloud_controller_ng/cloud_controller_ng.log
and /var/vcap/sys/log/cloud_controller_worker/cloud_controller_worker.log?
This should give us more details about what's going on. The ctl script logs
do not provide enough details.

Thanks,
Peter

On Wed, Apr 6, 2016 at 6:12 PM Ranga Rajagopalan <
ranga.rajagopalan(a)gmail.com> wrote:

I tried v231. Unfortunately, same issue.
--
Thanks,

Ranga