Re: [cf-bosh] Re: Re: [Metron Agent] failed to generate job templates with metron agent on top of OpenStack Dynamic network


Amit Kumar Gupta
 

Hi Yitao,

I would recommend either using manual networks, or following the GitHub
issue Ben R created above to see if the BOSH team can figure out the root
cause of this issue with dynamic networks.

Best,
Amit

On Tue, Apr 12, 2016 at 6:46 PM, Yitao Jiang <jiangyt.cn(a)gmail.com> wrote:

Thanks Amit.

If using dynamic network, the OpenStack support allocate ip address, using
the ip address to configure metron_agent and launch vm, will it possible
solve the issue?

On Tue, Apr 12, 2016 at 1:24 PM, Amit Gupta <agupta(a)pivotal.io> wrote:

This will not work with dynamic networks. Many jobs in cf-release rely
on data from BOSH to determine their IP so that configuration files can be
rendered up-front by the director rather than at runtime, requiring system
calls to determine IP. metron_agent is one such job, and it tends to be
colocated with each other job (it is what allows all system component logs
to be aggregated through the loggregator system), so this would require all
Cloud Foundry VMs to be on a manual network. You don't need to manually
pick the IPs, you just need to tell BOSH which IPs in the network not to
use and specify these in the "reserved" range.

Since so many different components depend on being able to determine
their IP via BOSH data, there's no quick workaround if you want to stick to
using dynamic networks, but we're aware of this current limitation.

Best,
Amit

On Mon, Apr 11, 2016 at 7:23 PM, Yitao Jiang <jiangyt.cn(a)gmail.com>
wrote:

Is it a bug of CF or Bosh ?

On Fri, Apr 8, 2016 at 12:08 PM, Ben R <vagcom.ben(a)gmail.com> wrote:

I have the same issue. It has to do with every release since bosh 248.
However, dynamic networks with older bosh releases + cf-231/cf-231 work.

This must be a bug.

Ben R


On Thu, Apr 7, 2016 at 8:55 PM, Yitao Jiang <jiangyt.cn(a)gmail.com>
wrote:

Hi,guys

When deploy CF on top of OpenStack with dynamic network, the jobs
failed with metron-agent
Error filling in template 'syslog_forwarder.conf.erb' (line 44:
undefined method `strip' for nil:NilClass)

here's related logs

​D​
etecting deployment changes
----------------------------
Releases
cf
version type changed: String -> Fixnum
- 233
+ 233

Compilation
No changes

Update
± canaries:
- 1
+ 0

Resource pools
No changes

Disk pools
No changes

Networks
dynamic-net
+ name: dynamic-net
subnets
10.0.0.0/24
cloud_properties
+ net_id: 0700ae03-4b38-464e-b40d-0a9c8dd18ff0
+ security_groups: ["Test OS SG_20160128T070152Z"]
+ dns: ["114.114.114.114", "8.8.8.8"]

+ range: 10.0.0.0/24
+ name: Test OS Sub Internal Network_20160128T070152Z

+ type: dynamic


Jobs
stats_z1

± networks:

- {"name"=>"cf1"}

+ {"name"=>"dynamic-net"}


Properties
No changes


Meta
No changes


Please review all changes carefully

Deploying
---------

Are you sure you want to deploy? (type 'yes' to continue): yes


Director task 57
Started preparing deployment > Preparing deployment. Done (00:00:03)


Error 100: Unable to render instance groups for deployment. Errors
are:
- Unable to render jobs for instance group 'stats_z1'. Errors are:

- Unable to render templates for job 'metron_agent'. Errors are:

- Error filling in template 'syslog_forwarder.conf.erb' (line
44: undefined method `strip' for nil:NilClass)

Task 57 error

For a more detailed error report, run: bosh task 57 --debug
as the ip manged by OpenStack, bosh cannot get the actual ip address
of each vm until vm alive, this lead to the generated job spec doesn't
contain ip address infos
so, must i have to configure network type to manual?​

snippets of deployment yml

1001 - name: dynamic-net
1002 subnets:
1003 - cloud_properties:
1004 net_id: 0700ae03-4b38-464e-b40d-0a9c8dd18ff0
1005 security_groups:
1006 - Test OS SG_20160128T070152Z
1007 dns:
1010 - 114.114.114.114
1011 - 8.8.8.8
1012 range: 10.0.0.0/24
1013 name: Test OS Sub Internal Network_20160128T070152Z
1014 type: dynamic

​Rendered job spec

{"deployment"=>"staging-01", "job"=

{"name"=>"stats_z1", "templates"=>[{"name"=>"collector",
"version"=>"6c210292f18d129e9a037fe7053836db2d494344",
"sha1"=>"38927f47b15c2daf6c8a2e7c760e73e5ff90
dfd4", "blobstore_id"=>"23531029-0ee1-4267-8863-b5f931afaecb"},
{"name"=>"metron_agent",
"version"=>"2b80a211127fc642fc8bb0d14d7eb30c37730db3", "sha1"=>"150f2
7445c2ef960951c1f26606525d41ec629b2",
"blobstore_id"=>"e87174dc-f3f7-4768-94cd-74f299813528"}],
"template"=>"collector", "version"=>"6c210292f18d129e9a037fe70
53836db2d494344", "sha1"=>"38927f47b15c2daf6c8a2e7c760e73e5ff90dfd4",
"blobstore_id"=>"23531029-0ee1-4267-8863-b5f931afaecb"}, "index"=>0,
"bootstrap"=>true,
"name"=>"stats_z1", "id"=>"99f349d0-fb5d-4de7-9912-3de5559d2f19",
"az"=>nil,

*"networks"=>{"dynamic-net"=>{"type"=>"dynamic",
"cloud_properties"=>{"net_id"=>"0700ae03-4b38-464e-b40d-0a9c8dd18ff0",
"security_groups"=>["Test OS SG_20160128T070152Z"]},
"dns"=>["114.114.114.114", "8.8.8.8", "10.0.0.13"], "default"=>["dns",
"gateway"],
"dns_record_name"=>"0.stats-z1.dynamic-net.staging-01.microbosh"}}*,
"properties"=>{"collector"=>{"aws"=>{
"access_key_id"=>nil, "secret_access_key"=>nil},
"datadog"=>{"api_key"=>nil, "application_key"=>nil},
"deployment_name"=>nil, "logging_level"=>"info", "interv
als"=>{"discover"=>60, "healthz"=>30, "local_metrics"=>30,
"nats_ping"=>30, "prune"=>300, "varz"=>30}, "use_aws_cloudwatch"=>false,
"use_datadog"=>false, "use
_tsdb"=>false, "opentsdb"=>{"address"=>nil, "port"=>nil},
"use_graphite"=>false, "graphite"=>{"address"=>nil, "port"=>nil},
"memory_threshold"=>800}, "nats"=>
{"machines"=>["10.0.0.127"], "password"=>"NATS_PASSWORD",
"port"=>4222, "user"=>"NATS_USER"},
"syslog_daemon_config"=>{"address"=>nil, "port"=>nil, "transport
"=>"tcp", "fallback_addresses"=>[], "custom_rule"=>"",
"max_message_size"=>"4k"},
"metron_agent"=>{"dropsonde_incoming_port"=>3457, "preferred_protocol"=>"udp
", "tls"=>{"client_cert"=>"", "client_key"=>""}, "debug"=>false,
"zone"=>"z1", "deployment"=>"ya-staging-01",
"tcp"=>{"batching_buffer_bytes"=>10240, "batchin
g_buffer_flush_interval_milliseconds"=>100},
"logrotate"=>{"freq_min"=>5, "rotate"=>7, "size"=>"50M"},
"buffer_size"=>10000, "enable_buffer"=>false}, "metron_
endpoint"=>{"shared_secret"=>"LOGGREGATOR_ENDPOINT_SHARED_SECRET"},
"loggregator"=>{"tls"=>{"ca_cert"=>""}, "dropsonde_incoming_port"=>3457,
"etcd"=>{"machine
s"=>["10.0.0.133"], "maxconcurrentrequests"=>10}}},
"dns_domain_name"=>"microbosh", "links"=>{},
"address"=>"99f349d0-fb5d-4de7-9912-3de5559d2f19.stats-z1.dyn
amic-net.ya-staging-01.microbosh", "persistent_disk"=>0,
"resource_pool"=>"small_z1"}​

--

Regards,

Yitao

--

Regards,

Yitao

--

Regards,

Yitao

Join cf-dev@lists.cloudfoundry.org to automatically receive all group messages.