Re: Syslog Drain to Logstash Problems


John Tuley <jtuley@...>
 

I don't have an answer, so here are some diagnostic questions to help you
debug:

Have you set doppler.blacklisted_syslog_ranges to something that includes
your destination? (This would be bad.)
Do you have syslog_drain_binder running? (It should be colocated with
doppler on the loggregator VMs.) Does it seem healthy?
Can you use the HTTP API to etcd to look at the keys in
/loggregator/services. You should see an entry per (app ID, drain URL)
pair. If those aren't present, then the syslog drain binder isn't working,
and doppler won't get bindings.
If you have the collector enabled, look in your metrics for one containing
"DopplerServer.messageRouter.numberOfSyslogSinks". If that number seems
accurate, then it's likely networking (which you have already ruled out) or
something really weird in the code.



– John Tuley

On Wed, Jun 10, 2015 at 10:35 AM, Steve Wall <
steve.wall(a)primetimesoftware.com> wrote:

I was able submit a log entry from the loggregator VM using -

nc -w0 10.xx.xx.xx 5000 <<< "logging from loggregator"

and to test UDP

nc -u -w0 10.xx.xx.xx 5000 <<< "logging from loggregator"


Which leads me to believe the networking is working properly. Any other
thoughts?
Thanks!
Steve

On Wed, Jun 3, 2015 at 6:14 PM, Josh Ghiloni <jghiloni(a)ecsteam.com> wrote:

We’ll check that, thanks!

Josh Ghiloni
Senior Consultant
303.932.2202 o | 303.590.5427 m | 303.565.2794 f
jghiloni(a)ecsteam.com

ECS Team
Technology Solutions Delivered
ECSTeam.com





On Jun 3, 2015, at 15:41, John Tuley <jtuley(a)pivotal.io> wrote:

Steve,

Until recently (cf-release v198), binding a syslog service required
restarting the app. If you're post-v198, it *should* Just Work.

However, one of the things that could be in your way is network
security. In order to forward logs to your drain, your loggregator servers
must be able to access that server. This is the most common cause we see of
systems failing to forward to syslog drains.

Please let us know if you have more questions.

– John Tuley

On Wed, Jun 3, 2015 at 12:37 PM, Steve Wall <
steve.wall(a)primetimesoftware.com> wrote:

Hello,
We are having problems draining log messages to Logstash. The drain is
setup as a user provided service.

cf cups logstash-drain -l syslog://xx.xx.xx.xx:5000

And then bound to the service.

cf bind-service myapp logstash-drain

But no log messages are coming through to Logstash. Or more
specifically, we are using ELK and the messages aren't seen through Kibana.

We were able to log into the DEA and using netcat (nc), messages were
successfully submitted to the ELK stack.

nc -w0 -u xx.xx.xx.xx 5000 <<< "logging from remote"

Any suggestions on how to debug this further?
-Steve


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

Join cf-dev@lists.cloudfoundry.org to automatically receive all group messages.