Syslog Drain to Logstash Problems


Steve Wall <steve.wall@...>
 

Hello,
We are having problems draining log messages to Logstash. The drain is
setup as a user provided service.

cf cups logstash-drain -l syslog://xx.xx.xx.xx:5000

And then bound to the service.

cf bind-service myapp logstash-drain

But no log messages are coming through to Logstash. Or more specifically,
we are using ELK and the messages aren't seen through Kibana.

We were able to log into the DEA and using netcat (nc), messages were
successfully submitted to the ELK stack.

nc -w0 -u xx.xx.xx.xx 5000 <<< "logging from remote"

Any suggestions on how to debug this further?
-Steve


John Tuley <jtuley@...>
 

Steve,

Until recently (cf-release v198), binding a syslog service required
restarting the app. If you're post-v198, it *should* Just Work.

However, one of the things that could be in your way is network security.
In order to forward logs to your drain, your loggregator servers must be
able to access that server. This is the most common cause we see of systems
failing to forward to syslog drains.

Please let us know if you have more questions.

– John Tuley

On Wed, Jun 3, 2015 at 12:37 PM, Steve Wall <
steve.wall(a)primetimesoftware.com> wrote:

Hello,
We are having problems draining log messages to Logstash. The drain is
setup as a user provided service.

cf cups logstash-drain -l syslog://xx.xx.xx.xx:5000

And then bound to the service.

cf bind-service myapp logstash-drain

But no log messages are coming through to Logstash. Or more specifically,
we are using ELK and the messages aren't seen through Kibana.

We were able to log into the DEA and using netcat (nc), messages were
successfully submitted to the ELK stack.

nc -w0 -u xx.xx.xx.xx 5000 <<< "logging from remote"

Any suggestions on how to debug this further?
-Steve


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Josh Ghiloni
 

We’ll check that, thanks!

Josh Ghiloni
Senior Consultant
303.932.2202 o | 303.590.5427 m | 303.565.2794 f
jghiloni(a)ecsteam.com<mailto:jghiloni(a)ecsteam.com>

ECS Team
Technology Solutions Delivered
ECSTeam.com<http://ECSTeam.com>

On Jun 3, 2015, at 15:41, John Tuley <jtuley(a)pivotal.io<mailto:jtuley(a)pivotal.io>> wrote:

Steve,

Until recently (cf-release v198), binding a syslog service required restarting the app. If you're post-v198, it should Just Work.

However, one of the things that could be in your way is network security. In order to forward logs to your drain, your loggregator servers must be able to access that server. This is the most common cause we see of systems failing to forward to syslog drains.

Please let us know if you have more questions.

– John Tuley

On Wed, Jun 3, 2015 at 12:37 PM, Steve Wall <steve.wall(a)primetimesoftware.com<mailto:steve.wall(a)primetimesoftware.com>> wrote:
Hello,
We are having problems draining log messages to Logstash. The drain is setup as a user provided service.

cf cups logstash-drain -l syslog://xx.xx.xx.xx:5000

And then bound to the service.

cf bind-service myapp logstash-drain

But no log messages are coming through to Logstash. Or more specifically, we are using ELK and the messages aren't seen through Kibana.

We were able to log into the DEA and using netcat (nc), messages were successfully submitted to the ELK stack.

nc -w0 -u xx.xx.xx.xx 5000 <<< "logging from remote"

Any suggestions on how to debug this further?
-Steve


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Steve Wall <steve.wall@...>
 

I was able submit a log entry from the loggregator VM using -

nc -w0 10.xx.xx.xx 5000 <<< "logging from loggregator"

and to test UDP

nc -u -w0 10.xx.xx.xx 5000 <<< "logging from loggregator"


Which leads me to believe the networking is working properly. Any other
thoughts?
Thanks!
Steve

On Wed, Jun 3, 2015 at 6:14 PM, Josh Ghiloni <jghiloni(a)ecsteam.com> wrote:

We’ll check that, thanks!

Josh Ghiloni
Senior Consultant
303.932.2202 o | 303.590.5427 m | 303.565.2794 f
jghiloni(a)ecsteam.com

ECS Team
Technology Solutions Delivered
ECSTeam.com





On Jun 3, 2015, at 15:41, John Tuley <jtuley(a)pivotal.io> wrote:

Steve,

Until recently (cf-release v198), binding a syslog service required
restarting the app. If you're post-v198, it *should* Just Work.

However, one of the things that could be in your way is network
security. In order to forward logs to your drain, your loggregator servers
must be able to access that server. This is the most common cause we see of
systems failing to forward to syslog drains.

Please let us know if you have more questions.

– John Tuley

On Wed, Jun 3, 2015 at 12:37 PM, Steve Wall <
steve.wall(a)primetimesoftware.com> wrote:

Hello,
We are having problems draining log messages to Logstash. The drain is
setup as a user provided service.

cf cups logstash-drain -l syslog://xx.xx.xx.xx:5000

And then bound to the service.

cf bind-service myapp logstash-drain

But no log messages are coming through to Logstash. Or more
specifically, we are using ELK and the messages aren't seen through Kibana.

We were able to log into the DEA and using netcat (nc), messages were
successfully submitted to the ELK stack.

nc -w0 -u xx.xx.xx.xx 5000 <<< "logging from remote"

Any suggestions on how to debug this further?
-Steve


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


MJ
 

We had the same problem due to missing templates in our manifest.

We initially used the example manifest snippet shown at https://github.com/cloudfoundry/loggregator which mentions only the doppler template. Looking at https://github.com/cloudfoundry/loggregator/blob/develop/manifest-templates/cf-lamb.yml we later determined that we also needed the syslog_drain_binder and metron_agent templates for a complete loggregator deployment.

-Mike

On Jun 10, 2015, at 9:35 AM, Steve Wall <steve.wall(a)primetimesoftware.com<mailto:steve.wall(a)primetimesoftware.com>> wrote:

I was able submit a log entry from the loggregator VM using -

nc -w0 10.xx.xx.xx 5000 <<< "logging from loggregator"

and to test UDP

nc -u -w0 10.xx.xx.xx 5000 <<< "logging from loggregator"


Which leads me to believe the networking is working properly. Any other thoughts?
Thanks!
Steve

On Wed, Jun 3, 2015 at 6:14 PM, Josh Ghiloni <jghiloni(a)ecsteam.com<mailto:jghiloni(a)ecsteam.com>> wrote:
We’ll check that, thanks!

Josh Ghiloni
Senior Consultant
303.932.2202<tel:303.932.2202> o | 303.590.5427<tel:303.590.5427> m | 303.565.2794<tel:303.565.2794> f
jghiloni(a)ecsteam.com<mailto:jghiloni(a)ecsteam.com>

ECS Team
Technology Solutions Delivered
ECSTeam.com<http://ecsteam.com/>





On Jun 3, 2015, at 15:41, John Tuley <jtuley(a)pivotal.io<mailto:jtuley(a)pivotal.io>> wrote:

Steve,

Until recently (cf-release v198), binding a syslog service required restarting the app. If you're post-v198, it should Just Work.

However, one of the things that could be in your way is network security. In order to forward logs to your drain, your loggregator servers must be able to access that server. This is the most common cause we see of systems failing to forward to syslog drains.

Please let us know if you have more questions.

– John Tuley

On Wed, Jun 3, 2015 at 12:37 PM, Steve Wall <steve.wall(a)primetimesoftware.com<mailto:steve.wall(a)primetimesoftware.com>> wrote:
Hello,
We are having problems draining log messages to Logstash. The drain is setup as a user provided service.

cf cups logstash-drain -l syslog://xx.xx.xx.xx:5000

And then bound to the service.

cf bind-service myapp logstash-drain

But no log messages are coming through to Logstash. Or more specifically, we are using ELK and the messages aren't seen through Kibana.

We were able to log into the DEA and using netcat (nc), messages were successfully submitted to the ELK stack.

nc -w0 -u xx.xx.xx.xx 5000 <<< "logging from remote"

Any suggestions on how to debug this further?
-Steve


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


John Tuley <jtuley@...>
 

I don't have an answer, so here are some diagnostic questions to help you
debug:

Have you set doppler.blacklisted_syslog_ranges to something that includes
your destination? (This would be bad.)
Do you have syslog_drain_binder running? (It should be colocated with
doppler on the loggregator VMs.) Does it seem healthy?
Can you use the HTTP API to etcd to look at the keys in
/loggregator/services. You should see an entry per (app ID, drain URL)
pair. If those aren't present, then the syslog drain binder isn't working,
and doppler won't get bindings.
If you have the collector enabled, look in your metrics for one containing
"DopplerServer.messageRouter.numberOfSyslogSinks". If that number seems
accurate, then it's likely networking (which you have already ruled out) or
something really weird in the code.



– John Tuley

On Wed, Jun 10, 2015 at 10:35 AM, Steve Wall <
steve.wall(a)primetimesoftware.com> wrote:

I was able submit a log entry from the loggregator VM using -

nc -w0 10.xx.xx.xx 5000 <<< "logging from loggregator"

and to test UDP

nc -u -w0 10.xx.xx.xx 5000 <<< "logging from loggregator"


Which leads me to believe the networking is working properly. Any other
thoughts?
Thanks!
Steve

On Wed, Jun 3, 2015 at 6:14 PM, Josh Ghiloni <jghiloni(a)ecsteam.com> wrote:

We’ll check that, thanks!

Josh Ghiloni
Senior Consultant
303.932.2202 o | 303.590.5427 m | 303.565.2794 f
jghiloni(a)ecsteam.com

ECS Team
Technology Solutions Delivered
ECSTeam.com





On Jun 3, 2015, at 15:41, John Tuley <jtuley(a)pivotal.io> wrote:

Steve,

Until recently (cf-release v198), binding a syslog service required
restarting the app. If you're post-v198, it *should* Just Work.

However, one of the things that could be in your way is network
security. In order to forward logs to your drain, your loggregator servers
must be able to access that server. This is the most common cause we see of
systems failing to forward to syslog drains.

Please let us know if you have more questions.

– John Tuley

On Wed, Jun 3, 2015 at 12:37 PM, Steve Wall <
steve.wall(a)primetimesoftware.com> wrote:

Hello,
We are having problems draining log messages to Logstash. The drain is
setup as a user provided service.

cf cups logstash-drain -l syslog://xx.xx.xx.xx:5000

And then bound to the service.

cf bind-service myapp logstash-drain

But no log messages are coming through to Logstash. Or more
specifically, we are using ELK and the messages aren't seen through Kibana.

We were able to log into the DEA and using netcat (nc), messages were
successfully submitted to the ELK stack.

nc -w0 -u xx.xx.xx.xx 5000 <<< "logging from remote"

Any suggestions on how to debug this further?
-Steve


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


John Tuley <jtuley@...>
 

Mike,

Thanks for finding that. I've filed a bug
<https://www.pivotaltracker.com/story/show/96801752> to get the README
fixed.

– John Tuley

On Thu, Jun 11, 2015 at 10:22 AM, Mike Jacobi <jacobi(a)adobe.com> wrote:

We had the same problem due to missing templates in our manifest.

We initially used the example manifest snippet shown at
https://github.com/cloudfoundry/loggregator which mentions only the
doppler template. Looking at
https://github.com/cloudfoundry/loggregator/blob/develop/manifest-templates/cf-lamb.yml we
later determined that we also needed the syslog_drain_binder
and metron_agent templates for a complete loggregator deployment.

-Mike



On Jun 10, 2015, at 9:35 AM, Steve Wall <steve.wall(a)primetimesoftware.com>
wrote:

I was able submit a log entry from the loggregator VM using -

nc -w0 10.xx.xx.xx 5000 <<< "logging from loggregator"

and to test UDP

nc -u -w0 10.xx.xx.xx 5000 <<< "logging from loggregator"


Which leads me to believe the networking is working properly. Any other
thoughts?
Thanks!
Steve

On Wed, Jun 3, 2015 at 6:14 PM, Josh Ghiloni <jghiloni(a)ecsteam.com> wrote:

We’ll check that, thanks!

Josh Ghiloni
Senior Consultant
303.932.2202 o | 303.590.5427 m | 303.565.2794 f
jghiloni(a)ecsteam.com

ECS Team
Technology Solutions Delivered
ECSTeam.com <http://ecsteam.com/>





On Jun 3, 2015, at 15:41, John Tuley <jtuley(a)pivotal.io> wrote:

Steve,

Until recently (cf-release v198), binding a syslog service required
restarting the app. If you're post-v198, it *should* Just Work.

However, one of the things that could be in your way is network
security. In order to forward logs to your drain, your loggregator servers
must be able to access that server. This is the most common cause we see of
systems failing to forward to syslog drains.

Please let us know if you have more questions.

– John Tuley

On Wed, Jun 3, 2015 at 12:37 PM, Steve Wall <
steve.wall(a)primetimesoftware.com> wrote:

Hello,
We are having problems draining log messages to Logstash. The drain is
setup as a user provided service.

cf cups logstash-drain -l syslog://xx.xx.xx.xx:5000

And then bound to the service.

cf bind-service myapp logstash-drain

But no log messages are coming through to Logstash. Or more
specifically, we are using ELK and the messages aren't seen through Kibana.

We were able to log into the DEA and using netcat (nc), messages were
successfully submitted to the ELK stack.

nc -w0 -u xx.xx.xx.xx 5000 <<< "logging from remote"

Any suggestions on how to debug this further?
-Steve


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Josh Ghiloni
 

It appears that we do not have the syslog_drain_binder templates as Mike mentioned — thanks guys! We’ll update that and give ‘er a go.

Josh Ghiloni
Senior Consultant
303.932.2202 o | 303.590.5427 m | 303.565.2794 f
jghiloni(a)ecsteam.com<mailto:jghiloni(a)ecsteam.com>

ECS Team
Technology Solutions Delivered
ECSTeam.com<http://ECSTeam.com>

On Jun 11, 2015, at 11:10, John Tuley <jtuley(a)pivotal.io<mailto:jtuley(a)pivotal.io>> wrote:

Mike,

Thanks for finding that. I've filed a bug<https://www.pivotaltracker.com/story/show/96801752> to get the README fixed.

– John Tuley

On Thu, Jun 11, 2015 at 10:22 AM, Mike Jacobi <jacobi(a)adobe.com<mailto:jacobi(a)adobe.com>> wrote:
We had the same problem due to missing templates in our manifest.

We initially used the example manifest snippet shown at https://github.com/cloudfoundry/loggregator which mentions only the doppler template. Looking at https://github.com/cloudfoundry/loggregator/blob/develop/manifest-templates/cf-lamb.yml we later determined that we also needed the syslog_drain_binder and metron_agent templates for a complete loggregator deployment.

-Mike



On Jun 10, 2015, at 9:35 AM, Steve Wall <steve.wall(a)primetimesoftware.com<mailto:steve.wall(a)primetimesoftware.com>> wrote:

I was able submit a log entry from the loggregator VM using -

nc -w0 10.xx.xx.xx 5000 <<< "logging from loggregator"

and to test UDP

nc -u -w0 10.xx.xx.xx 5000 <<< "logging from loggregator"


Which leads me to believe the networking is working properly. Any other thoughts?
Thanks!
Steve

On Wed, Jun 3, 2015 at 6:14 PM, Josh Ghiloni <jghiloni(a)ecsteam.com<mailto:jghiloni(a)ecsteam.com>> wrote:
We’ll check that, thanks!

Josh Ghiloni
Senior Consultant
303.932.2202<tel:303.932.2202> o | 303.590.5427<tel:303.590.5427> m | 303.565.2794<tel:303.565.2794> f
jghiloni(a)ecsteam.com<mailto:jghiloni(a)ecsteam.com>

ECS Team
Technology Solutions Delivered
ECSTeam.com<http://ecsteam.com/>





On Jun 3, 2015, at 15:41, John Tuley <jtuley(a)pivotal.io<mailto:jtuley(a)pivotal.io>> wrote:

Steve,

Until recently (cf-release v198), binding a syslog service required restarting the app. If you're post-v198, it should Just Work.

However, one of the things that could be in your way is network security. In order to forward logs to your drain, your loggregator servers must be able to access that server. This is the most common cause we see of systems failing to forward to syslog drains.

Please let us know if you have more questions.

– John Tuley

On Wed, Jun 3, 2015 at 12:37 PM, Steve Wall <steve.wall(a)primetimesoftware.com<mailto:steve.wall(a)primetimesoftware.com>> wrote:
Hello,
We are having problems draining log messages to Logstash. The drain is setup as a user provided service.

cf cups logstash-drain -l syslog://xx.xx.xx.xx:5000

And then bound to the service.

cf bind-service myapp logstash-drain

But no log messages are coming through to Logstash. Or more specifically, we are using ELK and the messages aren't seen through Kibana.

We were able to log into the DEA and using netcat (nc), messages were successfully submitted to the ELK stack.

nc -w0 -u xx.xx.xx.xx 5000 <<< "logging from remote"

Any suggestions on how to debug this further?
-Steve


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Josh Ghiloni
 

That did solve our problems. Thanks everyone!

Josh Ghiloni
Senior Consultant
303.932.2202 o | 303.590.5427 m | 303.565.2794 f
jghiloni(a)ecsteam.com<mailto:jghiloni(a)ecsteam.com>

ECS Team
Technology Solutions Delivered
ECSTeam.com<http://ECSTeam.com>

On Jun 12, 2015, at 09:20, Josh Ghiloni <jghiloni(a)ecsteam.com<mailto:jghiloni(a)ecsteam.com>> wrote:

It appears that we do not have the syslog_drain_binder templates as Mike mentioned — thanks guys! We’ll update that and give ‘er a go.

Josh Ghiloni
Senior Consultant
303.932.2202 o | 303.590.5427 m | 303.565.2794 f
jghiloni(a)ecsteam.com<mailto:jghiloni(a)ecsteam.com>

ECS Team
Technology Solutions Delivered
ECSTeam.com<http://ecsteam.com/>





On Jun 11, 2015, at 11:10, John Tuley <jtuley(a)pivotal.io<mailto:jtuley(a)pivotal.io>> wrote:

Mike,

Thanks for finding that. I've filed a bug<https://www.pivotaltracker.com/story/show/96801752> to get the README fixed.

– John Tuley

On Thu, Jun 11, 2015 at 10:22 AM, Mike Jacobi <jacobi(a)adobe.com<mailto:jacobi(a)adobe.com>> wrote:
We had the same problem due to missing templates in our manifest.

We initially used the example manifest snippet shown at https://github.com/cloudfoundry/loggregator which mentions only the doppler template. Looking at https://github.com/cloudfoundry/loggregator/blob/develop/manifest-templates/cf-lamb.yml we later determined that we also needed the syslog_drain_binder and metron_agent templates for a complete loggregator deployment.

-Mike



On Jun 10, 2015, at 9:35 AM, Steve Wall <steve.wall(a)primetimesoftware.com<mailto:steve.wall(a)primetimesoftware.com>> wrote:

I was able submit a log entry from the loggregator VM using -

nc -w0 10.xx.xx.xx 5000 <<< "logging from loggregator"

and to test UDP

nc -u -w0 10.xx.xx.xx 5000 <<< "logging from loggregator"


Which leads me to believe the networking is working properly. Any other thoughts?
Thanks!
Steve

On Wed, Jun 3, 2015 at 6:14 PM, Josh Ghiloni <jghiloni(a)ecsteam.com<mailto:jghiloni(a)ecsteam.com>> wrote:
We’ll check that, thanks!

Josh Ghiloni
Senior Consultant
303.932.2202<tel:303.932.2202> o | 303.590.5427<tel:303.590.5427> m | 303.565.2794<tel:303.565.2794> f
jghiloni(a)ecsteam.com<mailto:jghiloni(a)ecsteam.com>

ECS Team
Technology Solutions Delivered
ECSTeam.com<http://ecsteam.com/>





On Jun 3, 2015, at 15:41, John Tuley <jtuley(a)pivotal.io<mailto:jtuley(a)pivotal.io>> wrote:

Steve,

Until recently (cf-release v198), binding a syslog service required restarting the app. If you're post-v198, it should Just Work.

However, one of the things that could be in your way is network security. In order to forward logs to your drain, your loggregator servers must be able to access that server. This is the most common cause we see of systems failing to forward to syslog drains.

Please let us know if you have more questions.

– John Tuley

On Wed, Jun 3, 2015 at 12:37 PM, Steve Wall <steve.wall(a)primetimesoftware.com<mailto:steve.wall(a)primetimesoftware.com>> wrote:
Hello,
We are having problems draining log messages to Logstash. The drain is setup as a user provided service.

cf cups logstash-drain -l syslog://xx.xx.xx.xx:5000

And then bound to the service.

cf bind-service myapp logstash-drain

But no log messages are coming through to Logstash. Or more specifically, we are using ELK and the messages aren't seen through Kibana.

We were able to log into the DEA and using netcat (nc), messages were successfully submitted to the ELK stack.

nc -w0 -u xx.xx.xx.xx 5000 <<< "logging from remote"

Any suggestions on how to debug this further?
-Steve


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev