Forwrad container metrics to syslog drain


Mehran Saliminia
 

Hi,

does anybody know what is the best way to forward container metrics to a user-specified syslog drain?


Carlo Alberto Ferraris
 

Will just drop this here in case somebody wants to add something: https://github.com/cloudfoundry/loggregator/issues/150


Geoff Franks <geoff@...>
 

Have you taken a look at firehose-to-syslog (https://github.com/cloudfoundry-community/firehose-to-syslog)?

On Oct 4, 2016, at 8:59 AM, Carlo Alberto Ferraris <carlo.ferraris(a)rakuten.com> wrote:

Will just drop this here in case somebody wants to add something: https://github.com/cloudfoundry/loggregator/issues/150


Mehran Saliminia
 

yes, but it needs admin credentials to fetch the container metrics. We want to deliver metrics to the syslog drain endpoint which user binds to his application as a third party log management system.


Geoff Franks <geoff@...>
 

Ah, in that case, check out https://github.com/cloudfoundry-community/kibana-me-logs and https://github.com/cloudfoundry-community/docker-boshrelease in conjunction with https://github.com/cloudfoundry-community/logstash-docker-boshrelease.

kibana-me-logs is a cf plugin that takes an app, uses the logstash-docker-boshrelease to create a logging service for the app, and then pushes a kibana app bound to the same service, so that users can see their app's log data in kibana.

If you don't want to use kibana, you can probably find the relevant info for the service broker in the docker-boshrelease, for being able to provision a service that provides a syslog drain URL for an application.

On Oct 5, 2016, at 1:41 AM, Mehran Saliminia <msaliminia(a)gmail.com> wrote:

yes, but it needs admin credentials to fetch the container metrics. We want to deliver metrics to the syslog drain endpoint which user binds to his application as a third party log management system.


Daniel Mikusa
 

I think this might also be helpful.

https://github.com/cloudfoundry/noaa#sample-applications

Sample app that shows how to connect to loggregator and stream container
metrics. It can run as an admin or regular user. I think it's probably
the simplest example of how to get container metrics from loggregator.

Dan


On Wed, Oct 5, 2016 at 9:21 AM, Geoff Franks <geoff(a)starkandwayne.com>
wrote:

Ah, in that case, check out https://github.com/
cloudfoundry-community/kibana-me-logs and https://github.com/
cloudfoundry-community/docker-boshrelease in conjunction with
https://github.com/cloudfoundry-community/logstash-docker-boshrelease.

kibana-me-logs is a cf plugin that takes an app, uses the
logstash-docker-boshrelease to create a logging service for the app, and
then pushes a kibana app bound to the same service, so that users can see
their app's log data in kibana.

If you don't want to use kibana, you can probably find the relevant info
for the service broker in the docker-boshrelease, for being able to
provision a service that provides a syslog drain URL for an application.

On Oct 5, 2016, at 1:41 AM, Mehran Saliminia <msaliminia(a)gmail.com>
wrote:

yes, but it needs admin credentials to fetch the container metrics. We
want to deliver metrics to the syslog drain endpoint which user binds to
his application as a third party log management system.


Mehran Saliminia
 

thank you for your response! it works for app logs with a regular user token but streaming ContainerMetrics will only succeed if you have admin credentials.


Johannes Tuchscherer
 

That is not quite true. From the Firehose you can get ContainerMetrics for
an app as long as you have access to that app. The Firehose CLI plugin for
example shows all application related messages - log messages,
httpstartstop events and container metrics. And that plugin works even for
a 'normal' space developer.

Here is the method from the noaa library that you can use to get access to
that data:
https://github.com/cloudfoundry/noaa/blob/master/consumer/async.go#L53-L55

Johannes

On Wed, Oct 5, 2016 at 4:09 PM Mehran Saliminia <msaliminia(a)gmail.com>
wrote:

thank you for your response! it works for app logs with a regular user
token but streaming ContainerMetrics will only succeed if you have admin
credentials.


Johannes Hiemer <jvhiemer@...>
 

That's very neat. Thanks for that hint Johannes!

On 5 Oct 2016, at 16:18, Johannes Tuchscherer <jtuchscherer(a)pivotal.io> wrote:

That is not quite true. From the Firehose you can get ContainerMetrics for an app as long as you have access to that app. The Firehose CLI plugin for example shows all application related messages - log messages, httpstartstop events and container metrics. And that plugin works even for a 'normal' space developer.

Here is the method from the noaa library that you can use to get access to that data: https://github.com/cloudfoundry/noaa/blob/master/consumer/async.go#L53-L55

Johannes

On Wed, Oct 5, 2016 at 4:09 PM Mehran Saliminia <msaliminia(a)gmail.com> wrote:
thank you for your response! it works for app logs with a regular user token but streaming ContainerMetrics will only succeed if you have admin credentials.


Mehran Saliminia
 

I have executed this sample app for 5 minutes:
https://github.com/cloudfoundry/noaa/blob/master/samples/app_logs/main.go/#L32-L39

As the sample app comments I also expected to get ContainerMetrics as well but I only receive eventType:LogMessage.


Daniel Mikusa
 

The app_logs demo is just logs. Look at either the firehose demo or the
container_metrics demo.

https://github.com/cloudfoundry/noaa#logs-and-metrics-firehose

or

https://github.com/cloudfoundry/noaa#container-metrics

Dan

On Wed, Oct 5, 2016 at 11:40 AM, Mehran Saliminia <msaliminia(a)gmail.com>
wrote:

I have executed this sample app for 5 minutes:
https://github.com/cloudfoundry/noaa/blob/master/
samples/app_logs/main.go/#L32-L39

As the sample app comments I also expected to get ContainerMetrics as well
but I only receive eventType:LogMessage.