Re: Throttling App Logging


Aleksey Zalesov
 

By the way how do you comply with the requirement to persist all the logs? CF logging system is lossy by its nature so can drop messages.

We have similar requirement - some log messages must be reliably delivered and stored. After some considerations we decided to use RabbitMQ service for delivering these kind of messages from apps to log storage. All other logs go through metron-doppler chain as usual.

Aleksey Zalesov | CloudFoundry Engineer | Altoros
Tel: (617) 841-2121 ext. 5707 | Toll free: 855-ALTOROS
Fax: (866) 201-3646 | Skype: aleksey_zalesov
www.altoros.com <http://www.altoros.com/> | blog.altoros.com <http://blog.altoros.com/> | twitter.com/altoros <http://twitter.com/altoros>

On 22 Sep 2015, at 11:01, Daniel Jones <daniel.jones(a)engineerbetter.com> wrote:

Thanks all.

Sadly the client has a regulatory requirement for some apps that all logs must be persisted for a number of years, so we can't drop messages indiscriminately using the loggregator buffer. They're a PCF customer, so I'll raise a feature request through the support process.

Cheers!

On Mon, Sep 21, 2015 at 5:24 PM, Rohit Kumar <rokumar(a)pivotal.io <mailto:rokumar(a)pivotal.io>> wrote:
It isn't possible to throttle logging output on a per application basis. It is possible to configure the message_drain_buffer_size [1] to be lower than the default value of 100, which will reduce the number of logs which loggregator will buffer. If the producer is filling up logs too quickly, loggregator will drop the messages present in its buffer. This configuration will affect ALL the applications running on your Cloud Foundry environment. You could play with that property and see if it helps.

Rohit

[1]: https://github.com/cloudfoundry/loggregator/blob/develop/bosh/jobs/doppler/spec#L60-L62 <https://github.com/cloudfoundry/loggregator/blob/develop/bosh/jobs/doppler/spec#L60-L62>

On Mon, Sep 21, 2015 at 2:57 AM, Daniel Jones <daniel.jones(a)engineerbetter.com <mailto:daniel.jones(a)engineerbetter.com>> wrote:

Is it possible with the current logging infrastructure in CF to limit the logging throughput of particular apps or spaces?

Current client is running CF multi-tenant, and has some particularly noisy customers. It'd be nice to be able to put a hard limit on how much they can pass through to downstream commercial log indexers.

Any suggestions most gratefully received!

Regards,

Daniel Jones
EngineerBetter.com




--
Regards,

Daniel Jones
EngineerBetter.com

Join cf-dev@lists.cloudfoundry.org to automatically receive all group messages.