Noisy Neighbor Nozzle CLI - Quickly Determine High Volume Log Producers #loggregator


Adam Hevenor
 

Nearly all log reliability problems on CF can be attributed to two main factors. Improper scaling or "noisy neighbor" applications. For the former you can install logmon, check the scaling recommendations published by Pivotal and learn about our capacity planning methodology in our whitepaper.  For the latter Operators have had to rely on their own solutions until now. The Noisy Neighbor Nozzle is a set of nozzle applications and a CLI Plugin that make it easy to identify applications that are logging excessively (after some time playing with it we built in a color threshold at 1,000,000 logs per minute). We worked with several customers as part of Pivotal's early access program to make this nozzle easy to deploy and quickly utilize from the Command Line. We also have included an example "accumulator" that will send the results to datadog and allow operators to operationalize their monitoring of these events. One more thing...it's backwards compatible to nearly all versions of Cloud Foundry since it uses a simple methodology of counting logs through the Firehose.  



To learn more about how this helps Operators check out my post on Rapid Troubleshooting of the Cloud Foundry Logging System on Medium

Join cf-dev@lists.cloudfoundry.org to automatically receive all group messages.