Introducing Log-Cache - A Horizontally Scalable Self Pruning Cache for Observability (built by #loggregator)


Adam Hevenor
 

Problem Summary

The current implementation of recent logs and container metrics on CF Application Runtime is slow and increases latency as operators scale up their deployments. Over the past year we have patched with concurrent processing and circuit breakers but the problem masked a larger problem we set out to fix this summer with an improved persistance architecture

Solution Summary

Creating an improved persistance layer has lead us to discover many additional advantages of offering a new restful API. To demonstrate these advantages we developed a log-cache-cli plugin that effectively replaces the cf logs command with a cf tail command. The tail command borrows many of it's flags and UX from the popular unix tail command, and allows you to follow, and window the cache in ways logs and logs --recent did not. 

Using Log Cache and Service Level Objectives

We have a couple of months of operation experience with this new release and we are working through some final packaging steps to include this in cf-deployment. For now you can include it in cf-deployment using an ops file and learn about the operation track record on our Product FAQ

Feedback and next steps

This feature is paving the way for a new developer UX as well as some new loggregator improvements we are considering and I will post soon. If you have feedback or ideas about using Log-Cache and/or the CLI we'd love to hear from you here or in slack.