Re: Multi-Line Loggregator events and the new Splunk "HTTP Event Collector" API
|
Rohit Kumar
Hi Mike,
toggle quoted message
Show quoted text
As Erik mentioned in the last thread, multi-line logging is something which the loggregator team would like to solve. But there are a few questions to answer before we can come up with a clean solution. We want a design which solves the problem while not breaking existing apps which do not require this functionality. Before implementing a solution we would also want to answer if we want to do it for both runtimes or just Diego, since the way log lines are sent to Metron differ based on the runtime. If we were to implement the solution which you described, where newlines are escaped with a '\', I guess the expectation is that loggregator would internally remove the escape character. This has performance implications because now some part of loggregator will need to inspect the log message and coalesce the message with the succeeding ones. We will need to do this in a way which respects multi-tenancy. That means now we are storing additional state related to log lines per app. We will also need to decide how long loggregator needs to wait for the next lines in a multi-line log, before deciding to send the line which it received. To me that's not a simple change. I am happy to continue this discussion and hear your thoughts on the existing proposal or any other design alternatives. Thanks, Rohit On Wed, Oct 7, 2015 at 10:45 AM, Mike Youngstrom <youngm(a)gmail.com> wrote:
Splunk recently released its new "HTTP Event Collector" that greatly |
|
|