Re: Logstash and Multiline Log Entry
Mike Youngstrom <youngm@...>
I replied a while ago but it appears the mailing list dropped it. Here istoggle quoted message Show quoted text
Thanks for the response Erik.
* Loggregator's dropsonde protocol didn't allow for a clean way to
enforce/tag multi-line data consistency - something we are about to put
forward a proposal to remedy.
** Timestamps are not a clean mechanism for reliably re-assembling a
multi-line log - some combination of app-instance and order-of-output would
need to be tacked on, or a decent vector-time implementation.
It seems these features above are focused on breaking apart a multi-line
event into multiple events then re-assembling them somewhere else. I think
I've been looking at the issue from a different perspective. I've been
looking for ways to allow an application to get multiple lines into a
single Loggregator event. That would be good enough to solve my problems
without all the complexity of breaking up and reassembling a series of
events. Even if Loggregator had ways to add metadata to join multiple
events into 1 we are still faced with the problem of how the
dea_logging_agent knows when an app logs multiple stdout/err lines when
should those be considered a multi-line message.
Do you have any thoughts on how an app might hint to the logging agent that
a message is a multi-line message? It seems to me that is the real problem
I need solved. Or am I missing something?
On Tue, Aug 11, 2015 at 1:26 PM, Erik Jasiak <mjasiak(a)pivotal.io> wrote:
Hi Steve and Simon; hello again Mike,