Logstash and Multiline Log Entry
Steve Wall
Hello,
We are sending CF logs message to an ELK stack. Multiline logs message are
broken out into several log messages in Logstash. One end per line of the
multiline log message. This is problematic when stack traces dumped to the
log. Each line of the stack trace is translated into a log message. Trying
to view this through Kibana is nearly impossible. Logstash provides a Grok
feature allowing for the manipulation of the log messages. One common
solution is to create a Grok filter that using a timestamp to indicate when
a log entry starts and to combine all lines until the next timestamp into
one log message. The problem is that CF adds a timestamp to every line. Has
anyone come up with a good Grok expression to handle multiline log message
coming out of CF?
Thanks!
Steve
We are sending CF logs message to an ELK stack. Multiline logs message are
broken out into several log messages in Logstash. One end per line of the
multiline log message. This is problematic when stack traces dumped to the
log. Each line of the stack trace is translated into a log message. Trying
to view this through Kibana is nearly impossible. Logstash provides a Grok
feature allowing for the manipulation of the log messages. One common
solution is to create a Grok filter that using a timestamp to indicate when
a log entry starts and to combine all lines until the next timestamp into
one log message. The problem is that CF adds a timestamp to every line. Has
anyone come up with a good Grok expression to handle multiline log message
coming out of CF?
Thanks!
Steve