Re: Doppler/Firehose - Multiline Log Entry


Aliaksandr Prysmakou <prysmakou@...>
 

Hi Jim,

Thank you for sharing it with us.

Two things - for Java, we are working toward defining an enhanced metric format that will support transport of Multi Lines.
Could you please share a link to track status of this work?
So there are "first class citizen"(Java) and others (Ruby, Python etc)?
Is it so rare case that we should use workarounds?

The second is this workaround that David Laing suggested for Logstash. Think you could use it for Splunk?

With the Java Logback library you can do this by adding "%replace(%xException){'\n','\u2028'}%nopex" to your logging config[1] , and then use the following logstash conf.[2]
Replace the unicode newline character \u2028 with \n, which Kibana will display as a new line.

mutate {
gsub => [ "[@message]", '\u2028', "
"]
^^^ Seems that passing a string with an actual newline in it is the only way to make gsub work

}
to replace the token with a regular newline again so it displays "properly" in Kibana.
[1] github.com/dpin...ication.yml#L12 <https://github.com/dpinto-pivotal/cf-SpringBootTrader-config/blob/master/application.yml#L12>
[2] github.com/logs...se.conf#L60-L64 <https://github.com/logsearch/logsearch-for-cloudfoundry/blob/master/src/logsearch-config/src/logstash-filters/snippets/firehose.conf#L60-L64>

On Mon, Mar 14, 2016 at 11:11 AM, Mike Youngstrom <youngm(a)gmail.com <mailto:youngm(a)gmail.com>> wrote:
I'll let the Loggregator team respond formally. But, in my conversations with the Loggregator team I think we're basically stuck not sure what the right thing to do is on the client side. How does the client trigger in loggregator that this is a multi line log message or what is the right way for loggregator to detect that the client is trying to send a multi line log message? Any ideas?

Mike

On Mon, Mar 14, 2016 at 10:25 AM, Aliaksandr Prysmakou <prysmakou(a)gmail.com <mailto:prysmakou(a)gmail.com>> wrote:
Hi guys,
Are there any updates about "Multiline Log Entry" issue? How correctly deal with stacktraces?
Links to the tracker to read?
----
Alex Prysmakou / Altoros
Tel: (617) 841-2121 ext. 5161 <tel:%28617%29%20841-2121%20ext.%205161> | Toll free: 855-ALTOROS
Skype: aliaksandr.prysmakou
www.altoros.com <http://www.altoros.com/> | blog.altoros.com <http://blog.altoros.com/> | twitter.com/altoros <http://twitter.com/altoros>




--
Jim Campbell | Product Manager | Cloud Foundry | Pivotal.io | 303.618.0963

Join cf-dev@lists.cloudfoundry.org to automatically receive all group messages.