Re: Follow up on multiple line log outputs in CF

George Li


so basically Loggregator does not support such correlation itself and it is
totally left to the downstream processor to do the aggregation, right?
When my service write exception stack trace into the log as follows:

2015-05-18 14:40:41 +0000 WARN
[] GET http://localhost:3004/ threw
exception: FileNotFoundException -
E:\Src\Registrar\escrow\service\config\ (The system
cannot find the file specified) Method)<init>(

Loggregator sends the following to downstream logstash:

2015-05-18 14:40:41 +0000 [App/0] OUT [escrow] [pool-3-thread-1052] WARN
[] GET threw
exception: FileNotFoundException - config\ (No such file
or directory)
2015-05-18 14:40:41 +0000 [App/0] OUT
2015-05-18 14:40:41 +0000 [App/0] OUT<init>(
2015-05-18 14:40:41 +0000 [App/0] OUT

Ideally we want to keep all these as a single log "event" in the final log.
The way to do this in logstash is to config logstash so it would aggregate
all lines not staring with a timestamp (or any identifiable marker) to
previous lines:

input {
file {
codec => multiline {
pattern => "^%{TIMESTAMP_ISO8601} "

So I think my exact question is if there is a similar way to config
loggregator so it does the same thing.

In the previous thread!msg/vcap-dev/B1W6_vO0oyo/84X1eAtFsKoJ,
David Lee said "...As for multiline output, we should have fixed this a
couple of builds back...", I wonder what was really fixed related to
multiline output at that time.



On Sun, May 17, 2015 at 8:26 AM, James Bayer <jbayer(a)> wrote:


can you explain the scenario that is impacting you currently? e.g. reading
a java stack trace? the app index is shown in the log line. when using the
loggregator websocket client to retrieve logs you should also get the
application id. perhaps that is enough to correlate whatever UI you have on
top of logstash?

On Tue, May 12, 2015 at 7:40 AM, Li, George <>


this is a follow up on the archived posting!msg/vcap-dev/B1W6_vO0oyo/84X1eAtFsKoJ.
I cannot find any new postings on that thread.
I am using Cloud Foundry version
"6.11.2-2a26d55-2015-04-27T21:11:44+00:00" and want to know what options I
have to handle multiple line logs in a multi-tenant environment. Since
multiple instances of multiple applications are all sending logs to a
single Logstash server, is it best to avoid having multiple lines in my
log? I can live with sticking to single line logs except for outputting
exception stack trace, not to mention that I only have control over my


cf-dev mailing list

Thank you,

James Bayer

Join to automatically receive all group messages.