Re: Missing routing logs from "cf logs app-name"

Matthew Sykes <matthew.sykes@...>

Simon, what level of cf-release are you using? We've seen some extremely
high log drop rates (~40% during log spurts) to log sinks with v209 and
v210. May or may not be related since we were focused on application logs...

In some cases it was doppler dropping the messages because the truncating
buffer was overrun, in some cases it was a bug where sinks were not picked
up by doppler because it's poller ran out of sockets due to a connection
leak, and in others still, we just never saw the message get to one of the

The http start/stop stuff is usually due to requests that take longer than
a minute (the starts get purged). I don't know if that makes sense for
these requests.

On Wed, Aug 12, 2015 at 11:19 AM, Simon Johansson <simon(a)>

Gwenn, our buffer is set to 500(default is 100).

On Wed, Aug 12, 2015 at 3:24 AM, Gwenn Etourneau <getourneau(a)>

Simon not sure is related but don,t if you don't consume enough quickly
the message, the buffer will be exhausted.

Did you try to increase the buffer size ?

On Wed, Aug 12, 2015 at 5:47 AM, Simon Johansson <
simon(a)> wrote:

Howdie! Not really a dev related question, but since there is no cf-user
list I'm trying my luck here. :)

Im looking into an issue where we don't get enough RTR logs from cf logs
/ firehose-to-syslog
I'm using cf-env[1] as the test app, and Im hitting
For each request the app itself logs the the path requested and the
gorouters logs the same information but a bit more verbose.

$ cf logs cf-env-test
2015-08-11T11:43:41.24+0200 [RTR/0] OUT -
[11/08/2015:09:43:41 +0000] "GET /kehehehe HTTP/1.1" 404 0 18 "-"
"curl/7.43.0" x_forwarded_for:""
response_time:0.008503156 app_id:7f1944a2-2197-43b5-9334-0be7c8b9b40e
2015-08-11T11:43:41.31+0200 [App/0] ERR - -
[11/Aug/2015 09:43:41] "GET /kehehehe HTTP/1.1" 404 18 0.0006

I shoot off 1000 requests and would expect to see 1000 of each log type.
But only the App logs are correct.

$ grep "App/" /tmp/cf-logs | wc -l # /tmp/cf-logs comes from cf logs
cf-env-test > /tmp/cf-logs
$ grep "RTR/" /tmp/cf-logs | wc -l

Looking at the gorouters
nsaadmin(a)a99a5339-4308-43db-b3a4-59442169861d:~$ grep "GET /kehehehe"
/var/vcap/sys/log/gorouter/access.log | wc -l
nsaadmin(a)1d8ef2c2-2ec1-417d-8882-0234de251c60:~$ grep "GET /kehehehe"
/var/vcap/sys/log/gorouter/access.log | wc -l
So the gorouter process definitely logs the right amount of requests to

In metron_agent.stdout.log on the gorouters there are a lot of lines like

matching HTTP start message found for {low:6001753075995010215
matching HTTP start message found for {low:8668459744461365080
matching HTTP start message found for {low:13205430422726438703

Has anyone seen this before?


Matthew Sykes

Join { to automatically receive all group messages.