Date
1 - 6 of 6
Failure of metron agent in cloudfoundry
|
Sam Nelson
Just chiming in real quick: the specific line of code that displayed that
toggle quoted message
Show quoted text
the panic is showing comes from here: https://github.com/cloudfoundry/loggregatorlib/blob/507ff1f4ef7749879a14b12fd2c42d654c99b2f2/servicediscovery/servicediscovery.go#L57 Any error from the etcd storeadapter other than storeadapter.ErrorTimeout and storeadapter.ErrorKeyNotFound will cause a panic. The error that was output was "Standby internal error", but with that code being as old as it is, I honestly couldn't guess what it actually means. The modern code should attempt to reconnect instead of panicking. It also uses etcd watch events instead of polling an endpoint using ListRecursively. -Sam On Wed, Apr 6, 2016 at 1:08 PM, Bharath Posa <bharathp(a)vedams.com> wrote:
Yeah. I can try. Before that can I know cause of failure for this one. Bcz |
|
|
|
Bharath
Yeah. I can try. Before that can I know cause of failure for this one. Bcz
it is the current deployment we are having. We want to know what exactly went wrong with it. Just curious in debugging it. :-) Bharath On 6 Apr 2016 23:58, "Jim CF Campbell" <jcampbell(a)pivotal.io> wrote: newer release? are you running? wrote: the metron agent in all the jobs failed . The metron.stderr.log provided the below log github.com/cloudfoundry/loggregatorlib/servicediscovery.(*serverAddressList).Run(0xc208052120, 0x12a05f200) github.com/cloudfoundry/loggregatorlib/servicediscovery/servicediscovery.go:57/var/vcap/data/compile/metron_agent/loggregator/src/ +0x3c3 /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:105created by main.main +0x1351 /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:187 +0x7a /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:107main.main() +0x137e 0xc20800bd40) github.com/cloudfoundry/gunk/workpool/workpool.go:71 +0x151/var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:44 +0x189created by github.com/cloudfoundry/gunk/workpool.New 0xc20800bd40) github.com/cloudfoundry/gunk/workpool/workpool.go:71 +0x151/var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:44 +0x189created by github.com/cloudfoundry/gunk/workpool.New 0xc20800bd40) github.com/cloudfoundry/gunk/workpool/workpool.go:71 +0x151/var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:44 +0x189created by github.com/cloudfoundry/gunk/workpool.New 0xc20800bd40) github.com/cloudfoundry/gunk/workpool/workpool.go:71 +0x151/var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:44 +0x189created by github.com/cloudfoundry/gunk/workpool.New 0xc20800bd40) github.com/cloudfoundry/gunk/workpool/workpool.go:71 +0x151/var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:44 +0x189created by github.com/cloudfoundry/gunk/workpool.New 0xc20800bd40) github.com/cloudfoundry/gunk/workpool/workpool.go:71 +0x151/var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:44 +0x189created by github.com/cloudfoundry/gunk/workpool.New 0xc20800bd40) github.com/cloudfoundry/gunk/workpool/workpool.go:71 +0x151/var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:44 +0x189created by github.com/cloudfoundry/gunk/workpool.New 0xc20800bd40) github.com/cloudfoundry/gunk/workpool/workpool.go:71 +0x151/var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:44 +0x189created by github.com/cloudfoundry/gunk/workpool.New 0xc20800bd40) github.com/cloudfoundry/gunk/workpool/workpool.go:71 +0x151/var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:44 +0x189created by github.com/cloudfoundry/gunk/workpool.New 0xc20800bd40) github.com/cloudfoundry/gunk/workpool/workpool.go:71 +0x151/var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:44 +0x189created by github.com/cloudfoundry/gunk/workpool.New github.com/cloudfoundry/loggregatorlib/cfcomponent/registrars/collectorregistrar.(*CollectorRegistrar).Run(0xc20800c0c0) github.com/cloudfoundry/loggregatorlib/cfcomponent/registrars/collectorregistrar/collector_registrar.go:41/var/vcap/data/compile/metron_agent/loggregator/src/ +0x201 /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:78 +0xf7dcreated by main.main 0x0, 0x0) 0xc20800bc50, 0x0, 0x0)/usr/local/go/src/net/http/server.go:1728 +0x92 github.com/cloudfoundry/loggregatorlib/cfcomponent.Component.StartMonitoringEndpoints(0xc20801ece0,/usr/local/go/src/net/http/server.go:1808 +0xba 0xc20802aac0, 0x9, 0x7ff232301940, 0xa393a8, 0x834ff0, 0xb, 0x0, 0x0, 0x0, ...) github.com/cloudfoundry/loggregatorlib/cfcomponent/component.go:76 +0x719/var/vcap/data/compile/metron_agent/loggregator/src/ 0x7ff232301940, 0xa393a8, 0x834ff0, 0xb, 0x0, 0x0, 0x0, ...)main.startMonitoringEndpoints(0xc20801ece0, 0xc20802aac0, 0x9, /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:118 +0x3e /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:80 +0xfb8created by main.main 0x0, 0x0, 0x7ff2322ffbe0, 0xc20802acf8) 0x20, 0x0, 0x0, 0x0)/usr/local/go/src/net/fd_unix.go:269 +0x4a1 0xffff, 0x0, 0x0, 0x0, 0x0)/usr/local/go/src/net/udpsock_posix.go:67 +0x124 github.com/cloudfoundry/loggregatorlib/agentlistener.(*agentListener).Start(0xc208010fc0)/usr/local/go/src/net/udpsock_posix.go:82 +0x12e github.com/cloudfoundry/loggregatorlib/agentlistener/agent_listener.go:46/var/vcap/data/compile/metron_agent/loggregator/src/ +0x3a5 /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:83created by main.main +0x101a metron/legacy_message/legacy_unmarshaller.(*legacyUnmarshaller).Run(0xc20802a9d0, 0xc208052540, 0xc2080523c0) /var/vcap/data/compile/metron_agent/loggregator/src/metron/legacy_message/legacy_unmarshaller/legacy_unmarshaller.go:30 +0x71 /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:86created by main.main +0x1096 metron/legacy_message/legacy_message_converter.(*legacyMessageConverter).Run(0xc20802e020, 0xc2080523c0, 0xc208052300) /var/vcap/data/compile/metron_agent/loggregator/src/metron/legacy_message/legacy_message_converter/legacy_message_converter.go:28 +0x5c /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:87created by main.main +0x10dc 0x0, 0x0, 0x7ff2322ffbe0, 0xc20802add0) 0x20, 0x0, 0x0, 0x0)/usr/local/go/src/net/fd_unix.go:269 +0x4a1 0xffff, 0x0, 0x0, 0x0, 0x0)/usr/local/go/src/net/udpsock_posix.go:67 +0x124 /var/vcap/data/compile/metron_agent/loggregator/src/metron/eventlistener/event_listener.go:54/usr/local/go/src/net/udpsock_posix.go:82 +0x12e +0x3a0 /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:89created by main.main +0x1108 github.com/cloudfoundry/dropsonde/dropsonde_unmarshaller.(*dropsondeUnmarshaller).Run(0xc20802c080, 0xc2080525a0, 0xc208052300) github.com/cloudfoundry/dropsonde/dropsonde_unmarshaller/dropsonde_unmarshaller.go:65/var/vcap/data/compile/metron_agent/loggregator/src/ +0x71 /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:91created by main.main +0x114e 0xc208052300, 0xc208052420) /var/vcap/data/compile/metron_agent/loggregator/src/metron/message_aggregator/message_aggregator.go:57 +0x5c /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:94created by main.main +0x11ca 0xc208052480) /var/vcap/data/compile/metron_agent/loggregator/src/metron/varz_forwarder/varz_forwarder.go:31 +0x51 /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:97created by main.main +0x1237 github.com/cloudfoundry/dropsonde/dropsonde_marshaller.(*dropsondeMarshaller).Run(0xc20801e0a0, 0xc208052480, 0xc2080524e0) github.com/cloudfoundry/dropsonde/dropsonde_marshaller/dropsonde_marshaller.go:59/var/vcap/data/compile/metron_agent/loggregator/src/ +0x5f /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:100created by main.main +0x12a3 /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:111 +0x7a /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:103created by main.main +0x130e 0x7ff2322ffbe0, 0xc2080daa00) 0x0)/usr/local/go/src/net/fd_unix.go:242 +0x40f 0xc20805af78, 0xc208059000, 0x1000, 0x1000, 0x75b940, 0x0, 0x0)/usr/local/go/src/net/net.go:121 +0xdc 0x1000, 0xc208012000, 0x0, 0x0)/usr/local/go/src/net/http/transport.go:1270 +0x6e 0x7ff2322ffbe0, 0xc2080dad28)<autogenerated>:125 +0xd4 0x0)/usr/local/go/src/net/fd_unix.go:242 +0x40f 0xc20805ad68, 0xc2080fb000, 0x1000, 0x1000, 0x75b940, 0x0, 0x0)/usr/local/go/src/net/net.go:121 +0xdc 0x1000, 0xc208012000, 0x0, 0x0)/usr/local/go/src/net/http/transport.go:1270 +0x6e 303.618.0963<autogenerated>:125 +0xd4
|
|
|
|
Jim CF Campbell
Hi Barath,
toggle quoted message
Show quoted text
That release is over a year old. Is there any chance you can upgrade to a newer release? Jim On Wed, Apr 6, 2016 at 12:23 PM, Bharath Posa <bharathp(a)vedams.com> wrote:
Hi Jim --
Jim Campbell | Product Manager | Cloud Foundry | Pivotal.io | 303.618.0963 |
|
|
|
Bharath
Hi Jim
toggle quoted message
Show quoted text
Its version 206 Bharath On 6 Apr 2016 23:25, "Jim CF Campbell" <jcampbell(a)pivotal.io> wrote:
Hi Bharath, |
|
|
|
Jim CF Campbell
Hi Bharath,
toggle quoted message
Show quoted text
This looks like it might be older code. What version of Cloud Foundry are you running? Thanks, Jim On Mon, Apr 4, 2016 at 11:09 PM, Bharath Posa <bharathp(a)vedams.com> wrote:
Hi all, --
Jim Campbell | Product Manager | Cloud Foundry | Pivotal.io | 303.618.0963 |
|
|
|
Bharath
Hi all,
we are recently running cloudfoundry in openstack. while recent saw the metron agent in all the jobs failed . The metron.stderr.log provided the below log ------------- panic: 402: Standby Internal Error () [0] goroutine 28 [running]: github.com/cloudfoundry/loggregatorlib/servicediscovery.(*serverAddressList).Run(0xc208052120, 0x12a05f200) /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/loggregatorlib/servicediscovery/servicediscovery.go:57 +0x3c3 created by main.main /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:105 +0x1351 goroutine 1 [chan receive]: main.forwardMessagesToDoppler(0xc20802c040, 0xc2080526c0, 0xc20801ece0) /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:187 +0x7a main.main() /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:107 +0x137e goroutine 5 [syscall]: os/signal.loop() /usr/local/go/src/os/signal/signal_unix.go:21 +0x1f created by os/signal.init·1 /usr/local/go/src/os/signal/signal_unix.go:27 +0x35 goroutine 7 [select]: github.com/cloudfoundry/gunk/workpool.worker(0xc208052600, 0xc20800bd40) /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:71 +0x151 created by github.com/cloudfoundry/gunk/workpool.New /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:44 +0x189 goroutine 8 [select]: github.com/cloudfoundry/gunk/workpool.worker(0xc208052600, 0xc20800bd40) /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:71 +0x151 created by github.com/cloudfoundry/gunk/workpool.New /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:44 +0x189 goroutine 9 [select]: github.com/cloudfoundry/gunk/workpool.worker(0xc208052600, 0xc20800bd40) /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:71 +0x151 created by github.com/cloudfoundry/gunk/workpool.New /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:44 +0x189 goroutine 10 [select]: github.com/cloudfoundry/gunk/workpool.worker(0xc208052600, 0xc20800bd40) /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:71 +0x151 created by github.com/cloudfoundry/gunk/workpool.New /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:44 +0x189 goroutine 11 [select]: github.com/cloudfoundry/gunk/workpool.worker(0xc208052600, 0xc20800bd40) /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:71 +0x151 created by github.com/cloudfoundry/gunk/workpool.New /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:44 +0x189 goroutine 12 [select]: github.com/cloudfoundry/gunk/workpool.worker(0xc208052600, 0xc20800bd40) /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:71 +0x151 created by github.com/cloudfoundry/gunk/workpool.New /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:44 +0x189 goroutine 13 [select]: github.com/cloudfoundry/gunk/workpool.worker(0xc208052600, 0xc20800bd40) /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:71 +0x151 created by github.com/cloudfoundry/gunk/workpool.New /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:44 +0x189 goroutine 14 [select]: github.com/cloudfoundry/gunk/workpool.worker(0xc208052600, 0xc20800bd40) /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:71 +0x151 created by github.com/cloudfoundry/gunk/workpool.New /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:44 +0x189 goroutine 15 [select]: github.com/cloudfoundry/gunk/workpool.worker(0xc208052600, 0xc20800bd40) /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:71 +0x151 created by github.com/cloudfoundry/gunk/workpool.New /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:44 +0x189 goroutine 16 [select]: github.com/cloudfoundry/gunk/workpool.worker(0xc208052600, 0xc20800bd40) /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:71 +0x151 created by github.com/cloudfoundry/gunk/workpool.New /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/gunk/workpool/workpool.go:44 +0x189 goroutine 17 [select]: github.com/cloudfoundry/loggregatorlib/cfcomponent/registrars/collectorregistrar.(*CollectorRegistrar).Run(0xc20800c0c0) /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/loggregatorlib/cfcomponent/registrars/collectorregistrar/collector_registrar.go:41 +0x201 created by main.main /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:78 +0xf7d goroutine 18 [IO wait]: net.(*pollDesc).Wait(0xc2080103e0, 0x72, 0x0, 0x0) /usr/local/go/src/net/fd_poll_runtime.go:84 +0x47 net.(*pollDesc).WaitRead(0xc2080103e0, 0x0, 0x0) /usr/local/go/src/net/fd_poll_runtime.go:89 +0x43 net.(*netFD).accept(0xc208010380, 0x0, 0x7ff2322ffbe0, 0xc20802aba8) /usr/local/go/src/net/fd_unix.go:419 +0x40b net.(*TCPListener).AcceptTCP(0xc20802e060, 0x596f54, 0x0, 0x0) /usr/local/go/src/net/tcpsock_posix.go:234 +0x4e net/http.tcpKeepAliveListener.Accept(0xc20802e060, 0x0, 0x0, 0x0, 0x0) /usr/local/go/src/net/http/server.go:1976 +0x4c net/http.(*Server).Serve(0xc2080527e0, 0x7ff232301bb8, 0xc20802e060, 0x0, 0x0) /usr/local/go/src/net/http/server.go:1728 +0x92 net/http.(*Server).ListenAndServe(0xc2080527e0, 0x0, 0x0) /usr/local/go/src/net/http/server.go:1718 +0x154 net/http.ListenAndServe(0xc20802abb0, 0xf, 0x7ff232301b90, 0xc20800bc50, 0x0, 0x0) /usr/local/go/src/net/http/server.go:1808 +0xba github.com/cloudfoundry/loggregatorlib/cfcomponent.Component.StartMonitoringEndpoints(0xc20801ece0, 0xc20802aac0, 0x9, 0x7ff232301940, 0xa393a8, 0x834ff0, 0xb, 0x0, 0x0, 0x0, ...) /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/loggregatorlib/cfcomponent/component.go:76 +0x719 main.startMonitoringEndpoints(0xc20801ece0, 0xc20802aac0, 0x9, 0x7ff232301940, 0xa393a8, 0x834ff0, 0xb, 0x0, 0x0, 0x0, ...) /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:118 +0x3e created by main.main /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:80 +0xfb8 goroutine 19 [IO wait]: net.(*pollDesc).Wait(0xc208010450, 0x72, 0x0, 0x0) /usr/local/go/src/net/fd_poll_runtime.go:84 +0x47 net.(*pollDesc).WaitRead(0xc208010450, 0x0, 0x0) /usr/local/go/src/net/fd_poll_runtime.go:89 +0x43 net.(*netFD).readFrom(0xc2080103f0, 0xc2080c8000, 0xffff, 0xffff, 0x0, 0x0, 0x0, 0x7ff2322ffbe0, 0xc20802acf8) /usr/local/go/src/net/fd_unix.go:269 +0x4a1 net.(*UDPConn).ReadFromUDP(0xc20802e068, 0xc2080c8000, 0xffff, 0xffff, 0x20, 0x0, 0x0, 0x0) /usr/local/go/src/net/udpsock_posix.go:67 +0x124 net.(*UDPConn).ReadFrom(0xc20802e068, 0xc2080c8000, 0xffff, 0xffff, 0xffff, 0x0, 0x0, 0x0, 0x0) /usr/local/go/src/net/udpsock_posix.go:82 +0x12e github.com/cloudfoundry/loggregatorlib/agentlistener.(*agentListener).Start(0xc208010fc0) /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/loggregatorlib/agentlistener/agent_listener.go:46 +0x3a5 created by main.main /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:83 +0x101a goroutine 20 [chan receive]: metron/legacy_message/legacy_unmarshaller.(*legacyUnmarshaller).Run(0xc20802a9d0, 0xc208052540, 0xc2080523c0) /var/vcap/data/compile/metron_agent/loggregator/src/metron/legacy_message/legacy_unmarshaller/legacy_unmarshaller.go:30 +0x71 created by main.main /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:86 +0x1096 goroutine 21 [chan receive]: metron/legacy_message/legacy_message_converter.(*legacyMessageConverter).Run(0xc20802e020, 0xc2080523c0, 0xc208052300) /var/vcap/data/compile/metron_agent/loggregator/src/metron/legacy_message/legacy_message_converter/legacy_message_converter.go:28 +0x5c created by main.main /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:87 +0x10dc goroutine 22 [IO wait]: net.(*pollDesc).Wait(0xc208010530, 0x72, 0x0, 0x0) /usr/local/go/src/net/fd_poll_runtime.go:84 +0x47 net.(*pollDesc).WaitRead(0xc208010530, 0x0, 0x0) /usr/local/go/src/net/fd_poll_runtime.go:89 +0x43 net.(*netFD).readFrom(0xc2080104d0, 0xc2080e0000, 0xffff, 0xffff, 0x0, 0x0, 0x0, 0x7ff2322ffbe0, 0xc20802add0) /usr/local/go/src/net/fd_unix.go:269 +0x4a1 net.(*UDPConn).ReadFromUDP(0xc20802e098, 0xc2080e0000, 0xffff, 0xffff, 0x20, 0x0, 0x0, 0x0) /usr/local/go/src/net/udpsock_posix.go:67 +0x124 net.(*UDPConn).ReadFrom(0xc20802e098, 0xc2080e0000, 0xffff, 0xffff, 0xffff, 0x0, 0x0, 0x0, 0x0) /usr/local/go/src/net/udpsock_posix.go:82 +0x12e metron/eventlistener.(*eventListener).Start(0xc208060400) /var/vcap/data/compile/metron_agent/loggregator/src/metron/eventlistener/event_listener.go:54 +0x3a0 created by main.main /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:89 +0x1108 goroutine 23 [chan receive]: github.com/cloudfoundry/dropsonde/dropsonde_unmarshaller.(*dropsondeUnmarshaller).Run(0xc20802c080, 0xc2080525a0, 0xc208052300) /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/dropsonde/dropsonde_unmarshaller/dropsonde_unmarshaller.go:65 +0x71 created by main.main /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:91 +0x114e goroutine 24 [chan receive]: metron/message_aggregator.(*messageAggregator).Run(0xc208052180, 0xc208052300, 0xc208052420) /var/vcap/data/compile/metron_agent/loggregator/src/metron/message_aggregator/message_aggregator.go:57 +0x5c created by main.main /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:94 +0x11ca goroutine 25 [chan receive]: metron/varz_forwarder.(*VarzForwarder).Run(0xc20802c500, 0xc208052420, 0xc208052480) /var/vcap/data/compile/metron_agent/loggregator/src/metron/varz_forwarder/varz_forwarder.go:31 +0x51 created by main.main /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:97 +0x1237 goroutine 26 [chan receive]: github.com/cloudfoundry/dropsonde/dropsonde_marshaller.(*dropsondeMarshaller).Run(0xc20801e0a0, 0xc208052480, 0xc2080524e0) /var/vcap/data/compile/metron_agent/loggregator/src/ github.com/cloudfoundry/dropsonde/dropsonde_marshaller/dropsonde_marshaller.go:59 +0x5f created by main.main /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:100 +0x12a3 goroutine 27 [chan receive]: main.signMessages(0xc20802b4b8, 0x8, 0xc2080524e0, 0xc2080526c0) /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:111 +0x7a created by main.main /var/vcap/data/compile/metron_agent/loggregator/src/metron/main.go:103 +0x130e goroutine 33 [syscall, locked to thread]: runtime.goexit() /usr/local/go/src/runtime/asm_amd64.s:2232 +0x1 goroutine 31 [IO wait]: net.(*pollDesc).Wait(0xc208010a00, 0x72, 0x0, 0x0) /usr/local/go/src/net/fd_poll_runtime.go:84 +0x47 net.(*pollDesc).WaitRead(0xc208010a00, 0x0, 0x0) /usr/local/go/src/net/fd_poll_runtime.go:89 +0x43 net.(*netFD).Read(0xc2080109a0, 0xc208059000, 0x1000, 0x1000, 0x0, 0x7ff2322ffbe0, 0xc2080daa00) /usr/local/go/src/net/fd_unix.go:242 +0x40f net.(*conn).Read(0xc20802e150, 0xc208059000, 0x1000, 0x1000, 0x0, 0x0, 0x0) /usr/local/go/src/net/net.go:121 +0xdc net/http.noteEOFReader.Read(0x7ff232301d08, 0xc20802e150, 0xc20805af78, 0xc208059000, 0x1000, 0x1000, 0x75b940, 0x0, 0x0) /usr/local/go/src/net/http/transport.go:1270 +0x6e net/http.(*noteEOFReader).Read(0xc20801f2e0, 0xc208059000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0) <autogenerated>:125 +0xd4 bufio.(*Reader).fill(0xc208053200) /usr/local/go/src/bufio/bufio.go:97 +0x1ce bufio.(*Reader).Peek(0xc208053200, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /usr/local/go/src/bufio/bufio.go:132 +0xf0 net/http.(*persistConn).readLoop(0xc20805af20) /usr/local/go/src/net/http/transport.go:842 +0xa4 created by net/http.(*Transport).dialConn /usr/local/go/src/net/http/transport.go:660 +0xc9f goroutine 32 [select]: net/http.(*persistConn).writeLoop(0xc20805af20) /usr/local/go/src/net/http/transport.go:945 +0x41d created by net/http.(*Transport).dialConn /usr/local/go/src/net/http/transport.go:661 +0xcbc goroutine 36 [IO wait]: net.(*pollDesc).Wait(0xc20810a3e0, 0x72, 0x0, 0x0) /usr/local/go/src/net/fd_poll_runtime.go:84 +0x47 net.(*pollDesc).WaitRead(0xc20810a3e0, 0x0, 0x0) /usr/local/go/src/net/fd_poll_runtime.go:89 +0x43 net.(*netFD).Read(0xc20810a380, 0xc2080fb000, 0x1000, 0x1000, 0x0, 0x7ff2322ffbe0, 0xc2080dad28) /usr/local/go/src/net/fd_unix.go:242 +0x40f net.(*conn).Read(0xc20802e318, 0xc2080fb000, 0x1000, 0x1000, 0x0, 0x0, 0x0) /usr/local/go/src/net/net.go:121 +0xdc net/http.noteEOFReader.Read(0x7ff232301d08, 0xc20802e318, 0xc20805ad68, 0xc2080fb000, 0x1000, 0x1000, 0x75b940, 0x0, 0x0) /usr/local/go/src/net/http/transport.go:1270 +0x6e net/http.(*noteEOFReader).Read(0xc2080dd000, 0xc2080fb000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0) <autogenerated>:125 +0xd4 bufio.(*Reader).fill(0xc2081086c0) /usr/local/go/src/bufio/bufio.go:97 +0x1ce bufio.(*Reader).Peek(0xc2081086c0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0) /usr/local/go/src/bufio/bufio.go:132 +0xf0 net/http.(*persistConn).readLoop(0xc20805ad10) /usr/local/go/src/net/http/transport.go:842 +0xa4 created by net/http.(*Transport).dialConn /usr/local/go/src/net/http/transport.go:660 +0xc9f goroutine 37 [select]: net/http.(*persistConn).writeLoop(0xc20805ad10) /usr/local/go/src/net/http/transport.go:945 +0x41d created by net/http.(*Transport).dialConn /usr/local/go/src/net/http/transport.go:661 +0xcbc ---------------------- I tried to restart it using monit restart still it is failing . any body got idea on this error regards Bharath |
|
|