Re: expected? doppler log "lost election for cluster leader"


Rohit Kumar
 

Hi Guangcai,

The log messages are coming from the syslog_drain_binder process which is
colocated with dopplers. The syslog_drain_binder is used to poll the
CloudController for active syslog drain URLs for apps. At any point we only
want one syslog_drain_binder to be active, so that the CloudController
doesn't get overloaded with requests. The election process is done to
ensure that.

To answer your question, yes these messages are expected. Secondly, the
syslog_drain_binders will run for election after a specified timeout has
expired. All of them try to create a key in etcd but only one succeeds and
becomes the leader. The exact logic can be found here
<https://github.com/cloudfoundry/loggregator/blob/develop/src/syslog_drain_binder/elector/elector.go#L38-L58>
.

Rohit

On Mon, Sep 14, 2015 at 1:10 AM, Guangcai Wang <guangcai.wang(a)gmail.com>
wrote:

Hi all,

I have 2 doppler instances. I found if one of doppler won election for
cluster leader, the other will frequently log "lost election for cluster
leader" as follows. Is it expected?


{"timestamp":1442214238.411536455,"process_id":7212,"source":"syslog_drain_binder","log_level":"info","message":"Elector:
'doppler_z1.0' lost election for cluster
leader.","data":null,"file":"/var/vcap/data/compile/syslog_drain_binder/loggregator/src/syslog_drain_binder/elector/elector.go","line":57,"method":"syslog_drain_binder/elector.(*Elector).RunForElection"}
{"timestamp":1442214253.724292278,"process_id":7212,"source":"syslog_drain_binder","log_level":"info","message":"Elector:
'doppler_z1.0' lost election for cluster
leader.","data":null,"file":"/var/vcap/data/compile/syslog_drain_binder/loggregator/src/syslog_drain_binder/elector/elector.go","line":57,"method":"syslog_drain_binder/elector.(*Elector).RunForElection"}
{"timestamp":1442214269.286961317,"process_id":7212,"source":"syslog_drain_binder","log_level":"info","message":"Elector:
'doppler_z1.0' lost election for cluster
leader.","data":null,"file":"/var/vcap/data/compile/syslog_drain_binder/loggregator/src/syslog_drain_binder/elector/elector.go","line":57,"method":"syslog_drain_binder/elector.(*Elector).RunForElection"}
{"timestamp":1442214284.720170259,"process_id":7212,"source":"syslog_drain_binder","log_level":"info","message":"Elector:
'doppler_z1.0' lost election for cluster
leader.","data":null,"file":"/var/vcap/data/compile/syslog_drain_binder/loggregator/src/syslog_drain_binder/elector/elector.go","line":57,"method":"syslog_drain_binder/elector.(*Elector).RunForElection"}
{"timestamp":1442214300.056922436,"process_id":7212,"source":"syslog_drain_binder","log_level":"info","message":"Elector:
'doppler_z1.0' lost election for cluster
leader.","data":null,"file":"/var/vcap/data/compile/syslog_drain_binder/loggregator/src/syslog_drain_binder/elector/elector.go","line":57,"method":"syslog_drain_binder/elector.(*Elector).RunForElection"}


I also want to know in which conditions/situations they will reelect for
cluster leader again.

Join cf-dev@lists.cloudfoundry.org to automatically receive all group messages.