Browse Source

Scrap the connection test in the client metadata update

Previously, if the cluster metadata was giving us back a broker which we
suspected was unavailable (since it was already in our 'dead' set) then we would
wait for the connection, and mark it as unavailable if the connection failed
(otherwise, we simply do what the cluster tells us and let the
producers/consumers deal with the connection errors). This was handy since it
let us back off nicely if a broker crashed and came back, retrying metadata
until the cluster had caught up and moved the leader to a broker that was up.

I'm now of the opinion this was more trouble than it's worth, so scrap it. Among
other things:
 - it does network IO while holding an important mutex, which is a bad pattern
   to begin with (#263)
 - it can mask real network errors behind "LeaderNotAvailable" (#272)

The unfortunate side-effect of scrapping it is that in the producer and consumer
we are more likely to fail if we don't wait long enough for the cluster to fail
over leadership. The real solution if that occurs is to wait longer in the
correct spot (`RetryBackoff` in the producer, currently hard-coded to 10 seconds
in the consumer) instead of this hack.
Evan Huus 11 years ago
parent
commit
71e343a459
1 changed files with 1 additions and 10 deletions
  1. 1 10
      client.go

+ 1 - 10
client.go

@@ -544,16 +544,7 @@ func (client *Client) update(data *MetadataResponse) ([]string, error) {
 		client.metadata[topic.Name] = make(map[int32]*PartitionMetadata, len(topic.Partitions))
 		for _, partition := range topic.Partitions {
 			client.metadata[topic.Name][partition.ID] = partition
-			switch partition.Err {
-			case NoError:
-				broker := client.brokers[partition.Leader]
-				if _, present := client.deadBrokerAddrs[broker.Addr()]; present {
-					if connected, _ := broker.Connected(); !connected {
-						partition.Err = LeaderNotAvailable
-						toRetry[topic.Name] = true
-					}
-				}
-			case LeaderNotAvailable:
+			if partition.Err == LeaderNotAvailable {
 				toRetry[topic.Name] = true
 			}
 		}