--- title: Benchmarking etcd v2.2.0-rc --- ## Physical machines GCE n1-highcpu-2 machine type - 1x dedicated local SSD mounted under /var/lib/etcd - 1x dedicated slow disk for the OS - 1.8 GB memory - 2x CPUs ## etcd Cluster 3 etcd 2.2.0-rc members, each runs on a single machine. Detailed versions: ``` etcd Version: 2.2.0-alpha.1+git Git SHA: 59a5a7e Go Version: go1.4.2 Go OS/Arch: linux/amd64 ``` Also, we use 3 etcd 2.1.0 alpha-stage members to form cluster to get base performance. etcd's commit head is at [c7146bd5][c7146bd5], which is the same as the one that we use in [etcd 2.1 benchmark][etcd-2.1-benchmark]. ## Testing Bootstrap another machine and use the [hey HTTP benchmark tool][hey] to send requests to each etcd member. Check the [benchmark hacking guide][hack-benchmark] for detailed instructions. ## Performance ### reading one single key | key size in bytes | number of clients | target etcd server | read QPS | 90th Percentile Latency (ms) | |-------------------|-------------------|--------------------|----------|---------------| | 64 | 1 | leader only | 2804 (-5%) | 0.4 (+0%) | | 64 | 64 | leader only | 17816 (+0%) | 5.7 (-6%) | | 64 | 256 | leader only | 18667 (-6%) | 20.4 (+2%) | | 256 | 1 | leader only | 2181 (-15%) | 0.5 (+25%) | | 256 | 64 | leader only | 17435 (-7%) | 6.0 (+9%) | | 256 | 256 | leader only | 18180 (-8%) | 21.3 (+3%) | | 64 | 64 | all servers | 46965 (-4%) | 2.1 (+0%) | | 64 | 256 | all servers | 55286 (-6%) | 7.4 (+6%) | | 256 | 64 | all servers | 46603 (-6%) | 2.1 (+5%) | | 256 | 256 | all servers | 55291 (-6%) | 7.3 (+4%) | ### writing one single key | key size in bytes | number of clients | target etcd server | write QPS | 90th Percentile Latency (ms) | |-------------------|-------------------|--------------------|-----------|---------------| | 64 | 1 | leader only | 76 (+22%) | 19.4 (-15%) | | 64 | 64 | leader only | 2461 (+45%) | 31.8 (-32%) | | 64 | 256 | leader only | 4275 (+1%) | 69.6 (-10%) | | 256 | 1 | leader only | 64 (+20%) | 16.7 (-30%) | | 256 | 64 | leader only | 2385 (+30%) | 31.5 (-19%) | | 256 | 256 | leader only | 4353 (-3%) | 74.0 (+9%) | | 64 | 64 | all servers | 2005 (+81%) | 49.8 (-55%) | | 64 | 256 | all servers | 4868 (+35%) | 81.5 (-40%) | | 256 | 64 | all servers | 1925 (+72%) | 47.7 (-59%) | | 256 | 256 | all servers | 4975 (+36%) | 70.3 (-36%) | ### performance changes explanation - read QPS in most scenarios is decreased by 5~8%. The reason is that etcd records store metrics for each store operation. The metrics is important for monitoring and debugging, so this is acceptable. - write QPS to leader is increased by 20~30%. This is because we decouple raft main loop and entry apply loop, which avoids them blocking each other. - write QPS to all servers is increased by 30~80% because follower could receive latest commit index earlier and commit proposals faster. [hey]: https://github.com/rakyll/hey [c7146bd5]: https://github.com/coreos/etcd/commits/c7146bd5f2c73716091262edc638401bb8229144 [etcd-2.1-benchmark]: etcd-2-1-0-alpha-benchmarks.md [hack-benchmark]: ../../hack/benchmark/