Ugorji Nwoke 86ce10f272 codec: misc refactoring, clean up and optimizations hace 5 años
..
.gitignore a2154fa654 codec: do not generate fast-path for uncommon types, and generate sort.Sort wrappers hace 5 años
README.md 420fad79aa codec: mirror benchmarks into bench sub-directory hace 5 años
bench.sh 86ce10f272 codec: misc refactoring, clean up and optimizations hace 5 años
bench_test.go 420fad79aa codec: mirror benchmarks into bench sub-directory hace 5 años
doc.go 420fad79aa codec: mirror benchmarks into bench sub-directory hace 5 años
shared_test.go 86ce10f272 codec: misc refactoring, clean up and optimizations hace 5 años
values_test.go 3797a84a37 codec/bench: make bench/values_test.go a symlink hace 5 años
x_bench_gen_test.go 420fad79aa codec: mirror benchmarks into bench sub-directory hace 5 años
x_bench_test.go 86ce10f272 codec: misc refactoring, clean up and optimizations hace 5 años
z_all_bench_test.go b4462fc2f4 codec: bench: add benchmark very quick suites, and add more benchmark options hace 5 años
z_all_x_bench_gen_test.go 0ecb224b1b codec: bench: added github.com/mongodb/mongo-go-driver/bson to benchmarks hace 5 años
z_all_x_bench_test.go b4462fc2f4 codec: bench: add benchmark very quick suites, and add more benchmark options hace 5 años

README.md

go-codec-bench

This is a comparison of different binary and text encodings.

We compare the codecs provided by github.com/ugorji/go/codec package, against other libraries:

github.com/ugorji/go/codec provides:

Other codecs compared include:

Data

The data being serialized is a TestStruc randomly generated values. See https://github.com/ugorji/go-codec-bench/blob/master/codec/values_test.go for the definition of the TestStruc.

Run Benchmarks

See https://github.com/ugorji/go-codec-bench/blob/master/codec/bench.sh for how to download the external libraries which we benchmark against, generate the files for the types when needed, and run the suite of tests.

The 3 suite of benchmarks are

  • CodecSuite
  • XSuite
  • CodecXSuite

    # Note that `bench.sh` may be in the codec sub-directory, and should be run from there.
    
    # download the code and all its dependencies
    ./bench.sh -d
    
    # code-generate files needed for benchmarks against ffjson, easyjson, msgp, etc
    ./bench.sh -c
    
    # run the full suite of tests
    ./bench.sh -s
    
    # Below, see how to just run some specific suite of tests, knowing the right tags and flags ...
    # See bench.sh for different iterations
    
    # Run suite of tests in default mode (selectively using unsafe in specific areas)
    go test -tags "alltests x" -bench "CodecXSuite" -benchmem 
    # Run suite of tests in safe mode (no usage of unsafe)
    go test -tags "alltests x safe" -bench "CodecXSuite" -benchmem 
    # Run suite of tests in codecgen mode, including all tests which are generated (msgp, ffjson, etc)
    go test -tags "alltests x generated" -bench "CodecXGenSuite" -benchmem 
    
    

Issues

The following issues are seen currently (11/20/2014):

  • code.google.com/p/cbor/go fails on encoding and decoding the test struct
  • github.com/davecgh/go-xdr/xdr2 fails on encoding and decoding the test struct
  • github.com/Sereal/Sereal/Go/sereal fails on decoding the serialized test struct

Representative Benchmark Results

Please see the benchmarking blog post for detailed representative results.

A snapshot of some results on my 2016 MacBook Pro is below.
Note: errors are truncated, and lines re-arranged, for readability.

Below are results of running the entire suite on 2017-11-20 (ie running ./bench.sh -s).

What you should notice:

  • Results get better with codecgen, showing about 20-50% performance improvement. Users should carefully weigh the performance improvements against the usability and binary-size increases, as performance is already extremely good without the codecgen path.

See https://github.com/ugorji/go-codec-bench/blob/master/bench.out.txt for latest run of bench.sh as of 2017-11-20

  • snippet of bench.out.txt, running without codecgen *

    BenchmarkCodecXSuite/options-false.../Benchmark__Msgpack____Encode-8         	   10000	    183961 ns/op	   10224 B/op	      75 allocs/op
    BenchmarkCodecXSuite/options-false.../Benchmark__Binc_______Encode-8         	   10000	    206362 ns/op	   12551 B/op	      80 allocs/op
    BenchmarkCodecXSuite/options-false.../Benchmark__Simple_____Encode-8         	   10000	    193966 ns/op	   10224 B/op	      75 allocs/op
    BenchmarkCodecXSuite/options-false.../Benchmark__Cbor_______Encode-8         	   10000	    192666 ns/op	   10224 B/op	      75 allocs/op
    BenchmarkCodecXSuite/options-false.../Benchmark__Json_______Encode-8         	    3000	    475767 ns/op	   10352 B/op	      75 allocs/op
    BenchmarkCodecXSuite/options-false.../Benchmark__Std_Json___Encode-8         	    3000	    525223 ns/op	  256049 B/op	     835 allocs/op
    BenchmarkCodecXSuite/options-false.../Benchmark__Gob________Encode-8         	    5000	    270550 ns/op	  333548 B/op	     959 allocs/op
    BenchmarkCodecXSuite/options-false.../Benchmark__JsonIter___Encode-8         	    3000	    478130 ns/op	  183552 B/op	    3262 allocs/op
    BenchmarkCodecXSuite/options-false.../Benchmark__Bson_______Encode-8         	    2000	    747360 ns/op	  715539 B/op	    5629 allocs/op
    BenchmarkCodecXSuite/options-false.../Benchmark__VMsgpack___Encode-8         	    2000	    637388 ns/op	  320385 B/op	     542 allocs/op
    BenchmarkCodecXSuite/options-false.../Benchmark__Sereal_____Encode-8         	    5000	    361369 ns/op	  294541 B/op	    4286 allocs/op
    -------------------------------
    BenchmarkCodecXSuite/options-false.../Benchmark__Msgpack____Decode-8         	    5000	    370340 ns/op	  120352 B/op	    1210 allocs/op
    BenchmarkCodecXSuite/options-false.../Benchmark__Binc_______Decode-8         	    3000	    443650 ns/op	  126144 B/op	    1263 allocs/op
    BenchmarkCodecXSuite/options-false.../Benchmark__Simple_____Decode-8         	    3000	    381155 ns/op	  120352 B/op	    1210 allocs/op
    BenchmarkCodecXSuite/options-false.../Benchmark__Cbor_______Decode-8         	    5000	    370754 ns/op	  120352 B/op	    1210 allocs/op
    BenchmarkCodecXSuite/options-false.../Benchmark__Json_______Decode-8         	    2000	    719658 ns/op	  159289 B/op	    1478 allocs/op
    BenchmarkCodecXSuite/options-false.../Benchmark__Std_Json___Decode-8         	    1000	   2204258 ns/op	  276336 B/op	    6959 allocs/op
    BenchmarkCodecXSuite/options-false.../Benchmark__Gob________Decode-8         	    5000	    383884 ns/op	  256684 B/op	    3261 allocs/op
    BenchmarkCodecXSuite/options-false.../Benchmark__JsonIter___Decode-8         	    2000	    907079 ns/op	  301520 B/op	    7769 allocs/op
    BenchmarkCodecXSuite/options-false.../Benchmark__Bson_______Decode-8         	    2000	   1146851 ns/op	  373121 B/op	   15703 allocs/op
    
  • snippet of bench.out.txt, running with codecgen *

    BenchmarkCodecXGenSuite/options-false.../Benchmark__Msgpack____Encode-8         	   10000	    124729 ns/op	    6224 B/op	       7 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__Binc_______Encode-8         	   10000	    119745 ns/op	    6256 B/op	       7 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__Simple_____Encode-8         	   10000	    132501 ns/op	    6224 B/op	       7 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__Cbor_______Encode-8         	   10000	    129706 ns/op	    6224 B/op	       7 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__Json_______Encode-8         	    3000	    436958 ns/op	    6352 B/op	       7 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__Std_Json___Encode-8         	    3000	    539884 ns/op	  256049 B/op	     835 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__Gob________Encode-8         	    5000	    270663 ns/op	  333548 B/op	     959 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__JsonIter___Encode-8         	    3000	    476215 ns/op	  183552 B/op	    3262 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__Bson_______Encode-8         	    2000	    741688 ns/op	  715539 B/op	    5629 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__VMsgpack___Encode-8         	    2000	    649516 ns/op	  320385 B/op	     542 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__Msgp_______Encode-8         	   30000	     57573 ns/op	       0 B/op	       0 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__Easyjson___Encode-8         	    5000	    366701 ns/op	   92762 B/op	      14 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__Ffjson_____Encode-8         	    3000	    568665 ns/op	  219803 B/op	    1569 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__Sereal_____Encode-8         	    5000	    365595 ns/op	  296303 B/op	    4285 allocs/op
    -------------------------------
    BenchmarkCodecXGenSuite/options-false.../Benchmark__Msgpack____Decode-8         	   10000	    244013 ns/op	  131912 B/op	    1112 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__Binc_______Decode-8         	    5000	    280478 ns/op	  131944 B/op	    1112 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__Simple_____Decode-8         	    5000	    247863 ns/op	  131912 B/op	    1112 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__Cbor_______Decode-8         	   10000	    244624 ns/op	  131912 B/op	    1112 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__Json_______Decode-8         	    3000	    571572 ns/op	  170824 B/op	    1376 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__Std_Json___Decode-8         	    1000	   2224320 ns/op	  276337 B/op	    6959 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__Gob________Decode-8         	    5000	    387137 ns/op	  256683 B/op	    3261 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__JsonIter___Decode-8         	    2000	    913324 ns/op	  301472 B/op	    7769 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__Bson_______Decode-8         	    2000	   1139852 ns/op	  373121 B/op	   15703 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__Msgp_______Decode-8         	   10000	    124270 ns/op	  112688 B/op	    1058 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__Easyjson___Decode-8         	    3000	    521070 ns/op	  184176 B/op	    1371 allocs/op
    BenchmarkCodecXGenSuite/options-false.../Benchmark__Ffjson_____Decode-8         	    2000	    970256 ns/op	  161798 B/op	    1927 allocs/op